Overview
Developed a contextual AI nutrition assistant during Teladoc's 2-day internal hackathon. The app uses OCR to scan food labels and extract nutritional information, then combines it with user health data to deliver personalized dietary guidance through a conversational chat interface. The concept predated the mainstream availability of Vision-Language Models — and was later adopted by the company to improve its food-logging and nutrition systems.
The Challenge
Nutrition apps typically require manual data entry, which creates friction that kills engagement. The challenge was to eliminate this friction entirely using AI — letting users point a phone at any food label and instantly receive personalized guidance, not just raw data. All of this had to be built and demonstrated in under 48 hours.
Technical Approach
- OCR pipeline to extract nutritional information from food label images in real time
- Structured extraction: calories, macros, serving sizes, ingredient lists
- User health profile intake: dietary goals, restrictions, medical conditions
- Conversational chat interface combining OCR outputs with health context
- Personalized response generation: specific guidance, not generic nutrition facts
- Mobile and web-compatible implementation for the hackathon demo