ML engineering project
AI Burger Flipper
AI Burger Flipper tackles full-stack automation in fast food by integrating natural language processing, computer vision, and robotic manipulation into one cohesive system. The project demonstrates how AI can handle an enviroemtn with varying lighting, imprecise positioning, and natural human interaction. **System Components:** • **Human Detection** - MediaPipe Pose estimation (confidence threshold τ = 0.5) continuously monitors the kiosk camera to detect customer presence and automatically launch the ordering interface • **Voice Interface** - Flask-SocketIO server with dual worker threads handles speech recognition (Google Speech API) and text-to-speech (gTTS), enabling natural conversation with the LLM for order intake • **Vision Pipeline** - Dual detection system using Hough Circle Transform for burger localization and inverse-square light intensity for gripper fingertip tracking  **Camera Calibration & Geometry:** The system uses camera intrinsics and homography to map image coordinates to real-world positions. Distance to target is computed from the burger's known diameter and detected radius: $$ d = \frac{f \cdot D}{2r} $$ Horizontal and vertical offsets guide the servo control loop, with base rotation angle computed as: $$ \theta = \arctan\left(\frac{\Delta x}{d}\right) $$  **Results & Statistics:** Over 50 independent trials, the arm achieved X = 43 successful burger retrievals (ˆp = 0.86). A one-sided z-test against baseline p₀ = 0.5 yielded z ≈ 5.09 and p < 2×10⁻⁷, with 95% CI: [0.76, 0.96].  **Challenges Overcome:** The project survived accidental data loss (7 months of code reconstructed), mechanical failure (servo controller replacement), vision calibration drift under varying lighting, and grip stability issues solved by adding rubber bands to fingertips for increased friction.  **Tech Stack:** Built with **Python**, **Flask-SocketIO**, **OpenCV**, **MediaPipe**, **LLaMA 3.1**, and **Lynxmotion SSC-32U** servo controller. System integrates real-time computer vision with natural language understanding for truly autonomous food service.
Problem
A robotic food-service system must understand an order, find a burger under imperfect lighting, and manipulate it reliably with a physical arm.
My role
Built the end-to-end robotic-arm system spanning voice interaction, vision calibration, grasp control, mechanical iteration, and experimental evaluation.
Approach
Combine LLaMA-assisted ordering, MediaPipe presence detection, OpenCV geometry, calibrated camera-to-arm coordinates, and a 6-DOF robotic arm.
Outcome
Completed 43 successful retrievals across 50 trials for an 86% documented success rate.
Implementation
- Used MediaPipe and speech services to open and operate the ordering flow.
- Detected burgers with Hough circles and tracked the gripper with visual cues.
- Controlled a Lynxmotion SSC-32U-based 6-DOF arm using calibrated geometry.
Evaluation
- Ran 50 independent burger-retrieval trials.
- Reported the success proportion, confidence interval, and a one-sided baseline comparison.
Results
- 43 of 50 successful retrievals (86%).
- The documented 95% confidence interval is 0.76 to 0.96.
Constraints
- Lighting and target placement varied between trials.
- Camera pixels had to map to physical servo coordinates.
- Grip stability and mechanical reliability affected software outcomes.
Tradeoffs
- Classical geometric vision is interpretable and fast but sensitive to calibration and lighting.
- The integrated demo prioritizes end-to-end behavior over industrial food-safety readiness.
Next improvements
- Add depth sensing, closed-loop force feedback, and broader object-placement tests.
- Separate the ordering and manipulation services for easier fault isolation.