michaelml.dev / Michael_Portfolio.ipynb
AI-driven fast food automation system that fuses LLaMA 3.1 with a 6-DOF robotic arm for autonomous ordering and delivery. Uses MediaPipe for customer detection, Flask-SocketIO for natural language ordering, and OpenCV vision pipeline for precise burger grasping. Achieved 86% success rate over 50 trials.
[ ]
code · AI Burger Flipper
project = load_project("ai-burger-flipper")
project.summary()Output
AI Burger Flipper
AI-driven fast food automation system that fuses LLaMA 3.1 with a 6-DOF robotic arm for autonomous ordering and delivery. Uses MediaPipe for customer detection, Flask-SocketIO for natural language ordering, and OpenCV vision pipeline for precise burger grasping. Achieved 86% success rate over 50 trials.
Computer VisionOpenCVRoboticsFlask-SocketIOMediaPipeLLMPythonSpeech RecognitionHough TransformServo Control
[ ]
code · Problem definition
project = load_project("ai-burger-flipper")
project.problem()Output
ProblemA robotic food-service system must understand an order, find a burger under imperfect lighting, and manipulate it reliably with a physical arm.
ApproachCombine LLaMA-assisted ordering, MediaPipe presence detection, OpenCV geometry, calibrated camera-to-arm coordinates, and a 6-DOF robotic arm.
OutcomeCompleted 43 successful retrievals across 50 trials for an 86% documented success rate.
[ ]
code · Constraints
project = load_project("ai-burger-flipper")
project.constraints()Output
Constraints
- Lighting and target placement varied between trials.
- Camera pixels had to map to physical servo coordinates.
- Grip stability and mechanical reliability affected software outcomes.
[ ]
code · Role and implementation
project = load_project("ai-burger-flipper")
project.implementation()Output
Role and implementation
My role: Built the end-to-end robotic-arm system spanning voice interaction, vision calibration, grasp control, mechanical iteration, and experimental evaluation.
- Used MediaPipe and speech services to open and operate the ordering flow.
- Detected burgers with Hough circles and tracked the gripper with visual cues.
- Controlled a Lynxmotion SSC-32U-based 6-DOF arm using calibrated geometry.
[ ]
code · Architecture
project = load_project("ai-burger-flipper")
project.show_architecture()Output
Customer DetectionMediaPipe continuously monitors kiosk camera for human presence (33 pose landmarks, min confidence 0.5)
Voice Interface LaunchAutomatically spawn Flask-SocketIO server with dual worker threads for speech I/O and open browser to chat UI
Natural Language OrderingStream customer speech to LLaMA 3.1 for intent parsing, maintain conversation history in thread-safe queue
Dual Vision DetectionHough Circle Transform finds burger center/radius, inverse-square intensity tracks gripper fingertips
Geometric MappingCamera calibration + homography convert pixel coords to real-world offsets, compute distance via focal length
Servo Control LoopCalculate rotation angle θ, update base/lift positions with distance-scaled feedback until grasping criteria met
[ ]
code · Demo and media
project = load_project("ai-burger-flipper")
project.media()Output
[ ]
code · Evaluation
project = load_project("ai-burger-flipper")
project.evaluation()Output
Evaluation
- Ran 50 independent burger-retrieval trials.
- Reported the success proportion, confidence interval, and a one-sided baseline comparison.
[ ]
code · Results
project = load_project("ai-burger-flipper")
project.results()Output
Results
- 43 of 50 successful retrievals (86%).
- The documented 95% confidence interval is 0.76 to 0.96.
[ ]
code · Tradeoffs and failures
project = load_project("ai-burger-flipper")
project.tradeoffs()Output
Tradeoffs
- Classical geometric vision is interpretable and fast but sensitive to calibration and lighting.
- The integrated demo prioritizes end-to-end behavior over industrial food-safety readiness.
[ ]
code · What I would improve
project = load_project("ai-burger-flipper")
project.improvements()Output
Improvements
- Add depth sensing, closed-loop force feedback, and broader object-placement tests.
- Separate the ordering and manipulation services for easier fault isolation.
[ ]