AI Burger Flipper
I connected a conversational ordering interface to a vision-guided robotic arm. The small-scale prototype detects a customer, confirms an order, locates the burger, and coordinates the arm’s approach, grasp, and delivery.
The problem
An ordering interface and a robotic arm operate on very different inputs. One receives a person’s words; the other needs physical coordinates and a sequence of motions. Connecting them requires the confirmed order to lead into reliable visual localization and manipulation.
The project explored that complete interaction using a camera, speech services, a language model, and a six-axis arm. Lighting, camera placement, and gripper contact all influenced the physical result.
My contribution
I built the end-to-end robotic-arm system spanning voice interaction, vision calibration, grasp control, mechanical iteration, and experimental evaluation. My work connected the software stages with the physical adjustments needed to retrieve the burger consistently.
Architecture
The paper describes MediaPipe pose detection with a 0.5 confidence threshold as the entry point. Detecting a customer launches the Flask-SocketIO ordering interface. Speech recognition supplies the user’s input, LLaMA 3.1 supports order interpretation, and text-to-speech returns the response.
After order confirmation, the system moves into visual target localization and robotic control. Keeping that transition explicit makes it possible to trace how an interaction becomes a physical action.
Vision and calibration
Hough-circle detection identifies the burger’s image center and radius. The paper combines that observation with the burger’s known diameter and an effective focal length to estimate distance. Camera calibration establishes the relationship between image measurements and the arm’s workspace.
The fixed example uses a focal length of 600 pixels, a 10 cm diameter, and a 50-pixel radius. It produces a distance of 60 cm. These values illustrate the equations described in the paper.
def target_distance(focal_px, diameter_cm, radius_px):
return focal_px * diameter_cm / (2 * radius_px)from math import atan, degrees
def base_angle(offset_cm, distance_cm):
return degrees(atan(offset_cm / distance_cm))
The second visual path tracks the gripper fingertips using their light-intensity cues. Together, the target and fingertip observations help express the remaining offset between the gripper and the burger.
Robotic control
The Lynxmotion SSC-32U controller drives the arm’s servos. Target distance and offset inform base rotation and the approach. The manipulation sequence then closes the gripper, transfers the object, and releases it at the destination.
Mechanical contact mattered alongside the control software. Adding rubber bands to the fingertips increased friction during retrieval. The paper also describes camera calibration drift and a servo-controller replacement during development.
The opening animation uses a fixed illustrative grasp-and-transfer sequence. Its joint positions and timing are generated locally for the page.
Results
The paper records 43 successful burger retrievals across 50 trials, giving an observed success rate of 86%. This result describes the small-scale prototype and its experimental retrieval task.
Vision calibration and grip stability were central to the trials. The photographs document those implementation details, while the figure fixtures make the geometry and motion sequence repeatable for this explanation.
Paper
AI-Driven Automation in Fast Food: Integrating Large Language Models with Robotic Systems, by Michael Rusu, Carter Nebrich, Charles Weihem, and Kathryn Crawford, March 2025.
Read the project paper ↗