Award-winning ML project

Guido

Guido started with a simple question: how can we give wheelchair users more independence in everyday movement? In America, 1 in 4 people live with a disability, and for many wheelchair users getting around can still require physical effort or outside help. We wanted to build something that merged autonomy and voice control so movement could happen on the user's terms. **What Guido Does:** • **Voice-Driven Control** - Users can say commands like "move forward," "turn left," or ask Guido to navigate to a destination without needing a joystick • **Autonomous Navigation** - Guido builds a real-time map of its surroundings with LiDAR, detects obstacles, and drives to waypoints in the background • **Exploration in Unknown Spaces** - The system can explore unfamiliar environments, map them on the fly, and avoid repeatedly retrying bad goals **Technical Deep Dive:** The core design choice was making Guido a **layered autonomy system** instead of one monolithic script. Command handling, supervisory logic, and Nav2 execution run in separate ROS 2 nodes so high-level intents never talk to wheel motion directly. For navigation, we separated **mapping** from **localization**. **SLAM Toolbox** maintains the map and pose estimate, while a dedicated goal bridge feeds saved destinations and autonomy goals into **Nav2**. In exploration mode, Guido uses **frontier-based planning** to find the boundary between known and unknown cells, cluster candidate frontiers, rank them using path cost and information gain, and blacklist timed-out frontiers so the robot does not get stuck retrying bad goals. We also added **health monitoring** for LiDAR scans, odometry, and TF freshness, plus stop and timeout behavior in both the supervisor and motor bridge so the system fails safely when data goes stale or motion commands are interrupted. **Hardware:** • **NVIDIA Jetson Orin Nano** - Main onboard compute • **LD19 360° LiDAR** - Real-time perception and mapping • **Arduino + motor driver shield** - Low-level motor interface • **MPU6050** - Gyroscope and accelerometer feedback • **Dual DC motors with encoders** - Closed-loop mobility platform **Software:** • **ROS 2 Humble** running on Ubuntu • **SLAM Toolbox** for mapping and localization • **Nav2** for goal planning and autonomous execution • **Serial bridge** between Jetson and Arduino for motor control • **Voice-to-text pipeline** streaming commands into the system • **Agent layer** for interpreting commands and triggering actions **Challenges We Ran Into:** One of the hardest problems was navigation in unknown environments. Guido cannot assume it already knows the space, so it has to explore, build a usable map, and stay safe at the same time. Other major challenges included: • Reliable real-time voice recognition • Synchronizing sensor data with movement commands • Maintaining stable serial communication between Jetson and Arduino • Getting every subsystem to work together consistently in the real world **What We're Proud Of:** • Building a fully working autonomous wheelchair system from scratch at smaller scale • Real-time LiDAR mapping with obstacle avoidance • A voice-to-action pipeline that can directly control movement • A modular ROS-based architecture that can scale over time • End-to-end hardware and software integration **What We Learned:** We learned very quickly that real-world robotics is way messier than simulation. Latency, sensor noise, and motor inconsistency all show up immediately once hardware is actually moving. We also learned: • How to design reliable real-time systems • How to structure ROS-based robotics projects • Why safety layers and fail-safes matter • How to connect high-level AI behavior with low-level hardware control **What's Next:** The next step is scaling Guido to a full-sized wheelchair platform and improving reliability in real environments. We're also planning to: • Improve path planning in more complex indoor spaces • Integrate richer mapping for outdoor navigation • Make voice interaction feel more natural and conversational • Add stronger safety and redundancy systems The long-term goal is to take Guido from a prototype into something people could genuinely rely on every day. **Built With:** ROS 2, NVIDIA Jetson Orin Nano, LiDAR (LD19), Arduino, Python, C++, SLAM Toolbox, Vosk / ElevenLabs, Google ADK, React, TailwindCSS