A wearable that had to see

As engineering students, we constantly face high-stress situations — exams and deadlines that feel impossible in the moment. When your heart is racing and your thoughts are scattered, all you want is someone calm beside you saying, “It’s okay. Do this next.”

Survival situations are no different. Stress clouds judgment, and small mistakes can become dangerous. So we built Twin: a built-in friend, in a hard hat, designed to guide you when things get overwhelming.

We submitted it to MakeUofT 2026 and won 1st Place: Meta Glasses.

What it does

Twin is an AI-powered survival assistant integrated directly into a hard hat. It sees your surroundings through a camera, monitors biometric data like heart rate, and gives real-time guidance when you need it.

1. The user presses a button and asks a question
2. Twin captures the surrounding environment with a camera
3. It processes the visual input
4. It analyzes the user’s question
5. It responds with context-aware survival guidance

How we built it

Twin runs on a hybrid embedded system:

• Arduino for sensor input (heart rate and environmental data)
• Raspberry Pi for processing and AI inference
• Computer vision for environmental recognition
• Serial communication between Arduino and Pi for real-time data exchange

When the button is pressed, the Arduino signals the Raspberry Pi. The Pi captures an image, runs the vision stack (YOLO / OpenCV), interprets the question, and generates a response.

Challenges we ran into

The hardest part was reliable serial communication between the Arduino and the Raspberry Pi. Baud-rate mismatches caused inconsistent data — delayed or corrupted readings.

We also had to balance processing speed against wearable hardware limits. Off-device inference is easy to demo and easy to miss. Localized YOLO-v8 models got us from “it works on the laptop” to “it works on a person.”

What we’re proud of

• A hardware-triggered AI interaction system
• Real-time environmental analysis on the hat
• Embedded systems talking to an on-device vision stack
• A functional wearable prototype in a weekend

What we learnt

• How to configure a Raspberry Pi for AI inference
• How to interface Arduino hardware with a Linux box
• How much baud-rate matching and serial debugging matter in embedded systems
• What it actually costs to put hardware and AI in the same real-time loop

What’s next

• Continuous multi-frame awareness (always-on perception instead of push-to-talk)
• Better hazard detection models
• Lower latency and better power
• Construction safety and disaster response, not just a hackathon hat