All projects
  • DeepNeuron · Team
  • Completed

Omni AR

An AR assistant that watches a scene, remembers it, and answers questions about it later. Built for everyday recall, and for people living with memory impairment.

The Omni AR interface describing a live scene: camera view with scene description, voice controls and memory actions.
The Omni AR interface describing a live scene: camera view with scene description, voice controls and memory actions.

Stack

  • Python
  • Flask
  • React
  • Gemini Vision
  • Whisper
  • SSE

Links

Source

What I built

  • A hand-rolled vector memory store with normalized cosine similarity, time-windowed recall, and disk persistence.
  • A low-latency streaming voice pipeline: Whisper → Gemini intent router → sentence-chunked parallel TTS over SSE, tuned for time-to-first-audio.

The Omni AR glasses prototype on a desk, tethered over USB-C.