⚠️ SAFETY DISCLAIMER MIRA is an assistive supplement for navigation only. It does not detect obstacles, traffic, stairs, potholes, or moving hazards. Always use your white cane and follow your orientation and mobility (O&M) training. Do not use this app as your sole guide in unfamiliar or dangerous environments. MIRA is not a replacement for professional O&M instruction or any mobility aid.
MIRA is a voice-first Android app for blind and low-vision users in India. Using only your voice, it:
- Navigates you outdoors to any destination described in natural language (English or Tamil) with turn-by-turn spoken guidance and distinct vibration patterns.
- Reads indoor signs using the phone camera so you can find rooms, floors, and departments in unfamiliar buildings — with no beacons or pre-mapped infrastructure.
- Identifies elevator buttons and tells you which physical button to press for your floor.
All features work in both English and Tamil. The app is designed so a blind user can complete every action with voice alone — zero required screen taps.
mira/
├── app/src/main/java/com/vaazhi/app/
│ ├── core/
│ │ ├── IntentSchema.kt # Shared data models (ParsedIntent, NavStep, etc.)
│ │ └── SessionState.kt # Single source of truth — StateFlow-based session
│ │
│ ├── voice/
│ │ ├── AudioRecorder.kt # VAD-gated microphone → 16kHz WAV
│ │ ├── WhisperSTT.kt # Whisper STT (self-hosted or OpenAI API fallback)
│ │ ├── GeminiIntentParser.kt# Gemini 2.5 Flash → structured ParsedIntent JSON
│ │ ├── VaazhiTTS.kt # On-device TTS (ta-IN + en-IN)
│ │ └── VoicePipeline.kt # Orchestrates: record → transcribe → parse → dispatch
│ │
│ ├── outdoor/
│ │ ├── PlacesClient.kt # Google Places API (New) — text search
│ │ ├── DirectionsClient.kt # Google Directions API — walking steps
│ │ ├── HapticController.kt # Distinct vibration per maneuver type
│ │ └── OutdoorNavigationModule.kt # Full outdoor flow orchestrator
│ │
│ ├── indoor/
│ │ ├── signage/
│ │ │ ├── SignageScanner.kt # ML Kit on-device OCR (offline)
│ │ │ ├── GeminiSignReader.kt # Gemini multimodal sign interpretation
│ │ │ ├── StepCounter.kt # Accelerometer-based step detection
│ │ │ └── IndoorSignageModule.kt # Full sign-sweep loop orchestrator
│ │ └── elevator/
│ │ ├── ElevatorPanelReader.kt # Gemini multimodal elevator panel reader
│ │ └── ElevatorFlowModule.kt # Full elevator flow orchestrator
│ │
│ ├── ui/
│ │ ├── MainActivity.kt # Single activity, CameraX, permissions
│ │ └── MainViewModel.kt # Central orchestrator, intent router
│ │
│ └── VaazhiApplication.kt
│
├── res/
│ ├── layout/activity_main.xml # High-contrast, single FAB, camera preview
│ ├── values/strings.xml # Bilingual strings + accessibility descriptions
│ ├── values/colors.xml # WCAG AA high-contrast palette
│ ├── values/themes.xml
│ └── xml/shortcuts.xml # App Actions + static launcher shortcuts
│
├── build.gradle # All dependencies + BuildConfig key injection
└── AndroidManifest.xml
Voice shortcut / FAB tap
│
▼
VoicePipeline.listen()
AudioRecorder (VAD-gated WAV)
│
▼ WhisperSTT (self-hosted / OpenAI)
│
▼ GeminiIntentParser → ParsedIntent JSON
│
▼
MainViewModel.dispatchIntent()
├── NAVIGATE_OUTDOOR → OutdoorNavigationModule
│ PlacesClient → DirectionsClient → HapticController + TTS
│
├── NAVIGATE_INDOOR → IndoorSignageModule
│ Camera → SignageScanner (ML Kit, offline)
│ └── if text detected → GeminiSignReader (online) or raw OCR (offline)
│ StepCounter ("walk 10 steps, then scan again")
│
└── ELEVATOR → ElevatorFlowModule
Camera → ElevatorPanelReader (Gemini multimodal)
"Press the 3rd button from the top"
| Tool | Version |
|---|---|
| Android Studio | Hedgehog (2023.1.1) or newer |
| JDK | 17 |
| Android device | API 26+ (Android 8.0+), with TalkBack installed |
| Google Play Services | Required for location |
git clone https://github.com/your-org/mira.git
cd mira
# Open in Android Studio: File → Open → select the mira/ folderCopy the template and fill in your API keys:
cp local.properties.template local.propertiesEdit local.properties:
GEMINI_API_KEY=your_gemini_key
GOOGLE_MAPS_API_KEY=your_maps_key
WHISPER_BASE_URL=http://YOUR_SERVER_IP:9000
OPENAI_API_KEY=sk-... # only if using OpenAI fallback
USE_OPENAI_WHISPER_FALLBACK=false| Key | Where |
|---|---|
GEMINI_API_KEY |
Google AI Studio — free tier available |
GOOGLE_MAPS_API_KEY |
Google Cloud Console — enable Places API (New) and Directions API |
WHISPER_BASE_URL |
See Whisper self-hosting below |
OPENAI_API_KEY |
platform.openai.com — only needed as fallback |
The simplest way to run a Whisper HTTP server:
# Using Docker (faster-whisper backend)
docker run -d -p 9000:9000 \
-e ASR_MODEL=base \
-e ASR_ENGINE=faster_whisper \
onerahmet/openai-whisper-asr-webservice:latest
# Verify it works:
curl http://localhost:9000/v1/audio/transcriptions \
-F model=whisper-1 \
-F file=@test.wavSet WHISPER_BASE_URL=http://<your-server-ip>:9000 in local.properties.
If you cannot run a self-hosted server, set USE_OPENAI_WHISPER_FALLBACK=true
and provide your OPENAI_API_KEY. This sends audio to OpenAI's servers — inform
users of this privacy implication.
./gradlew assembleDebug
adb install app/build/outputs/apk/debug/app-debug.apkOr press Run ▶ in Android Studio.
Settings → Accessibility → TalkBack → Turn On
Use a Bluetooth keyboard or the gesture-navigation mode to operate the device.
Enable these in your Google Cloud project:
- ✅ Places API (New) — text search for destinations
- ✅ Directions API — walking route steps
- ✅ Maps JavaScript API — (optional, for future web companion)
Restrict your API key to com.vaazhi.app package in Cloud Console to prevent abuse.
| Item | PRD spec | What's built | Reason |
|---|---|---|---|
| Cloud TTS fallback | Listed | Not built | On-device Google TTS covers ta-IN since Android 8+; cloud would add latency and offline failure modes |
| GPS-triggered step advancement | Implied | Delay-based timer (TODO marked) | FusedLocationProviderClient lifecycle wiring requires Activity context; marked as TODO in OutdoorNavigationModule |
| Polyline off-route detection | Implied | Announced on voice "I'm lost" | Full polyline geometry adds ~300 LOC; flagged as TODO inline |
| Tamil ML Kit OCR | Desired | Latin recognizer only | ML Kit Tamil script module is experimental as of 2024; flagged as TODO in SignageScanner.kt |
| Firebase | Optional per PRD | Not included | Keeping dependencies minimal; add Crashlytics to VaazhiApplication if needed |
| Full BII App Actions | Listed | SEARCH_ACTION intent filter | BII enrollment requires Play Console review; current approach works without it |
MIRA is an assistive supplement for navigation. It does not detect obstacles, traffic, stairs, potholes, moving hazards, or any physical dangers. Always use your white cane and follow your orientation and mobility (O&M) training at all times. Do not use this app as your sole guide in unfamiliar or dangerous environments. MIRA is not a replacement for professional O&M instruction, a guide dog, or any certified mobility aid. The developers are not liable for any injury, accident, or harm resulting from the use of this application.
This disclaimer is spoken aloud (in English and Tamil) on first app launch.
PRs welcome. Before contributing:
- All new spoken strings must have both
_enand_tavariants. - No UI interaction may be required for any core feature — voice-only path must work.
- Any new API call must have an explicit offline/error spoken fallback.
- New action types: add to
Actionenum + Gemini few-shot examples only — no rearchitecting.