All projects

Agentic AI · Speech

Language Mirror

A voice-conversation prototype I orchestrated across hosted speech, translation and language models, offering guided speaking practice in five languages.

Context
AI Partner Catalyst hackathon · project
Stack
FastAPI, Gemini, ElevenLabs, Next.js, React 19, SSE

How it works

Explainer animation

Explainer animation, not a screen recording. How a spoken practice turn moves through hosted speech, translation and language models.

  1. Pick a persona

    15 dialect personas across 5 languages (Italian, Spanish, French, German, Japanese), voiced through 3 ElevenLabs voices

  2. Hosted models, one pipeline

    Google Speech-to-Text, Google Translate, and Gemini on a FastAPI backend: a six-stage SSE pipeline with a seven-model fallback chain

  3. Spoken replies

    ElevenLabs Multilingual v2 text-to-speech for spoken replies

  4. The frontend

    Next.js 16 / React 19 / MUI frontend with GSAP animations, live subtitles, and language-mismatch detection

Highlights

  • 15 dialect personas across 5 languages (Italian, Spanish, French, German, Japanese), voiced through 3 ElevenLabs voices
  • Google Speech-to-Text, Google Translate, and Gemini on a FastAPI backend: a six-stage SSE pipeline with a seven-model fallback chain
  • ElevenLabs Multilingual v2 text-to-speech for spoken replies
  • Next.js 16 / React 19 / MUI frontend with GSAP animations, live subtitles, and language-mismatch detection

How it was built

For the AI Partner Catalyst hackathon I orchestrated a voice tutor from hosted services. A learner's speech goes through a six-stage streaming pipeline on a FastAPI backend: Google speech-to-text, language detection through Google Translate, a Gemini reply in a fixed JSON format, and ElevenLabs speech synthesis. If the learner uses the wrong language, the pipeline stops before calling Gemini to save quota, and a seven-model chain handles quota failures. A Next.js frontend shows live subtitles. Limits: every model is hosted and nothing was trained; there are no tests or measurements.