Technical lead at a speech-AI lab serving oral-language communities. I own the stack end to end: self-supervised acoustic units and vocoders, LLM Guide and Validator agents with an evaluation harness as the release gate, and the FastAPI backend and voice apps that five field products run on.
Most of my work sits where research-grade speech models meet a product that real people, often with no written language and spotty connectivity, have to use every day. I take the whole problem: model, evaluation, backend, app.
Text-free pipelines: self-supervised acoustic units (XEUS), motif tokenizers, pitch-conditioned unit vocoders (HiFi-GAN), ASR and TTS adaptation, data collection with speakers in the field.
Built the full Tripod speech pipeline and ran pilots with Terena and Sateré-Mawé speakers in Brazil.
Multi-role agents on the Claude API with explicit tool boundaries, a validator role that refuses and redirects, and an evaluation harness (golden sessions, LLM judge, per-role cost accounting, budget gates) as the release gate.
Nothing reaches a field team until it passes the harness; the same pattern served 50+ enterprise clients at an ed-tech company.
FastAPI, SQLAlchemy and PostgreSQL on Cloud Run with auth, app-scoped access and migrations; Flutter apps for iOS, Android and web; React consoles for the team behind the model.
One shared backend behind five products, from audio capture to a screen-text-free LLM room.
Each entry is the problem as the team had it, what I built, and what came out of it. The lab work is methods-level; the data and the languages' material stay with the communities.
Communities with oral-only languages needed a way to produce and check translated speech with no writing system to lean on, and no existing model knew their language.
A text-free pipeline end to end: an XEUS-based acoustic tokenizer with a vocabulary of 100 discrete units, a BPE motif tokenizer over those units, a pitch-conditioned HiFi-GAN v2 unit vocoder at 16 kHz, and Tripod Lab, the experiment-tracking and training dashboard with staged quality gates that decides what moves forward.
A working speech-to-units-to-speech loop, pilots with Terena and Sateré-Mawé speakers, and a method the linguistics lead could turn into specifications and tickets for the rest of the team.
Walk a translation team through a passage by voice alone, with an AI guide that helps without taking over, and a way to prove each prompt or model change did not make it worse.
Guide and Validator roles on Claude with explicit tool boundaries, text-to-speech and server-sent events inside a Flutter room with no on-screen text; plus the evaluation harness: a fixed set of golden sessions replayed and scored by an LLM judge, per-role cost accounting and budget gates wired in as the release gate.
Prompt and model changes ship only after the harness passes, with cost per session known before release. The same harness runs a regression suite over the agents' tool use.
Five products (Internalization Room, Facilitator Desk, Sound Necklace, Oral Collector, Annotation Studio) built by a small team, each needing auth, storage, access rules and secrets, without five backends.
A shared FastAPI, SQLAlchemy and PostgreSQL backend on Google Cloud Run with authentication, app-scoped access control, Alembic migrations and Secret Manager; the Oral Collector Flutter app for monolingual audio capture on iOS, Android and web; and the React/TypeScript consoles (Tripod Console, Meaning Map UI, Annotation Studio for layer-wise studies of XEUS).
One codebase the whole lab deploys from, with a multi-agent development workflow (isolated worktrees, dedicated reviewers, Linear) that lets two or three engineers ship like a larger team.
Generate personalized educational content for dozens of enterprise clients with answers accurate enough to put in front of teachers and students.
Multi-agent architectures and monitoring pipelines over enterprise content indexing, with human-in-the-loop validation; non-blocking microservices for generation; CI/CD for tests and linting. As Tech Lead, worked with the CTO to scale the ML products and bridged the product and engineering squads.
Personalized generation for 50+ B2B clients with higher answer accuracy and lower operating cost.
The setup I use to take a Linear ticket to a merged pull request with coding agents: agent skills, Maestri roles, a local decision inbox exposed over MCP so a human settles only the real decisions, and a single end-to-end test slot.
Habits from shipping models to people who cannot file a bug report in writing.
A prompt or model change is a code change. It ships when the golden sessions pass and the cost per session is known, not when it looks good in a demo.
Backends and apps grow behind tests; migrations are proven by snapshot, upgrade and downgrade before they touch a database with field data.
I turn a linguist's method into specifications, tickets and acceptance criteria, and review the engineers' work against them.
Isolated worktrees, one reviewer per task, and a decision inbox so the people decide only what the ticket and the docs do not settle.
Have a speech or LLM problem with real users behind it? Send a few lines about the problem and the constraints, and we can take it from there.
joaocarvoli@hotmail.com