Can you run your company by talking to it?
Voice-Operated Company
- Whisper
- Claude
- ElevenLabs
- FastAPI
Relevant services: AI Digital Teammates · Thrivbe AI
Hypothesis
If the AI OS holds all company state, the fastest interface isn't a dashboard — it's your voice. Speak a thought, and the system should route it: file a task, search memory, check the pipeline, delegate work to an agent.
What we built
A voice bridge: a progressive web app on the phone streams speech to a server, Whisper transcribes, an LLM picks from a manifest of ~17 kernel-backed tools (search contacts, recall memory, delegate to the OS worker, send messages…), and the answer comes back as streamed speech.
The interesting part is the safety architecture. High-stakes tools (sending messages, making calls, approving decisions) are never executed from model output — the call is staged, the assistant says "confirmation required", and only a short spoken yes from the same session, within a time window, executes it. That's a deterministic code path, not a prompt instruction.
Learnings
- Confirmation gates belong in code, not prompts. A model can be talked out of
a prompt rule; it cannot be talked out of an
ifstatement. - Tool definitions drift. The kernel now serves a canonical tool manifest and a weekly job diffs the voice bridge against it — schema drift gets caught by a timer, not by a failed call in production.
- Privacy review changed the architecture: once a tool result contains third-party personal data, follow-up model calls pin to a verified cascade of no-training-on-prompts models. Worth doing before, not after.
Log
- 2026-07-11 — Brought under git with a deploy script; high-stakes gate, tool-drift timer, and privacy model-pinning all live. Proactive attention-push to the phone added.
- 2026-06-10 — First working voice loop: speak → transcribe → tool call → spoken answer.
