Can a spreadsheet join your meeting and answer questions out loud?

Hypothesis

Communities keep their memory in spreadsheets: who is here, what they care about, which call discusses what, what came out of it. In a live conversation that memory is useless, because reading it means someone stops listening, shares a screen, and scrolls. If the sheet could speak, the person who knows the answer would no longer have to be the person holding the laptop.

What we built

A voice agent over a real community-inquiry sheet (a participatory inquiry convened by Ben Roberts), then gave it a seat in the meeting.

Three layers. First, both tabs of the sheet were converted into a structured markdown knowledge base: 23 people, 11 topics, 17 venues, 17 resources, 5 harvests and 2 groups on one tab, plus roughly 90 "connection" rows on the other that link a person to a topic or venue along with the reasoning for the match. About 18,000 words in total.

Second, an ElevenLabs conversational agent (gpt-4o-mini) on top of that knowledge base, prompted to answer the questions a convenor actually asks: who should I talk to about this, which venue covers that topic, what has already been harvested.

Third, meeting presence via Recall.ai. Their bot can render a webpage as its camera and microphone, so we deployed a 46-line HTML page that starts an ElevenLabs session, put it behind Caddy on one of our servers, and pointed the bot at it. The meeting's audio becomes the agent's microphone and the agent's replies stream back into the room. One command sends it into any Zoom, Meet or Teams call. In a live Google Meet test the bot joined within seconds and answered questions straight from the sheet.

Learnings

Retrieval versus full-prompt injection is a real coin flip at this size. The first version injected the entire knowledge base into every turn, which is legitimate for a document this small and gives perfect recall on "list everything" questions. We switched to RAG (multilingual e5 large instruct, up to 20 retrieved chunks) so each question pulls only relevant rows. The cost of that choice is exactly the broad question: an agent doing retrieval cannot reliably enumerate a whole category. The fix was honesty in the prompt, telling it to answer with what was retrieved, say the sheet may hold more, and suggest narrowing the question. Retrieval buys focus and pays for it in completeness.

A webpage can be a meeting participant. The Recall output-media path means anything that runs in a browser can hold a seat in a call. Our whole "agent body" is a static HTML file with an animated orb. That is a much wider door than it first appears: any browser-based interface, not just a voice widget, can now be inside the meeting rather than beside it.

The hard part was etiquette, not capability. A voice agent's default behaviour is to answer every utterance it hears, which in a group call is socially unusable. It greets the room on join, then narrates over people thinking aloud. We removed the greeting entirely so it joins silently, and enabled ElevenLabs' skip_turn system tool with prompt rules that make silence the default: it speaks only when addressed by name, and re-mutes immediately after answering. One address, one answer, silence. Turn-taking turns out to be the interface, and it belongs in the tool layer, not in a politeness instruction.

Log

• 2026-07-28 — Built the sheet-to-knowledge-base conversion, created the ElevenLabs agent, switched it from full-prompt injection to RAG, deployed the meeting page, and dispatched the bot into a live Google Meet where it answered from the sheet. Added silent join and the skip_turn re-mute rule after the first test showed the agent talking over the room.