Does a knowledge graph remember a community better than flat search does?

Hypothesis

Our AI OS already has a memory. One SQLite index holds everything we know about the communities we coordinate: group chats, CRM records, notes, network data, the internal wiki. Questions go in through a hybrid retrieval layer that runs keyword and vector search side by side, fuses the rankings, and hands the top dozen chunks to a model that writes a short answer. It works, and it has been the backbone of most of what we build.

But the questions we actually ask it are not really search questions. They are questions about a network. Who is working on what. Who shows up in three different rooms and plays a different part in each. Who should meet whom. Flat retrieval answers those by finding passages that sound like the question, which is a different operation from following a relationship.

So the hypothesis: if you extract entities and relationships up front and store them as an actual graph, questions shaped like a graph should get better answers. GraphRAG is not a new idea, and there is now a mature open-source implementation of it. Time to find out whether the idea survives contact with our own data.

What we built

We put cognee (Apache-2.0, around 29.5k stars) up against our incumbent memory, on the same corpus, with the same questions.

The corpus is the tracked community group chats described in Unified Group Memory: 33,882 messages across 125 groups, roughly 7MB of text, read-only, with personal and family conversations excluded by rule. Same scoping as before.

The challenger ran in an isolated virtual environment on our own server, fully embedded and self-hosted: Kuzu for the graph, LanceDB for the vectors, SQLite underneath, with gpt-4o-mini doing entity extraction. We bucketed the corpus one document per chat per day, which is the same chunking decision that worked for the incumbent, and ran a pilot on the 150 most recent buckets. That produced a knowledge graph of 1,329 nodes and 3,223 edges in about 90 seconds, for well under a dollar of extraction cost.

Then 20 identical questions to both systems, transcripts saved side by side. Five warm-ups, then fifteen deliberately hard ones: multi-hop chains, contradiction detection, how a topic shifted over weeks, single buried facts, open asks nobody answered, and matchmaking.

The incumbent went in with a real advantage. It had the full multi-source history, including the wiki and the CRM. The challenger had 150 days of chat and nothing else.

Learnings

The graph won, and not narrowly. Roughly 12 of 20 questions went to cognee outright, one to the incumbent, the rest split or too close to call. It won despite seeing a fraction of the data.

The wins cluster exactly where the hypothesis predicted. Anything people-shaped or enumeration-shaped went to the graph, often lopsidedly. Asked which people appear in more than one group chat and what role they play in each, our unified memory returned names with the roles missing and said so honestly; cognee returned the people, both chats, and a role in each. Asked to trace a chain from a report to its presenter to the organisations behind it to others connected to those organisations, the incumbent named the organisations and then suggested we go read the chat history ourselves. Asked for concrete open asks across all groups, the incumbent said the context did not contain any; cognee listed five, with who, what, and where. Same corpus. The difference is that one system had already done the work of deciding what a person is.

Its best moment was about us, not the communities. Asked to find contradictions across the chats, cognee noticed that one of our own workspace chats reported "all decisions logged and implemented" on 10 July, and then on 16 July reported a seventh consecutive day with zero new kanban tasks and drafts sitting unapproved for 168 hours. That is not chat trivia. That is an operational problem we had not noticed, surfaced by a memory system that was asked a completely generic question.

It has no taste. Asked what numbers had been shared recently, cognee dutifully reported a scheme promising 5 to 25 percent daily returns on a $500 minimum, listed under "Trading Investment Opportunity", presented with the same neutral confidence as everything else. It is an obvious scam post in a group chat. The graph extracted it as a fact about the world because that is what extraction does. Anything built on this needs a layer that knows the difference between something that was said and something that is true.

The comparison was not entirely fair, in the incumbent's favour. Some of what looks like architecture is parameters. Our retrieval layer fetches twelve chunks and is instructed to answer in 150 words or less. Several of its losses were enumeration questions, where a twelve-chunk ceiling and a word limit make a complete list structurally impossible. We are not going to claim a clean architectural verdict until we relax those and re-run.

The one thing it still does better is combine sources. The incumbent's single clear win came from a question that needed the wiki and the CRM and the chats at once. cognee only had chats. That is not a limitation of graphs, it is a limitation of what we ingested, and it points at the obvious next experiment.

Running the test audited the incumbent for free. Asked who is currently looking for funding, our unified memory answered with five company names that came from the inbox source and are plainly junk. That is not a retrieval failure. That is spam sitting in the memory, which we would not have found without watching the system answer a question we already knew the shape of. Also, both systems inherit unresolved sender identities from the messaging bridge, so some answers still name people by raw account identifiers rather than by name.

The verdict is "add", not "replace". Entity extraction genuinely buys something that no amount of tuning a flat retriever will produce, and it buys it cheaply. But a graph of chats is not a memory of a company. The interesting system is both.

Log

• 2026-07-29 — The full backfill fought back, twice. A one-shot ingest of all 5,798 chat-day buckets was killed by the OOM reaper at 5.7GB on a 7.6GB box. The memory-capped retry ran eight hours and died on something more interesting: the embedded graph database crashes with an assertion failure when it reopens and replays a write-ahead log left behind by a killed subprocess — and a clean rebuild hit the same crash with no OOM involved, so this is an upstream bug at our scale, not our residue. Current workaround: every 500-document slice runs as its own process, so each clean exit checkpoints the log away before it can grow teeth, all inside a memory-capped cgroup so the worst case is a dead lab process instead of a dead server.
• 2026-07-28 — Set up cognee in an isolated environment on our own server (embedded Kuzu, LanceDB, SQLite, gpt-4o-mini extraction). Piloted 150 recent chat-day buckets into a 1,329-node, 3,223-edge graph in about 90 seconds. Ran 20 identical questions against both systems and saved the transcripts. The graph took roughly 12 outright, the incumbent 1. Started the full 33,882 message backfill. Next: relax the incumbent's retrieval ceiling and re-run the losses to separate architecture from parameters, and try cognee's MCP server as a read-only network-graph tool inside our coding agent.