Can a network tell you which two organisations should meet — and can you prove it's not guessing?

Hypothesis

The ecosystem graph shows who is already connected. The more useful question is the one it can't answer: of 569 organisations, which pairs should be talking and aren't?

Recommendation engines are easy to build and easy to fool. Any embedding model will happily produce a ranked list of "similar" organisations, and the list will look plausible whether or not it means anything. So the real hypothesis was about measurement, not matching: can we produce suggestions and hold a number that says how good they are?

What we built

Two layers over the same 569 organisations, both toggleable at /ecosystem.

Similar vision — semantic similarity between what organisations say they do. The scoring kernel is a direct port of Max Semenchuk's hromada-strategy-collab, which matches Ukrainian municipalities for inter-municipal cooperation. It blends a bipartite best-match score against a centroid score, with length-capped averaging so that an organisation with a long, comprehensive mission statement can't hub the entire graph. We ported it verbatim, tests included, then adapted three things for a corpus of marketing copy rather than municipal strategy documents.

Complements — directional. Claude Opus 5 reads each organisation and extracts what it offers and what it needs against a closed 25-tag vocabulary. An edge appears when one organisation's offer meets another's need, so the link has a direction and a stated reason: "Mesh Fair offers market access — Toniic needs it."

To measure any of it we needed ground truth nobody had written down. It turned out to be hiding in the org descriptions: organisations that name their partners in prose. Mining those mentions — with guards against the obvious cheats, like one name being a substring of the other — produced 44 known-good pairs to score against.

Learnings

• The measurement was worth more than the model. Our first validation computed median rank over surviving pairs only — conditioning the statistic on the very thing it measured, and reporting a flattering rank of 2. Rescored honestly against all 568 candidates, the real number was 20. Still good (random is 284), but the flattering version would have hidden every subsequent regression.

• Geography made it worse. The spec locked a 25% weight on shared location, on the reasonable intuition that co-located organisations are easier to introduce. Measured, that weight cost real accuracy: recall@10 went 0.386 → 0.455 → 0.477 as the geo weight fell 0.25 → 0.10 → 0.00. We kept a small weight as a tiebreaker and wrote the evidence into the spec so the next reader doesn't restore the "obvious" value.

• Nine agencies shared one sentence. Nine Oslo event agencies had the identical line "Event Byrå Oslo" in their descriptions and scored ~0.9 against each other. The standard fix — down-weighting common phrases — provably cannot work here: for a two-line organisation, a uniform weight leaves a weighted average unchanged. Shared boilerplate had to be dropped at the corpus level instead.

• A duplicate can hide a fact. The reason strings are named after the matched need, so two different offers hitting one need produced the same sentence twice. Because only the first two reasons are shown, the duplicate was displacing a real second reason. Deduplicating the display strings — but never the score, since two offers covering one need is genuinely stronger — made suggestions more informative, not just tidier.

• Where it still fails is legible. The pairs we miss cluster at ranks 11–50, not at 300. That's a truncation artefact rather than a semantic failure, which is the good kind of wrong: it tells you the ranking works and the cutoff is the thing to tune.

Signal, honestly measured: a known-good pair sits at median rank 20 out of 568, 75% land inside the top 69, and 64% survive into the shipped graph. Suggestions are suggestions — the graph proposes, and a human decides whether there's a conversation worth having.

Log

• 2026-07-31 — Both layers live at /ecosystem. Rank regression wired into the test suite so accuracy can't silently rot.