Semantic routing
A routing technique that uses embedding similarity to send each request to the right model, agent, or workflow — instead of brittle keyword rules or expensive LLM classifiers.
Semantic routing solves the "which path should this request take" problem efficiently. You precompute embeddings for canonical example queries per path (path = which agent, which model, which workflow). Each new request gets embedded; cosine similarity decides routing.
Compared to alternatives: keyword rules are brittle (synonyms break them); LLM classifiers are accurate but cost a full LLM call per route decision; semantic routing approaches LLM accuracy at near-zero per-request cost.
In 2026 the leading open-source library is semantic-router by Aurelio Labs. Production use cases: intent classification for chatbots, agent selection in multi-agent systems, query type detection in search.
Frequently asked
How accurate is semantic routing?+
With well-curated examples, 90–95% on most classification tasks. Less accurate than a tuned LLM classifier, more accurate than keyword rules, dramatically cheaper than either.
Where does semantic routing fail?+
On highly nuanced classification where context matters more than surface meaning. For those cases, fall back to LLM classifiers. Hybrid systems (semantic routing as first pass, LLM verification on low-confidence) work well.