31 August 2026 · Dustin Erasmus, Technical and Digital Platforms Director

Why We Use Semantic Search in Our AI Chatbots

Every AI chatbot we build for a client under AI Voice & Chatbot Agents is only as good as its ability to find the right answer before the language model ever starts writing a response. Get the search wrong, and the smartest model in the world just makes something up, or shrugs and gives a vague non-answer — neither of which is acceptable when a real customer is asking about their account, their order, or a policy that actually matters to them. That's why every chatbot we host runs semantic search rather than keyword matching; it's the single biggest quality lever in that product, more than the choice of language model sitting on top of it. Keyword search matches words. Semantic search matches meaning, and that distinction sounds small right up until you watch it play out against a real visitor asking a real question in their own words rather than the exact phrasing a knowledge base happens to use.

Why semantic search wins

A visitor might ask "how much does it cost to get this sorted out" when the knowledge base talks about "fees" and "charges." Someone else might ask "when's the last day to sign up" when the source page actually says "closing date." Keyword search sees no shared words between the question and the answer, and finds nothing, even though the exact information the visitor needs is sitting right there in the knowledge base, phrased differently than they happened to ask it. When that happens, the chatbot either misses the match entirely and says it doesn't know, or — worse — the underlying language model tries to answer from general knowledge instead of the client's actual content, and gets the specifics wrong in a way that sounds confident and looks like an answer, even though it isn't grounded in anything real.

Semantic search doesn't have that ceiling, because it isn't comparing spelling in the first place. Every piece of a client's knowledge base — FAQs, policies, service pages, site content — is converted into a numeric representation of what it's actually about, not just which words happen to appear in it. A visitor's question gets converted the same way at the moment they type it, and the system finds the closest matches by meaning rather than by shared vocabulary. "How much does it cost" and "what are the fees" land in the same neighbourhood of that representation even though they don't share a single keyword, because the underlying model has learned that they're asking the same thing. That's the property that makes the rest of this approach possible: it scales with however a real person happens to phrase a question, not with however the content happened to be written.

Understands paraphrasing, synonyms, and everyday language instead of requiring visitors to guess the site's exact wording
Keeps working as a knowledge base grows and its vocabulary gets more varied — keyword search degrades, semantic search doesn't
Surfaces the right answer before the AI model starts writing, which is what actually stops it from making things up
Still catches exact terms, acronyms, and reference numbers verbatim, because we run it alongside keyword search rather than instead of it

That last point matters: we didn't throw keyword search away when we added semantic search on top of it. Each chatbot blends both — semantic search for meaning, keyword search for exact matches — and merges the two ranked lists into a single result set rather than picking one approach and living with its blind spots. That gives us the best of both: a chatbot that catches the paraphrased, conversationally worded question and, in the same query, still catches the exact case number, product code, or acronym a keyword-only system would have nailed anyway. Dropping keyword matching entirely to chase semantic search alone would trade one blind spot for another; running them together closes the gap that either one leaves on its own, which is the whole point of blending them rather than choosing between them.

How we use it

This runs underneath every chatbot we host as part of AI Voice & Chatbot Agents, invisibly to the visitor, who never sees any of the search mechanics and shouldn't have to — from their side, they just ask a question and get an answer that's actually grounded in the client's real content. What's different on our side, and what actually determines whether that experience holds up once a knowledge base grows past a handful of pages, is how the underlying search is built rather than which chat interface sits on top of it. The five pieces below are what make that reliable in practice: how content gets turned into something searchable by meaning, how that search stays isolated between clients, and how it keeps working without manual intervention every time a client updates their own site.

Embeddings — every chunk of a client's knowledge base is converted into a vector (a list of numbers representing meaning) using a dedicated embeddings model. The same happens to each visitor question at chat time.
Similarity search — we compare the question's vector against every knowledge chunk's vector and pull the closest matches, the ones that mean the same thing regardless of phrasing.
Blended ranking — semantic results and keyword results are merged using a ranking method that rewards content appearing near the top of either list, so a strong keyword match and a strong semantic match both surface.
Per-client isolation — every client's knowledge base, embeddings, and search results stay fully separated, no cross-contamination between chatbots.
No retraining required — when a client updates their site or knowledge base, an Automated Workflow & Lead Routing job re-crawls and re-embeds the changed content automatically. No model retraining, no downtime, no manual step for the client.

The same shift is happening in search engines

This isn't a one-off trick built only for chatbots — it's the same underlying shift we work with under Answer Engine Optimisation (AEO/GEO). Tools like ChatGPT, Perplexity, and Google's AI Overviews retrieve and cite content the same way our chatbots do: by matching meaning, not exact keywords, when deciding what to surface and quote in response to someone's question. A client's content that's structured to be understood by meaning — clear headings, direct answers, content organised around what people actually ask rather than around internal jargon — performs better in both places at once: inside their own chatbot, answering their customers directly, and inside the AI search results those same customers are increasingly using instead of a classic Google search to research a purchase before they ever actually reach the client's own site at all.

That's the real payoff of building this in-house rather than bolting on a generic vendor widget that treats search as an afterthought: the same investment in making content machine-understandable compounds across every AI touchpoint a client has, not just the one chat window running on their own site. Structure content once, for meaning rather than for keywords, and it pays off inside the chatbot, inside AI search results, and inside whatever the next AI-driven discovery channel turns out to be — because the underlying skill being built is understanding what content actually means, which doesn't go stale the way keyword tricks tuned to one specific search algorithm eventually do. A tactic aimed at gaming one search engine's current ranking formula stops working the day that formula changes; understanding what a piece of content actually means doesn't expire the same way, whichever platform ends up doing the retrieving.

Why this matters if you're evaluating an AI chatbot

If you're comparing chatbot vendors or evaluating whether to build one in-house, ask this question directly: does it use semantic search, or keyword search dressed up behind a modern-looking chat interface? A lot of "AI chatbots" on the market are simply a language model bolted onto keyword search from a decade ago, with none of the retrieval layer actually upgraded to match the model sitting on top of it. The AI can be as good as anyone wants to pay for — if it can't reliably find the right information in the first place, it's guessing, and a confident-sounding guess is worse than no answer at all when it turns out to be wrong. Ask a vendor to show you what happens when a real question doesn't match the knowledge base's exact wording; that single test tells you more than any feature list.

We build the search layer to match the quality of the model sitting on top of it, rather than treating retrieval as a solved problem and spending the whole budget on the flashier, more visible half of the system. That's the part that doesn't show up in a sales demo, where a handful of pre-tested questions can make almost any chatbot look competent regardless of what's actually happening underneath. It's also the part that decides whether a real visitor, asking a real question in their own words on a bad day, gets a real answer grounded in the client's actual content — or gets handed a confident-sounding guess that happens to be wrong, which is the failure mode that actually costs a business trust. Trust lost that way is hard to earn back, which is exactly why it belongs at the front of the build, not the end.