Two banks can deploy the exact same model and still get different accuracy. The gap sits in what each chatbot is allowed to look up before it answers.
Two chatbots, same model, different answers
A vendor demo runs on GPT-5 or Gemini. So does the chatbot at the bank next door. Ask both what changed on a customer's mortgage rate this week, and you can get two different answers from the identical model. What changed was what each chatbot was allowed to check before it answered.
What a static knowledge base actually is
Most banking chatbots launch on a knowledge base: a fixed set of documents, FAQs, and product sheets, indexed once and queried from then on. It's fast to build and cheap to run, and it works fine right up until something in the bank changes.
A fee moves. A policy gets updated by compliance on a Tuesday. The knowledge base doesn't know, because nobody re-indexed it, and the chatbot answers from what it still believes is current. eGain's research into chatbot failure names the pattern directly: "the model is rarely the real problem. The knowledge feeding it is" (eGain, 2026).
What changes with live retrieval
Live, or dynamic, retrieval means the chatbot queries current systems, a website, a database, a core, at the moment a customer asks. That's instead of a snapshot taken once at build time. The answer reflects what's true right now, instead of what was true when someone last updated a document.
The research backs the shift. One peer-reviewed study on reducing hallucination in generated output found that adding retrieval-augmented generation cut the hallucination rate from 68% to 10% against the same base model (Béchard & Marquez Ayala, NAACL 2024). The model stayed the same. The retrieval layer underneath it is what moved.
Google Cloud's own case study on Hong Leong Bank shows the same pattern in production. Marina, HLB's assistant, handles 67% of customer support queries and reaches 99% AI accuracy running on Gemini 2.5 Flash (Google Cloud, 2026). For pre-login questions, Marina retrieves answers by searching HLB's public website live, instead of relying on a manually updated database that goes stale between updates. That's the static-versus-dynamic shift, in production, from the vendor's own account.
It's also why researchers keep testing whether retrieval-based systems recognize when their own retrieved context doesn't actually support an answer, the exact moment a chatbot should say it doesn't know instead of guessing anyway (AAAI, 2026).
Retrieval and governance solve two different problems
Fixing retrieval fixes what the chatbot is working from. It doesn't fix what happens after the model generates an answer, right or wrong. That's a separate layer: policy enforcement and an audit trail that catches a bad answer before it reaches a customer. Backbase calls this the Authority Layer, and it enforces the same checks regardless of which model or retrieval setup sits underneath it.
This is the same principle behind Backbase's Conversational Banking: every response gets grounded in live systems and the bank's own policies before it reaches a customer, so an answer is never invented. A bank needs both layers. Confusing one for the other is how a chatbot ships that's either well-governed and wrong, or accurate and unaccountable.
The same logic applies to which model answers a question in the first place, a separate decision covered in Why the model isn't the moat.
What to ask a vendor before you trust the accuracy number
Does the chatbot query live systems, or a knowledge base indexed at some point in the past? Ask when it was last re-indexed and how.
What happens when a product, rate, or policy changes? If the answer involves a manual re-upload, the chatbot is running on a static base regardless of what the model is.
Can you trace a specific wrong answer back to its source? If a vendor can't show which document or system fed a given response, there's no way to fix the actual cause when accuracy drops. That kind of traceability is a core requirement of the Banking OS underneath a production deployment.
Frequently asked questions
Is a chatbot's accuracy mostly about which AI model it uses?
Usually not. Model choice matters, but a chatbot working from an outdated knowledge base produces wrong answers no matter which model generates the response. Retrieval architecture sets the ceiling.
What's the difference between a static knowledge base and live retrieval?
A static knowledge base is indexed once and queried from a fixed snapshot. Live retrieval queries current systems at the moment a customer asks, so the answer reflects current data.
Does fixing retrieval also fix compliance risk?
No. Retrieval determines whether an answer is accurate. A separate governance layer determines whether an inaccurate or non-compliant answer reaches the customer anyway. Both are needed.
How much can retrieval architecture actually change accuracy?
Documented improvements are significant: one peer-reviewed study found retrieval-augmented generation cut a hallucination rate from 68% to 10% on the same base model. Hong Leong Bank's Gemini-based assistant reaches 99% accuracy in production, per Google Cloud's own case study.
