The support bot started confidently answering with the wrong docs — Debug Lab

Debug LabSEV2

The support bot started confidently answering with the wrong docs

#rag-retrieval-failure

00:00MEDIUM impact

Current signals

Irrelevant Answer Rate

31%

Retrieval Latency (s)

2.1s

Relevant Chunks Retrieved (of top-k)

4/20

Embedding Model Version Match

2/10

Incident

Support engineers are reporting the internal RAG assistant is giving confident, well-formatted answers that cite the wrong documentation. It started after last night's embedding reindex.

Started 40 minutes ago

Working hypotheses

Nothing yet — investigate the evidence to surface possible causes.

Stuck?

The support RAG assistant is confidently citing the wrong documentation for about a third of queries, starting shortly after last night's reindex.

Overview reviewed.

· Answers are confidently wrong — this is a retrieval problem, not a generation problem.

Investigation timeline

12:43Incident started
12:44Review incident overview