What Is Hybrid RAG

Where we left off

If you've done naive_rag, you already know dense retrieval: embed a query, embed every chunk, rank by cosine similarity, take the top k. It works because embeddings capture meaning, so a question and its answer end up close together in vector space even when they don't share a single word.

That strength is also dense retrieval's weakness. Meaning is exactly what an embedding model is good at, and exactly what a firmware build number, a part number, or a product SKU doesn't have much of. A string like 20240115 or E3D-CHT-04 means almost nothing on its own, an embedding model has no real signal to place it precisely, it just gets folded into "whatever the surrounding paragraph is about."

The specific gap

Hybrid RAG combines dense retrieval (good at meaning, paraphrase, synonyms) with sparse retrieval (good at exact token overlap: IDs, model numbers, acronyms, rare proper nouns), and fuses the two rankings into one. Neither retriever is replaced, both run, and their results are combined so a question can be answered by whichever one actually has the signal for it.

The code, piece by piece

QUESTION = "What is firmware build 20240115 for?"

This question is built entirely around one exact, rare token. The correct document does contain that exact string, but buried in a paragraph about video calls and brick walls, nothing about the sentence around it screams "firmware."

scored = [
    (path.name, cosine_similarity(query_vector, vector))
    for path, vector in zip(paths, doc_vectors)
]

Same dense retrieval as naive_rag, nothing new here, six documents, one query, ranked by cosine similarity. The point of this lesson isn't the code, it's what the ranking looks like once it runs.