Tuning RRF's k

Where we left off

Lesson 8 introduced RRF's constant k (60, the standard default) without explaining what changing it actually does. This lesson does two things: shows k's real effect on a fused ranking, using a synthetic example where dense and sparse genuinely disagree, then sweeps k on this course's own ten labeled questions, and is explicit about a trap that sweep falls straight into.

The code, piece by piece

synthetic_dense = ["X", "Y", "P", "Q", "R", "S"]   # X is #1, dead last in sparse
synthetic_sparse = ["P", "Q", "Y", "R", "S", "X"]  # Y is #3 in both, consistent

X gets a spectacular rank-1 finish in one ranking and a terrible one in the other. Y never wins outright, but never does badly either. This is the case k actually controls: does one great finish outweigh one terrible one, or does consistent-but-unspectacular win out?

Why this lesson is the contaminated case, on purpose

The README convention set for this series (docs/RAG-SERIES-PLAN/README.md) flags exactly this: tuning a hyperparameter against the same labeled set you later report a score on invalidates that score. This lesson does precisely that, sweeping k against the same ten questions Lesson 17 scores hybrid retrieval on, deliberately, so you see the trap from the inside rather than being told about it in the abstract. A real pipeline keeps a tuning set and a reporting set separate, or, more honestly at this scale, just keeps k at its standard default (60) and doesn't pretend five or ten questions can respectably tune anything.