Normalizing Scores Before Fusion

Where we left off

Lesson 8's RRF never looks at scores, only rank, and that's usually the right default (Lesson 7 showed why picking a weight is fragile). But RRF throws away real information: it can't tell "barely edged out the runner-up" from "crushed the field," both are just "rank 1." Sometimes that distinction matters, a reranking step, a confidence threshold before answering at all, anything downstream that wants "how sure are we," not just "what's the order." This lesson looks more closely at how to normalize scores onto a comparable scale when you do need them, picking up where Lesson 7's quick min-max pass left off.

The code, piece by piece

def min_max_normalize(scores: dict[str, float]) -> dict[str, float]:
    low, high = min(values), max(values)
    return {name: (score - low) / spread for name, score in scores.items()}

Lesson 7's approach: stretch the lowest score to 0, the highest to 1, everything else proportionally in between. Simple, always lands in [0, 1], but sensitive to the shape of the distribution, see below.

def z_score_normalize(scores: dict[str, float]) -> dict[str, float]:
    mean = statistics.mean(values)
    stdev = statistics.pstdev(values)
    return {name: (score - mean) / stdev for name, score in scores.items()}

A different question: not "where does this score fall between the min and max," but "how many standard deviations from average is this score." A document exactly average scores 0, one far above average scores well above 1. This doesn't force anything into [0, 1], the scale is now "standard deviations," which needs its own handling before mixing with a dense score.