Introducing ChromaDB

Where we left off

Lesson 19 measured the actual cost of a linear scan at scale. This lesson introduces the fix: chromadb, a real, open-source vector database, still entirely local (no server, no account, no API key beyond Gemini's), that indexes vectors instead of scanning every one of them on every query.

What changes, and what doesn't

The four-stage shape (chunk, embed, retrieve, generate) doesn't change at all. What changes is where the vectors live and how they're searched: a Python list you scan yourself, versus a database that indexes them and answers "most similar" queries without checking every record one by one. This lesson only replaces the storage and retrieval half, nothing about chunking or generation is any different from Lesson 15.

The code, piece by piece

chroma_client = chromadb.Client()
collection = chroma_client.create_collection(name="notes")

chromadb.Client() with no arguments creates an ephemeral, in-memory database, everything it stores disappears when the program exits, the database equivalent of this course's own Python list, just with real indexing underneath. A collection is chromadb's name for what this course has been calling a "vector store," a named group of records.

collection.add(
    ids=[path.stem for path in paths],
    documents=texts,
    embeddings=vectors,
    metadatas=[{"source": path.name} for path in paths],
)

Unlike this course's list-of-dicts (one dict per record, each with its own text/embedding/source keys), chromadb wants four parallel lists, matched up by position: unique ids, the documents (raw text), the embeddings (vectors this course already computed with embed_texts, chromadb doesn't compute its own), and metadatas (one dict of extra fields per record, same idea as Lesson 12's source field).

results = collection.query(query_embeddings=[query_vector], n_results=2)

The retrieval half of Lesson 6's retrieve(), expressed as a method call instead of a manual sort: pass a query embedding (still computed with this course's own embed_texts, chromadb never sees the raw query text here), get back the n_results closest matches, already ranked.