Wrapping It as a Service

Where we left off

Lesson 23 split this course's pipeline into ingest() (run once) and ask() (run per question), specifically because that boundary matches how a real web service works: build the index at startup, answer requests as they arrive. This lesson wires those two functions into an actual, if minimal, FastAPI app.

The code, piece by piece

@asynccontextmanager
async def lifespan(app: FastAPI):
    chroma_client = chromadb.Client()
    app.state.collection = ingest(NOTES_DIR, chroma_client)
    yield

FastAPI's lifespan runs once, when the app starts (and again, after yield, when it shuts down, unused here). ingest() runs exactly here, producing a collection stored on app.state, a place any request handler can reach it from. This is the "expensive setup happens once" half of Lesson 23's whole point, now literally wired to the server's own startup event instead of the top of a script.

@app.get("/ask", response_model=AskResponse)
def ask_endpoint(q: str, k: int = 2) -> AskResponse:
    return AskResponse(answer=ask(q, app.state.collection, k))

One route, GET /ask?q=..., that does nothing but call ask() (Lesson 23, unchanged) against the collection built at startup. This is the entire service: ingest() once, ask() per request, exactly the division Lesson 23 drew.

with TestClient(app) as test_client:
    response = test_client.get("/ask", params={"q": question})

TestClient runs the FastAPI app in-process, no real network socket involved, so uv run python lesson.py can exercise the API the same way every other lesson's main() runs and prints something, without needing a second terminal running a live server.