Pixi-EQ is our agentic RAG system: it answers questions from your own documents with cited sources. We built it because the naive pattern — embed everything, retrieve top-k, generate — fails in a specific, repeatable way: it produces confident prose that sounds grounded and is not.
Three components moved it from plausible to checkable.
1. Query planning before retrieval
Real questions are rarely retrievable as written. “Can we ship this order?” decomposes into inventory state, credit status, and fulfillment policy — three different lookups, possibly three different systems. Pixi-EQ plans queries first: it rewrites the question into explicit sub-queries, each with a target source. When the plan cannot name its sources, that is already useful signal — the question is probably out of scope, and saying so beats hallucinating an answer.
2. Multi-step retrieval with intermediate checks
Single-shot retrieval assumes one hop is enough. In practice, answering a real operational question means following references: a policy cites a definition; a definition lives in another document. Pixi-EQ retrieves, reads what came back, and decides whether another hop is needed — with a hard budget on steps so latency stays bounded.
3. Answer verification against sources
Before an answer is returned, each factual claim is checked back against retrieved passages. The design principle: unsupported claims get removed, or the answer downgrades from conclusion to “here is what I found.” A grounded system should visibly refuse to overclaim.
What evaluation has to measure
Evaluation should not be vibes. Build the evaluation set from real operational questions — each with a known source document and expected answer shape — and hold the system to three metrics: citation precision (does the cited passage actually support the claim), refusal rate on out-of-scope questions (refusals are a feature), and end-to-end latency under realistic document volumes.