A team is building a RAG pipeline over long technical manuals stored in Cloud Storage. They plan to compute embeddings for each chunk of text and store them for vector search. The manuals are dense and full of cross-references, and the team must balance retrieval precision against the cost of computing and storing embeddings. Which chunking strategy most directly supports both goals?
- ASplit each manual into overlapping chunks of a few hundred tokens with a small overlap between adjacent chunks, then embed each chunk Correct
- BEmbed the entire manual as a single document so each manual is represented by exactly one vector and storage cost is minimised
- CSplit each manual into one chunk per sentence and embed every sentence individually for maximum granularity
- DEmbed only the table of contents and section headings of each manual, then retrieve full sections by heading match at query time
Why A is correct: Chunking into a few hundred tokens keeps each vector focused on a single topic so cosine similarity discriminates between sections, while a small overlap preserves context that straddles chunk boundaries. This is the standard RAG trade-off that buys retrieval precision without exploding the vector count.
Why B is wrong: One vector per manual minimises storage but is tempting only on cost grounds: a single embedding for an entire dense document averages away topic detail, so retrieval cannot locate the specific section that answers a user query, defeating the point of RAG.
Why C is wrong: Per-sentence chunks are tempting because finer granularity sounds like better precision, but very short chunks lose surrounding context, multiply embedding cost dramatically, and force the generator to stitch many tiny snippets together to answer one question.
Why D is wrong: Indexing only headings is tempting because it is cheap, but headings are short and generic, so the embedding distance between user questions and headings is a poor proxy for actual content relevance and many useful passages will never be retrieved.