The best chunking strategy is one that maximizes retrieval precision while preserving enough local context for Claude to answer without guessing. In developer documentation, a strong default is semantic chunking: split by headings and logical sections, keep code blocks intact, and use modest overlap. Many teams land around 300–800 tokens per chunk with 50–150 token overlap, but the “right” size depends on the structure of your docs. Chunk too small and you lose prerequisites and definitions; chunk too large and retrieval returns noisy blocks that waste tokens and dilute relevance.
A practical chunking playbook for developer docs:
API reference: chunk per endpoint or per parameter group (small/medium chunks).
How-to guides: chunk by step group (“Setup”, “Run”, “Troubleshooting”).
Concept pages: chunk by subheading, but include the definition paragraph with related constraints.
Code examples: keep each example as a single chunk; don’t split mid-codeblock.
Tables/config matrices: include the table plus 1–2 paragraphs of explanation; consider normalizing table rows into consistent text lines.
Also store metadata per chunk: version, product, lang, doc_type, heading_path, url. This improves retrieval more than endlessly tweaking chunk length because you can filter and boost the right document types for a query (“how to” questions prefer doc_type=howto).
Chunking and retrieval are a joint system with your vector database. Use Milvus or Zilliz Cloud to store chunk embeddings plus metadata, then evaluate retrieval quality using real queries (search logs, FAQ queries). Track “retrieval hit rate” (did top-k include the correct section?) and adjust chunk boundaries where failures occur. Claude Opus 4.6’s large context window is helpful, but you still don’t want to feed it irrelevant chunks. Good chunking is the fastest path to both better accuracy and lower cost.