★ Markdown-Aware chunking, visualized
Markdown-Aware Chunking for RAG
Markdown-aware chunking splits the document at headings, so each chunk is one section. It records the heading path, for example 'Refund policy > How to request a refund > Processing times', as metadata you can store with the embedding.
- Runs in your browser
- No signup
1,181 characters · loading tokenizer… · nothing leaves your browser
Markdown · 400 chars · overlap 40
6 chunks
- 49
- avg tokens
- 29 / 71
- min / max
- 0
- overlap tokens
- $5.86
- embed 1M docs
Loading the tokenizer; token counts are estimates until it's ready.
one chunkoverlapClick a chunk to inspect it.
Chunk inspector
Click any coloured chunk in the document to see its text, size, heading path and what it shares with its neighbours.
Chunk sizes
Tokens per chunk (estimated). A tight cluster means predictable chunks.
Embedding cost assumes text-embedding-3-small at $0.02 per 1M tokens (OpenAI, checked 8 Nov 2026), counting overlap twice.
Export config
Copy-ready code for the strategy and settings above. Comments flag where a library differs.
from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter
headers = [("#", "h1"), ("##", "h2"), ("###", "h3"), ("####", "h4")]
by_heading = MarkdownHeaderTextSplitter(headers_to_split_on=headers, strip_headers=False)
sections = by_heading.split_text(text) # each Document carries its heading path in .metadata
# Cap section size; heading metadata is copied to every piece.
capper = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=40)
chunks = capper.split_documents(sections)Your retrieval is only as good as your chunks.
I build RAG systems with evals, so you know when the answers get better or worse.
How it works
Markdown-aware chunking splits the document at headings, so each chunk is one section. It records the heading path, for example 'Refund policy > How to request a refund > Processing times', as metadata you can store with the embedding.
Headings inside code fences are ignored. When a section is longer than the size limit, it falls back to recursive splitting inside that section, and every piece keeps the section's heading path.
When to use it
Good fit
- Documentation, help centres and knowledge bases written in markdown.
- When the section title is part of the answer's meaning.
- When you want to filter or boost retrieval by section.
Watch out for
- Very short sections become tiny chunks with little to embed.
- Documents with no headings fall back to plain recursive splitting.
- Inconsistent heading levels produce odd heading paths.
Where to start
Use a size cap that fits a typical section, and prepend the heading path to each chunk's text before embedding so the vector carries the context too.
Then test it: write down twenty real questions, check which chunks come back for each, and compare strategies on the same set. The main visualizer lets you put two strategies side by side.
Questions engineers actually ask
Should the heading be in the chunk text or only in metadata?
How do I do this in LangChain?
What about HTML or PDFs?
What chunk size should I use for RAG?
Does my text leave the browser?
Other strategies
More from the workbench
Fine print
Tools: on the house.
Engineer: by the hour.
The visualizer was free. Production RAG with evals is what I build.
I'm Himanshu, an AI and full-stack engineer. I build MVPs, AI agents and the growth plumbing around them for early-stage startups.
