New blogMaking an AI Agent You Can Trust With Money
logo

★ Sentence-Based chunking, visualized

Sentence-Based Chunking for RAG

Sentence chunking first splits the text into sentences, here with the browser's built-in sentence segmenter, then packs whole sentences into chunks until the next one would exceed the size limit.

  • Runs in your browser
  • No signup

1,181 characters · loading tokenizer… · nothing leaves your browser

chars
chars

Sentence · 300 chars · overlap 60

6 chunks

54
avg tokens
29 / 71
min / max
28
overlap tokens
$6.50
embed 1M docs

Loading the tokenizer; token counts are estimates until it's ready.

# Refund policy for annual plans Annual plans are billed once a year, up front. This page explains when you can get money back and how long it takes. ## Eligibility You can request a full refund within 30 days of the first annual charge. After 30 days, refunds are prorated for the unused months, minus any discount you received for paying annually. Renewals follow the same rule: the 30-day window starts again on each renewal date. ## How to request a refund Open Settings, then Billing, and choose "Request refund". Pick the reason that fits best. If you are cancelling because of a bug, include the error message; it often gets fixed faster than the refund is processed. ### Processing times Card refunds usually appear within 5 to 10 business days, depending on your bank. Bank transfers can take longer. We send an email when the refund leaves our side. ## Exceptions Refunds are not available for usage-based charges such as extra API calls or storage, because those costs are incurred when you use them. Accounts closed for abuse are not eligible. ## Contact Still stuck? Email the billing team with your account ID and the last four digits of the card on file.

one chunkoverlapClick a chunk to inspect it.

Chunk inspector

Click any coloured chunk in the document to see its text, size, heading path and what it shares with its neighbours.

Chunk sizes

Tokens per chunk (estimated). A tight cluster means predictable chunks.

1293714535361169chunk size (tokens), bucket start

Embedding cost assumes text-embedding-3-small at $0.02 per 1M tokens (OpenAI, checked 8 Nov 2026), counting overlap twice.

Export config

Copy-ready code for the strategy and settings above. Comments flag where a library differs.

from langchain_text_splitters import NLTKTextSplitter
# pip install nltk, then: python -m nltk.downloader punkt_tab

# Packs whole sentences up to chunk_size (characters). Overlap is applied to the merged pieces.
splitter = NLTKTextSplitter(chunk_size=300, chunk_overlap=60)
chunks = splitter.split_text(text)

Your retrieval is only as good as your chunks.

I build RAG systems with evals, so you know when the answers get better or worse.

Fix my RAG
01

How it works

Sentence chunking first splits the text into sentences, here with the browser's built-in sentence segmenter, then packs whole sentences into chunks until the next one would exceed the size limit.

Overlap is done in whole sentences: the last sentences of a chunk that fit in the overlap budget are repeated at the start of the next one. No chunk ever starts or ends mid-sentence.

02

When to use it

Good fit

  • Prose where each sentence carries a complete fact: policies, FAQs, articles.
  • When readable chunks matter, for example when you show retrieved text to users.
  • Question answering over documents without much heading structure.

Watch out for

  • One very long sentence becomes one oversized chunk (flagged in the tool).
  • Abbreviations and numbered lists can confuse sentence detection.
  • Ignores headings, so context from the section title is lost.
03

Where to start

Pack enough sentences to hold one idea, often a short paragraph, and overlap by one sentence. Check a few oversized chunks to see whether your text has run-on sentences or lists.

Then test it: write down twenty real questions, check which chunks come back for each, and compare strategies on the same set. The main visualizer lets you put two strategies side by side.

04

Questions engineers actually ask

How are sentences detected?
With Intl.Segmenter, the browser's Unicode sentence segmentation, with a simple punctuation fallback. Production pipelines often use NLTK or spaCy for the same job.
What happens to a sentence longer than the chunk size?
It becomes its own chunk and is flagged as oversized, because splitting it would break the strategy's main promise.
Is sentence chunking better than recursive?
Often similar on prose. Sentence chunking guarantees clean boundaries; recursive respects paragraphs. Compare both on your own questions.
What chunk size should I use for RAG?
There is no universal number. A few hundred tokens with a small overlap is a common starting point, but the right size depends on your documents and questions. Build a small evaluation set and compare sizes on it.
Does my text leave the browser?
No. Splitting, tokenizing and the visualization all run in your browser. Nothing you paste is sent anywhere.
05

Other strategies

06

More from the workbench

07 · AI & LLMsToken CounterToken counts per model, cost per call and how much of the context window you are using.02 · AI & LLMsLLM API Cost CalculatorYour monthly bill across Claude, GPT, Gemini and open models, side by side.13 · AI & LLMsAI Agent vs HireThe fully loaded cost of a hire versus an AI agent doing the same work, review included.

Fine print

Tools: on the house.
Engineer: by the hour.

The visualizer was free. Production RAG with evals is what I build.

I'm Himanshu, an AI and full-stack engineer. I build MVPs, AI agents and the growth plumbing around them for early-stage startups.

Hire me →My projects