Skip to content
Nappai Documentation

Recursive Character Text Splitter

Recursive Character Text Splitter cuts long texts into pieces of a maximum size. It first tries to cut between paragraphs, then between lines and then between words, so each piece makes sense on its own. It is the recommended splitter for most documents.

Long documents are cut into pieces before being stored in a vector database: that way each search returns only the piece that answers, not the whole document.

When to use it

  • Preparing manuals, contracts or FAQs to store them in a vector database.
  • Cutting long texts before passing them to a model.
  • Starting with a splitter that works well without tuning.

Configuration

FieldWhat it does
Chunk SizeMaximum size of each piece, in characters. Default: 1000.
Chunk OverlapHow many characters two consecutive pieces share, so an idea is not cut in half. Default: 200.
InputThe texts to cut, usually the output of a component that loads files.
SeparatorsThe characters to cut on, in order of preference. Empty, paragraph breaks, line breaks and spaces are used.

Output

The Data output holds the pieces, each with the data of the document it comes from. Connect it to Ingest Data of a vector database, such as Chroma DB.

Example

A school prepares its notes for the students’ assistant.

  1. Load the notes with a file component and connect it to Input.
  2. Leave Chunk Size at 1000 and Chunk Overlap at 200.
  3. Connect Data to Ingest Data of a vector database.

Good practices

  • Start with the defaults and adjust Chunk Size if search results fall short or bring too much text.