Skip to content
Nappai Documentation

Language Recursive Text Splitter

Language Recursive Text Splitter cuts texts written in a programming or markup language, such as HTML, Markdown or LaTeX, respecting their structure: functions, sections or tags.

Long documents are cut into pieces before being stored in a vector database: that way each search returns only the piece that answers, not the whole document.

When to use it

  • Preparing technical documentation in Markdown or HTML pages for an assistant.
  • Storing source code so an agent can search it.

Configuration

FieldWhat it does
Chunk SizeMaximum size of each piece, in characters. Default: 1000.
Chunk OverlapHow many characters two consecutive pieces share, so an idea is not cut in half. Default: 200.
InputThe texts to cut, usually the output of a component that loads files.
Code LanguageThe text’s language, for example markdown, html, latex, js or python (default).

Output

The Data output holds the pieces, each with the data of the document it comes from. Connect it to Ingest Data of a vector database, such as Chroma DB.

Example

A product team keeps its documentation in Markdown.

  1. Connect the files to Input and choose markdown in Code Language.
  2. Connect Data to a vector database for the internal assistant.

Good practices

  • Start with the defaults and adjust Chunk Size if search results fall short or bring too much text.