Retrieval techniques

Retrieving means finding, among all your chunks, the ones that answer the question. Basic search by meaning already works well for many questions, but fails on others: exact codes, questions that need a filter, chunks that do not say what they are about, or counting questions. Each technique on this page fixes one of those cases, and almost all of them combine by connecting components.

Which technique for which question

Results measured on 17 October 2026 with a test knowledge base (9 Spanish documents from a coffee machine repair service, 40 questions). Each cell shows the percentage of questions where the right answer was the first result / was among the first five. The rows with Local Rerank were measured again on 18 October with its current model. There are few questions per type (6 to 10): they show how each technique behaves, not what will happen with your data.

Question type	Example	Basic search	Best measured technique	With it	Added cost per question
Phrased in other words	”How often should the gaskets that seal the portafilter be renewed?“	80 % / 100 %	Multi Query Retriever	90 to 100 % / 100 %	1 model call, about 1.2 s
Exact code or reference	”Which part is JT-777?“	50 % / 88 %	Local Rerank	75 % / 100 %	No language model, about 1.2 s
Needs a filter (brand, month, type)	“What does error E07 mean on the Vellani Aurora?“	33 % / 33 %	Contextual chunks + hybrid + Local Rerank	33 % / 100 %	No language model at question time, about 2 s
The chunk does not say what it is about	”What did the September report recommend about the water softeners?“	67 % / 100 %	Contextual chunks (basic search)	67 % / 100 %	None at question time
Relationships or counts	”How many open incidents is Lucía Ortega assigned?“	0 of 6 right answers	Neo4j GraphRAG in Text2Cypher with 2 examples	5 of 6	1 model call, about 1.1 s
Not in your documents	”How much does a new Vellani Aurora 3G cost?”	A similarity threshold cannot tell them apart	Local Rerank with Minimum Relevance 0.01	Rejected 2 of 4, without rejecting any of the 36 that did have an answer, but by a very thin margin: one right answer scored 0.011	Nothing beyond the reranking

Two more things from the same measurements:

Reranking also helps when the chunk does not say what it is about: the answer was the first result in 5 of 6 of those questions, against 4 of 6 without reranking. Even so, the best combination still starts by preparing the chunks with Contextual Text Splitter.
Counting questions are not solved with chunks. The best chunk combination answered 2 of 6 correctly. If your users ask “how many”, “who has the most” or “which ones are open”, that data must be in a database that is queried. See GraphRAG with Neo4j.

The techniques, one by one

Similarity search

What any vector database does with Search Type set to Similarity: it returns the Number of Results chunks closest to the question. It is fast (tens of milliseconds) and had the answer among the first five for 83 % of the questions about documents. It is the base for everything else.

Score threshold

With Search Type set to Similarity with score threshold, the database drops results below Search Score Threshold. It removes obvious noise, but it does not detect unanswerable questions: in our measurements, the question about the price of a new machine (not in the documents) looked more similar to the documents than most questions that did have an answer, because it shared the model name. For that, use the relevance of Local Rerank (see below).

Diverse results (MMR)

Qdrant offers MMR (Max Marginal Relevance) in Search Type: instead of the five closest chunks, which sometimes say the same thing, it returns chunks close to the question but different from each other. Useful when you have many near-identical documents (versions, monthly reports). In our measurements it did not change the result clearly.

Keyword search (BM25)

BM25 Retriever finds the chunks that contain the question’s words and gives more weight to rare words, such as a code. It ignores case, accents and punctuation, and compares hyphenated codes as a whole: “jt-777?” finds “JT-777:”. It reads the chunks of the database you connect to Parent document vectorstore (the database’s Vector Store output).

Does BM25 need anything special? No. It uses the same chunks you already stored in the vector database, as they are, with no second ingestion or other format: on every run it reads them from the database (all of them in FAISS, Chroma and Qdrant; up to 10,000 in the others) and builds its keyword index there and then. If you chunked with Contextual Text Splitter, it also searches each chunk’s context sentence. The only thing to watch is volume: with very large collections, reading them on every question takes longer.

On its own, it found exact codes as well as search by meaning (88 % among the first five) and did worse on questions phrased in other words. Its place is hybrid search.

Hybrid search (keywords + meaning)

Ensemble Retriever queries several retrievers at once and merges their lists. For hybrid search, connect to Retrievers the Retriever output of BM25 Retriever and the Retriever output of your vector database. In Weights you decide how much each one counts, in the order you connected them: 0.5, 0.5 is a good starting point.

With equal weights, the answer was among the first five for 83 % of the questions, the same as basic search, and for 100 % of the exact-code questions (88 % without it). It costs no model calls and adds a few milliseconds.

The video in Reranking with Local Rerank builds this hybrid search and reranks it.

Multi Query

Multi Query Retriever asks a model to rewrite the question in several ways (Number of Questions, 3 by default), searches with all of them and merges the results. On questions phrased in other words it put the answer first in 90 % to 100 % of cases depending on the run, because the model does not always write the same versions (80 % without it). It costs one model call per question, about 1.2 seconds. Each run’s log shows the questions it searched with.

This video shows how Multi Query Retriever finds an answer that plain search misses when the question uses other words, and how compression keeps only the useful results.

What you see in the video:

A question phrased in other words, along the lines of “if a repair breaks again a few months later, do you charge me again?”. Plain search does not bring back the repair warranty section.
Multi Query Retriever writes 3 versions of the question and also searches with the original one: now it finds that section.
The run’s log shows the questions it searched with.
Contextual Compression Retriever, with a model that reviews 10 results and drops the ones that do not answer, keeps only a few: the log says how many it found and how many it kept.

Self Query: filters written by a model

Self Query Retriever reads the question and writes the search with its filters. “Error E07 on the Vellani” becomes “error E07” with the filter brand = Vellani. In Metadata Schema you describe the data that can be filtered on (the same format as Metadata Tagger).

With each document’s data (type, brand, month), it had the answer among the first five for 83 % of the questions that needed a filter, against 33 % without filters.
Each run’s log shows the search and the filter the model wrote, for example marca = "Vellani".
It never fails because of the filter: if the model’s answer cannot be read, if the database cannot apply the filter (a range on a text field) or if nothing matches it, it searches without the filter and the log says so.
It works with FAISS, Chroma DB, Qdrant, PGVector and other databases in the palette. With FAISS it got the same 83 % as with Qdrant.
List the values a field takes (enum in the schema) so the model writes them as they are stored.
It costs one model call per question, about 1 second.

This video shows how Self Query Retriever turns a question about a brand and a month into a filter, and finds what plain search misses.

What you see in the video:

In the ingestion flow, the Metadata Tagger schema with each chunk’s data: brand, document type, month and codes.
Self Query Retriever over Qdrant, with the same schema in Metadata Schema and the allowed values of each field.
Asked “Which incidents did the Vellani machines have in July?”, the model writes the filter and(eq("brand", "Vellani"), eq("month", "2026-07")).
With that filter, the two July Vellani chunks come back, with all three incidents: Cafetería Triana, Cervecería Macarena and Cafetería San Bernardo.
Plain search misses some of them.

If you prefer fixed filters, or filters the agent fills in, use the database’s own filters, in the next section.

Metadata filters in the vector database

FAISS, Chroma DB, Qdrant, PGVector, Neo4j and Elasticsearch share the same filter, Filter by Metadata: rows of field, operator and value (Manual) or a JSON such as {"brand": "Vellani"} (Dynamic). It applies to the search, to the Retriever and to tool mode, and the agent can fill in the value itself. How to set it up: Filter by metadata.

Measured on the six databases with the 6 questions that need a filter (first result / among the first five):

How it filters	Result	Added cost
No filter	33 % / 33 %	—
Fixed filter (`brand`, `month` or `type` equal to the question’s)	33 % / 100 %	None: same latency as without a filter
The agent writes the filter (tool mode, DeepSeek)	33-50 % / 83 %	The model call the agent already makes

The filter puts the answer among the first five for every question, but does not lift it to first place: inside the right document, similarity still decides. Combine it with Local Rerank if you need the answer on top.
The agent wrote a correct filter for all 6 questions; the gap with the fixed filter comes from how it rewrote the search.
With list data, such as Metadata Tagger tags, Contains gave the same 100 % in FAISS, Qdrant, PGVector and Elasticsearch. Chroma DB and Neo4j do not look inside lists.

This video shows how a brand filter leaves out the chunks of other brands, first with a fixed condition and then with an agent that fills in the value.

What you see in the video:

Without a filter, “What does error E07 mean?” returns chunks from different brands: the same code means different things on each one.
With one row in Manual mode (Field brand, Operator Equal, Value Vellani), every result is from Vellani.
The list of operators, with In and Contains for the code tags, and how several conditions are combined.
An Agent with the database as a tool, where the AI fills in Condition 1: Value: the tool call shows filter_1_value: "Vellani" and the answer is the Vellani E07 (blocked flowmeter, part VL-1180).
The filter works the same way in FAISS, Chroma DB, PGVector, Neo4j and Elasticsearch.

Reranking with Local Rerank

Local Rerank takes the candidates of a retriever and orders them again with a model that reads the question and each chunk together. It is more precise than comparing vectors and runs inside the platform: free, multilingual (Spanish, English and many more) and with no API key.

Field	What to set
Retriever	The Retriever output of your database or of an Ensemble Retriever. Ask it for more results than you want to keep, for example 12 or 20.
Results to Keep	How many of the best are returned. 4 or 5 is usual.
Minimum Relevance (advanced)	Drops results with a relevance (from 0 to 1) below this value. With 0 (default) nothing is dropped. Right answers can score as low as 0.01, so only very low values are safe; values such as 0.15 already drop right answers.
Model (advanced)	The default one is multilingual. The others are smaller and English only.

With basic search plus Local Rerank, the answer was the first result for 77 % of the questions about documents, against 60 % without reranking, and exact-code questions went from having it first in 50 % of cases to 75 %. Time: about 1.2 seconds per question with 20 candidates, and it grows with the number of candidates.

This video compares plain search with hybrid search plus reranking, on a question about a part reference.

What you see in the video:

Qdrant alone, with its top 5 results, misses part VL-2210.
BM25 Retriever and Qdrant joined in Ensemble Retriever with Weights 0.5, 0.5, followed by Local Rerank: it asks for 20 candidates and keeps the best 5.
The first result is the spare parts list of the Vellani Aurora manual, where “VL-2210: PT1000 temperature probe” appears, with a relevance of 0.56.
The same combination, connected as an Agent tool, answers the question correctly.

Reranking with Cohere Rerank

Cohere Rerank does the same with Cohere’s service: it needs its credential and costs per use. If your texts are not in English, choose a multilingual model in Model (rerank-multilingual-v3.0 or rerank-v3.5), because the default one is for English. We did not include it in the measurements.

Compression with a model

Contextual Compression Retriever cleans up another retriever’s results with a model. With Compressor set to LLMChainFilter, the model reads each result and drops the ones that do not answer. In our measurements it kept 2 or 3 of 10 results on average, never dropped the right answer, and put it first for 88 % of the questions (75 % without it). It costs one model call per result (10 calls for 10 results), in parallel: about 1 second. It is the default compressor. LLMChainExtractor, which keeps only the sentences that answer, put the answer first in 94 % but took 8.8 seconds; EmbeddingsFilter costs no model calls but its threshold has to be tuned for your embeddings model: at 0.4 one question was left with no results.

The video in Multi Query ends with Contextual Compression Retriever filtering 10 results.

Small chunk, whole document

Multi Vector Retriever searches small chunks, which are found more precisely, and returns the bigger piece of the document they belong to, a section or the whole document. One node in Prepare Chunks cuts your documents for the vector database (Piece Size 2000 and Chunk Size 400 by default); another in Search finds them. Useful when the chunk that is found is too short to answer, for example a product sheet or a procedure.

With 400-character chunks and 2000-character pieces, the answer was among the first five results for 93 % of the questions about documents (83 % with plain search), but each search returned 8,400 characters instead of 3,100: plain search with as much text (13 results) found 97 %. Its value is that the agent reads whole sections instead of loose fragments, not that it finds more.

This video compares loose chunks with the whole section that Multi Vector Retriever returns, on a question about a procedure.

What you see in the video:

A Multi Vector Retriever node in Prepare Chunks cuts the documents into 400-character chunks that remember their 2,000-character section, and stores them in Qdrant.
Another node in Search searches those chunks and returns the whole section.
Asked how to replace the NTC probe of a Jaltec Brío 2G step by step, plain search returns loose fragments that are missing steps.
The search with sections returns the full procedure, up to the final check.

Graph search

Neo4j GraphRAG searches by meaning, or by meaning and keywords, over the nodes of a graph, follows the relationships of each result, or writes the query that answers the question. It is the only technique that answered counting questions correctly. See GraphRAG with Neo4j.

The recommended setup

For manuals, policies and FAQs, this is the combination that worked best:

Ingestion. Contextual Text Splitter (Chunk Size 700 to 800, Chunk Overlap 100 to 150, a fast and cheap model) → Metadata Tagger only if you will filter → vector database (Qdrant, or Chroma DB or FAISS if you have no server) with FastEmbed Embeddings (Local). Keep the document type, brand and period on every chunk.
Search. Database in Retriever with Number of Results 20 → Local Rerank with Results to Keep 5. If your users ask a lot about codes, put an Ensemble Retriever with BM25 Retriever and the database in front, with Weights 0.5, 0.5.
Answer. Connect the Tool output of Local Rerank to an Agent, with instructions that ask it to answer only with what it finds.
Records. If you also have incidents, orders or tickets, give the same agent a tool that queries them: Neo4j GraphRAG in Text2Cypher or an SQL query.

With contextual chunks, hybrid search and Local Rerank, the answer was the first result for 80 % of the questions about documents and among the first five for all of them, against 60 % and 83 % for basic search. The cost: one model call per chunk when preparing documents and about 2 seconds of reranking per question.

This flow is built step by step in Step-by-step examples.

How to make the agent say “I don’t know”

An agent with a search tool always gets something back, even when it has nothing to do with the question. To keep it from making things up, combine two things:

Filter out what is unrelated, carefully. With Minimum Relevance in Local Rerank at 0.01, 2 of the 4 unanswerable questions were rejected and none of the answerable ones, but only just: one right answer scored 0.011 and one unanswerable question 0.12. A higher value already drops right answers, so try it with your own questions. A similarity threshold between vectors cannot do this.
Say it in the instructions. Some unanswerable questions look a lot like your documents (an error code that does not exist on a model that does) and only the agent notices, when it reads the results. Add something like this to the agent’s System Prompt:

Answer only with the information your tools return.
If it is not there, say you do not have that data and do not make it up.

That is how the assistant in the video answers when asked the price of a new machine: it says it does not have that data instead of making it up.

If you want the component itself to do that check, use the RAG Agent: it scores every document and, if none is useful, answers in the language of the question that it could not find the information.