Local Rerank
Local Rerank takes the results of a search and orders them again by how well each one answers the question. It keeps the best ones and, if you want, drops the ones that have nothing to do with it. The model that does this runs inside the platform itself: it needs no credential or account with any provider, there is no cost per search and your texts never leave NappAI.
A search by meaning compares the question and each piece separately, so the right piece sometimes ends up in position 8 or 15. Local Rerank reads the question and each piece together and gives them a relevance score from 0 to 1. That is why it does better with codes, references and specific questions.
When to use it
- The search finds the right piece, but not always among the first ones.
- Your users ask about error codes, part references or order numbers.
- You want to pass the agent only the four or five most useful pieces, instead of twenty.
- You want to drop results unrelated to the question, so the agent says it does not know instead of making something up.
- Your documents are in Spanish, English or several languages and you don’t want to pay for a reranking service.
How it works
- Ask the search for more results than you want to keep, for example 20.
- Local Rerank scores each one against the question.
- It returns the best ones, as many as you set in Results to Keep (for example, 5), from most to least relevant.
The results can come in two ways: by connecting a retriever to Retriever (for example, the Retriever output of Qdrant or of Ensemble Retriever) or by connecting a list of results to Search Results (for example, the Results output of a vector database). If you connect both, the retriever is used.
Configuration
| Field | What it does |
|---|---|
| Search Query | The question the results are ordered for. When an agent uses the component as a tool, the agent writes it. |
| Retriever | A retriever whose results are reordered. Set it to return more results than you keep, for example 20 in its Number of Results. |
| Search Results | The results to reorder, if you don’t connect a retriever. |
| Results to Keep | How many of the best results are returned, from 1 to 50. Default: 5. |
Advanced options
| Field | What it does |
|---|---|
| Model | The model that scores the results. Default: BAAI/bge-reranker-v2-m3-int8, which understands Spanish, English and many other languages. Xenova/ms-marco-MiniLM-L-12-v2, Xenova/ms-marco-MiniLM-L-6-v2, jinaai/jina-reranker-v1-turbo-en and jinaai/jina-reranker-v1-tiny-en are smaller and only for English texts. BAAI/bge-reranker-base is also available. For texts in Spanish, keep the default. |
| Minimum Relevance | Results scoring below this, from 0 to 1, are dropped. With 0 (default) none are dropped. Right answers can score as low as 0.01, so only very low values are safe; values such as 0.15 already drop right answers. |
Output
- Results: the reordered results for Search Query, each with its text, its data and its score in
rerank_score. Pass them to a prompt or an agent. - Retriever: a retriever that searches and reorders in one step. Connect it where a retriever is expected, such as Retrievers of RAG Agent. It needs a retriever connected to Retriever.
Use as an agent tool
Connect Local Rerank to the Tools input of an Agent and the agent searches and reorders on its own: it writes Search Query with the question. In the Tool tab, Tool Name and Tool Description are the name and explanation the agent uses to recognise the tool and decide when to use it. It is called reranked_search by default; change the description to say what your documents contain, for example “Searches the coffee machines’ technical manuals”.
Example
A coffee machine repair service wants its assistant to find in the manuals what each error code means.
- Add a Qdrant database with Operation set to
Retriever, the manuals collection and Number of Results set to 12. - Connect its Retriever output to the Retriever input of Local Rerank and set Results to Keep to 4.
- Connect Local Rerank to the agent’s tools, with the description “Searches the technical manuals”.
- Ask “The Q90 shows error E07, what does it mean?”. The agent gets the four most relevant pieces, and the first one explains E07 for that model, not for another machine with the same code.
What it improves: measured figures
Measured on 18 October 2026 on a test knowledge base (9 documents of a fictional repair service, 40 questions). These figures come from a single synthetic set: they show how the technique behaves, not a promise for your data.
- The right piece first: asking for 20 results and keeping 5, the piece that answers came first for 77 % of the questions, against 60 % with search by meaning alone.
- Exact codes and references: with that same setup, the answer was in the top 5 for every question (1.00), against 0.88 with search by meaning alone.
- Together with pieces that carry context: combining Contextual Text Splitter, hybrid search and Local Rerank was the best setup in the test: the right piece came first for 80 % of the questions and in the top 5 for all of them, against 60 % and 83 % with search by meaning alone.
- Questions the documents cannot answer: with Minimum Relevance at 0.01, 2 of the 4 unanswerable questions were dropped, and none of the 36 that did have an answer, but by a very thin margin: one right answer scored 0.011 and one unanswerable question 0.12. At 0.15, 4 questions that did have an answer would have been dropped too.
- Time: about 1.2 seconds per question with 20 results, on our test machine. The more results you ask for, the longer it takes.
Good practices
- Ask the search for 3 to 5 times the results you keep: for example, 20 to keep 5.
- If your pieces don’t say what they are about (a step of a procedure that only says “the part”), prepare them first with Contextual Text Splitter: together with hybrid search and Local Rerank it was the best combination in the test.
- Leave Minimum Relevance at 0 unless you have tried a threshold with your own questions. To make the agent say it does not know, its instructions are what works reliably.
- Even with a threshold, tell the agent in its instructions to say so when the answer is not in the results. A question about something that does not exist but looks very much like something that does can pass the threshold.
- The first search takes a little longer because the model is loaded; the following ones find it ready.
Local Rerank or Cohere Rerank
| Local Rerank | Cohere Rerank | |
|---|---|---|
| Where it runs | Inside NappAI | On Cohere’s service |
| Credential | None needed | Cohere API credential |
| Cost | No cost per search | What Cohere charges per search |
| Your texts | Stay on the platform | Are sent to Cohere |
| Drop irrelevant results | Yes, with Minimum Relevance | No |
| What it takes | A retriever or a list of results | A list of results |
| Agent tool | Yes, it searches and reorders on its own | No |
Video
This video compares plain search with hybrid search plus reranking, on a question about a part reference.
What you see in the video:
- Qdrant alone, with its top 5 results, misses part VL-2210.
- BM25 Retriever and Qdrant joined in Ensemble Retriever with Weights
0.5, 0.5, followed by Local Rerank: it asks for 20 candidates and keeps the best 5. - The first result is the spare parts list of the Vellani Aurora manual, where “VL-2210: PT1000 temperature probe” appears, with a relevance of 0.56.
- The same combination, connected as an Agent tool, answers the question correctly.
Related components
- Contextual Text Splitter: adds context to each piece before it is stored; the best preparation for reranking.
- Ensemble Retriever: combines search by words and by meaning before reordering.
- Qdrant: a vector database to start from.
- Cohere Rerank: the same idea with Cohere’s service.
To see how it fits in a complete system, see Search techniques in the RAG guide.