Scorer
Scorer evaluates content against the criteria you give it and assigns a score from 1 to 100, where 100 is the best. An AI model reads the content and scores it; if the text is long, it evaluates it in parts and takes the average. If you ask for it, it also adds an explanation of the score.
When to use it
- Rate the quality of your support team’s replies for clarity and friendliness.
- Score applications or proposals against your company’s requirements.
- Measure how urgent a request is so you can prioritize the highest scores.
- Give an agent a way to evaluate texts with the same criteria every time.
Configuration
| Field | What it does |
|---|---|
| Data (required) | The content you want to score: a text, a message or a record. |
| Criteria (required) | The evaluation criteria, in your own words. For example: “Clarity of the message: clear = high score, unclear = low score”. |
| Additional Context | Extra information or instructions about what is being evaluated. Optional. |
| Max chunks | Maximum number of text parts that are evaluated. Default: 5. Keeps the cost down for long texts: anything beyond that number is not taken into account. |
| Include Justification | When on, adds the explanation for the score. |
| Model (required) | The AI model that does the evaluation. Connect a model component here. |
Advanced options
| Field | What it does |
|---|---|
| chunk size | Size of each part, in characters. Default: 1500. |
| chunk overlap | Characters shared by two consecutive parts, so ideas are not cut in half. Default: 150. |
Output
- Score: a record with the original content and a
scorefield with the score. If Include Justification is on, it also includes ajustificationfield. If the model does not give a valid score,scoreis 0. - Tool: the component as a tool for an agent: the agent sends the text to score and gets back its score (and justification, if enabled), with the criteria configured here.
Example
Every week you want to review the quality of your team’s email replies. You connect each reply to Scorer with the criterion “Answers the customer’s question, is friendly and proposes a clear next step” and turn on Include Justification. Replies with a low score are sent to the team lead along with the explanation.
Good practices
- Write specific criteria and explain what deserves a high score and what deserves a low one.
- Score each piece of content separately: connect a single text or record per run, or go through them with Loop.