HuggingFace
HuggingFace connects your flows to AI models published on Hugging Face. You can use it in two ways: as the model of an agent, by connecting its Model output to the agent’s Model input, or on its own, so that it answers the text that reaches its Input.
When to use it
- Give an agent the model it uses to understand requests, decide which tools to use and answer.
- Write, summarise or classify texts inside a flow, without an agent.
- Process a list of texts one by one, for example all the reviews of a month, with Batch Mode.
- Use a specific open model, or your own model published on Hugging Face.
Configuration
In the Common tab:
| Field | What it does |
|---|---|
| Model ID (required) | The model that answers. Choose it from the list or choose custom to type another one. Default: meta-llama/Llama-3.3-70B-Instruct. |
| Custom Model ID (required) | Appears when you choose custom in Model ID. Type the model’s identifier as shown on Hugging Face. |
| Max New Tokens | The maximum length of the answer, in tokens (pieces of words). Default: 512. |
| Temperature | How varied the answers are: low values give steadier, more predictable answers; high values, more creative ones. Default: 0.1. |
| Inference Endpoint (required) | The address of the service that runs the model. Leave it as it is to use Hugging Face. If you type the address of your own service, it is used as is, without adding the Model ID. |
| Credential (required) | The credential of the service HuggingFace API. Pick one from the list (Choose an option) or create one with Add new credential. See Credentials. |
| Connection | What to connect |
|---|---|
| Input | The text the model must answer when it works on its own, for example the message of a Chat Input or the result of a Prompt. |
Advanced options
In the Advanced tab:
| Field | What it does |
|---|---|
| System Message | General instructions for the model when it works on its own: its role, the tone, what to avoid. You can type them or connect them from another component. |
| Stream | Shows the answer in the chat while it is being written, instead of waiting until it is finished. It only has an effect in the chat. Default: off. |
| Task | The kind of task asked of the model: generate text, summarise or translate. Default: text-generation. |
| Top K and Top P and Typical P | Other ways to control how varied the answer is. Temperature is usually enough. |
| Repetition Penalty | How much repetition is penalised; 1.0 means no penalty. |
| Retry Attempts | How many times it tries to connect to the service. Default: 1. |
| Other options | Model Keyword Arguments: fine-tuning settings for advanced users. Leave them as they are if you don’t need them. |
Batch processing
With the Batch Mode switch on, the component runs once for each item of the list you connect to Batch Input and returns all the results together as a list. Parallel Executions sets how many items are processed at the same time (1 to 5). Skip if Empty Batch Input skips the component when the list arrives empty, and Skip Branch if Empty Output stops the following components when the result is empty. More in Key concepts.
Output
| Output | What it contains |
|---|---|
| Model | The configured model. It sits at the top of the component. Connect it to the Model input of an agent or of any other component that asks for a model. |
| Text | The model’s answer to the text received in Input. Connect it, for example, to a Chat Output or to another component that works with text. |
The two ways of using it exclude each other: when you connect the Model output, the component hides Input and Text, because the model now works inside the other component; and if you connect Input, Model is hidden.
Example
The cooperative Huerta Viva wants to summarise its product sheets with an open model. It connects a Prompt to HuggingFace’s Input, chooses the credential and keeps the default model. The Text output with the summary is saved in the next step of the flow.
Good practices
- Use a low Temperature when you need exact answers, such as classifying or extracting data, and raise it for more creative texts.
- If you use the model with an agent, configure only the model here: the instructions and the tools are set in the agent.