URL
URL opens the web pages you give it and passes their text to the flow. It is the simplest way for an agent or a model to work with information published on a website.
If you only want specific parts of the page, such as headings or links, or need to read addresses that arrive in a list of data, use URL Advanced.
When to use it
- Summarize an article or a press release from its address.
- Give an agent the content of your website’s terms or FAQ page.
- Compare the product pages of several suppliers.
- Check a page periodically to spot changes in its content.
Configuration
| Field | What it does |
|---|---|
| URLs | The addresses of the pages you want to read. Click + to add more than one, or write them on a single line separated by commas. If they don’t start with http:// or https://, the component adds it. |
| Selectors to extract | Optional. The parts of the page whose text you want to keep, separated by commas: tag names such as h1, p or page selectors such as .article. Only the text of those parts is returned. Leave it empty to get all the text. |
Output
The Data output returns one item per page, with its text and details such as the source address and the page title.
Connect it to an agent or a model to summarize or analyze the content, or to Parse Data to turn it into text.
Example
A marketing team wants a summary of what a competitor publishes:
- Add URL and enter the address of the competitor’s news page in URLs.
- Connect Data to an agent with the instruction “Summarize the news on this page in three sentences”.
- Send the summary to the team’s channel.
Good practices
- Some websites don’t show their content until it loads in a browser, or require signing in; in those cases the text may arrive incomplete.
- Make sure you are allowed to use the content of the pages you read.
Related components
- URL Advanced: extracts title, text, links and images, and reads addresses from a list of data.
- RSS Feeds: reads the articles of RSS feeds.