Image Understanding
Image Understanding analyses images with Google’s Gemini models. It can do two things: describe in words what it sees, following your instructions, or detect the objects that appear and tell you where each one is in the image.
When to use it
- Automatically write descriptions for your catalogue’s product photos.
- Review photos of incidents or building work and count the items that appear.
- Check whether a photo of a store shelf contains the products it should.
- Crop each product out of a group photo to handle it separately.
Configuration
You need a Google Gemini credential with your Google key. See Credentials.
| Field | What it does |
|---|---|
| Credential (required) | The Google Gemini credential used to analyse the images. |
| Images | The images you want to analyse. You can connect one or several. |
| Prompt | What you want to know about the image. Default: Describe the image. With Object Detection, use it to say which objects to look for, for example “Detect every bottle”. |
| Model (required) | The Gemini model that analyses the images. If the list is empty, press the refresh button after choosing the credential. |
| Tool Function (required) | Which analysis to run: Describe Image (default) to get a description, or Object Detection to locate objects. |
Advanced options
These only appear with Object Detection.
| Field | What it does |
|---|---|
| Generate Overlay Image | Creates a copy of the image with a box and a label over each detected object. On by default. Turn it off if you only need the data, so it takes less time. |
| Extract Individual Objects | Crops each detected object as a separate image. Off by default. |
Output
The Data output depends on the analysis you chose:
- Describe Image: a single result with the description (
image_description), the number of images and their names. If you connect several images, they are described together in one answer. - Object Detection: one result per image, with the list of detected objects (
detections), each with its label and position, and the number of objects (num_objects). If you turned on the advanced options, it also includes the image with the boxes and the crop of each object.
If an image cannot be analysed, its result includes an error field and the other images are still processed.
Example
A supermarket wants to check from photos that its drinks shelves are fully stocked.
- Connect the shelf photos to Images.
- Choose the Google Gemini credential and a model in Model.
- In Tool Function, choose Object Detection, and in Prompt type “Detect every bottle and every empty gap on the shelf”.
- Connect Data to a component that compares
num_objectswith the expected number and alerts the manager if products are missing.
Related components
- Describe Image: describes images with the language model you connect.
- Easy OCR: reads the text that appears in an image.
- Gemini OCR: extracts the text of documents with Gemini.