Skip to content
Nappai Documentation

Amazon S3

Amazon S3 reads files stored in an Amazon S3 bucket: one specific file or the files in a folder. It can return the text of the files, so an agent or a model can work with it, or the files themselves, to pass them on to another component.

Use it when the documents your flow works with are already in your company’s Amazon storage.

When to use it

  • Read the PDF contracts or invoices that another application drops into an S3 folder.
  • Load the internal documentation stored in a bucket into a knowledge base an agent can search.
  • Review only the new files in a folder every day.
  • Get the list of files in a folder, with their details, without downloading their content.

Configuration

The component needs an AWS S3 bucket credential with the access key, the secret key, the bucket and the region. See Credentials.

FieldWhat it does
Credential (required)The Amazon S3 credential used to access the bucket.
S3 ObjectThe specific file you want to read. When you pick one, the component reads only that file.
S3 PrefixThe folder you want to read. Used when you don’t pick a file in S3 Object.
File TypeWhich files are read or listed from a folder: all or documents (text documents only). Default: all.
S3 Object KeyWhen you connect data to Data with file, the name of the field in that data that holds the file to read. Default: file_id.
S3 Prefix KeyWhen you connect data to Data with file, the name of the field that holds the folder to read. Default: folder_id.
S3 Session TokenOptional. A temporary session token, if your Amazon account requires one.

Advanced options

FieldWhat it does
Data with fileConnect a list of data here to read a different file or folder for each item. The component takes the path from the fields named in S3 Object Key and S3 Prefix Key and adds the content it reads to each item.
Always Include RegexA pattern the files in a folder must match to be read, for example \.pdf$ for PDFs. You can enter several, separated by commas.
Ignore Files RegexA pattern for the files or folders you don’t want to read. You can enter several, separated by commas.
Download File contentIn the Files output, when reading a folder, also downloads the content of each file. When off, the output returns only the list of files with their details.
Max Concurrency DownloadHow many files are downloaded at the same time. Default: 10.
Crawl Only FoldersIn the Files output, returns only the folders, without the files. Off by default.
Max DepthIn the Files output, how many levels of subfolders are explored. Default: 4.
Check Point Database Directory, Check Point Database and Check Point TablePoint to a table that lists the files already processed. Files listed there are not read again. Leave them empty if you don’t use one.
Batch SizeLimits how many files of a folder are read in each run. 0 means no limit.

Output

  • Data: the text of each file read, along with details such as its name and location. Connect it to an agent, a model or a document store.
  • Files: the files as they are, to pass them to components that work with files. If you read a folder without turning on Download File content, it returns the list of files with their details.

Example

An accounts team receives supplier invoices in the invoices/pending folder of a bucket:

  1. Add Amazon S3, pick the credential and, in S3 Prefix, the folder invoices/pending.
  2. In Always Include Regex, enter \.pdf$ to read only PDFs.
  3. Connect Data to an agent that extracts the supplier, the date and the amount from each invoice.
  4. Save the result to a spreadsheet.

Good practices

  • Read only the folder you need: the more specific it is, the faster the run.
  • Use Batch Size when the folder holds many files and you prefer to process them a few at a time.