Skip to main content

Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Guzzle provide this feature for reading and writing data from and to AWS.

We can use this datastore for reading data from Amazon (aws) and do validation and transformations on that data and write that data on CSV, JSON, XML, excel, etc. files.

This datastore is use for both source and target tabs.

  1. Only Ingestion activity supports File based connectors.
  2. Guzzle only supports Access Key based authentication for S3

Steps to create Datastore for S3#

  1. Click on the action button (image alt text) from the Datastores section in Left Navigation and select Amazon S3 connector. Alternatively user can launch from Create New Datastore link in Activity authoring UI or Copy Data tool

  2. Enter the Datastore name for the new datastore and click OK

  3. Update the connection name or leave the default. You can refer to Connection and Environments for more details

  4. Enter the additional properties for the storage account as described below:

Access Key IDAWS access key IDYes
Secret KeyAWS secret access key.
Specify the secret key.
For specify secret key the following options are available:
1. Manual: Provide secret key directly.
2. Azure Key Vault: To use Azure key vault feature user have to integrate Key Vault with Guzzle for that visit here. Give value of the key vault name and secret name where secret key is stored in Azure Key Vault instance.
RegionRegion of the S3 bucketNo
BucketS3 Bucket nameYes
PathThis is the folder path within the S3 bucket. You can specify / (root path) to point to entire containerYes

Interface for Amazon S3 datastore#

Known Limitation#