Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Guzzle provide this feature for reading and writing data from and to AWS.

We can use this datastore for reading data from Amazon (aws) and do validation and transformations on that data and write that data on CSV, JSON, XML, excel, etc. files.

This datastore is use for both source and target tabs.

info

1. Only Ingestion activity supports File based connectors.
2. Guzzle only supports Access Key based authentication for S3

Steps to create Datastore for S3

1. Click on the action button () from the Datastores section in Left Navigation and select Amazon S3 connector. Alternatively user can launch from Create New Datastore link in Activity authoring UI or Copy Data tool

2. Enter the Datastore name for the new datastore and click OK

3. Update the connection name or leave the default. You can refer to Connection and Environments for more details

4. Enter the additional properties for the storage account as described below:

Property	Description	Required
Access Key ID	AWS access key ID	Yes
Secret Key	AWS secret access key. Specify the secret key. For specify secret key the following options are available: 1. Manual: Provide secret key directly. 2. Azure Key Vault: To use Azure key vault feature user have to integrate Key Vault with Guzzle for that visit here. Give value of the key vault name and secret name where secret key is stored in Azure Key Vault instance.	Yes
Region	Region of the S3 bucket	No
Bucket	S3 Bucket name	Yes
Path	This is the folder path within the S3 bucket. You can specify / (root path) to point to entire container	Yes

Interface for Amazon S3 datastore

Known Limitation