Access an AWS S3 Bucket

An S3 Bucket is a container for objects stored in Amazon S3. Every object is contained in a bucket.

S3 Bucket Details

Access Authorization to connect to S3 services is granted by one of two possible methods:

• via the EC2 Instance Profile: Select this box to ensure that the EC2 server on which the Gateway is run has an instance profile to access the S3 bucket, using the Identity Access Management (IAM) role attached to the EC2 instance. Configure the same using the instanceProfile attribute of the element <Authentication instanceProfile="false">.

  Note: The instanceProfile value must be an xsd schema supported Boolean value. The valid values for xsd:boolean are true, false, 0 and 1. Values that are capitalized (for example, TRUE) or abbreviated (for example, T) are not valid.

• via the user's Credential File

  • Define the location and name (on your local server) of the Credential File to make requests to AWS.

    Note: If Instance Profile value is set to true, then the Credential File path value must point to a valid AWS Credential file.

  • Define the relevant Profile Name defined in the Credential File.

    Note: If Instance Profile value is set to true, then Profile Name is a required field and cannot be empty.

Bucket Identification of where the file is located then needs to be provided via the following parameters:

• Region: Define the Region in which Amazon S3 Bucket is deployed.

• Bucket Name: (Case-sensitive) Define the name of the Bucket which hosts the file's location.

• File Description:

  • Object Key: (Case-sensitive) For Extractors, define this object key to read from S3 bucket. This should be the Object Key definition which may contain any folders and the file's name.

  • Output Folder: For Loaders, defining the full object key is not required. Select the output folder in which output EIWM//SVG/CSV file should be placed. This is optional to allow you to specify the folder part of the Object Key as the Gateway automatically generates the filename (normally derived from the input source or changed by the mapping configuration of the #MODEL_NAME# manifest attribute). If the output folder does not exist it will be created. When writing to a bucket this will just be the folder part of the Object Key as the filename is normally derived from the input source or changed by the Gateway's mapping configuration.

    Note: When loading to S3 bucket the object key is not required and the object key is automatically set to the output name of EIWM (applicable only to the loader).

When you want to access an AWS S3 Bucket, you must authenticate your calls. It depends on where the Gateway is instantiated:

• When running the Gateway on an EC2 instance (that is, from within AWS): You should rely on the Instance profile for authentication of your calls to the S3 bucket.

  In this case, the instance profile internally authenticates the call for the user (that is, without the use of any credential file). Instance profiles use an EC2 attached role to authenticate calls to other Amazon Resource Names (ARN).

• When running the Gateway on their own server (that is, from outside AWS): You should rely on the credential file for authentication of their calls to the S3 bucket. A credential file is stored on your system and it contains AWS credentials for accessing AWS resources.

• The Gateway also supports the temporary credentials. The format for temporary credential file is as follows:

  *[aveva-iam-user]
  region=YOUR_REGION_HERE
  aws_access_key_id=YOUR_ACCESS_KEY_HERE
  aws_secret_access_key=YOUR_SECRET_KEY_HERE

  [default]

  source_profile=aveva-iam-user

  role_arn=arn:aws:iam::YOUR_ACCOUNT_NO_HERE:role/YOUR_ROLE_NAME_HERE

  region=YOUR_REGION_HERE*

• Set your credentials in the AWS credentials profile file on your local Windows system, located at:

  C:\Users\USERNAME \.aws\credentials

• Save the credential file format without any extension or with only .txt extension. For more information, refer to the relevant AWS documentation on AWS command line interface.

• There can be multiple profiles in a credential file. You must be very careful when handling a credential file. It must never be shared.

• When reading or writing to S3 buckets, files can be located in folders. In this case, the folder name must be specified in the configuration.

  Note: For security reasons, it is recommended to restrict a user's access to only those AWS resources required by their IAM role. For example:

1. Create an IAM user whose policy does not have access to any AWS resource.

2. Create an IAM role with the following policy to access a specific Amazon S3 Bucket:

   {
   "Version": "2012-10-17",
   "Statement": [
   { "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::<s3-bucket-name>" ] }
   ,
   
   { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:PutObjectAcl" ], "Resource": [ "arn:aws:s3:::<s3-bucket-name>/*" ] }
   ]
   }

   Note: The above example is the minimum recommended configuration. If it is not suitable for your environment, you should investigate what is the most suitable approach for your environment.

3. Assign this role to the IAM user created in Step 1 above.

Define a bucket policy to restrict access to an S3 bucket. For more information, refer to the relevant AWS documentation on how to restrict access to S3 buckets.