S3 Account and User Import

User Access Keys

To authenticate using Access Keys, provide an access token and security key for an AWS user that has permissions to create files in the specified bucket. The token and security key can be obtained from the AWS Console. For more information on creating AWS users and security keys in the AWS Console, refer to the Managing access keys for IAM users article.

IAM User Roles

The IAM User Role method allows access to the S3 bucket to be controlled via AWS roles without the exchange of any secret keys. Authentication is accomplished by creating a customer-owned AWS role which has permissions to write to the customer’s bucket.  In addition, the role is configured so that a provided Gainsight AWS user can assume the role.

To configure IAM Roles authentication:

IMPORTANT: In the JSON examples below, elements  in green should be substituted with values from the AWS instance.

1. Create an AWS role, with the following policy, substituting in the ARN for your bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::aptrinsic-test-bucket"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::aptrinsic-test-bucket/*"
        }
    ]
}

2. Define Trust Relationship for the PX role, by substituting the PX Subscription ID obtained from the top of the Company Details page in your PX subscription. (https://app.aptrinsic.com/settings/subscription)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::581300747314:user/gspx-ext-s3-exports"
                ]
            },
            "Action": "sts:AssumeRole",
            "Condition": {                
                "StringEquals": {
                    "sts:ExternalId": "PX subscription ID"
                }
            }
        }
    ]
}

3. Create a bucket with a policy that grants access via the role created in Step 1 to the bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "allow-access-px-bucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::99999:role/s3-export-role"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::aptrinsic-test-bucket/*"
        }
    ]
}

Bucket Permissions

You need to provide an authentication token (and security token) that provides read/write access to the import folder in the bucket.

Process/Data Flow

Schedule: 

The PX import job runs every 15 minutes. If any input files are found, the file is imported. If more than one file is found in a folder, the files are imported serially.

Data Flow: 

For each day’s data, you need to create a subdirectory with a name created using the current date in the format dt=YYYYMMDD. The exported user file needs to be placed in that subdirectory. When a data file is ready to be processed, place an empty file named “_SUCCESS” into the directory to signal that the write process is complete and PX can begin processing the file. Once the job is complete, the files are moved to the processed folder. For more details, refer to the Validate Import using Status File section of this article.

Input Queue Example:

 ./current 
   /dt=20210228 
      _SUCCESS 
      users.csv.gz

Processed Files Example:

./processed 
   /20210228 
      status.txt
        _SUCCESS
        users.csv.gz

Bucket Structure

You need to configure the Bucket Structure as defined in the following sections.

CSV Structure

The following is the CSV structure suggested by Gainsight PX:

• Standard CSV (RFC0180)
  For more information, refer to the CSV specifications article.
• Gzipped, with extension “.csv.gz”.
• Filename prefix (example: “users”) is not significant.
• All columns must be defined in the User attributes area of PX, the column headers should be the “apiName”, not the View Name of the attribute.
• Data must be parseable according to types defined in PX (see below) • Limited to 350,000 rows per file

Note: The id column is mandatory.

Data Types

 The following are the data types:

• String: Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double quotes. If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote. 
• Numeric: Integer or floating-point values 
• Boolean: “true” or “false” (not case sensitive)
• Date: Should be in the format specified by the Date Format value on the configuration screen.
  The valid characters are listed here.
  Note: The proper abbreviation for month is “MM”, not “mm”

Typical values are: 

• Date Only: 

                 Format String: YYYYMMDD

                 Example Data: 20211225 

• Date/Time (UTC time-zone): 

                Format String: yyyy-MM-dd'T'HH:mm:ssX 

                Example Data: 2021-06-25T13:55:42Z

Validate Import Using Status File 

When an import job finishes, the imported file is moved to the processed folder and a status.txt file is created. The first line contains the result, either “SUCCESS” or “ERROR”. Subsequent lines contain a human-readable message with more details about the results of the import.

Success Example:

SUCCESS
4155 records were uploaded

Error Example:

ERROR
Parsing Errors:[
       Unknown PX Column: 'flantana_carteret',
       Missing required Column: 'id' 
]

Validate Import Using Status Endpoint

You can provide an optional HTTP endpoint for receiving notifications from PX when jobs are finished (errors or success).

• Example Endpoint: https://www.example.com/prd/data-export
• Method: POST
• Authentication: x-api-key header with api key provided by customer
• Body: { folder_path: “{path to processed folder}”, success: {true|false}, message: “{load result details}” }
• Response: 200 okay, body: String

Error Handling

If any of the above validations fail, the file is skipped. The errors are listed in the status.txt file. For more information, refer to the Validate Import Using Status File section.