Setting up API Outpost

What is Outpost

Outpost provides a front end for Darwinium's API to allow Personally Identifiable Information to be processed, anonymized before it leaves your cloud environment. Any PII data gathered will be end-to-end encrypted and transferred directly to your configured S3 storage.

This can both improve your security posture as Darwinium is not able to access your customers' data, as well as aid in regulatory compliance as you are able to control exactly in which region the data stays.

Like Darwinium Edge workers, Outpost also can execute your rules internally to allow PII to be processed before anonymization, for instance through Regex matching.

Currently only AWS is supported. Contact Darwinium if you wish to deploy on another cloud provider.

What you Need:

• An AWS account.
• Access to the Darwinium Portal
• The ability to create an API gateway.
• The ability to grant Darwinium some restricted IAM roles to configure and maintain lambdas.

Step 1 - Note your ExternalID from the Darwinium Portal

In the Darwinium portal, navigate:

• Make sure in desired node (top right green circle) -> Node settings (left side menu) -> Edge Deployment (tab)
• Add a new target (see 1)
• Assign a Target Name
• Naming Tips: Target names should be short but unique, as they are included within the naming of the lambda function
  • A good choice is the brand/channel that Darwinium will be deployed against.
  • Example: retailweb
• Select Darwinium Outpost - AWS API Gateway as the CDN Vendor (see 2)

A system generated External ID for this target will appear. Copy this down somewhere - it will be used within the AWS configuration roles.

Looks like:

8aabb990-abcd-1234-b6c3-aa0bb0ccf8ac

Step 2 - Note your AWS AccountID

Login to the AWS administrator console and click on your username to the top right. Note your AWS Account ID:

{height="" width=""}
Note your AWS Account ID

Step 3 - Create API Gateway

Create an HTTP API Gateway.
Follow the AWS Instructions to create an API Gateway

Step 4 - Create IAM execution role

Now we need to create the AWS Policies and Roles that will permit Darwinium to run lambdas. First is to Create Lambda Execution Role to allow access to your S3 bucket.

When you are using a Customer Hosted (BYO) S3 bucket, the Darwinium Lambda functions will write encrypted PII data directly to your S3 bucket on your infrastructure, with no part of this write process involving Darwinium infrastructure for maximum security and performance.

In the AWS console: Go to IAM > Roles > Create Role

Specify a custom trust policy with the following contents:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "lambda.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

On the Add Permissions menu add the following policies:

AWSLambdaBasicExecutionRole

S3 write policy (if using BYOS3 storage)

Extra Considerations for BYOS3 Event Storage

BYO S3 Event Storage outlines the process for optionally setting up Darwinium with your own S3 bucket to store event details. If using this functionality, the CloudFront additionally needs to be able to write to your S3, hence why adding the S3 write-only policy that was created upon that setup.

If your BYO S3 Bucket is in a different AWS account to your Lambdas and Lambda execution role, you must also create a bucket policy for it as described in BYO S3 installation.

You will also need to create a duplicate S3 write-only IAM policy for attaching to your Lambda execution role in the account with your Lambdas. This should be identical to the main S3 write-only IAM policy created in the account with the S3 bucket.

Save and note the role ARN

Choose a name for your role and note down the ARN. Will look something like

arn:aws:iam::123456789:role/lambdarole

Step 5 - Create Policy: Management Policy

This IAM policy will be used by Darwinium to create Lambdas and attach them to the API Gateway

Go to IAM > Policies > Create Policy

Find and replace all of the CAPITALISED IDs in the following custom policy with the following

GATEWAYARN = Your API Gateway ARN
Looks like: arn:aws:apigateway:us-east-1::/apis/iy4tsg3xpi

AWSACCOUNTID = Your AWSAccountID from step 2
Looks like: 123456789

EXECUTIONROLEARN = Your execution role ARN from step 4
Looks like: arn:aws:iam::123456789:role/lambdarole

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "iam:GetRole",
        "iam:PassRole"
      ],
      "Resource": [
        "EXECUTIONROLEARN"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "lambda:CreateFunction",
        "lambda:GetFunction",
        "lambda:DeleteProvisionedConcurrencyConfig",
        "lambda:UpdateFunctionConfiguration",
        "lambda:PutFunctionConcurrency",
        "lambda:UpdateFunctionCode",
        "lambda:ListProvisionedConcurrencyConfigs",
        "lambda:AddPermission",
        "lambda:GetFunctionConcurrency",
        "lambda:PutProvisionedConcurrencyConfig",
        "lambda:DeleteFunction",
        "lambda:PublishVersion",
        "lambda:DeleteFunctionConcurrency",
        "lambda:EnableReplication*",
        "lambda:GetPolicy"
      ],
      "Resource": [
        "arn:aws:lambda:*:AWSACCOUNTID:function:dwn*"
      ]
    },
    {
      "Sid": "VisualEditor2",
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:POST"
      ],
      "Resource": [
        "GATEWAYARN/routes",
        "GATEWAYARN/integrations",
        "GATEWAYARN"
      ]
    },
    {
      "Sid": "VisualEditor3",
      "Effect": "Allow",
      "Action": [
        "apigateway:DELETE"
      ],
      "Resource": [
        "GATEWAYARN/integrations/*"
      ]
    },
    {
      "Sid": "VisualEditor4",
      "Effect": "Allow",
      "Action": [
        "apigateway:PATCH"
      ],
      "Resource": [
        "GATEWAYARN/routes/*"
      ]
    },
    {
      "Sid": "VisualEditor5",
      "Effect": "Allow",
      "Action": "lambda:ListVersionsByFunction",
      "Resource": "*"
    }
  ]
}

Save and note the policy name

Save the edited result as the JSON Policy and note the name of the IAM policy as you will add it to the following role.

Step 6 - Create Role: Management IAM Role

This IAM Role will be assumed by Darwinium to create Lambda functions and attach them to an API

• Go to IAM > Roles > Create Role
• Find and replace all of the CAPITALISED IDs in the following custom policy with the following

EXTERNALID = The Darwinium ExternalID that you obtained in step 1
Looks like: 8aabb990-abcd-1234-b6c3-aa0bb0ccf8ac

DWNACCOUNTID = Darwinium's AWS AccountID. Contact Darwinium personnel for this value.
Looks like: 123456789

Use the edited result as the custom trust policy for your new IAM Role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::DWNACCOUNTID:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": [
                        "EXTERNALID"
                    ]
                },
                "ArnLike": {
                    "aws:PrincipalArn": "arn:aws:iam::DWNACCOUNTID:role/crossaccountassumers/*"
                }
            }
        }
    ]
}

(optional, only if duplicate targets required) If multiple identical Darwinium targets, can add them to the ExternalId list This is unusual but can happen when sharing repositories between nodes with unbalanced targets (eg. sandbox has single distribution for everything, production has multiple distributions).

On Add Permissions menu, select the Management Policy that you created in step 5.

Choose an appropriate name for the role, and note down the role's ARN once created. It will look like
arn:aws:iam::123456789:role/dwndeploymentrole

Step 8 - Add the IDs into the Darwinium Portal

Navigate to the Darwinium portal and login as a user who has sufficient privileges to edit node settings.

Navigate:

Select correct node (top right green circle) > Node settings (left side menu) > Edge Deployment (tab) > Target of the EXTERNALID you were using > Deployment config

Enter the details of the AWS constructs you have made:

API Gateway ARNS
Your AWS API Gateway ARNS (possibly created in step 3),

Looks like: arn:aws:apigateway:us-east-1::/apis/iy4tsg3xpi

Management IAM Role ARN

The role to permit Darwinium to deploy lambdas/functions to the CloudFront distribution, created in step 5

Looks like: arn:aws:iam::123456789:role/dwndeploymentrole

Execution role ARN

The role to permit execution, (including writing to S3 if BYOS3) created in step 5

Looks like: arn:aws:iam::123456789:role/lambdarole