Custom Deployment of Darwinium at the Edge using your own infrastructure

Darwinium ships with its own deployment manager that manages the process of uploading and deploying edge artifacts generated by the platform to your CDN vendor. There are, however, instances where customers need to handle this deployment themselves due to change management and infrastructure management considerations.

 Darwinium can be customized such that the process of deployment is delegated to your own internal CI/CD pipelines and Infrastructure-as-code tooling such as Terraform, AWS CloudFormation or Pulumi. In order to facilitate this, Darwinium provides functionality that enables these systems to access build artifacts

An overview of the standard Darwinium deployment lifecycle

Going from journey-to-edge consists of 2 phases: build and deploy.

1. When a user pushes a change to the main branch of a Darwinium node’s git repository, an automatic process compiles defined scripts, features, journeys and manifests into a collection of edge-ready artifacts. The composition of these artifacts will vary, depending on the CDN targeted. This is stored as a release on Darwinium’s SaaS infrastructure.
2. A user in a role with Manage Deployments may then select a built release and deploy this to CDN infrastructure using the credentials saved in the node’s configuration UI
3. Darwinium’s Backend API infrastructure is updated to support the journey deployment through an internal B/G deployment mechanism

Darwinium provides deployment management through a Deployment Manager UI (accessible from admin > nodes). This provides the ability to roll-back to any release created in the node’s build history

Customized lifecycle using your own tooling

In a customized lifecycle, Darwinium provides the build infrastructure, and the customer provides necessary provisions for deployment.

The process can be described as follows:

1. User pushes a change to the main branch of the node’s Darwinium git repository
2. The journey is automatically built into edge-ready artifacts that are then stored on Darwinium’s SaaS platform
3. Darwinium fires a webhook containing details of the build (such as the commit hash, the author, and where to download the build artifacts) to the customer’s CI pipeline
4. Customer pipeline receives the webhook, downloads a .tar.gz bundle of the webhooks from Darwinium, and unpacks it to a nominated location
5. Prior to commencing deployment, the customer calls the active deployment API to notify Darwinium’s backend to begin supporting worker communications specific to the version being deployed
6. For each enabled deployment target defined, there exists a corresponding folder in the unpacked build artifact. This folder contains a file, workers.json, which provides details of the worker (lambda function in the case of Cloudfront) and the route upon which it should exist (Function association for Cloudfront)
7. Using a combination of workers.json and the files contained in the artifact, the customer’s IaaC system (terraform, for example) defines appropriate cloud infrastructure and configuration for deployment

Deployment Target configuration

Journeys.yaml, located on the node’s git repository defines one-or-more targets. A target defines:

• The CDN vendor used (type: ‘cloudfront’ or ‘cloudflare’)
• A target name (name)
• Whether the target is enabled (enabled)
• A list of hostnames for which the target should be used (valid_host_list)  
  
  

Darwinium supports targets of mixed types, as well as multiple targets of the same type to support complex usecases such as when different AWS accounts are used to cover different domains.

The Deployment Artifact bundle contains folders that correspond to the names of each target.

Webhook

Darwinium can be configured to trigger a webhook when an automated build is completed. This webhook is highly customizable and can define your own headers, HTTP method, query parameters and body.

The webhook template is defined in YAML and expects the following parameters to be defined:

Template variables

The following template variables are provided with examples of usecases where they may be used:

Understanding workers.json

Each deployment target defined in journeys.yaml results in a folder in the deployment artifact bundle containing the worker source code and a workers.json file

Cloudfront targets

Cloudflare targets

Currently the official Cloudflare terraform modules lag behind capabilities provided by wrangler and the cloudflare edge workers API. As a result, Darwinium has a customized version of the Cloudflare Terraform Provider that supports additional file uploads. The following will need to be included in your Terraform's required providers:

terraform {
  required_providers {
    cloudflare = {
      source = "app.terraform.io/dwn-edge-deployment/cloudflare"
      version = "4.7.0-dwn-0.1"
    }
  }
}

A custom Terraform API key will be needed to access this provider. Darwinium will provide you with this at time of setup

Example – integrating with Github Actions and Terraform 

 A boilerplate project demonstrating how Darwinium build artifacts can be integrated into a github action that triggers a terraform step can be found below. It is strongly recommended that you review this sample, particularly for Cloudflare deployments:

In the example, the github action responds to a Darwinium-invoked webhook and uses steps to download and unpack the artifact from Darwinium using an MTLS credential stored in the repo secrets before invoking a terraform step to read workers.json and synchronize lambda@edge functions and function associations from this definition.