Troubleshooting Deployments

After setting up Darwinium, you should follow Deployment Confirmation to ensure that a journey has been published to your CDN infrastructure. You should test your journey by visiting the URLs for any steps defined.

Below are some possible scenarios:

No new events are visible in Investigations

Check your journey

If you are not able to see events following a deployment, the first thing to do is check your journey and it's steps:

1. Go to the Workflows section of the Darwinium portal
2. Open journeys.yaml and check that:
   1. The journey you expect to be seeing events from is declared in the journeys section of this file
   2. Targets are defined and correspond to the hostnames you are expecting in your journey steps. In particular, you should ensure that the appropriate domains are declared
3. Open the journey that you anticipate events being generated from - ensure that an event type is set and that the hostname corresponds exactly to the one you are attempting to capture events for
4. Force a rebuild/re-deploy:
   1. Open a terminal in the workflows environment
   2. Run the following:

      git commit --allow-empty -m "forcing rebuild"
      git push

   3. Use deployment manager to deploy a new build to your infrastructure (or your terraform pipeline if applicable).
   4. Open the node build history by going to admin > nodes > (node) > deployments - ensure that the latest deployment has occurred successfully

 Check if workers have been deployed to your cloud provider

This check assumes that you have appropriate access to your CDN Dashboard. It is recommended that your infrastructure team completes this process.

Cloudflare:

1. Go to https://dash.cloudflare.com
2. Select your site
3. From the left-hand navigation menu, select Workers Routes. You should see a list of workers deployed by Darwinium. Each Darwinium worker's name is prepended with "dwn_"

Cloudfront:

This guide assumes that you are a user with access to the AWS console and have experience in the use of Lambdas and Cloudfront.

1. Open the AWS Console
2. Navigate to Cloudflare
3. Select the Distribution ID corresponding to your Darwinium Deployment (see: CloudFront Deployment)
4. Select "Behaviors" from the top nav. You should see a list of behaviors corresponding to steps in your Darwinium journeys.

Check Lambdas have been created and function associations have been setup:

1.  Select a behavior from the list and click "edit"
2. Scroll to the bottom to reveal function associations. There should be at least one Darwinium function associated with the behavior

Check that Lambdas exist:

1. In the AWS console, navigate to "Lambdas"
2. Select "functions" from the left-hand navigation menu.
3. You should see at least 1 function prefixed with "dwn_" in your function list if a successful deployment has occured

A page or API on my site seems to be taking too long to respond after deploying Darwinium

• If your backend services are working fine, the most likely cause of this issue is journey configuration. Check your step dependencies and ensure that there are no models or calls that cause a delay in the response being sent back to the end-user machine.
• If configuration looks good, the next step is to republish your journey by following the guide at the top of this page.
• If this fails, the issue may be the result of a timeout in an API call made to Darwinium or a 3rd party service. By default, the worker is designed to timeout in 2000ms or less with external calls. Please contact Darwinuim support (see: Contacting Darwinium) to assess the issue further.

There has been a catastrophic failure- How do I remove Darwinium in an emergency?

Darwinium's Edge workers have been designed to fail-open; this means that in the unlikely event that Darwinium's back-end API is down, your request will continue to process after the worker attempts to reach Darwinium and fails. However, if for some reason your system gets into a catastrophic state, the first  thing to do is remove all of Darwinium's workers/routes from your edge infrastructure.

On Cloudflare

1. Follow Check if workers have been deployed to your cloud provider
2. For each Darwinium route:
   1. click edit
   2. click remove on the modal dialog that appears

On CloudFront:

1. Open the AWS Console
2. Navigate to CloudFront
3. Select the Distribution ID corresponding to your Darwinium Deployment (see: CloudFront Deployment)
4. Select "Behaviors" from the top nav. You should see a list of behaviors corresponding to steps in your Darwinium journeys.
5. For each Darwinium behavior:
   1. Click "edit"
   2. Scroll to Function associations
   3. Change associations for Viewer request, Viewer response, Origin Request, Origin Response to "No Association"
   4. Save Changes