Introduction

The following components of Darwinium deployment can be included in the health check scope:

• Edge Workloads - Amazon CloudFront/Lambda@Edge (when using CloudFront as a CDN)
• Edge Workloads - Cloudflare (when using Cloudflare as a CDN)
• S3 buckets (when using self-managed PII storage)

Edge Workloads - Amazon CloudFront/Lambda@Edge

Amazon CloudFront is integrated with Amazon CloudWatch and automatically publishes operational metrics for distributions and Lambda@Edge functions. These metrics can be accessed via the CloudFront console or by using the CloudFront API or CLI.

The CloudFront metrics don't count against CloudWatch quotas and don't incur any additional cost.

A CloudWatch dashboard is created for CloudFront and Lambda@Edge metrics (a CloudFormation sample template is provided).

Standard CloudFront metrics

• Requests - the total number of viewer requests
• Bytes downloaded - number of bytes downloaded by viewers for GET, HEAD, and OPTIONS requests.
• Bytes uploaded - number of bytes uploaded to the origin using POST and PUT requests.
• 4xx error rate
• 5xx error rate
• Total error rate

Additional CloudFront metrics

These metrics are recommended for more in-depth monitoring of CloudFront. They incur additional cost. For reference please check https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/viewing-cloudfront-metrics.html#monitoring-console.distributions-additional-pricing

• Cache hit rate - the percentage of all cacheable requests for which CloudFront served the content from its cache.
• Origin latency - the total time spent from when CloudFront receives a request to when it starts providing a response to the network (not the viewer), for requests that are served from the origin, not the CloudFront cache.
• Error rate by status code

Note, that when you turn on additional metrics, CloudFront starts sending metrics to CloudWatch in the US East (N. Virginia) Region. A fee is charged for each metric. This rate is charged only once per month, per metric (up to 8 metrics per distribution). This is a fixed rate, so your cost remains the same regardless of the number of requests or responses that the CloudFront distribution receives or sends. Use CloudWatch Pricing Calculator to estimate the costs https://aws.amazon.com/cloudwatch/pricing/#Pricing_calculator.

Standard Lambda@Edge metrics

The following default metrics are available for Lambda@Edge functions with no additional charge:

• Error rate for Lambda@Edge
• Lambda execution errors
• Lambda invalid responses
• Lambda throttles

Amazon CloudWatch Alarms

Alerting can be configured in Amazon CloudWatch based on the metrics collected for CloudFront and Lambda@Edge services. A number of alerts are optionally included in the provided sample CloudFormation template.

Please refer to “Using Amazon CloudWatch alarms” guide for details.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html

Edge Workloads - Cloudflare

Monitoring Cloudflare Account

Cloudflare provides insights into health and performance of Cloudflare assets via extensive analytics functionality.

Root-level analytics includes an overview of metadata related to your Cloudflare account, analytics related to specific properties and products. A GraphQL API is available to visualise the analytics and log information available on the Cloudflare dashboard for more advanced use cases..

Types of analytic information

• Workers Analytics Engine - analytics about anything using Cloudflare Workers.
• Cloudflare Web Analytics - free, privacy-first analytics for your website without changing your DNS or using Cloudflare’s proxy
• Account and zone analytics - metrics related to the requests and traffic on your Cloudflare accounts and zones.
• Cloudflare Network Analytics visibility into network and transport-layer traffic patterns
• GraphQL Analytics API

Read more about analytic functionality in Cloudflare documentation https://developers.cloudflare.com/analytics/

Monitoring Cloudflare Workers

Information about Workers traffic can be viewed in Workers Metrics dashboard and zone-based Workers analytics dashboard in Cloudflare console. Workers metrics can help you diagnose issues and understand your Workers workloads by showing performance and usage of your Workers.

To view Worker metrics:

1. Log in to the Cloudflare dashboard
2. Select Workers & Pages
3. In Overview, select your Worker to view its metrics

There are two metrics that are useful for evaluating Worker health:

• request success and error
• invocation status statuses

Worker invocation statuses indicate whether a Worker script executed successfully or not.

Request data analytics aggregated per zone, ie for all scripts assigned to any routes defined for a zone can be accessed in Zone Metrics dashboard in Analytics > Workers in your Cloudflare dashboard.

• Following metrics are available via Zone Metrics dashboard:
• Subrequests - requests triggered by calling fetch from within a Worker script
• Uncached - requests answered directly by your origin server or other servers
• Cached - requests answered by Cloudflare’s cache
• Bandwidth - bandwidth usage for all scripts on a zone broken down by cache status.
• Status codes - requests broken down by HTTP status code.
• Total requests - successful requests, failed requests, and subrequests, categorized by HTTP status code

Additionally, Cloudflare technology partners (Datadog, Splunk, New Relic etc) provide advanced functionality for Cloudflare Workers monitoring. Please refer to a specific provider for details.

For more information regarding Worker monitoring refer to Cloudflare documentation https://developers.cloudflare.com/workers/learning/metrics-and-analytics/

Cloudflare alarms

Cloudflare offers alerting for various components of the platform.

Depending on the plan, it is also possible to configure webhooks which allow integration with external services such as Slack and PagerDuty.

Refer to Cloudflare documentation for full information on available alarms https://developers.cloudflare.com/fundamentals/notifications/notification-available/

S3 monitoring

S3 storage buckets can be monitored using Amazon CloudWatch. AWS CloudTrail provides log processing for S3 calls.

A CloudWatch dashboard is created for S3 storage (a CloudFormation sample template is provided).

Logging Amazon S3 API calls using AWS CloudTrail

AWS CloudTrail provides records of actions taken by a user, role, or an AWS service in Amazon S3. CloudTrail captures a subset of API calls for Amazon S3 as events, including calls from the Amazon S3 console and code calls to the Amazon S3 APIs.

Logging requests using server access logging

Server access logging provides detailed records for the requests that are made to a bucket.

Refer to AWS documentation to enable server access logging for S3 storage https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html

S3 metrics in CloudWatch

Amazon CloudWatch metrics for Amazon S3 can help understand the consumption and performance of Edge applications in relation to S3 data storage.

The following metrics are available in CloudWatch:

• Storage metrics
• Request metrics
• Replication metrics

Amazon CloudWatch Alarms

Alerting can be configured in Amazon CloudWatch based on the metrics collected for S3 service. A number of alerts are optionally included in the provided sample CloudFormation template.

Please refer to “Using Amazon CloudWatch alarms” guide for details.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html