Creating and Editing Steps

Step Summary

Step Name

Step Name is a unique name given to the Step to identify it. Keeping the Step Name as simple as possible will make searching for the Step within Investigations easier.

Event Type

Is the Type of the Event that will be written into Investigations when this Step occurs. Event Type is used by Features and decision logic so it is important to select the Event Type with the closest match to the purpose of the Step. 

Alternatively if "(Do not generate event)" is chosen, then the Step will not be recorded into Investigations and cannot be referenced by Features or decision logic in the future.

Event Type Reference

Import event data from dependencies

Importing event data allows data from a previous step to be imported from a previous Step within the Journey into the Event currently being processed.

A simple example of this is username, which is only entered once by the end-user during an AccountLogin Step. 

This username value is not available to the current Step, unless it is imported as a Step Dependency inside the Import event data from dependencies Step Configuration.

In our example, the username is supplied as a part of a form POST to the Step Named "My Login Step". 

The "Dependency Example" has a Count Distinct Feature which counts the number of Devices that have accessed the account within the last 7 days. The "Dependency Example" Step must import the "Request Body Processed" dependency from "My Login Step".

The reason for this is that the "Request Body Processing" is where the username is extracted into an attribute.

Refer to the Understanding Dependencies article for more information on Dependencies.

Triggers

Trigger configuration specifies which HTTP requests will invoke or "Trigger" the Step. A Trigger consists of a number of conditional configuration elements based on the HTTP Request URI that must all be satisfied in order for the Trigger to be met and invoke the Step.

Host

The Host setting allows the Trigger to match only on requests for that Host. If left empty then the Trigger will match on all HTTP requests regardless of the Host portion of the request URL. 

Host Wildcards

The Host setting can optionally begin with a wildcard operator (*) or a subdomain wildcard operator(**). 

Wildcard Operator (*)

The wildcard operator (*) can be used to match any second level subdomain.

"*example.com" will match "example.com" AND "www.example.com"

"*.example.com" will match "www.example.com" but NOT "example.com"

IMPORTANT: take special care with the placement of dot characters when using a wildcard operator or subdomain wildcard operator as "*.example.com" will match "www.example.com" but NOT "example.com"

Method

Method specifies that HTTP Method for the Trigger. GET is used by the client browser for loading web pages and content into the browser and is typically used to Trigger a Step when a page is loaded. POST is used by the client browser to send data and content from the browser to the website and it typically used to extract login information, content and other application specific data elements into Attributes.

Url

Url specifies the path portion of the HTTP requests that will Trigger the Step.

Url Match Type

URL match type determines which URLs will or will not trigger the step.

Exact

When Exact Match Type is selected, the Trigger Url condition will only be satisfied if the full HTTP Request URI exactly matches the configured value. The means all sub-paths or queries will NOT be considered a match

For example, given the Url is set to "/my/example", then:

• "/my/example" will match. 
• "my/example?param=val will NOT be considered a match as it has a query
• "my/example/browse" will NOT be considered a match as the path does not match
  

ExactWithQuery

ExactWithQuery matches in the same way as Exact but permits any query (which is essentially ignored).

So here, with URL "my/example" then:

• "/my/example" will match. 
• "my/example?param=val" will match, query is permitted
• "my/example/browse" will NOT be considered a match as the path does not match

Prefix

Prefix Match Type matches the configured Url against the start of the URI path string, treating anything following the configured value to be a wildcard. The means all sub-paths will be considered a match.

For example, given the Url is set to "/my/example", then 

• "/my/example" will match
• "my/example?param=val" will match, query is permitted
• "my/example/browse" will match, different follow-on path permitted.

Glob

Glob Match Type enables wildcard matching of paths using one or more wildcard (*) operators inside the configured Url. A wildcard operator will match zero or more of any character.

Wildcard operators can only appear either as a path segment or at the end of the path.

Invalid Usage:

• "/my/example/*.png" is not valid as the wildcard operator is at the beginning of the path segment
• "my*/example/" is not valid as the wildcard operator is at the end of a path segment, not the end of the path.

With "/my/*/here/*" as URL:

• "/my/example/here/" will match
• "/my/example/here/another" will match 
• "/my/example" will NOT match
• "/myexample/here will NOT match

Match Type Summary

Data Mappings

Data Mappings allow both the inspection and modification of the HTTP Request and subsequent Response. Data can be extracted and injected to/from Darwinium Attributes, Jinja Templates and static string content.

Request

The Request Header and Body Data Mapping occurs inside the proxy, before forwarding occurs. This means a Request can be modified or inspected based on conditional logic before arriving at your Web server.

URL Mappings

URLs consist of the following syntax components:

URL Mappings allows some or all of the URL syntax components to be extracted or modified as required. The most common use-case is the extraction of URL Query parameters for use in Workflow decisions, however many other scenarios are possible such as changing the Host based on geography or traffic limits.

URL Mappings is available inside Data Mappings > Request of the Step configuration. Click on the edit icon of the URL syntax component to display the Injection and Extraction Menu to perform Data Mapping operations. Please refer to the Injection and Extraction Menu article for details of the options available.

The URL Query syntax component can contain one or more name/value pairs. The Query section of the URL Mapping Step configuration allows the entry of multiple name/operation pairs. To add a new mapping, click the Add button. Parameter name is the name of the URL Query Parameter and the Operation selects the operation to take place on the value.

The example below shows the URL Query Parameter named "myparam" being extracted into the Attribute "custom.general_purpose['myparam']" for use in Workflow decisions.

If a configured Query Parameter does not exist, the operation will be ignored and the next configured Query Parameter operation will be processed.

Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.

Header Mappings

HTTP Headers are a list of name/value pairs that let the HTTP Client and HTTP Server pass additional information in a Request or Response. Request Headers contain information about the resource being fetched or about the client requesting the resource. Web Applications also utilise Headers for many purposes, such as cookie and session management along with application specific purposes.

The Header Mappings allow the entry of name/operation pairs. To add a new Header Mapping, click the Add button. Set Header Name to the name of the required HTTP Header and Operation select the operation to take place on that Header.

Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.

The example below illustrates the extraction of:

• "X-UserName" application header into the "identity['ACCOUNT'].username.username" attribute
• "Content-MD5" application header into the "custom.general_purpose['content_md5']" attribute
• "X-Frame-Options" application header into the "custom.general_purpose['frame_options']" attribute
• "Client-Peer" ip address into the "custom.general_purpose['client_peer_ip']" attribute

Body Mappings

Darwinium Body Mappings allow full access to the HTTP body, to extract data elements, modify or replace the entire HTTP body.

The Request Body contents depend on the HTTP Method and the "Content-Type" header.

For most resource GET requests (page loads) the HTTP Request body is empty. However requests with a HTTP POST or PUT method have a Body, containing data sent from the client to the HTTP server. For POST and PUT methods, the "Content-Type" header is used to describe the contents of the HTTP body to the server.

Body Type

The Body Type setting determines how the body is processed and can either be set to:

• body_rule - the Request Body is processed as a single document with only one operation available on the entire Request Body. This is useful for setting the Request Body for URL Callouts (External API calls) or extracting or injecting a single data element. Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.
• multipart_body_rules - the Request Body is processed in multiple named parts based on the Content-Type. HTML form data ("Content-Type: multipart/form-data"), HTML, XML, JSON and URL Encoded data can all be processed by referencing the data Field name and the operation to perform on that field.

Click on the Add button to add a new Field/Operation pair. Set the Field to the appropriate name of the Multipart Body and then select the desired Operation. Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.

The example below shows the HTML body being replaced with a templated Content string.

Wait for these dependencies to be satisfied before mapping the request

Click on the edit button to set and dependencies that might be required before the request Data Mapping occurs. For example, if the Score from a Model is added as a HTTP header on the request, it must be set as a request mapping dependency so that the request mapping waits for the Model to complete before processing the request mappings. For more information please refer to the Understanding Dependencies article.

Response

The Response Header and Body Data Mapping occurs inside the proxy, AFTER the Response has been sent from your web server but BEFORE the client receives the response.

This means a Response can be modified or inspected based on conditional logic before arriving at your Web server. 

If the Request is a GET for a HTML Page, the page can be modified before the client receives it. This is very useful for injecting Darwinium Profiling code or Captcha or other forms of Step-up authentication without having to modify application logic running on your web server.

Header Mappings

HTTP Headers are a list of name/value pairs that let the HTTP Client and HTTP Server pass additional information in a Request or Response. Request Headers contain information about the resource being fetched or about the client requesting the resource. Web Applications also utilise Headers for many purposes, such as cookie and session management along with application specific purposes.

The Header Mappings allow the entry of name/operation pairs. 

To add a new Header Mapping, click the Add button. Set Header Name to the name of the required HTTP Header and Operation select the operation to take place on that Header.

Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.

Body Mappings

Darwinium Body Mappings allow full access to the HTTP body, to extract data elements, modify or replace the entire HTTP body.

The Request Body contents depend on the HTTP Method and the "Content-Type" header.

Body Type

The Response Body Mappings are similar to the Request Body Mappings, however they operate on the Response Body.

The Body Type setting determines how the body is processed and can either be set to:

• body_rule - the Request Body is processed as a single document with only one operation available on the entire Response Body. Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.
• multipart_body_rules - the Response Body is processed in multiple named parts based on the Content-Type. HTML form data ("Content-Type: multipart/form-data"), HTML, XML, JSON and URL Encoded data can all be processed by referencing the data Field name and the operation to perform on that field.

Click on the Add button to add a new Field/Operation pair. Set the Field to the appropriate name of the Multipart Body and then select the desired Operation. Please refer to the Injection and Extraction Menu article for more details on the Operations that are available.

Wait for these dependencies to be satisfied before mapping the response

Click on the edit button to set and dependencies that might be required before the response Data Mapping occurs. For example, if the Score from a Model is added as a HTTP header on the response, it must be set as a response mapping dependency so that the response mapping waits for the Model to complete before processing the response mappings. For more information please refer to the Understanding Dependencies article.

Workflows

Workflows is where Models, Features, URL/API Calls and Signals are generated and passed to the Decision Strategy to make a decision on the event. Please see the Decisioning Overview if you are unfamiliar for an introduction into Darwinium decisioning.

Models

Models is where Models, Rules, Features and URL/API Calls can be made. To add a new model (Model, Rules, Features etc), click the + button.

Name

First give the model a name. All Models are identified by a name. The name should be meaningful and unique, so that it can be referenced by the Decision Strategy or other Models.

Action

Next select the desired action by clicking on action:

• Execute Code/Model/Rules - The Path should include the full filename and path relative to the current Journey file.
  • Rules - Executes a '.rules' Rule Based System file. Please refer to the Rule Editor article for detailed information on creating and editing rules.
  • Model/Code - Executes an Assembly Script '.ts' Model or Code file. Please refer to the Advanced Decisions article for more information.
• Call URL... - can be used to invoke a RESTful API call. It can be used to either pass data e.g Case Management System, SMS Alert etc or collect data to enrich the current Event or Journey. Please refer to the Call URL article for more information.
• Generate Features... - Select this option to add a Feature Set. The Path should include the full name and path of the Feature file relative to the current Journey file.  Please refer to the Feature article for more information on Features.

Dependencies

Configure any Dependencies required by the action. For example, if a Model or Rules requires a Feature Set then the Feature Set should be configured as a Dependency of the Model/Ruleset. Please refer to the Dependencies article for information.

Condition

The Condition setting allows for conditional execution of the Model based on a boolean expression provided in Query Syntax. If the condition is not satisfied then the Model will not be executed. Conditions are optional and do not need to be specified unless conditional execution is required.

To set a Condition, click the edit button under Condition:

This will open a condition editor allowing the boolean Condition to be entered:

Clicking the close button saves the changes to the condition and can now be seen inside the editor:

Proxy Actions

Proxy Actions allow (optional) direct control of underlying TCP/HTTP connection. 

Proxy Actions can be configured to occur either before the request is proxied (or "forwarded") to its destination or after the destination has processed the request but before the response has been returned to the end-user.

By creating a new Rules file and setting the type to Proxy Action it can now be used to create Proxy Action rules. 

Proxy Action rulesets have two new rules available from the Add Rule button:

All Proxy Action Rules have an optional Score, Signal and Category like other rules.

Proxy Terminate Rule

The Proxy Terminate Rule prevents the HTTP connection from any further forwarding. The Termination can be a HTTP Redirect, abrupt TCP connection drop or a standard HTTP Success/Error status code response.

Return with HTTP status code

The Proxy Terminate Rule prevents the HTTP connection from any further forwarding an immediately return to the client with the configured HTTP status code. 2xx Success or 4xx/5xx error codes can be returned and the HTTP response body can be generated using the "Content on Terminate" configuration.

HTTP Redirect

HTTP Redirect tells the client to obtain the request from an alternative URL.

HTTP Silent Drop

Silent Drop Is a TCP level termination of the connection, resulting in what looks like a connection error to the HTTP client. This can be used to cause interference once a bad actor scenario such as Account Takeover, Bot etc has been detected.

Proxy Accept Rule

The Proxy Accept Rule simply forwards the connection onto the destination as normal. This can be useful inside nested If/Else rules involving Proxy Terminate Rules.

Before Request to Proxied Destination

Allows the execution of Proxy Action Rules once Darwinium has accepted the HTTP request, but before the request has been forwarded to the destination.

This allows the Request to be modified OR to prevent the forward altogether using a Proxy Terminate rule.

Modifications could for example include adding HTTP headers containing risk scores, device or user behaviour data to your application server.

Proxy termination may also include returning a HTTP redirect or HTTP error based on scores or behaviour before the request arrives at your web server.

Before Response to end-user

Allows the execution of Proxy Action Rules after the request has been forwarded and processed by the destination, but before the response has been returned to the client by Darwinium.

This allows the Response to be modified OR to replace the entire response using a Proxy Terminate rule.

Before Request and Before Response share the same configuration options, the only difference being when the execution takes place.

Name

Name is a unique name given to the Proxy Action to identify it. Keeping the Name as simple as possible will make searching within Investigations easier.

Action

The only option available is to Execute a Ruleset which has been configured to Proxy Action Rule type (instead of Scoring Model).

Use the Path option to set the filename and path relative to the Journey file.

Dependencies

Dependencies can be used to set any Dependencies for the Proxy Action, such as Models, Features etc.

Refer to the Understanding Dependencies article for more information on Dependencies.

Condition

The Condition setting allows for conditional execution of the Action based on a boolean expression provided in Query Syntax. If the condition is not satisfied then the Action will not be executed. Conditions are optional and do not need to be specified unless conditional execution is required.

To set a Condition, click the edit button under Condition:

This will open a condition editor allowing the boolean Condition to be entered:

Clicking the close button saves the changes to the condition and can now be seen inside the editor:

Content (on terminate)

Allows the entire HTTP Request or Response body to be set in the case that the Proxy Action Rules resulted in a Terminate. The Content (on terminate) is ignored if the Proxy Action Rules result in a Proxy Accept Rule.

Headers

Contains a list of HTTP Headers that appear in the request/response. The Content-Type header must be set along with any other headers that are to appear in the response. For more information on HTTP Headers please refer to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type.

Use the + button to add a new Header.

Enter the name of the header into the Header Name field.

Next select what should be injected into the header field. Typically this is static String content.

The example below shows the setting of the Content-Type header to JSON (application/json).

Template markup may be used to inject attribute values or formatting as required. Please refer to the Template Markup article for more information.

Body

Body is used to define the contents of the HTTP request or response body. The String content option can be used for simple bodies whilst template files can be used for more complex data.

Template markup may be used to inject attribute values or formatting as required. Please refer to the Template Markup article for more information.

Decision Strategy

A Decision Strategy is a special type of Ruleset. It strictly runs last and represents the overall risk and decision assigned to the event. The Decision Strategy determines the logic that makes the final decision on what should occur with the event by using Rules that access all previous Features, Signals, Scores and Results. Decision Strategy files must use a '.strategy' file extension. Please see the Decision Strategy article for more detailed information.

Name

Name is a unique name given to the Decision Strategy to identify it. Keeping the Name as simple as possible will make searching within Investigations easier.

Click on the edit button to set the Name of the Decision Strategy:

Action

Next, click on the edit button below Action to set the path to the Decision Strategy file. Enter the path and filename to the Decision Strategy file relative to the current Journey file: