Features are numeric values to be computed as the result of data analysis over previous events in time, such as Velocity, Distinct Count, Time Since, Specificity etc.
Features:
- are used as data inputs for Rules, Models and Decision Strategies.
- have a unique name so that they can be referenced without ambiguity.
- are grouped together into a Feature Set (a .feature.yaml file).
A Feature Set outputs:
- a map of Feature Name: Feature Value pairs.
Rules
Rules are boolean (yes/no) logic to be applied onto event data, whether direct profiling data, inferred features or model outputs.
Rules:
- have a unique name (called a 'Signal'). The Signal is only emitted when the Rule evaluates to True.
- have an optional Category which is not unique. The Category can be emitted by any that are True in a group of similar Rules.
- have an optional Rule score, allowing the Rule Set to hybrid as a linear Model.
- are grouped together into a Rule Set (.rules file).
A Rule Set outputs:
- a list of Signals, from all Rules that were True.
- a list of Categories, from all Rules that were True.
- (optional) a Score, from adding the scores of all Rules that were True.
Models
Models allow for complex transformations and combinations of data, resulting in a numeric continuous Score.
A Score often provides a better way to compute probability of a certain risk and apply subsequent decision thresholds compared to Rules alone.
Models:
- have a unique model name.
- can be manually constructed from Rule Sets, or imported from machine learning processes.
A Model outputs:
- a Score.
- a Percentile, between 0 and 100 based on where it sits along Score distribution.
- a Result, categoric string summarising the risk or prediction.
Decision Strategy
The Decision Strategy is a special singular Rule Set slot, that strictly runs last and represents the overall risk and decision assigned to the event.
It determines the logic that makes the final decision on what should occur with the event by using Rules that access all previous Features, Signals, Scores and Results.
A Decision Strategy outputs the same things as a Rule Set, but with:
- a Result from fixed set of options: Pass, Challenge, Alert, Reject, Review.
Examples
Features: Velocity of Failed Account Logins
A Velocity feature is created that counts the number of failed login attempts made on an account within a period of 1 hour.
It is given the name "account_failed_login_vel_1hr".
Rule Sets can now reference the "account_failed_login_vel_1hr" feature and and have Rules on the feature, such as comparison of the value with a threshold.
This may also allow the final Decision Strategy to produce an Outcome to lock the account for 24 hours if the value of "account_failed_login_vel_1hr" is greater than 4. E.g The account should be locked if 5 or more failed login attempts have occurred within a 1 hour period.
Rules: Signal Categories for Bots
Bots such as search engine crawlers and web scrapers can been seen as "good bots" as they help improve search engine rankings. Conversely, Bot traffic originating from malware or known attack sources can be seen as "bad bots".
As such, separate Rules may be setup to generate a Signal indicating which specific kind of bot was detected such as SITE_MONITOR, WEB_CRAWLER and SEARCH_ENGINE_CRAWLER.
This allows for detailed reporting and analysis of traffic data, but makes decisioning logic more complex. However, a Rule with a condition based on "good bots" would require a logical OR on all of the "good bots" Signals. E.g if SITE_MONITOR or WEB_CRAWLER or SEARCH_ENGINE_CRAWLER. This also becomes difficult to maintain as new Bot detection models and signals are created, as each Rule would need to be updated to be made aware that the newly created Bot signal is classified as part of the "good bots" behaviour.
This can be resolved by the usage of Signal Categories. In this example the SITE_MONITOR, WEB_CRAWLER and SEARCH_ENGINE_CRAWLER signals would all be configured to have the Signal Category GOOD_BOT.
The Decision Strategy and Models can now simply detect the presence of the GOOD_BOT Signal Category instead of the individual Signals. This allows the addition of new "good bot" detection and Signals along with the Signal Category GOOD_BOT, without the need to update existing logic.
Models: Account Takeover and Scam Models
Assessing risk of an event is not one continuous sliding scale, but several different ones. Different risks have different indicators that may contradict or dilute each other that need to be assessed separately.
Take an example of a payment at a bank:
The bank wants to check that there is no risk of account takeover; that a malicious actor has managed to log into someone's account and is attempting to exit funds. Example rules that would help here are: device is new to the account, high number of failed login attempts from the IP and location, and device spoofing anomalies.
But the bank also wants to check there is no scam risk; a genuine customer is being tricked and manipulated into making a payment themselves to a malicious actor. Here the payment passes the account takeover ruleset with no risk at all; it's the customer's usual device they always use, the customer passed all login authentication, their location is normal and there are no spoofing concerns on their device.
The solution is to create two different Models that are trained and calibrated to separately detect the risk of each in isolation. A decision strategy would decide what to do with the payment.
Decision Strategy: Payment Decision
Imagine there are two models running on a payment; an Account Takeover and Scam model producing separate risk scores. How can the bank decide what to do with a payment?
A Decision Strategy is where the final decision logic is stored that would make use of these Models to decide what to do.
- Block: (Account Takeover Risk = High) or (Scam Risk = High)
- Challenge: (Account Takeover Risk = Medium)
- Review: (Scam Risk = Medium)
Why?:
- Challenging Account Takeover scenarios that elicit a medium risk score with a chosen step-up has several advantages. Firstly, this approach has a good chance of preventing the payment when it is initiated by a fraudster. Secondly this approach offers false positives a better user experience than a direct block. Finally, it doesn't waste operations time reviewing the case when identity confirmation is all that is needed.
- Challenging the Scam scenarios that have medium risk scores would have a low prevention chance, as a genuine customer is able to pass any step-up or authorization needed.
- Instead, the medium Scam risks are reviewed. This may involve a temporary hold of funds or an operations call out to the customer to ask questions of the intent behind the payment.