Rules vs machine learning for effective fraud prevention

Which is better – rules or machine learning? Learn more about the benefits and challenges of each to decide which approach could help you address your fraud risk.

What’s the most effective enterprise fraud management approach to minimize payment processing losses?

Merchants were hit by a series of critical business challenges during the course of the last three years. Yet at the forefront of some of their concerns remains the accelerating cost of online fraud – with online payment fraud forecasted to surpass $206 billion between 2021 and 2025.¹ The challenge now is to manage fraud more effectively, while still working to prevent more traditional tactics such as stolen credit cards and identities.

To do so, merchants have two obvious enterprise fraud management options:

1. Traditional rule-based fraud detection and prevention
2. Adaptive machine learning fraud prevention

Yet many struggle to decide between the two. Is one better than the other? Or can both be used in concert?

Rule-based vs machine learning fraud prevention modeling: A decade-long debated

Over the past 10 years, fraud prevention leaders have been taught that machine learning modeling is more effective than traditional rules and filters for minimizing payment processing losses. But that may not be the whole story. Basic analysis has demonstrated that while machine learning is a solid approach, it may lack validation against real-world transactional traffic.

Settling the rule-based vs machine learning debate would require analysis of historical data that has run in both scenarios, in order to generate actual performance comparisons for each. However, this would require a massive amount of transactional data with the same schema and the known outcome represented in chargeback and decline rates. Up until now this hasn’t been possible.

Why choosing an effective fraud prevention system matters

Global fraudsters have been attacking merchants since the dawn of eCommerce – and they’ve discovered that the best way to defeat fraud prevention layers is to understand how they work. They test and validate different scenarios to determine which would result in a successful attack, and either use those methods themselves or sell them to other fraudsters.

This is a major issue for fraud management tools, in that they become less and less effective as fraudsters continue to test their limits, and design attacks to circumvent filters. As a result, fraud prevention teams simply can’t keep up with continually changing attack techniques – exposing merchants to significant financial and reputational damage.

The pros and cons of a rule-based approach

There are, of course, both advantages and disadvantages of a rule-based fraud prevention approach.

Advantages

Effective rule-based fraud prevention approaches require human interaction. It can sometimes be relatively easy for analysts monitoring events to spot errors and make a swift correction. This can save the day for merchants, especially where machine learning is deployed but hasn’t yet learned to identify these issues. Smaller and growing merchants may benefit from a rule-based fraud detection approach. For example, putting in place rules that don’t allow an order to ship out where the IP address is in a different country than the billing address may help reduce fraud.

Rules also make it easier for analysts to understand why a transaction was declined, affording them the opportunity to create new rules as needed. And if they see new fraud patterns in transactional data, additional rules can be created to tackle them. Again, that has value in scenarios where machine learning models are still getting up to speed.

Disadvantages

But while rule-based fraud prevention may be a good choice for smaller organizations, these systems can become overwhelmingly complex and cumbersome to maintain as they grow. Once a rule is created, it can’t be changed. That means rule-based systems grow and grow as fraud evolves and the bad guys invent new ways to circumvent them. It can be challenging knowing which rules to discard and which to keep. The need for more staff to manually observe trends and create new rules can become prohibitive.

Leveraging machine learning to detect fraud

Fraudsters are constantly evolving, employing innovations such as automated testing tools, and technology designed to mask domains, devices, and even IP addresses. This is where the advantages of machine learning can help.

Advantages of machine learning

By creating algorithms to process large datasets with multiple variables, organizations can leverage the advantages of machine learning to rapidly discover correlations that might indicate more sophisticated fraud attempts. Not only do they identify trends and patterns that the human eye might miss, they’re also able to continuously adapt to keep pace with the fraudsters. This can result in a higher degree of precision in identifying fraud. Effective fraud prevention using machine learning powered by clean, robust, and proprietary data helps enable a model that is more sophisticated than those available to purchase ready-made. Better models powered by better data may help provide an advantage over fraudsters.

Whereas rules are created in response to a particular fraud pattern recognition, machine learning considers 300 to 500 signals before deciding a transaction is fraudulent. These systems typically also look at multiple factors based on administrator input when they are in the initial learning phase. This stops fraud prevention with machine learning models from getting stuck on a particular trend – enabling them to capture more fraudulent attempts with fewer false positives. Their adaptive abilities outshine rules by fighting fraud protection in real time, detecting patterns as they start to form, and using models based on thousands of transactions to identify future fraudulent activity. Plus, these models can be retrained using the freshest data, making fraud prevention using machine learning more precise.

Machine Learning – used the right way

As with any fraud management tool, the way it is used makes a difference. Machine learning models are not a silver bullet for fraud prevention if they are not properly designed and trained, or fail to leverage the necessary features.

As machine learning algorithms have become more complex in production, models are no longer capable of providing easy-to-understand results that analysts require to determine the effectiveness of the model and why transactions are declined. This lack of transparency and explainability, for example in neural network modeling, can be challenging for those who rely on complete data sets to determine future modeling.

There are ways to address this, including SHAP (Shapley Additive Explanations) or LIME (local interpretable model-agnostic explanations) algorithms which help explain what machine learning models are doing behind the scenes. By visualizing the output, SHAP can be used to more easily explain a model’s prediction through computation of each feature’s contribution. Meanwhile, LIME focuses on interpreting local models by modifying a sample and observing the results. Both have their advantages, but they don’t necessarily provide the straight line needed to provide a clear picture of transactional behaviors.

Machine learning fraud management tools require time to absorb data and analyze information to produce results and learn user behavior. And it needs a sufficient volume of data. Once it’s functional, fraud prevention using machine learning can be automated. This ramp up/training period typically requires 1-2 months as rejections/chargebacks must be realized (identified) to begin training the model. Fully realized historical data allows for this training to be shortened, assuming the features required are included in the data set.

Combining the best aspects of both rule and machine learning-based systems to decrease fraud rates

The good news is that this doesn’t have to be an “either/or” decision for merchants. In fact, organizations can benefit from combining the best aspects of both rule and machine learning-based systems to drive down fraud rates without increasing false positives.

While machine learning models are “powering up”, rules can step in to do some of the heavy lifting for enterprise fraud protection. Because rules are based on particular trends, administrators can write them on the fly while fraud prevention using machine learning is being trained to see the same patterns. Using this approach, merchants can protect themselves and their customers from fraud with the immediacy of rules and the sophistication of machine learning. In other cases, anti-fraud platforms might use machine learning algorithms to suggest new rules for analysts to create, based on evolving fraud patterns.

Organizations which have an open mind to using both fraud management resources will ultimately be better placed to manage eCommerce fraud risk effectively than those that do not.

If you’re ready to learn more about the many ways PayPal's 20+ years of building risk models can benefit your business, visit Manage Risk here.