More and more organizations are seeking to operationalize their decisions into software-enabled platforms as part of their digital transformation.
These same organizations have invested quite heavily (and recently) in big data and machine learning. This has led them to understand how fast business situations evolve when operating at web scale and how this pace is outdating their decision models. This is particularly important for telcos and enterprises whose business is based on their users being connected to their ecosystem. The decisions are now made in a split second—in single digit milliseconds, to be precise—and are based on data streaming from multiple event sources.
A new Gartner report, How to Use Machine Learning, Business Rules and Optimization in Decision Management, delves into the what, how, and why of enterprise-level decision management in the age of machine learning and reiterates the importance of the codification of decision rules for process automation.
The report correctly points out the key technologies and considerations required to fully take advantage of streaming data for powerful, contextual decisioning, and there’s one extra part we would add to Gartner’s thoughts on this.
Must Read: Business support systems
First—let’s define our terms.
Table Of Contents
What is decision management?
In case you’re new to the term, decision management involves using data, analytics, and rules to generate fact-based, structured outcomes that are largely automated. To illustrate, imagine a customer indicates interest in buying a product or upgrading their service. When this happens, the decision management solution gets activated in real time, enabling the application to take a particular data-driven action that allows the business to upsell or cross-sell to the customer with the just the right offer at just the right moment.
Many companies now use intelligent decisioning engines to help end-users make the best choices. In fact, by 2023, more than 33% of large organizations will be practicing decision intelligence and decision modeling
But fast-data decisions are no longer based on static rules. Although traditionally, data has been primarily used for analytics, thanks to things like 5G, IoT, and automation, insights now become outdated by literally the next iteration of your machine learning algorithm retraining.
The only way to stay on top of things is to optimize your decisions by immediately incorporating machine learning outcomes into the rules in your decisioning process. This is what we call contextual decisioning, and doing it correctly has become a very big challenge for enterprises still relying on legacy outdated technology to handle modern-day use cases.
Why businesses are struggling with decision management
Per the Gartner report, data and analytics leaders have been struggling to determine what kind of decision logic should be set as business rules, for two main reasons:
- Data and analytics leaders who understand business intelligence (BI) or machine learning (ML) analytics often don’t understand rule processing. At the same time, rule experts often don’t typically understand analytics.
- While machine learning plays an important role in decision management, it doesn’t provide complete algorithms for structured decisions. Furthermore, expertise in optimization techniques is usually restricted to management science, operations research, or logistics specialists who work outside of analytics and BI teams, which often prevents teams from figuring out the best way forward with each decision.
Tips for better decision management
Gartner outlines some great recommendations and best practices in the report for enabling better decisions from data and analytics solutions, including:
1. Using analytics in conjunction with rules
Gartner recommends identifying the most important decision criteria and calculating decision weights and coefficients using machine learning. It’s a mistake, they say, to assume that analytics are better than rules or that rules are better than analytics. They’re both critically important. Gartner recommends using a decisioning system as a service to improve agility and reusability. This is a point that we should look into a bit deeper in the context of extremely low latency systems that rely on decisions and actions within milliseconds.
2. Using optimization techniques in “trade-off” situations
By using optimization techniques in decisioning, it becomes much easier to determine the best decisions in situations where there are competing goals and constraints. Some examples of optimization techniques include constraint-solving, linear and nonlinear programming, and genetic algorithms, among others.
The report provides various optimization examples, including airline ticket sales and seat availability; transportation organizations planning load building in trucks; and route planning for delivery. These are good examples of situations that require decisions that don’t need to be acted upon immediately, which covers the majority of use cases. However, in telecommunication-industry scenarios, where you often have conflicting factors, such as subscriber experience management and network bandwidth usage, network slices and use case service level agreements, and network security and low latency, the need for faster, better, and more reliable decisions becomes extremely important.
Real-time decisioning: A top need
Another key challenge businesses face with contextual decisioning is latency.
In the past, business processes were seldom in real time or measured in milliseconds. Enterprises could leverage big data technologies such as a data lake or data warehouse to process the data and then proceed to incorporate the insights generated into manual processes, or, in cases of codification, kick off a project to do the same.
Today, businesses are dealing with rising data volumes and the rolling out of 5G and IoT deployments, increased customer expectations around performance, and aspirations for digital transformation to automate processes, which means their decisions now need to be instantaneous to tap into the most relevant moment of engagement.
As such, businesses across the board are turning to real-time decisioning engines. In fact, Gartner anticipates that by 2022 more than half of major new business systems will incorporate continuous intelligence that uses real-time contextual data to improve decisions.
How to produce contextual, real-time decisions at scale
Suffice it to say that in today’s fast-paced world, businesses have a very short window of opportunity to respond to events and produce a desired result, such as preventing fraud or engaging a customer. Waiting too long—even half a second too long—can lead to missed revenue opportunities, data leaks, brand reputation damage, and other poor outcomes.
The new gold standard for processing data and responding to an event is 250-milliseconds — which means that decisioning needs to happen within 10 milliseconds, which means there’s no time for data to travel to and from a data lake.
While the Gartner report mentioned above addresses the business rules and optimization of the said rules, when looking at the picture holistically, there are other key capabilities that come into play for ensuring optimal contextual decisioning in low latency:
- Stream processing, for easy moving of the data to the appropriate processing layer.
- In-memory, real-time data stores for fast access to contextual data correlating to the incoming event information.
- Business rules, Gartner noted, optimized with machine learning outcomes.
- Streaming aggregation to keep a running track of key performance indicators (KPIs) via things like materialized views.
But an enterprise can have all of the above and still find contextual decisioning at scale, within the latency expectations, to be extremely challenging.
Why?
Because if you need to add a tech layer for every capability, you’re also adding latency, and in the age of 5G and IoT, high latency, or even regular latency, is simply a non-starter.
Volt Active Data is the only data platform purpose-built to provide the full suite of contextual decisioning capabilities at scale, without compromising on latency or data accuracy. We’re able to do this because we offer everything in a single, unified platform, so that you can have a simplified stack that avoids the pitfalls of tech layers.
To learn more about the importance of contextual decisioning and what you can do right now to improve your organization’s capabilities around it, check out the full Gartner report.
Gartner, How to Use Machine Learning, Business Rules and Optimization in Decision Management, W. Roy Schulte, Pieter den Hamer, 14 January 2021
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
