Imagine you’ve got a complex system with mountains of data flooding your models and demanding assessment. You’re tasked with analyzing this system and all of that data, and then making an optimal decision about the future state of that system. How do you do it?
If you’re like many on the cutting edge of analytics, then there’s a real chance you’ll turn to machine learning tools to help you determine the right path to take. After all, the processing power that machine learning systems bring to decision management is enormous, and certainly there are patterns and correlations that machines can identify that are impossible for humans to do as quickly, if identify at all without technological aid.
But while machine learning might cut it for simple systems or even some complicated systems, it is not going to make inroads when systems are truly complex.
Consider the following analogy.
A simple system can be compared to a two-dimensional shape, say a square. A square has four equal sides and four 90-degree vertices, and is something a first grader can wrap their head around.
A complicated system is a little more difficult to understand than a square, but not a lot more. This time, instead of a square, we can consider a cube: the three-dimensional shape has six faces, eight vertices, and requires 12 straight lines to draw. It’s something the average first grader would struggle with, but with a couple more years of math lessons they’ll manage.
A complex system, on the other hand, might describe a shape with 32 dimensions. How many vertices in a 32-dimensional shape? More than 4 billion. Not only is the very notion of 32-dimensional space difficult for most people to conceptualize, but being able to conceive of a 4 billion verticed anything is difficult even in two or three dimensions.
It’s less of a problem for machines, of course, who don’t suffer from a human’s natural reversion to simplification in the face of complex problems. Yet there are limits to the processing a machine can do and efficiently analyzing billions of data points with billions more interactions between them all is close to that limit. And by the time we imagine a complex system with a hundred different parameters and existing in 100 dimensions, we’re way beyond them. Indeed, analyzing just the corners of a system like this would require all of the computing power that a company the size and scope of Google has at its disposal in all of its data centers globally, and then some!
Complex systems like this exist, by the way – they aren’t hypothetical. I know because at Cosmo Tech we analyze and forecast dozens of these systems every single day with a server that fits in a couple of datacenter bays not far from my office.
So how do we do it when the best machine learning backed by the computing power of dozens of data centers cannot?
We model and simulate.
A complex system is dynamic by nature and often very little has been observed and will be observed. As a result, even targeted modeling with machine learning techniques have limits as it is impossible to configure the system to forecast events that have never happened before.
Instead of trying to understand a complex system by mapping every inch of every system and sub-system to get a yes/no answer, we model the system using technology invented just for this purpose and then run simulation after simulation to identify the most likely future outcomes. Our models include expert knowledge about how each of the sub-systems in a complex system work and, by coupling these sub-systems together to mimic real world circumstances, we can capture the emergent phenomena that machine learning approaches cannot and accurately predict what will happen next, even if that future state has never occurred before.
Our modeling technology and simulation approach means dealing with systems with 32, 50, or 100 different dimensions is not only possible, but common. What’s more, forecasting these systems is not a dream but a standard expectation for our clients which include some of Europe’s biggest industrial actors from across a range of verticals because, yes, the complex modeling and simulation approach works for any complex system.
AI Trends has argued that machine learning appears to be an essential capability in the decade ahead for organizations and companies like Google, Microsoft, and Apple are investing heavily in it for consumer and enterprise products alike. Yet for all the promise of machine learning there are still some significant pitfalls, and accurately comprehending and forecasting complex industrial systems is one of them. There’s a reason Gartner has positioned machine learning at the peak of inflated expectations on its hype cycle and just ahead of its “trough of disillusionment”: there are limits to what machine learning can do for the complex business systems of today’s global world.
Luckily, there are no such limits for Cosmo Tech’s complex systems approach and, as a complement to machine learning, it is just as essential for a company moving through its digital transformation.