Its quarter century (g)old

Evolution of process mining

In 2022, Chat GPT made news for moving scaling to a million users in 5 days. In contrast, the first mention of neural networks dates back to 1943. After 80 years, a technology that was built using the underlying neural net concept, scaled at a lightening pace. So that leads us to ask - why do certain technologies scale faster while others don't? Its about the complex interplay between - relevance, demand and a supporting ecosystem. And some luck. Yeah, its about being at the right place at the right time. Going by that standard, process mining, for all its relevance and impact had a pretty slow start. At this juncture, when process mining is celebrating its silver jubilee, let us delve into the origin of process mining, its evolution, the current state and the potential directions that it could take from here.

It all started with this thought of Prof. Wil Van der Aalst - "What if I could reconstruct the process from its execution logs?". This is the idea he had introduced in the year 1999 in his publication "Process Design by Discovery". For almost one decade after that, we could only find it in mentions of IEEE conference proceedings and in the Prom software. In parallel, this was the era when there was massive digitization taking place across industries. From 2010s, commercial software like Celonis, Disco and others gained prominence. It was also the time when I got introduced to this concept through the famous Coursera program - "Process Mining: Data science in action". The commercial software were able to integrate themselves into the larger ecosystem of big data analytics and machine learning. Thanks to the live integrations with the source systems SAP, Salesforce these firms were able to provide an execution layer to support process management. The segment started growing leaps and bounds.

Fast forward to the present, there are over 30 process mining vendors and each of them is approaching the market in different ways. Some of the broad themes which the vendors are trying to address are,

• Discovery and analysis - This is one of the basic features in all process mining tools. It is about the ability to read process event logs and generate the process maps. Using that model, almost all vendors, provide capabilities to configure your KPIs and dashboards.

• Process Improvement - Some software vendors have tailored their capabilities to support the framework of systematic process improvements like Six Sigma. That includes capabilities like automatic detection of opportunities, benchmarking, root cause analysis and monitoring of improvements

• Comparative process mining - This refers to the capability to compare a process against some standards to compare the KPIs or execution. It could be useful in auditing for compliances against standard process or other process variants.

• Process Simulation - Using the digital twin of the process to conduct to predict the outcomes of process changes and do what-if analyses

• Automation enablement - Process mining and automation complement each other well. While process mining highlights the bottlenecks or inefficiencies, automation solves some of them. So many automation vendors have also acquired process mining capabilities to support the identification of areas for automation.

• Process Documentation - Some vendors have provided the capabilities to for end to end process management by including features to document and maintain BPMN models and compare it against the execution

• Task mining - This is an important feature that helps unpack the intelligence from the event data available in UI logs. These UI logs that are generated from the clicks, keystrokes and data entries are useful in understanding the manual effort involved in performing the tasks

• Other advanced use-cases like machine learning workbench, predictive process monitoring and prescriptive analyses are also provided by some of the vendors.

I had just detailed out the major themes; a more detailed explanations can be found in research reports like Gartner and Forrester. Complementary version of these reports can be downloaded from many of the vendor websites.

Now we enter treacherous zone of forecasting what the future evolution might look like. There are three major themes that could play out,

• Process mining tools go 'invisible' - Don't get me wrong, they would continue to exist, but like electricity, they will be an invisible layer powering things. They could connect different legacy systems be the layer that integrates process data , rules of execution, alert / action definitions etc., Some of the analysis and visualization might be taken over by the BI tools and specialized simulators, process designers, automation systems - all powered by the 'invisible layer' - could evolve.

• Gen AI unlocks new capabilities in process intelligence - This is an evolution I am near certain about. Gen AI will become another layer between the user and the core process mining engine. This will help in the realizing many gen AI use cases like - contextualization, democratization, content creation etc., . A more detailed article on this will follow.

• OCPM - Object centric approach to process mining is a novel way that overcomes the limitations of traditional process mining. It captures the interaction between systems and objects so that processes can be followed more holistically across entire business. The adaption might be steep, but I believe, will bring about a lot of value to many complex processes.

Its a journey that's just getting started. This was meant to give a broad overview and outline without getting bogged into the details. I will follow this up with detailed articles on some of these topics. Meanwhile, please share any of the broad use-cases or trends that you find most relevant to your industry.