Robotic Process Automation (RPA), low code and no code systems, are all systems that enable businesses to automate processes. However, they all suffer from a large gap between the expected benefits to business and what’s observed in reality. Numerous surveys have suggested that 30%-50% of RPA projects fail, and only 3% of bot deployments reach scale.
Let us dig deeper and find out why.
The promise of RPA has always been that it can do mundane tasks that otherwise would have to be done by a human. In general, that is the primary purpose of computers. The real difference with RPA is in the expectation of who the developer is. In RPA, and also the low code/no code systems, there is the expectation that the business user or the citizen developer will build the automation. There is no requirement of formal computer programming experience from these business users — at least that is the initial marketing pitch, and the demos demonstrate how easy it is for a non-programmer to create an automation via a drag-n-drop UI. And when it works it does seem magical!
But, the dirty secret about any automation is that the initial creation of the software is just the marriage ceremony, married life is what happens next.
The initial creation of automation (or the marriage ceremony), is a beautiful well orchestrated demonstration of good intent and promises. It is the happy path or sunny day scenario. Seasoned software developers know that it is only 20% of the work.
The Day 2 — Day N of automation (or married life), is full of surprises, edge cases, exceptions, changes in requirements, changes in expectations, changes in environment, changes in people, growth and acquisitions. Or as developers call it, “an average day at work”.
The complexity of Day 2 and beyond forced business users to hand over the RPA tools to the IT department which created a Center of Excellence where developers (either in house or consultants) were hired to maintain the automations. That still does not explain the large rate of failure and frustration in RPA projects. We don’t hear of general software projects failing that often. What gives?
Let us double-click into what challenges IT faces when implementing RPA.
The top 3 reasons why many RPA projects fail
Reason #1: The programming environment is inherently unstable
RPA systems, more often than not, fill in the functionality gaps between existing business tools. For example, taking an invoice from an email and extracting data from it, performing validations and updating the general ledger; or receiving a record from Salesforce and creating a corresponding record in Netsuite after cross-checks and validations. There are many such processes that humans are manually doing today that RPA helps automate.
However, there is a reason why these gaps exist. Either it is hard to configure the existing products to fill in these gaps, or the existing products don’t have the requisite features. The latter happens when the diversity of integrations required are too large for the software vendors to grapple with, or the environment is quite fluid and unpredictable. For example, a website could change its look and feel breaking automation that depends on it; or a payroll system might change its behavior without coordinating with any financial planning tools that depend on it.
Humans, who fill in those product gaps, are adaptive and don’t crash when a website moves the “Submit” button to a different spot on the page, or when the payroll system rolls out an improved version of their software.
Dealing with a bunch of third party tools and services is hard, and that’s what RPA is tasked to do.
The inherent instability of these environment makes implementation hard, even for the very best developers. The automations created regularly break and are constantly hungry for fixes. This results in “bots” being hungry for “baby sitting”, and the realization that we can only have a few bots. Did we say “married life”?
Reason #2: The business requirements are not clear and change a lot
I have built software products all my life and I know that gathering requirements is not easy. There is a reason why “product manager” is a full-time job. If we don’t invest in figuring out what the requirements are up front, it will lead to later changes which will be more expensive to do. Some of the requirement for a product manager is fulfilled by process mining tools that are deployed in conjunction with the RPA solution. But deploying these tools is also a chore and RPA projects inevitably require many meetings with various stakeholders and business users across the organization with the objective of figuring out the business processes in great detail. Then, based on the stability, complexity and value to business of the process, it is determined whether a process is fit for automation.
If we simply ask the business user to describe a business process in detail, more likely than not, they will miss some key scenarios. Humans have evolved to very quickly figure out the next best step in any situation. But, we have had no evolutionary need to chronicle all the possible paths to take in a complex workflow. Planning is hard, reacting is easy.
Business requirements also change often. In a growing business, processes are created and modified almost every quarter to help the company evolve to the next level of scale and maturity. Acquisitions and mergers can force changes to processes. Governmental regulations may throw a wrench into business process as well. These factors make it uneconomical to produce automation for processes that are subject to change.
Reason #3: The developers are not all in-house
Finance, Sales, Marketing, HR and Support are not software development organizations. When business leaders decide to bring in automation tools, they have to decide between hiring developers in-house or leveraging consultants. Today, it is hard to hire talent in the automation space as there are not enough developers trained in these specialized tools. Further, the business users are trying to reduce their workload by offloading it to a machine, and it seems antithetical to the idea to hire a team of humans that now you have to manage. Most enterprises chose to have a combination of consultants and in-house developers to build out the Center of Excellence.
Creating highly maintainable software is very hard when most of team are part-time consultants. Long term owners of software are critical for the success of any software product. If there is churn in the staff maintaining automation, it reflects in poor maintenance of the automation leading to poorer long term results.
So, where did RPA succeed?
RPA succeeds in automating repeated tasks that are narrow in scope and don’t change much. As long as that process is not in an unstable environment and the requirements are not going to change, it is likely to succeed in saving a lot of time for humans. However, those kind of processes are a small fraction of what could and should be automated. Gartner estimates that hyper-automation market to reach $600B by 2022, and RPA will barely scratch the surface of that.
We need to think beyond RPA.
The Future of Business Automation
No. It isn’t RPA.
Notice that the fundamental reason for all of the problems around automation is that automation needs to evolve on Day 2, and that is hard to do with RPA. The humans that are currently driving the business processes manually, are inherently able to handle and adapt to unforeseen conditions and are able to rewrite the process as they go along.
The biggest difference between humans and machines is how they behave when something goes wrong.
Can we build such a system that has human-like error handling?
Yes. With the recent advances in deep learning and compiler design, this has become achievable.
Below are the 5 requirements for automation to become human-like.
The above table shows the fundamental difference between machines and humans. When a piece of automation realizes it does not know how to do an action, it just crashes. A human on the other hand, would simply ask the manager, learn the new fact or skill, and continue. There is a reason why we do not need to do “process discovery” before hiring a new employee. The expectation is that the employee will ask if anything is confusing and learn from the answer. So, the training cost of a human gets spread out over time, unlike software engineering which front-loaded with a lot of the development cost.
The table below shows a qualitative representation of this observation. Each business will have different numbers but the picture helps visualize the problem better.
Humans can be trained incompletely and yet be productive to the business.
Machines have to be trained more thoroughly upfront because of cost of making changes later in the lifecycle is steep.
Good software practice is to articulate all edge cases up front and build that into the software. That leads to a large up front cost.
Humans with time get more useful to the business without explicit effort in retraining them. We call them “experienced employees”.
Machines, on the other hand, get ossified and hard to change, and after some time require a rewrite of logic.
If we can handle errors without crashing and change automation without a development cycle, we can fundamentally change the TCO of automation.
Such a system would exhibit:
- Low up front cost of training.
- Low cost of adaptation to new business logic.
- Constantly increasing benefit to business with decreasing investment.
The Solution
In a previous article I had articulated the journey towards building such a system that dramatically increasing success rates and reduces TCO of automation. There are two breakthroughs that such a system will possess:
- Ability for the computer to ask, learn and continue. This allows the computer to reach out to the user on edge cases and exceptions and learn new facts or procedures and carry on — just like a human.
- Ability to program in natural language. This allows the computer to reach out to the user with questions in natural language. Further, the user can understand the code even if someone else wrote it.
We, at Kognitos Inc., have built such a system and foresee that many others will come forward with similar solutions. The business process automation industry will finally be able to tackle the TCO and scaling problems around automation, leading to more efficient and dynamic businesses in the future.
Discover the Power of Kognitos
Our clients achieved:
- 75%manual data entry eliminated
- 30 hourssaved on invoicing per week
- 2 millionreceipts analyzed per year
