As generative AI models like GPT-4 continue to advance they bring unprecedented capabilities for a wide range of tasks to include writing articles, generating code, and facilitating ideation. However, they also carry with them certain risks and biases that can have a significant impact on the reliability and repeatability of executing code. In this blog we will explore the inherent dangers of relying on generative AI for code execution and discuss how Kognitos, a next-generation automation platform, helps mitigate these risks.
The Risks of Generative AI in Code Execution
To illustrate the risks associated with using generative AI by itself for automation, let’s consider an example. We attempted to use GPT-4 to perform a simple task: find the sell price of a product based on a list price and a partner’s discount rate. We utilized the following prompt:
“””
calculate the sell price
get the partner’s discount rate
the result is the product price * the discount rate
Calculate the sell price
Run the above. When you don’t know something, ask.
“””
On running the above prompt with two different inputs: “$100, 3” and “$100, 30“, GPT-4 provided the answers $300 and $130, respectively. In the first case, its bias lead it to assume the rate as an absolute number, while in the second case, it treated the rate as a percentage. This subtle discrepancy, present in just a few steps, could have far-reaching consequences when scaled to an enterprise end to end process.
Let’s take a look at a different example. We asked GPT to be a medical insurance claims processor. The user will provide the claim details and the patient’s insurance policy and it will have to approve a reimbursement amount based on the insurance policy. We then tested out the following GPT prompt:
“””
Insurance Policy:
Insurance Company: ABC Health Insurance
Policy Holder: John Smith
Policy Number: 123456789
Coverage Details:
In-Network Provider: 80% coverage
Out-of-Network Provider: 50% coverage
Deductible: $500
Out-of-Pocket Maximum: $5,000
Past Claims:
1. Dr Alex Dunphy
Total Cost: $300
Reimbursed: $0
2. Dr Prabhat Kumar
Total Cost: $700
Reimbursed: $400
—
Claim
Provider Name: Dr. Jane Doe
NPI Number: 1234567890
Tax ID: 123456789
Address: 123 Main Street, Suite 200, Anytown, USA
Cost Breakdown:
Consultation fee: $150
Medication fee: $50
Total Cost: $200
“””
On running the above prompt, gpt responded back with this:
“””
Based on the insurance policy provided, we need to determine if Dr. Jane Doe is an in-network or out-of-network provider. Since this information is not provided, I will assume that Dr. Jane Doe is an in-network provider for this calculation.
Insurance Policy:
In-Network Provider: 80% coverage
Out-of-Network Provider: 50% coverage
Deductible: $500
Out-of-Pocket Maximum: $5,000
Past Claims:
1. Dr Alex Dunphy
Total Cost: $300
Reimbursed: $0
2. Dr Prabhat Kumar
Total Cost: $700
Reimbursed: $400
Current Claim:
Provider Name: Dr. Jane Doe
Total Cost: $200
Since the deductible has not been met ($300 from Dr. Alex Dunphy and $200 from Dr. Prabhat Kumar, totaling $500), the remaining deductible amount is $0.
As Dr. Jane Doe is assumed to be an in-network provider, the insurance will cover 80% of the cost after the deductible has been met. In this case, the total cost of the claim is $200.
Reimbursement Calculation:
Total Cost: $200
Insurance Coverage: 80%
Reimbursement Amount: $200 * 0.8 = $160
However, since the deductible has just been met, the reimbursement amount will be $0 for this claim. The patient will be responsible for the full $200 cost of this claim.
Please note that if Dr. Jane Doe is an out-of-network provider, the reimbursement calculation would be different.
“””
One can clearly see that not only did GPT’s bias make it automatically assume Dr. Doe to be an in-network provider, but it also incorrectly reasoned that the deductible wasn’t fully paid, when in fact John had already paid the $500 deductible in his previous two visits.
As illustrated by the above two examples, it’s clear that GPT-4 can sometimes incorporate its own biases or opinions while processing user requests, leading to potential discrepancies in the output. This underlines the challenges that emerge when using generative AI by itself for automation or code execution, particularly when deterministic and unbiased execution is crucial for automating manual processes. Within an Enterprise, mistakes such as those noted above could have significant effects and are not tolerable for automation.
The inherent creativity of generative AI models is both their strength and weakness. While they excel at tasks requiring imagination and ideation, they may introduce biases when it comes to executing code in a predictable and repeatable manner.
Addressing AI Bias in Computation
The issue of AI bias in computation is more serious than many might realize. So far, the focus has primarily been on addressing racial and social biases in AI, but the risk of biased computation is just as significant. To ensure the safety and reliability of code execution, we must be vigilant in exposing and addressing these risks.
For instance, imagine a scenario where an HR system calculates an employee’s raise based on GPT-4’s understanding of what is considered a “normal” raise. In this case, GPT-4 might unintentionally introduce biases stemming from its training data, leading to unfair or even discriminatory outcomes.
In addition to the risk of biased computation in AI, generative models like GPT-4 also perform poorly at repetitive tasks. The nature of these models is to generate diverse and creative outputs based on patterns identified in their training data. However, this can lead to inconsistencies when it comes to tasks that require strict adherence to a specific pattern or repetition. This could lead to incorrect or incomplete results, which can have serious consequences, especially in safety-critical applications. Also, Generative AI models are computationally expensive, especially when applied to repetitive tasks. The amount of resources required to process and generate results might not be proportional to the task’s simplicity, leading to inefficiencies in resource utilization.
Thus, there is a need for a system with “computational intelligence” – one that is able to understand human language and reliably execute instructions just like traditional software.
In a follow up blog we will dive into the details of “Computational Intelligence” and how Enterprises can still leverage the benefits of Generative AI, in a way that is precise and consistent.
Want to Unlock the Power of Generative AI for Your Business Today
