# Intelligent Document Processing: The Reading Is Probabilistic, the Deciding Must Not Be

> What intelligent document processing is, its six stages, why OCR and RPA fall short, and why extraction and decisions need to be held to different standards.

Source: https://www.kognitos.com/blog/intelligent-document-processing/

Published May 5, 2026 by Kognitos. Rewritten October 9, 2026.

## TL;DR

Intelligent document processing is the automated capture, extraction, classification, validation and routing of information from business documents, including unstructured ones that OCR alone cannot handle. The useful way to evaluate it is to separate the two things it does. Reading a document is inherently probabilistic. Deciding what to do with what was read must not be. Most IDP failures come from treating those as one problem.

**Key Takeaways:** IDP runs six stages: capture, classification, extraction, validation, exception handling and integration. OCR handles standardized documents and degrades on handwriting, poor scans and complex tables. RPA fed imperfect data breaks. Extraction accuracy and straight-through processing rate are different measurements and should be reported separately. The decision layer is where auditability is required and where probabilistic systems create problems.

## What is intelligent document processing?

Intelligent document processing, usually shortened to IDP, is the automated handling of information contained in business documents: capturing the document, identifying what kind it is, extracting the relevant fields, validating them, resolving anything that fails validation, and passing the result into the systems that act on it.

It exists because most enterprise documents are not forms. Invoices arrive in hundreds of supplier-specific layouts. Remittance advices arrive as email bodies, PDFs and spreadsheet attachments. Contracts, claims, bills of lading, certificates and correspondence carry the information a process depends on, in a structure nobody standardized.

IDP is distinguished from simple document capture by what it does after reading. Capture turns an image into text. IDP determines what that text means in the context of a business process, and what should happen next.

## The six stages

**Capture.** Documents arrive by email, portal, scanner, EDI, or API, in mixed formats and quality.

**Classification.** The system determines document type, which governs everything downstream. An invoice and a credit note require different handling, and misclassification at this stage propagates.

**Extraction.** Relevant fields are identified and pulled. Extraction targets differ by document type: invoices need supplier, amount, line items and purchase order reference; contracts need parties, dates, obligations and termination clauses; claims need claimant, coverage and incident detail.

**Validation.** Extracted values are checked. Amount ranges, date formats, required-field presence, cross-field consistency, and agreement with records held elsewhere.

**Exception handling.** Anything failing validation is resolved, either automatically or by a person.

**Integration.** The validated result is posted into the ERP, case system or workflow that acts on it.

Most discussion of IDP concentrates on extraction, because that is the visibly difficult part. In production, validation and exception handling determine whether the deployment works.

## Where OCR and RPA fall short

Optical character recognition combined with robotic process automation is a reasonable first step and covers a narrow band of documents well: high volume, standardized, consistent layout.

OCR degrades predictably outside that band. Handwriting, non-standard fonts, poor image quality, and documents containing tables, graphs and other visual elements all reduce accuracy. OCR also has no view of the wider process, so it cannot tell whether what it read makes sense.

RPA operates on what it is given. Fed incomplete or incorrect data, bots fail, which is why they are commonly described as brittle. RPA is well suited to processes with few exceptions, and document processes have many, because variation in documents produces exceptions by definition.

The combination therefore handles the standardized portion of document volume and stalls on the remainder, which in most organizations is the larger share and the more expensive one.

## The distinction that matters

Here is the point that should govern how IDP is evaluated.

**Reading a document is probabilistic. Deciding what to do with what was read must not be.**

Extraction is a recognition problem. Given a layout the system has not seen, confidence varies, and no extraction engine is completely accurate. That is acceptable and inherent, and it is where machine learning genuinely helps: modern models handle layout variety far better than template-based OCR.

The decision that follows is a different kind of problem. Whether this invoice can be posted, whether this claim is covered, whether this certificate satisfies the specification, whether this value falls within tolerance. These are judgments against rules, and they carry consequences that are audited.

Those two layers have different requirements:

*Extraction and decision are different problems with different requirements*
|  | Extraction | Decision |
| Nature | Recognition | Rule application |
| Variability | Expected | Unacceptable |
| Needs | Accuracy, confidence scoring | Reproducibility, explainability |
| Failure mode | Wrong value read | Wrong action taken on a correct value |

Problems arise when a single probabilistic system performs both. Output varies between runs, the reasoning cannot be inspected, and when an auditor asks why a particular document was treated a particular way, the answer is a confidence score rather than a rule.

An organization can tolerate an extraction engine that is 94 percent accurate, because validation catches the remainder. It cannot tolerate a decision layer that treats the same input differently on different days.

## Measuring it properly

Two metrics are commonly reported as one, and separating them is the fastest way to understand where a deployment is failing.

**Extraction accuracy** measures how often fields are read correctly.

**Straight-through processing rate** measures how often a document completes without human intervention.

These diverge for informative reasons. High extraction accuracy and low straight-through rate means validation rules are rejecting correctly-read documents, usually because downstream records disagree with the document rather than because the document was misread. Low extraction accuracy with a high straight-through rate is worse, because it means errors are passing validation and reaching production systems.

Reported together as one number, neither problem is visible.

## Implementing it

Five steps that determine whether a deployment works.

**Define document types and extraction targets for each process.** Different types require different fields. Define them before deployment rather than discovering them during it.

**Test extraction against your own document population.** Performance varies substantially by document type and format variety. Test on at least a hundred real documents from your actual intake, not vendor samples.

**Configure validation rules per critical field.** Amount ranges, date formats, required-field presence, cross-field consistency. Validation is what catches extraction errors before they reach downstream systems.

**Design exception handling before go-live.** Every deployment has exceptions. Decide who reviews each type, what information they need, and what each resolution triggers. Undefined exception paths create backlogs.

**Measure extraction accuracy separately from straight-through rate,** for the reasons above.

## Where Kognitos fits

Kognitos operates as the reasoning and decision layer above document capture, working alongside the extraction tooling and systems of record already in place rather than replacing them.

Three properties are relevant to the distinction drawn above. Processing logic is written and read in plain English, so the rule applied to a document can be inspected rather than inferred. Execution is deterministic, so the same document under the same rules produces the same outcome and can be reperformed. And every decision produces a record of what was determined and on what basis, which is what an auditor asking about a specific document actually needs.

Exceptions are handled conversationally: when a document does not fit an existing rule, the person who owns the process explains the handling in plain English and the system applies it from then on, without a developer.

To be clear about scope, Kognitos is not an OCR engine or a capture platform. Document capture and extraction tooling remain part of the stack. What Kognitos addresses is everything after extraction: validating, deciding, handling exceptions, and posting the result with an inspectable record of why.

For related material, see our guides on [document automation](/blog/document-automation/), [optical character recognition](/blog/optical-character-recognition/), [AI document management systems](/blog/ai-based-document-management-systems/), [intelligent document processing in insurance](/blog/intelligent-document-processing-insurance/), and [top AI document processing platforms](/blog/top-ai-document-processing-platforms-enterprise-2026/). To see deterministic document decisioning with a full audit trail, [book a demo](/book-a-demo/) or [try the platform](https://app.us-1.kognitos.com/).

## FAQ

### What is intelligent document processing?

Intelligent document processing is the automated capture, classification, extraction, validation, exception handling and integration of information held in business documents, including unstructured documents that standard OCR cannot reliably handle. It differs from document capture in determining what extracted text means within a business process and what should happen next.

### What are the stages of the IDP pipeline?

Six: capture, where documents arrive in mixed formats; classification, determining document type; extraction, pulling the relevant fields; validation, checking values against rules and records; exception handling, resolving anything that fails; and integration, posting the result into the system that acts on it. Validation and exception handling usually determine whether a deployment succeeds.

### How is IDP different from OCR?

OCR converts an image into machine-readable text. IDP determines what that text means in context, validates it, resolves exceptions, and routes the result. OCR performs well on standardized, high-quality documents and degrades on handwriting, poor scans, non-standard fonts and complex tables, and has no view of the wider process to judge whether a result is plausible.

### Why do OCR and RPA struggle with document processing?

OCR loses accuracy outside standardized layouts, and RPA operates on whatever it is given, so imperfect extraction causes bots to fail. RPA suits processes with few exceptions, while document processes generate exceptions whenever documents vary. Together they handle the standardized portion of document volume and stall on the remainder.

### What accuracy should intelligent document processing achieve?

Accuracy varies by document type and format variety, which is why it should be tested on a representative sample of at least a hundred real documents from your own intake rather than vendor samples. Extraction accuracy should be measured separately from straight-through processing rate, since the gap between them identifies whether failures originate in extraction or in downstream validation.

### What is the difference between extraction accuracy and straight-through processing rate?

Extraction accuracy measures how often fields are read correctly. Straight-through processing rate measures how often a document completes without human intervention. High accuracy with a low straight-through rate indicates validation is rejecting correctly-read documents, usually due to disagreement with downstream records. Low accuracy with a high straight-through rate indicates errors are passing validation and reaching production systems.

### What is the best intelligent document processing software?

There is no single best choice, because performance varies by document type and format variety. Test each candidate on at least a hundred real documents from your own intake, report extraction accuracy separately from straight-through processing rate, and check how each handles validation, exceptions and an audit trail for the decisions made after extraction.