TL;DR
Intelligent document processing is the automated capture, extraction, classification, validation and routing of information from business documents, including unstructured ones that OCR alone cannot handle. The useful way to evaluate it is to separate the two things it does. Reading a document is inherently probabilistic. Deciding what to do with what was read must not be. Most IDP failures come from treating those as one problem.
Key Takeaways: IDP runs six stages: capture, classification, extraction, validation, exception handling and integration. OCR handles standardized documents and degrades on handwriting, poor scans and complex tables. RPA fed imperfect data breaks. Extraction accuracy and straight-through processing rate are different measurements and should be reported separately. The decision layer is where auditability is required and where probabilistic systems create problems.
What is intelligent document processing?
Intelligent document processing, usually shortened to IDP, is the automated handling of information contained in business documents: capturing the document, identifying what kind it is, extracting the relevant fields, validating them, resolving anything that fails validation, and passing the result into the systems that act on it.
It exists because most enterprise documents are not forms. Invoices arrive in hundreds of supplier-specific layouts. Remittance advices arrive as email bodies, PDFs and spreadsheet attachments. Contracts, claims, bills of lading, certificates and correspondence carry the information a process depends on, in a structure nobody standardized.
IDP is distinguished from simple document capture by what it does after reading. Capture turns an image into text. IDP determines what that text means in the context of a business process, and what should happen next.
The six stages
Capture. Documents arrive by email, portal, scanner, EDI, or API, in mixed formats and quality.
Classification. The system determines document type, which governs everything downstream. An invoice and a credit note require different handling, and misclassification at this stage propagates.
Extraction. Relevant fields are identified and pulled. Extraction targets differ by document type: invoices need supplier, amount, line items and purchase order reference; contracts need parties, dates, obligations and termination clauses; claims need claimant, coverage and incident detail.
Validation. Extracted values are checked. Amount ranges, date formats, required-field presence, cross-field consistency, and agreement with records held elsewhere.
Exception handling. Anything failing validation is resolved, either automatically or by a person.
Integration. The validated result is posted into the ERP, case system or workflow that acts on it.
Most discussion of IDP concentrates on extraction, because that is the visibly difficult part. In production, validation and exception handling determine whether the deployment works.
Where OCR and RPA fall short
Optical character recognition combined with robotic process automation is a reasonable first step and covers a narrow band of documents well: high volume, standardized, consistent layout.
OCR degrades predictably outside that band. Handwriting, non-standard fonts, poor image quality, and documents containing tables, graphs and other visual elements all reduce accuracy. OCR also has no view of the wider process, so it cannot tell whether what it read makes sense.
RPA operates on what it is given. Fed incomplete or incorrect data, bots fail, which is why they are commonly described as brittle. RPA is well suited to processes with few exceptions, and document processes have many, because variation in documents produces exceptions by definition.
The combination therefore handles the standardized portion of document volume and stalls on the remainder, which in most organizations is the larger share and the more expensive one.
The distinction that matters
Here is the point that should govern how IDP is evaluated.
Reading a document is probabilistic. Deciding what to do with what was read must not be.
Extraction is a recognition problem. Given a layout the system has not seen, confidence varies, and no extraction engine is completely accurate. That is acceptable and inherent, and it is where machine learning genuinely helps: modern models handle layout variety far better than template-based OCR.
The decision that follows is a different kind of problem. Whether this invoice can be posted, whether this claim is covered, whether this certificate satisfies the specification, whether this value falls within tolerance. These are judgments against rules, and they carry consequences that are audited.
Those two layers have different requirements:
| Extraction | Decision | |
|---|---|---|
| Nature | Recognition | Rule application |
| Variability | Expected | Unacceptable |
| Needs | Accuracy, confidence scoring | Reproducibility, explainability |
| Failure mode | Wrong value read | Wrong action taken on a correct value |
Problems arise when a single probabilistic system performs both. Output varies between runs, the reasoning cannot be inspected, and when an auditor asks why a particular document was treated a particular way, the answer is a confidence score rather than a rule.
An organization can tolerate an extraction engine that is 94 percent accurate, because validation catches the remainder. It cannot tolerate a decision layer that treats the same input differently on different days.
Measuring it properly
Two metrics are commonly reported as one, and separating them is the fastest way to understand where a deployment is failing.
Extraction accuracy measures how often fields are read correctly.
Straight-through processing rate measures how often a document completes without human intervention.
These diverge for informative reasons. High extraction accuracy and low straight-through rate means validation rules are rejecting correctly-read documents, usually because downstream records disagree with the document rather than because the document was misread. Low extraction accuracy with a high straight-through rate is worse, because it means errors are passing validation and reaching production systems.
Reported together as one number, neither problem is visible.
Implementing it
Five steps that determine whether a deployment works.
Define document types and extraction targets for each process. Different types require different fields. Define them before deployment rather than discovering them during it.
Test extraction against your own document population. Performance varies substantially by document type and format variety. Test on at least a hundred real documents from your actual intake, not vendor samples.
Configure validation rules per critical field. Amount ranges, date formats, required-field presence, cross-field consistency. Validation is what catches extraction errors before they reach downstream systems.
Design exception handling before go-live. Every deployment has exceptions. Decide who reviews each type, what information they need, and what each resolution triggers. Undefined exception paths create backlogs.
Measure extraction accuracy separately from straight-through rate, for the reasons above.
Where Kognitos fits
Kognitos operates as the reasoning and decision layer above document capture, working alongside the extraction tooling and systems of record already in place rather than replacing them.
Three properties are relevant to the distinction drawn above. Processing logic is written and read in plain English, so the rule applied to a document can be inspected rather than inferred. Execution is deterministic, so the same document under the same rules produces the same outcome and can be reperformed. And every decision produces a record of what was determined and on what basis, which is what an auditor asking about a specific document actually needs.
Exceptions are handled conversationally: when a document does not fit an existing rule, the person who owns the process explains the handling in plain English and the system applies it from then on, without a developer.
To be clear about scope, Kognitos is not an OCR engine or a capture platform. Document capture and extraction tooling remain part of the stack. What Kognitos addresses is everything after extraction: validating, deciding, handling exceptions, and posting the result with an inspectable record of why.
For related material, see our guides on document automation, optical character recognition, AI document management systems, intelligent document processing in insurance, and top AI document processing platforms. To see deterministic document decisioning with a full audit trail, book a demo or try the platform.
