How does an AI learn to process invoices?

What makes an invoice an invoice? For the human accountant, this is obvious at first glance from the respective characteristics of the document – but an ECM/DMS used to face incomparably higher hurdles in the context of digital document processing. Today, however, artificial intelligence or machine learning can significantly increase the hit rate in document classification and deliver a huge boost in the automation of enterprise-wide document management.

Letter opener or e-bill?

According to a recent survey by Bitkom Research, 4 out of 10 companies now create e-invoices. They are already reaping the benefits that digital invoice processing offers them, such as faster processes, fewer errors and lower costs. However, this contrasts with a high proportion of companies where “unstructured invoice formats such as the digital invoice in PDF format (60 percent) or the paper invoice (87 percent) are still widely used”.

For the latter, the letter opener is most likely the most important component of daily incoming invoice processing. Modern ECM/DMS solutions with AI support now enable businesses to finally replace manual instruments like this and rely on comprehensive automation through an end-to-end digital approach.

The challenge of document classi­fi­ca­tion

The challenge with all types of documents is that they need to be properly categorized. This is true for both analog paper documents and “semi-digital” electronic PDFs. Who usually makes this decision? Accountants who are tasked with scanning documents or forwarding them to the appropriate colleagues. Errors can occur both here and later in the process, for example, when accounting data is transferred to other systems. Even integrated OCR functionality in a scanner does not provide a reliable guarantee that all information can be extracted correctly from the document on the first try.

Document classi­fi­ca­tion using machine learn­ing

Content services platforms and information management applications such as yuuvis® Momentum, which recognize and process content using machine learning, represent the next step in document classification. The term “artificial intelligence” is commonly used in this context, but it does not apply directly. This is due to the fact that an AI system is initially dumb – intelligence must first be trained. It has to learn what defines an invoice. In contrast to traditional document management systems, which use document item data to record information about the customer, vendor, discount, VAT, final total, and so on, AI systems take a different approach. Using “Natural Language Processing (NLP)”, natural language is captured and processed via algorithms.

Basis for machine learning: valid training data

The quality of the data used to train an ECM system with AI support is the determining factor for its “intelligence quotient”. For this purpose, the company’s historical invoice data is used, which is known to be correct. After being trained in this manner, the system can be “unleashed” on new documents.

By setting threshold values, the clerk can check the probability with which the system recognizes that it is a specific customer. If this threshold value is already appropriately high, a new check is no longer necessary. If, for example, the system assumes that 99.98 percent of the document is an invoice, there is little doubt. Because all item data recognized in the document is estimated and displayed as a percentage, the clerk can perform the validation step by step. The results of the human evaluation are fed back into the system, improving recognition quality in subsequent runs.