Challenges with data ingestion tools for audit

Data ingestion tools

Audit firms are being held back from the technology revolution that has delivered efficiency and reliable high-quality to other industries by a lack of reliable, standardised data. Firms aren’t able to perform analytics at scale, which means expensive and highly-trained audit staff spends days tidying and reconciling evidence that could be checked in seconds if the data were properly formatted. This makes audits more expensive and reduces the time available for audit teams to focus on risk and quality.

Data is the fuel that powers many audit firm’s mission-critical processes. However, to be of any use to an audit company, the data needs to be plentiful, readily available, and clean. This is where data ingestion tools come into play.

What are data ingestion tools?

Data ingestion tools prepare data for analysis, so you can see the big picture hidden in your data. Automated data ingestion tools work by integrating data from disparate sources, such as Enterprise Resource Management (ERP) systems and HR systems. The integrated data is then transferred to a safe location the auditor has access to where it can be deposited and analysed. The data can be stored in a database, data warehouse, document store, data mart, spreadsheets, either in your own environment or in a data extraction provider’s environment through a SaaS (Software as a Service) data platform. Sounds simple, right? Well. It should be, however, many data ingestion tools out there are not keeping up with the complexities and amounts of data out there.

Data ingestion tools challenges


Back when ETL tools were created, it was easy to write scripts or manually create mappings to cleanse, extract, and load data. However, data today has grown in volume and become highly diversified – for example, now the audit of a mid-sized company is likely to involve data from ERP, HR, payroll, CRM, and potentially workforce management systems, many of them creating data that simply wouldn’t have been recorded 10-15 years ago. The old methods of data ingestion aren’t quick enough to keep up with the volume and scope of modern data sources. As a consequence, today, there is a move towards automated data ingestion tools.


Because there is an explosion of new and rich data sources like inventory barcodes, geo-tagging, automated fixed asset record keeping, mobile phone stock counting tools, and other connected devices, auditors can find it tricky to get and understand all the data they need for testing. This is because of the complexity involved in connecting to that data source and cleaning the data acquired from it, like identifying and eliminating faults and schema inconsistencies in data.


Several different factors make data ingestion expensive. The infrastructure to support different data sources and proprietary tools can be very costly to maintain in the long run and maintaining a staff of experts to support the ingestion pipeline is not cheap. Especially where clients aren’t confident managing their own data. The auditor can find they get inconsistent evidence from period to period, making standardisation of analysis reliant on paying expensive data preparation staff makes it difficult to meet compliance standards throughout the process.

Data security

Security is an ever-present issue when moving data around. Data is often staged at numerous steps during ingestion, which can introduce risks that your client’s confidentiality is compromised by your extraction, and makes it difficult to meet compliance standards throughout the process.

With corporate data growing enormously each year, it becomes challenging for audit firms to keep track of and collect every data point they need to perform a quality audit. Auditors need to turn to next-generation data ingestion tools to get access to the data needed for risk assessment and assurance.

Next-generation data ingestion – EB Integration Engine

Engine B’s data ingestion product, the EB Integration Engine, is solving these data challenges for auditors by automating the extraction process and mapping data to a common format which enables scalable analysis and gives organisations more control and consistency over their data extraction tools.

EB Integration Engine ingests data and integrates it into the Audit Common Data Model (CDM), which prepares it for integration with all the firm’s technology. Alternatively, the data can be analysed in Microsoft Power BI or any client analytics tool.

Need to know more about EB Integration Engine? Download our product guide for more information.