Artificial intelligence and internal auditing: Risks and opportunities

Our world is changing significantly, and risks are becoming more complex, interdependent, unpredictable, and materializing faster than ever before. Technology is transforming entire industries and internal audit is not immune to this dynamic. The speed at which AI is being adopted is remarkable, so internal audit must evolve to face the challenges that will define the effectiveness of our profession in the future, while noting that technology is not only an auditable subject, but also a tool we can use.

Generative pre-trained transformer (GPT) applications, like ChatGPT, use computer algorithms to analyze structures, relationships and patterns of large data sets and can produce graphics, images, music, text, audio and even computer code. These last three uses represent particularly useful opportunities for auditors because they can significantly improve the way our work is done.

Use Cases

Audit Planning and Project Management

AI can help internal auditors by automating many of the tasks involved in the planning and project management processes. For example, it can verify that recurring reviews are scheduled and staffed, costs are calculated and compared to budgets, prepare and send notifications to the appropriate stakeholders, identify over-scheduled staff members, and update the relevant project management tools.

Fieldwork

Since AI uses algorithms to identify and understand patterns and anomalies, internal auditors are using it do research that involves multiple search criteria simultaneously, identify specific points of interest within the results, compare those to benchmarks or other criteria, combine this information as needed, and display the results.

AI can help auditors identify trends, patterns, and other transactions of interest within data sets. The application of this capability generally includes the search for anomalous text, audio, video, and numerical transactions that may indicate a risky pattern underlying them.

Auditors can identify best practices and correlations that may indicate centers of excellence within their organizations.

AI can create presentations for status updates, turn audio to text for narratives, and turn text to audio for voice messaging.

Risk Assessment

AI can help populate risk registers, which are a common, yet time consuming tool used to capture details like the nature, owner, and mitigation measures assigned to identified risks.

By using results from previous audits and other reports, as well as data analytics, AI can help populate risk registers, quantify complex portions of it, and update risk ratings like impact, likelihood, and velocity.

Operational Quality and Fraud

Since our focus is on the achievement of objectives and the role risk plays hindering them, AI can help identify and understand patterns, anomalies, and areas of risk more quickly and accurately. This can help auditors identify errors, omissions, duplicates, mis-categorized items, and similar issues. By arranging those results by operating unit, product line, timeframe, or similar criteria, it can then identify and flag higher risk areas within the organization for further review.

Another use case for AI is the identification of transactions that may circumvent existing controls, fail to follow preset rules, or deviate from the expected sequence of events. While testing for these patterns and deviations has been standard practice for many auditors in processes like Purchasing, Accounts Payable, Accounts Receivable, and Travel and Entertainment, AI can eliminate the multi-step manual procedures that these tests often require and do the work faster, real time, and on the entire population.

Reporting

AI can generate reports more efficiently by automating many of the tasks involved in the reporting process, such as retrieving, collating, and arranging standardized components of the document. AI can also check grammar and identify issues in the report. This can result in faster and more impactful reports that are free from errors and inconsistencies. By automating repetitive tasks, AI can help reduce the workload of human workers and allow them to focus on more creative and strategic aspects of their work.

Compliance Monitoring

AI can help internal auditors monitor for compliance by analyzing large data sets and identifying patterns that may indicate non-compliance. Triggers can be set and adjusted as needed, providing tiered notifications, tracking their resolution over time, and providing reminders when severity and corrective actions fail to meet the organization’s expectations.

In general, AI can help auditors identify and track areas of risk within processes faster and more comprehensively, improve the quality and speed of fieldwork, and create more impactful audit reports that serve as catalysts for change.

Required Skills

Although AI has immense potential, it requires the intervention of skilled humans to help it do its work. Training is typically done using large data sets and through iterative feedback, AI is taught what is acceptable and what is not.

For example, while AI can improve how audit reports are written, the results generated by large language models (LLM) must be examined and auditors should be familiar with their organization’s writing style, preferences, jargon, and communication format. Auditors must also make sure that the correct audit issues are included, and the information presented for each of them is accurate, objective, clear, concise, constructive, complete, and persuasive given the preferences of the report readers and decision-makers. Instances of LLMs generating false information by deviating from facts or contextual logic, referred to as hallucinations, means that auditors must review AI results carefully to make sure the information reported is factual.

When it comes to data analytics, AI depends on and learns from the data it is fed, so auditors need to know their organization’s data, the quality of that data, and how frequently it is being updated. If the AI tool is taught using inadequate data, at the execution phase it could be analyzing outdated, inaccurate, or otherwise unreliable data.

Internal auditors also need to understand the processes that AI is running against. In Purchasing for example, items are often returned or moved, so the related transactions may show reversals, adjustments, and reclassifications and that may be fine. However, auditors may still want to raise questions about a process that is inefficient. Conversely, unusual transactions like expenses for advertising and promotions, public relations, consulting services or the purchase of long-life machinery may be acceptable, yet be flagged as questionable, until rules are put in place to train the model to better determine what is reportable and how, and what is not.

Additional skills include knowledge of statistical and data analysis, data science and data management so auditors can better assess false positives and be on the lookout for false negatives. These may occur as the organization makes natural changes like deploy new products and services, open or close operating units, hire and terminate employees and vendors, and similar changes that are characteristic of evolving organizations today.

Auditors should also monitor the emerging standards and frameworks for the development and use of AI. This includes:

• NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0)

• ISO/IEC 23053:2022 Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML)

• The European Union (EU) Artificial Intelligence (AI) Act, and

• The OECD AI Principles.

Recommendations

There are many ways to benefit from software that as recently as a year ago, many people didn’t know existed. Studies are showing that AI is improving performance in the workplace, and auditors can benefit as well.

Naturally, a lot of people would assume that AI software is thoroughly tested before it is made available to others, and that developers are following information security protocols, but in many cases that is unknown. Internal auditors need to be careful as they adopt these technologies to make sure they are simultaneously preventing risks that may result in poor results or hurt the organization’s ability to achieve its objectives.

Internal auditors need to learn as fast as the rest of the organization is learning about AI, or better yet faster. This includes identifying innovative approaches we can adopt and deploying those effectively within our units. We must also think ahead to anticipate upcoming risks and opportunities by applying forward and transformative thinking to what we do, why we do it, how we do it, and how we can add value in a world that some claim may be unrecognizable in a not-too-distant future.

When it comes to AI, avoidance is hardly an option. Blindly accepting, deploying, and using it is not acceptable either as evidenced by the instances of bias, data leakage, and deep fakes that have already surfaced. So, auditors and management alike must find suitable vendors with vetted products to partner with and deploy these tools mindfully.

Final Thoughts

Professionals worldwide are already finding ways to increase the quality, depth, breath, and speed of their work. As our world continues to change, we must keep pace, smartly embrace these developments, and cautiously, but confidently, embrace the emerging opportunities before us. After all, fortune favors the bold.

Dr. Hernan Murdock, CIA, CRMA is VP - Audit Content at ACI Learning