Audit

What better way to start the first post in the “CPAs” series than with a world-class lady. Angelique Koopman, Partner Audit Innovation at Baker Tilly Berk, has been following MindBridge since our early days and recently visited our office. We could not resist asking her a lot of questions.

In this series, we will interview various CPAs in the profession as we travel across the globe, asking them for their opinions and personal stories. If you ever would like to share your own experience as a CPA, please do not hesitate to reach out directly.

Solon:  Where do you think analytics and artificial intelligence (AI) would be the most useful in the audit process today?

Angelique:  Planning and risk assessment are often based on professional judgment. And what I would like to see is the use of more data analytics in the planning phase of the audit so we can substantiate our considerations and our risk assessments with facts (quantitative data). This way we are collecting evidence to support our risk assessments early on.

Solon:  So how should we address that deficiency, and what impact do you think it has during the audit process? What surprised me recently is that there are firms that do it in the review phase, then they miss something, and they have to go back. If you do integrate next-generation AI systems in the planning phase, do you think it would help to decrease this need to re-do portions of the audit?

Angelique:  Yes, when you are doing your risk assessment or your planning, it’s better to have a set of preliminary analytics based on facts. That way, you can substantiate the choices you make which can help you decide to perform all the procedures or just a subset of procedures. Right now, we do risk assessments to limit the audit work. If you do not plan your audit well enough, at the end of an audit, you will find that you missed something somewhere and then you have to go back and do additional work, and I believe this is one of the best arguments for using analytics more in the planning phase.

Solon:  At the end of the day, is it just about analytics, or are there other industry best practices we can also be including?

Angelique:  It is the combination. It’s a combination of automating rigorous procedures and it’s about automating specific aspects related to the industry. I strongly believe in an industry driven approach. It is more of the rule-based part using the industry expertise. But I also believe that analytics must be used in combination with the auditor with all his knowledge and the holistic view of the client. Giving the auditor the technology to do better, to improve professional judgment, to improve decisions based on intuition or feeling and to substantiate those decisions more comprehensively. That is really important. To be honest, I don’t believe in just having a tool or platform only, because, in the end, an auditor needs to evaluate the outcome. But what you can do is help the auditor from performing tedious tasks that could otherwise be automated. Such tools provide the auditor with more insights and knowledge such as making cross-references and analyzing data from more than one dimension. What we normally do with audit analytics, is that we have a question, then we build the query and the query gives an answer that’s different, so I believe in the combination of both analytics and best practices in combination, that it will help auditors become more focused on delivering better results.

Solon:  There is a lot of confusion over the role AI plays in the audit. What would you tell a traditional audit partner who is evaluating if he/she should leverage machine learning and artificial intelligence (AI)? 

Angelique:  Mostly when people hear about AI, they believe that it is science fiction. And they might think it is in the preliminary stage and are not aware of the fact that nowadays lots of things in our society are run by algorithms. We use them everywhere. Unfortunately, we are not using it in the audit on a large scale yet. Maybe, we think it is scary. However, when we get into a plane, we fully trust the airplane, but in the audit world, we do not trust algorithms yet. Basically, it’s about combining statistical methods with machine learning. So I think they need to know what machine learning is and how it works. We have to explain it with simple examples, like how does a program operate and learn? As an example how to identify a cat from a picture? In fact, AI is about automating tasks that are beyond the capacity of human beings. Machine learning is a way of making complex analysis possible. Automating repetitive audit tasks makes even greater business sense when you combine it with machine learning and AI algorithms.

Solon:  People talk about the robot taking the job of an accountant… 

Angelique:  It’s very similar to the team that supports you when you are auditing a bank, and the bank has financial instruments, and you have team members who are specialized in that type of audit; they have the know-how of the financial instruments. Or when you are auditing a real estate company, you use the expertise of people who know that space, you cannot do it by yourself. It’s only expertise enshrined in a computer program. That’s the difference.

I was telling Robin and Eli, knowing that this technology is here, and at this point, it would be hard to go back to audit and do the role I used to do without leveraging this technology. I can clearly see the value it adds, I can do a manual sample, but recognizing that with AI I can sample the whole ledger with a higher degree of assurance. And given that it is my signature on that audit statements, I want to do the best job with the best tools available.

Solon:  So using artificial intelligence (AI) is going to become a matter of professional integrity?

Angelique:  I think, this is something in the audit community which is starting now. And I gave the same message at a recent conference in Belgium. People start to think about it, and they are starting to say, “Yes, why not? Change is coming!”

Originally posted on Solon Angel’s Linkedin post

Auditors love statistical sampling and so does the MindBridge Ai team. Why wouldn’t we—statistical sampling uses the laws of probability to measure sampling risk. We truly believe that statistical sampling largely outperforms judgment sampling.

Don’t get us wrong—the experience of a Partner is undisputed and their ability to focus the audit and identify areas of risk in financial statements is the key to a successful audit and, of course, the peace of mind once an audit opinion is issued. However, once the risk areas are identified and a population of thousands (if not millions) of transactions remains to be reviewed, experience alone might not be sufficient to select a test set.

Statistical sampling and the laws of probability make it feasible to pull a test set from a huge population so an audit team can use it to give reasonable assurance that the overall population is free from material misstatements.

Can audit teams give the same reasonable assurance and test fewer transactions?

There are many commonly used methods to enhance statistical sampling in the audit profession. For example, Monetary Unit Sampling is one (very, very popular) way that a test set can enrich the items most likely to be of interest to auditors. In Monetary Unit Sampling, every dollar is regarded as a distinct unit and given equal chance of being in a transaction picked for testing. Therefore, the higher the amount of a transaction is, the more likely it will be part of the test set. In an audit context, this is great as larger transactions, the ones that are more likely to cause a material misstatement, are more likely to be picked for testing.

How did MindBridge come up with an improved sampling methodology?

First, let’s clarify what improved means. For us improved means to enrich data in a meaningful way so it makes the whole testing process more efficient. Similar to Monetary Unit Sampling, we enrich a data set with additional information to make it more likely that transactions get sampled that are of audit interest.

The intelligent sampler

We run a number of tests to calculate a risk score for every single transaction in a data set. This risk score is the basis of our intelligent sampler which pulls a stratified sample set across the whole population. As with Monetary Unit Sampling, every transaction has a chance for being selected for field testing; however, the ones more likely to be of audit interest having a higher chance.

What does “of audit interest” mean?

“Of audit interest” means the likelihood that a transaction is misstated due to fraud or error. In a number of field experiments, the Ai Auditor™ has proven its ability to identify and push fraud- and error-prone transactions to the top of our risk ranking.

This ability of our Ai Auditor™ to identify transactions which are more likely to be fraudulent gives audit teams the ability to reach the desired confidence level with a smaller test set of transactions. MindBridge’s intelligent sampler is a focusing tool that helps the auditor identify audit areas that they should spend the most time on, but at the same time makes sure they also spend enough time in other areas beyond the high-risk transactions.

Comparing statistical random sampling with intelligent sampling

Our data science team created a general journal for educational purposes. This journal consists of 2966 transactions of which 24 transactions are non-compliant. The non-compliance rate in this file is 0.8%.

Assuming an acceptable error rate of 1%, it would be required to test a sample set of 230 transactions to achieve a 90% confidence utilizing statistical random sampling methodology.

Our intelligent sampler takes an equal number of samples from the Low and Medium/High-risk categories. This ensures that every transaction has a chance of being selected in the sample, but those with higher risk have a higher chance. Using this method the auditor has a 90% chance of discovering non-compliance in the file with just testing a sample set of 14 transactions.

Approach	Confidence level / error rate	Required Sample Size
Statistical Random Sampling	90%/1%	230
Intelligent Sampling	90%/1%	14

 

If you’re interested to learn more about MindBridge™ and our intelligent sampler, or would like to run your own A/B test,  please get in touch.

Throughout my investing and business life, I’ve been amazed by the number of accounting scandals and re-statements that have occurred. The issue of regulatory oversight is raised in the press, but even with increased enforcement and higher penalties, a new accounting issue is never far behind, and it’s continued to get worse. Just look at the impact on British Telecom’s stock because of the irregularities found in accounting in their Italian division (read about it here).

One week into joining MindBridge Ai, I can see how we are focused on the goal of solving one of the massive issues underlying these scandals, and I can see how we are building an expert system to help both internal and external auditors provide us with higher levels of assurance about the financial statements they are sworn to certify.

And the baseline issue we are dealing with is the amount of financial data they need to consume. There has always been too much of it. The Big Data problem facing auditors is that they are asked to “go fishing in a murky lake” and find accounting anomalies and irregularities for further review. To this point, accountants and regulators are agreed that this should be done using statistical account sampling. However, the quality of the sample can only be measured after it has been sampled and analyzed. And with the time pressure on providing audited results, if an irregularity or anomaly is found, the process has to start again. And this leads to costly re-statements which impact shareholder value and confidence in the audit firms.

To solve this, Mindbridge Ai is leveraging machine learning AI to provide auditors with a more highly justifiable sample before the audit begins. Our Auditor AI service can ingest and analyze the entire leger and pinpoint areas they should be investigating early on with a higher degree of justification against the standard, and currently accepted sampling techniques. This gives auditors a higher assurance rate and helps them improve upon the current approaches that only meet the standard, and not move past it and towards the answers we, the press and firms like British Telecom need.