3 ways artificial intelligence helps with audit planning
Artificial intelligence (AI) is transforming the audit industry, enabling more efficient and cost-effective audits while redefining the nature of audit planning.
Do you know how AI can help plan and execute more effective audit engagements?
With AI-enhanced auditing, you get rapid insights into how financial transactions are performing across 100% of the data – with none of the downsides of traditional sampling methods. You also get timely visibility in assessing business risks and identifying areas where special consideration is required.
Join John Colthart, VP of Growth & General Manager, Audit and Assurance, for an in-depth walkthrough of how MindBridge Ai Auditor helps plan better audits. In this webinar, you will learn:
- How AI helps assess and review business risks faster
- How Ai Auditor helps identify areas for special audit consideration
- How interim audit reviews compress the time needed for end-of-year audits
You will walk away with a better understanding of AI and knowing three ways in which AI-enhanced audits help plan more effective engagements.
Again, good morning, good afternoon, and good evening to everyone on the line. Thank you for joining us for the webinar on three ways AI helps with audit planning. Obviously, as you can tell by our first screen here we're all about shift happens, we're trying to make sure that folks know a little bit about artificial intelligence, what machine learning's all about, and how it can impact you here in the world.
And in this case, the world of audit and accounting. Most of us obviously know what an audit's all about, but just taking it directly from the AICPA's website, here's the real skinny version of it. And if you read through this, you're basically looking at the statements that we make around audit, is that the auditor is "reasonably sure," i.e, they have a high degree of confidence or assurance that the financial statements they've just audited are free from material misstatement.
And when we think through that, when we think through what the auditor needs to do in order to get to those views on the financial statement, there's lots of information and lots of data. And I think at times we sort of go back and fall back into a bad paradigm of understanding whether there is in fact, a problem with auditing. And I'm here to kind of remind everyone that absolutely there is.
Just a few short weeks ago The Globe and Mail and a publisher of newspaper out of Toronto, Canada published this article. Why it's Getting Harder to Trustthe Auditors. And we feel that this is a really important piece of the puzzle. We know that in the landscape today of an audit, and we're going to talk about planning here in a second, but in a landscape of a full audit engagement, there is a disconnect between what your clients are looking for or you as the clients looking for, versus what the external auditor does.
And if you're on the internal side, in the internal audit side working for the CFO or the chief audit exec, you've got that same dilemma, what they need and want is disparate to the time and energy you might have to put in. So I don't know that it's necessarily that it's getting harder to trust the auditors, I think we've got an expectation gap, and I think part of this starts right from the beginning, when we start planning out and thinking through the audits.
Now, one of the things that is in passing articles like that that you saw where it's getting harder is the amount of instances and the amount of times where we start seeing auditors' names appearing in newspapers linked in some way, shape or form to some financial disaster at an organization.
And the Association for Certified Fraud Examiners put out a Report to Nations every couple of years, and their most recent one is as alarming as the last one, maybe even more so. In that, there's a surge, there's a continued surge of loss in the financial markets due to error and intentional items like fraud, surging almost to 4 trillion annually, and that's just based on their estimates of 5% of loss applied to the GDP worldwide.
That doesn't include other types of business challenges that you run into that might be impacting you based on not fully appreciating what's happening in that business of yours. So, this is just one viewpoint. And we have a pretty strong viewpoint here at MindBridge Ai that a lot of it comes down to analyst fatigue.
A lot of it comes down to the fact that things that we're looking for, and how we're looking for that data doesn't make sense anymore. So we have challenges where the analyst is fatigued because they're doing the majority of the work manually, and we've got a tools mismatch. And if we look at the term "reasonable assurance" and we think about that in its articulation of what sampling really means, it's a coping mechanism to solve the data problem.
We have to take a smaller subset to put our human eyeballs on, in order to validate these pieces of information because it's fairly clear that most teams, whether the CFO directly or indirectly through the external audit partner that they've selected would rarely put enough investment to research every single transaction, to understand and ensure that there isn't some type of issue.
And as I said, that Report to Nations report that the ACFE puts out continues to remind us of these challenges. Now kudos to you internal auditors and external auditors, you make up roughly 29%, 30% of all fraud found around the world. But that means 70% is still going unaccounted for or when we do find it, it's found by things like a tip line or a mistake.
Remember those old facsimiles, the buzzing sound you would get when you picked up the line and someone was trying to send a document? These are how most of these fraud situations are still found today. It's just that we've upgraded a bit, we've got telephone lines to dial in, we've got, you know, anonymous website areas that you can go in and post. But the net result is we do have a challenge in the world today with audit, and we'd like to think that we can help start solving some of those challenges for the team.
Now I did put a couple of polls in here. So, I'm going to put a poll up here on the screen. So what is the biggest challenge as you see it? Is it too much data? Not enough time? Hard to find the right stakeholders? Not necessarily that you're confident in your plan.
So what did you just tell me? Well, time and data, which is fantastic because we're going to spend time talking about these two areas, right, because this is exactly what we're seeing around the world in every country I visit, these two things are the most prominent. Now there's still a little bit of a challenge getting the right stakeholders, getting the right plan in place, but a lot of times it comes down to your ability to execute as quickly as possible, and doing that with the enormous amounts of data that have just come in.
All right, so what is an audit plan? How do we do it? Well again, using my good friends over at the AICPA, I pulled it up, and just thought I'd throw it here for a second. When you look at the audit plan, it's all around making sure that we spend time understanding stakeholders, understanding risks, understanding potential areas for us to really focus on and harness the energies of the audit team.
And then the other side is actually finding time to manage through the process, you know, how many team members do I need on this audit? What are their skill levels and capabilities? Do I have staff ones and twos? Do I have, you know, a bunch of seniors because of the potential risk that we found as we've interviewed and done a lot of the qualitative analysis?
And then there's the supervisory work that you have to do as you go through the overall plan. So, clearly there's a lot of things that have to happen when you plan the audit. And what I'd like to do is actually take a different tangent that you may not expect in a webinar. I'm actually going to go a little bit product, a little bit slides for the rest of the remaining time that we spend together.
So a little bit about the setup itself. What are the things that we care about as we're setting up the audit? Well, we want to make sure that we have some data. We want to make sure that we understand what types of things we're going to run, and whether that's going to be influenced typically, by our engagement management tool or for a CPA firm or a practitioner out in the external audit.
Then on the internal audit, there would have been a plan or a schedule created maybe last fall, maybe over the summer that influences what you're going to go spend time on. I want to talk specifically about how we get started leveraging Ai Auditor to actually fill out some of the details. Let's talk about the setup, let's talk about how we can start planning better with the teams that we have and getting oriented.
So obviously, I've got six engagements on the go right now, or six organizations and then a bunch of engagements underneath that on the go at any given time. Let's talk about why this is important to look at and to start processing when we think about, how do we get going on actually planning that audit? So we have to do a lot of qualitative work, we're doing interviews with the CFO, with various people for internal controls, but we're also starting to amass a bit of data.
And we need to very quickly get that into a sensible and easy-to-review way, so they can understand the potential risk. You know, now for a lot of organizations that might actually be for a lot of firms doing an interim engagement. So, we're going to very quickly rip through creating a new organization and a new interim. So, we're going to with Funky Plumbing Co.
And if you have a client ID, if you're an external firm and you want a bill code, throw that in there. And I'm going to go ahead and start building my team right from the beginning, right? If I'm an audit partner I just signed this new client, I'm ready to go, I'm going to ask Danielle to spend some time with me, and you can expect to ask Eric to spend some time. And they're going to be part of the management of this engagement or of this organization.
It allows me to very quickly get them going, notify them that, "Hey, we've closed that client, we're ready to start rolling." I'm going to build a new engagement. I have the opportunity to go in and decide exactly what it is that can fit the industry, I can go ahead and copy from previous engagements.
Now, here's a great thing about planning your audit. One of the biggest challenges that most firms we see and most teams that in terms of internal audit see is just how do you get going? How do you start using the same types of philosophies across the firm? We all have that professional practice team or the quality control team, but really, you want to get to a point where you can essentially systemize the way that you're going to assess information and risk so that everyone in your team can be as smart as the other people in that team.
So we can go ahead and use that to pull over previous examples. I can go in and supply what type of engagement this is. If I already know the financial management system, the ERP, I can go ahead and pick it. You know, maybe it's Sage 50. And I can go through and decide, "Well, who else on the team is probably going to work with me?" So, we'll grab Chris and Stefan, and make them auditors.
And this is very important, because there's different ways and different roles that people get to play as you go through the engagement. If you think about, "Well, what is the plan? And what is the purpose?" It's finding the right expertise to fit that right plan. So I know that Chris is great at construction payables. I'm going to assign him, I'm going to add him into the fold, I'm going to help him help me. So again I get to pick what type of engagement it actually is.
I'm going to do an interim. Now I know that most teams where we are based on the year, you know, we're just into the 11th month, a lot of teams have finished up their September 30th reporting, they're getting ready for their 12/31, for December 31st year ends for businesses. Maybe I need to start doing some more work up front to make sure this plan is right.
So I'm going to do that interim. And I'm going to go ahead and add in some information. Now, what you see here is, you know we teach you, right? Artificial intelligence and the audit plan. I'm going to walk through some of these areas and show you where some of that smart can be, how you can actually be empowered as an auditor, as a certified or chartered professional accountant, wherever you are in the world, whatever phrasing you use, how you can actually get a little bit closer to leveraging better technologies to do some of these things.
So we're going to just do a very basic vanilla view here, and I'll show you some work that I've already done, you know the old typical cooking show example of once and done. But we're actually going to look at all this data at the same time. I can go ahead and connect with Intacct or NetSuite or QuickBooks Online, I can drag and drop files from different ERPs. And I'm just going to go and grab that interim dataset with nine months of data.
And here's where the smarts are happening. It's analyzing anything I'm submitting into the system. And again, rather than having just a single person out there going and building these things, you can have your entire team enabled to start using analytics as part of the planning process.
The system has automatically understood what's in my dataset, you can see all these codes that are coming from my data, and it's already automatically mapped things to where it thinks they should be. If I had user, if I had source, if I had a difference of entry date versus posted date, we can drag those across if it didn't automatically pick it up.
But this is where some of that AI comes in. And when people hear the term AI, they're sometimes a little bit confused, right? AI is actually an ensemble of lots of things working towards making them seem a bit smarter. In this case we've used things like natural language processing, you know, natural language understanding. Remember when you click on that home button of your phone, you hold it and you ask that assistant to do something for you?
It's using natural language understanding and processing to triangulate what you're trying to achieve, then it'll go do that task. We do the exact same type of thing with your data. So it knows where the columns are, it knows what, kind of, the data should look like based on what's there. Again, this is to give you the opportunity to just verify a few things as you're loading in that data, but it let's you see where things are.
We can see that we've got quite an interesting number of transactions that have a higher number of entries, and then I've got a few more that just have a little bit more complexity, they've got a lot more entries inside those transactions. I can take a look at any one of them if I want to and preview what the data is. But I'm going to go ahead and just accept the next. And again, if I had copied over previous engagement settings, some of these things would have just happened for me.
But let's say that I do, in fact, want a materiality threshold, I'll throw something simple in there, $50,000, every single financial management system has different ways to flag what their manual entry codes are, otherwise we try to dynamically do it. And there's lots of keywords that you might want to use as part of searching and sifting for something that might be just a little bit odd or a little bit anomalous.
So, I'm going to go ahead and finish that import. It's going to tell me that there's a bunch of accounts that haven't quite yet been verified. Again, you're going through this for the first time, so you might have to do a little bit of work. But again, we've used that AI componentry to dynamically figure out, "Hey, based on the data you gave me, I see bank account, it's likely going to fit into this level of reporting hierarchy."
Now, I know there's a couple of things that I probably should have fixed in here, but they're not going to make a material difference to my audit that we're going to run through here together. I just want to get going. So I'm going to start the analysis. So, the system will dynamically now take this data, it will try to detect what it's all about.
So it didn't register that this was Sage 50, but if you had done a transaction by account report, a full detail report out of Sage 50, you know, that old ACCPAC or the old Peachtree, you can literally just throw it at it, it'll know what to do, it'll pack it in to the system and start doing the analysis. So, we get that ability to really refine and look through all of the data.
So let's think about what we've just gone through, right? We've gone through very quickly getting auditors assigned and getting seniors assigned to help me manage my organization. I've loaded in some data. I've managed to fix all of the settings that I want. Again, if you've got a common practice based on your guidelines you can now replicate that across the entire firm, every single engagement, every single engagement by industry, by ERP or financial management system, etc, you get the ability to start actually harnessing, you know, your collective as well as artificial intelligence to make your audit plan a little bit better.
Now, while this is going through I'm just going to highlight a couple of quick things here in the product that you'll be able to start understanding. So again, we talked about access management, that you'll have that ability to see who's the manager within this, who's an auditor within this, they have different roles and responsibilities, we'll get into that because obviously when you're planning the audit you're going to want to give them things to do.
And then it also has the ability to...you have the ability to look at the various weightings that are being used on what we call control points. Now, control points are essentially those triggers and those pieces of interesting facts about the data itself. And we're looking for very interesting things that are happening within your dataset. You'll notice it's finished, I'll come back and look at the actual results in a second.
But this essentially just gives you sort of the blueprint of what has been assessed within the data. So that's one of the very first things that the system does, is it gathers all the data, it runs it through this pipeline, it's looking for things like manual entries, things whether passes or fails or two-digit Benford's law statistical test, it's looking for rare flows, it's looking against our expert score, supervised machine learning stuff.
And it's all these things all in an effort to put you into the driver's seat at looking where the potential risk is in the data. So you'll see that I've now got this data in here, and it's given me a really great snapshot of what's happening. So, we've picked an interim analysis, so it's only picked obviously the first nine months of the year, that's what I see here in my time horizon, and it's started to assess roughly, in those first nine months, where it sees the potential risk or the potential areas you need to focus on based on the data that it sees.
So it sees 17 transactions, there are almost 2,000,000 at medium risk. It sees just over 7.2 in low risk, and nothing in the high risk. Now, because this is an interim, you may not just take that as a grain of salt and build your entire plan based on there being no high risk. What you probably want to do is actually start looking at, "Well, where are the potential areas that I need to look at?"
So down here at the bottom, we start giving you a bit of a assessment of the risk by the overall account grouping. So my asset is relatively dark green here, as I radiate out obviously current assets, AR, these are all this nice dark, rich green. This means that when I'm thinking about my plan and I'm thinking about how I'm going to get going with this client, I may not have to spend as much time as I would have done if I wasn't using data analytics as part of that planning process.
What it's done is it's looked at all the various things that could happen to the transactions hitting the receivables and it's now given me that. Now, when we do the larger engagement, when we finish the end of the year, which we'll do a rollover here in a second, you'll actually be able to see how we can start bringing in some of these other pieces of data for receivables to just validate that that is in fact, low risk.
But let's look here, we've got inventories. Inventory of raw materials looks to be an area going a little bit, you know, towards that lime green. I can radiate over and see equity needs a bit of a review. And if I look at my expense line there's a couple of things happening here in rent and utilities that, you know, might be fairly interesting for me to review.
On the whole, maybe this jibes with all the qualitative analysis you've already done in terms of understanding what's happening in that client of yours that plumbing construction company. So, right away you get to see very quickly where the system, from an unbiased perspective associates risk. It does this on general ledger as an example of type of analysis we do, it does this across 28 different control points, the majority of which are an element of unsupervised or supervised machine learning, as well as your common rules that you're expecting to look for.
You know, it's pulling out things like anything that was a weekend post, there's 192 of them, anything that was manual entry, those types of things will be available here on the system for you to assess that risk. We also start giving you a trend analysis of how things are going, obviously if you add more and more years'worth of data, and I'll show this with more data in a second, you'll be able to see that flow of what's going on within the data.
So let's just pop back to our slides for a second and talk about a little of what we just saw. So, when we think about the audit planning process, you know, we need to very quickly set it up, we need to understand the type of data that's available, and we need to move from point-to-point. The next thing we did is we actually had that data load so quick that I didn't even get a chance to go through all the functions, and in fact, areas of focus or areas of risk.
Now again, this is really important because what we're really trying to do is ensure that when you're thinking about the audit plan, when you're thinking about where you're going to invest that minimal amount of time, 47% of you on the line today in the poll said there's not enough time, that's one of the biggest challenges with performing an audit plan and getting ready for the audit, there's just not enough time.
The other side of you, 33% of you, said there's too much data. So how do we very quickly get to this point of bridging those things together? How do we start working on building the plan, just assess the risk, you know, whether you look at ISA 315 and the whole concept of risk management, and how you mitigate that as part of your audit plan, we give that to you in seconds, we start showing you exactly where the system is identifying risks.
And this is like having, like think about it, it's like having tens or hundreds of bookkeeping professionals and auditors sifting through the data in seconds. It went through a couple of thousand transactions in under a few minutes, and provided us line of sight to where it felt there was some anomalies, where it felt there might be some potential errors for me to focus on.
And that's the type of thing that we want to do even before we get going too far into the audit. So, just a couple of little points there, right? We want to make sure that we're in a good place, we have the ability to go down and get to lots more data. Now what I'm going to do at this point is, I'm just going to pause again, and I'm going to do a quick little poll.
And I'm going to talk about, you know, the types of technologies you use today because that will influence some of the things that we again talk about here in a second. And while we're doing that, I'm going to look at the question and answer and chat window, and just see if anyone's got some interesting questions for us. So again, I'm going to launch the poll for you here, it will take just a second to show up on your screen.
It should show up and show you, what are you using for your tools? How are you actually getting to the plan? Are you using a computer-assisted audit tool? Are you using some form of AI already? Are you using any technology at all? So let's see what happens in just a couple of seconds. While we're doing that, I'm just going to go into the question and answer and just verify.
So, bear with us just for a second as that polling is in progress, we will get there in just a second. So, the question and answer looks relatively free of things. And I think we're good. If you have any questions, throw them up there in the question room, throw them up there in the chat and we'll get right back to you.
I've got a couple of folks on standby just helping us out. All right, I'm going to close off this poll, just let me get back to here. We've got all of you here voting. So, I was going to say, anyone want to guess? But you're all on mute. Unsurprising to me, and hopefully not a huge surprise to you, Excel. So Excel is by far our biggest-used technology as it relates to audit planning.
And I think that makes sense, we went from a time period where we were all looking at ledger books and those types of, sort of, manual offerings, we got into this world of having great spreadsheet tools in the late '80s, early '90s. And what do most accounting and business professionals learn and economics professionals learn in school? They learn Excel.
They use the spreadsheet tool, right to really analyze and access the data. What we're here to say is that there might be just a little bit more that you can do. So, we're going to close the poll here, just give me one second. Go back to the screen. I'll just verify that you guys can see the next screen.
And let's talk about now the next part of your audit plan. So, we're going to do this by actually going through the process. So we've done a little bit of an assessment, maybe we've built a small audit plan for the interim, we had some folks going look at some of the transactions, now let's actually do the rollover and let's take a look and see how the results can then be put into a more formalized plan to get the team fully engaged and fully enabled to go and actually work the audit.
So we're going to go back to our friendly tool here, we're going to go right back to the beginning of the data, inside the engagement. We're going to go ahead and roll this data over by starting a new analysis. We'll do GL still just for a second.
We are going to come back and talk about payables and receivables, but let's just go ahead and do the full year. Let's load in that full 12 months plus a few extra weeks or months for us to actually look at the dataset. Let me grab this guy here. Obviously, grabbing data before the year started and after the year ended allows us to start working on things like cut-off testing, understanding a little bit more of the data.
So here we go, data's in, column still look great. Do we need to change any of the settings? Do we need to change any of the data perspectives? That's what we'll go through here in a couple of quick steps. Keep the materiality of $50,000. Nothing else came from our interim work. And now it's going to go ahead and tell me, "Ooh, we have a challenge, there's 16 new accounts unverified."
Let's go take a look at that. So why is this important? This is really important because this is exactly what happens in your world today. If you were using Excel if or you were using a computer-assisted audit tool that can't very quickly understand what happened and what's happening, where there's deltas, you're going to find yourself spending a lot of energy trying to figure out why something's not balanced, why something isn't reported in this way.
We very quickly can see all those that are unverified, we're happy that they're all part of the accounts receivable bucket. Now we're ready to go and start the analysis. So it's now going to go through, and it's going to do a lot of that work. While this is doing some of the work, so it's not doing all the cutover, it's now showing me all the balances.
We're just going to go back to my beginning and show you some work that's already been done. So we're going to grab the MindBridge University data, we're going to take a look at the engagement, and we're going to walk through some of the other capabilities that the solution has. So as you see, I'm using the exact same dataset. So this isn't…you know, pull your over your eyes, it's just I'm trying to get to some of the pieces faster, I've gone ahead and loaded up my previous years, so I've got four years of data in there.
And I've also got accounts payable and accounts receivable. So now, to load all of this data probably takes me another 15 minutes or so. I don't want to spend your time doing that. I just want to show you what we can do to really start creating that high-powered workload for the team, and really round up the audit plan as we start getting ready to do the field work. So that same data, now with a full year, gives me a slightly different view of risk.
I now, in fact, have some transactions applied, I actually have a little bit more medium risk to deal with, and the dollars are sizeable. And as you can remember, the first part of the year wasn't so bad, right? If we go back across time this is exactly as we saw it in the interim. But look at where some of the risk is happening, at the year cutover, we can get even more granular, so I'm going to look at this time sequence across the cutover and see exactly where those risk points happen.
And we can start seeing the data has shifted just a little bit. If you remember, inventory of raw materials was trending a little bit higher risk, that likely means that since it's now in the dark green, in this richer green, we've had some changes happen to the books that make it less of an anomaly.
But what's increased in anomalous behavior or potentially something to focus on is the equipment over here. And then if you remember, expenses did look to be a little bit odd and it looks like we still have a little bit of an issue. But because the size has gotten bigger, it also means the monetary flow is a little bit bigger there, so we probably want to spend some time there.
So again, the idea here is that we very quickly see all of that information, we can see that we're balanced, total debits and credits, we can see what type of control points. So again, put this back in the context of what we're trying to discuss here today, we want to talk very specifically about ensuring that we are able to plan in an appropriate way, so we can see very quickly the risk breakdown by volume of transaction, stratification of transaction based on the control points that have been triggered.
We'll talk more about that in a second. We can see it by timeframe, and we can see it by actual account hierarchical view. Right in the same data as we talked about before, right? We can start seeing the data now trended, so now that I've got all five years of data in here I can start looking at all of these pieces, I can see where the risk potentially is.
I can actually go in and see if there's any other pieces, I see can a spike here in a write-off, that's probably not fantastic, Days expenses in AP, looking like it's trended up, it's the end of the year, but then down at the end of March that might be something we need to look at. But we very quickly can see those trends. And of course, it wouldn't be complete in terms of looking at it from a planning perspective unless we could understand if there's anything out of sequence.
So, in this case, there's three accounts that weren't included in the closing balance, why is that? Right? We want to look at those types of anomalies. We've got five accounts that aren't matching in the account groupings, but it looks like all of this transaction is balanced, so that's probably a good thing. On the other side, all the reports, I can look at the roll forward report, so I can now go ahead and see what changed.
And so think about this, if you weren't doing any interim today as a part of your planning and as part of your execution, you now have the ability to do that interim with very little effort. Load the data, process through, look at some of the detail transactions, we'll get into that a second. But then the minute you roll over, it's going to reassess, see if anything was backdated, see if anything was altered or changed or removed from that interim, which would highlight another control breach that you want to investigate.
Again, part of the planning process is getting you ready to execute, and do all the field testing, all the field work with your client. That's what we want to make sure that you actually have the opportunity to do. So if you think about all the different pieces of information that you're seeing here on the screen, I very quickly get the ability to download any of these reports, get to see whether I'm in balance, see if there's anything unbalanced, and all that data is available for me to now actually use and leverage as part of my audit plan.
Now what's great about what's happening around the world in terms of standards because that always comes up a little bit, is all of this information is part of that enrichment of the data, it allows me to very quickly see where that risk is. And if you think about what the IWASB has just put out, that actually I think the close for feedback has just happened, but in ISA 315 around risk management they're talking about leveraging all of this that you're looking at in things like your audit evidence, which means everything you planned, if you do an interim you're going to be able to take a lot of this right into the actual evidence to start balancing your work effort across this audit.
Again, being better planned, getting you back to less time, right, when you gave me the sense that time is one of the biggest issues, data is the next biggest issue. So let's spend a little bit of time on data and let's actually dig in, because even though this isn't necessarily exclusively about audit planning, I think it's really important to understand what happened so that you can understand why this is so important for you as you think about the plan.
So, what happened? Well, the Ai Auditor decided it would look at every single entry and look at every single transaction, and in the case of General Ledger it looked across all of our core typical rules and thoughts of what we want to look for, manual entries, weekend posts, last three digits in zeros and nines, etc, it's added in statistical models like Benford's Law, and it's added machine learning capabilities to really refine a focusing score.
We call it a risk score because it's a little bit easier way to really define what this is. So when you think about using your Excel or using your current CAT tool, imagine getting 100% of every transaction and entry assessed in minutes and have it completely filtered and ready for your review.
And what does that mean? Well, as I said, every single bit of the transactions are reviewed and risked. Now, I'm not surprised to see these big etree numbers, it pops two percenters of my transactions being identified as a slightly higher risk. But let's look at this other one, which isn't huge, but it's also not, you know, small.
And it's got a focusing score of 53, which in our case means it's tipped the balance, it's more of a high risk potential, you should probably spend some time on it. And when I open up that transaction it shows me the entry, in this case it's a fairly small transaction, five entries, but you know, I'm a little bit confused by this entry at the same time, looks like the customers cancelled the job.
Okay, I understand that and appreciate that, but just from an eyeball perspective I'm not seeing some other accounts that maybe I'm used to seeing in here, you know, if I'm doing sales and I've got things like roofing and hardware and lumber, in my mind. I'm thinking, "Well, why is my cost of sales or my raw materials not fitting in this transaction?" Maybe I want to look for that in a second.
But what the system does is it actually it tells you and articulates where it feels the challenges are. So we've got three parts in this transaction that end in zeros, they happen to be all sales accounts, probably not as uncommon as we think. So maybe that's not all that relevant and interesting. The whole thing was a manual entry, it's of course a material value, so it's above that $50,000. So these two account lines, everything else is fine.
But imagine that, we now actually go and pinpoint exact entries that are above that materiality threshold versus the transaction as a whole and just do it as a matter of course. We also take time to try to do the math and figure out, "Hey, has this transaction been reversed in any way, shape or form?" Yes it has. By the way, here's the journal entry number that it corresponds to 2949.
So I could click on that hyperlink, go and look at the details that I need to to validate whether this is interesting or not. And then of course, I get the ability to go and look at some of the more high-powered or high-caliber pieces of algorithms. So I can see that it's a rare flow. So what does that kind of mean?
Well, it means that this transaction is quite rare. Think of all the businesses that you audit or if you're in a single business, let's just think of all the departments, all the countries, all the regions in the world that you audit. Imagine having a system that can look at all of that data, right, I think that was the second highest. First was time, second was actual having too much data. Imagine if you could go through every single transaction, every single entry, and understand specifically for that company, for that business unit, for that geography, what's rare?
That would be hard to do unless you have lots and lots of time. And it may not even be possible based on human cognitive capacity for dimensionality. What this system does, is it looks at all the factors and says, "Hey, this one? Really hot, right?" Ninety-two out of a score from zero to 100 of potent of rareness. And it's talking about how the actual data is represented, if I click on that, it's showing me that the whole transaction there's just something about again, that data flow, that monetary flow that just seems a bit off.
One of these fails Benford's, a couple of them are unusual amounts. And our expert score, again, you've now got a system that is looking at every bit of data but it's doing such with artificial intelligence techniques like supervised learning, where a human expert, multiple of them by the way, have helped us actually define what types of things to look at, and we algorithmically go after every single bit of your data and try to find out what's happening with that data.
So again, this one's not quite as high as the rare flow as an indicator but now it's there. So now let's think about what that means as it relates to our audit plan. So I wanted to at any point in time, I'd have the opportunity to very quickly go in and look for my outliers, it's one of the things you've got to do in an audit.
So let's go and look at how we built the plan, right? So we're going to go into the data table as a whole, and we're going to say something really, really simple that says, I want to go and look at anything that is an outlier. Well, it's clearly stated as one of our control points. I wouldn't solely just use that one. But if I take outlier I go from my 12,000 entries, I now have 264 to look at.
That's fantastic, what a focusing agent. I now know there's only 264 entries that are considered outliers for this business. And I see them ranked from high to low, and I can see where they are. Now the thing that's interesting about this is I can also go into any one of those and further refine my view, and be able to get to a point where I see exactly what's in there.
So I can go into this one here. Ooh, happens on what most people would classify as a data that isn't worked. I can see a little bit of the details, but I may want to dig in a little bit further, and again see all of those pieces. What that allows me to do very quickly, is refine what I might need to work on. Now what I want to do, is I want to take a look at that data, select it all, and put it into my audit plan.
So how would I do that in the easiest fashion possible? Well, the easiest way to go through that is, I'm just going to reset all of my filters here. I'm actually just going to go and ask a question. But I want to refine it a little bit more. So I want to say, show me…let's start on the transactional level, transactions that are outliers, and experts fail over...I don't know, let's pick even higher than our materiality, let's say 75,000.
Are there any transactions that fit my request? No, there isn't. Fantastic. Let's take out that last little bit. Start again. Remember how in Google and other types of search engines you kind of have to retype things, well the system here is actually quite dynamic because what it's doing is it's actually building an expression very quickly.
It's translated expert score to be referred to as our expert score analysis, it's added the ability to make this an and or an or, and it allows me to switch between transactional view versus entry view right here in the screen. Once I see all of this data and I see that there's 12 items to pick from, I can get to select them all and add them to my plan, just through a couple of quick tasks.
Now again, think about how we do all this, My Plumbing Co. And this is outlier review. Remember what this is all about, planning and assessing, so we're going to go and ask Danielle, take the first pass at it, maybe she's our senior because she's one of the managers.
We'll give her a due date. Let's not be too nasty, let's give it to her for… She's coming up. Now Danielle will have all these transactions associated to her. So when you think about the plan and you think about what you're trying to do and accomplish, we very quickly give you the ability to sift through all of the data naturally, use expressions that are in your head, find the filtered list, add them to the audit plan, and at the end of the day part of that plan is to help manage how you get there.
So we have lots of other webinar resources that you can go and look at for, you know, what the Ai Auditor is all about. You can go and understand what some of these other elements are. But essentially, you can see all the folks that are working on our engagement, when they were assigned that part of the engagement, and you'll notice that Mike's got quite a few things here that are all from the payable sub-ledger.
We can go and look at things that are open, versus closed, versus who they're assigned to within our team. We can go and make sure that we just look at the things that are resolved as an audit leader, right? That might be what you need to do, right, that oversight function is part of the audit plan. Okay, let's go and see, were there any comments made?
Did anything happen to this transaction? What is the history of the transaction? All very quickly and easily looked at inside our audit plan. Now of course, you can take these out and put them back into engagement management tool, working papers-type solution without any problem, all of this data comes with it. Everything you see on the screen can very quickly come out and be looked at in the information, right?
Might have some information that needs to be dealt with. All of this is available in your audit plan, you can see everything, you can see what's closed versus not closed, how many transactions for each transactional bucket, very, very quickly. Now just in the essence of time, let's talk a little bit about things like receivables, payables.
Again, planning your engagement, how do we get to a view of the data? Who do we need to look at? Who are more likely customers in this case on receivables that I should go and spend time on? We give you all the right visualization to actually go in and see the details and see the data. So I'm not going to look at all these nice high green ones in their entirety, but I'm probably going to go look at Futherton because it's a deep rich red, but it's also fairly significant in size.
I can do this on a customer viewpoint. I can do it from an internal controls perspective and look at employee ID 5423. The fact that in the case of Futherton and this, they're both the biggest and the highest red, and they're relatively the same size, I may want to go into those details.
So let me look at those, and let me also look at which customers, right, they got the customer number, which ones are actually impacted? And you can see, for some reason, this employee seems to do a lot of work on receivables for this client. It was in deep, rich red.
There are some potential challenges with them. What are the challenges? Well, things done on a weekend, they're part of the top 2% of the ledger, they've got a rare transactional flow, the money looks a little bit odd. You know, if there are a flurry of activity by that customer. These are the types of things that you can very quickly look at, see, and plan.
Now we're going to head back into the PowerPoint just to, kind of, do some wrap-up, have a little bit more about closing out on the plan. I've got one more poll for you to give me some assistance on here, I'm just going to set it up. And of course, as you can imagine it's a little bit leading, I want to understand where you're heads are at.
We spent a little bit of time on how we do things on the plan, but we haven't really talked about probably the core piece, which is the AI. And I think that's very much on purpose, it's on purpose because the AI shouldn't be the prominent thing. AI is what's going to enable you to plan more effectively, understand the risks, and actually start redefining things like your reasonable assurance.
So we'll give that a couple of seconds. I just need a few more votes to get us up to that plus 75% range before I can close it. It's really quite interesting to see where everyone's heads are at. This time question always comes up. All right. And I'm sort of again leading you into a point of view, but I'm really liking the result.
Let me just give you a second more, just take a quick look at some of the question and answer, see if there's anything that pops out. All right, I'll be able to have a little bit of time for Q&A, I've got a few questions in the queue here which I'll line up. We're going to go ahead and close that poll. I'll share the results.
This may surprise some of you. I'm quite surprised. I'm also quite happy. Quite a lot of you are looking to implement this in the next few months. A few of you, so more than 50% in the next 12 months. That's great, almost 50%. That's amazing.
And for those of you in that 15-month camp, those who are fitting in 1 to 2 years, also fantastic. And I love the fact that there are people that still aren't quite across the chasm, they're still thinking, "Maybe not AI for me." That's okay, all good news, all good things to see. So let's talk about finishing this off with the AI and finishing off with why I didn't touch on it quite a strongly as we teased you in the header for this event.
I think audit planning is inherently a challenge. You've told me that Excel is the predominant capability that you're using or tool that you're using along with a little bit of you, a few of you using CAP tools. So inherently you're probably not looking at 100% of the data, you're probably not able to assess everything.
And combine that with the fact that you told me that there's too much data, and that you don't have enough time to plan effectively. Let's start thinking about how we can redefine this reasonable assurance. Let's think about how we can actually work together technology with professional for a slightly better outcome. And when I say slightly better, I mean significantly better when it comes to your confidence interval, your overall sense of assurance, making sure that you've actually focused on everything your client might be interested in as you get ready for the plan.
So what we do is we try to redefine reasonable assurance. We do that leveraging machine learning and AI technology, and it's essentially working together with you to assess all of that data, be able to understand it from an interim perspective, rollover perspective, what changed in your risk profile, all those pieces, being able to have one consistent platform that anyone can use.
Then I spent a lot of time on the tool because I think this is a big piece of it, where else can you get a solution that's going to partner AI and machine learning and all those fancy statistical algorithms with every single one of your audit team members? See how easy it was for me to, you know, grab things as Daniele or the manager Stefan and the rest of the team?
We very quickly can get you from having that analytics function, in a box sitting in someone's desk that only a few get to use, and we can now balance across your entire team. If you looked at the way that we build audit plans dynamically by saying, "Hey, I want to go look at outliers. I want to go pick revenue. I want to…" Whatever the test is that you have to actually accomplish, you very quickly can get all that data, you can assign it to the right staff member.
You can know that they have all the information about those transactions and those entries at the click of a mouse, and that makes it way more impactful for planning your time, planning the audit and its overall risk because we can now start seeing some of the areas that are really important. All of this that you've seen today is much greater and you're going to have to spend a little bit more time with us, but we really do believe that Ai Auditor is going to transform the way we think about financial auditing.
We're going to be going deeper and deeper into your datasets. We're going to be looking at flows of information that is not customary in the audit profession today. But with that we're going to blend all those tests and standards that you've got to abide by, and get it done in an effective way. And probably the biggest aha moment why I didn't show too much of the actual raw, technical, data sciency, AI-y stuff is because we firmly believe that AIs won't replace auditors or CPAs.
What we believe is that CPAs that use AI will replace those that don't. So when you think about the firm of the future, when you think about where you're going to be, when you think about the profession, everything from audit plans, everything from engaging with the data and information to better do that plan, having consistency across all of your team, you need to start looking at tools that might be able to give you that harnessing.
And we believe AI is just one of the components that you need to start embracing and making it easier for your team to work as a team in a decentralized way in terms of getting all the information out to all the right and relevant people, with the right roles in the system, right, all those things I've already shared with you and allow you to really get in there and dig in the data.
So there are some next steps that I'm going to throw up on the screen, things for you to think about. And while you do that, I'm just going to pop over to the question and answer and start answering some of these questions for you. So we have a question in the Q&A around how do results map into management assertions, and what would the auditor need… Sorry, the auditor would need that insight to assure them whether scope covered by AI is enough, and that's absolutely spot on.
So I appreciate that question, how do they map into the assertion? A lot of what's happening in the AI space and in AI in accounting, is about taking everyone on a journey. And the starting point for you as practitioners is to actually work together with the technology and with the change leadership to actually get to that point where we map it into how the firm wants to create the view of that assertion.
So the assertion could be, you know, we had a strong sales growth in this region. Great, go and validate that very quickly, pick the region, pick revenue, see all the sales entries, see what it looks like. There might be other assertions that are relating to reclasses and things like that.
You can very quickly start seeing all of the data in one place across all the different factors, and you should be able to then knock those directly into what you're doing. So the documentation aspect is actually kind of a follow on. So this is kind of interesting. A follow-on question from a different user, but I think it's a follow on, because it's, how do we see the change in the documentation aspect of audit work?
And when you think about audit plans, I mean, a large portion of that is documenting what you're going to do, it's like the Sesame Street model, tell show tell, write down what you're going to do, go do the work, show people what you did. Those are great questions Danielle, I really love it. So, I really do think that it will change drastically, because the level of information that you're going to be able to assess very, very quickly, some of those high-end visualizations that you saw, the, you know, customer analysis, by user analysis, the risk profile of accounts, I think what's going to happen is, you're going to be able to leverage those types of elements as part of your documentation.
I think some of the documentation will be based on, you know, sort of, changing that perspective of why we assess the risk a certain way. And so, when you think about the control points which I touched on, I went into the engagement settings really early on in the tool, but having a standardized view of that across your firm, across your team, is kind of what's there.
So you can take this all out, put it into working papers, it's all exportable, easy to do at the right level of detail. But I think you can work more and more so in the tools themselves. So the question of risk comes up quite a lot, and I actually on purpose, so the question is, you know what is high? What is medium?
What is low risk? And what type of criteria is considered? Every firm will sort of take a different slant on the criteria. So, inside the service, inside Ai Auditor you actually have the ability to create sort of firm-like standards on how you weight certain criteria. And that's really important because part of this whole audit thing is about yourselves as practitioners being able to use and actually, you know, be standing behind your professional judgment.
So, every single team is going to have a slightly different view on this. But if you look at the high-medium risk, it basically means that on the weightings that you've decided to use, anything that triggers a score over 50 goes in high, anything between 30 and 50 goes in medium, and anything underneath that goes low. So essentially, with your team understanding those control-point paradigms that we use in AP, AR, and general ledger analysis, we give you the opportunity to get there.
So Tom, glad you're interested. How do we actually start using this? How do we try it out? Well, the biggest thing is, is obviously get in touch with us. We have quite a few different programs. We've got a jumpstart program which is all about trying to help you understand what the tool does.
I think back to the last question around what .is high, medium, and low, thank you Joyce for that. By the way, again I didn't thank you, do appreciate that. It goes hand in hand I think with Tom's question, is, our jumpstart program is probably the best place to get started, it's a way for you to start working on this either in your firm if you're an internal auditor, or work within a couple of your engagements that you're about to do.
And it's actually blending in change management, which is really a critical piece. Right? I talked a little bit about the journey. The journey is important. It's something that isn't one and done. You're not going to buy a tool that says AI on it and have this really nice staff, and tomorrow be able to say that you're an AI-enabled firm. So, the jumpstart program really takes you through that.
It helps you understand what we do, how we do it in live engagements together. So that's pretty awesome and fantastic. So, I think just based on timing we're right up at the hour mark. I really want to appreciate the attentiveness and the engagement, lots of great questions. If there is anything that I didn't answer that you specifically asked, the team here over at MindBridge will send you a quick note.
There will be a recording available, you should get an email directly in your inbox not to too long after you've finished hearing my voice so you can replay it and share it with your friends. Listen, this is a really big journey, coming together, understanding how AI is going to impact me, starting with the audit plan all the way through to your engagement.
Listen, it's going to be fun, exciting. Get onboard, rally your team, this is going to be a starting point. And if you've read anything about the fourth industrial revolution recently, Klaus Schwab is putting a stake in the ground, the chairman of the World Economic Forum in 2016 said that "this is one area audit," it was I think number 14 out of his list of 25, AI, robotics process automation, robots will change audit.
We're on the forefront, we're early, we're going to get there together. It's a great journey, hopefully you can join us on it. Reach out to us if you need to. Have an awesome morning, afternoon or evening. Thanks a lot everyone.
MindBridge Ai Technology, Mission and Product Overview
Learn about our MindBridge Ai technology, why we created it and how we take a hybrid approach cross-correlating business rules, statistical methods and machine learning to deliver better financial analysis.
Interview with Becky Shields of Kingston Smith
Kingston Smith, a top 20 UK chartered accountancy and audit firm is leveraging MindBridge’s Ai Auditor. Becky Shields talks about the value that Ai Auditor brings to the firm.
How AI is Disrupting Audit and Introduction to Ai Auditor
Learn about the history of AI and why it is emerging now and how AI is currently disrupting audit, followed by a demonstration of the Ai Auditor platform.