Select Page

Using AI to assess financial risk in times of crisis

Stuart Cobbe, Director of Growth, Europe
With the thought of revenue losses, recession, and business survival on everyone’s minds, audit practices and finance organizations are redefining how they assess financial risk. By using the power of artificial intelligence and data analytics, we can improve our understanding of where risk occurs in the business and use financial data to identify critical trends in revenue and expenditures.
How can your firm use AI to navigate financial crisis?
Join Stuart Cobbe, Director of Growth and data analytics expert, for this interactive webinar on how AI is used to help auditors and finance organizations assess their financial position.
In this webinar, you will learn:
  1. How to use data-driven approaches to understand cashflow
  2. How to demonstrate revenue and expenditure lines
  3. How to identify those accounts that are risky in your receivables

This webinar includes a Q&A session to answer any questions you have.

You will walk away with a better understanding of how AI-based analytics and visualizations deliver financial insights and help mitigate risk.

Who should attend
  • CPAs and CAs interested in using AI for audit
  • CFOs, controllers, and internal auditors interested in using AI for financial data analysis
  • MindBridge Ai Auditor users who are analyzing financial data

Video Transcript

Hi everybody, thanks very much for joining me today. A little bit about me, I am a chartered accountant by trade. I trained in the mid-tier firm in the UK where I had the good opportunity to help stars and define their data analytic strategy.  Part of which was exploring and implementing artificial intelligence within audit as well as in other areas of their business and there were a few other kind of data analytics projects that we undertook there, following that I started a financial statements reporting tool which has recently been acquired by MindBridge and now I am director of growth for MindBridge.

So really focusing on the end-to-end process making sure that Auditors and accountants understand the techniques the tools such as ours apply and making sure that everybody has the right skill sets and understand how to see value through these kinds of techniques.

So, who are we as MindBridge? We believe that every organization deserves to understand and trust the data that is vital to their business. If you're an auditor, that means the financial data of your client likewise for advisors or members of corporate finance teams. We enable Auditors to uncover unknowns in their data to make decisions with confidence. So really a huge part of our goal is to make sure that those unknown unknowns are surfaced quickly and easily as part of our tool. And we have a wide variety of wrapping services that we offer to audit advisory corporate finance teams in the like to make sure that people can see value.

So, from an artificial intelligence perspective, I mean just a little bit of background, how did we get here? You know what has changed in the kind of modern environment? Why is there so much hype around artificial intelligence at the moment just for a little bit of history. I mean, the term artificial intelligence has been around for quite some time. I mean really, it's almost as old as computers themselves few of the key landmarks. I mean everybody knows about deep blue beating Garry Kasparov with what's called a good old-fashioned AI system, you know, those systems are really where a lot of the internal knowledge of chest systems, which is what this was is baked into the functioning of the algorithms themselves. A lot of the sort of decision making is baked in by a set of experts.  Things started to change really in the 2010s where the rise of neural nets led to a much broader application of artificial intelligence techniques, you know, you have the like of ImageNet which is very well-known image classification competition and more recently. We've seen other problems in other domains also be tackled by these machine learning algorithms a lot of them in your own net. Some of them are simpler. Some of them are just kind of linear regression or somewhere in between 2017.

For instance, we saw an algorithm perform in on a comfortable level to doctors at diagnosing cancer from scans. And recently we had a team by “deepmind” owned by Google create a bot that plays Starcraft, which is emerged as this grand challenge for artificial intelligence, which we'll talk about a little bit more. So, what are two of the big things that have really changed in order to enable this AI spring? I mean really, it's got two pillars. The first is a huge amount of data being captured and stored in a usable format and some of the competition's that I'll talk about over the next 10 minutes or so.

They're very reliant on well-organized very properly labeled data sets, which often businesses don't have sometimes they do it's a question that businesses have to ask themselves when they're looking to implement these kinds of techniques and the second pillar is really a huge increase in computational power. So, it's effectively throwing a huge amount of individual small decisions at a data set and allowing the computer to learn some of the patterns that are embedded within that data set, relationships between different data points or other patterns that might exist. So, where are we from an academic performance perspective. I mean this visualization is come from a really great resource called, I really encourage you to all check it out. If it's if this is an area of interest for you, they compare how artificial intelligence is being implemented. They look at how it's being scored on various academic benchmarks. And they also look at where the hot spots are globally, and this is a comparison of how AI algorithms within that image net classification is performing compared to humans and you can see that actually quite a few years ago algorithms were outperforming humans and really the task is just an image is being presented to the algorithm and the algorithm needs to tell what it is, you know, the classic example is this a cat or not or is it an animal or what have you.

 What we've seen recently over the last year or two years is some of the problems that traditionally were more difficult to required higher-level reasoning also of started to fall. We've also seen machines and algorithm starting to outperform humans on them. This example is the general language understanding evaluation benchmark, which is a collection of different language tasks that are posed of artificial intelligence algorithms. And really the goal here is to test the algorithms ability to draw conclusions from in some cases limited data sets. And again, you can see in early 2019 the test algorithms which tend to be put together by very highly professional teams or expensive teams for very focused objective.  But still when the challenge becomes more and more open-ended there's a gap between the algorithm’s performance and the human performance.

So, this is the visual question answering challenge which combines both image problems as well as language problems, effectively the machine is presented with a picture be it a boy hitting a baseball and the algorithm needs to describe what's going on in that photograph. So, it requires not just an understanding of not just perception skills and being able to understand what's in the picture but also language skills and being able to describe what's going on and whilst the gap exists. You can see that this the line of the algorithm is slowly approaching. So, we are getting closer and closer to human performance on this and you might also wonder why the human performance is so low and a big part of that is got to do with different opinions would humans answer these same questions as how that benchmark tends to be derived.  So, long story short, you know great progress is being made and a lot of this progress is on a lot of more complex challenges that require higher level reasoning.

So, one example of this that I mentioned earlier is StarCraft 2, deepmind released a bot that recently beat some of the world's best StarCraft 2 players and in the blog that they release that described their technique and describe some of the challenges that exist with Starcraft 2. They kind of listed five reasons why this represented a very good challenge for artificial systems. The first is game theory. So, anticipating other actions and understanding some of the incentives that exist before an action is taken. Imperfect information, in the game of Starcraft, you can only see a limited portion of the map. You have to make decisions based only on the information available to you. The decisions are made in real time often very quickly. There's a huge action space which means that there's a variety of different routes strategies and actions that can be performed at any one moment. And it also involves long-term planning. So, decisions made very early in a game impact the decisions that may need to be made later. And when I see these kinds of five core features of this as a great challenge for artificial intelligence systems, a lot of them are married in business. I mean, ultimately when I look at a business, it has many of these tenants and in many regards the challenges that businesses face is even more difficult on each of these five elements than the challenges that are faced in Starcraft.

So, I just wanted to show you a short clip together with a quote that came from one of those talk Starcraft players that was part of this training together with deep mind. You know, what I really wanted to impress was the complexity of the system that was built and the amount of effort that went into it both from the perspective of training the model as well as from discussing the results with the Starcraft community as a whole. The key takeaway for me here and one of the lessons that I learned when I was reading this was that this algorithm is defining new ways of playing the game. You know, this is TLO's quote, TLO being one of the top Starcraft players that helped to train the model you saying that this algorithm demonstrated strategies that he hadn't thought of before and it indicates that there may still be new ways of playing the game and he mentioned that this understanding came during the training phase so what I would really like to impress on you as that even when an algorithm is getting to its kind of final state or when it when it's not quite at the position where it outperforms humans there still a lot of useful insight and learning that can be generated from this.

We're also seeing these techniques being applied in the response to covid-19. There is a great initiative by folding at home, which is leveraging people's home computers in order to figure out access Paths of protein folding paths protein structures that allow vaccines and treatments for covid-19 to stop the spread of the virus or to help treat patients. So really just to reinforce, you know, even where the topic is really complex artificial intelligence can help us understand how the system operates and where to look effectively.

So, what does that mean from an AI from an audit in accountancy perspective? One of the kinds of key elements that we're seeing across the audit and accountancy space is that an increasing number of firms are either researching or deploying these techniques in multiple service arms be at audit or advisory or tax and so on. So, this is from an FRC report that was released in March 2020 and you can see that across the six firms that were that were surveyed almost all of them were either deploying artificial intelligence techniques, piloting them or researching them in a number of different ways be it machine learning natural language processing or predictive analytics. And from the FRC's perspective really these techniques are set up in such a way to improve audit quality and they're also very glad to see that's which makes us happy, you know pleased to see audit firms researching and piloting these techniques, which is great.

We're also seeing this more broadly in the world of business, you know, this is a report from McKinsey & Company in November 2019 where a number of businesses were surveyed about that the impact of artificial intelligence on their operations and on their business. One of the biggest areas was marketing and sales and second to that is the improvement of product and service development, which is really where MindBridge as a product, it’s ai auditor as a product sets are in this product and service development. So, it's improving the way that Auditors accounted some advisors are providing that service back to their own client. From the perspective of MindBridge, I mean one of the key elements that we set out to do was to tackle this problem of financial loss due to fraud and error. It actually was a topic that we were trying to broaden our scope away from a little bit not just about fraud. We are also useful at the planning stage of audits during the due diligence process for corporate finance engagements and m&a engagements, for instance.

What we were really trying to do in the early days was reduce the amount of financial loss that was going undetected which is somewhere between four trillion dollars give or take. I mean this data is a little bit out of date particularly now due to the coronavirus issues, which I'll touch on in a little bit. So really with the increased focus on business health business viability and the growing pressures on employees within companies. This is coming back into focus for us in quite a big way. Only briefly I mean what are some of the pillars of our products that we really rely on to do this? I mean the first is that we are detecting anomalies and financial risk using machine learning and AI we're doing that across a hundred percent of the data that's being loaded by doing silver redefining the way to gain reasonable assurance be that in an audit setting or elsewhere and then we do it all relatively quickly and when I say machine learning and artificial intelligence, how exactly do we apply it using Ai auditor or within our platform?

 We have an ensemble AI approach which means we take a variety of different techniques some of which have been around for quite some time such as you know Bedford’s law together with some of the most cutting-edge techniques that exist in the data science world and folding those best practices into the world of accountancy. So, looking for outliers, using techniques such as stochastic outlier selection, and it's quite important to note that a lot of our techniques are unsupervised in their nature which is important in a moment like this where businesses are changing rapidly and adapting quickly to the situation that's ahead of them. On top of this, we layer a number of visualizations and different ways for people to add value to the process and I'll show you what this looks like with some concrete examples of today's crisis some simulated data. We have together with some anonymize data from businesses that we've seen of what what's going on in today's crisis.

 So one of the topics that we've had quite a lot of discussions around internally and it's a theme that we're hearing from a lot of our customers as well as to what extent can I rely on these techniques in a moment of deep systemic change and there's this concept of concept draft that's quite important in this kind of moments. I mean, there's quite a lot of evidence that what's happening now is unprecedented. I mean, I think we've all seen the news of the jobless claims and how extraordinary it is compared to what's come before. There's a lot of echoes of the financial crisis of 2008. It's clear that for a lot of businesses. We're entering a totally new era of operating along. This will exist is yet to be seen but nevertheless there's this question over whether the conclusions that are being drawn from data prior to the coronavirus crisis is applicable to the data that's happening now.  And I'm going to try and talk about some of the techniques that we use in the tool and what our viewer, what my view is more specifically on these techniques.

 So, the first is a time series modeling technique called a rhema and we actually use sarima as opposed to rhema which means that we are a little bit better at incorporating seasonal trends in our time series forecasting. We use this in the trending dashboard as part of AI auditor. You'll see this in the tool in a minute. But really what the goal here is to provide advisors with the way to understand how a business is performing compared to expectations. What I've got on screen at the moment is an example of a time series. So, this is Australian beer sales and you can see that there's a clear upwards trend in the data that the model was trained on the forecast is shown in a teal and the actual data following that is shown in a red and in this particular data set there was a system shock so effectively beer sales flattened out almost entirely and the model was unable to predict this and actually it's worth stating that in a system as complex as the world economy where you have a huge number of people operating and a huge number of variables and inputs is very difficult to predict these kinds of systems shocks and that's as true of the time series analysis that's in our tool as it is in any other kind of machine learning or data science driven approach that you may see elsewhere.

However, once the model is retrained with the shock included in it, you can see that the relationships become understood very quickly and the model adapts to the new norm quite effectively and it's also worth bearing in mind that some elements of the relationships between the data still existed and the model still understood appropriately even before this new data was incorporated and by that, I mean if I skip back to the prior slide the seasonality still exists after the system shock and that element of the relationship is still clear in the forecast and it's still clear in the forecast after the data including the shock has been included in the training data. So, you know the key takeaway here is really that these techniques are often poor at predicting that such a shock will happen as to be expected. But once the data with the shock is being included, you can get a lot of meaningful results from leveraging this kind of analysis and I'll tie this back into some you know, traditional ways of looking at businesses such as liquidity, expense ratios, and so on within the tool with a couple of examples.

 Another way that we really leverage machine learning and data science in the in the tool today is to identify fraudulent transactions or anomalous transactions within the data sets that we're looking at so general ledger, accounts payable and accounts receivable transactions being the core data sets that we're looking at today, and it's worth bearing in mind that. When you combine different methodologies in our case, it's a combination of rules based, and statistical machine learning based. It overcomes some of the weaknesses in single model systems and from our experience. It's also far more effective at finding those unusual or strange transactions than using any one of those methodologies on their own and really when I was talking about AI ensemble, you know, this is one of the kind of key tenants is that when you combine multiple signals you can on multiple different approaches. You create a more robust system overall, you know, this is critical systems design 101. It's my belief that many of the indicators of corporate risk will stay the same during a crisis like this, you know, ultimately the ways that people will be committing fraud be at corporate fraud or employee fraud are likely to be the same. This is been my gut sense. There hasn't been any research that I've seen to back this up, but I would vouch quite a lot of money on it.

So how do we tie this back into actually using AI for accountants’ advisors and auditors during a system shock? I mean, there's kind of three key questions that people really should be asking generally when leveraging these kinds of techniques in a business-critical format, but they become particularly important at a moment like this the first is data governance. So, understanding what data the model has been trained on does it include the shock period, you know does it include this Q1 of this year understanding the model’s strengths and weaknesses. So understanding that's sarima models tend to take on the trajectory of what existed in the in the period immediately preceding and having that business acumen understanding what is likely to change and what is not and also understanding how to drive value from the insights that you might get out of the tool. And all of this is kind of mirrored by what we're hearing from the ICA, this is Richard Anning saying that accountants will need to understand at a basic level how AI works and how to work with an AI Specialist, you know, they need to have more of that technology knowledge and skills than as perhaps be needed in the past. And this was March 2020. So, you know highly relevant highly up to date. So, what I'm going to do now is talk about some of the features within AI auditor and talk about some of the techniques that we've been exploring as a team of accountants and auditors within MindBridge to look for financial risk within simulated or real data sets. And so first to provide a bit of context, I mean three of the kind of key pillars the I like to tick off when I'm doing this kind of work.

The first is AI detection. So, finding that needle in the haystack. The second is AI prediction. So, understanding complex relationships be it time series or bit between transactions and the third is data visualization. Taking all of those insights, taking all of the enriched information that these techniques provide to people and making it digestible understandable so that so that you can drive business insight from it. So, what are some of the themes that we're hearing during this coronavirus crisis, I mean, the first is quite obvious “cash is king” you're hearing a lot of advice from accountancy firms rightly so. From major outlets about the need to focus on cash the need to understand what a business is cash position is like and what its expenditures are like the second is an expected uptick in corporate fraud so in most recessions, it uncovers a lot of bad corporate practices. It uncovers a lot of zombie firms, It uncovers instances where perhaps a company was operating just on the edge and bending the rules in order to stay alive. You know, the classic example would be Bernie Madoff and the third is remote monitoring. You know, how do individual organizations ensure that they’re employees are not misappropriating assets ensuring that the company is staying true to his its objectives and so on.

So, I saw something not that long ago, you know just literally minutes before this podcast saying that previous recessions show a direct correlation between a fall and economic output and a rise in fraud hence why we as an organization are really focusing on this message of financial risk and helping our clients understand how to leverage these techniques in order to uncover some of these elements. So firstly, focusing on the cash element going to be talking about three kind of main categories the first being liquidity the second being accounts receivable and the third being expenditures. So, what techniques can people leverage our technology today in order to understand their clients better? So, from a liquidity perspective, how can you use an advisor to get a deep understanding of a business quickly. We're going to focus on Revenue expenditure working capital ratios and current and quick ratios.

What I'm going to do is jump over to the tool quickly. What I'm looking at here is a simulated data set. This is covering a few years up to December 19 this data set. We've also simulated up to at April 2020 and then forward with some kind of reasonable looking kind of business performance. Shall we say, so regular-looking business, you know kind of quite a flat profile not a huge growth in transaction numbers. I'm just jump straight into the trending dashboard. So I'm really going to focus on revenue to begin with you can see the up to December 19 revenue is relatively flat as you would expect if I jump over to the same data set but including April, I mean a dip in revenue is to be expected. So, you know, the question is to what extent does this impact other areas of the business, you know, at least understanding the impact of revenue and expenditure is a good place to start, how is that company perhaps changing the way that its operating power its customers reacting and so on, so really we're going to focus on the use of these ratios.

One of the key questions for me is in cash receipts to revenue. So, for this particular business we can see this is effectively the cash receipts which is a debit to the bank account over the revenue values. We can see 2019 has a relatively flat profile and then in 2020 a big peak in March as the company trying to recover all of those debts relatively quickly. We also saw a decrease in revenue up above followed by a big fall in cash receipts to revenue as both metrics started to fall. Now tying this back to some of the sarima concepts that I was talking about earlier. We've got this expected ranges here. Now, the expected range is what we believed to be from our use of Arima, where we expect that value to fall for every month of the current year and that's our 95% confidence interval. So, it basically means we expected to fall within that range 95% of the time and there's a few things to take away from here.

 I mean, the first is that I guess unexpectedly the first few months of the quarter look quite unusual. The tool is flagging them as outliers. We also see quite a big expansion in that expected range which indicates that the algorithm is becoming much less certain about the direction that this is going to take. As the year progresses these kinds of techniques will help you identify instances where the client’s business will start to settle, you know, perhaps that confidence interval narrows or perhaps the actual results falls more tightly in line with that confidence interval.  

From the perspective of days expenses and accounts payable. So how long is the business taking to pay its accounts payable. We're seeing a consistent drop in that which is likely to do with the lag between expenses and the actual payment and from a day sales outstanding. We're also seeing a big drop. So as expected the business is taking longer to pay out. The business is taking longer to no. Sorry.  It is recovering its debts quicker from its customers. So, a big increase again in that kind of expected range.

From a pure liquidity perspective Also worth noting a couple of things here. I mean interestingly actually the current ratio and quick ratio is increasing despite a big decrease in in Revenue. This is likely to do with the fact that apologies This is likely to do with the fact that whilst what we've got here effectively is my cash and cash equivalents plus my accounts receivable divided by my current liabilities. And as this is a simulated furniture production company and it's got a large amount of manufacturing, the majority of its expenditure is actually on direct costs, which is something that it can control quite well. So, this current liability figure is falling quite quickly. And I can vouch that by going further up and viewing that.

On the straight year-on-year we can see this big dip in February March and April which indicates that the company's slowing of purchases slowing of purchases for direct cost is having quite a big effect on its cash position. Of course, switching over to my December 2020. I can watch as these various things progress over time and looking at revenue to expense. For instance, I can see that these few periods between January and April were highly unusual. But after that the trend stabilizes a little bit you can see that this this trendline falls well within the expected range which means the as the crisis progresses. I've got quite a powerful tool to quickly check which businesses are operating within some kind of norm and which businesses aren’t that's really the idea is being able to get to a quick understanding, quite a deep understanding relatively quickly across a portfolio of businesses.

So, from an account’s receivable perspective, there's a number of different ways to leverage the accounts receivable analysis again to get quite a deep understanding of what's going on. Really what we're looking for here are instances such as accounts receivable account. So, customer counts that are steadily growing or have been steadily growing over time which indicates that there's some issue with that customer account or some issue with the way that things are being recorded for it and so on.

So, jumping over to this account’s payable analysis for this particular data set again. You know, you've got this consistent increasing trend which at a quick glance is worthy of inspection, you know particularly in times where everybody will be looking to slow down their payments and this is kind of one of these key elements that perhaps what we used to assess, you know, a year-on-year balance check doesn't go to enough detail in order to really identify where the risk lies in these kinds of time. So being able to view it being able to view the visualization the data trend over a much larger period is quite important, it's quite critical and it gives a much deeper level of understanding. So, for instance in this particular dataset, we can go investigate which elements which individual suppliers are contributing to that slow steady increase and then we can query with that particular supplier or customer. Why is this happening, and we can help identify earlier where there may be issues in recovering debts from customers.

Crucially, a lot of our traditional value-add relating to identifying strange transactions is still relevant at this time. So, leveraging the accounts receivable specific control points is still a huge part of understanding where the risk lies within a particular company and understanding how to recover some of these elements or how to advise or what conclusion you should place on a going concern working paper.

So, the next example I am going to walk through is an expenditure example, really what we're looking here is a number of elements. I mean a lot of companies are at the moment assessing which bits of their outflow which bits of their expenditure they can tail off in order to maintain their cash position. One of the key questions is really what's the fixed cost base. What does that look like compared to the variable cost base? There's a lot of scenario planning going on at the moment. And really the objective here is to provide people with quite a deep quick data point that they can compare those scenarios, those budgets to.

So, as part of this trending dashboard going back to the general ledger data set what we've added is this fixed versus variable ratio, which is effectively the proportion of fixed costs versus the proportion of variable costs for this purpose. We're really only using quite a simple metric of which nominal codes represent fixed or flexible. But again, it's quite a strong way for you to vouch this against other businesses. And in this particular case when you're looking at the 2020 data in its entirety, you can see that the trend continues upward which would perhaps be a little bit worrying, you know, you would expect towards the end of the year perhaps that some of that variable some of that variable expenditure on direct costs would start to come back.

The last area of expenditure where the tool is really quite useful is an identifying strange expenditure profile. So, you know using some of the analytics that is in the tool taking a little bit further from excel to identify where and how often some of those large one-offs might happen. So in this particular example, we've got a spreadsheet which can provide to clients have kind of worked on with clients in the past that takes the general ledger all of that enriched data together with our control points and kind of goes a little bit further allows people to filter on perhaps transaction type or period allows them to get a medium level view.

Medium level of granularity into their businesses data and really the purpose here is to identify those unusual transactions understand how large they are and understand How likely they are to fall again. So, in this particular case, you know, we had one large transaction in February of 320,000 as an advisor as an auditor I might want to investigate that and understand right how likely is such an expenditure transaction to occur in 2020 and how big of an impact is it going to have on my business in order to find this what I've done is I've selected purchase invoices and I found any that have triggered outlier anomaly. So just one of those examples where we're really seeing people take the analytics and the data enrichment and the tool that one step further in order to answer their very specific question.

Another example from a different business is just strange expenditure profiles. So, you know, it's looking for elements of those visualizations where there's a big peak in previous periods or big trough and just getting some concept as to why that is happened and how likely that is to happen in 2020 and here we've got a number of clients who are doing this in a relatively kind of streamlined way and are able to answer these questions relatively effectively through the combination of good training on the tool and good data exports and understanding how some of these techniques work.

So, the next area that I really wanted to talk about was what's corporate fraud as we mentioned earlier in the economist article and a number of others there's this expectation the corporate for defraud is likely to increase or that were likely to see a big uptick in instances of it that have occurred in the past you know just that they happen to be kind of revealed what I'm going to do is talk you through some anonymous data sets that I've seen over the last couple of months The indicate for me businesses that I would like to investigate deeper, you know be an operating unit or an audit client here and advise your client or your will have you so really the key questions that I'm trying to ask when I'm doing this is leveraging the risk overview in the data table to identify where there's a weakness of an IT system like in the ERP or bookkeeping system for instance or indicators of management override or where management override is a very important part of preparing a business's financial numbers.  

So, this is one example this particular business.  This is the general ledger data, what we're looking at here is a count of transactions over time in the general ledger this actually balanced out. So, by checking the opening and the closing trial balance, we had assurance that this was exactly the data set that was making up the financial statement and what you can see is that there's a huge variance in dates that are being posted. You know, there were dates all the way back to 2005 or 2001 within this particular dataset to me that indicates that those manual entry going on around that date recognition and in a time like this a lot of incentives are going to be in place to bring forward revenue recognition, perhaps or miss stated expenditure in some way and by manipulating the dates it's potentially quite an easy way to do this.

When I leveraged this with some of the machine learning risk analytics that we have on the transactional level we can see that a lot of those risky transactions it in and around the year-end now these existed in January 2020 so perhaps not so, you know, not exactly coronavirus related for these particular transactions, but for this client, I would certainly want to be looking at where they are recording transactions in and around quarter end and what information their, perhaps putting to their board of putting to their shareholders, you know, it's this kind of profile. I would be looking for large upticks in risky transactions in and around period closure.

Going a little bit deeper into this particular client, you know using some of the analysis that we looked at before I filtered for credit notes. So what I was interested in knowing is are there any credit notes post year-end perhaps that might be impacting this business and indeed there was a relatively large medium risk credit note post year-end which may or may not be picked up by traditional approaches, but it's certainly interesting for me and it's certainly the kind of analysis that I would want to be doing for instance at March 30th close.

Just incidentally I saw a piece of research earlier that mentioned the most businesses are still reporting on time. So, a lot of this analytics can be leveraged kind of today. So from an unusual patterns perspective, I mean here what we're really looking for is strange profiles of revenue or expenditure recognition that again indicate that management override or management judgment is a huge part of revenue recognition for this particular business and also potentially indicators that they have historically relied on the period in and around the year-end to make revenue look like a particular way.

 Another example of a business that I saw recently big uptick in March, you know, a big uptick in the revenue recognition at March which for me was quite unusual particularly when there's kind of this slow small upward trend in the year for me, you know, I would be a little bit worried about the judgments going on although in some businesses revenue recognition transactions are large around the year end in this instance it's certainly something I'd be looking to investigate. On what basis are they making those judgments particularly where there's an incentive from VC private equity investors to show some kind of revenue in the quarter.

Looking at that same business by risk, you know when of the risky transactions being recognized so these are sales invoices. This isn't even manual journals and you can see a huge uptick in actual sales invoices both low, medium, and high risk being posted in that period. So, for me, this is unquestionably the kind of thing that I want to be looking at. You know, why is there this huge uptick of sales invoices being recognized and how has this happened? You know, despite the fact that the rest of the economy Is kind of on the down?

Really to kind of wrap up some of these elements, I mean in terms of building a competency and leveraging these kinds of techniques within your practice or within your service offering we talked quite a lot about change management. We talked quite a lot about skills and understanding and how these elements are vital to seeing value from these techniques really mirroring what Richard Anning head of the tech committee of the ICAW stated, you know building a competency involves combination of both capability and capacity. And when I talk about capability, you know, you'll notice that two of those two things, you know, foundational skills business skills. Those are skill sets that accountants, auditors, advisors will need to have in spades anyway to do their job well.

The key element that we're layering on top of that is data literacy understanding what the data represents its understanding what kind of questions can be asked of it and when to rely on it and adding to that the time to use these techniques the resources and the tools to make them effective. So, giving them the appropriate data sets giving access to your teams to the appropriate data sets and the right tooling to do these things effectively.

 I strongly believe that innovation happens at these intersections of ideas and concepts and cultures and for me, you know, the big space that I've been trying to straddle for the last few years is this intersection between data science and audit this intersection between data science and financial advisory. I do think that there's a huge amount of value and there's still a lot that we have to learn as a company and a lot that we have to learn as an industry. So, despite all of the craziness that's going on despite the bad news that you're seeing. I am truly excited about what the future holds for us as an industry, and I think that many of the skills that are required to see value from these techniques or skills that accountants and financial advisors have in spades anyway.

Just to bring some of the kind of more detailed elements on the data literacy side, it's an understanding of how the data is stored and ability to manipulate it and an awareness of potential issues some of which we've discussed today and an ability to tell a story from the data, so understanding what this means for this business and how to take it kind of a step further.  What further questions to ask so okay revenue is falling, but how does that impact my receipts how does that impact my accounts receivable balances and so on and turning that insight into action be it from an audit perspective or from a business advisory perspective.

So, I just wanted to finish with a quote from the Boston Consulting Group, they put out an article in April the rare the rise of the AI powered company in the post-crisis world. From their perspective, you know, they really believe that “winners will reinvent themselves by putting software data and AI at the core of their organizations”. And thanks everyone for joining as well and have a good day.