ISA 315 revised: What it means for risk assessment procedures, and data analytics

Two characters discuss the benefits of data analytics in light of ISA 315 revisions.

ISA 315 (revised) and Data Analytics: Risk assessment procedures reimagined

The revised standard has been published as of December 2020, and you might be wondering what impact it has on your firm’s risk assessment procedures and how you can address the requirements. There are many useful sources of information on the changes, notably the IAASB’s Introduction to ISA 315. IFAC also published a helpful flowchart for ISA 315 during the work programme, which walks through the various steps required to assess risk of material misstatement.

There are a number of improvements to the standard, including an enhanced focus on controls (particularly IT controls), stronger requirements on exercising professional scepticism and documentation, and considerations around the use of data analytics for risk assessment. The new standard comes into effect from 15th December 2021, so now is the time to start planning how you will address the changes in your audit. Below we discuss some key considerations on how analytics can support a strong risk assessment.

A chart explaining risk assessment and data analytics as part of the ISA 315 revision by IFAC.

Credit: https://www.ifac.org/system/files/publications/files/IAASB-Introduction-to-ISA-315.pdf

So how can data analytics support your risk assessment according to ISA 315? The areas identified above in red show the different procedures that can be supported by the use of these techniques. A key element of the revised standard is that this should be an iterative process conducted throughout the audit. This means using data analytics tools that can be easily refreshed with the latest information will better support this requirement than more traditional approaches.

Identifying risks of material misstatement at the financial statement level

Data analytics can support the risk assessment procedures laid out in ISA 315 by analysing previous and current accounting data to the financial statement level. This allows the auditor to see the material balances in the accounts, and if machine learning is applied, where the concentration of risky transactions lies. This is where the knowledge gained in the blue boxes above can be brought to bear. Comparing understanding gained through observation to the data is a powerful way to sense check and identify areas for further investigation.

Identifying risks of material misstatement at the assertion level

Specific analyses can target assertion risks and show where there are particular problems with an assertion. To do so effectively, several different analytics tests can be applied and combined to develop a good indicator of an assertion risk, for example accuracy. These can then be applied in an automatic way to give the auditor the information needed for their risk assessment.

Determine significant classes of transactions, account balances or disclosures (COTABD)

Combining assertion analytics with the ability to profile similar transactions can help auditors identify significant classes of transactions or balances. Analytics can help to produce similarity scores, but also to identify sets of transactions that are unusual. This can indicate previously unknown business processes that may require a separate assessment of their control environment.

Assess inherent risk by assessing likelihood and magnitude

Following identification of risk, the audit can guide their assessment by understanding the level of unusualness. Data analytics can provide finer grain evaluations of risk rather than simply risky or not. This can help support assessments aligned with the spectrum of inherent risk as defined in the standard.

Assess control risk

Data analytics such as process mining or automated testing of segregation of duties can help to inform or test control risk. These analytics can provide more comfort around the controls risk assessment and help to identify deviations in the control environment that require further examination.

Material but not significant COTABD

Where COTABD has been determined as material but not significant, recurring analytics can ensure that this assessment remains valid. Anomaly detection methods can be particularly helpful here, allowing the auditor to regularly check that nothing unusual has occurred since the initial assessment was undertaken.

Next Steps: ISA 315 and Data Analytics

Audit methodologies will need to reflect the revised workflow, with particular emphasis on the iterative nature of the risk assessment and ensuring that auditors are prompted to exercise professional scepticism and document it at every stage. Data analytics can help to ensure that the information used to continuously conduct risk assessment is timely, appropriate and relevant.

These improvements to the standard will result in a stronger audit approach and an advancement towards industry adaption data and analytics technologies. With AI audit software, accountants and auditors can gain deeper insights into their client’s financial data, in less time. Overall, the audit software can increase the efficiency of their processes, so they can focus on delivering better results, in time for the ISA 315 (revised) December 15th, 2021 deadline.Ā 

Leveraging AI for your substantive procedures for Accounts Receivable and Accounts Payable

abstract lines up showing ar-ap procedures

Artificial intelligence (AI) and machine learning (ML) technologies can streamline traditional audit procedures for Accounts Receivable (AR) and Accounts Payable (AP) in audits of financial statements.

This blog will consider applications of AI and ML technologies using the MindBridge platform for both substantive analytical procedures as well as detailed testing of specific items.

What does the MindBridge platform do?

MindBridge Ai Auditor, in addition to core general ledger analysis, includes dedicated AR and AP modules that automatically analyze subledger data and, without any scripting, provide high-value visualizations and transaction-level analysis of data.

These capabilities allow you to leverage subledger-level insights and anomalies as critical inputs to your audit procedures and identify risks of material misstatement.

How MindBridge empowers you to perform effective and substantive analytical procedures for AR and AP

Substantive analytical procedures can be a powerful complement to traditional sampling and external confirmations. That is, provided that the auditor is comfortable with the internal controls in place regarding purchasing and sales cycles and has validated the accuracy and completeness of the subledger data.

Trends and patterns

Ai Auditor allows you to visualize how monthly AR and AP balances or net monthly activity track over multiple years at customer vendor levels, and in aggregate. Consistent patterns in these trends in the face of consistent sales and purchasing patterns (respectively) may provide audit evidence that subledger information is not materially misstated.

Vendors and customers related to the entity subject to audit are flagged directly in the summary detail as well.

AP AR screenshot showing summary detail

Key performance indicators

Days Outstanding and Turnover Ratios are calculated at the customer and vendor level and are visualized on a monthly basis, allowing you to identify where there are periods of potential distress or deteriorating quality (e.g. is the volume of cash receipts slowing?). Similar to ending balances and activity, you are also able to compare certain customers or vendors against each other along the lines of these metrics to expose patterns of interest.

Key performance monthly indicator screenshot

Aging

Aging at the customer and vendor level is automatically calculated and captured across respective buckets of days outstanding (0-30 days, 31-60 days, etc.). Consistent breakdown in the relative proportion of these aging buckets across multiple years of subledgers may provide audit evidence that subledger information is not materially misstated at the balance sheet date.

For certain entries that are significantly aged or stale, you’re able to drill-in to all the transactions with a particular customer or vendor and ascertain which invoice(s) are contributing to those totals and whether they could be at risk of bad debt.

Risk of bad debt screenshot

How MindBridge streamlines detailed testing of AR & AP subledger data

Navigating and querying transactional level data via the Data Table in Ai Auditor is a powerful and effective way to explore and validate subledger activity.

Control Points, which are various statistical, rules-based, and machine learning tests, are run against every transaction. The results are summarized on a dashboard that supports interactions like filtering and drill-through.

Dashboard based on control point screenshot

Combining the query building capabilities of the Data Table with Control Point tests, you can efficiently identify relevant populations for sampling and have selections for external confirmation requests or alternative procedures testing (like subsequent receipts, for example) automatically identified on a risk-stratified basis. These selections can then be exported to Excel in one clickĀ  to populate confirmation requests and/or to be included in supporting documentation.

The results of the transactional risk analysis may also be of particular interest to large entities and small businesses alike to provide insight into where there may be process improvements or gaps to consider in internal controls.

Take the first step towards AI-driven audit procedures on the AR and AP subledgers

To learn more, contactĀ sales@mindbridge.ai.

Our tips for artificial intelligence in accounting

assurance audits

A common thread among the advice given by Accounting Today’s Top 100 Most Influential peopleĀ was exemplified by Alan Anderson of Accountability Plus:

ā€œBecome technology sponges. Embrace technology and explore avenues that can positively impact what accountants do on a daily basis to help their clients.ā€

With our unique experience in deliveringĀ AI-enabled auditing solutionsĀ to accountants around the world, we asked our experts what advice they would give to firms thinking about, or already engaged in, their journey towards artificial intelligence.

Solon Angel, Founder &Ā Accounting Today Top 100 Most Influential People

ā€œTechnologies that bring new efficiencies are the enablers of innovative process designs. It’s easy to get carried away by the hype in new technology cycles but keeping an eye on the fundamentals and measuring them against their utilities enables you to swiftly create value for all stakeholders.ā€

Scott Rockefeller, Director, Sales

ā€œWith artificial intelligence, incremental improvement is much better than postponed perfection. Waiting to see how the market evolves means you’re much further behind whenever your arbitrary inflection point comes up. The prudent strategy is to test the waters now and to get ahead of the competition.ā€

Azalia Shamsaei, Product Manager

ā€œArtificial Intelligence is not hype, it’s real, and it’s here! AI has huge potential to transform our lives, and it’s already impacting industries worldwide. Audit must embrace technology and change – as auditors of the future, you should leverage tools that incorporate new technologies and explore new ways to audit beyond today’s and yesterday’s ideas.

There is no point in delaying the change, it’s just a matter of time to adapt and know more about it. Being an early adopter not only gives you a competitive edge in hiring the right people, it also provides your clients with more effective and efficient audits that exceeds their expectations.ā€

John Colthart, Senior VP, Sales

ā€œSince the beginning, MindBridge has worked to help our clients change their business models, recruit new talent, and most importantly, enhance their existing processes. You can’t wait for standard setters, regulations, and ā€œsign off.ā€

Technology is a fundamental part of our social and business fabrics and its use allows you to grow, retain, and encourage evolution in the profession. As leaders and individuals, you can be part of the change by moving into the 21st century to be technologically enabled.ā€

James Moffatt, CPA, Director of Sales

ā€œEveryone finds themselves so busy with client commitments that they forget to plan for next year. And the year after that. Taking the time to make time to evolve your audit practice with enabling technology means you’re better prepared for the future.ā€

Gillian Fischer,Ā Global Manager, Customer Advocacy

ā€œWith the flurry of new opportunities around us, the pace of change can feel overwhelming. The truth is, not everything is changing. It’s your core values that stay constant. Being a trusted advisor, remaining relevant, committing to your clients, empowering your people, and maintaining integrity — these are things we typically cannot compromise.

Return to your core values for the courage to make a change in your organization and have the confidence in knowing that where you’re going is part of the story of who you are.ā€

Gordon Roxon, Account Executive

ā€œOvernight success takes time. Get on with it!ā€

Kevin Smiley, Account Executive

ā€œAI is a blue ocean opportunity, providing firms the unique ability to redefine their market boundaries and make their competition irrelevant. CPAs went through digitization using tools to automate and improve their existing way of working without really altering it fundamentally.

CPAs now are going through digital transformation by moving from one way of working to entirely new ones, capturing far more value than was possible using low-scale, low-leverage legacy business.

Firm leaders need to ask themselves: Do I want to invest in AI to create demand for my firm now or wait until I’mĀ  compelled to in the fight for my firm’s life for market share?ā€

To learn about MindBridge Ai Auditor in 10 minutes,Ā watch this video.

Democratizing financial and audit analytics with AI

auditing profession

PwC recently shared that in 2018 alone, 12 zettabytes of financial services industry (FSI) information was generated, but less than 0.5% was actually leveraged by businesses. This financial data explosion comes from an ever-increasing number of ERP and CRM systems employed by businesses and their partners, gathering and consolidating different payment, expense, inventory, and maintenance cost ledgers across the organizational landscape.

Contributing to this low rate of analytic engagement is the fact that current methods for analyzing financial data are slow; limited by time, capabilities and skill sets of the people and the software available to support the auditors and financial analysts.

Current analytics tools can’t keep pace

Let me illustrate this problem with a reasonably routine example. An internal auditor is asked to perform a risk analysis on a general ledger with five million rows. The Microsoft Excel limit for analysis is one million rows, preventing the auditor from using what is considered the world’s most popular analytical software. This means the auditor must engage their internal data analytics team, made up of specialized resources trained in the use of one the popular Computer Assisted Audit Tools (CAATs). This team in turn will write a script to perform the required analysis of the entire general ledger, or to save time, sample the data and attempt to extrapolate the risk.

The problem is, once the script work is scheduled and completed, the analysis might be too late for the audit process, or the priorities of the organization may have shifted to the point where the analysis is moot. If the team decided to use a sample and extrapolate the risk, they may have significantly less than a 100% risk analysis, putting the organization at risk.

It’s therefore no wonder that businesses are not making use of analytics; a world running on machine generated data, human-dependent analysis cannot keep pace.

AI enables analytics for all

This puts artificial intelligence (AI) front and center as the means of breaking this dependency on specialized resources and human-speed analysis. MindBridge is focused on developing the MindBridge Ai Auditor for auditors and finance managers to load financial data and analyze it on their own.

This allows auditors to bypass these specialized data science resources and tap directly into the power of their own data, effectively democratizing the access to financial analytics.

Microsoft CEO Satya NadellaĀ shares that,

The core currency of any business going forward will be the ability to convert their data into AI that drives competitive advantage.

MindBridge is in complete agreement with Satya’s view, and I look forward to sharing more with you on this.

The human-AI relationship for CPAs: More, better, and faster

sample size in auditing

Every week I interview entrepreneurs and experts from around the world toĀ share their big ideaĀ about new forms of value creation and the potential we can unlock when technology augments the unique strengths of people to deliver remarkable impact.

Transforming financial auditing

I got inspired by the big idea behind MindBridge Ai, hence I invitedĀ CTO, Robin GrossetĀ to my podcast. We explored the challenges in the financial auditing practice, and how, even after decades of automation, much of the practice is still very manual and sample based, leaving huge opportunities for fraud. Beyond that, we discussed why a human/machine approach will always provide the optimal combination to create exponential impact.

The thing that triggered me most from my interview with Robin

ā€œThe existing ways that we are analyzing or auditing financial transactions are inadequate with the rules based system, you’re only going to find something that you anticipate.

What’s the bigger value here?

If we only find what we anticipate, i.e., the cases that are highlighted based on the rules we have set, then what is the magnitude of what we are NOT catching? Robin addressed this by highlighting recent research from the Association of Certified Fraud Examiners. The number appears to be a number beyond imagination — but to put it in perspective — For every $1 we label as ā€˜fraud, misconduct, or irregularity,’ we’re missing out on $15. So, with current systems we’re only tracing 6.6%, and missing out on 93.4%! Translated, this is $500 for every person on the planet – every single year. And apparently (until recently) nobody was making a big issue about this, arguing ā€˜it’s not necessary to do anything different, this is the way we work’. This is a typical example of complacency and inertia in the workplace.

It’s about time the rules-based systems are going to be replaced by self-learning systems that are 24/7 active on finding new patterns, i.e., the +93% we’re missing out on. It’s the only way to win the fight against fraud. Doing nothing is not an option as data volumes and the number of channels we operate in keep increasing with extreme pace.

What significant fraud detection opportunity is raised?

What would be the impact on the economy and on society as a whole if this was solved? From my perspective, this is not only about finding the leaks in our systems, but very much about what we could do with the difference. Just look at the challenges we’re facing in health, education, or for example, public safety, simply because budgets are cut every single year. If these organizations would be able to 10x their ability to find fraud, misconduct, and irregularity — what could they do with that difference?

I would assume that there are many more areas like this to be uncovered — an opportunity and obligation for all of us to be sensitive about. I concur with Robin’s advice to look beyond the established conventions and existing standards. Only then will we be able to disrupt the status quo and increase (competitive) advantage.

On that notion — I concur 100% with Robin’s vision that the way to go about this is human-centric AI. In many industries, ā€˜black-box’ automation won’t work. Just think about how to explain black-box decisions in court? You’ll always need a person with a high-level understanding of the business context. Therefore, it’s about augmentation, not automation. Augmentation will allow human auditors to take their game to the next level, perform a better service to their clients, and be able to back their decisions up with clearly articulated evidence.

In other words: don’t be afraid that AI will take our jobs. It will not.

That said, doing nothing is not an option either: human auditors using AI will replace auditors who don’t. That’s an idea worth thinking about — also if you’re not an auditor.

Listen to theĀ big idea behind MindBridge Ai, and why it has the potential to transform the way financial auditors deliver value.

Our year in stories: 2018

internal audit purpose

We’re grateful to be a part of the world’s journey towards AI. Far more than an academic abstraction, 2018 was a leap forward in the practicalities of AI and machine learning across many different applications, with our own vision for the transformation of audit and financial analysis gaining momentum across the globe.

Here are some of the best and brightest spots of our year together in AI.

ā€œ must be developed and used while respecting people’s autonomy, and with the goal of increasing people’s control over their lives and their surroundings.ā€ – Montreal Declaration for a Responsible Development of AI

The social and ethical challenges of AI are just beginning to be realized, and the recent signing of the Montreal Declaration for a Responsible Development of AI is a big step forward in providing the framework for responsible technology development. As theĀ first private sector signatoryĀ to the Declaration, we reinforced our commitment to responsible, human-centric AI systems.

•

Through a passion for enabling technology,Ā Samantha Bowling, CPA, CGMA, wasĀ named a 2018 Innovative PractitionerĀ by CPA.com. As the first to successfully use AI in auditing for small businesses, non-profits, and local government, Samantha’s firm, Garbelman Winslow, leads the pack in improving processes and reducing the risk of material misstatements.

•

ā€œWe need to figure how to free up more data so that AI can thrive.ā€ – Leon Katsnelson, Director & CTO, Strategic Partnerships and Data Science Ecosystem, IBM,Ā speaking at IMPACT AI

The inaugural IMPACT AI conference was held on May 24th, bringing industry thought leaders and technology experts to an audience of over 550 people. In addition toĀ promoting AI education, the goal of the event was to increase and elevate more women in technology. Watch Navdeep Bains, Canadian Minister of Innovation, Science and Economic DevelopmentĀ discuss the influence of AI and stay tuned for details on next year’s conference.

Industry reform was a big theme in accounting this year, withĀ scandals for the Big FourĀ and the UK Competition and Markets AuthorityĀ recommending major shake-ups. Our CEO, Eli Fathi,Ā reminded usĀ how technology can play a critical role in reform.

•

The first known case of AI helping to investigate a human CPA committing over $2.8M in embezzlement fraud was documented on theĀ ACFE Insights blog.

•

ā€œAI is transforming the way auditors do business and the exponential pace of change is requiring CPAs to get up to speed quickly.ā€ – Tom Hood, CPA, President & CEO, Maryland Association of CPAs

With dozens of events, webinars, seminars, and forums under our belts in 2018, two notable ones were our AI & the Future of Accounting roadshow, in partnership with theĀ Canadian Trade Commissioner Service, and ourĀ expert CPA panel in December. While the roadshow introduced AI to audiences across eight cities, the expert panel delivered practical advice and recommendations tailored directly for auditors. We were also recognized by industry associations and media this year, including being selected as theĀ Top New Product of 2018 by Accounting TodayĀ and theĀ Best Machine Learning Solution for Regulatory ComplianceĀ by Central Banking.

•

After a successful pilot with universities across North America, weĀ launched our University Alliance ProgramĀ in July to educate and train accounting students on the use of AI in auditing. As this year ends, the momentum will continue into 2019 with more than double the amount of institutions on board, over 1300 students completing the program, and a wealth of new curriculum materials and case studies being generated.

•

Our partnerships with accounting firms around the world exploded, growing our user base to well over 200 organizations. Relationships such as withĀ Garbelman Winslow,Ā KNAV P.A., andĀ Kreston Reeves, solidify the value that AI brings to auditing and help us continually improve the MindBridge platform.

For our development team, 2018 was a year of transition as we went from launching the first release of MindBridge Ai Auditor to continuous delivery of major new features for users. February saw new functionality such as Natural Language Processing (NLP) and accounts payableĀ launched at a marquee eventĀ in partnership with the Canadian Trade Commissioner Service at Canada House in London, UK, while the rest of the year saw delivery of discrete pieces of value for users, such as interim audit reviews, the data ingestion wizard, and theĀ amazing Filter BuilderĀ used by auditors to create their own AI-enabled tests and logic.

What will 2019 bring? We firmly believe that AI is still in its revolution stage for many, bringing aboard new players all the time, while others continue to work with AI-based audits every day. We’ll continue to share and educate along the way, and hope that you’ll let us know how we’re doing.

Why we signed the Montreal Declaration for a Responsible Development of Artificial Intelligence

auditing sample

With artificial intelligence (AI) influencing every aspect of our lives, and its continued growth in research and commercialization opportunities, the question isn’t whether we should develop it responsibly, it’s a question of how.Last year, over 400 participants came together at the Forum on the Socially Responsible Development of Artificial Intelligence to discuss themes of cybersecurity, legal liability, moral psychology, jobs, and other areas to begin the conversation around the impact of artificial intelligence systems (AIS) on humans. Given that it’s now possible to create autonomous systems capable of performing tasks that were once the sole domain of human intelligence and have strong influence on data-driven decisions, it’s imperative to consider the potential effects of AI on ethical and social concerns. How will AI impact security and privacy? What is the impact on social equality and cultural diversity? Will AI disrupt careers and upend the job market?

These are tough questions and the result of the 2017 forum was a draft declaration setting out a framework of ethical guidelines for the development of AI. After a months-long consultation process with the public, experts, and government decision makers, the finalĀ Montreal Declaration for a Responsible Development of Artificial IntelligenceĀ was signed on December 4, 2018 at the Society for Arts and Technology.

As of today, we are theĀ first private sector signatoryĀ to the Declaration, reinforcing our commitment towards an ethical framework for AIS technology development. The Declaration has three main objectives:

  1. Develop an ethical framework for the development and deployment of AI
  2. Guide the digital transition so everyone benefits from this technological revolution
  3. Open a national and international forum for discussion to collectively achieve equitable, inclusive, and ecologically sustainable AI development

How the Montreal Declaration applies to us

As MindBridge is building an AI platform to help people analyze and understand vast amounts of their data in ways never thought of before, it’s critical to follow a development philosophy that keeps our users at the center of the loop. Because we’re building it for you.

We firmly believe that AI is not meant to replace humans, rather its greatest benefit is to empower people to make better decisions for themselves and society without imposing constraints based on any specific beliefs. As the Declaration’s ā€œRespect for autonomyā€ principle guides:

AIS must be developed and used while respecting people’s autonomy, and with the goal of increasing people’s control over their lives and their surroundings.

Another principle is democratic participation, where ā€œAIS processes that make decisions affecting a person’s life, quality of life, or reputation must be intelligible to their creators.ā€ Our human-centric approach to the MindBridge platform embodies this philosophy within every aspect of the system. Our CTO, Robin Grosset, explains the details and provides concrete examples inĀ his recent blog.

We embraced these and other principles long before the Declaration was signed, so it required little thought to join and become the first private company to get on board. Now that it’s official, we look forward to working with industry, government, and other parties to ensure a responsibly-developed AI future for all of us.

Our approach to human-centric artificial intelligence

internal auditing

Where is the AI?

Artificial intelligence (AI) is all around us, it powers the helpful voice on my phone and it’s in the digital assistant on my kitchen counter. Actually, I have to admit liking to say ā€œAlexa turn on Christmasā€ to turn my Christmas lights on and off. It’s just a simple end-point computer, like a terminal, communicating with a cloud-based service which does all the hard work of interpreting what I say and figuring out what to do.

Many AI systems are not as obvious as Alexa, they surround us, yet we don’t see them. Take the ads on my Facebook feed, for example, an algorithm is figuring out what it knows about me and then what ads will likely work best. Even with Google, what appears to be just a search box is much smarter. If you ask the question ā€œWhat is the population of Canada,ā€ Google is not just searching documents using its famous PageRank algorithm, it’s doing much more. It’s figuring out that an infographic is the best way to communicate the population of Canada to me and showing this alongside its other insights. It also knows flight numbers and does different things depending on context.

What we think is a simple search is much more. AI is sometimes quite subtle and helping us in ways we may not realize.

Good experience design often makes our little AI helpers invisible to us. Two of the ten Dieter Rams principles of good design are, ā€œGood design is unobtrusiveā€ and ā€œGood design is as little design as possible.ā€ We can see why subtle or invisible AI happens; it is considered good design.

Does MindBridge hide its AI?

We have a philosophy that when our AI provides insight or direction to users, we give them the feedback they need to both see it and understand it. We believe in human-centric AI, which means the human is the central part of the system and they should be able to understand what the AI is telling them and have explanations at each stage. The AI needs to communicate and therefore, being visible is an essential element in the trust relationship we are endeavoring to create.

Having said that, sometimes we can’t help ourselves and occasionally we make the experience seamless and require users to click on little information tabs to find out more. This is a design principle called ā€˜progressive disclosure’ and allows a user to select the level of detail they want.

So where is our AI? How do you know it’s there and working? Let’s take three examples from ourĀ AI Auditor productĀ and walk through the techniques and the design considerations.

#1 Unobtrusive but verifiable

Auditors often have to classify items in audit tools manually. They may need to say what kind of money is held in a certain type of account, whether it’s a cash asset, a liability, or maybe a non-capital expense. This process of instructing a software tool in what something means is laborious and repetitive. I think it’s fair to say nobody wants to do it but it’s required to get an accurate view of the finances. This is a great candidate for automation with AI.

MindBridge has a built-in account classifier that uses the human-readable label on financial accounts to determine what kind of account it actually is. This is a form of language processing and we use two methods, the first is a simple search which works well for well-labelled accounts, the second is a Neural Network Classifier which learns how people classify accounts. The net effect (excuse the pun ☺) is that most users of MindBridge spend little to no time telling our system what an account is. It just knows. We do recommend, however, that users review its findings to confirm or correct them. Our AI also learns from these interactions.

This is what it looks like as its working: It appears to be loading data, pretty unobtrusive and just doing its thing.This is what it looks like when the user verifies the outcome. The user has the option to change the classification of the account. This is the only real clue that something smart has just happened.You could be forgiven for not noticing that a lot of work is happening but there are some real time savings here. Below are some charts of simple text search methods vs. a hybrid of text search and AI together. On simple and well-labelled accounting structures, the accuracy of a text search is indistinguishable from an AI. But as we get a little more complex, we see big wins. Further, as the complexity grows to involve a massive organization’s accounts, you see that the simple text search accuracy breaks down and doesn’t cope at all. Conversely, the AI method keeps on punching through the problem and gets it done. The time savings at the complex level is huge; we are talking hours, if not days, of human time saved in laborious activities.

#2 Search that tells you what it understands and gives you options

The MindBridge search interface is a little different than what you’re used to, as we want everything to be understandable and explicable even at the level of a search box. Have you ever typed a search into Google and not got the results you wanted? Chances are you ended up not scrolling to page 2, typed in a slightly different question, and got what you wanted by trial and error.

At MindBridge, we value the AI being visible and explaining itself so that our users can figure out what part of the question is driving the view of data. Here we see a search user interface where the user types their query. There is no AI yet.The user hits go! The AI system parses the language and uses natural language processing (NLP) techniques to unpack what is being requested. Our NLP AI understands language in general but also common accounting terminology. It highlights the important terms in the query and filters the transaction list accordingly.Note that the highlights are clickable so that a user can determine other possible paths and verify that the AI has understood the question. It also understands complex semantics like conjunctions, which are combinations of terms such as AND, OR, or NOT logical expressions. This allows more complex questions to be posed and answered.

In this way, MindBridge users can not only search vast amounts of transaction data for specific scenarios, they can do this without writing an SQL query or using similar technical languages. The AI is effectively reading back their query to them to help in the understanding of what’s driving the results and showing other possibilities. This user interface is very artful as it provides both progressive disclosure and explainable AI, all in a search box.

For transparency, MindBridge has filed a patent for methods used in this search interface. We believe in ā€˜AI for Good’ and human-centric AI and we use patent protection to ensure the freedom to do the work we do.

#3 Ensemble AI

Ensemble AI is the main event at MindBridge and it guides much of our work. We consider its primary role to be a focusing function for people and, as we specialize in finding insights and irregularities in financial data, it allows us to do this in a robust and explainable way.

So how does Ensemble AI work?

First, we need to understand that the ensemble is not just one method or algorithm but many. It’s like having a panel of experts with different types of knowledge and asking each of them what they think about a given transaction or element of data. The system then combines all the insights from the individual algorithms together.

For example, AI Auditor includes standard audit checks, so some of these ā€œexpertsā€ are following simple audit rules while others follow advanced AI techniques and algorithms. The point of the ensemble model is that they all work together like an orchestra and, as the user is the conductor of the orchestra, they can select what’s important to them and the combination of results from the ensemble is presented in an easy to follow way.

Here’s an example of one of the detailed views of the ensemble at work (click to enlarge). You see all the little rectangles which have the larger red or green highlights, these are the individual AI capabilities in the ensemble.Let’s dig deeper into two of these capabilities.

Expert score

One example of an AI method we use is an ā€˜Expert System.’ This is a classical AI method that draws on the knowledge of real-world accounting practice to identify unusual transactions.

How do we capture real-world knowledge? We work closely with audit professionals and quiz them with surveys and specific questions about risky transactions, allowing us to construct an expert system that knows hundreds of account interactions and their associated concerns. We can run this method very quickly on large amounts of data, allowing us to scale human knowledge and highlight issues that a human user looking at a small sample could easily miss.

Rare flows

Ensemble AI can also identify unusual things using empirical methods. This leverages the science of what is usual or unusual, such as another method we use called ā€˜Rare flows.’ This part of Ensemble AI is a method of unsupervised learning from a family of algorithms known as outlier detection. The nice thing about unsupervised learning algorithms is they bring no bias, they simply identify what’s in the data and thus let the data speak for itself.

The purpose of this method is to uncover unusual financial activity. We apply this method to all financial activity but theĀ specific PCAOB guidance on material misstatementsĀ says:

ā€œThe auditor also should lookĀ to the requirements in paragraphs .66–.67A of AU sec. 316, Consideration of Fraud in a Financial Statement Audit,Ā for … Ā significant unusual transactions.ā€

This algorithm finds unusual activity and highlights them and we also perform this type of analysis with several different ensemble techniques. One of the nice things about the ensemble is that you’re not relying on one method, and these techniques can look at account interactions, dollar value amounts, and other outlier metrics to bring them all together.

Why human-centric AI is needed in auditing

Most audit standards today, including the international standards, were the result of years of experience in previous cases of accounting irregularities. As such, they are great at identifying the problems of the past. The limitation is that the typical rules-based system approach to finding irregularities can never identify a circumstance that is not anticipated, and this is why we should apply AI methods like those described above.

A future-looking audit practice needs to adapt to new circumstances. Every industry is changing as the result of AI adoption and the idea that we can uncover new and unusual activity, and explain why it is being flagged, is a key strength of AI systems used by forward -looking audit professionals.

This is why we need AI in auditing. In the words ofĀ John Bednarek, Executive Director of Sales Operations, Marketing & Strategic Business Development at MindBridge, ā€œAuditors using AI will replace auditors who don’tā€. The simple reason for this is auditors who leverage AI will be faster and more complete in their work, providing a better service to their clients.

Ethical AI goes beyond legal AI

internal audit sampling

The recent case of the Statistics Canada project toĀ use personal financial data from banks to study the spending habits of CanadiansĀ provides a very clear lesson in the ethics of AI. In this case, Statistics Canada has clear legal authority to request and use this data and it’s very likely that the proposed project conforms with ethical standards for AI and analytics. There is also an excellent case that this project will provide significant public benefit. However, it’s also clear that the project failed to gain a moral license from Canadians and by failing in this regard, they have put the project and perhaps their freedom to operate at risk.

Shining a light on the project

At this point, the details about the project are difficult to come by and I have not seen evidence of any public consultation or public notice of the project. This project came to light through aĀ news story published by Global News on Oct 26, 2018. Based on the news reports and a bias towards the general good intentions of government bureaucracy, we can infer that Statistics Canada finds its current survey-based approach to collecting data on Canadian spending habits deeply inadequate. I also expect that the bureaucrats involved saw the opportunity to provide a more accurate picture of Canadian spending habits, more efficiently, and with less burden on the members of the Canadian public. After consulting with Justice, they also determined that they have the legal authority to do so and they honestly believe that Canadians by and large trust Statistics Canada with their personal data. So they made the decision to use the legislation governing Statistics Canada and request data from the banks. I also expect that bureaucrats knew that this request could be misunderstood by the public so they decided to act out of the public eye, trusting that the banks would comply without fuss. Of course, this project will benefit the banks greatly.

What possibly could go wrong?

Application to analytics and AI

I want to stress that there was no malice in the bureaucratic intentions behind this project. To the contrary, I see the motivations as things we want to encourage: innovation, efficiency, improved quality, and Canadian competitiveness. Where things may have went wrong is a long-standing bureaucratic culture of secrecy. The causes and solutions to this problem with bureaucratic culture is a topic for another day.

No doubt there will be calls for changes to the Statistics Act but I think cries for wholesale changes are misguided. Overall, the Act provides a good example of a legal framework for analytics. I’m not saying that events such as this should be ignored, rather the justice department should be tasked with reviewing the act and regulations with the goal of Ā improving the legislation — perhaps by making public consultation mandatory when Statistics Canada wants to collect personal data indirectly.

Legislative frameworks for analytics and AI must do a few things well:

  • They must protect privacy
  • They must ensure that the collection and use of personal data contributes to the general social welfare broadly defined
  • They must protect the ability to innovate

On this last point, legislative frameworks must be flexible and protect against egregious misuse while relying on social and market mechanisms to align activity with public expectations. Authority granted by legislation must protect against the right to innovate being blocked by a radical few. By these tests, the Statistics Act stands up well.

Having legal authority to do something is not the same as acting morally or ethically. In general, ethical use of personal data requires that the data subjects explicitly consent to the collection and use of their data. One can assume that the data subjects have given a license to the analytics organization to use their data for the intended purposes but, in practice, this is complicated and there are exceptions to this approach. One such exception is that the use serves the public good. From what I understand of the proposed use of data by Statistics Canada, this test is clearly met.

How we can do better

So what went wrong? The personal data in the possession of the banks was created as part of delivering banking services. The public expectation, perhaps naively, is that that is the only use they have consented to. The attempt by a third party to access and use this data to develop profiles of consumer spending habits goes well beyond their expectations. In this case, the legal authority to do this is irrelevant and disturbing. At the very least, a public education campaign describing why this is important to Canada and Canadians and how each individual will be protected in the process would have gone a long way to easing the public’s concern.

More fulsome consultation and offering individuals with the ability to opt out would likely have eliminated all barriers and created a positive opinion of the project. Each time an organization tries to fly under the radar when accessing large quantities of personal data, they create a risk of public backlash that will saddle the industry with stifling regulation.

The AI industry needs the right to ethically innovate and to do this, we need a regulatory environment that gives latitude to innovate. This requires the public to be confident that industry members will act ethically within the bounds of the legislation. Each time the AI industry goes against these expectations, the right to innovate is put at risk.