{"id":8924,"date":"2021-02-16T00:01:03","date_gmt":"2021-02-16T05:01:03","guid":{"rendered":"http:\/\/www.jasonapollovoss.local\/?p=8924"},"modified":"2021-03-15T09:34:59","modified_gmt":"2021-03-15T13:34:59","slug":"fraud-and-deception-detection-text-based-analysis","status":"publish","type":"post","link":"https:\/\/jasonapollovoss.com\/web\/2021\/02\/16\/fraud-and-deception-detection-text-based-analysis\/","title":{"rendered":"Fraud and Deception Detection: Text-Based Analysis"},"content":{"rendered":"

Research analysis relies on our trust.<\/span><\/h3>\n

Among the many factors we consider as fundamental investors are assessments of a company\u2019s strategy, products, supply chain, employees, financing, operating environment, competition, management, adaptability, and so on. Investment professionals conduct these assessments to increase our understanding, yes, but also to increase our trust in the data and the people whose activities the data measure. If we cannot trust the data and the people who created it, then we will not invest. In short, we must\u00a0trust\u00a0<\/em>management.<\/span><\/p>\n

 <\/p>\n

Our fraud and deception detection methods are only okay.<\/span><\/h3>\n

But by what repeatable method can we evaluate the trustworthiness of companies and their people? Usually the answer is some combination of financial statement analysis and \u201ctrust your gut.\u201d Here is the problem with that:<\/span><\/p>\n

 <\/p>\n

1. Time and Resource Constraints<\/span><\/h3>\n

Companies communicate information through words more than numbers. For example, from 2009 to 2019, the annual reports of the Dow Jones Industrial Average\u2019s component companies tallied just over 31.8 million words and numbers combined, according to AIM Consulting. Numbers only made up 13.5% of the total.<\/span><\/p>\n

Now, JP Morgan\u2019s 2012 annual report is 237,894 words. Let\u2019s say an average reader can read and comprehend about 125 words per minute. At this rate, it would take a research analyst approximately 31 hours and 43 minutes to thoroughly read the report.\u00a0The average mutual fund research analyst in the United States makes around $70,000 per year,<\/a>\u00a0according to\u00a0WallStreetMojo<\/em>. So that one JP Morgan report costs a firm more than $1,100 to assess. If we are already invested in JP Morgan, we\u2019d perform much of this work just to ensure our trust in the company.<\/span><\/p>\n

Moreover, quantitative data is always publicly released with a significant time lag. Since a company\u2019s performance is usually disclosed quarterly and annually, the average time lag for such data is slightly less than 90 days. And once the data becomes public, whatever advantage it offers is quickly traded away. Most investment research teams lack the resources to assess every company in their universe or portfolio in near real time, or just after a quarterly or annual report is released.<\/span><\/p>\n

Conclusion: What is that old line? Oh, yeah: Time is money.<\/strong><\/span><\/p>\n

 <\/p>\n

2. Trusting our gut does not work.<\/span><\/h3>\n

Despite the pan-cultural fiction to the contrary, research demonstrates we cannot detect deception through body language or gut instinct.\u00a0In fact, a meta-analysis of our deception-spotting abilities found a global success rate just 4% better than chance<\/a>. We might believe that as finance pros we are exceptional. We would be wrong.<\/span><\/p>\n

In 2017, we measured deception detection skills among finance professionals. It was the first time our industry\u2019s lie detection prowess had ever been put to the test<\/a>. In short: ouch! Our overall success rate is actually worse than that of the general population: We did not score 54%, we earned an even-worse-than-a-coin-toss 49.4%.<\/span><\/p>\n

But maybe our strengths are in our own sector. Put us in a finance setting, say on an earnings call, and we\u2019ll do much better, right? Nope, not really. In investment settings, we could detect deception just 51.8% of the time.<\/span><\/p>\n

There is more bad news here (sorry): Finance pros have a strong truth bias. We tend to trust other finance pros way more than we should. Our research found that we only catch a lie in finance 39.4% of the time. So that 51.8% accuracy rate is due to our tendency to believe our fellow finance pros.<\/span><\/p>\n

One other tidbit: When assessing statements outside of our domain, we have a strong 64.9% deceptiveness bias. Again, this speaks to our industry\u2019s innate sense of exceptionalism.\u00a0In an earlier study<\/a>, our researchers found that we believe we are told 2.14 lies per day\u00a0outside of work<\/em>\u00a0settings, and just 1.62 lies per day\u00a0in work<\/em>\u00a0settings. This again speaks to the truth bias within finance.<\/span><\/p>\n

Finally, we believe we can detect lies within finance at a 68% accuracy rate, not the actual 51.8% measured. Folks, this is the very definition of overconfidence bias and is delusion by another name.<\/span><\/p>\n

Conclusion: We cannot trust our guts.<\/strong><\/span><\/p>\n

 <\/p>\n

3. Auditors\u2019 techniques audit numbers.<\/span><\/h3>\n

But what about auditors? Can they accurately evaluate company truthfulness and save us both time and money? Yes, company reports are audited. But auditors can only conduct their analyses through a micro-sampling of transactions data. Worse still, auditors\u2019 techniques, like ours, are largely focused on that very small 13.5% of information that is captured numerically. That leaves out the 86.5% of text-based content.<\/span><\/p>\n

Further, because financial statement analysis \u2014 our industry\u2019s fraud detection technique \u2014 is one step removed from what the auditors see, it is hardly reliable. Indeed, financial statement analyses are just table stakes: Ours probably won\u2019t differ much from those of our competitors. Just looking at the same numbers as everybody else is unlikely to prevent fraud or generate alpha.<\/span><\/p>\n

And what about private markets? The investment research community has spent an awful lot of time looking for investment opportunities in that space in recent years. But while private market data are sometimes audited, they lack the additional enforcement mechanism of public market participants\u2019 due-diligence and trading activities. These can sometimes signal fraud and deception.<\/span><\/p>\n

Conclusion: There has to be another tool to help us fight deception.<\/strong><\/span><\/p>\n

 <\/p>\n

 <\/p>\n

Scientifically Based Text Analyses to the Rescue<\/span><\/h3>\n

Starting with James W. Pennebaker\u2019s pioneering work<\/a>, researchers have applied natural language processing (NLP) to analyze verbal content and estimate a transcript\u2019s or written document\u2019s credibility. Computers extract language features from the text, such as word frequencies, psycholinguistic details, or negative financial terms, in effect, dusting for language fingerprints. How do these automated techniques perform?\u00a0Their success rates are between 64% and 80%<\/a>.<\/span><\/p>\n

In personal interactions, as we noted, people can detect lies approximately 54% of the time. But their performance worsens when assessing the veracity of text.\u00a0Research published in 2021 found that people have about a 50% or coin-flip chance to identify deception in text. A computer-based algorithm, however, had a 69% chance.<\/a><\/span><\/p>\n

But surely adding people to the mix improves the accuracy? Not at all. Our overconfidence as investors sabotages our ability to catch deception even in human-machine hybrid models. The same researchers explored how human subjects evaluated computer judgments of deception that they could then overrule or tweak. When humans could overrule, the computer\u2019s accuracy dropped to a mere 51%.\u00a0When human subjects could tweak the computer judgments in a narrow range around the algorithms\u2019 evaluation, the hybrid success rate fell to 67%<\/a>.<\/span><\/p>\n

Computers can give investment pros a huge advantage in evaluating the truthfulness of company communications, but not all deception detection methods are one size fits all.<\/span><\/p>\n

One computer-driven text-based analysis,\u00a0published in 2011<\/a>, had the ability to predict negative stock price performance for companies whose 10-Ks included a higher percentage of negative words. By scanning documents for words and phrases associated with the tone of financial communications, this method searched for elements that may indicate deception, fraud, or poor future financial performance.<\/span><\/p>\n

Of course, those businesses whose stock prices were hurt by this technique adapted. They removed the offending words from their communications altogether.\u00a0Some executives even hired speech coaches to avoid ever uttering them<\/a>. So word-list analyses have lost some of their luster.<\/span><\/p>\n

 <\/p>\n

 <\/p>\n

Where Do We Go from Here?<\/span><\/h3>\n

It may be tempting to dismiss all text-based analyses. But that would be a mistake. After all, we have not thrown away financial statement analysis, right? No, instead we should seek out and apply the text-based analyses that work. That means methods that are not easily spoofed, that assess\u00a0how language is used<\/em>\u00a0\u2014 its structure, for example \u2014 not\u00a0what language is used<\/em>.<\/span><\/p>\n

With these issues in mind, we developed\u00a0Deception And Truth Analysis (D.A.T.A.)<\/a>\u00a0with\u00a0Orbit Financial<\/a>. Based on a\u00a010-year investigation of those deception technologies that work in and out of sample<\/a>\u00a0\u2014 hint: not reading body language \u2014 D.A.T.A. examines more than 30 language fingerprints in five separate scientifically proven algorithms to determine how these speech elements and language fingerprints interact with one another.<\/span><\/p>\n

The process is similar to that of a standard stock screener. That screener identifies the performance fingerprints we want and then applies these quantitative fingerprints to screen an entire universe of stocks and produce a list on which we can unleash our financial analysis. D.A.T.A. works in the same way.<\/span><\/p>\n

A key language fingerprint is the use of articles like a, an, and the, for example. An excess of these is more associated with deceptive than truthful speech. But article frequency is only one component: How the articles are used is what really matters. And since articles are directly connected to nouns, D.A.T.A is hard to outmaneuver. A potential dissembler would have to alter how they communicate, changing how they use their nouns and how often they use them. This is not an easy task and even if successful would only counteract a single D.A.T.A. language fingerprint.<\/span><\/p>\n

The other key findings from recent D.A.T.A. tests include the following:<\/span><\/p>\n