Key Scientific Paper Redux – Why Do Lie-Catchers Fail?

For those of you new to Deception And Truth Analysis (D.A.T.A.) and our grounding in science, it is a well-established stat that people, including pros, are only about 54% successful at deception detection. Our latest Key Scientific Paper Redux entitled Why Do Lie-Catchers Fail? A Lens Model Meta-Analysis of Human Lie Judgments[i]is one of the few papers that examines possible reasons for people being so bad at deception detection.

Why Do Lie-Catchers Fail?: Study Details

Three main questions are addressed by the paper: 

1. What cues do people use in deception detection?
2. What is the degree of overlap, if any, between subjective and objective cues to deception?
3. What is the source of people’s poor deception detection?

Regarding #3, above, there are two main theories as to why people are poor at deception detection:

• People rely on invalid cues when judging deceptiveness. This is known as the wrong subjective cue hypothesis.
• A lack of valid deception cues limits accuracy. In other words, there is no Pinocchio’s nose, which is known as the weak objective cue hypothesis.

To evaluate the two theories above the researchers used a statistical technique known as Brunswik’s lens model. This is a method for studying human predictions about criteria – in this case, is someone being deceptive – that are probabilistically related to cues. Examples of cues include a doctor making a diagnosis based on patient symptoms, or a college admissions officials making predictions about an applicant’s likely success at their college, and so on.

Coupled with the lens model equation, the researchers conducted a total of four meta-analyses (i.e. a study of studies) to test the above two hypotheses. 

Why Do Lie-Catchers Fail?: Major Findings

Meta-analysis 1 – Cues to Perceived Deception

Researchers explored the various cues to deception that have been researched in the scientific literature. Crucially, they examined the cues that those seeking to detect deception perceive are valid, as well as the actual validity of those cues in detecting deception.

1. There is a large gap between a person’s perceptions about valid behavioral cues to deception and actual underlying deception.
2. Of the 66 cues to deception, 41 are, at a statistically significant level, believed by people to be valid cues to deception at the 0.05 level.
3. Of the 66 cues to deception, 21 are, at a statistically significant level, believed by people to be valid cues to deception at the 0.001 level.
4. That said, not a single one of the actual cues to deception has a greater than 50% correlation to actual deceptive behavior.

Meta-analysis 2 – Cues to Perceived and Actual Deception

This examination was designed to explore Hypothesis 1, the wrong subjective cue hypothesis. In other words, those trying to detect deception may be bad at it because they are relying on behavioral cues that are not actually reliable indicators of deception.

Clearly then, this hypothesis can be rejected if there is a strong correlation between perceived cues to deception and actual cues to deception. That’s because a strong correlation between the two would indicate people’s perceptions about the correct deceptive cues to behavior are, in fact, correct.

Of the 66 cues, 57 of them had been researched more than once. The remaining 9 cues had not been and were excluded from the analyses. For the 57 cues, the following was found:

1. The correlation between the perceived cues to deception and the actual cues to deception = 59.3%. This indicates pretty strong evidence that people’s perceived cues to deception and the actual cues to deception are fairly accurate. To confirm this, however, Meta-analysis 3 digs a little deeper.
2. Mean absolute Z for the association of cues to deception = 9%.
3. Mean absolute Z for the association of cues to judgments of deception = 25%.
4. A t-test done with 56 degrees of freedom found these two means differ significantly = t(56) = 8.22, p < 0.001.
5. In other words, the perceived behavioral cues are more strongly related to judgments of deception than to actual deception. This can be interpreted as the perceivers over-emphasizing the deceptive cues too strongly.
6. Further tests were done to see if perceivers – those judging deception – infer deception from truth cues, or if they infer truth from deception cues. There was no evidence in the data of this.  

Meta-analysis 3 – Within Study Evidence

Hartwig and Bond relied on two different datasets to generate their perceived cues and their actual cues. In order to examine whether or not this fact introduced bias into the results that would complicate interpretation, they looked for “within study” evidence. That is, some of the studies that made of the meta-analysis had actual cues data, too. This allowed for them to test for bias.

Their focus was on instances in which researchers reported correlations among deception and three or more cues. Additionally, they looked at those same cues on the same set of “senders” (i.e. those people in the studies who were tasked with attempting to deceive others). There were 25 sender samples meeting these criteria with 1,422 senders and judgments made by perceivers who numbered 2,250. Using statistical techniques they converted the correlations into cross-correlations to conduct their examinations.

1. For the 25 sender samples, correlations between actual deception and perceived cues varied widely. The minimum correlation was -68%, while the maximum was 97%. Overall, the median correlation was 54%.
2. Of the 25 sender samples, 22 of them had a positive cross-correlation.
3. In the within-study study results, the more strongly connected a cue to deception, the stronger the connection with perceived deception, too. The correlation, r, associated with these results is 72%, and with a narrow confidence interval (i.e. a low variance) of between 70% to 74%.
4. In 22 of the studies, the researchers had figures relating to the accuracy of the deception judgments made by perceivers. Here the correlation was 60%. In other words, those attempting to perceive deception seem to be, on balance, using correct cues to deception.
5. However, while they are using correct cues they are over-emphasizing their cues in their assessment. This is because the absolute mean of perceived deception, 27%, is statistically significantly higher than the mean of actual deception, 17%. A two-tailed t-test (24) = 3.89 with p < 0.005.
6. Last, the researchers examined the idea that their averaging of cues may be obscuring perceivers’ ability to focus in on the “correct” cues, that is those that are most strongly correlated with actual deception. To test this, they looked at the maximum correlation of an actual cue to deception, 39% and compared it with the maximum correlation to a perceived cue of 61%. Once again, the cues are more strongly related to perceived deception, rather than actual deception, t-test (24) = 3.72, p < 0.005.

Meta-analysis 4 – Multiple Cues

This final meta-analysis sought to investigate whether poor deception detection can be attributed to incorrect decision-making strategies, or due to a lack of valid deception cues. For the above three meta-analyses it was assumed that perceivers were judging deception from a single cue, but what if perceivers, instead, use multiple cues? This would mean that looking at single item correlations would be an inappropriate analysis.

In a lens analysis, the correlation between perceived deception and actual deception is made up of two terms: one which involves the correlation of the errors in predicting another person’s deceptiveness and that person’s perceived deceptiveness. If these error terms are uncorrelated, the correlation between actual deception and perceived deception is the product of three factors: RDec, RPer, and G. RDec is the multiple correlation for predicting actual deception from cues. RPeris the multiple correlation for predicting perceived deception from those same cues. Last, G is the correlation between predictions of a deceiver’s deception from cues and the receiver’s perception of those same cues.

To estimate these three factors Hartwig and Bond sought studies in which multiple cues to deception had been used by those evaluating another’s deceptiveness. They found 59 multiple cue deception predictions that met their criteria, representing 3,428 deceivers. They also found 30 multiple cue predictions of perceived deception that satisfied their criteria, that was made up of 1,178 deceivers, and 3,497 perceivers.

1. For predicting deception from multiple cues the median RDec = 46%, with the number of cues being used by those attempting to detect deception, and thus entering into the correlations being between 2 and 38 cues.
2. Across the 59 multiple correlations there is no relationship between the magnitude of RDec and the number of cues entering into it.
3. In order to combine multiple correlations together into a single value of RDec the researchers used statistical techniques to land on a single correlation value for RDec= 36%, with the 95% confidence interval being 33% to 38%.
4. For RPer the median correlation = 61%, with the number of cues entering into it being between 2 and 16 cues.
5. Across the 59 multiple correlations there is no relationship between the magnitude of RPer and the number of cues entering into it.
6. Converting the multiple correlations into a single value for RPer = 63%, with the 95% confidence interval being between 60% to 65%.
7. Again, looking at these two correlations, RDec= 36% < RPer = 63%, it is obvious that it is easier to predict perceived deception than actual deception.
8. Bond and DePaulo reported that the relation between actual deception and perceived deception is 21%.
9. The above three figures allow for an inference of the value of G in the lens model of 93%. Thus, we have:

• R-acc = R-Dec x R-Per x G or 21% = 36% x 63% x 93%

The above lens model demonstrates something very powerful, namely the accuracy of deception detection is:

• Most constrained by a lack of valid cues to deception, since R-Dec = 36%
• Less constrained by judge’s unreliability in using those cues, since R-Per = 63%
• Unconstrained by matching of behaviorally-based predictions of deception with predictions of deception judgment, since G = 93%.

Why Do Lie-Catchers Fail?: Sub-Findings

1. Hartwig and Bond’s work shows that in contrast with past work, people seem to be relying on signals to detect deception that are different than the ones they report using. This is indicated by the wide gaps between the believed correlation of a behavioral cue with deception and the actual correlation, such as in the case of gaze aversion. This indicates that there is a general lack of self-awareness.
2. Among the strongest correlates in use when someone judges another to be deceptive are someone else’s incompetence, ambivalence, lack of spontaneity, and plausibility. These findings are interesting because when the technique employed to surface these techniques is relying on the self-reports of deception detectors, they do not report these things. Again, this suggests a lack of self-awareness on the part of most people.
3. If those attempting to detect deception lack self-awareness, then what are they relying on when judging veracity? From a look at the meta-analyses it appears that they are judging others as deceptive when they provide implausible, illogical accounts with few details, particularly few sensory details.

Conclusions

1. The wrong subjective cue hypothesis, that people rely on invalid cues when judging deceptiveness is not supported by the research.
2. The weak objective cue hypothesis, that there is a lack of valid deception cues when judging deceptiveness is supported by the research.

Quotes of Note

• “It is worth noting that the strongest cues to deception judgments are not single behaviors but global impressions, such as ambivalence.”
• “People do not seem to know what behaviors they use when judging veracity. The behaviors they claim to use are largely inaccurate, but the behaviors they actually rely on show a substantial overlap with objective cues to deception. Simply put, intuition outperforms explicit notions about deception.”
• “[O]ne study showed that perceivers’ performance was slightly enhanced by both bogus training (in cues that are not actually related to deception) and training in actual cues to deception. This suggests that to the extent that training in valid cues is effective at all, it might be the act of itself rather than its content that is responsible for improvements in performance, possibly by creating more motivated lie-catchers.”

[i]Hartwig, Maria and Charles F. Bond, Jr. “Why Do Lie-Catchers Fail? A Lens Model Meta-Analysis of Human Lie Judgments.” Psychological Bulletin Vol. 137, No. 4 (2011): pp. 643-659