Faster Than the Speed of Thought

A bat and a ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost?

This is one of three questions on the classic version of the cognitive reflection test (CRT) – a survey designed to measure people's tendency to jump to intuitive (but wrong) conclusions without first taking a moment to think. The intuitive answer is 10 cents, suggested by the contrast between $1.00 and $1.10. But that's incorrect. If the ball cost 10 cents, then the bat would cost $1, or $0.90 cents more than the ball. The correct answer is 5 cents, making the bat cost $1.05. Many of you have probably heard of this problem before.

As with any test of reasoning, there is a game going on, to some degree. If you understand the "tricky questions" game, then you know to expect counter-intuitive answers. Even so, many people perform abysmally - about 50% of college students get the ball and bat problem wrong. The traditional explanation is that there is a trait ("tendency to reflect") that problems like these measure and the tendency to reflect is distinct from the big five personality traits, intelligence, and many other psychological constructs. At the same time, performance on the three-question CRT has been correlated with all kinds of interesting outcomes: risk-taking behavior, belief in the paranormal, vulnerability to decision-making biases, moral judgments, etc.

Later research has cast doubt that the CRT measures a "pure" tendency to reflect. Numeracy, reading ability, and other psychological processes – unsurprisingly – are folded into CRT performance. And variation in how people answer the test questions suggest that some people don't have to inhibit an intuitively wrong answer at all; that is, they don't have to reflect to do well on the test. But there is something compelling about the simplicity and power of the CRT.

Closely connected with the test is the idea that if you just got people to stop and think for a second, they would be better problem solvers. Or better decision-makers.

I think about the CRT often in the contexts of conversations. Imagine you had a conversational partner who always said the first, most intuitive thing to come to his mind. We don't have to imagine this, necessarily, because life on the internet offers plenty of examples.

It's not just that people seem to want to reach judgments very quickly – and that people do reach judgments very quickly – it's that the infrastructure of our internet conversations promotes instant, non-thinking reactions.

It would be interesting to look at data on how quickly people comment after watching a video or reading an article. Considering the research on how willing people are to share news articles without reading them, I wouldn't be surprised at all to find that many people – in many circumstances – would be willing to comment on articles they haven't read. Or comment on articles where they had only read the headline. Or comment on articles where they had only read someone else's comment and it made them really mad.

Perhaps being a content creator has made me hyper-aware of this reactive culture. There are many thoughtful comments on my videos. But sometimes commenters will react to what they think you said, rather than what you actually said. Or they will focus on an offhand comment you made, rather than the general point you're trying to make. Or they will react to what the title may have implied, rather than what was directly stated. Or they will use your content to work out some seemingly unrelated personal bugaboo.

Under these circumstances, I'd like to tell them to take a closer look at the content. Or to take a moment between reading (or listening) and commenting. To just wait a beat.

There is a world, I think, where reflection could just be the thing that everyone does. The tendency to react without thinking is not necessarily an enduring personality trait – at least, I haven't seen much evidence that it is – it has more to do with culture, social norms, technological infrastructure, habit, circumstance, and opportunity.

References

The classic paper on the CRT:

Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic perspectives, 19(4), 25-42. As of this writing, available here: https://pubs.aeaweb.org/doi/pdf/10.1257/089533005775196732

A recent exploration of the components of performance of the CRT is below. Incidentally, this piece is also a broadside against the traditional interpretation of the CRT I outlined above. Among their interesting findings: a multiple-choice version of the CRT test, where the incorrect, intuitive answers were not answer choices still resulted in a lot of incorrect answers. This makes the CRT seem to be measuring something closer to "the ability/tendency to reframe a problem," in addition to numeracy.

Patel, N., Baker, S. G., & Scherer, L. D. (2019). Evaluating the cognitive reflection test as a measure of intuition/reflection, numeracy, and insight problem solving, and the implications for understanding real-world judgments and beliefs. Journal of Experimental Psychology: General, 148(12), 2129–2153. https://doi.org/10.1037/xge0000592

On "sharing without reading," see:

Ward, A. F., Zheng, J., & Broniarczyk, S. M. (2023). I share, therefore I know? Sharing online content‐even without reading it‐inflates subjective knowledge. Journal of Consumer Psychology, 33(3), 469-488. As of this writing, you can find an SSRN version of the paper here: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4132814

A paper on the "miserly processing" interpretation of CRT performance, correlating CRT performance with vulnerability to cognitive heuristics and biases. The miserly processing interpretation is that CRT measures the tendency to avoid involved cognitive processing (rather than cognitive ability, which is a measure of what you could process):

Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & cognition, 39(7), 1275-1289.

Another by the same team, which expands the three-question CRT to seven:

Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An expansion of the Cognitive Reflection Test. Thinking & reasoning, 20(2), 147-168.