As we make clinical decisions every day, we assess probabilities in a subjective fashion. And in doing so, we tend to fall into very predictable traps \u2014 traps we can get better at avoiding if we learn about how they ensnare us. That requires familiarizing ourselves with a bit of history.<\/p>\n
Several decades ago Casscells and colleagues published the results of an interesting experiment (N Engl J Med<\/em> 1978; 299:999). They asked 60 Harvard medical students, residents, and attending physicians the following question: \u201cIf a test to detect a disease whose prevalence is 1\/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming that you know nothing about the person\u2019s symptoms or signs?\u201d<\/p>\n Interestingly, only 11 of 60 (18%) participants gave the correct answer. What is your answer? If you guessed 95%, you are in good company \u2014 but wrong. Twenty-seven of the 60 (45%) gave that incorrect response. The correct answer is actually 2%.<\/p>\n How can we explain such poor performance on a question that seems straightforward and very clinically relevant? Cognitive psychologists would say the high error rate was because the question expressed the false-positive rate as a percentage rather than as a natural frequency. Gigerenzer, Cosmides and Tooby, and others say that the mind has evolved to understand natural frequencies and that our intuition frequently fails when probabilities are presented in other formats such as percentages (Med Decis Making<\/em> 1996; 16:273; Cognition <\/em>1996; 58:1).<\/p>\n To test this theory, Cosmides and Tooby performed a second set of experiments. First they posed Casscells\u2019 original question to a group of Stanford undergraduates. They found that this group fell into the same trap as the Harvard group: Only 3 of 25 (12%) gave the correct answer. Then they phrased the question in a different way using natural frequencies: \u201cOne out of every 1000 Americans has disease x. A test has been developed to detect when a person has disease x. Every time the test is given to a person who has the disease, the test comes out positive. But sometimes the test also comes out positive when it is given to a person who is completely healthy. Specifically, out of every 1000 people who are perfectly healthy, 50 of them test positive for the disease\u2026\u201d<\/p>\n When the question was asked using natural frequencies, 19 of 25 participants (76%) got the correct answer. These and other experiments demonstrate that our intuition can be set up to succeed or to fail by how questions are framed. Simply changing the way numbers are formatted can have a dramatic effect on how well we reason.<\/p>\n I suspect that many subjects in the original experiment did not see a quick answer, so they guessed that the true positive rate was the complementary probability of the false positive rate of 5%. The percentage format encourages this error because it causes people to lose track of what the denominator, or reference class, represents. Probabilities presented as natural frequencies force people to recognize that the false-negative rate and the true positive rate have different denominators and are not complementary probabilities. It then becomes more obvious that one has to create either a 2×2 table or a branching algorithm to determine the true positive rate based on the incidence of disease and the false-positive rate.<\/p>\n There are other ways our intuition can be fooled by how numbers are framed. For example, expressing treatment effects as relative risk reduction rather than absolute risk reduction or number needed to treat can exaggerate a treatment effect. That type of exaggeration can be used as a sales gimmick by those who are trying to push a product. Any shopper who has seen products priced at $9.99 rather than $10.00 knows that our intuition can be tricked by how numbers are presented.<\/p>\n The Casscells experiment also illustrates a fallacy known as base-rate neglect, which I will discuss in the next post in this blog series. \u00a0For now, here\u2019s a brief probability problem\u00a0that comes in part from Reichlen et al. (N Engl J Med<\/em>\u00a02009; 361:858).\u00a0See if you can solve it.\u00a0I present it here as a multiple-choice question, for purposes of illustration.\u00a0Choose your answer, and then discuss your thinking about the question and about probability fallacies in general in the comments. But please don\u2019t give away the answer to other readers.<\/p>\n An emergency department decides to perform serum troponin testing on all patients with any type of chest complaint. They suspect\u00a0that the incidence of documented myocardial infarction in this subgroup is only about 1%, but they are determined not to miss a single MI.\u00a0They choose a high-sensitivity troponin assay with\u00a0a sensitivity of 95% and a specificity of 80%.\u00a0For\u00a0one of these\u00a0patients with a positive troponin, what\u00a0are the odds of having an MI?<\/em><\/p>\nSorry, there are no polls available at the moment.\n","protected":false},"excerpt":{"rendered":" As we make clinical decisions every day, we assess probabilities in a subjective fashion. And in doing so, we tend to fall into very predictable traps \u2014 traps we can get better at avoiding if we learn about how they ensnare us. That requires familiarizing ourselves with a bit of history. Several decades ago Casscells […]<\/p>\n","protected":false},"author":379,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[949,950],"class_list":["post-11014","post","type-post","status-publish","format-standard","hentry","category-general","tag-decision-making","tag-heuristics"],"_links":{"self":[{"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/posts\/11014","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/users\/379"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/comments?post=11014"}],"version-history":[{"count":0,"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/posts\/11014\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/media?parent=11014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/categories?post=11014"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.nejm.org\/cardioexchange\/wp-json\/wp\/v2\/tags?post=11014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}