# Why Everyone Needs to Study Statistics, Even Doctors

It seems kind of important that a doctor can correctly interpret the following scenario:

• A 40-year old woman has a positive mammography in a routine screening.

The doctor is told the following about scanning for breast cancer:

• 1% of women at age forty who participate in routine screening have breast cancer.
• 80% of women with breast cancer will get positive mammographies (which means there are 20% false negative).
• 9.6% of women without breast cancer will also get positive mammographies (known as a 9.6% false positive)

What is the probability that she actually has breast cancer?

Only 15% of the doctors surveyed estimated the correct probability; most doctors estimated the probability to be between 70% and 80%, which is wildly incorrect.

That’s kind of scary.

This is a classical example of where Bayes’ Theorem should be used.

The problem, according to an editor at Read the Sequences, is that Bayesian reasoning is very counterintuitive. People do not employ Bayesian reasoning intuitively, find it very difficult to learn Bayesian reasoning when tutored, and rapidly forget Bayesian methods once the tutoring is over. This holds equally true for novice students and highly trained professionals in a field.

Here is Bayes Theorem:

OR:

Clear as mud, right?

No wonder people claim that it is not intuitive.

But let’s try to apply it to the mammography screening.

We are trying to solve for the probability of having cancer, given a positive screening. This is P(A|B).

Here is what the other terms mean:

• P(A) = the overall probability of cancer; in this example, this probability is one percent, or .01
• P(B|A) = probability of positive screening, given that you have cancer; in this example, this probability is 80 percent, or .80
• P(B) = the overall probability of getting a positive result; this is the most challenging number to calculate.

To calculate the P(B), let’s reword the original scenario.

Let’s assume that 1,000 women of age 40 take the mammography screening. Of these 1,000 women, 1% have cancer, which is 10 women. Of these 10 women, the mammography test will give the correct result for 80% of them, or 8 women. Of the 990 women in this group who do not have cancer, the mammography test will falsely say that 9.6% of the women do have cancer, which is approximately 95 women.

Thus, in total, out of the 1,000 women who take the mammography test, 103 (8 + 95) will test positive, or about a 10.3 percent chance, or .103 (103/1000). This is P(B).

Let’s put it all together now, using the first formula in the blue box above:

P(B|A) * P(A) = =.8*.01 = .008 (in English, these are the odds of testing positive, given that you have cancer times the chance of having cancer)

P(B) = .103 (from above) (in English, this is the probability of testing positive)

Dividing one by the other, we get P(A|B) =  .008/.103 = .078, or 7.8%

We could have also calculated the denominator using the alternative formula shown above.  We already know that

P(B|A) * P(A) = =.8*.01 = .008 these are the odds of testing positive given that you have cancer times the chance of having cancer

P(B|not A) * P(not A) = .096 * .99 = .095 these are the odds of testing positive given that you do not have cancer times the chance you do not have cancer.

Thus,

P(B|A) * P(A) plus P(B|not A) * P(not A) = .008 + .095 = .103 (in English, this is the probability of testing positive)

So to answer the original question: if a woman tests positive for cancer, there is only a 7.8% chance that she has cancer; dramatically different than the 70 to 80% probability that doctors had estimated.

Perhaps an easier way to think of this would be to go back to the example of 1,000 women.

We have shown 103 of these women will test positive; however, only 8 of them will have cancer, 95 will not have cancer. Thus the probability of having cancer, given that you tested positive is 8/103, or 7.8%.

As noted above, this may not be an intuitive result.

When you are told that 80% of women with cancer will test positive, that is not the same question as asking that if a woman tests positive, what are her chances of having cancer.

Since there are so many women more women who do not have breast cancer, and some of those women will test positive, that creates a much larger population of women who test positive and don’t have cancer (95) compared to the number who test positive and have cancer (8).

In conclusion, if you made it this far, then:

• you are a stat nerd like me (less than one percent chance)
• you need to find better reading material (greater than 99 percent chance)

I also wonder, given that a person has read this far, what the odds are that they will hit the like button…

## 76 thoughts on “Why Everyone Needs to Study Statistics, Even Doctors”

1. Maybe this is why it’s a good idea to get a second opinion. But don’t go to a doctor for your second opinion, go to a mathematician.

Liked by 5 people

1. And maybe this is why you sometimes hear on the intercoms at hospitals, “Dr. So-and-so to the ER, stat!”

Liked by 2 people

2. 🙂 I never thought of that; who knew stat people were in such demand!

Liked by 1 person

2. I pushed “like” even if I kind of skipped the numbers part. LOL!
I agree that its scary about those Drs giving such a crazy high answer! I don’t do Math but I even knew the percent was much lower.
I also was thinking of how ironic on this timing of your post. I go soon for my appt to test all those percentages.

Liked by 1 person

1. And all I want is the money you offered to pay your readers. LOL! 😉
Thanks!

Liked by 1 person

2. once I monetize my blog, I will share the wealth… 🙂

Like

3. I don’t expect my Doctor to understand statistics or mathematics. It’s why our Doctors can plug data into an algorithm and get the answers they need. That’s a better tool for them. Same as I don’t expect a statistician to know what the signs of breast cancer are.

Liked by 3 people

4. Hopefully, competent radiologists and oncologists who deal with breast cancer will have a good idea (intuitively) of the likelihood that a 40-year-old woman with a positive mammography actual has breast cancer without performing a mathematical calculation and will use additional tests and analysis before announcing a diagnosis. I wonder if the doctors surveyed were radiologists and oncologists who deal with breast cancer. If they were, that’s very concerning. Even so, it is surprising that MDs may be as bad at math as I am.

Liked by 2 people

1. you would hope they are versed in such matters; but this was just one example of how many people, even very bright people, do not understand some key statistical principles.

Liked by 1 person

1. Yep. I get your point. MDs at one time were even good at mathematical principles. If you asked lawyers a similar question, the percentage who could construct the right equation and get the right answer would be about 1%.

Liked by 1 person

2. I think many times, at least when I was in school, people who went on to med school were usually the smartest people in the class. I don’t think I could say the same thing about those who went to law school…

Like

5. I actually didn’t mind my college Stat classes when I was studying for my BSN. I preferred my statistical application class over the actual general stats though!

Liked by 1 person

6. petespringerauthor says:

I used to think math was cool. Are you trying to change my opinion?

Liked by 3 people

7. I got overwhelmed early on, why we need both kinds of people and a way to connect them.

Liked by 1 person

8. You explained this perfectly (I did have to read this twice though). My “intuitive” thought when I read the initial numbers was somewhere under 10%, but It may be a fluke and I may have already forgotten everything. Great post!

Liked by 1 person

1. thanks, glad you enjoyed it. And nice job estimating the percentage – you’re a natural!

Liked by 1 person

9. Wow not gonna lie a lot of this went over my head LOL I’m definitely not a math person nor did I ever take stats in school but you bring up a good point about math being used in different ways in every profession (even ones you don’t think would need it)!

Liked by 2 people

1. I think stats is more important to take in high school than calculus, but it seems most schools don’t do it that way…

Liked by 1 person

Odds of me hitting like? 100%

Did I like my stats class? A resounding NO.
Do I like or excel at math. Also a resounding NO.

What are the odds I followed the logic of this post?
You’re the stats guy. I’ll let you run the numbers.

Liked by 2 people

11. Ok you totally lost me. Not getting through stats is the reason I don’t have a business degree. I got through everything else though. Not sure I would want a doctor reciting a bunch of statistics at me if I was worried about cancer. It would be like dealing with a robot. Oh wait my doctor is like a robot only a robot would probably remember me and why he said to come back in two weeks.

Liked by 2 people

1. I think losing people while trying to explain something is one of my specialties 🙂

and yes, I would want a doctor more skilled in cancer treatment than stats, but hopefully he or she is getting help with such calculations from somewhere…

soundsl like you need a new doctor…

Liked by 1 person

12. I tried following that calculation and your reasoning, but it all went tits up…

Liked by 1 person

13. My brain experienced statistical overload, but your discussion did make sense. I won’t need a second opinion about clicking “Like” for this post. Done!!

Liked by 1 person

1. You managed to ignite some of my memories from taking a Stats class in college. And . . . I stayed awake!

Liked by 1 person

2. you must have had a lot of caffeine in you!

Like

1. no need to apologize; I’m used to losing people all the time when I try to explain something 🙂

Liked by 1 person

1. It’s the numbers that lost me. Both my children are great at maths. One did a phd in computer science. The other one did a phd in something else but lectures in stats (amongst other things) at university. They didn’t get it from me. 😂

Liked by 1 person

2. two PhDs! quite impressive…I’m sure they picked up a love of reading from you… 🙂

Liked by 1 person

3. I think they did! I didn’t really give them much choice. 😂💖

Liked by 1 person

14. Well it seems I’m a stats nerd too. I calculated the rounded 8% using the second method with is intuitive [for me anyway]. Great post.

Liked by 1 person

1. thanks, Robbie. I like the second approach as well, since it is more intuitive. I would have picked you as a stats nerd 🙂

Liked by 1 person

15. I had to take statistics for a few months in high school and obviously it went great considering I am now a history major. I couldn’t quite keep up with the maths because it’s very late and also it’s maths but I do see your point.

Liked by 1 person

1. Thanks and it’s not started yet but will be starting soon!

Liked by 1 person

2. well good luck when school does start; we’ve been in school for two weeks…

Liked by 1 person

3. Thanks and hope your classes are going well- it must be nice to see your students again!

Liked by 1 person

16. Oh dear..at least the stats (somewhere) were correct I wasn’t the only one who thought it was a clear as mud…Maths and stats I think I flunked those in school then went on to work in a bank..funny enough I passed that entrance exam ..must have paid them enough…lol

Liked by 1 person

1. all that knowledge is in there somewhere! and I thought there would be some less than enthusiastic responses to this post 🙂

Liked by 1 person

17. If I had taken Statistics in college, I would have flunked. I now teach math in a fun way so children won’t hate it like I did.

Liked by 1 person

1. I thnk it is critical to instill a love of learning in children at the youngest age possible. Kudos to you for making that a priority!

Liked by 1 person

1. Yes! YES!! I wish you could see me subitizing, adding and subtracting using rocks, estimating with a jar full of shells…

Liked by 1 person

1. They are so fascinating. As part of my psych degree education I had to take, “Research Methods.” It was a requirement for all psych majors. It was the closest thing to a religious experience I ever had. The world made so much sense. We also had to write three research papers in APA style. The first one was terrifying-for all of us. But, I aced the class. I still have my textbooks from 1983.

Liked by 1 person

2. My wife had the same class at her college; challenging but so rewarding…

Liked by 1 person

3. Where did she go? I graduated from University of South Florida with a psych BA and minors in mass communications, snd photojournalism. Higher Ed is one of my vices, like chocolate and coffee.

Liked by 1 person