04. Diagnostic workshop: why predictive values change
Lesson
LEARNING OBJECTIVES
Reproduce diagnostic metrics in two hypothetical populations, explain the role of underlying disease frequency, and avoid transferring predictive values without examining assumptions.
LABEL THE COUNTS
For this workshop, rows are test positive/negative and columns are reference-standard disease present/absent:
a = true positive; b = false positive; c = false negative; d = true negative.
Check the labels and complete counts before interpreting any percentage.
WORKED POPULATION A
Suppose 1,000 people are assessed, 100 have the reference-standard condition and 900 do not. Under the hypothetical test performance below:
a = 90, b = 45, c = 10, d = 855.
Sensitivity = 90/100 = 90%.
Specificity = 855/900 = 95%.
Positive predictive value = 90/135 = 66.7%.
Negative predictive value = 855/865 = approximately 98.84%.
WORKED POPULATION B
Now imagine a different population of 10,000 people, of whom 100 have the condition. Assume the same sensitivity and specificity for teaching:
a = 90, b = 495, c = 10, d = 9,405.
Sensitivity remains 90% and specificity 95%.
Positive predictive value = 90/585 = 15.4%.
Negative predictive value = 9,405/9,415 = approximately 99.89%.
The condition frequency is 10% in A and 1% in B. The lower frequency in B means that false positives comprise a much larger share of positive results under these assumptions. Neither the name of the test nor identical sensitivity/specificity guarantees the same predictive value.
WHAT THE EXAMPLE DOES AND DOES NOT SHOW
These are invented complete tables, not clinical recommendations or measured local accuracy. The example assumes stable sensitivity and specificity, which should not be assumed automatically when the population, disease spectrum, threshold or measurement process changes.
Do not transfer predictive values from a deliberately selected case/control diagnostic sample to a screening population without a defensible approach. Also assess uncertainty, consequences of false results, reference-standard limitations and the place of the test in an actual pathway.
YOUR TASK
Open Research lab → Epidemiology 2 × 2, choose Diagnostic accuracy, and reproduce both tables. Save all four metrics with the counts, labels and population assumptions.
https://aces-af.org/learn/index.php?r=tool&tool=table
Write a 120-word explanation for a nontechnical colleague of why the proportion of true cases among positive results differs. Then list three reasons the same assumed sensitivity/specificity might fail to transfer to a new study setting.
SELF-CHECK AND FEEDBACK
1. Which metric starts with everyone who tested positive? Positive predictive value.
2. Did lower disease frequency change sensitivity by arithmetic in this example? No; sensitivity was held fixed by assumption.
3. Does an undefined result from a zero denominator mean zero accuracy? No; the requested metric cannot be calculated from that denominator.
PORTFOLIO OUTPUT
Two labeled result records, a plain-language explanation and an applicability-assumptions list.
FURTHER READING
FDA: Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/statistical-guidance-reporting-results-studies-evaluating-diagnostic-tests-guidance-industry-and-fda
Reproduce diagnostic metrics in two hypothetical populations, explain the role of underlying disease frequency, and avoid transferring predictive values without examining assumptions.
LABEL THE COUNTS
For this workshop, rows are test positive/negative and columns are reference-standard disease present/absent:
a = true positive; b = false positive; c = false negative; d = true negative.
Check the labels and complete counts before interpreting any percentage.
WORKED POPULATION A
Suppose 1,000 people are assessed, 100 have the reference-standard condition and 900 do not. Under the hypothetical test performance below:
a = 90, b = 45, c = 10, d = 855.
Sensitivity = 90/100 = 90%.
Specificity = 855/900 = 95%.
Positive predictive value = 90/135 = 66.7%.
Negative predictive value = 855/865 = approximately 98.84%.
WORKED POPULATION B
Now imagine a different population of 10,000 people, of whom 100 have the condition. Assume the same sensitivity and specificity for teaching:
a = 90, b = 495, c = 10, d = 9,405.
Sensitivity remains 90% and specificity 95%.
Positive predictive value = 90/585 = 15.4%.
Negative predictive value = 9,405/9,415 = approximately 99.89%.
The condition frequency is 10% in A and 1% in B. The lower frequency in B means that false positives comprise a much larger share of positive results under these assumptions. Neither the name of the test nor identical sensitivity/specificity guarantees the same predictive value.
WHAT THE EXAMPLE DOES AND DOES NOT SHOW
These are invented complete tables, not clinical recommendations or measured local accuracy. The example assumes stable sensitivity and specificity, which should not be assumed automatically when the population, disease spectrum, threshold or measurement process changes.
Do not transfer predictive values from a deliberately selected case/control diagnostic sample to a screening population without a defensible approach. Also assess uncertainty, consequences of false results, reference-standard limitations and the place of the test in an actual pathway.
YOUR TASK
Open Research lab → Epidemiology 2 × 2, choose Diagnostic accuracy, and reproduce both tables. Save all four metrics with the counts, labels and population assumptions.
https://aces-af.org/learn/index.php?r=tool&tool=table
Write a 120-word explanation for a nontechnical colleague of why the proportion of true cases among positive results differs. Then list three reasons the same assumed sensitivity/specificity might fail to transfer to a new study setting.
SELF-CHECK AND FEEDBACK
1. Which metric starts with everyone who tested positive? Positive predictive value.
2. Did lower disease frequency change sensitivity by arithmetic in this example? No; sensitivity was held fixed by assumption.
3. Does an undefined result from a zero denominator mean zero accuracy? No; the requested metric cannot be calculated from that denominator.
PORTFOLIO OUTPUT
Two labeled result records, a plain-language explanation and an applicability-assumptions list.
FURTHER READING
FDA: Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/statistical-guidance-reporting-results-studies-evaluating-diagnostic-tests-guidance-industry-and-fda
My study notes
Stored only in this browser. Do not enter patient information.Saved in this browser. No account needed.