The alert threshold: balancing sensitivity and specificity

The correct answer: D. Specificity increases and sensitivity decreases, so there are fewer false alarms but more missed cases

✅ Why this answer

Raising the threshold means the model alerts only on cases with higher scores. Two things happen together:

• Fewer patients without the condition cross the threshold, so false alarms drop and specificity rises.
• Some real patients with medium scores do not cross the threshold, so they are missed and sensitivity falls.

❌ Why the other options are wrong

• (A): changing the threshold alone does not improve both together; that needs a better model.
• (B): this is the effect of lowering the threshold, not raising it.
• (C): the threshold changes who is classified as positive, so sensitivity and specificity change.

💡 Key concept

• Sensitivity: of the patients who really have the condition, how many did the model catch? Low sensitivity means missed cases (false negatives).
• Specificity: of the patients who do not have it, how many did the model leave alone? Low specificity means false alarms (false positives).

Moving the threshold trades one for the other: raising it increases specificity and lowers sensitivity, and lowering it does the opposite.

Choosing the threshold is a clinical and operational decision, not just a technical one: what does a missed case cost? And what does a false alarm cost in the team’s time and trust in the system? So it is set with physicians, and reviewed after go-live based on actual results.

🔗 Related facts and questions

• The 2×2 table: true positive (TP), false positive (FP), true negative (TN) and false negative (FN). From it: sensitivity = TP ÷ (TP + FN), specificity = TN ÷ (TN + FP), and positive predictive value (PPV) = TP ÷ (TP + FP): of those the model alerted on, how many really have the condition?
• The effect of a rare condition: at a fixed threshold, sensitivity and specificity do not depend on how common the condition is, but PPV changes with prevalence. Example: 1,000 patients, 1% of whom will deteriorate, and a model with 90% sensitivity and 90% specificity. It catches 9 of the 10 real cases, and alerts wrongly on 99 of the 990. So PPV = 9 ÷ 108, only about 8%.
  Practice question: why do nurses complain about a model with 90% sensitivity and 90% specificity? → Because most of its alerts are false when the condition is rare.
• The ROC curve and the area under it (AUC): it plots sensitivity against the false positive rate (1 − specificity: the share of patients without the condition who are wrongly alerted on) at every possible threshold. An AUC of 0.5 means random guessing, and 1.0 means perfect discrimination. Changing the threshold moves you along the same curve; improving the model lifts the whole curve.
• Which way do we move the threshold? If missing a case is dangerous and checking it is cheap (sepsis screening, for example), we lower the threshold and accept more alerts. If the next step is costly or risky, we raise it.
  Practice question: a sepsis screening model misses many cases. Which way is the threshold moved? → Down (higher sensitivity).
• After go-live: the override rate, the response time to alerts and their effect on the clinical outcome are monitored. The threshold is set again if the patient population or the way of documenting changes, because a model’s performance drifts over time (model drift).
• Calculating positive predictive value: "Positive predictive value (PPV)" (in the full bank).
• Alert governance to fight alert fatigue: a related question in the full bank.