GCSP · Talent / Research
2026
Prevalence-shift evaluation for pediatric CXR triage: a 12% screening model can miss more cases at 35% surge prevalence unless you sweep thresholds and calibration.
ClinRad-Bench is my Talent/Research anchor: an evaluation framework for pediatric chest X-ray triage under prevalence shift, not a clinically deployed model.
A model tuned at 12% prevalence can look strong on paper and miss more cases at 35% surge prevalence because the same threshold trades off against a different base rate. The repo stress-tests that shift at τ=0.58 with 200 bootstrap resamples on synthetic cohorts.
What I worked on during 2026. Hover underlined terms for quick definitions.
ClinRad-Bench is a GCSP reflective capstone distilled from Pneumonia detectionreal radiograph training on 368 held-out images and Module 11compound-error benchmark work. It is not a clinically deployed model. Cohort images come from a synthetic DGP; bootstrap CIs, ECE, permutation tests, and rubric logic are real and pytest-gated.
Pneumonia detectionEfficientNetV2B0 pipeline at 83% validation accuracy on 368 held-out radiographs taught me clinical metrics. Module 11Visual Compounding Problem benchmark: errors multiply through the perception-action loop. Both showed the same gap: headline accuracy hides prevalence shift, calibration drift, and compounding failure.
A triage model tuned on a 12% screening population can miss more real cases when deployed into a 35% surge population, because the same threshold trades off against a different base rate. ClinRad-Bench makes that failure mode explicit with prevalence sweeps at 2%, 12%, and 35%, selective prediction at τ=0.58, and 200 bootstrap resamples to bound sensitivity swings.
Running `scripts/run_benchmark.py --sweep` writes comparison_table.mdCross-profile table with bootstrap confidence intervals and a screening-vs-surge permutation p-valueNonparametric test whether missed-case rates differ between prevalence profiles.
Every number is labeled synthetic in the README Limitations section. The methodology is real; the CXR images are a DGPData-generating process: code that produces labeled features with known ground truth.
Rigorous evaluation only matters if a lab can afford to run it. That pushed me to BenchHostMetered robotics benchmarking platform: unit economics, idempotent submit, and certificate verification as the entrepreneurship answer to the same Grand Challenge.
Headline accuracy without prevalence sweeps and calibration checks is a deployment risk, not a result. Discovery tools are not credible until their evaluation methodology is explicit.