Day 3: Thursday 10 October

STATISTICAL METHODS FOR DIAGNOSTIC ACCURACY IN MEDICAL RESEARCH

This one day course is predominately designed for researchers who want to analyse their own diagnostic accuracy data. Typically, this will be analysis on results from research carried out in an early development/exploratory phase but the methods covered will also be applicable to research in an advanced stage where tests are studied in a clinical setting. The course content is geared towards teaching “how to” rather than “why” and will make extensive use of practical sessions so that participants can gain hands-on experience of analysing diagnostic accuracy data. In the practical sessions, participants will use the free statistical software R (https://cran.r-project.org/) for completing the analyses.

Prerequisites.

Participants should have a basic understanding of statistics up to the level of confidence intervals and p values. A rudimentary knowledge of the basics of diagnostic accuracy would be advantageous but it is not necessary as we will revise the basics in the first session. Participants will be expected to bring their own laptops and have R and RStudio installed.

By taking this course:

The course will consist of four sessions with demonstrations and practical exercises.  Participants will revise their understanding of common measures of diagnostic accuracy, learn how to calculate measures of diagnostic accuracy from data and quantify uncertainty in their estimates. Then learn when and how you should use ROC analysis, how to compare two diagnostic tests and finish with methods for finding optimal thresholds.

Course content in detail

1. Single summary estimates of diagnostic accuracy.

• Common measures of DA - sensitivity, specificity,
• Other summary measures- Diagnostic odds ratio, Youden’s index. Etc.
• LR+ and LR-
• Bayes theorem, PPV and NPV and prevalence
• Simple sample sizes calculations.

2. Receiver operating characteristic

• ROC space and ROC curves (pros and cons)
• Area under the curve measures (partial area under the curve)
• Smooth (parametric) ROC curves

3. Hypothesis tests for comparing the accuracy of two tests

• Tests based on either sensitivity or specificity
• Comparing two ROC AUC’s curves
• Paired and non-paired designs

4. Methods for finding optimal thresholds.

• Methods based on sensitivity and specificity alone
• Optimal thresholds based on costs (weighting for false pos and false negs)
• Graphical summaries (Net-benefit measure etc)