Diagnostic Evidence Workshops 2022
Wednesday, 22 June 2022 to Friday, 24 June 2022, 10am - 4pm
St Hugh's College Oxford, St Margaret's Road, Oxford, OX2 6LE
Hosted by NIHR Community Healthcare MICClick here to book
A three-day workshop aimed at clinicians, researchers, the diagnostics industry, and those working in guideline development and regulatory affairs.
The NIHR Community Healthcare MIC is pleased to offer a series of workshops for 2022 that will build on the educational programme previously delivered by NIHR DEC Oxford. Running sequentially, the workshops cover:
- Workshop 1: An introduction to evaluation of diagnostics
- Workshop 2: Cost-effectiveness analysis of diagnostic tests for development and adoption
- Workshop 3: Statistical methods for diagnostic accuracy in medical research
These workshops can be taken individually or combined depending on your requirements.
Each workshop includes lunch and refreshments and is £195 to attend. You are not obliged to attend all three; you can attend one (£195), two (£390) or three (£585) workshops to suit your requirements.
Taught by academic clinicians, health economists and statisticians together with external experts from NICE and the NIHR, these workshops aim to equip you with the necessary skills to evaluate IVD and non-IVD diagnostic technologies to support the development of evidence portfolios for regulatory approval (UKCA and the CE-IVDR).
This series is aimed at all professionals who wish to advance their understanding of diagnostic test evaluation, including those working in industry, academia, research funding and regulatory affairs organisations.
- Associate Professor Gail Hayward, University of Oxford and Deputy Director, NIHR Community Healthcare MedTech and In Vitro Diagnostics Co-operative.
- Dr Ian Newington, National Institute for Health Research i4i Programme and the SBRI Programmes.
- Professor Chris Hyde, University of Exeter and NICE Diagnostic Advisory Committee
INTRODUCTION TO EVALUATION OF DIAGNOSTICS
The opening workshop of the series will be taught by front-line diagnostics researchers and will provide participants with a basic overview of the components which one might employ for the evaluation of diagnostic tests, including basic health economics and statistics, searching for evidence, and pragmatic study design. Tutors will discuss some of the challenges faced by developers of diagnostic tests and potential sources of funding for development and evaluation projects respectively.
There are no prerequisites for attendance. This workshop has been formulated so that it is accessible to all who have an interest in gaining a basic understanding of the evaluation of diagnostic tests.
The course will consist of three sessions of seminars separated by lunch and coffee breaks. Some of the seminars will incorporate short practical exercises.
Course content (subject to minor changes)
1. Tests as part of a clinical pathway
2. Introduction to study design for pragmatic evaluation of diagnostics
3. Basic statistics for diagnostic medicine
4. Basic health economics for diagnostic evaluation
5. Challenges of diagnostic test development – the industry perspective
6. Searching for existing evidence to support regulatory approval and other purposes
7. NIHR funding streams
COST-EFFECTIVENESS ANALYSIS OF DIAGNOSTIC TESTS FOR DEVELOPMENT AND ADOPTION.
This one day course aims to explain how cost-effectiveness analysis of diagnostic tests is conducted and used in the research, development and evaluation of diagnostics. It is aimed at those involved in the development and evaluation of diagnostics, in both academic and commercial settings.
Participants should have a basic understanding of measures of diagnostic accuracy and clinical effectiveness, as well as basic descriptive statistics (means, ranges). Previous knowledge of cost-effectiveness concepts would be an advantage, but these are not necessary as they will be revised in the first session. For the practical session, participants will need a laptop with Microsoft Excel installed.
By taking this course
Participants will revise the core concepts underlying cost-effectiveness analysis, including incremental cost-effectiveness ratios. They will learn how diagnostic tests can be evaluated using cost-effectiveness methods, and will get practical experience of working with a pre-built decision model and performing sensitivity analysis to understand how cost-effective a diagnostic would be in different settings. They will also learn about the diagnostic evaluation evidence requirements used by NICE, and consider contemporary issues related to cost-effectiveness analysis of diagnostics.
Course content (subject to minor change)
- Introduction to cost-effectiveness for diagnostics
- Evaluating the cost-consequences of testing
- Quality-adjusted life years
- Incremental cost-effectiveness ratios
- Linking test results to outcomes
- Introduction to decision models
2. Generating and using cost-effectiveness evidence for diagnostics
- Identifying which evidence is needed
- Revising evidence sources
- Selecting parameters
- Model assumptions
3. Evidence for implementation in routine clinical practice – NICE evaluations
4. Practical: Identifying evidence, decision modelling, and evaluating determinants of cost-effectiveness
- Search strategies
- Performing deterministic sensitivity analysis
- Identifying threshold values in early evaluations
5. Outstanding issues in cost-effectiveness of diagnostics: AMR, diagnostic delay and going beyond the QALY.
STATISTICAL METHODS FOR DIAGNOSTIC ACCURACY IN MEDICAL RESEARCH
This one day course is predominately designed for researchers who want to analyse their own diagnostic accuracy data. Typically, this will be analysis on results from research carried out in an early development/exploratory phase but the methods covered will also be applicable to research in an advanced stage where tests are studied in a clinical setting. The course content is geared towards teaching “how to” rather than “why” and will make extensive use of practical sessions so that participants can gain hands-on experience of analysing diagnostic accuracy data. In the practical sessions, participants will use the free statistical software R (https://cran.r-project.org/) for completing the analyses.
Participants should have a basic understanding of statistics up to the level of confidence intervals and p values. A rudimentary knowledge of the basics of diagnostic accuracy would be advantageous but it is not necessary as we will revise the basics in the first session. Participants will be expected to bring their own laptops and have R and RStudio installed.
By taking this course:
The course will consist of four sessions with demonstrations and practical exercises. Participants will revise their understanding of common measures of diagnostic accuracy, learn how to calculate measures of diagnostic accuracy from data and quantify uncertainty in their estimates. Then learn when and how you should use ROC analysis, how to compare two diagnostic tests and finish with methods for finding optimal thresholds.
Course content in detail (subject to minor change)
1. Single summary estimates of diagnostic accuracy.
- Common measures of DA - sensitivity, specificity,
- Other summary measures- Diagnostic odds ratio, Youden’s index. Etc.
- LR+ and LR-
- Bayes theorem, PPV and NPV and prevalence
2. Receiver operating characteristic
- ROC space and ROC curves (pros and cons)
- Area under the curve measures (partial area under the curve)
- Smooth (parametric) ROC curves
3. Sample size calculations for diagnostic accuracy studies
4. Methods for finding optimal thresholds.
- Methods based on sensitivity and specificity alone
- Optimal thresholds based on costs (weighting for false pos and false negs)
- Graphical summaries (Net-benefit measure etc)