Testing an artificial intelligence algorithm for detecting newborn hip dysplasia on ultrasound scans
| ISRCTN | ISRCTN49436239 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN49436239 |
| IRAS number | 316325 |
| Secondary identifying numbers | IRAS 316325 |
- Submission date
- 18/06/2024
- Registration date
- 19/06/2024
- Last edited
- 19/06/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Neonatal Diseases
Plain English Summary
Background and study aims
The study aims to evaluate the impact of an AI algorithm on the diagnostic accuracy, speed and confidence of healthcare professionals in diagnosing developmental dysplasia of the hip (DDH) on ultrasound scans. The study will involve 10 readers, who will interpret 70 ultrasound scans of baby hips, with and without AI assistance. The scans will include 35 normal and 35 abnormal cases, all of which have been obtained during routine screening in the NHS. The study will also assess the stand-alone performance of the AI algorithm.
Who can participate?
Consultants/attendings (specialising in Paediatric Orthopaedic Surgery) and registrars/residents. Specialist physiotherapists who take part in hip screening as part of their clinical practice.
What does the study involve?
10 readers of varying seniority will be recruited from eight NHS Trusts. This will include five consultant/attending surgeons, four registrars/residents and one specialist physiotherapist. Readers will interpret each scan with and without AI assistance, with an intervening 2-week "washout" period. Each reader will mark seven anatomical points (landmarks, used to determine the diagnosis) in each scan. They will provide their overall confidence score (scale of 1 to 5, 1 = not confident, 5 = very confident) in annotating all the points apart from the labrum.
Using a panel of two paediatric orthopaedic surgeons who specialise in DDH as ground truth, the stand-alone performance of the AI algorithm will assessed, alongside its impact on reader’s accuracy, mean review time per scan and self-reported diagnostic confidence.
What are the possible benefits and risks of participating?
The results may show the utility of the AI algorithm as an assistive diagnostic tool. There are no risks of participating.
Where is the study run from?
Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford (UK)
When is the study starting and how long is it expected to run for?
February 2024 to January 2025
Who is funding the study?
National Institute of Health and Care Research (NIHR) (UK)
Who is the main contact?
Mr Abhinav Singh, Abhinav.singh@ndorms.ox.ac.uk
Contact information
Public, Scientific, Principal Investigator
Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences
University of Oxford
Botnar Research Centre
Old Road
Oxford
OX3 7LD
United Kingdom
 ORCID ID |
0000-0002-7329-6792 |
|---|---|
| Phone | +44 (0)1865227374 |
| Abhinav.singh@ndorms.ox.ac.uk |
Study information
| Study design | Retrospective multicentre and multireader observational cohort study |
|---|---|
| Primary study design | Observational |
| Secondary study design | Cohort study |
| Study setting(s) | Medical and other records, University/medical school/dental school |
| Study type | Diagnostic, Safety, Efficacy |
| Scientific title | Developing and testing computer-assisted diagnostic tools for screening of developmental dysplasia of the hip in newborns: a multi-reader multi-case study |
| Study acronym | DeTeCT DDH |
| Study hypothesis | An assistive AI algorithm can improve the diagnostic accuracy, speed and self-reported confidence of clinicians in diagnosing developmental dysplasia of the hip (DDH) on ultrasound scans. |
| Ethics approval(s) | Ethics approval not required |
| Ethics approval additional information | Approved 14/03/2023, Health Research Authority (2 Redman Place, Stratford, London, E20 1JQ, UK; +44 (0)2071048000; approvals@hra.nhs.uk), ref: 23/HRA/0966 REC approval was waived for the collection of a retrospective fully anonymised dataset. Ethical approval is not required for the multi-reader multi-case study of healthcare professionals. |
| Condition | Developmental dysplasia of the hip in newborns, diagnosed by ultrasound scan |
| Intervention | A retrospective dataset of 70 newborn ultrasound scans will be compiled to include 35 normal and 35 abnormal (dysplastic [25]/dislocated [10]) hips. The case balance is intended to better mimic clinical practice whilst still being statistically powered to detect a suspected difference in accuracy. 10 readers of varying seniority will be recruited from eight NHS Trusts. This will include five consultant/attending orthopaedic surgeons, four orthopaedic registrars/residents and one specialist physiotherapist. Readers will interpret each scan with and without AI assistance in two different sessions. There will be an intervening 2-week "washout" period to minimise reader memory of the reviewed scans. Each reader will mark seven anatomical points (landmarks) used to determine the diagnosis on each scan. They will provide their overall confidence score (scale of 1 to 5, 1= not confident, 5= very confident) in annotating all the points apart from the labrum. Using a panel of two paediatric orthopaedic surgeons who specialise in DDH as ground truth (reference standard), the stand-alone performance of the AI algorithm will assessed, alongside its impact on the reader’s accuracy, mean review time per scan and self-reported diagnostic confidence. Subgroup analysis will be performed by the seniority of the reader. |
| Intervention type | Other |
| Primary outcome measure | Reader and AI algorithm performance will be evaluated as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Area Under Receiver Operating Characteristic Curve (AUC). Where the hip is abnormal on the ultrasound and readers correctly identify this classification as abnormal, it will be counted as a true positive, an incorrect diagnosis of normal by the reader will be a false negative. Where the hip is normal on the ultrasound, its correct classification by the reader will be a true negative and an incorrect classification will be a false positive. The performance measures listed above will be compared for each reader with and without AI assistance. The performance of the AI algorithm alone will also be evaluated as a comparative measure. |
| Secondary outcome measures | Reader speed will be evaluated as the mean review time per scan, with and without AI assistance. Reader confidence will be evaluated via a self-reported score (scale of 1 to 5, 1= not confident to 5 = fully confident), with and without AI assistance. |
| Overall study start date | 01/02/2024 |
| Overall study end date | 31/01/2025 |
Eligibility
| Participant type(s) | Health professional |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Sex | Both |
| Target number of participants | 10 |
| Total final enrolment | 10 |
| Participant inclusion criteria | Consultants/attendings (specialising in Paediatric Orthopaedic Surgery) and registrars/residents. Specialist physiotherapists who take part in hip screening as part of their clinical practice. |
| Participant exclusion criteria | Any healthcare professional who does not review newborn hip ultrasound scans (either autonomously or under direct supervision) in their clinical practice |
| Recruitment start date | 01/03/2024 |
| Recruitment end date | 31/05/2024 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centres
Botnar Research Centre
Old Road
Oxford
OX3 7LD
United Kingdom
Tremona Road
Southampton
SO16 6YD
United Kingdom
Colney
Norwich
NR4 7UY
United Kingdom
London
SW17 0QT
United Kingdom
Stanmore
HA7 4LP
United Kingdom
Westcliff-on-sea
SS0 0RY
United Kingdom
Uxbridge
UB8 3NN
United Kingdom
South Wharf Road
London
W2 1NY
United Kingdom
Wrythe Lane
Carshalton
SM5 1AA
United Kingdom
Eaton Road
West Derby
Liverpool
L12 2AP
United Kingdom
Sponsor information
University/education
Research Governance, Ethics & Assurance
Boundary Brook House
Churchill Drive
Oxford
OX3 7GB
England
United Kingdom
| Phone | +44 (0)1865616480 |
|---|---|
| RGEA.Sponsor@admin.ox.ac.uk | |
| Website | http://www.ox.ac.uk/ |
"ROR" |
https://ror.org/052gg0110 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 31/12/2024 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Published as a supplement to the results publication |
| Publication and dissemination plan | Planned publication in a high-impact peer-reviewed journal |
| IPD sharing plan | All data generated or analysed during this study will be included in the subsequent results publication |
Editorial Notes
18/06/2024: Study's existence confirmed by the HRA.
