FRACT-AI: A study comparing the finding of broken bones on X-Rays by artificial intelligence to the findings by clinicians of varying grades and professional backgrounds
| ISRCTN | ISRCTN19562541 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN19562541 |
| IRAS number | 310995 |
| ClinicalTrials.gov number | NCT06130397 |
| Secondary identifying numbers | NIHR204982, IRAS 310995, CPMS 52221 |
- Submission date
- 17/11/2023
- Registration date
- 11/03/2024
- Last edited
- 06/09/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Ongoing
- Condition category
- Other
Plain English Summary
Background and study aims
Some recent research has shown two things regarding how we look at x-rays to look for fractures, the first is that artificial intelligence (AI) shows that it can competently locate fractures, and the second is that in the emergency department, one of the most frequent problems is mistakes being made in locating fractures on x-rays.
This study brings those two things together, looking at how an AI performs in comparison to a human clinician, and further looks at how the human performs when helped by the AI. The human clinicians will represent several hospital professions (emergency medicine, orthopaedics, radiologists, radiology, physiotherapy and emergency nurses) and levels of experience ranging from junior staff to experienced consultant-level clinicians. The clinicians will look at 500 x-rays, of which half have a fracture, and half do not. The study will measure how many of them are correctly identified.
Who can participate?
Hospital professions (emergency medicine, orthopaedics, radiologists, radiology, physiotherapy and emergency nurses) in the trusts involved.
What does the study involve?
The study will begin recruitment at the start of September for eight weeks, with the first month of x-ray interpretation (where the participants don't have AI to help them) scheduled to begin on the 1st of November. After a period of time of one month (to reduce the chances of x-rays being remembered), the same x-rays will be looked at for a second time with the help of the AI.
What are the possible benefits and risks of participating?
There will be no risk to participants in this study. X-rays interpreted will be old x-rays that are not being used actively in the treatment of a patient.
Where is the study run from?
The study will occur over five NHS trusts in the Thames Deanery - Oxford University Hospitals, Royal Berkshire, Buckinghamshire Healthcare, Frimley Health and Milton Keynes University Hospital (UK)
When is the study starting and how long is it expected to run for?
December 2022 to June 2025
Who is funding the study?
National Institute for Health and Care Research (NIHR) (UK).
Who is the main contact?
Professor Alex Novak, alex.novak@ouh.nhs.uk
Doctor Sarim Ather, sarim.ather@ouh.nhs.uk
Contact information
Scientific, Principal Investigator
John Radcliffe Hospital, Headley Way, Headington
Oxford
OX3 9DU
United Kingdom
 ORCID ID |
0009-0006-4086-3152 |
|---|---|
| Phone | 0300 304 7777 |
| alex.novak@ouh.nhs.uk |
Scientific, Principal Investigator
John Radcliffe Hospital, Headley Way, Headington
Oxford
OX3 9DU
United Kingdom
| ORCID ID |
0000-0001-9614-5033 |
|---|---|
| Phone | 0300 304 7777 |
| sarim.ather@ouh.nhs.uk |
Public
John Radcliffe Hospital, Headley Way, Headington
Oxford
OX3 9DU
United Kingdom
| ORCID ID |
0009-0001-0288-954X |
|---|---|
| Phone | 0300 304 7777 |
| max.hollowday@nhs.net |
Study information
| Study design | Multicentre multiple-reader multiple-case study |
|---|---|
| Primary study design | Observational |
| Secondary study design | Multiple-reader multiple-case study |
| Study setting(s) | Internet/virtual, Medical and other records |
| Study type | Diagnostic |
| Participant information sheet | 44612 PIS 0.2.pdf |
| Scientific title | FRACT-AI: Evaluating the Impact of Artificial Intelligence-Enhanced Image Analysis on the Diagnostic Accuracy of Frontline Clinicians in the Detection of Fractures on Plain X-Ray |
| Study acronym | FRACT-AI |
| Study hypothesis | Previous research has demonstrated AI's promising diagnostic performance in the location of fractures, and similarly X-Ray interpretation in the emergency department has been found to be a frequent source of error. These elements have not yet been compared in a UK clinical setting. This study hypothesises that artificial intelligence (Boneview) is more accurate at accurately locating fractures on plain X-Rays than that those interpreted by human clinicians. |
| Ethics approval(s) |
Approved 13/12/2022, UK Healthcare Research Authority (2 Redman Place, London, E20 1JQ, United Kingdom; +44 207 104 8000; queries@hra.nhs.uk), ref: R80145/RE001 |
| Condition | Location of fractures on plain X-rays by artificial intelligence and human clinicians. |
| Intervention | The study broadly compares X-ray interpretation between human clinicians and an artificial intelligence designed to locate fractures in plain x-rays - Boneview. 500 plain x-rays will be interpreted by each participant, 50% pathological and 50% normal. The reference for the 50:50 divide will be set by a panel of specialised musculoskeletal radiologists, each x-ray being reviewed by at least two musculoskeletal radiologists to establish 'ground truth'. There will also be additional parameters ascribed to each x-ray, including the nature of the pathological finding, the location of the anatomy being investigated, and the difficulty of the image to interpret. Plain x-rays not to be included are: -X-ray Skull -X-ray Cervical spine -Postsurgical X-ray -Follow-up x-ray for known fracture -Paediatric x-ray (<18 years) The human clinicians, hereafter termed 'readers', will show a range of specialities and levels of experience. 18 clinicians in total will be split evenly between 6 specialities: emergency medicine physicians, surgeons in trauma & orthopaedics, radiologists, radiographers, physiotherapists, and finally emergency nurse practitioners (nurses specialising in minor trauma injuries in the emergency department). Within these groups, consisting of 3 clinicians each, there will be a person of consultant/equivalent level (>10 years' practice), registrar/equivalent (5-10 years' practice), and senior house officer/equivalent (>5 years' practice). The images will be run through Boneview. The readers will then interpret the same set of randomised images twice, once without the aid of Boneview, and after a washout period of no less than a month, a second time with the assistance of Boneview. The study will therefore compare AI on its own with unassisted readers, and with AI and readers working in synchrony. As a subset, data will also be collected on the professional background of the reader, and their level of experience. |
| Intervention type | Device |
| Pharmaceutical study type(s) | Not Applicable |
| Phase | Not Applicable |
| Drug / device / biological / vaccine name(s) | Gleamer Boneview |
| Primary outcome measure | Clinician readers will be asked to identify the presence or absence of fracture by placing a marker on the image at the location of the fracture (if present) and to rank their confidence for fracture identification. Confidence rating will take the form of a Likert scale from 1 to 10, 1 being least confident, 10 being very confident). |
| Secondary outcome measures | There are no secondary outcome measures |
| Overall study start date | 13/12/2022 |
| Overall study end date | 30/06/2025 |
Eligibility
| Participant type(s) | Health professional |
|---|---|
| Age group | Adult |
| Sex | Both |
| Target number of participants | 18 |
| Total final enrolment | 16 |
| Participant inclusion criteria | 1. Healthcare professional from the following professions/specialities: 1.1. Emergency medicine physicians 1.2. Surgeons in trauma and orthopaedics 1.3. Radiologists 1.4. Radiographers 1.5. Physiotherapists 1.6. Emergency nurse practitioners |
| Participant exclusion criteria | 1. Not from the above listed professions (emergency medicine physicians, surgeons in trauma and orthopaedics, radiologists, radiographers, physiotherapists, emergency nurse practitioners.) 2. Radiologists already musculoskeletal specialists, |
| Recruitment start date | 01/09/2023 |
| Recruitment end date | 31/10/2023 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centres
Headley Way
Headington
Oxford
OX3 9DU
United Kingdom
London Road
Reading
RG1 5AN
United Kingdom
Whielden Street
Amersham
HP7 0JD
United Kingdom
Frimley
Camberley
GU16 7UJ
United Kingdom
Eaglestone
Milton Keynes
MK6 5LD
United Kingdom
Sponsor information
Government
Old Rd, Headington
Oxford
OX3 7LD
England
United Kingdom
| Phone | +44 1865 226764 |
|---|---|
| enquiries@nihr.ac.uk | |
| Website | https://www.crn.nihr.ac.uk/ |
"ROR" |
https://ror.org/05fj7ar22 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/04/2024 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Data sharing statement to be made available at a later date |
| Publication and dissemination plan | Planned publications in mind for this study are the British Medical Journal and the Emergency Medicine Journal |
| IPD sharing plan | The current data sharing plans for this study are unknown and will be available at a later date |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Participant information sheet | version 0.2 | 01/12/2023 | No | Yes | |
| Protocol file | version 0.7 | 01/12/2023 | No | No | |
| Protocol article | 05/09/2024 | 06/09/2024 | Yes | No |
Additional files
Editorial Notes
06/09/2024: Publication reference added.
03/05/2024: The overall study end date was changed from 31/03/2024 to 30/06/2025. Total final enrolment added.
01/12/2023: Trial's existence confirmed by the National Institute for Health and Care Research (NIHR) (UK).
