AI triage in general practice
| ISRCTN | ISRCTN10744072 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN10744072 |
| IRAS number | 331286 |
| Secondary identifying numbers | CPMS 60999, NIHR153121 |
- Submission date
- 21/05/2024
- Registration date
- 23/10/2024
- Last edited
- 07/11/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Ongoing
- Condition category
- Other
Plain English Summary
Background and study aims
Patients can currently ask for help from their GP practice at any time by completing an online consultation form on the internet. This goes directly to the GP practice and patients do not have to wait in a queue. However, the system cannot identify and prioritise those patients needing urgent help. This may cause delays in patient care. A possible solution is for the online consultation system to automatically identify urgent and emergency requests as soon as they are submitted by using an artificial intelligence (AI) add-on feature called AI Triage. The research team want to find out if computers trained to identify urgent and emergency requests using AI triage are accurate and can reduce these delays. The study will also find out if AI triage works fairly for a diverse range of patients, whether it affects staff workload, whether it is good value for money and whether there are any disadvantages. This study aims to ensure that patients who require urgent appointments and treatment from GP surgeries are identified quickly and so avoid harm as a result of delays in care.
Who can participate?
A minimum of 20 GP practices across England will test the AI Triage intervention and another 20 GP practices will continue to use Patchs without AI triage. These 40 practices are expected to receive at least 2928 urgent and emergency (combined) online consultation requests in the 12-month intervention period, from patients of all sexes, ages, and with any clinical condition.
What does the study involve?
The study is testing whether computers trained to identify urgent and emergency requests using AI can reduce delays in completing these requests and what factors such as the organisation, patient characteristics and other resources can influence detection and decision making. It will study the Online Consultation system ‘Patchs’, which has already been used by NHS GP practices in diverse communities in England for at least 12 months. 20 control practices will continue to use the existing Patchs system and 20 intervention practices will be given the add-on AI Triage feature. The time it takes to complete an urgent or emergency request will be compared in the 20 practices that do not have the AI triage feature with the 20 practices that do have it, before and after the AI triage feature was introduced.
What are the possible benefits and risks of participating?
Benefits: If the AI triage feature reduces care delays, patients who need urgent and emergency help may receive it sooner.
Risks: This study uses routinely collected data to evaluate a UKCA-marked medical device already used in routine clinical practice. Consequently, risks for participants are minimal. Security arrangements are in place to protect this data.
Where is the study run from?
The University of Manchester
When is the study starting and how long is it expected to run for?
April 2022 to April 2026
Who is funding the study?
NIHR Health and Social Care Delivery Research (HSDR) Programme
Who is the main contact?
Dr Sarah Croke, sarah.croke@manchester.ac.uk
Contact information
Public, Scientific, Principal Investigator
Centre for Primary Care and Health Services Research, Williamson Building, University of Manchester
Manchester
M13 9PL
United Kingdom
 ORCID ID |
0000-0001-9975-4782 |
|---|---|
| Phone | +44 (0)161 306 6000 |
| benjamin.brown@manchester.ac.uk |
Study information
| Study design | Randomized controlled clinical investigation study of a medical device |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | GP practice, Internet/virtual |
| Study type | Treatment |
| Participant information sheet | Not available in web format, please use the contact details below to request a patient information sheet |
| Scientific title | Evaluating the impact of artificial intelligence triage in online consultations to reduce delays in urgent primary care: interrupted time series analysis and quantitative process evaluation |
| Study acronym | AI TRIAGE IMPACT |
| Study hypothesis | AI Triage reduces delays in completing online consultations defined as urgent and emergency by GP practice staff at the patient-level. |
| Ethics approval(s) |
Approved 29/02/2024, HSC REC A (-, -, -, United Kingdom; +44 (0)28 9536 1400; RECA@hscni.net), ref: 24/NI/0022 |
| Condition | Artificial intelligence triage in online consultations to reduce delays in urgent primary care |
| Intervention | Study Parts: 1. Interrupted Time Series Analysis: the study will compare how quickly GP practices can complete urgent and emergency online consultations before and after introducing Al Triage. 2. Quantitative Process Evaluation: the study will measure how well Al Triage is implemented, how many practices use it, and how accurate it is. (A related qualitative process evaluation is described in a separate protocol (IRAS ID: 335429)). Participants: GP practices that have been using the PATCHS online consultation system without Al Triage will be recruited for at least 12 months. These practices will be randomly assigned to two groups: one getting Al Triage immediately and the other getting it later ('control' group). The study aims to recruit at least 20 practices for each group and to collect a minimum of 2928 urgent and emergency online consultations across both groups over 12 months. Intervention GP Practices: Practices in the Al Triage group will be contacted by the company that developed PATCHS, Spectra Analytics, to start using Al Triage for a year using their usual process. This involves being contacted by email by Spectra Analytics who provide training and safety instructions on how to use Al Triage. Al Triage will automatically identify urgent and emergency cases based on what patients write in their online consultations. GP practice staff will receive automated flags for urgent and emergency cases, and patients will get advice on what to do next, for example, attend the emergency department if an emergency. Control GP Practices: Practices in the control group won't be contacted about Al Triage by Spectra Analytics until the end of the 12 months. Al Triage is a selling point of PATCHS so this approach helps to avoid their potential disappointment. Al Triage is available to all GP practices using PATCHS in normal clinical practice, so any GP practice without Al Triage can ask to use it during the study anyway (including those in the control group). Control GP practice staff will NOT see automated flags for urgent and emergency cases, and patients will NOT get advice on what to do next, for example, attend the emergency department if an emergency. Outcome Measurement: The time it takes to resolve urgent and emergency online consultations will be compared between the Al Triage and control groups. The accuracy of Al Triage, how it affects people from different backgrounds, and how it might impact health service resources will also be assessed. Potential Benefits: If Al Triage works well, patients in the participating practices may experience quicker responses to urgent and emergency cases. This study aims to provide evidence for the broader use of Al Triage in the NHS, and guides will be created on using it effectively, regardless of the outcome. Patient and Public Involvement: The study team talked to seven patients and had discussions with 10 GP practice staff to get their thoughts on our study plans. They chose our primary outcome measure and the level of improvement that would make a real difference in people's health. Based on their input, the team also added specific goals to see how the use of Al Triage might affect differences in health outcomes among different groups of people and how it might impact the resources in the healthcare system. Randomisation: GP practices will be allocated one-to-one into intervention (Al Triage now) or control (Al Triage later) groups in blocks of 5 practices in each arm (control and intervention) with a target of 20 in each group. Depending on the response rate, the block size may be increased or decreased, and through a combination of randomisation and algorithmic approaches using appropriate statistical software, attempt to match the two groups as closely as possible on patient population size, Index of Multiple Deprivation quintile, monthly volume of online consultations, and baseline levels of the primary outcome measure. The possibility of matching GP practices will also explored on geographic region, rurality, number of whole-time equivalent GPs per 1000 patients, and levels of patient morbidity, though given the target sample size is relatively small this may not be possible. Where characteristics cannot be matched, the team will attempt to adjust for them in our statistical models. This is a single-blind study. The UoM research team will not know which GP practices are allocated to intervention or control groups. Due to the nature of the intervention GP practices cannot be blinded. The UoM research team will allocate GP practices to "0" or "1" groups using the approach described above. The allocation sequence will be provided to Spectra Analytics which will use statistical software to randomly allocate "0" or "1" to the intervention group. They will then approach GP practices in the intervention group to offer them Al Triage as described elsewhere. Spectra Analytics will retain the mapping key and not share it with the UoM research team until after the data analysis is finalised. Analysis of the interrupted time series analysis (including the primary outcome measure) will not take place until the end of the intervention period. |
| Intervention type | Procedure/Surgery |
| Primary outcome measure | The proportion of delays in completing urgent and emergency online consultations at the patient level (delayed urgent online consultations + delayed emergency online consultations) ÷ (total urgent patient online consultations + total emergency online consultations) at baseline versus the intervention comparison |
| Secondary outcome measures | 1. The proportion of delayed emergency online consultations at the patient level (disaggregated primary outcome measure; denominator = number of emergency online consultations) at one timepoint 2. The proportion of delayed urgent online consultations at the patient level (disaggregated primary outcome measure; denominator = number of urgent online consultations) at one timepoint |
| Overall study start date | 01/04/2022 |
| Overall study end date | 30/04/2026 |
Eligibility
| Participant type(s) | Patient |
|---|---|
| Age group | Mixed |
| Lower age limit | 16 Years |
| Sex | Both |
| Target number of participants | Planned Sample Size: 2928; UK Sample Size: 2928 |
| Participant inclusion criteria | GP practices: GP practices must meet both the following criteria to be eligible for the study: 1. Currently actively using PATCHS without AI Triage 2. Actively used PATCHS without AI Triage for at least 12 months Patients: 1. Patients who use PATCHS in the intervention and control GP practices who have not opted out of sharing their anonymised data for research purposes with UoM. 2. Patients who are at least 16 years old, though carers can use PATCHS on behalf of patients under 16 years old if they have guardianship or parental responsibility (as manually verified by the GP practice in PATCHS). |
| Participant exclusion criteria | Not meeting the participant inclusion criteria |
| Recruitment start date | 29/02/2024 |
| Recruitment end date | 01/03/2025 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
United Kingdom
Sponsor information
Hospital/treatment centre
Directorate of Research and Business Engagement, Floor 2 Christie Building, Oxford Road
Manchester
M13 9PL
England
United Kingdom
| Phone | +44 (0)161 275 2725 |
|---|---|
| medicaldevices@manchester.ac.uk | |
| Website | https://www.manchester.ac.uk/ |
"ROR" |
https://ror.org/027m9bs27 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 30/04/2027 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Not expected to be made available |
| Publication and dissemination plan | This research protocol will be publicly registered on the ISRCTN (International Standard Randomised Controlled Trial Number) registry. Findings will be published in open-access peer-reviewed scientific journals. Analysis code will be made available. We will also produce short evidence summaries communicating key findings in an accessible way, which will be hosted on publicly available websites (e.g. www.patchs.ai) and disseminated to participating GP practices and patients by Spectra Analytics via email. GP practices will be encouraged to share these findings with their patients, for example by publishing them on their website. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study are not expected to be made available because study participants have only consented to share their data with The University of Manchester. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Other files | version 1 | 10/08/2023 | 04/11/2024 | No | No |
| Other files | version 1.1 | 16/10/2023 | 04/11/2024 | No | No |
| Protocol file | version 1.1 | 01/03/2024 | 07/11/2024 | No | No |
| Statistical Analysis Plan | version 1.1 | 01/03/2024 | 07/11/2024 | No | No |
Additional files
Editorial Notes
07/11/2024: Internal review.
04/11/2024: The following updates were made:
1. The study acronym was added.
2. Appendices were added as additional files showing (1) PATCHS screenshots and (2) Anonymisation Assessment.
3. The study protocol and statistical analysis plan were added (not peer-reviewed, v1.1, dated 01/03/2024).
23/10/2024: This study record was submitted for registration on 21/05/2024. The editorial response was provided after engaging with the study’s public contributor group and was received by the registry on 23/10/2024. As this date was after the recruitment start date (01/05/2024) the study was automatically marked as retrospectively registered.
21/05/2024: Study's existence confirmed by the National Institute for Health and Care Research (NIHR) (UK).
