QualDash: Designing and evaluating an interactive dashboard to improve quality of care
| ISRCTN | ISRCTN18289782 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN18289782 |
| Secondary identifying numbers | 34994 |
- Submission date
- 30/10/2017
- Registration date
- 09/11/2017
- Last edited
- 14/07/2022
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English Summary
Background and study aims
Over 100 audits are undertaken in the NHS, each focusing on a different area of care. They provide healthcare professionals, managers, and those responsible for planning NHS services (commissioners) with important information about whether a hospital is meeting expected standards and how the care provided compares to other hospitals. This can encourage hospitals to make improvements. However, some healthcare professionals and managers have found it difficult to make use of audit information. One way that might help people to understand the information better is to present it in a visual or graph-based form, known as a ‘dashboard’. Dashboards are already being used in the NHS but existing dashboards are still images, presenting information in a standard format. The aim of this project is to use computer technology to develop a dashboard that is interactive, allowing healthcare professionals, managers, and commissioners to easily and quickly explore audit information to understand where improvements in care delivery should be made.
Who can participate?
Staff from NHS hospitals and NHS commissioners.
What does the study involve?
Staff and commissioners are interviewed to gather their ideas about how audit information is used, challenges in using audit information, and how these challenges might be overcome. These ideas are used to design a dashboard and to develop a plan for introducing the dashboard so healthcare professionals, managers, and commissioners understand why the dashboard is being introduced and know how to use it. The design of the dashboard depends on what healthcare professionals, managers, and commissioners tell us is useful and will be tailored to meet the needs of different users. It is likely to include the ability for staff to: compare a hospital’s performance with that of hospitals located nearby and hospitals that are further away but of similar size; explore how much the performance of a hospital varies, identifying areas where there is room for improvement but also identifying areas where there is good practice that can be disseminated more widely; see a visual image of how a hospital’s performance has changed over time, to help them decide if changes in the way care is delivered are having the anticipated effect; and explore the relationship between different types of information recorded in the audit and how this compares with other hospitals, for example seeing if differences in outcomes for patients are due to differences in how care is delivered or how the hospital is resourced. The dashboard is made available to the 5 NHS hospitals and commissioners via the internet and participants are observed for one year on how it is used. Staff interviews are done to get feedback and use this to make improvements to the dashboard during this time. The amount that the dashboard is used is recorded and the quality of care in hospitals that did and did not use the dashboards are compared. The findings will provide knowledge about the impacts of the dashboard, which features of the dashboard provide most benefit, and what is needed to support the use of the dashboard.
What are the possible benefits and risks of participating?
Staff who participate in the study will have the opportunity to inform both the design of an interactive web-based quality dashboard and the strategy for introducing the dashboard, helping to ensure that the dashboard meets their needs. Over a one year period, staff will have the opportunity to use the dashboard to explore audit data and use it to inform quality improvement initiatives.Risks and burdens for staff are anticipated to be minimal. The most significant burden will be on staff time, and so on all data collection activities a balance will be sought between obtaining the data needed for the study with ensuring that these activities do not place unnecessary burden on staff time or cause fatigue. The part of the study that involves observing how people use the dashboard may be perceived by staff as an intrusion, and therefore presentations will be given in each Trust to explain the purpose of the research. No observations of patient care will be undertaken.
Where is the study run from?
The study is run from the University of Leeds (UK) and involves 5 NHS Trusts.
When is the study starting and how long is it expected to run for?
October 2017 to June 2020
Who is funding the study?
National Institute for Health Research (UK)
Who is the main contact?
Dr Rebecca Randell
r.randell@leeds.ac.uk
Contact information
Scientific
Faculty of Health Studies
University of Bradford
Bradford
BD7 1DP
United Kingdom
 ORCID ID |
0000-0002-5856-4912 |
|---|---|
| Phone | +44 (0)1274 234144 |
| r.randell@bradford.ac.uk |
Study information
| Study design | Non-randomised; Both; Design type: Process of Care, Complex Intervention, Management of Care, Qualitative |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Non randomised study |
| Study setting(s) | Hospital |
| Study type | Treatment |
| Participant information sheet | Not available in web format, please use the contact details below to request a patient information sheet |
| Scientific title | QualDash: Designing and evaluating an interactive dashboard to improve quality of care |
| Study hypothesis | The aim of this study is to develop and evaluate QualDash, an interactive web-based quality dashboard that supports clinical teams, quality sub-committees, NHS Trust boards, and commissioners to better understand and make use of National Clinical Audit data, thereby leading to improved quality of care and clinical outcomes. |
| Ethics approval(s) | University of Leeds School of Healthcare Research Ethics Committee (SHREC), 03/08/2017, ref: HREC16-044 |
| Condition | Generic health relevance (organisation and delivery of services) |
| Intervention | The project is based on MRC guidance for the design and evaluation of complex interventions. To ensure that QualDash has a robust theoretical basis and to enhance the probability of its widespread implementation, the principles of realist evaluation (which involves building, testing and refining the theories of how and in what contexts an intervention works) are combined with the principles of co-design. The project comprises five phases: Phase 1: Interviews with members of clinical teams, quality sub-committees, and boards across five NHS acute Trusts and relevant commissioners are used to articulate how NCA data are currently used (or not) in practice, identifying blockages to effective use and how these might be overcome. Interviews consider a range of NCAs but focus on the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet). Initial requirements for the design of QualDash derived from the interview data are discussed at a workshop with suppliers of other NCAs to determine which requirements are generalisable to all NCAs. Phase 2: QualDash is developed through an iterative process, involving focus groups with clinical teams, quality sub-committee members, and board members from one Trust, relevant commissioners, and patients and carers. A controlled user experiment assesses comprehension, usability, and acceptability of QualDash prototypes, in comparison with existing formats for feedback of NCA data. Phase 3: An implementation strategy for QualDash, tailored to the five Trusts and relevant Clinical Commissioning Groups (CCGs), isdeveloped through focus groups with clinical teams, quality sub-committees, and boards from the five Trusts and relevant commissioners. Planned implementation activities are delivered across the five Trusts and with relevant commissioners. Phase 4: QualDash is made available in the five Trusts and relevant CCGs. A controlled interrupted time series (CITS) analysis investigates the impacts of QualDash, using process measures from the included NCAs. Ethnographic observations and interviews over 12 months provide insight into contexts and mechanisms that lead to those impacts. A questionnaire is used to gather data on perceived usefulness of QualDash. Phase 5: The feasibility of conducting a cluster randomised controlled trial (CRT) of QualDash is assessed. If progression criteria are met, a CRT will be designed, using the CITS results to decide what effect size the trial should be powered to detect. Two focus groups explore the suitability of QualDash for a range of other NCAs. |
| Intervention type | Other |
| Primary outcome measure | 1. For MINAP, the primary outcome is the composite process measure Cumulative Missed Opportunities for Care (CMOC), which has nine components (pre-hospital ECG, acute use of aspirin, timely perfusion, prescription at hospital discharge of aspirin, thienopyridine inhibitor, ACE-inhibitor, HMG-CoA reductase inhibitor, beta blocker, referral for cardiac rehabilitation). This will be calculated using monthly audit data obtained from MINAP for 24 months pre-intervention and 12 months post-intervention. 2. For PICANet, the primary outcome is the use of non-invasive ventilation first for patients requiring ventilation. Monthly data will be obtained from PICANet for 24 months pre-intervention and 12 months post-intervention. |
| Secondary outcome measures | 1. Data completeness for MINAP and PICANet is measured using monthly data obtained from MINAP and PICANet for 24 months pre-intervention and 12 months post-intervention 2. Additional secondary outcomes to be included in the CITS will be determined on the basis of findings from Phase 1 regarding interviewees’ perceptions of the value of particular measures and the extent to which they can be impacted by quality improvement initiatives and through consultation with our Lay Advisory Group. 3. Percentage of intended users who use QualDash, with data collected from log files over the 12 month period of the evaluation, complemented by qualitative data collected through observations to understand how and in what contexts QualDash is used at 12 months 4. Perceived usefulness of QualDash and intention to continue using it after the study period is assessed using data collected through a questionnaire based on the Technology Acceptance Model, complemented by qualitative data collected through semi-structured interviews at 12 months |
| Overall study start date | 01/10/2017 |
| Overall study end date | 31/12/2020 |
Eligibility
| Participant type(s) | Patient |
|---|---|
| Age group | Adult |
| Sex | Both |
| Target number of participants | Planned Sample Size: 58; UK Sample Size: 58 |
| Total final enrolment | 61 |
| Participant inclusion criteria | Staff who work in the participating NHS Trusts and relevant commissioners. Within the each Trust, we will recruit members of relevant clinical teams (cardiology, paediatric intensive care), the quality sub-committee, and Trust boards. |
| Participant exclusion criteria | Participants who do not fulfil the inclusion criteria. |
| Recruitment start date | 13/11/2017 |
| Recruitment end date | 30/11/2020 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
LS1 3EX
United Kingdom
Sponsor information
University/education
Faculty of Medicine & Health
Leeds
LS2 9JT
England
United Kingdom
"ROR" |
https://ror.org/024mrxd33 |
|---|
Funders
Funder type
Government
No information available
Results and Publications
| Intention to publish date | 14/07/2021 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | Planned publication and dissemination activities include Open Access publications in a range of academic journals (e.g. BMJ Quality & Safety, Implementation Science, Journal of the American Medical Informatics Association), and presentation of findings at a national conference, such as the HSRUK Symposium or the HQIP-sponsored Clinical Audit for Improvement conference. A draft final report will be submitted to the funder by 14/07/2020, with publication of the final report in the NIHR journal Health Services & Delivery Research likely within 12 months of that date. An end-of-project dissemination event will also be held for NCA suppliers, approximately 6 months after completion the project, with presentations video recorded and made available on the project website. |
| IPD sharing plan | Beyond inclusion of anonymised quotations from interviews and focus groups and field note extracts in publications and presentations, the qualitative datasets generated during the current study are not expected to be made available. This is because of the recognised challenges of making sense of qualitative data without an understanding of the context in which it was gathered. The questionnaire data will be available upon request from Rebecca Randell (r.randell@leeds.ac.uk), following publication of the final report, subject to the necessary ethical approvals being obtained. To access the audit data used for the CITS, this will need to be requested from the audit suppliers. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol article | 25/02/2020 | 11/05/2021 | Yes | No | |
| Results article | 01/05/2022 | 14/07/2022 | Yes | No |
Editorial Notes
14/07/2022: Publication reference and total final enrolment added.
11/05/2021: Publication reference added.
05/05/2020: Due to current public health guidance, recruitment for this study has been paused.
31/03/2020: The following changes were made to the trial record:
1. The overall end date was changed from 30/06/2020 to 31/12/2020.
2. The recruitment end date was changed from 31/05/2020 to 30/11/2020.
3. The primary contact details were updated.
29/03/2019: The condition has been changed from "Specialty: Health services and delivery research, Primary sub-specialty: Health Services and Delivery Research; UKCRC code/ Disease: Generic Health Relevance/ No specific disease" to "Generic health relevance (organisation and delivery of services)" following a request from the NIHR.
