Evaluation of the 5Rs programme (a revision year approach to GCSE maths resits)
| ISRCTN | ISRCTN23703392 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN23703392 |
| Secondary identifying numbers | R2135001 |
- Submission date
- 09/07/2019
- Registration date
- 25/07/2019
- Last edited
- 12/12/2022
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English Summary
Background and study aims
Full-time students aged 16-19 who narrowly miss the pass mark for GCSE maths (Grade 4) must continue studying it to receive funding for the rest of their further education. Only 22.6% of those aged 17+ re-taking GCSE maths achieved a pass mark (grade 4 or higher) in 2018. This suggests current approaches to teaching GCSE resit maths need improvement. There is some small-scale evidence that the 5Rs approach has had a positive impact on student attendance rates and increased the proportion passing their GCSE maths resits. 5Rs has been developed to improve maths skills and outcomes by focusing on revision rather than teaching new information (since students have already covered the curriculum). It aims to cover mathematical basics, plug knowledge gaps, improve exam technique and introduce students to alternative mathematical methods that they will not have encountered in school. It has been designed to look and feel different to students’ previous school experiences and emphasises self-motivated study outside lessons.
Who can participate?
The study is open to post-16 education and training providers in England (including FE colleges, sixth form colleges, school sixth forms and independent training providers). All students aged 16-19 who are resitting GCSE maths having previously obtained a Grade 3 or below can participate.
What does the study involve?
The settings will be allocated at random either to receive 5Rs or to continue with their usual practice. Those receiving 5Rs will deliver it in all GCSE maths resit lessons across the year.
What are the possible benefits and risks of participating?
The hope is that participants will be more likely to achieve a pass grade in their GCSE maths resit: the risk is that they will be less likely. However, preliminary evidence for the effectiveness of 5Rs is promising.
Where is the study run from?
88 settings have been recruited to the project. They are mainly Further Education colleges, with a sizeable minority of sixth form colleges and sixth forms in schools. Recruitment was led by the Association of Colleges and 5Rs training and support will be provided by Julia Smith Ltd.
When is the study starting and how long is it expected to run for?
The trial will be running across the academic year 2019/2020
Who is funding the study?
The study is being funded by the Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Elizabeth Coleman, izzy.coleman@york.ac.uk
Contact information
Public
York Trials Unit
Department of Health Sciences
ARRC Building
University of York
Heslington
York
YO10 5DD
United Kingdom
 ORCID ID |
0000-0003-4210-1865 |
|---|---|
| Phone | +44 (0)1904 321512 |
| izzy.coleman@york.ac.uk |
Study information
| Study design | Multi-site cluster randomised controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | Efficacy study of the 5Rs approach to GCSE Maths resits, a two-arm cluster randomised trial of 16-19 year olds in post-16 settings (intervention vs business-as-usual) |
| Study acronym | 5Rs evaluation |
| Study hypothesis | The central aim of the trial is to evaluate the impact of 5Rs on GCSE maths attainment for students aged 16-19 who are resitting GCSE maths to try to achieve a pass grade (ie gain a Grade 4 or above). The primary research question is: 1. How effective is 5Rs compared to “teaching as usual” in improving outcomes in maths GCSE scores for resit students aged 16-19? The secondary research questions, all framed in the context of comparing resit students aged 16-19 following the 5Rs programme with those receiving “teaching as usual”, are: 1. How effective is 5Rs in improving student pass rate for GCSE maths? 2. Does 5Rs have an impact on student attitudes towards maths, as measured by the adapted Attitudes Toward Mathematics Inventory (ATMI)? 3. Does 5Rs have an impact on student retention rates as measured by exam attendance? The final research questions explore the impact of 5Rs on subgroups of the resit 16-19-year-old students: 1. Does 5Rs have a greater benefit for students doing resits in May/June 2020 rather than November 2019? 2. What is the effect of 5Rs on those who have ever been eligible for free school meals (FSM)? |
| Ethics approval(s) | Approved 29/07/2019, Health Sciences Research Governance Committee (University of York, York, YO10 5DD, UK; Tel: +44 (0)1904 32(1344)), ref: HSRGC/2019/352/B |
| Condition | GCSE maths resit lessons |
| Intervention | Participants are randomised at the setting level to follow the 5Rs approach or business-as-usual - masking to treatment not possible in this educational environment. In the intervention condition, teachers receive one day of training at the beginning of each term. The 5Rs programme will be delivered as a replacement for the standard GCSE resit lessons for an entire academic year. It is anticipated that this will be around 3 hours a week. |
| Intervention type | Behavioural |
| Primary outcome measure | Resit GCSE Maths raw mark from November 2019 or May/June 2020, which will be obtained directly from settings when available (about two months after exams). Results from the latest sitting will be used in the analysis, i.e. combining results from November and June as appropriate. |
| Secondary outcome measures | 1. GCSE maths grade is measured by achievement of grade 4 or above at resit GCSE (ie. achieving a pass) as a binary measure obtained from the exam board via the settings (November 2019 or June 2020, whichever is more recent) 2. GCSE attendance is measured using present/absent at each of the three papers obtained from the exam board via the settings (November 2019 or June 2020, whichever is more recent) 3. Student attitude towards mathematics is measured using an adapted version of the Attitudes Toward Mathematics Inventory (ATMI) in April/May 2020 |
| Overall study start date | 30/11/2018 |
| Overall study end date | 30/09/2020 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Other |
| Lower age limit | 16 Years |
| Upper age limit | 19 Years |
| Sex | Both |
| Target number of participants | Aim to recruit 80 settings (clusters), estimate average of 60 students per setting i.e. 4800 |
| Total final enrolment | 88 |
| Participant inclusion criteria | Post-16 education and training providers in England are eligible to take part in the trial. This includes FE colleges, sixth form colleges, school sixth forms, university technical colleges (UTCs) and independent training providers. Settings are eligible to participate if the following criteria are met: 1. They have a minimum of 15 students aged 16-19 re-taking GCSE maths in the year 2018-2019 and expect this number to stay constant or rise in 2019-2020. 2. GCSE students are enrolled by September 2019 for the full academic year. Students will be eligible to participate providing they have not already achieved a pass grade at GCSE maths (ie not achieved grade 4 or above) and are studying to re-sit GCSE maths in November 2019 or Summer 2020. Age: 16-19. All genders. |
| Participant exclusion criteria | Settings will not be eligible if any of the following apply: 1. They operate roll-on roll-off recruitment of students 2. They are involved in the Maths for Life trial funded by EEF 3. They or their staff have previously been trained in or used the 5Rs programme, including accessing the 5Rs materials available on the AQA website. AQA have agreed to check potential participants against their records and will confirm whether or not they have been previously involved (data protection regulations do not allow us direct access to this information) 4. They have been selected as a Centre for Excellence in Maths (CfEM) |
| Recruitment start date | 08/03/2019 |
| Recruitment end date | 10/07/2019 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Holborn
London
WC1A 1HU
United Kingdom
Sponsor information
Charity
5th Floor, Millbank Tower
21-24 Millbank
London
SW1P 4QP
United Kingdom
| Phone | +44 (0)207 802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
University/education
Research Innovation Office
Innovation Centre
York Science Park
Innovation Way
Heslington
York
YO10 5DG
England
United Kingdom
| Phone | +44 (0)1904 321726 |
|---|---|
| sally.baker@york.ac.uk | |
| Website | https://www.york.ac.uk/ |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 31/08/2021 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | Current publication and dissemination plan as of 13/10/2021: All three documents (protocol, Statistical Analysis Plan, final report) can be found on the EEF project webpage: https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/the-5rs-approach-to-gcse-maths-resits/ Previous publication and dissemination plan: 1. The protocol can be found on the EEF project webpage: https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/the-5rs-approach-to-gcse-maths-resits/ 2. The SAP will also be published here (expected towards the end of 2019) 3. The final report will be openly published on the EEF website (est submitted end July 2021, published end August 2021), with the possibility of subsequent journal articles, practitioner articles and conference papers |
| IPD sharing plan | Current IPD sharing statement as of 13/10/2021: Type of data stored: 1. Anonymised project data 2. A standardised dataset including pupil identifiers with fields renamed or recoded to support secondary analysis 3. A copy of the draft evaluation report to help validate the data and any other documents or materials to support secondary analysis 4. Syntax or Do files 5. A submission form listing the filenames and declaring the legal basis for processing Repository name and weblink: EEF data archive, currently run by Fischer Family Trust (FFT) but probably moving to ONS before the end of this project. There is no weblink. Requesting access: The datasets generated during and/or analysed during the current study will be available upon request from the EEF data archive managed by the Office for National Statistics. Participant consent: participants are informed that their anonymised data will be deposited with the EEF data archive and can withdraw from the research if they do not wish their data to be shared Any legal/ethical restrictions: strict procedures for protecting the integrity and confidentiality of the data processed on behalf of the EEF. All data processing activities must take place in accordance with the current Data Protection Act and GDPR. Independent evaluators must clearly specify and justify their legal bases for processing personal and any special data, data processing roles, parties with access to data and retention periods, in all relevant project documents. EEF evaluation data are processed by the archive manager on the basis of legitimate interests, according to the GDPR, Article 6, Paragraph 1(f), taking responsibility for protecting the fundamental rights and freedoms of the data subjects, and ensuring their interests are protected at all times. Type of data shared: the EEF archive will be accessible to the wider research community for secondary analyses that provide public benefit and are in line with the missions of the EEF, DfE and ONS. When available and for how long: data was submitted to the data archive on 2 September 2021; evaluation data are retained in the archive indefinitely, with the exception of names and pupil identifiers, which are retained until the end of the academic year in which the pupil reaches age 20. Previous IPD sharing statement Type of data stored: 1. Anonymised project data 2. A standardised dataset including pupil identifiers with fields renamed or recoded to support secondary analysis 3. A copy of the draft evaluation report to help validate the data and any other documents or materials to support secondary analysis 4. Syntax or Do files 5. A submission form listing the filenames and declaring the legal basis for processing Repository name and weblink: EEF data archive, currently run by Fischer Family Trust (FFT) but probably moving to ONS before the end of this project. There is no weblink. Requesting access: At the moment, the main use is FFT providing anonymised data extracts to EEF’s designated archive evaluator (Durham University). Once the EEF archive has been transferred to the ONS, it is intended that it will be accessible to the wider research community. Evaluators will analyse the data via the ONS Secure Research Service (SRS). The datasets will, in time, be added to the EEF archive, also hosted by the ONS SRS and managed by EEF’s data processor for the archive. Participant consent: participants are informed that their anonymised data will be deposited with the Fischer Family Trust and can withdraw from the research if they do not wish their data to be shared Any legal/ethical restrictions: strict procedures for protecting the integrity and confidentiality of the data processed on behalf of the EEF. All data processing activities must take place in accordance with the current Data Protection Act and GDPR. Independent evaluators must clearly specify and justify their legal bases for processing personal and any special data, data processing roles, parties with access to data and retention periods, in all relevant project documents. EEF evaluation data are processed by the archive manager on the basis of legitimate interests, according to the GDPR, Article 6, Paragraph 1(f), taking responsibility for protecting the fundamental rights and freedoms of the data subjects, and ensuring their interests are protected at all times. Type of data shared: evaluators will submit data to the DfE for matching to NPD attainment data, after which the DfE will release the matched data to the ONS. At the moment, the main use is FFT providing anonymised data extracts to EEF’s designated archive evaluator (Durham University) for the purpose of conducting secondary and longitudinal data analyses in order to track impact over time (using additional matching to NPD data obtained from the DfE), check data archive integrity and produce methodological outputs for the EEF Evaluation Advisory Board. Once the EEF archive has been transferred to the ONS, it is intended that it will be accessible to the wider research community for secondary analyses that provide public benefit and are in line with the missions of the EEF, DfE and ONS. When available and for how long: data will be submitted to the data archive within one month of report publication (July/August 2021); evaluation data are retained in the archive indefinitely, with the exception of names and pupil identifiers, which are retained until the end of the academic year in which the pupil reaches age 20. The anonymous pupils matching reference (PMR) is retained indefinitely. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | version 1.0 | 06/06/2019 | 11/08/2022 | No | No |
| Funder report results | 23/09/2021 | 03/11/2022 | No | No | |
| Statistical Analysis Plan | 07/09/2020 | 03/11/2022 | No | No |
Additional files
Editorial Notes
12/12/2022: Contact details updated.
03/11/2022: Statistical analysis plan and funder report added.
11/08/2022: Uploaded protocol (not peer-reviewed) as an additional file.
13/10/2021: The publication and dissemination plan has been updated.
09/09/2020: Following school/college closures and the resultant cancellation of GCSE exams during the COVID-19 pandemic, the GCSE grades awarded for June 2020 were not considered valid for the evaluation and it was not possible to administer the maths attitudes measure, so none of the primary or secondary outcome measures were available. An evaluation using case study and teacher survey data, plus data from the November 2019 resits is planned.
09/08/2019: The total final enrolment was added (88 clusters).
08/08/2019: The recruitment end date was changed from 03/07/2019 to 10/07/2019.
31/07/2019: Ethics approval details added.
24/07/2019: Trial's existence confirmed by funder.
