Evaluation of English Mastery
| ISRCTN | ISRCTN14494646 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN14494646 |
| Secondary identifying numbers | NATCEN REF P13637 |
- Submission date
- 09/04/2020
- Registration date
- 15/04/2020
- Last edited
- 28/06/2022
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English Summary
Background and study aims
The intervention being tested is English Mastery, a programme which trains and provides resources to English teachers to deliver a knowledge-rich Key Stage 3 English curriculum.
The expected outcomes from this intervention at the teacher level are an increase on teachers’ effectiveness to teach literary texts, grammatical concepts and vocabulary; better understanding of how to use assessment to drive planning and delivery and give feedback to pupils and reduced teacher workload and improved confidence. The programme is expected to help pupils possess the key competences required to succeed at GCSE, A’ level, university and beyond.
Who can participate?
English Mastery is designed to be delivered by secondary English teachers. For the evaluation we focus on Year 7 and trace their progress in Year 8.
What does the study involve?
The evaluation is a two-armed cluster randomised controlled trial with 100 schools, half of them under the English Mastery intervention and half of them in a control group.
All pupils in the treatment group enrolled in Year 7 in the academic year 2019-20 are expected to receive the intervention for a period of two years. The trial will use Key Stage 2 English SATs results as a baseline measure of academic attainment (composite score incorporating grammar, spelling, punctuation, language strategies, reading and comprehension). To track progress in learning as a primary outcome, we will measure participating pupils’ English attainment in both treatment and control groups at the end of the intervention by administering the GL Progress Test in English (PTE).
The primary outcome is a standardised measure of pupils’ English skills and reading comprehension, the GL Progress Test in English (GL PTE). Both the GL PTE SPAG subscale, which measures spelling, grammar and punctuation, and the ‘reading comprehension’ subscales will be included in the analysis as secondary outcomes. Pupils will be assessed in June 2021, at the end of Year 8 (second year of the intervention).
What are the possible benefits and risks of participating?
The programme is expected to help pupils gain the key competences required to succeed at GCSE, A’ level, university and beyond. There are no risks for participating students.
Where is the study run from?
1. National Centre for Social Research (NatCen) (UK)
2. English Mastery (UK)
When is the study starting and how long is it expected to run for?
February 2019 to December 2021
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
Malen Davies
malen.davies@natcen.ac.uk
Contact information
Scientific
35 Northampton Square
Clerkenwell
London
EC1V 0AX
United Kingdom
 ORCID ID |
0000-0003-0653-767X |
|---|---|
| Phone | +44 0207 549 8589 |
| valdeep.gill@natcen.ac.uk |
Scientific
35 Northampton Square
Clerkenwell
London
EC1V 0AX
United Kingdom
| ORCID ID |
0000-0001-8840-4875 |
|---|---|
| Phone | +44 0207 549 8519 |
| robert.wishart@natcen.ac.uk |
Study information
| Study design | Implementation and process evaluation |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Implementation and process evaluation |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | English Mastery: A cluster-randomised efficacy trial |
| Study acronym | English Mastery |
| Study hypothesis | 1. Participating in English Mastery improves Year 8 pupils’ English attainment, as measured in GL’s Progress Test in English. 2.1. Participating in English Mastery improves Year 8 pupils’ spelling, punctuation, and grammar, as well as reading comprehension, as measured by the GL subscales of PTE. 2.2. Receiving training and lesson materials reduces teacher workload in and out of class, as measured by a survey of English Mastery teachers. 2.3. Participating in English Mastery improves long-term attainment as measured by GCSE results. |
| Ethics approval(s) | Approved 11/07/2019, NatCen Social Research Research Ethics Committee (35 Northampton Square, London EC1V 0AX; Recadmin@natcen.ac.uk; +44 0207 250 1866), ref: P13637 English Mastery Trial |
| Condition | English attainment at the end of Year 8 |
| Intervention | EM programme aims to implement a coherent and cumulative approach to curriculum design for English teaching including bespoke assessments to improve achievement in the subject of English. This is done by providing teachers with subject-specific training, curriculum materials and ongoing in-school support and coaching. EM is a comprehensive programme with two pathways: the traditional curriculum for children reaching age-related expectations, and the foundation curriculum for those working below expectations. All pupils receive the same dosage and study the same topics, but the foundation curriculum is adapted to be accessible to lower attaining pupils (for example, using abridged texts). The English Mastery curriculum is implemented in regular classrooms of participating schools in England. English Mastery lessons are designed to be delivered six times a week in a way that fits the school timetable. The units run across each school term over two years, the academic years 2019-20 and 2020-21, in which the pupils will be in year 7 and 8. To facilitate timely programme training delivery among treatment group schools, schools needed to be assigned to treatment and control groups on a rolling basis before the beginning of the new academic year (2019-20). Schools agreeing to participate in the trial will be allocated to one of the two groups utilizing a ‘minimisation’ algorithm with three prognostic factors and a 1:1 ratio between treatment and control groups, therefore with 50 schools in each group. Pupils will be assessed in June 2021, at the end of Year 8, the second year of the intervention. Teachers will complete 3 online surveys on their workload during the intervention period. These surveys will be completed in: September-October 2019 (survey 1), May 2020 (survey 2), and May 2021 (survey 3). From the sample of trial participants (in both treatment and control schools), 60 individuals will be randomly chosen in each school to take part in testing at the end of the intervention. This group corresponds to the sample of evaluated participants. |
| Intervention type | Not Specified |
| Primary outcome measure | Current primary outcome measures as of 28/06/2022: 1. Teachers' perception of EM programme design and how the programme is delivered measured using a survey at three timepoints (September 2019, May 2020 and May 2021) and in-depth interviews with teachers at three timepoints (February 2020, October 2020 and April 2021) 2. Teachers' perception of key programme challenges measured using a survey at three timepoints (September 2019, May 2020 and May 2021) and in-depth interviews with teachers at three timepoints (February 2020, October 2020 and April 2021) 3. Teachers' perception of whether they have benefited from the programme and changed their teaching practices measured using a survey at three timepoints (September 2019, May 2020 and May 2021) and in-depth interviews with teachers at three timepoints (February 2020, October 2020 and April 2021) 4. Teachers' perception of the benefits of English Mastery on students in relation to 1) students’ enjoyment of studying English and 2) progress in reading and writing measured using a survey at three timepoints (September 2019, May 2020 and May 2021) and in-depth interviews with teachers at three timepoints (February 2020, October 2020 and April 2021) 5. Teachers' perception of whether local and national contextual issues, including changes to school regulations and inspections, affected delivery and adaptation measured using a survey at three timepoints (September 2019, May 2020 and May 2021) and in-depth interviews with teachers at three timepoints (February 2020, October 2020 and April 2021) Previous primary outcome measure (not measured due to cancellation of RCT): English skills and reading comprehension of pupils, measured using the GL Progress Test in English (GL PTE) at 2 years |
| Secondary outcome measures | 1. Spelling, grammar, and punctuation measured using the GL PTE SPAG subscale at 2 years 2. Reading comprehension measured using the ‘reading comprehension’ subscales at 2 years 3. Teachers’ workload measured using data collected from an online survey of Year 7 English teachers at 6 months, 1 and 2 years Updated 28/06/2022: Not measured due to cancellation of RCT |
| Overall study start date | 11/01/2019 |
| Overall study end date | 01/09/2021 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Sex | Both |
| Target number of participants | 100 schools, 6000 pupils selected for testing |
| Total final enrolment | 0 |
| Participant inclusion criteria | Non-academically selective, state secondary schools in England |
| Participant exclusion criteria | Schools: 1. Fewer than 4 h of English teaching timetabled per week in Year 7 or 8 2. Grade 4 according to OFSTED school classification 3. Previously partnered with English Mastery Pupils: 1. Pupils whose parents have withdrawn them from the trial |
| Recruitment start date | 01/02/2019 |
| Recruitment end date | 25/06/2019 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
1 Nicholas Road
London
W11 4AN
United Kingdom
Sponsor information
Charity
35 Northampton Square
Clerkenwell
London
EC1V 0AX
United Kingdom
| Phone | +44 0207250 1866 |
|---|---|
| info@natcen.ac.uk | |
| Website | http://natcen.ac.uk/ |
"ROR" |
https://ror.org/057z98j75 |
Funders
Funder type
Government
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 30/01/2022 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Stored in repository |
| Publication and dissemination plan | We will publish an Evaluation Report in January 2022, which will include the impact estimates for the primary and secondary outcomes. This will be published on the study page of the Education Endowment Foundation website. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study will be available upon the submission of a data request. Data will be archived with the Fischer Family Trust (FFT) for EEF. Access to this is provided only on submission of a data request, and any NPD data is accessed via the ONS SRS. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Funder report results | 29/04/2022 | 28/06/2022 | No | No |
Editorial Notes
28/06/2022: Owing to the COVID-19 pandemic and associated school closures the design was modified and the RCT was cancelled. A total of 97 schools were enrolled onto the trial. The researchers did not enrol any pupils onto any testing because the RCT was cancelled. The following changes were made to the trial record:
1. Publication reference added.
2. The primary and secondary outcome measures were updated.
3. The study design was changed from 'Two-arm cluster randomised controlled trial with 100 schools (1:1 the English Mastery intervention or control group)' to 'Implementation and process evaluation'.
4. Total final enrolment added.
14/04/2020: Trial’s existence confirmed by Education Endowment Foundation.
