Embedding formative assessment to improve pupil attainment: A randomised trial in schools
| ISRCTN | ISRCTN10973392 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN10973392 |
| Secondary identifying numbers | N/A |
- Submission date
- 06/07/2015
- Registration date
- 16/07/2015
- Last edited
- 15/08/2022
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English Summary
Background and study aims
Embedding Formative Assessment (EFA) is a whole school professional development programme that aims to develop the quality of feedback provided by teachers to their pupils. The idea is that teachers can guide themselves through the materials to run a carefully structured series of workshops. The pack was developed by SSAT with Dylan Wiliam. It aims to improve pupils' test scores through the improved feedback they receive. There is evidence that feedback is effective in improving pupils' learning. To be effective and to be formative, pupils need to change what they would have done as a result of the feedback, spend their efforts differently and often in very specific areas.
Who can participate?
Approximately 120 state-funded schools in England who are happy to commit to running the EFA programme if they are placed in the treatment group and not to run the programme if they are placed in the control group. Within the schools, we mainly focus on the test scores of those in school Year 10 (age 14-15) when the two year programme starts.
What does the study involve?
Schools are randomly allocated to one of two groups: treatment or control group.
The treatment group will receive the EFA pack, a day’s training and materials to deliver 18 monthly internal workshops (“Teacher Learning Communities”) in schools.
The control group will receive a payment of £300 (the cost of buying the pack) but will otherwise be asked to continue as if they were not participating in the study
What are the possible benefits and risks of participating?
As there is some evidence of improved test-scores from improved feedback, a possible benefit is the improvement of these test scores. Risks, other than missing out on a potentially beneficial programme if placed in the control group, are minimal.
Where is the study run from?
The study is run by The Schools Network (SSAT) and the evaluation is being run by the National Institute of Economic and Social Research. Both organisations are based in London.
The study includes schools from across England.
When is the study starting and how long is it expected to run for?
April 2014 to August 2017. The intervention will start in the academic year beginning September 2015, and will continue for two academic years. After this time, treatment and control schools will be free to start, continue, or end the programme as they wish.
Who is funding the study?
Education Endowment Foundation (UK).
Who is the main contact?
Dr Jake Anders
Contact information
Scientific
National Institute of Economic and Social Research
2 Dean Trench Street
London
SW1P 3HE
United Kingdom
 ORCID ID |
0000-0003-0930-2884 |
|---|
Study information
| Study design | Interventional school-level cluster randomised controlled trial; the participants are not blinded, but the outcome measurement is. The trial is being organised by a single-centre but includes schools from across England. |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | School |
| Study type | Quality of life |
| Participant information sheet | Available from the trial website https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/embedding-formative-assessment |
| Scientific title | Embedding formative assessment to improve pupil attainment: A school-level cluster randomised controlled trial |
| Study hypothesis | Use by schools of the "Embedding Formative Assessment", a pack that promotes a systematic approach to developing high quality feedback through continuing professional development, will improve children's performance in academic tests at age 16. |
| Ethics approval(s) | The National Institute of Economic and Social Research (NIESR) adheres to the Ethics Guidelines of the Social Research Association (SRA). |
| Condition | Academic performance at age 16 |
| Intervention | Treatment: Receipt of "Embedding Formative Assessment" pack, attendance at launch event, and two years of mentoring to support its usage. Control: Business as usual and payment of £300 (market price of pack). |
| Intervention type | Behavioural |
| Primary outcome measure | The outcome measures are all pupil attainment measured using General Certificate of Secondary Education (GCSE) results collected through the National Pupil Database (NPD); these are taken at the end of compulsory schooling (age 16). The primary outcome is the Attainment 8 score. This is calculated as the sum of an individual’s best 8 GCSE qualifications (of which two must be maths and English, which are also double weighted). At present the points attached to each grade are 8 points for A*, 7 for A, 6 for B, 5 for C, 4 for D, 3 for E, 2 for F, and 1 for G, but this will change for the cohort we use (in 2017 it is planned to be 8.5 points for A*, 7 for A, 5.5 for B, 4 for C, 3 for D, 2 for E, 1.5 for F, and 1 for G). We will use the prevailing grade points system, while will also cover GCSEs with the reformed grades system Identical data will be collected on children in control schools. All outcomes are at the individual level. By using NPD data for pre- and post-test the measurement will be blinded, since pre-test will use attainment at age 11 (i.e. pre-randomisation) and post-test will be marked by GCSE markers who do not know the identity of the schools whose examination scripts they are marking. |
| Secondary outcome measures | 1. GCSE English grade (using the prevailing grade points system) 2. GCSE Maths grade (using the prevailing grade points system) All above outcomes for a disadvantaged sub-sample (defined as those eligible for Free School Meals) All above outcomes for a high-attaining sub-sample (defined as those in the top quintile of performers in end of Key Stage 2 tests taken at age 11) All above outcomes for a low-attaining sub-sample (defined as those in the bottom quintile of performers in end of Key Stage 2 tests taken at age 11) |
| Overall study start date | 01/04/2015 |
| Overall study end date | 31/08/2017 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Sex | Both |
| Target number of participants | 120 schools (clusters), with an estimated 100 participants per age group within each. |
| Total final enrolment | 25393 |
| Participant inclusion criteria | The study will include secondary schools drawn from across England. Within each school, the trial will primarily focus on pupils starting Year 10 when the intervention begins. In addition, we will analyse the outcomes of those starting Year 11 when the intervention begins, as secondary outcomes. Furthermore, where possible, opt-out consent will be sought for pupils in year groups lower down the school (with the exception of the bottom year, usually Year 7, since they will not yet be in the school at time of requesting consent) in order to make it possible for longer-term impacts to be explored (although this is beyond the scope of this protocol). In order to be considered, schools will have to agree to provide student data in order that it is possible to apply for data from the National Pupil Database, to allow CPD time for all teachers to participate in the Teaching and Learning Communities, and to cooperate with the project and evaluation teams during the trial (further details of these requirements are outlined in the Memorandum of Understanding with Schools, included with this document in Appendix E). SSAT will advertise the trial on its website and also approach schools via its existing schools network. Where possible it will aim to recruit schools that have larger populations of individuals receiving Free School Meals. |
| Participant exclusion criteria | 1. Parents may choose to opt their pupils out of the trial; they will be excluded from all analysis. 2. Independent schools are excluded from the trial. |
| Recruitment start date | 21/04/2015 |
| Recruitment end date | 08/05/2015 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centres
SW1P 3HE
United Kingdom
EC1V 8AR
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21-24 Millbank
London
SW1P 4QP
United Kingdom
| Website | http://educationendowmentfoundation.org.uk/ |
|---|---|
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
No information available
Results and Publications
| Intention to publish date | 31/12/2020 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository |
| Publication and dissemination plan | We will publish an Evaluation Report in Spring 2018, which will include the impact estimates for the primary and secondary outcomes. This will be published on the study page of the Education Endowment Foundation website. |
| IPD sharing plan | Current IPD sharing statement as of 15/08/2022: The datasets generated and analysed during the current study have been archived in the Education Endowment Foundation (EEF)’s data archive, and will be available from this data archive in the future, on application to the EEF. Consent for this archiving and administrative data linkage was obtained from participants as part of the study. At the time of writing, access is not yet possible and the timing of access being made available is dependent upon the EEF, rather than the project team. The data archived includes all variables used as part of the analyses reported, named in line with a consistent data specification provided by the EEF (see appendix 3 of their policy on data archiving: https://d2tic4wvo1iusb.cloudfront.net/documents/evaluation/archiving-evaluation-data/Archiving-evaluation-data-analysed-in-the-SRS-Project-completion-and-archiving-form_v2.pdf). Application for access to the EEF data archive is contingent on providing detail of the public benefit of the research to be carried out, in line with the EEF data archive privacy notice: https://educationendowmentfoundation.org.uk/privacy-notices/privacy-notice-for-the-eef-data-archive; this will be assessed by the EEF as archive data owners, rather than the project team. Archive data are held and shared within a secure environment (UK Office for National Statistics Secure Research Service; ONS SRS), for which applicants must meet relevant requirements (including application for recognition as an accredited researcher from the ONS), as mitigation against potential identifiability risks. Previous IPD sharing statement: Data analysed as part of this project have been archived and are available on application to the Education Endowment Foundation Data Archive. Further details available from the Education Endowment Foundation: https://educationendowmentfoundation.org.uk/projects-and-evaluation/evaluation/evaluation-guidance-and-resources/archiving-evaluation-data |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Funder report results | 01/07/2018 | No | No | ||
| Results article | 03/03/2022 | 11/04/2022 | Yes | No | |
| Protocol (other) | 08/02/2016 | 12/08/2022 | No | No | |
| Statistical Analysis Plan | 31/01/2018 | 12/08/2022 | No | No |
Editorial Notes
15/08/2022: IPD sharing statement updated.
12/08/2022: Protocol and statistical analysis plan added.
11/04/2022: The following changes have been made:
1. Publication reference added.
2. The Individual participant data (IPD) sharing statement has been added.
12/05/2021: The final enrolment number has been added.
23/10/2018: The following changes were made:
1. Online report added to publication and dissemination plan.
2. The intention to publish date was updated from 31/03/2018 to 31/12/2020.
