A large-scale trial of a one-to-one intervention designed to improve the working memory and numeracy skills of 7 – 8 year-old children who show low attainment in arithmetic
| ISRCTN | ISRCTN37136039 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN37136039 |
| Secondary identifying numbers | 21053 |
- Submission date
- 26/11/2021
- Registration date
- 10/12/2021
- Last edited
- 03/12/2021
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English Summary
Background and study aims
This is a trial of the Working Memory + Arithmetic (WM+A) intervention that seeks to improve pupils’ working memory, as well as the way they organise information about numbers and operations (addition and subtraction) in their long-term memory. It builds on evidence from cognitive science, which suggests that numeracy difficulties may be related to poor working memory capacity. A previous trial of the approach showed that the intervention had a positive impact on maths outcomes, with children in the intervention schools making the equivalent of three additional months’ progress in maths. Considering the positive results, it was decided to test the programme in a larger number of schools through an effectiveness trial and assess the impact on pupils eligible for Free Schools Meal (FSM). This effectiveness trial will test a more scalable version of the intervention, which includes a revised ‘train the trainer model’, assessing its potential for further rollout.
Who can participate?
The intervention was designed by the University of Oxford to improve the numeracy skills of 7 – 8-year-old children who show low attainment in arithmetic at the end of Key Stage 1.
What does the study involve?
The study aims to recruit between 200 – 240 schools in England to participate, with 10 Year 3 pupils being nominated to take part from each school. Schools will be allocated to one of two groups, to receive either the intervention or business-as-usual maths teaching, with an equal chance of being in either group (like tossing a coin). The trial will look at pupils’ maths outcomes at the end of the first school year using a combination of two standardised maths measures and the sub-test ‘Number Skills’ of the British Ability Scales (BAS3, which was the assessment used during the efficacy trial).
What are the possible benefits and risks of participating?
Schools allocated to treatment will receive IWM+A training and be expected to deliver the IWM+A programme, while schools allocated to control will be expected to carry on with business as usual (BAU). Schools assigned to the control group will be given £1000 on completion of all assessments. We see no potential risks in participating.
Where is the study run from?
The study is run by RAND Europe (UK). The delivery team that developed the intervention and are responsible for training and implementation are the University of Oxford (UK)
When is the study starting and how long is it expected to run for?
From December 2019 to March 2023
Who is funding the study?
The Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Elena Rosa Brown
Contact information
Public
RAND Europe
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
| Phone | +44 (0)7816 334317 |
|---|---|
| erbrown@randeurope.org |
Study information
| Study design | Region-stratified two-arm cluster-randomized effectiveness trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use the contact details to request a participant information sheet |
| Scientific title | Improving Working Memory plus Arithmetic Effectiveness Trial |
| Study acronym | IWM+A |
| Study hypothesis | Number skills as measured by the number skills subtest of the British Ability Scales (BAS3) of pupils in schools receiving IWM+A will be better in comparison to those pupils in control schools receiving business as usual. |
| Ethics approval(s) | Determined to be exempt from further review 19/02/2020, RAND Human Subjects Protection Committee (1776 Main Street, PO Box 2138, Santa Monica, CA, USA; +1 (0)866 697 5620; hspcinfo@rand.org), ref: 2020-N0915 |
| Condition | Supporting low attaining pupils in maths at age 6/7 (Year 3) |
| Intervention | Random allocation will be at the school level to avoid contamination, with schools randomly allocated to the treatment condition delivering IWM+A and schools randomly allocated to control condition delivering business as usual. Randomisation will be stratified by region with schools as the unit of randomisation, and (Year 3) pupils as the unit of analysis. Given delivery and recruitment are organised regionally (i.e. with one TL responsible for recruiting and delivering in a single region) we will stratify by region. Randomising at the school level also reduces the chances of contamination between randomised groups as TAs will be delivering to all identified children in the school. Stratifying at the region/trainer level is particularly relevant given the regional nature of delivery with one TL delivering the training in each region (see the ‘Intervention’ section for more details). Regions to be included are: Cambridgeshire, Derbyshire, Devon, East Sussex, Essex, Greater Manchester, Merseyside, Nottinghamshire, South Yorkshire, Suffolk, West Sussex, West Yorkshire. Stratifying by region will ensure that TLs in each region have a number of schools they will be able to train. We will use a package in stata (randtreat) to avoid unequal allocation due to an uneven number of schools within each region. Randomisation will occur with a 50:50 allocation to treatment and control. Schools allocated to treatment will receive IWM+A training and be expected to deliver the IWM+A programme, while schools allocated to control will be expected to carry on with business as usual (BAU). Schools assigned to the control group will be given £1000 on completion of all endline assessments. These funds are to be used at the discretion of the school and could be used to buy an intervention programme of their choice, including IWM+A from September 2022. The value of the incentive is the same that was used in the efficacy trial. We will use the same adapted scale that was used in the efficacy trial. As IWM+A is a targeted intervention, schools will only be eligible for randomisation once target children have been identified and all requirements for randomisation have been completed (e.g. providing student identifiers and teacher scores on SNAP-IV, confirming information sheets have been shared with parents and carers) and data checked for completeness by the Oxford University team. Randomisation will be conducted in Stata by a member of the evaluation team, who will be blind to treatment allocation. The code used to randomise schools as well as all relevant variables will be saved and made available if requested. The trial allocation will be recorded in Excel and communicated to the implementation team in a PDF file to prevent editing. The Improving Working Memory + Arithmetic (IWM+A) intervention is for 6- to 8-year-old children who show low attainment in arithmetic at the end of Key Stage 1 (KS1) (children aged 5 to 7 years). It combines a WM, central executive intervention with an arithmetic intervention, and was designed to improve WM span and the way numbers and arithmetic operations are organised in long-term memory. Thus the intervention targets two components of WM, the central executive and the episodic buffer, which involves an input from long-term memory. The activities were designed using research findings about how attention can be promoted and how numerical information can be stored in long term memory and processed in effective ways. During the WM intervention, children learn strategies for keeping information in mind so that their increased WM capacity allows them to learn basic number concepts, such as additive composition and the inverse relation between operations. As their WM improves, children can learn these concepts and represent numbers and operations in their long-term memory in a new way. The combination of WM with an additional component that aims to enhance the understanding of underlying principles of arithmetic is particularly promising, as Craick & Lockhart famously showed that deeper, more elaborate processing leads to better learning (Craik & Lockhart, 1972). This is supported by previous studies by the Oxford University Team (Nunes et al., 2008; Nunes et al., 2014). In addition, there is evidence of the arithmetic element of the intervention being successful, with previous evaluations of similar approaches from the same Oxford University Team showing a positive impact on pupils’ numeracy ability (Worth et al., 2015). WM+A is delivered through one-hour one-to-one sessions by Teaching Assistants (TAs) and includes the following components: a working memory programme, delivered over five sessions, and a number and arithmetic operations component, delivered over five subsequent sessions. During each session, the TA works for half an hour with one child while a second child plays specially designed online games in the same room to practice the strategies. After half an hour, the children swap activities. The sessions should take place once a week for ten weeks TAs receive 2 days of training in the approaches – 1 day focused on the working memory aspect and a second on the arithmetic elements. |
| Intervention type | Behavioural |
| Primary outcome measure | Pupils’ number skills measured using the number skills subtest of GL Assessment’s British Ability Scales 3rd Edition (BAS3) at endline (at the end of Year 3) |
| Secondary outcome measures | 1. Pupils working memory is measured using the WM Battery Test for Children at endline 2. Pupils’ wider maths attainment will be measured using GL Assessment’s Progress Test in Maths 8 (PtM8) at endline 3. Pupils’ attention and behaviour will be measured using SNAP-IV Teacher Attention rating Scale at baseline (at the start of Year 3) and endline |
| Overall study start date | 13/12/2019 |
| Overall study end date | 20/03/2023 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Sex | Both |
| Target number of participants | 10 pupils per school at 200 – 240 schools |
| Total final enrolment | 2100 |
| Participant inclusion criteria | Classroom teachers will be responsible for selecting children to participate in IWM+A before the start of the trial (i.e. pre-randomisation). Pupils will be the ten lowest-attaining students in mathematics at the end of their KS1, according to the teachers’ judgements. In the previous trial, teachers were encouraged to consider KS1 results in their nominations. For the purposes of the trial, children will be identified in September of Year 3 (i.e. at the start of the school year). However, given that Year 3 teachers may not be familiar with their new class, the children’s former Year 2 (i.e. end of KS1) teachers will be asked to collaborate with Year 3 teachers to identify appropriate pupils. Teachers will be encouraged to use their own judgement, and may base this on their own evaluation of how children perform in the mathematics lessons and activities. In the efficacy trial teachers were asked to use KS1 assessments, but these were cancelled in 2021 as a result of the impact of COVID-19. For schools with more than one form entry, teachers of all the classes will cooperate and identify the 10 pupils showing lowest performance in numeracy across the year group. Schools will only be able to nominate 10 children, regardless of the size of the school. |
| Participant exclusion criteria | 1. Deafness, blindness, and/or physical restrictions that might interfere with a child’s ability to use the online games 2. Behavioural problems that might interfere with a child’s ability to work independently and in the same room as another child 3. Children whose level of fluency in English would prevent them from engaging with the computer games |
| Recruitment start date | 01/03/2021 |
| Recruitment end date | 20/09/2021 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
United Kingdom
Sponsor information
Charity
Millbank Tower
21-24 Millbank
London
SW1P 4QP
England
United Kingdom
| Phone | +44 (0)204 536 3999 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 20/03/2023 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in publicly available repository |
| Publication and dissemination plan | 1. Protocol 2. Statistical analysis plan 3. Final report |
| IPD sharing plan | Data from the trial (i.e. pupil identifiers, Pupil Matching Reference, pupil-level impact data) will be stored in the EEF Data Achieve. More details on the EEF data archive can be found at https://d2tic4wvo1iusb.cloudfront.net/documents/evaluation/archiving-evaluation-data/Archiving_evaluation_data_analysed_in_the_SRS_-_November_2020.pdf. In summary the archive stores pupil-level impact evaluation data including primary and secondary outcomes, data used in statistical analysis (i.e. compliance, fidelity, sensitivity). The archive is managed by FFT Education and is accessible through the Office for National Statistics (ONS) Secure Research Service (SRS). The approach to accessing data from the EEF archive is currently being piloted, but will be open to the research community for analysis in the SRS. Participants and their guardians have been informed through privacy notices and information letters that their data will archived and potentially shared with other parties after archiving, may be relinked to the National Pupil Database (NPD) or other datasets after archiving. Contact details of the evaluation team were provided should participants or their carers wish to remove their data from the evaluation or the archive at any time. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | version 1.0 | 01/08/2012 | 03/12/2021 | No | No |
Additional files
Editorial Notes
03/12/2021: Trial’s existence confirmed by RAND Human Subjects Protection Committee.
