The new progress measures for KS2 pupils – an alternative view.
Nicky, Ricky, Dicky and Dawn all attended the same school (It’s a Wokingham school and each child is currently funded less than any other borough across the country – a blog for a different day!) At KS1, they all achieved the same APS score of 15.
Nicky – scored, R 2a, W 2b and M 2c.
Ricky – scored R 2b, W 2b and M 2b.
Dicky – scored R 2c, W 2b and M 2a.
Dawn – (part of our Sp and Lang Resource Unit) scored R 1, W 1 and M 3.
As a result, regardless of their vastly different profiles, their KS2 estimates would all be identical (with progress being judged against these too).
If Dicky did exceptionally well in his Reading, meeting the expected standard from a 2 c starting point, his judgement of progress is still reliant of the performance of other children. How is this fair? No matter how well Dicky does, he is not in control of his own destiny. When you compare averages with averages, by default, for every child above average there is a child below average. This has nothing to do with effort or expectation but simple mathematics.
Dawn, who has an EHCP for SLI is an able mathematician and as a result will have increased expectation for all subjects due to her KS1 APS. As the KS2 reading paper with require more emphasis on writing, her level 1 writing starting point with be a hinderance, but her progress expectations with be determined by her peers who could have had a completely different assessment profile.
The new system worries me greatly. I don’t have all the answers or a suggestion of a foolproof method but whatever it is, there needs to be a progress expectation that every child is able to achieve irrespective of how other children performed. This system, I feel, is setting up children to fail. I am not advocating a return to levels, far from it, but what replaces them needs to be an improvement and not just different. My school motto is ‘Where Individuals Matter’ – and that is how progress should be judged.
Matt Hickey
headhighwood 
It’s a very interesting problem… and I still don’t know quite how I feel.
I wonder if one of the challenges of a system like this is that the number of children who would match the profile of Dawn would be so few that it would be an unhelpful comparison group – hence putting them all into one APS group.
The other alternative would be to look only at progress in one subject, i.e. comparing Dawn’s progress to others in Maths who had a L3, but in Writing comparing her to others who had a L1. The chap from the DfE I spoke to pointed out that actually this is a less reliable predictor than the overall APS – because we know for the vast majority of children their progress in one subject does affect their progress in others (particularly for Reading, of course).
I agree that there are concerns, and I’d like to see more detail about how they’ve been addressed, but I do think that it may be possible that this is the least-bad approach that’s feasible.
LikeLike
Thank you for sharing your thoughts on this. I also think, like Michael, that this may turn out to be the least-bad approach, at least as far as school accountability goes.
Also, the current VA model is quite sophisticated and I don’t think all 4 pupils would have the same KS2 VA estimate. The 2015 technical guidance on calculating VA is in the RAISEonline Library.
LikeLike
I think we are in interesting and uncertain times, which worries me greatly. I think the fact we might have to be happy with the ‘least bad approach’ speaks volumes about the current state of our profession.
I have copied the section from the raise library:
Individual pupil VA scores need to be calculated before a school VA score can be produced.
The first step is to use a statistical model to calculate an “estimated outcome” for all pupils that are at the end of KS2 in 2015. Each pupil’s KS2 estimate is calculated based on the actual KS2 outcomes of all pupils nationally with the same level of achievement at KS1. For example, calculation of an estimated outcome for a pupil who scored an average of 15 points at KS1 will be based on the actual KS2 outcomes of all pupils nationally that also scored an average of 15 points at KS1.
A pupil’s VA score is then calculated by subtracting their estimated KS2 outcome from their actual KS2 outcome. Using the KS1-2 mathematics VA measure as an example, if a pupil attains a level 4 in KS2 mathematics (equivalent to 27 points) and they are estimated to attain a Level 3 (equivalent to 21 points) by the VA measure, then the pupil has a VA score of +6 points (27 points – 21 points).
I don’t see anywhere that says the KS1 APS will be interpreted with subject variations, but if you have any links I would be really interested to read further.
My school has a Speech and Language Resource Unit, so the profile of Dawn is much more common and the (externally credited) progress those children make is potentially going to be undermined by the new measure. When I have 4 in a cohort of 30, the published data looks very different to the personalised progress I am able to show. I just hope the new regime is as receptive to the internal evidence of progress as much as it is with externally ‘validated’ data.
LikeLike
Look at Section B p.14 of the Technical Annex of the doc KS1-2 General VA Guidance 2015- unamended in the RAISE Library. The calculation seems to me to include subject variations at KS1.
LikeLike
Hmmmm… The READEV and MATDEV do appear to imply subject variations are taken into account, which differs from a lot that I have read about the new measure. I still have concerns about the average to average comparison and it not allowing all children the chance to succeed.
LikeLike
At least things have moved on from the 2013 DfE consultation document which proposed reporting scaled scores to parents and contained this appalling example of what this would look like…
In the end of key stage 2 mathematics test, Tom received a scaled score of
87. He did not meet the secondary readiness standard (100). This places him in the bottom 10% of pupils nationally. The average scaled score for pupils with the same prior attainment was 92, so he has made less progress in mathematics than other pupils with a similar starting point.
The proposal was dropped after the consultation but it is profoundly worrying that the DfE ever entertained the idea.
LikeLike
Having done a bit more research it appears that using KS1 subject variations to calculate KS2 VA estimates may indeed not be included in the method used for the progress measure in 2016. I had assumed any new VA model would be at least as sophisticated as at present. I can’t believe I have been so gullible. This would be a long way off a ‘least-bad’ approach.
LikeLike
That was the premise for writing the blog. The thought that children’s reading / maths progress will be judged on the average score of children who may have completely different reading / maths starting points, to me, seems fundamentally wrong.
LikeLike