This information paper is designed for members of the senior leadership team and data managers, and seeks to explore the challenges facing school leaders as they respond to the ever more sophisticated range of data available to schools.
Ofsted, the DfE and local authorities often base judgements about schools on the messages that they derive from these data systems, so it is vital these systems are understood.
Throughout this information paper, ‘Percentage of cohort getting 5A*-C incl GCSE English and Maths’ is abbreviated to ‘%5A*-C inc En+Ma’. Also, ‘Percentage of cohort getting A*-C in both GCSE English and Maths’ is abbreviated to ‘%A*-C En+Ma’.
June 2016 will be the first year that Progress 8 takes effect for school accountability, along with other headline measures. This is a major step forward as it means that the results of every child at every grade in several subjects will count, rather than just a few around the threshold, and whilst also accounting for their prior attainment. But as with any change, it is important to understand the detail and be prepared.
This paper considers the following key questions and prompts:
ASCL has liaised closely with the DfE from the beginning of the discussions about Progress 8. We have worked to ensure that, whatever views there may be about the formulation of Progress 8, the supporting documentation from the DfE and ASCL is consistent and that the results for 2014 and 2015 were made available to each school.
In particular, it was important to emphasise that Progress 8 is a relative measure, calculated each year on the basis of the actual results of all of the pupils taking exams at the end of KS4 that year. Therefore, nobody knows in advance of analysis on national results in August what the national average will be, and consequently, the answer to question 1 is ‘no’.
On the other hand, you will be able to calculate your Attainment 8 score and convert it to an average grade, which will be one of the new accountability measures (as will %A*-C En+Ma - see response to question 3 for further information).
This is a challenging question. The DfE’s Statement of Intent update for the June 2016 results, highlighting what will be appearing in the Performance Tables, usually appears towards the end of July. The latest guidance currently available (and updated 20 January 2016), is:
Progress 8 Measure in 2016, 2017, and 2018 (page 7) states:
What will be published in the performance tables and RAISEonline from 2016
The headline measures which will appear in the performance tables will be:
We are currently consulting on the addition of a fifth headline measure to show the percentage of pupils entered
Parents and the local media are still probably expecting %5A*-C inc En+Ma, so you may wish to use one of the new headline measures %A*-C En+Ma, which actually is likely to be similar because they are both threshold measures.
Threshold measures: where a pupil has to reach a certain standard or threshold to count; this can then be
expressed as percentage of cohort etc, eg % of pupils gaining 5A*-C grades.
Note that in practice:
results of a small range of pupils can affect a threshold measure eg those around C/D borderline for %A*-C, and
one element of a multiple threshold can dominate, eg in practice there is very little difference between:
%5A*-C inc En+Ma (av across schools = 57.1 per cent)
and %A*-C En+Ma (av across schools = 59.4 per cent)
(DfE SFR June 15 table 3a and 4a of pupils at end KS4 in state mainstream).
So, 96% of pupils who gained %A*-C En+Ma also gained %5A*-C inc En+Ma.
In 2013 figures were:
58.6% and 59.4% ie 99% of pupils who gained %A*-C En+Ma also gained %5A*-C inc En+Ma.
Graph 1 shows that big differences between the EBacc3 and Other3 ‘buckets’ (or elements) in 2014 and 2015. We know that most schools have changed their curriculum and exam entry policy for current Year 11 so that the majority of pupils are taking three EBacc subjects, so we are expecting the number of slots filled in the EBacc3 ‘bucket’ to increase in June 2016 (graph 2) with a concomitant rise in the national average score. So, if you are not changing your entry policy (and results are the same) then it is likely that your EBacc3 score will drop!
The 2015 results are still available on the DfE Tables Checking website. The ‘base’ data for June 2016 will be available between 13 and 27 June so that you can check who will be counted in the Performance Tables ‘cohort’ results – effectively the denominator in the percentages (your school should have received a letter from DfE).
David Blow has prepared spreadsheets downloadable from the ASCL website so that you can paste in the 2015 DfE pupil datafile (see Further information at the end of this paper).
It is particularly important for schools with the advantage of high prior attainment and very high threshold output attainment to look at these files carefully. Progress 8 will enable a fair and effective differentiation amongst schools which are currently clumped together with a high %5A*-C inc En+Ma figure. The Ofsted data dashboard quintiles were particularly flawed for these schools, and Progress 8 offers a fair way of analysing their relative performance.
To many people’s surprise, the answer is ‘no’. Although ‘Overall % Expected Levels of Progress’ is being removed from the headline measures, there will still be the overall mindset change which is needed for schools to move from threshold measures to average measures, which take into account the result of every child at every grade.
Graph 3 shows the results for every school in the country for 2015, plotting the attainment measure %5A*-C inc En+Ma against ‘overall % making expected levels of progress’. There is a remarkable correlation, showing that overall % making expected levels of progress is actually a proxy for attainment.
The underlying reason is that a ‘level’ at KS2 is very different from a ‘level’ (or GCSE grade) at KS4. Most pupils on KS2 level 5 will gain a GCSE grade B (and thus make three levels of progress), whereas it is much more difficult for a pupil on KS2 level 3 to gain a GCSE grade D. Thus, schools with high prior attainment will have high %5A*-C inc En+Ma and overall % making expected levels of progress.
Actually, the English department is probably doing better than the maths department (although both might be over-performing or both underperforming). This is explained by graph 4 which shows the change in ‘%A* in maths and English’ in the last six years. In 2015, maths = 7.2 per cent, English = 3.2 per cent, ie more than twice as many maths* grades as English
How can you know how the maths and English departments are performing across the range of prior attainment, not just at the threshold of %A*-C? Look at the National Subject Transition Matrices and download the ASCL spreadsheet to enable you to compare the results from your school, not just English and maths, but in a total of 40 common GCSEs, against what they should be if in line with national expectation.
These are published each year by DfE and Ofsted for 40 subjects. They are also available by gender (which helps single-sex schools) and by disadvantage or not.
They give the actual number of students nationally gaining each GCSE grade in a particular subject from a given KS2 starting point. This enables fair comparisons to be made taking into account the prior attainment of the pupils involved. David Blow’s spreadsheet has the National TMs pre-loaded, and so all you have to do is a one-off pasting in of your school’s data from the DfE Tables Checking data file, to enable you to get a picture for each subject.
Not at all. Currently, the KS2 test marks are converted into a fine score which can then also be converted into a sub-level and level, but Progress 8 is based around the fine score. A similar process will take place once we have the marks and scaled score; there will still be a clear, finely gradated KS2 input value.
This information paper has been written by David Blow, Head of The Ashcombe School in Surrey and an ASCL member. David has kindly provided a range of spreadsheets and information regarding performance measures, with links detailed below.
DfE Progress 8 School Performance Measure (20 January 2016)