Notices 20/02/2019

1) No Progress 8 for Y7 - Y9? See https://www.4matrix.com/node/6222

2) KS5 Value Added showing warning triangles? See https://www.4matrix.com/help/ks5_va_2018.pdf

3) Up-to-date information on getting help with 4Matrix: https://www.4matrix.com/help/getting_help.pdf

### Subject Summary Commentaries, Residuals, Distribution etc

#### Solution

This shows the number of pupils achieving grades in this range (or equivalent).  The national expectation of A*-C numbers for a typical school has also been calculated. This estimate is calculated from the published national A*-C rate for subjects in maintained schools.  The total derives from separate estimates for boys and girls.

Subject Residuals (June 2017)

The new Subject Summary and Subject Overview tools added as part of update 3.8.0.0 (June 2017) report a residual value which is different to the residuals reported elsewhere in 4Matrix.

All grades for subjects that contribute to performance measures are included in the pupil APS and residual calculations. This includes grades that are discounted for any reason, such as multiple entries into the same family of qualifications and pathway discounting. For example, a pupil who studies Chemistry will be on the 'single sciences' pathway. A subsequent entry in Core Science will not count in performance measures. For the purpose of calculating residuals, this pupils' APS will include grades from both sciences, they will be given a residual for both sciences and the Core Science subject residual will include the discounted grade.

This measure is the average of the distance between each pupil's grade and their overall average grade, reported in points (not grades, as per older tools in 4Matrix). Higher than average is scored as positive and lower is negative.  Thus a subject residual of 0.00 tells you that on the whole pupils did just as well in this subject as they did in all their subjects.  A subject residual of +1.50 means that the pupils performed one and a half points better in this subject than their overall performance.

The reason for reporting the difference in points (and not grades) is because there are 2 point scales in use for 2017 onwards; 9-1 and 8.5-1. On the 8.5-1 scale which is used for legacy GCSEs, the points difference between each grade is not equal. This can also apply to vocational subjects. To report a residual of +1.50 (one and a half) grades may not necessarily be correct given the grade boundaries, but to say that it is 1.50 points will be truthful to both new and old grade scales, and GCSE and non-GCSE qualifications.

Another key difference between old and new residual calculations is the treatment of subjects which do not count in performance measures. The new residual calculation does not include qualifications that are excluded from the performance measures. An example of this might be where a pupil studies a non-counting entry-level qualification alongside their GCSEs. The entry-level qualification is not included when working out this pupil's average point score, nor will this qualification be given a residual score. This is mainly due to the fact that we can only officially confirm the points values for subjects which contribute to performance measures.

9-1 Combined Science/Science Trilogy

This double award subject is eligible to count in up to 2 slots in the Progress 8 measure. A pupil with a grade 5-4 will receive 4.5 points per slot.  It is this score that is used to determine the grade count in Subject Summary and Subject Overview; 4.5 will register as a single grade 4 and show in the APS column as 4.5. This allows users to compare performance between subjects equally, using the same scale.

When working with residuals, this same pupil will be given 9 (2 x 4.5) points towards their APS. If they achieve a grade 6 in English Language, their APS will be (4.5 + 4.5 + 6) / 3 = 5. The residual for this pupil in Science is therefore 4.5 - 5 = -0.5

This is a fairer method than using (4.5 + 6)/2 because pupils that study the Core/Triple Sciences pathways will have all of their science grades used in the residual calculation.

Subject Residuals (Legacy)

This measure is the average of the distance between each student's grade and their overall average grade, but better than average is scored as positive and worse is negative.  Thus a subject residual of 0.00 tells you that on the whole pupils did just as well in this subject as they did in all their subjects.  A residual of +1.50 means that the pupils performed one and a half grades better than their overall performance.

Where available, a national residual is also shown for the subject. It is the difference between the school's residual and the national residual (i.e. the corrected figure) that appears at the top in the coloured strip. A school's subject residuals are ranked from the most positive to the most negative. The coloured strip represents subject ranks as quartiles (green, grey, yellow, red). The ranking takes account of entry size in each subject, so it is possible that a small entry subject with, say, +0.53 ranks below a core subject with +0.12.

The underlying currency for residuals is QCA points. We have calculated our own residual values for number grades which can be found in Appendix H, although we expect to change to the new 9-1 points later in 2017.

In cases where a pupil receives a U grade in the selected subject, we have to include a correction to the residual calculation to account for the points difference between grades G and U, worth 16 and 0 respectively. We either add or subtract 10 (depending on whether the difference in grades is negative or positive) from the residual calculation before dividing by 6 to get the grade equivalent.

Example 1:

A pupil is working at a grade G on average across all subjects. In Maths they are achieving a grade U. G is worth 16 QCA points. To calculate the residual for Maths for this pupil, take the difference (-16) and add 10 to get -6. Divide by 6 to see the difference in terms of the number of grades =  1. This means that this pupil's residual for Maths is -1.

Example 2:

A pupil is working at a grade U on average across all subjects. In History they are achieving a grade D. D is worth 34 points. Take the difference (+34 points) and subtract 10 to get 24. Divide by 6 to see the difference in terms of the number of grades = 4. This means that this pupil's residual for History is +4.

Residual Vs National (RESvNAT)

In  Subject Summary, the RESvNAT is the difference between the subjects internal residual and the national residual (i.e. the corrected figure). For example, if the internal residual for a subject is +0.50, and the National is +0.20, then the RESvNAT column will display +0.30, which means that pupils in this subject are working about a third of a grade above the national residual.

This figure can be used to put the effect of 'harder' and 'easier' subjects into perspective. Subjects that are deemed to be harder tend to have a lower residual internally when compared with easier subjects, and the same can be said at national level. An internal residual of +0.25 in a harder subject vs a national residual of +0.1 would give a RESvNAT value of +0.15. An internal subject residual in an easier subject of +1.00 vs a national residual of +0.95 would give a RESvNAT figure of +0.05. So, although the internal residual is lower for the harder subject, it is actually performing better than the easier subject when compared nationally.

Distribution

The measure is the average distance between each student's grade in this subject and their overall average grade. For example: if there are just 3 pupils, and one attains grade B but his average over all subjects is C; another attains D but has an overall average of C; and another is graded D with F; then the distances measured will be B to C (1.00), D to C (1.00) and D to F (2.00). The distance always shows as a positive number.  The average in this example is 1.33 grades. The higher this number, the more the results will appear spread-out. If the results in an individual subject correlate closely with the overall average, then the measure will be close to zero. In practice a value of around 0.5-0.6 grades will represent a norm value, i.e. an expected value of distribution to be found in ‘normal’ circumstances (‘Normal flutter’).  A value of 1.5 grades for a class with at least 10 pupils would be considered as unusually high, and worthy of explanation.

Average Points Score (APS)

This displays the Average Points Score achieved by pupils in this subject.

APS Vs National (APSvNAT)

In  Subject Summary, the APSvNAT is the difference between the subjects internal average points score and the subjects national average points score. Example: Geography has an internal points score of 40 (grade C). The APSvNAT figure is showing as +3.00. This means that the national APS for Geography is 37. Because we are working with QCA points, the points difference between grades in the standard A*-G scale 6. This means that the school is performing half a grade better than the national APS.

Entry

The number of entries is recorded.  Where possible, the national expectation of entries for a typical school with the same cohort size has also been calculated. This estimate depends on the published national entry rates by subject in maintained schools.  The total derives from separate estimates for boys and girls.  Although the figure relates to a notional 'typical' same-size school, it may nonetheless be of interest.

Gender

Average grades and subject residuals are calculated separately for boys and for girls. The differences in boys' and girls' performance are worked out for each indicator.  If the difference is, say, +1.00 then the boys as a whole will have done a grade better than the girls. (A negative difference, say, -0.5, says the girls did better.) To show where there is gender bias in performance, the gender differences in subject residuals have been ranked across all subjects. The pink or blue symbol indicates where girls did better than boys, or vice versa.

This analysis is similar to the analysis by gender; see above. Here the groups compared are not boys and girls, but pupils in the upper half of the school's academic range and those in the lower half. The ranking is done on each student's average grade across all subjects. Subject residuals are calculated separately for the upper range and the lower range, and the differences in high-flyers' and low-flyers' performance are worked out. A positive difference indicates that pupils in the upper range did better than those in the lower range. The indicator here is the subject residual, so it is quite possible for pupils in the lower range to do better than the high-flyers.  To show this clearly, the differences in subject residuals have been ranked across all subjects. The symbol shows where the upper range did better than the lower range or vice versa. Note that here again the ranking takes account of group size.

#### Article details

Article ID: 47

Category: Using 4Matrix