CAGs, TAGs and mutant algorithms

In 2020 and 2021, formal 'sit-down' school exams in England were cancelled because of the Covid-19 pandemic. Students, though, still needed to be recognised for their learning over many years of schooling, and to be able to progress to their next stage of education, or employment.

How could students be awarded fair grades?

That's a question that, with hindsight, the government botched.

Many of the problems, though, were evident at the time, and far better answers were quite possible. From March 2020 until August 2021, I was a close observer of these events as they happened, and I published a series of blogs commenting 'in real time', as shown here. And afterwards, I wrote a book - Missing the Mark: Why so many school exam grades are wrong, and how to get results we can trust - published by Canbury Press in August 2022, Chapters 10, 11, 12 and 13 of which tell the full, sorry, story.

Let me note that a key feature of many of these blogs is not the blog itself, but the richness of the comments - my thanks to all those who made such important, and truly excellent, contributions.

Trusting teachers is the best way to deliver this year’s exam results – and those in future years? (HEPI, 21 March 2020)

A*++ for Ofqual and the SQA: this year’s school exam grades could well be the fairest ever (HEPI, 4 April 2020)

Written evidence to the House of Commons Education Committee (submitted 4 April 2020, made public 4 May 2020)

Two and a half cheers for Ofqual’s ‘standardisation model’ for GCSE, AS and A level grades – so long as schools comply (HEPI, 18 May 2020)

No test is better than a bad test (HEPI, 1 June 2020)

Have teachers been set up to fail? (HEPI, 18 June 2020)

Written evidence to the House of Commons Education Committee (submitted 18 June 2020, made public 1 July 2020)

Written evidence to the House of Commons Education Committee (submitted 26 June 2020, made public 7 July 2020)

Grade inflation tragedy if teacher judgement undermined (TES, 30 June 2020)

Written evidence to the House of Commons Education Committee (submitted 3 July 2020, made public 15 July 2020)

We need to act now to make sure GCSE grades are fair  (TES, 8 July 2020)

Halfon is right: Ofqual has more to do (co-authored with Rob Cuthbert, HEPI, 14 July 2020)

Hindsight is a wonderful thing: Ofqual, gradings and appeals (HEPI, 23 July 2020)

Ofqual’s Summer Symposium, Slide 12: Examiner’s Report (HEPI, 26 July 2020)

Written evidence to the House of Commons Education Committee (submitted 28 July 2020, made public 8 September 2020)

The Great CAG Car Crash – What Went Wrong? (HEPI, 12 August 2020)

Written evidence to the House of Commons Education Committee (submitted 12 August 2020, made public 8 September 2020)

CAGs rule OK (HEPI, 18 August 2020)

The grading fiasco has not finished unravelling (Schools Week, 25 August 2020)

The exams catastrophe: 16 questions that must still be answered (HEPI, 2 September 2020)

Written evidence to the House of Commons Education Committee (submitted 25 September 2020, made public 6 October 2020)

Why the rank order contingency plan is flawed (Schools Week, 4 November 2020)

Can we trust this year’s exam results? (HEPI, 6 March 2021)

Why GCSEs 2021 might actually be good for teachers (TES, 22 March 2021)

The school grading drama unfolds in five acts (HEPI, 8 June 2021)

An A* in Reputation Management? Looking back at last summer’s results row – and ahead to this summer’s coming one (HEPI, 22 June 2021)

TAG appeals are Ofqual’s mess. Let them own it. (Schools Week, 27 June 2021)

An idea for managing this year’s unreasonable appeals process  (Sixth Form Colleges Association, 28 June 2021)

Higher education to the rescue: only universities can put sufficient pressure on Ofqual to deliver reliable grades (HEPI, 7 August 2021)

There were, of course, any number of blogs and journal articles authored by others, notably Rob Cuthbert's HEPI blog of 10 August 2020. Media coverage too was extensive; let me mention here just two particularly influential newspaper stories.

This piece by Liz Lightfoot, published by the Guardian on 20 June 2020, was one of the earliest reports of a particularly vicious injustice of the algorithm, the 'history trap', whereby a talented student would be denied a high grade simply because no student at that school had been awarded a high grade over the previous three years.

And on 8 August 2020 - that's the Saturday prior to the announcement of the A level grades on the following Thursday - the Guardian's front page main headline was "Nearly 40% of A level predictions to be downgraded in England". The accompanying piece told of Huy Duong, whose son was awaiting his A level results. Combining the skills of a mathematician and a masterful detective, Huy had put together some fragments of information provided by the exam regulator, Ofqual. As a result, he made an astonishing forecast: that the next Thursday's results would show that 39% of the grades that had been suggested by teachers (known as 'CAGs') would be over-ruled, and downgraded, by the algorithm.

This article created uproar across the country, and over the next few days, triggered a series of events, including some last minute changes to the appeals process.

On results day, Thursday 13 August, Huy's forecast was confirmed as being absolutely right.

The national uproar then became even louder, and a few days later, on Monday 17 August, the results of the algorithm were discarded, and the CAGs reinstated.