Mad, Bad and Dangerous Statistics

I don’t normally use this blog to bitch about other bloggers – the whole point about blogging is that it allows you to express your own opinions, however misguided they might be and it is up to your readers to decide if they are worth anything. So, this post is an exceptional one for me. Dr Eoin Clarke has a blog called The Green Benches where he produces a lot of apparently statistically based articles critiquing the government. It is pretty influential – the stats counter on the blog shows he’s had nearly a thousand times more views than me so lots of people must think he is saying worthwhile and interesting things. He doesn’t let my gentle and polite comments through his moderation process so here they are for you.

He has blogged today about how comprehensive schools have been proven to outperform independent schools by reference to data published by the Department for Education. In summary, this data (from page 21) shows that 80.7% of children at comprehensive schools get 5 GCSEs at A*-C compared with only 80.3% of children at independent schools. Dr Clarke uses this to support the following statement:

“Independent schools are free from state interference. They run their own show. These companies charge parents £1000s of pounds a year to enroll and educate little Timmy and Tabatha. But guess what? State Comprehensives under Local Education Authorities, burdened with all that state interference actually out-performed these independent schools in this year’s GCSEs. This is one of many reasons why the Free Schools agenda is mad, bad and dangerous.”
However, this is completely misleading. First, the data for independent schools, if you can be bothered to read the footnote to it states that it includes data for independent and non-maintained special schools. That is, schools for children whose special needs are such that they do not attend a mainstream school. Schools like the one that former Education Secretary Ruth Kelly sent her child to. The equivalent special schools in the maintained sector have their results separated out from those of comprehensives and achieved a pass rate of 1.3% getting the same measure of GCSEs trumpeted in the blog. Now, it may be that independent special schools select particularly intellectual special needs children, but somehow this seems unlikely and rather inconsistent with the concept of a special school. Even if the proportion of independently educated special needs children is low, at the very least this shows that Dr Clarke isn’t comparing like for like.
In any case, I hold no particular agenda for independent schools. Maybe some parents are wasting their money. Although they might also point to the figures for 5 GCSEs at A*-C including English and Maths which show a much higher proportion achieving this at independent schools to justify their decision.
The bigger issue is that the data does not really support the criticism of Free Schools. Free Schools are not allowed to be any more selective than comprehensives and are bound by the same admissions law(subject to the possibility of some exemptions, as yet unknown, as pointed out by Allan Beavis in his comment below), so their independence is different from that of the private schools. What private schools achieve is not particularly an indicator of what Free Schools might achieve. If, on the other hand, the hidden allegation is that Free Schools will inevitably become covertly selective, for example by being only located in areas where there are sufficient numbers of horrid pushy parents, another comparison that Dr Clarke might want to make is with the results for children in areas with selective schools.
That data shows that selective state grammar schools get 99.2% of their children achieving the measure Dr Clarke relies on. Big deal, they’re by nature selecting the very brightest children and dumping the rest into ghastly Secondary Moderns. Except, the data for Secondary Moderns shows that they get 77.8% achieving that same measure. Considering that they have by definition had the most academic children creamed off, and anecdotally at least, probably have a proportion of the ones from wealthier backgrounds who missed out on a grammar school place sent to independent schools instead, that looks like a tremendous result, a mere 3% behind their fortunate peers at comprehensives which include a wider selection of higher ability children. Overall, the percentage achieving Dr Clarke’s measure of success at GCSE seems to be nearer 90% in areas with selection rather than 81% in areas with comprehensives. If Free Schools are going to introduce selection into local areas, perhaps they are not nearly as mad, bad and dangerous as Dr Clarke believes.
I’d recommend anyone tempted by Dr Clarke’s blog to take all his statistical analyses with a very large pinch of salt.
(Updated to reflect the possibility of Free Schools’ Funding Agreements including exemptions from the admissions code)

6 thoughts on “Mad, Bad and Dangerous Statistics

  1. I have a bit of a problem with so called experts who use statistics to measure the development of children. The Free School environment is non coercive,so the definition of education is radically different than the antiquated notion of comparing test scores.

    Here in the States, we have similar if not more entrenched problems. Charter schools, which are corporately funded, use numbers to prove their kids are learning. I use my brain and my first hand experience in talking to children to see how they are doing. Statistics are something you should apply to appliances and automobiles. Children are future purveyors of our world and hopefully will be competent enough to solve the mountain of problems we have left for them. I see those who are currently in Free Schools as the leaders in humane action, because their learning environment supports their ability to problem solve, work in teams, display empathy and so on. What have statistics ever done?

    • I have some sympathy for the saying that you don’t fatten a pig by measuring it. I have, perhaps fallen into the same trap as Dr Clarke by attempting to draw broad conclusions from impersonal data – the point I was aiming at was to show that the data didn’t really support the argument he made and could more plausibly support completely inconsistent and opposite arguments. The success or failure of different models of schools won’t be determined by statistics alone, but things like exam scores are important in a system that requires certain qualifications to allow people to access the more rewarding and interesting jobs and careers.

      My post and the one it was criticising were written from the perspective of Free Schools as envisaged in the UK. These have some relation to the US model but are likely to be judged at least to some extent by how they compare with the existing publicly funded schools.

  2. It is not true that Free Schools are bound by the same Admissions Code. They are exemptions that can be written under the Funding Agreement. To date, we know nothing about what is in the Funding Agreements so it would be interesting to see what Annex B of these Agreements might include. I suggest you keep abreast of the new Education Bill and read this

    and this

    • Thanks. Not sure that it makes a lot of difference to the point I was making that (at least on the basis of the stats about getting five GCSEs) the areas which were wholly and openly selective seemed to do better than the entirely non-selective areas, but useful to have links to the state of the art on admissions policies.

  3. Pingback: Review of 2011 – 2. Something so objectionable they pay you for it | botzarelli

  4. Pingback: Any Colour You Like As Long As It’s Black | botzarelli

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s