'No evidence' exam targets work

BBC NEWS
'No evidence' exam targets work

There is "no quantified evidence" about whether threatening to close schools affects their pupils' performance, the National Audit Office says.

Its report on the use of rewards and sanctions in the public sector said people believed there was an effect but those judgements were subjective.

It said value added measures of school performance were good practice.

But "threshold" measures, such as the proportion getting good GCSEs, could distort schools' behaviour.

Motivation

The audit office (NAO) report concludes: "We found no quantified evidence of the effect of sanctions and rewards on levels of performance for the programmes in the survey."

It noted that people needed to be able to affect the outcome if they were to be motivated by some sanction or reward.

"Unless the agent can expect their behaviour to affect the outcome, the mechanism provides little motivation to strive for the desired levels of performance."

It said an example of a good measure was the "contextualised value added" assessment of how much progress pupils have made in schools.

This takes into account factors outside schools' control but which are known to affect how well children do, such as poverty and ethnicity.

This "is generally seen as a better performance measure than the previous 'gross output' measure which did not make any kind of adjustment", said the report.

Distortion

However, Prime Minister Gordon Brown's determination to drive out failure - as he put it - in England's schools uses a "floor target" for GCSE performance that does not work in this way.

The target is 30% of pupils getting five good GCSEs including English and maths.

The NAO report says that such "threshold" schemes may not reward those who improve the most as result of starting from a lower base.

Some of the schools identified by the Department for Children, Schools and Families as falling below the floor target were regarded by Ofsted inspectors as "outstanding".

The audit office also noted that simplistic targets could distort what went on in the education system.

"The literature also contains examples of cases where the choice of outcome measure has led to 'gaming' or strategic behaviour by agents, such as anecdotal evidence of 'cream-skimming' by schools to select the best students."

It added: "Threshold measures - measures concerned with achieving an absolute level of performance, such as the number of pupils achieving GCSE grade C or higher - are seen as particularly vulnerable to gaming."

Social segregation

The government had recognised this risk and introduced additional targets about the outcomes to be achieved for less well-performing groups, the NAO said.

The report also considered the impact of competition - and said children from poorer homes could lose out.

"In the case of schools, the costs to the consumer of switching are significantly higher than in the case of, for example, utility providers.

"Children from families that are less well-informed, or less mobile, are at risk of becoming 'stuck' in less well-performing schools, furthering the inequity.

"It also gives schools the incentive to pick the best pupils."

So secondary schools had become more efficient - in terms of better results - but the trade-off had been greater social segregation, it said.

Story from BBC NEWS:
http://news.bbc.co.uk/go/pr/fr/-/2/hi/uk_news/education/7638091.stm

Published: 2008/09/26 13:48:10 GMT

© BBC MMVIII