Hargreaves en verantwoordelijk gebruik van data/cijfers: het wonder van Oakland, onderwijs en zes principes

30 oktober 2013

 Andy Hargreaves haalt de film Moneyball aan om te laten zien dat data/cijfers op zichzelf niet goed of fout zijn, maar dat in het gebruik ervan veel mis gaat. In het bijzonder in het Amerikaanse onderwijs, zoals hij in een open brief in de Washington Post schrijft. Hoe kun je metingen verrichten op prestaties van mensen en tegelijkertijd dezelfde mensen verantwoordelijk laten zijn voor hun eigen prestaties? Hargreaves, die onlangs op de Conferentie Duurzaam Leren in Utrecht over hetzelfde onderwerp uitweidde,  kijkt met zorgen naar de ontwikkelingen in het Amerikaanse onderwijs. Vervolgens beschrijft hij zes principes voor 'using data' , met het honkbalteam van de Oakland Athletics - een sport waarin het ogenschijnlijk draait om statistieken -  als het professioneel voorbeeld. Lees en kijk mee.

One of the most unlikely bestselling books and blockbuster movies of the past decade is a story about how baseball statistics turned from being a nerdy preoccupation of obsessive fans into a powerful tool for dramatically improving team performance. Michael Lewis’s Moneyball showed how insightful use of a range of performance statistics to select and deploy players boosted the underfunded Oakland Athletics to World Series standard, where they faced and often defeated teams with triple their payroll.  For the Oakland A’s, statistics often trumped coaches’ intuitions. The use of performance stats that Oakland pioneered in the United States is now commonplace throughout professional sports.

Performance metrics are also ubiquitous in business. Most companies now monitor a plethora of indicators — from product defect ratios to speed of output, from customer satisfaction to internet “stickiness” — to pinpoint performance problems and prompt real-time interventions. The brutal metrics of victories and defeats  and of profits and losses,  hold sports teams and companies accountable to fans and shareholders alike. In business and sports, we can get data-driven improvement and accountability (or DDIA) at the same time. Why should public education be any different?

One of the biggest buzz-words in education today is accountability. Accountability is seen as a strategy for improvement, by rewarding the successful and eliminating or intervening forcefully with those who are not. And it is mainly enforced through indicators of student achievement derived from standardized test scores. Test scores purport to reveal the successes or failures of students, teachers, schools and even entire educational systems.

As in business and sports, student test score data also have a second purpose – to focus everyone’s efforts on just-in-time improvement by providing ongoing information through many tests about students’ progress. This enables teachers to monitor individual students and to intervene immediately if they start to fall behind. It provides principals with data that indicate how their teachers are performing, and to take appropriate action. And it enables districts and state departments to know what is happening in every school, so that corrective action can be taken before it is too late.

So are the opponents of student testing just unrealistic romantics who are out of date and out of touch? Perhaps they are not just resisting the rightful insistence that they should be held accountable to the public. They may also be rejecting the data tools that would enable them to improve.

In our policy brief Data-driven Improvement and Accountability (DDIA), published today by the National Education Policy Center, we examine the linkage between improvement and accountability in education, especially in relation to their use of data. Our paper is based on the research we have done on the use of data in high performing countries, in business and sports, as well as the advice we are now providing to state and provincial departments of education that are revamping their testing instruments and accountability policies.

What have we found? DDIA is, in itself, neither good nor bad. It all depends on how the data are defined and used. When DDIA is done thoughtfully, with due respect to the strengths and limitations of the data, it provides educators with valuable feedback on their students’ progress and difficulties that can inform  decision-making and even lead to changes in  practice. It can also give parents and the public meaningful information about student learning and the quality of the education that students are receiving.

In high-performing educational systems, businesses and sports teams, DDIA systems are based on data that are valid, balanced, usable, stable and shared. But in the United States, up until now, these are not the typical characteristics of DDIA, with the result that DDIA has generally impeded improvement and undermined accountability.

The answer is not to avoid data or abandon all testing but, rather, to learn from high performing systems and organizations elsewhere. Here are some of the key principles and practices identified in our report.

1. Validity: Measure what is valued instead of valuing only what can easily be measured, so that the purpose of schooling is not distorted.
2. Balance: Create a balanced scorecard of metrics and indicators that captures the full range of what the school system values.
3. Insist on credible, high quality data that are stable and accurate.
4. Design and select data that are usable in real time.
5. Develop shared decision-making and responsibility for data analysis and improvement.
6. Be the drivers, not the driven, so that statistical and other kinds of formal evidence complement and inform educators’ knowledge and wisdom concerning their students and their own professional practice, rather than undermining or replacing them.

Lees verder

Professor Andy Hargreaves is hoogleraar en een internationaal erkende autoriteit op het gebeid van onderwijsonderzoek en heeft diverse bekroonde boeken op zijn naam staan. Hij is geïnteresseerd in de relatie tussen onderwijs, leiderschap en duurzaamheid. Hij was vorige maand een van de inleiders op het tweedaagse congres van Duurzaam Leren: 'Waar sta je voor, als je voor de klas staat?' Dit is de link naar zijn presentatie.

Meer over de film Moneyball

Billy Beane (Brad Pitt) is de algemeen directeur van het weinig succesvolle honkbalteam Oakland Athletics ("A's"). Hij is teleurgesteld vanwege de nederlaag tegen de New York Yankees aan het einde van het seizoen 2001 en dreigt drie belangrijke spelers te verliezen. Tijdens een bezoek aan de Cleveland Indians ontmoet hij toevallig Peter Brand (Jonah Hill), een jonge econoom die radicale, vooral op statistiek gebaseerde ideeën heeft over hoe spelers te evalueren. Beane neemt Brand aan en ze gaan, de weerstand van coach Art Howe (Philip Seymour Hoffman) en de scouts negerend, op zoek naar ondergewaardeerde spelers. Uiteindelijk weten de A's twintig wedstrijden op rij te winnen.