I remember as a child going to the Christmas pantomime (Peter Pan)  at the King’s Theatre in Glasgow. In particular, I recall my frustration when the crocodile appeared and Captain Hook couldn’t see it. I kept on shouting “It’s behind you!!!” but he would never look in the right place. I thought he was really stupid- you can never find what you are looking for if you look in the wrong places.

In the past week or so, many Australian lists have been discussing the latest MCEETYA report.
National Assessment Program - ICT Literacy Years 6 and 10 Report.

MCEETYA is the Ministerial Council on Education, Employment, Training and Youth Affairs – big league stuff!!!.
From the Executive Summary of the report –

“Australia’s national goals for schooling assert that when students leave school they should be: confident, creative and productive users of new technologies, particularly information and communication technologies, and understand the impact of those technologies on society (MCEETYA, 1999: Goal 1.6). The Australian National Assessment Program includes the systematic assessment of the extent to which this goal is being achieved through triennial sample surveys of students in Years 6 and 10.”

From what I can see, the report documents the assessment of tasks students in years 6 and 10 were asked to complete that measured how good they were at
a) Using graphics software to make a flag and a photo album
b) Finding information from a closed web environment
c) Using Microsoft office to make documents and powerpoints

The results showed that nationally, 49% of year 6 students and 61% of year 10 students met some arbitrary proficiency standard. None of the tasks had any real communications focus (remember the C in ICT). There was no mention of any Web 2.0 technologies. It looks like a computer applications competency test.
I wonder how many of the year 10 students who couldn’t make a photo album using the software provided have albums in Flickr, MySpace or Facebook. Of those, I wonder how many learnt how to do it without any adult intervention.

But that’s Australia .... I went looking to see what the US was doing. Admittedly, I didn’t look very hard and what I found may be unrepresentative. Someone’s blog mentioned the State Educational Technology Directors Association.  SETDA is the principal association representing the state directors for educational technology. They have a suite of tools for determining educational technology effectiveness called Profiling Educational Technology Integration (PETI): Resources for Assessing Readiness & Use. Guess what the suite of tools is – a bunch of surveys that are Microsoft Word documents. These ask questions about how students find information and use PowerPoint, etc. Not a single mention of any Web 2.0 technology.



Part of the problem of course is that technologies change rapidly and bureaucracies lumber along, getting further and further behind. That then begs the question – are their findings relevant or useful? Are they looking in the right places? How will we ever be able to judge the effectiveness of current educational technology practice if we are always looking at what we were doing years ago? Is it even worth the  effort?