How can you tell if a student really understands what they are doing?

The Trouble with Assessment

There is a great emphasis these days on using educational technology. But how can you tell if a student really understands what they are doing? Assessing students’ “digital literacy” is often not done at all, and, when it is undertaken, it is not often done well.The reason is, I think, the same in each case: On the whole, teachers do not understand enough about it themselves to be able to evaluate what someone else is doing.For example, I have been in meetings where school inspectors, who are trained to be extremely critical (in an academic sense, rather than a negative one) have looked at a piece of desktop-published work and concluded that the student’s ed tech skills must be quite good. Clearly, such people have never heard of wizards or templates!Sometimes it comes down to a lack of skill in assessment itself. In one lesson I observed, the teacher said to the class, “Do you now all understand what modelling is?” One boy said “Yes”, to which the teacher responded, “Excellent! So you all understand what modelling is.”

Never mind, you may say. I will just make sure all my staff have a copy of the NETS standards. Unfortunately, that won’t help much, or at least not on its own. Why not? Because how will the teachers recognise when a skill is being demonstrated, or a particular standard achieved?

Take this one, for example:

“Communicate information and ideas effectively to multiple audiences using a variety of media and formats.”

What does that word “effectively” mean in behavioral terms, i.e., what would someone have to do in order to demonstrate that?

Or take this one:

“Exhibit leadership for digital citizenship.”

In order to assess that, you would have to know what is meant by “digital citizenship”, what is meant by leadership in that context, and what behavior would be exhibited by someone who had mastered that skill.Now, you may think I’m nit-picking because, in a sense, we all probably have a good idea what is meant by “digital citizenship”. But the point is that we may all have a different idea, and we may all have different views about how to spot it. So, should we throw up our hands in despair and walk away from the problem? No, of course not. What we need to do is to recognize it as an issue, and address it methodically. Let’s start with a couple of principles:

Assessment Principles

Assessment Principle 1
Assessing educational technology understanding always entails assessing higher-order skills like the ability to evaluate. That is true even in a “purely” skills context. For example, when using a word processor, you have to make judgments about paragraph spacing, bullets vs. numbers, and so on, in the light of what is best for the target audience.

Assessment Principle 2
You can never judge by appearances alone. I’ve already mentioned wizards and templates, but spreadsheet work is another minefield. Without asking a student to show the formulas she’s used, you can’t possibly know whether she has “got it”. Now let’s consider some approaches to assessment.

Assessment Methods

Talk to your co-workers
You need to come to a shared understanding of what the standards mean, and how you will recognize when they have been reached. One way of doing so is to assess a sample of each other’s students’ work, and discuss the outcome until you have all agreed on the grades.

Talk to your students
A bit radical this, in our current era of testing everything that moves. But you can ask students why they did X, and when would they use Y, to help you get inside their thinking processes.

Set open-ended tasks as far as possible
Here’s the deal: If you ask me to run a 100 yards, then I will run a hundred yards. But you are left with no idea about whether or not I could run two hundred yards. (I couldn’t, trust me, but that’s not the point.) That’s why project-based learning can be so effective when it comes to assessing educational technology understanding: The open-endedness of the approach gives students plenty of scope to show what they know, understand and can do.

Ask rich questions…
A question like “Do you understand what modeling is?” is ridiculous, because the answer will give you no information whatsoever. You need to ask questions that start with “How would you…” and “Please explain how…”.

… and wait for an answer
Most teachers suffer from two unfortunate conditions: they hate silence, and they love the sound of their own voice. In combination they lead to the commonly-observed scenario in which the teacher asks the class a question, and then answers it himself after (according to research done in the UK) two or three seconds.

Conclusion

As you can see, the “trick” in assessing ed tech understanding is to do a lot of talking and listening: With both co-workers and students. Just plonking them down in front of a computer for an hour and then judging the quality of the print-out really is not good enough! 

Terry Freedman is a U.K.-based education technology consultant and publishes the ICT in Education website at www.ictineducation.org, and the electronic newsletter “Computers in Classrooms

POSTED ON HOTCHALK.COM

Comments (4)

Subscribe to this comment's feed

Name

Email

Website

Title

Comment

smaller | bigger

Subscribe via email (Registered users only)

I have read and agree to the Terms of Usage.

Add Comment