Assessment is a really crucial part of learning and should be as performance-based as possible. That was number 9 or 12 truisms I cited as having influenced how I have been doing my work as a learning and performance consultant over a number of years. I invited you to support them with a book reference, a personal experience or a case study, or knock them down using similar sources of evidence. Last time I began to express some beliefs I hold about assessment. To complete the picture I want to say that I believe the two criteria of “validity” and “reliability” are guiding principles; they are much more than academic labels and are the best measure of the effectiveness of assessment.
The concept of validity is probed by a simple question, “If you say ‘watch me’ will the observer see you doing precisely what you’re required to do in the real world?” Watch me fill out this form while questioning a customer” is not quite the same as “Watch me fill out an image of the form on screen while reading a scenario and looking at a photograph of a customer.” At some point it is necessary to decide how far you can compromise where assessment is concerned. Is “Watch me land this simulation of a plane” quite as valid as “Watch me land this plane?”
“Reliability” is more to do with the structure and composition of the test and the conditions under which it is administered. For example if you test someone at the start of the day when they have no other commitments on their diary and have been given time to prepare, and then you use the same test on someone you’ve taken by surprise ten minutes after their working shift has ended and they are anxious about missing their bus, should you expect the same results?
So taking part one of this posting on assessment alongside this second part on validity and reliability, here are some statements that may be true and may be false assumptions, and I’d welcome your views and experience.
- Well designed, performance-oriented tests inform learners about job requirements and guide their learning.
- Tests must be both valid and reliable.
- Learners who are frequently tested do better than those who are tested less often.
- Learners generally take two kinds of tests: knowledge tests and performance tests.
- Knowledge tests tell you whether people have learned information important for safety, and acquired knowledge that regulates their performance.
- Skill checks and performance assessments measure the competence of learners and reveal gaps and weaknesses in the method, media and content of instruction as well as gaps in the learner’s understanding and skills.
- Errors can be used not only to identify learning gaps to close, but also to motivate learners to deepen their enquiries and seek information that they appear to lack. They can also be used heuristically to give the learner, in a safe no-penalty environment, a view of what happens when you do things wrong.
Please let us all know what you think.
PG's Tips 