Using Training Evaluation Systems to Evaluate Language Education

Starting in 1959, Donald Kirkpatrick developed a four level systematic approach to training evaluations.

(1)Reaction: the feelings the students have about the program,
(2)Learning: the degree to which they learned the required material,
(3)Application: the ability to transfer training to the work place,
(4)Results: the impact of training on the organization's bottom line.

Over 75% of business HR departments surveyed use some form of the Kirkpatrick system to evaluate business training and development programs.

One way to keep language education programs current and viable is to use program evaluations and make necessary corrections to materials and teaching methods.

Kirkpatrick's four-level evaluation system has become a standard for business and
industry because it provides comprehensive data to support training programs.

If adapted for use in Language Education programs, this system will provide data to ensure that students:
(1) like the language education program,
(2) are learning the language,
(3) are able to apply the language in everyday, school and work settings, and
(4) have the correct competencies to use the language for school, work or social situations.

In short, implementing the Kirkpatrick four-level evaluation system could go a
long way toward ensuring the success and reputation of language education programs.

Kirkpatrick's Levels of Evaluation Overview

Level 1 of Kirkpatrick's four-level model measures the reaction of students to the training program. The purpose of measuring reaction is to ensure that students are motivated and interested in learning.

Implementation guidelines:
Determine what you want to find out.
Design a form that will quantify reactions.
Encourage written comments and suggestions.
Seek honest reactions.
Develop acceptable language standards.
Measure reactions against the language standards.

Level 2 of Kirkpatrick's four-level model measures the knowledge acquired, skills improved, or attitudes changed as a result of the language training.

Implementation guidelines:
Use a control group, if feasible. (two classes at the same level)
Evaluate knowledge, skills, or attitudes before and after training.
Use a paper and pencil test to measure knowledge and skills.
Use a performance test to measure attitudes.
Use the results of the evaluation to design program improvements action.

Level 3 of Kirkpatrick's four-level model measures the transfer of training. Are students applying the new language, skills, or attitudes with their language abilities.

Implementation guidelines:
Use a control group, if feasible.
Allow enough time for a change in behavior to take place.
Survey or interview students and peers, who observe students behavior.
Choose 10 students or an appropriate sampling.
Repeat the evaluation at appropriate times.

Level 4 of Kirkpatrick's four-level model measures the result of training as it relates to factors such as language test results and language performance.

Implementation guidelines:
Use a control group, if feasible.
Allow enough time for results to be achieved.
Measure both before and after training, if feasible.
Repeat the measurement at appropriate times.

As a summary

Do the best job of summarizing the data that your interviews and test results allow as in education or training contexts paper tests can lack reliability and interviews can be manipulated.

The new design changes to the language programs should be tested before wholesale changes.

Experts advise against using just one evaluation strategy and advise collecting
data from multiple sources using multiple evaluation methods.

***********

ESL in Canada Blog URL
http://eslincanada.blogspot.com/