Using Training Evaluation Systems to Evaluate Language Education

Starting in 1959, Donald Kirkpatrick developed a four level systematic approach to training evaluations.

(1)Reaction: the feelings the students have about the program,
(2)Learning: the degree to which they learned the required material,
(3)Application: the ability to transfer training to the work place,
(4)Results: the impact of training on the organization's bottom line.

Over 75% of business HR departments surveyed use some form of the Kirkpatrick system to evaluate business training and development programs.

One way to keep language education programs current and viable is to use program evaluations and make necessary corrections to materials and teaching methods.

Kirkpatrick's four-level evaluation system has become a standard for business and
industry because it provides comprehensive data to support training programs.

If adapted for use in Language Education programs, this system will provide data to ensure that students:
(1) like the language education program,
(2) are learning the language,
(3) are able to apply the language in everyday, school and work settings, and
(4) have the correct competencies to use the language for school, work or social situations.

In short, implementing the Kirkpatrick four-level evaluation system could go a
long way toward ensuring the success and reputation of language education programs.

Kirkpatrick's Levels of Evaluation Overview

Level 1 of Kirkpatrick's four-level model measures the reaction of students to the training program. The purpose of measuring reaction is to ensure that students are motivated and interested in learning.

Implementation guidelines:
Determine what you want to find out.
Design a form that will quantify reactions.
Encourage written comments and suggestions.
Seek honest reactions.
Develop acceptable language standards.
Measure reactions against the language standards.

Level 2 of Kirkpatrick's four-level model measures the knowledge acquired, skills improved, or attitudes changed as a result of the language training.

Implementation guidelines:
Use a control group, if feasible. (two classes at the same level)
Evaluate knowledge, skills, or attitudes before and after training.
Use a paper and pencil test to measure knowledge and skills.
Use a performance test to measure attitudes.
Use the results of the evaluation to design program improvements action.

Level 3 of Kirkpatrick's four-level model measures the transfer of training. Are students applying the new language, skills, or attitudes with their language abilities.

Implementation guidelines:
Use a control group, if feasible.
Allow enough time for a change in behavior to take place.
Survey or interview students and peers, who observe students behavior.
Choose 10 students or an appropriate sampling.
Repeat the evaluation at appropriate times.

Level 4 of Kirkpatrick's four-level model measures the result of training as it relates to factors such as language test results and language performance.

Implementation guidelines:
Use a control group, if feasible.
Allow enough time for results to be achieved.
Measure both before and after training, if feasible.
Repeat the measurement at appropriate times.

As a summary

Do the best job of summarizing the data that your interviews and test results allow as in education or training contexts paper tests can lack reliability and interviews can be manipulated.

The new design changes to the language programs should be tested before wholesale changes.

Experts advise against using just one evaluation strategy and advise collecting
data from multiple sources using multiple evaluation methods.

***********

ESL in Canada Blog URL
http://eslincanada.blogspot.com/

Canada Summer Day School Survey

We had had many requests for a Toronto Day school during August 2007

Would you be interested in any of the following programs and dates?

Day school only 9 to 4 PM

Beginner English ages 8 to 15 --
Intermediate English ages 8 to 15 --

Beginner French ages 8 to 15 --
Intermediate French ages 8 to 15 --

Beginner Mandarin ages 8 to 15 --
Intermediate Mandarin 8 to 15 --

History for students ages 8 to 15 --
Geography for students ages 8 to 15 --
Social Studies for students ages 8 to 15 --
Mathematics for students ages 8 to 15 --
Science for students ages 8 to 15 --

Dates July 30 to August 24 (4 weeks)

Please write in dates if you want 2 or 3 week programs -----------

Weekend and Day trips on Saturday or Sunday:
Toronto zoo __
Niagara Falls __
Algonquin Park __
Alora Gorge __
1000 Islands __
Toronto Center Island __
Wonderland __
Royal Ontario Museum __
Ontario Science Center __
Ontario Place __
CNE __

Please use registration form and email eslincanada@gmail.com

Have a good summer
ESL in Canada

***********

ESL in Canada Blog URL
http://eslincanada.blogspot.com/

B.C. cracks down on private schools

The Toronto Star, Globe, Canada Press Headline reads:
B.C. Cracks down on Private Schools

The story continues with various comments:

BC Advanced Education Minister Murray Coell said Thursday the new rules will take effect this September.

New Democrat Rob Fleming criticized the regulations as "too little too late" and "half-measures" that will do nothing to help swindled students.

Confederation of University Faculty Associations of B.C. president Chris Petter stated that only a handful of private schools have been involved in improprieties.

The scandals prompted complaints from Indian and Chinese consuls, and China warned students to avoid Canadian private post-secondary schools because of shady operators.

The headlines should read:
Foreign organized criminals use Canada to defraud Chinese and Indian nationals.

The story should read:

Small groups of Chinese and Indian criminals posing as education consultants, agents and schools lure unsuspecting victims into buying an education from un-accredited universities, colleges and ESL schools. The foreign criminals set up the local consulting firms in India and China and the schools in Canada.

The story should read that the money paid to these criminals never leaves China or India. The money is divided up amongst the foreign criminals. The small office posing as a school in Canada is the scape goat. It is set up as a bad school deliberatly so the foreign criminals have a foreign organization to blame. The foreign criminals all act innocent stating that it is the schools fault, it is Canada's fault. This is all an elaborate scam for foreign criminals to escape detection and prosecution.

The students who arrive in Canada are victims of Chinese and Indian criminals who want them dumped into Canada where their only option is to chase after a school that has been set up as a front.

Canada is a double victim in this process. Canada is blamed for bad schools and bad regulation and then has to deal with the angry and swindled students. The foreign criminals have all escaped and live in luxury outside of Canada.

This problem can be solved very easy using lessons learned from the many swindles that took place in the investment, fitness and travel industries.

1. require licenses to set up any type of school
2. require an insurance bond equal to the outstanding tuitions
3. require full tuition deposits go to the school
4. require all school agents be registered, licensed and bonded
5. Canadian school staff to include a Canadian certified principal
6. Canadian school staff to include a head teacher with a faculty of education degree

The small license fees would pay for the extra provincial administration and if done properly would probably triple the number of foreign students coming to Canada to study.

And yes I am more than happy to help set up this program.

***********

ESL in Canada Blog URL
http://eslincanada.blogspot.com/