Since 1998, Ontario colleges have been mandated by the provincial government to collect and report performance data to measure how well they are meeting the needs of students, graduates and employers. The KPI initiative is intended to ensure colleges remain accountable, responsive and effective in meeting the needs of these three key stakeholders.
The following are a few of the Durham College highlights from the provincial government's latest KPI survey of Ontario colleges, released April 9, 2008.
Durham College:
Durham College is committed to providing our students with quality learning experiences and support in finding pathways to jobs and productive citizenship.
| Graduate Employment | Employer Satisfaction | Graduate Satisfaction | Student Satisfaction | Graduation Rate |
| 90.7 per cent | 95.6 per cent | 81.5 per cent | 77.4 per cent | 67.4 per cent |
Full results for all 24 colleges are available on the website www.collegesontario.ca [This link will open in a new window.]
Q.
How are KPI results calculated?
A
.
There are five KPIs used to measure performance:
Data for Graduate Satisfaction, Employer Satisfaction, and Graduate Employment are collected by an external survey consultant through telephone surveys.
Student Satisfaction surveys are completed in-class by colleges across the province. The survey results are then compiled by an external survey consultant.
Graduation Rate is calculated annually by each college.
Q.
Which graduates are contacted for the Graduate Satisfaction survey? When and how?
A.
Per an agreement with the Ministry of Training, Colleges and Universities, all colleges provide the consulting firm, Forum Research, with a list of the official graduates for each graduating term. For the 2008 reporting period, the graduating terms were Summer 2006, Fall 2006 and
Winter 2007. Graduates are surveyed by phone six months after graduation. These parameters were developed by the provincewide KPI Steering Committee (made up of college representatives and ministry personnel) and are consistent across all colleges.
In addition to being asked about their satisfaction with the programs and services they experienced at the college, graduates are asked about their employment status (which provides information for the Graduate Employment KPI) and whether they have gone onto further education.
Q.
Which employers are contacted for the Employer Satisfaction survey?
A.
As a part of the Graduate Satisfaction survey, each employed graduate is asked whether or not his or her employer (direct supervisor) could be contacted. If permission is granted, every effort is made by the consulting firm to contact the employer within a five-week period of having spoken with the graduate. If permission is not received, the employer is not contacted.
The purpose of the Employer Satisfaction survey is to assess the performance of the college, not the graduate. However, some graduates may not grant permission, feeling some trepidation with asking a new employer to take part in such a survey.
Q.
How is the Graduation Rate calculated?
A.
As defined by the Ministry of Training, Colleges and Universities, the graduation rate is based on tracking individual students who entered a program of instruction in a particular enrolment reporting period and assessing how many of those individuals completed the program a number of years later.
For each program of instruction, students who transfer out are removed from the group that is being tracked, and those who transfer in are added.
The normal program duration used to establish a student's program completion time frame equals approximately 200 per cent of the normal program duration.
For example, the 2007-2008 Graduation Rate is based on students who started one-year programs in 2005-2006, two-year programs in 2003-2004, and three-year programs in 2001-2002, and who had graduated by the 2006-2007 academic year.
Q.
How valid/reliable are the surveys and the results?
A.
The surveys used in the KPI data collection process were developed from surveys that had been previously used within the college system. Minor adjustments were made to select surveys by the provincewide KPI Steering Committee to meet the KPI objectives.
The surveys meet conventional standards with respect to 'satisfaction surveying' or 'opinion polling'.
The third-party consultants who were contracted to gather the data brought both experience within the college system and expertise in survey design to the planning tables.
Sample size plays a significant role in the reliability and validity of data. Small sample sizes can sometimes result in chance findings that are not accurate. Sample size for program level, by definition, is always smaller than the overall college level; thus, caution is urged in interpretation of data at the program level.
Q.
What is the margin of error for these surveys?
A.
The graduate sample has a margin of +/- 0.3 per cent, 19 times out of 20, at the system-level.
The employer sample has a margin of error of +/- 0.8 per cent, 19 times out of 20, at the system-level.
The student sample has a margin of error of +/- 0.2 per cent, 19 times out of 20, at the system-level.
Across the three completed graduate survey periods, more than 41,500 interviews were completed out of a total graduate population of approximately 60,700 Ontario college graduates. For Durham College, almost 1,500 graduates were surveyed out of a total graduate population of approximately 2,000.
More than 9,000 employers completed the survey across the province. Of these, 300 had employed Durham College graduates.
Nearly 99,000 Ontario college students in their second semester or later completed the survey. Of these, almost 4,200 were from Durham College. Typically, national public opinion polls in Canada contact 1,000 adults out of an adult population of about 20 million and have larger margins of error.
Q.
What scale was used and why?
A.
All decisions regarding survey design, instrumentation, and so on, were made by the provincewide KPI Steering Committee with input from the contracted research firms. It was decided to use a five-point scale, where:
Such five-point scales are often used in instances where satisfaction is being measured.
Only results registered as very satisfied and satisfied are considered and/or reported as 'positive results'.
Q.
How were the KPI figures reached?
A.
Each survey is comprised of many questions to provide valuable feedback to the colleges on how they might improve programs and services.
Embedded in each survey, however, are questions that are referred to as capstone questions - key questions that ask the respondent to give his or her overall opinion about the central issue. It is these capstone questions that are used to calculate the KPI figures.
In selecting/designing the surveys, the general goal was to gain a comprehensive picture of user satisfaction. Each question was designed to measure slightly different aspects of the respondent's experience, so the use of capstone questions, focusing on the overall satisfaction level, was adopted. This is a standard procedure in such surveying.
Q.
What does Durham College do with these ratings?
A.
The Office of Institutional Research and Government Relations is responsible for analysis of KPI data. Each program and service/facility area is provided with a KPI Report Card that includes current year results as well as five-year trends. Particular emphasis is placed on those aspects that have been identified as important to students. Each program and service/facility area is responsible for completing an improvement plan that will further enhance the student experience.
Q.
What improvements has Durham College made as a result of KPI feedback?
A.
Based on KPI results over the past few years, Durham College has continued to introduce a number of new initiatives designed to enhance the student experience. In June 2007, the college opened a 90,000-square-foot expansion to the Campus Recreation and Wellness Centre. Also in 2007, the college increased space for group study, with an additional 10,000 square feet added for student use; expanded its Health Centre to offer wellness services such as a full-service pharmacy, sexual health centre, and chiropractic and massage therapy; and a Student Food Services Committee was established to provide feedback.
Q.
Where can I find specific program information on KPIs for Durham College?
A.
The full survey results are available at the Campus Library at the Oshawa campus.
Q.
How can I find out more about graduate employment for Durham College graduates?
A.
Each year, Durham College produces a Graduate Employment Report based on the information compiled from the KPI survey results. The information contained in the report profiles the employment status of Durham College students who graduated from post-secondary programs.
View the most recent Durham College Graduate Employment Report 294KB PDF [This link will open in a new window.]
Q.
How can I find out more about the performance of specific programs?
A.
All Ontario post-secondary institutions are required to provide students with information on graduation rates, graduate employment rates and Ontario Student Loan default rates for government-funded post-secondary programs. While common methods have been used to calculate all program-related information provided by all colleges, there may be some differences between how universities and private colleges calculate their information. For instance, graduate employment rates for college graduates are based on surveys done six months after graduation, while university graduate employment rates are based on surveys done six months and two years after graduation.
To view Durham College information, click here OSAP Performance Indicators.
For additional information on the KPIs, please visit www.collegesontario.ca. [This link will open in a new window.]