What does a university education actually teach you?
In a rare bid to answer the age-old question, Queen’s University is trying to measure what students actually learn — beyond the subject they’re studying. It’s tracking skills employers say they seek, but which long have been seen as unmeasurable; things like the ability to think critically, work with others, be organized and analyze problems.
“Employers want to know more than the fact you know your field; they want to know you’re going to show up at a meeting on time, have thought about the issue and bring questions to ask,” said Jill Scott, vice-provost of teaching and learning.
“Measuring these kinds of skills is a game-changer. People may roll their eyes at standardized tests, but if we don’t have the data, we can’t have these conversations.”
Queen’s is one of six institutions experimenting with ways to measure these “learning outcomes” beyond just course marks, in a project funded by Higher Education Quality Council of Ontario (HEQCO). The others are the University of Toronto, The University of Guelph, George Brown College, Durham College and Humber College.
At Queen’s, some 2,000 students in psychology, drama, physics and engineering have agreed to take part in “soft skill” assessments, from standardized international tests to personal surveys and special questions stitched into their own class assignments.
Among the tools Queen’s is using:
• The Cognitive Learning Assessment (CLA) is an online 90-minute test of critical thinking, communication and problem-solving. Widely used by American colleges, the test measures these “transferable skills” a student could bring to different types of work, and are deemed valuable by employers. “One engineering student was shocked he was being asked to analyze a poem as part of the test,” said Natalie Simper, co-ordinator of the Queen’s assessment project. “This is a poem — but I’m an engineer!”
• The Critical Thinking Assessment Test (CAT) is a paper-based assessment with problems that test the ability to think critically. For these standardized tests, Queen’s offers free pizza as an incentive.
• Special questions embedded in an assignment can help test core skills, such as communication and problem-solving.
• Students fill out surveys that indicate whether their personal behaviour might help them succeed, from being motivated and organized to how they work in a group, their self-confidence and self-discipline.
Already Queen’s has found students improved most between first and second year on questions that were embedded in their own course assignments, rather than the free-standing standardized tests,
“It stands to reason we’d see discernible growth when the questions are related to their field of study, where they’re most motivated,” said Simper.
In the end, it’s all about flexibility, said Susan McCahan, University of Toronto vice-provost of innovation and undergraduate education, who is in charge of the U of T’s pilot project.
“A good university education should not train you for one particular job; that was never the intent. But will you be flexible enough for all your jobs? That will put you in good stead for your whole career.”