An agency of the government of Ontario

PERFORMANCEINDICATORSFORTHEPUBLICPOSTSECONDARYSYSTEMINONTARIO AConsultationPaperPreparedbythe HigherEducationQualityCouncilofOntario(HEQCO) ThepurposeofthisConsultationPaperistoseekyourcomments,feedback,andadviceonaHEQCOled projecttodevelopasetofperformanceindicatorstomonitor,evaluate,andimproveOntariospublic postsecondarysystem. Forclarity,thisprojectisdesignedtoallowHEQCOtobetterfulfillitslegislatedmandatetoprovidean evaluationoftheoverallstateofOntariospostsecondarysystem.Thisprojectisnotdesignedto evaluatetheperformanceof,orrank,individualinstitutions.Assuch,thisprojectisnotrelatedtoany datacollectionexerciseHEQCOmaybeaskedtoinitiatebytheMinistryofTraining,Collegesand UniversitiesinlightofitsrecentlyannouncedconsultationprocessaroundtransformationofOntarios postsecondarysector. Thethinkingofferedinthispaperisbaseduponadviceandguidancewehavealreadyreceivedfroma diversegroupofexpertsinthefieldsofhighereducation,performancemeasurement,andpolicy analysis,aswellasresearchwehaveconductedexaminingperformancemeasurementexercisesin otherjurisdictions. ThispaperprovidesthecontextforthePerformanceIndicatorsforthePublicPostsecondarySystemin Ontarioproject,hereinknownasthePerformanceIndicatorsProjectandourpreliminarythinkingona slateofindicatorstoassessOntariospostsecondarysystem.Thefirstsectionofthispaper(page2) outlinesthepurposeofthisproject,theguidingprinciples,andproposedtimelinesweusedtoframethe initialdiscussion.Thesecondsection(page4)summarizesmanyofthepointsthatweremadeduring ourpreliminaryconversationswithourgroupofexperts(seeAppendix1),andgivesasenseofthe discussionsanddebatesthatledtothisdraftlistofindicators.Thethirdsection(page10)liststhedraft setofindicatorsthatresultedfromthosediscussions. WearesendingthispapertoallOntariocollegesanduniversities,studentandfacultygroups, government,andotherswhocancontributetothedevelopmentofabettersetofsystemindicators thanwehavemanagedtocontemplatetodate.Inaddition,weareusingourwebsitetosolicitinput fromabroadaudienceknowledgeableabouthighereducationwhomaywishtocontributetothe discussion. Cerapportestactuellementencoursdetraduction.Nouslemettronsenlignedsquecestdisponible le11juillet2012. Thereareseveralwaystoprovideyourinput.Youcansendwrittensubmissionstousviaemailat: Indicators@heqco.ca(pleaseuseHEQCOPerformanceIndicatorsProjectasthesubjectline)orregular HEQCOPerformanceIndicatorsProject 1|P a g e

 mailat:HEQCOPerformanceIndicatorsProject,1YongeStreet,Suite2402,Toronto,ONM5E1E5.In addition,ifthereissufficientdemandfromasufficientcriticalmassinanarea,wearepreparedtotravel toyouforaninpersonconversation.WewouldappreciateyourfeedbackbyAugust31,2012. Wethankyouinadvanceforyourcontributiontothisimportantproject.

HEQCOPerformanceIndicatorsProject

2|P a g e

SECTION1:PURPOSE,PRINCIPLES,ANDTIMELINES
WhydidHEQCOinitiatethisproject? TwoofthelegislatedfunctionsofHEQCO1are: tomakerecommendationstotheMinisteronperformancemeasurestobeusedtoevaluate thepostsecondaryeducationsector,and toevaluatethepostsecondaryeducationsector,reporttotheMinisterontheresultsofthe evaluationandmakethereportavailabletothepublic.

Tohelpusimprovehowwefulfilltheserequiredmandates,weinitiatedaPerformanceIndicators ProjecttogenerateasetofinformativeandusefulindicatorstoassesstheperformanceofOntario's postsecondarysystem.TheprimarypurposesofthePerformanceIndicatorsProjectaretoinformpublic debate,discussion,andunderstandingoftheOntariopostsecondarysystem,andtoproviderelevant informationtogovernmentandinstitutionstoallowthemtobettermanageandhelpimprove postsecondaryeducationinOntario. Toinformthisexercise,wereviewedhighereducationperformancemeasurementregimesinother provincesandcountries.Inaddition,wesolicitedtheadviceofagroupofindividualswhoarenotonly informedaboutissuesrelevanttopostsecondaryeducationandperformancemeasurementexercises butwhoarealsopassionateaboutthepurposesandimportanceofeducation.Theseindividuals,listed inAppendix1,haveprovidedtheirtimeandexpertiseonaprobonobasisandwethankthemfortheir supportandadvice.Wetakethewillingnessofthisgrouptodevotetimeandattentiontothisprojectas testamentoftheirsenseoftheimportanceofthisexercisetotheadvancementofpostsecondary educationinOntario. Principlesguidingtheprojectandtheselectionofindicators Thefollowingarealistofprinciples,strategies,andkeypointsthatframedthisexerciseandthathelped guidetheselectionofappropriateperformanceindicators: 1. Ourtaskistodevelopindicatorstoevaluatetheperformanceoftheoverallsystem,notthe performanceofindividualinstitutionswithinthesystem.2 2. Ausefulsystemwideperformancematrixisonethatcanserveasaframeworkwithinwhich individualinstitutionscreateand/oraligntheirownperformanceassessments.However,given
HEQCOAct,2005.http://heqco.ca/SiteCollectionDocuments/heqco_act_2005_EN.pdf Inconsultationstodate,somehavequestionedwhetherOntarioreallytreatsandmanagesits44publichigher educationinstitutions(20universitiesand24colleges)asasystemorwhetheritreallyoperatesasacollectionof distinctsectors,thelargesttwobeingthecollegeanduniversitysectors.Weunderstandthedistinction.We preferthetermsystem,however,becauseourtask,totheextentpossible,istoassessthewholeOntariopublic postsecondarysystem,bothcollegesanduniversitiesandbecausethereareimportantlinkagesbetweendifferent sectorsinthesystem.
2 1

HEQCOPerformanceIndicatorsProject

3|P a g e

 therealitiesofthedifferencesbetweenthecollegeanduniversitysectorsinOntario,itmay makesensetoreportonthesetwosectorsseparately. Thebestsystemperformanceregimescovertherangeofactivitiesandcontributionsexpected ofpostsecondaryinstitutions.InOntario,thismeansthatanidealslateofindicatorsshould coverthebroadareasofeducation,researchandinnovation,andcommunitysupportand impact. Fourdomainsofperformanceindicatorsappeartocapturemostoftheinterestandrelevant informationdesiredinpostsecondarysystemevaluations.Thesefourbroaddomainsinclude: access,quality,socialandeconomicimpact,andsustainability/efficiency.Thedifficulttaskisto populatethesefourdomainswithrelevantandusefulperformanceindicators. Thebestindicatorsmeasureoutcomesorwhatactuallyhappenstostudentsinpostsecondary institutions.Thatsaid,someperformancemeasurementofinputsmaybeunavoidable. Ifsomethingcannotbemeasureditcannotbeanindicator.Whilesomemightsuggestthatthis principleisselfevident,wearestruckbyhowmuchtimeisspentconsideringfactorsthat peoplebelieveareimportantbutforwhichnoonehasbeenabletogenerateanacceptable measure.Theremaybeaspectsofpostsecondarysystemperformancethatarecriticalto measureandforwhichwedonotnowhaveareadilyavailablemeasure.Insuchcasesit behoovesustoarticulatewhatinformationmustbegatheredorwhatresearchmustbedoneto makethesecriticalmeasurements. Notallperformanceindicatorsarequantitative.Rather,somecanbeassimpleasayesno answeronwhetheraparticularattributeorprocessispresentorabsent,orevenasummaryof informationobtainedfromquestionsinasurveyinstrument. Totheextentpossible,andbecausetheprimarypurposeofasystemwideperformancematrix isimprovement,indicatorsshouldfocusonthefuture,i.e.wherethepostsecondarysector shouldbegoing,ratherthansimplycataloguingwhathashappenedinthepast. Thesuiteofperformanceindicatorsnotonlyreportsonthecurrentstateofthesystembutalso permitscomparisonsofOntariosperformancetootherrelevantjurisdictions. Noteveryperformancemeasureisperfectandnoperformancematrixiseither,especiallyatits introduction.However,performancemeasurementiskeytoimprovementandtoadvancingthe Ontariopostsecondarysystem.Thestrategythenistocomeupwiththebestsetof performancemeasurementspossibleatthetime,toapplythisassessmentdeviceand,most importantly,tocontinuallymonitor,assess,andrefinetheinstrument.

3.

4.

5.

6.

7.

8.

9.

10.

HEQCOPerformanceIndicatorsProject

4|P a g e

 ThepurposeofthePerformanceIndicatorsProjectistoassesstheoverallOntariopostsecondarysector. Ideally,thisperformanceassessmentmatrixcanthenserveastheframeworkwithinwhichindividual institutionsgeneratetheirownperformanceassessmentdevices.ThewaythatOntariomonitors performanceofitshealthcaresectorprovidesausefulmodelandexemplarofthisapproach.Initially, discussionsorganizedbytheOntarioHospitalAssociationsuggestedasetofsystemwideindicators. Overthecourseofmanyyears,theseweredevelopedandthenreleasedattheinstitutionallevel.More recently,usingtheseindicatorsasaframework,andmotivatedbytheExcellentCareforAllAct,all hospitalsintheprovincehavedevelopedqualityimprovementplansbasedonperformance measurementstailoredtotheirhospital,takingintoaccounttheirprioritiesandsituation.Onanannual basis,theOntarioHealthQualityCouncilassessestheseplansandreportsonanoverallassessmentof healthcareinOntario. OurhopeisthatthePerformanceIndicatorsProjectwillprovidethefirststepinthisprocess,i.e. identificationofasensibleandrelevantsetofindicatorstomonitorandassesstheperformanceofthe overallpostsecondarysystem.Then,ideally,institutionswouldusethisframeworktodevelopa performancemeasurementdeviceconsistentwiththeirmission,circumstances,andaspirations.For completeness,thismeansthataftertheslateofsystemindicatorsisdeveloped,eachcollegeand universityshouldconsidercarefullywhetherasystemperformancemeasureisapplicabletoitandto whatdegree.Similarly,governmentshouldnotinsistthatallofthesystemindicatorsareapplicablefor everyinstitutioninthesystemandevenifrelevant,thateachinstitutiondemonstratethesamelevelof performanceonthatindicator.Inthismanner,theexistenceofasetofsystemwideperformance indicatorscansupportdiscussionsofamoredifferentiatedOntariopostsecondarysysteminwhichthe achievementofthesystemisoptimizedbecauseeachinstitutioncontributesmore,andisrewarded,for whatitaspirestodoanddoesbest. Timelines OurintentionistofinalizetheslateofperformanceindicatorsinearlySeptemberandtoreleasean evaluationoftheOntariopostsecondarysectorusingtheseindicatorsinFall2012.Therefore,wewould appreciateyourinputbyAugust31,2012.

SECTION2:DISCUSSIONOFINDICATORS
Asnotedabove,examinationofsystemperformanceregimesinotherjurisdictionsandthe consultationswehaveheldtodatesuggestthatfourdomainsofindicatorslikelycapturemostofwhat appearstobeofgreatestinterestinexerciseslikethis.Thesedomainsare:access,quality,socialand economicimpact,andsustainability/efficiency. Access Accessensuringthatthereareenoughspotsforallqualifiedstudentsseekingapostsecondary educationisapredominantconcernofstudents,public,andgovernments.InOntario,inparticular, increasingcapacityhasbeenthedominantpolicyobjectiveoverthelastdecadeorsoandtheevidence

HEQCOPerformanceIndicatorsProject

5|P a g e

 suggeststhattheOntariopostsecondarysystemhasaccommodatedsignificantgrowth.3The governmenthassetapolicyobjectiveofachievinga70%postsecondaryattainmentrate4andhas establishedfundingincentivestopromotehigherenrolments.ItisdifficulttoimagineanOntario systemwideperformancematrixthatdidnotincludemeasurementsofaccess. Accesstopostsecondaryeducationismeasuredindifferentways. Postsecondaryparticipationmeasuresthepercentageofapopulationwithinadefinedagecohortwho attendpostsecondaryeducation.Thenumeratorinparticipationmeasuresisthenumberofstudentsin thatagegroupwhoareinpostsecondarystudiesandthedenominatoristhetotalnumberofpeoplein thatagecohort.Althoughanyagecohortcouldbeselectedtodetermineaparticipationrate,theone thatreceivesthegreatestattentionisthatofthosecompleting,orwithinseveralyearsofcompleting, highschoolusually1824yearoldssincethisisthemajorfeederofpostsecondarystudents.These measuresdonotnecessarilycaptureolderstudentsorotherlifelonglearnerswhomayenter postsecondarystudiesatanolderageorthroughnontraditionalpathways.Itisalsoimportanttonote thatparticipationmeasuresjustthatthoseenrolledandnotnecessarilywhethertheygraduated. Postsecondaryattainmentmeasuresthepercentageofthoseofadefinedagocohortinthepopulation whoholdapostsecondarycredential.ThepostsecondaryattainmentinOntarioforthosebetweenthe agesof2564in2011was64%accordingtotheLabourForceSurvey,meaningthatthispercentageof Ontariansinthatagerangeheldsomepostsecondarycredential.5However,thisdoesnotnecessarily reflecttheeffortsoftheOntariopostsecondarysystemasattainmentcaptureseducationalachievement regardlessofwherethateducationwasreceived.ItisclearthatthehighattainmentrateinOntariois assistedbythehighnumbersofOntarianswhoreceivedtheirpostsecondarycredentialinother provincesorcountries;howeveritisalsodiminishedbythosewhoattainedacredentialinOntarioand thenmovedoutofprovince. Asidefrommeasuringoverallparticipationorattainment,thereisconsiderableinterestin understandingthepostsecondaryachievementsandengagementofspecifictargetedgroups.Thisfocus hasbeenshapedbytherecognitionthatsomesegmentsofthepopulationareunderrepresentedin accesstopostsecondaryeducationinspiteofthegenerallyhighoverallparticipationandattainment scoresinOntario(andCanada)relativetootherOECDcountries.6Sinceapostsecondarycredentialis recognizedasbeingimportanttofutureeconomicandsocialsuccess,recentattentionhasbeen focussedontheparticipationofthesecurrentlyunderrepresentedgroups(e.g.,Aboriginals,students fromeconomicallydisadvantagedfamilies).Therefore,animportantmeasureofaccesscouldcapture theengagementoftheseunderrepresentedgroupsinpostsecondaryeducation.

HEQCOQuickStats:http://heqco.ca/enCA/Research/quickstats/Pages/default.aspx OntarioThroneSpeech.(2010).http://www.premier.gov.on.ca/news/event.php?ItemID=11282&Lang=EN 5 Norrie,K.andLin,S.(2009).PostsecondaryEducationalAttainmentandParticipationinOntario.Toronto: HigherEducationQualityCouncilofOntario.http://heqco.ca/SiteCollectionDocuments/AttainmentENG.pdf 6 Norrie,K.andZhao,H.(2011).AnOverviewofPSEAccessibilityinOntario.Toronto:HigherEducationQuality CouncilofOntariohttp://heqco.ca/SiteCollectionDocuments/AtIssue8AccessibilityENG.pdf

4 3

HEQCOPerformanceIndicatorsProject

6|P a g e

 Aswehavearguedinotherpapers,7itisimpossibletospeakaboutaccesswithoutalsobringinginthe issueofstudentfinancialaidandstudentdebtload.Therearesomewhoarguethatanassessmentofa postsecondarysystemmustincorporateameasureofthefinancialburdenonthestudent,intermsof eithertuitionlevelsordebtload.Evenifonearguesthatstudentdebtisanimportantmeasure,the capacityofstudentsinthesystemtorepaytheirloans,measuredperhapsbydefaultrates,mightbethe morerelevantmeasurethandebtaccrued.Wedonotminimizetheimportanceofthesevariables.But, ultimately,thesevariablesshouldbereflectedinaccessmeasures.Iftuitionsareunaffordable,orifthe financialaidsystemisinadequatetomitigatethesetuitionlevels,thenonewouldexpecttosee decreasedpostsecondaryparticipation. Quality Thequalityofeducationstudentsreceiveisthecurrencybywhichpostsecondarysystemsare evaluated;andifitisnot,itshouldbe. Thedifficultyisthatwhilequalityisacknowledgedbymanytobeamongthemostimportant performanceindicators,itgeneratesmuchcontroversyandhasbeenthemostdifficulttomeasure.Part oftheconceptualdifficultyhasbeenthelackofagreementoverwhatconstitutesaqualitymeasure. Thisproblemisexacerbatedbythefactthatweexpectourcollegesanduniversitiestoperformina numberofdomainssoqualitymeasuresforeducation,forexample,areobviouslygoingtodifferfrom ameasureofresearchquality. Wewillnotrehearseherethesubstantialliteratureonqualitymeasuresinhighereducation. However,ourreviewofthisliteratureandourconsultationstodatehaveledustothefollowing conclusionsaboutqualitymeasuresinaprovincialpostsecondaryperformancematrix. First,itisinconceivabletohaveapostsecondaryperformancereportcardthatdoesnotattemptto assessthequalityofpostsecondaryeducationintheOntariosystem.Wecantakecomfortfromthe observationthatthehighereducationworldhascometopeacewithqualityassessmentsinother areasrelevanttohighereducationeventhoughsimilarconcernsandcontroversiesexistaboutthe validityandadequacyofthespecificmeasuresselected.Forexample,weroutinelyandrepeatedlymake judgementsaboutthequalityofaprofessorsorinstitutionsresearcheventhoughwehaveconcerns aboutthevalidity,relevance,andmeaningofsomeofthespecificmeasuresusedsuchaspublication counts,sponsoredresearchfunding,citationcounts,oreventhepeerreviewsystemitself. Second,goingbacktotheprinciplesidentifiedabove,measurementsofqualityoftheOntario postsecondarysystemmustcover,asaminimum,qualityconsiderationsinthedomainsofboththe educationweprovidetothestudentsinthesystemandtheresearch/innovationcontributionsofthe system.And,betterindicatorsarethosethatpermitrelevantcomparisonofOntariotoother jurisdictions,andhopefullyovertimetoallowjudgmentofthesuccessofmacropolicymeasures.

Norrie,K.andLennon,M.C.(2011).TuitionFeePolicyOptionsforOntario.Toronto:HigherEducationQuality CouncilofOntariohttp://heqco.ca/SiteCollectionDocuments/AtIssueTuitionENG.pdf
7

HEQCOPerformanceIndicatorsProject

7|P a g e

 So,howisqualityofpostsecondaryeducationmeasured?Intheareaofeducation,somepromotethe importanceofstudentsatisfactionscores.Manyjurisdictionsassessstudentsatisfactionbutitis understoodandoftenarguedthattherearemanyvariablesthatinfluencestudentsatisfaction,only someofwhichmayrelatetohowmuchorhowwellstudentshavelearnedduringtheirpostsecondary education.8 Othersarguethatthebestmeasuresofeducationalqualityarederivedfromstandardizedtests.Oneof thegreatadvantagesofstandardizedtestsisthattheypermitapplestoapplescomparisonsacross jurisdictions,aswellasalongitudinalassessmentofthesystemovertime.Thetrickhere,ofcourse,isto developconsensusaroundwhichthingsshouldbemeasuredandthenagreementastowhichmeasures shouldbestandardized.So,forexample,fewwouldarguethattheacquisitionofbettercriticalthinking skillsisaqualitygoalofapostsecondaryeducation.But,evenwiththisconsensus,thereissome questionoverhowtomeasurecriticalthinkingability.Forexample,thereisconsiderablecontroversy whethertheCollegiateLearningAssessment,aninstrumentsometoutasthecurrentgoldstandardfor measuringcriticalthinking,isavalidmeasureofthisdesiredqualityoutcome.9 Withinthedomainofqualitymeasures,somepointoutthatthedominantreasonstudentsofferfor seekingapostsecondarycredential,andadominantreasongovernmentssupportapublichigher educationsystem,istograduatestudentswiththeskillsandeducationtofillorcreatejobsina knowledgebasedeconomy.Thisobservationleadssometopromotethenecessityofqualitymeasures thatcapturehowwellapostsecondarysystemgraduatesstudentswiththerightskillsetstosucceedin currentlabourmarkets.Itisunclearhowthisassessmentwouldbemade,althoughtheuseofsurvey instrumentsthataskemployersaboutthepreparednessofstudentsforworkareused.10Somehave arguedforextendingthesemeasuresbyaskingemployers(usingasimplesurveyinstrument)ofthe qualityandpreparednessoftheirnewemployeeswhoholdpostsecondarycredentials.11 Inourdiscussions,somehavenotedthepossibilitythatnoconsensuswillemergearoundspecific indicatorstoexaminequality.Thissuggestionleadssometorecommendthatinlieuofpresumedquality measures,aproxyistoinquirewhetherthepostsecondarysystemhasembeddedprocessesand practicesthatwouldallowforqualitymeasurementsandassurance.Thiswasthephilosophybehindthe

Somewillarguethatstudentsatisfactionwiththeireducationalexperiences,byitself,isanimportantmeasure regardlessofitscorrelationwiththequalityamountordepthoflearning.Othersarguethatacombinationof employmentandeducationalsatisfactioninfluencesoverallgraduatesatisfaction.SeeMcCloy,U.andLiu,S. (2010).WhataretheInfluencersofGraduateSatisfactionandLabourMarketOutcomesofOntarioCollege Graduates?AnAnalysisofOntariosCollegeGraduateSatisfactionSurveyResults.Toronto:HigherEducation QualityCouncilofOntario. 9 Arum,R.andRoksa,J.(2011).AcademicallyAdrift:LimitedLearningonCollegeCampuses.UniversityofChicago Press.USA 10 TheMinistryofTraining,CollegesandUniversitiescapturesannualemployersatisfactionratesofrecentOntario collegegraduatesasacomponentoftheiroverallemploymentprofile.Forthemostcurrentreportsee: http://www.tcu.gov.on.ca/pepg/audiences/colleges/serials/eprofile0910/profile10.pdf 11 Usher,A.(2012).MeasuringGraduateQuality.HigherEducationStrategyAssociates. http://higheredstrategy.com/measuringgraduatequality/
8

HEQCOPerformanceIndicatorsProject

8|P a g e

 originalUPRACprocessusedbytheCouncilofOntarioUniversitiesforqualitymeasurementinthe Ontariouniversitysectorandthemanylearningoutcomemappingexercisesintheprovince.12 Givensocietalexpectationsofpostsecondaryeducation,however,onealsowantstomeasurethe quality,oratleastthecompetitiveness,ofasectorinresearchandinnovation.Theresearch competitivenessofindividualinstitutionsisoftenmeasuredanditisnotdifficulttothinkhowonecould aggregatethesedatatocomeupwithsomesystemmeasure.Measurementsofinnovation,andhow muchapostsecondarysystemcontributestotheinnovationofajurisdiction,aremorecontroversial.13 Theseattemptshavetendedtorelyonmeasurementssuchasspinoffcompaniesorlicensing/royalty revenuefrompostsecondaryinstitutions,althoughthelimitationsoftheseindicesarealso acknowledged. Socialandeconomicimpact Manyhighereducationinstitutionsspeakeloquentlyandforcefullyabouthowtheysupportand underpinthesocialandeconomicwellbeingoftheircommunitiesandregions.Itisclearthatthe willingnessofgovernmentsandthepublictosupportapublichighereducationsystemismotivatedby itsunderstoodcontributiontoabetterqualityoflifeandhealthiereconomy.Itseemssensible, therefore,totrytoincorporatesomemeasuresofsocialandeconomicimpactinanassessmentofthe contributionofapostsecondarysystem. Wehavereceivedmanyanecdotesofhowpostsecondaryinstitutionsupliftthesocialfabricofa jurisdiction.Forexample,wehavebeendirectedtoindicesthatpurporttomeasurethequalityoflifeor liveabilityofcitiesorregions,14butitisunclearhowmuchofthismeasurecanbeattributedtothe contributionofthepostsecondarysystem.Thereissomeresearchthatfindslinksbetweencertaintypes ofpoliticalactivityandyearsofeducation.15 Monitoringeconomicimpactappearseasierbecausesomuchofthediscussionabouttheeconomic impactofpostsecondarysystemshastodowithjobsandjobcreation.Wearealsoawarethatstudents,
Seeforexample,OntarioUniversitiesCouncilonQualityAssurance.(2012).QualityAssuranceFramework http://www.cou.on.ca/relatedsites/theontariouniversitiescouncilonqualityassura/pdfs(1)/qualityassurance frameworkguidemay2012 13 Seeforexample,GovernmentofCanada.(2010).StateoftheNation2010CanadasScience,Technologyand InnovationSystem:ImaginationtoInnovationBuildingCanadianPathstoProsperityhttp://www.stic csti.ca/eic/site/sticcsti.nsf/vwapj/10059_IC_SotN_Rapport_EN_WEB_INTERACTIVEgood.pdf/$FILE/10 059_IC_SotN_Rapport_EN_WEB_INTERACTIVEgood.pdf 14 Seeforexample,TheEconomistIntelligenceUnitsLivabilityIndex http://www.eiu.com/site_info.asp?info_name=The_Global_Liveability_Report,andMercerQualityofLifeSurvey http://www.mercer.com/qualityoflivingpr#cityrankings 15 SeeRiddell,C.(2006).TheImpactofEducationonEconomicandSocialOutcomes:AnOverviewofRecent AdvancesinEconomics.CPRN'sSkillsandKnowledgeforCanada'sFuture:SevenPerspectivesTowardsan IntegratedApproachtoHumanCapitalDevelopmentproject.http://www.cprn.org/doc.cfm?doc=1490&l=en;and Curtis,J.,Grabb,E.,Perks,E.,andChui,T.(2004)."PoliticalInvolvement,CivicEngagement,andSocialinequality." InSocialInequalityinCanada:Patterns,Problems,andPolicies,4thed.,eds.J.Curtis,N.Guppy,andE.Grabb. Toronto:PearsonEducationCanada.
12

HEQCOPerformanceIndicatorsProject

9|P a g e

 whenaskedtoidentifythesinglemostimportantreasonforpursuingpostsecondaryeducation, invariablyidentifythedesiretogetagoodjobasthedominantmotivator. Sustainability/efficiency Manypostsecondaryperformanceregimesincludeassessmentsofthefinancialstateoftheinstitutions thatconstitutethesystem.Thereasonsforthisareobviousfinanciallysustainableandappropriately resourcedinstitutionsareconsideredprerequisitetoasustainablerobustsystem.Currently,Ontarios allocationtopostsecondaryeducation,$7.3Bin201213,16isamongthehighestofministry expenditures,toppedbyhealthandK12education.TuitionsinOntariouniversitiesareamongthe highestinCanada.17 Asidefromtheissueofhowmuchmoneyinstitutionsreceivetooperate,therearenowgreatercallsby thepublicandgovernmenttoholdthepublicsectoraccountableforhowitdeploysthefundsitreceives andwhethertheyarebeingallocatedinanefficientmannertoachievedesiredoutcomes.These efficiencyconsiderationsaremotivatedbythedesireofmanygovernmentstosupportaneducational systemthatdeliversahighqualityeducationtomorestudentswithnomore,orevenless,money.The recentDrummondReport18focussestheattentionoftheOntariogovernmentandOntariansonthe amountofmoneyapublicsectorhastodeliveritsservicesandwhatstructuralorprocessreformsmay increaseitsabilitytoofferserviceswithintightfinancialconstraints. Onemightthinkiteasytoidentifysomenumberfromtheextensivefinancialstatementsofinstitutions thatwouldattesttothefinancialhealthandsustainabilityoftheinstitutionandbyaggregation, therefore,ofthesector.Ifthereissuchanobviousnumber,wearenotsurewhatitis.Postsecondary institutionsarecomplexcreatures,withmultiplesourcesofrevenuethathavecomplexrelationships withoneanotherandnontrivialconstraintsontheiruse(e.g.,capitalversusoperating;researchfunds versusgeneraloperating;alltherestrictedtrustfunds,etc.).Furthermore,theauditedfinancial statementsofpostsecondaryinstitutionsareconstructedtoconformtoaccountingrulesand regulationsthatoftenmakeitdifficulttointerprettherelevanceormeaningofthenumberscontained inthem.Institutionsalsocarrysomeseriousfinancialliabilitiesaffectingsustainabilitythatarenoteasily observableintheirfinancialstatements,suchastheirongoingpensionliabilitiesandthefuturecostsof salariesandbenefitsofthetenuredfacultycomplement. Considerationsofsustainabilitycangobeyondthemoneyavailabletosupportthefaculty,staff,and studentswithinthesystem.Postsecondarysystemsalsorequireasubstantialandexpensivephysical planttosupportextensiveteachingandresearchprograms.Someargue,therefore,forexaminationof thesustainabilityofasystemscapitalinfrastructureandassets.
MinistryofFinance.(2012).OntarioBudget.Toronto:Queen'sPrinterforOntario,p.174. http://www.fin.gov.on.ca/en/budget/ontariobudgets/2012/papers_all.pdf 17 NorrieandLennon,2011,p.5 18 Drummond,D.(2012).CommissionontheReformofOntariosPublicServices:PublicServicesforOntarians:A PathtoSustainabilityandExcellence.Toronto:QueensPrinterforOntario. www.fin.gov.on.ca/en/reformcommission/
16

HEQCOPerformanceIndicatorsProject

10|P a g e

 Inourdiscussiontodate,somehavesuggestedthattheseconsiderationswillmakeitdifficulttoidentify quantitativelythefinancialhealthoforganizations.Rather,theyargue,abetterapproachistosee whetherinstitutionsengageinpracticesthatarethefoundationofsustainabilitythinkingorthatallow themtoplanproperlyforthelongterm.Inthiscontext,thesuggestionthathascomeupmostoftenis toinquirewhetherinstitutionsengageinmultiyearbudgeting. Finally,forthereasonsidentifiedabove,manygovernmentsareexaminingtheefficiencyorproductivity oftheirpostsecondarysystems.Thistypicallyinvolvesanexaminationoftheamountofoutputofthe systemrelativetotheresourcesmadeavailabletothem.19Theoutputscouldbethenumberof graduatesrelativetofundinglevelortheteachingloadperfacultymember.

SECTION3:RECOMMENDEDINDICATORS
Access 1. PercentageofOntariostudentswithinan1824(orso)agerangethatattendpostsecondary. Aquintessentialparticipationmeasureofthedominantandleadingcohortpursuingpostsecondary studies. Possibledatasource(s):StatisticsCanadaLabourForceSurvey:thepercentageof2024yearoldswho haveattended(currentlyorpreviously)college,trades,oruniversity.Allowsforcomparisonstoother provincesandpossiblyothercountries. 2. Differentialparticipationrateof1824yearold(orso)studentsfromfamilieswithinthehighest incomequartileandlowestfamilyincomequartile. Toidentifythedegreetowhichapostsecondarysystemiscapturingcurrentlyunderrepresented students.Familyincomedoesnotcorrelateperfectlywithallunderrepresentedgroupsofinterestbutit isprobablythevariablethatcapturesthemajorityofthem. Possibledatasource(s):StatisticsCanadaSurveyofLabourandIncomeDynamics.Allowsfor comparisonstootherregionsinCanada. 3. Percentageofstudentsenteringpostsecondaryeducationwhograduateinadefinedand reasonableperiodoftimegiventhelengthoftheirprograms. Afundamentalmeasurementthatgetsbeyondenrolmentandassesseswhetherstudentsactually graduate. Possibledatasource(s):MinistryofTraining,CollegesandUniversitiesKeyPerformanceIndicators(KPI) forgraduationrate.AllowsforcomparisontotheUnitedStates.

Seeforexample,LuminaFoundation.(2011).StrategicPlanGoal2025. http://www.luminafoundation.org/wpcontent/uploads/2011/02/Lumina_Strategic_Plan.pdf

19

HEQCOPerformanceIndicatorsProject

11|P a g e

 Quality 4. NumberofOntarioinstitutionsadjustedforpopulationintheTop100ofworldrankings. Anassessmentofglobalcompetitivenessofthesystem. Possibledatasource(s):TimesHigherEducationWorldRankingsofUniversities;ShanghaiJiaoTong rankings.Allowsforcomparisontootherprovinces,states,andcountries. 5. LiteracyandnumeracylevelsofstudentsintheOntariopostsecondarysystem. Literacyandnumeracyarefundamentalskillsexpectedofpostsecondarystudents.Manywell developedinternationallyrecognizedmeasurementdevices. Possibledatasource(s):CanadaparticipatesintheAdultLiteracyandLifeSkillsSurvey(ALL)andthe ProgrammeforInternationalAssessmentofAdultCompetencies(PIACC),bothofwhichallowfor comparisontootherprovincesandcountries. 6. Employersatisfactionwithpostsecondaryhires. Possibledatasource(s):TheresultsfromtheEmployerSatisfactionSurveyarepublishedbytheMinistry ofTraining,CollegesandUniversities.Theannualsurveycapturesemployersatisfactionofrecent Ontariocollegegraduatesacrossskillsetssuchascommunication,numeracy,andjobpreparation. CurrentlytheseresultsonlyexistforOntariocolleges. 7. OntariosshareofTriCouncil(NSERC,SSHRC,CIHR)funding. MeasurescompetitivenessoftheOntariopostsecondarysysteminresearch. Possibledatasource(s):Federalgrantingcouncils. Socialandeconomicimpact 8. Percentageofindividualswithapostsecondarycredentialwhovoterelativetothevotingrateof thosewithoutapostsecondarycredential. Thereisastrongviewthatapostsecondaryeducationshouldproduceengagedcitizens.Ifso,they shouldhaveahighervotingratethanthepopulationwithoutapostsecondaryexperience.Thisisalso oneofthefewsocialimpactsforwhichwehopewecangetreliableandreasonabledatatoassesssome contributionofpostsecondaryeducation. 9. Employmentrateofholdersofapostsecondarycredentialrelativetoemploymentrateofthose withoutapostsecondarycredential. Ausualmeasureofpostsecondarycontributiontotheeconomybutonethatcontrolsforeconomic cycleswheretheoverallrateofemploymentmaychangeduetofiscalandeconomiccircumstances. Allowsforcomparisonstootherprovincesandcountries. HEQCOPerformanceIndicatorsProject 12|P a g e

 Possibledatasource(s):StatisticsCanadaGeneralSocialSurveyorLabourForceSurvey,Ministryof Training,CollegesandUniversitiesannualsurveyofgraduatesforKPIsrelatingtoemploymentrateand relevanceofprogramtoemployment. 10. Numberofnewjobscreatedinthepastyearrelativetothetotalnumberofjobsintheprovinceat thebeginningoftheyear. Ifthepostsecondarysystemissupposedtofueltheknowledgebasedeconomy,thenproportionally morejobsshouldbecreatedinjurisdictionswithbetterpostsecondarysystems.Allowsforprovincial andinternationalcomparisons. Sustainability/efficiency 11. Grant+tuitionasa%ofprovincialGDP. Measuresdegreeofoperatingresourcesprovidedtothesystemadjustedforsizeoftheeconomy. Institutionsrequirereasonablefundingiftheyaretodeliveronthepublicgoalsexpectedofthem.Itis alsoimportanttobeabletocomparetheoperatingresourcesavailabletoOntariosysteminstitutions relativetotheircompetitorinstitutionsinotherjurisdictions.Grant+tuitionrepresentthegreat majorityofoperatingrevenuestopublicinstitutionsinthesystem. Possibledatasource(s):Datausedtomeasurepublicandprivateexpendituresoneducational institutionsasapercentageofGDPwithinCanadaarecombinedforthePanCanadianEducation IndicatorsProgram(PCEIP)andarepublishedinthereport,EducationIndicatorsinCanada:An InternationalPerspective.ThesedatacanbeusedtomakecomparisonsbetweenCanadianprovinces andothercountries. 12. Numberofinstitutionswhereannualoperatingrevenueexceedsannualoperatingexpenses. Measureswhetherinstitutionshavetherevenuetheyneedtomeettheirongoingannualexpenses. 13. Annualgrant+tuitionpercredentialawarded. Providesonemeasureofproductivitybycalculatingtherelationshipbetweentheongoingoperating resourcesprovidedtothesystemandthemostsignificantoutputofthesystemgraduates.

HEQCOPerformanceIndicatorsProject

13|P a g e

APPENDIX1:EXPERTADVISORYGROUP
DavidTrick,DavidTrick&Associates KevinLynch,ViceChair,BMOFinancialGroup KenSnowdon,KenSnowdon&Associates RobMacIsaac,President,MohawkCollege VivekGoel,President&CEO,PublicHealthOntario MinistryofTraining,CollegesandUniversities AlexUsher,HigherEducationStrategyAssociates SamAndrey,formerExecutiveDirector,OntarioUndergraduateStudentAlliance PeterGooch,SeniorDirector,Policy&Analysis,CouncilofOntarioUniversities ChavivaHosek,PresidentEmeritus,CanadianInstituteforAdvancedResearch(CIFAR) MichaelFullan,ProfessorEmeritus,OntarioInstituteforStudiesinEducation,UniversityofToronto HowardAlper,Chair,Science,TechnologyandInnovationCouncil(STIC)ofCanada

HEQCOPerformanceIndicatorsProject

14|P a g e