cookiesCe site Web utilise des cookies

Nous utilisons des cookies pour améliorer votre expérience et reconnaître vos préférences. Pour apprendre et modifier vos paramètres, consultez notre Énoncé de confidentialité du site Web.
En utilisant notre site, vous consentez à utiliser des cookies à moins que vous ne les ayez désactivés.

SAEA - Service d’appui à l’enseignement et à l’apprentissage

Centre for University Teaching Resources for Program Review

Program Review at the University of Ottawa

The strategies, resources, and tools used at the University of Ottawa in the context of cyclical program review and curriculum development have evolved quite a bit over the last 5-6 years. The basis of the pedagogical support we use today at the Centre for Innovative Pedagogies and Digital Learning (CIPDL) was initially developed by Bob Parson. Since 2012, many practices including the SWOC/SOAR analysis focus groups (for faculty and students) and curriculum analysis data collection now have an online option. As such, we have developed surveys, templates and guides to assist programs with the collection and analysis of data which will help them describe the structure and function of their program. We outline a brief snapshot of our process and tools below.

Each academic program at the University of Ottawa must follow the protocol for cyclical review every 7-8 years. Consisting of an initial self-evaluation of the program, internal recommendations by a Senate committee, external review and a final recommendation report, the timeline is structured as follows:

Figure 1: University of Ottawa timeline for the cyclical review of undergraduate programs

CurrMap

What is Curriculum Mapping?

Curriculum mapping is the visual representation of the underlying logic of curricular design for a given program (Maki, 2004). In higher education, it is most frequently used for two main purposes:

  1. To ensure the alignment and sequencing of learning outcomes and assessments across courses when developing a new program; and
  2. To evaluate the current alignment and look for any gaps, redundancies and inconsistencies in order to enhance an existing program (Uchiyama & Radin, 2009; Kopera-Frye, Mahaffy & Svare, 2008).

    Curriculum mapping is “a deliberate process of curriculum deconstruction in order to understand better how the sum of the parts relates to the whole” (Jackson, 2000, p.144).

Curriculum Mapping in Five Stages

An evidence-based approach to collaborative curricular enhancement can commonly be characterised by five stages:

Figure 2: Five stages of curriculum enhancement

CurrMap5St

Stage 1 – Data Collection and Analysis

Using a Canadian based internet survey tool (Fluidsurvey), instructors identify which program learning outcomes are covered by their courses, at which level these outcomes are presented (introductory, intermediate, advanced), how the outcome is covered (taught, practiced, assessed), and if assessed, by what means.

Figure 3: Screen capture of the learning outcome section of the survey

LOs

Figure 4: Screen capture of the evaluation methods section of the survey

LOs2

Stage 2 – Interpretation and Discussion

Once the course information is collected, tables can be generated within a spreadsheet to better visualize the program’s underlying framework. Current tables include the integration of learning outcomes across program courses, assessment methods used throughout the program, grade weight and distribution of assessments across the program, favoured instructional approaches, etc.

Figure 5: Sample learning outcome distribution table

distTable

Figure 6: Sample assessment distribution table

AssessTab

Stage 3 – Identification of Areas for Improvement

As a catalyst for reflection and interdepartmental discussion, a series of questions are provided (Kenny, 2014). These include:

  • What learning outcomes are most/least emphasized?
  • Where are the strengths and gaps in teaching and assessment across the program?
  • Do the instructional and assessment methods used best align with the intended learning outcomes?
  • What instructional/assessment strategies are most/least used?
  • Are the instructional and assessment methods used in the courses congruent with the discipline and the program’s/Institution’s mission/vision?
  • In terms of supporting student learning, how well are the selected/utilized instructional and assessment methods actually working?

Stages 4 and 5 - Development of an Action Plan and Implementation

An action plan is most effective when it is faculty driven, data informed and supported by curriculum design specialists, and includes specific responsibilities and timelines as well as mechanisms to sustain the curriculum review and enhancement process by systematically revisiting the five stage cycle (Wolf, 2007).

Resources / Curriculum Analysis Templates

Undergraduate programs

Graduate Programs

________________________________________________________________

Seamless Data Collection: uoSyllabus

As these practices evolve at the Centre for Innovative Pedagogies and Digital Learning (CIPDL), the questionnaire continues to grow and become more robust. Of particular interest is how to collect information regarding student achievement of program learning outcomes and track progress on a continuous basis.

A tool called uoSyllabus being tested in a pilot phase may help standardise the collection of this information. uoSyllabus is an online module allowing instructors to produce syllabi that conform with the rules outlined by Senate and offer the possibility of adding content which aligns with best practices. Once the information is entered, the module produces a syllabus in a PDF format.

The module offers the possibility to specify which learning outcomes, accreditation standards or attributes are covered by each course within a program, thus facilitating data collection for various accreditation reports which many programs (including programs in professional schools) are subject to. The module equally enables curriculum design specialists to continuously collect information, thus facilitating the analysis of curricular alignment for the purpose of program evaluation.

uoSyllabus is now being tested by four professional Masters programs from the School of Rehabilitation Sciences: Audiology, Speech-Language Pathology, Occupational Therapy and Physiotherapy. All programs have accreditation standards that are specified in uoSyllabus, thus making it possible to identify how these standards are met by program courses.

List of uoSyllabus Advantages

Student Faculty members Institution
Course syllabi are accessible (WCAG 2.0, level 2A) Course syllabi are accessible (WCAG 2.0, level 2A) Course syllabi are accessible (Accessibility for Ontarians with Disabilities Act, 2005)
Information is easily identifiable uoSyllabus online module is accessible (WCAG 2.0, level 2A) Centralised database of PLOs
Automatically sent to students via uoZone (online student portal) Online management of past, present and future syllabi Control panel for Program Chairs with pre-formatted statistic tables of PLOs
Public syllabi are searchable User-friendly navigation Responds to Senate requirements (uOttawa academic regulation 8.5)
  Public syllabi are searchable Accurate image of how PLOs are covered by programs
  Multiple professors can contribute to a course syllabus Longitudinal data (instead of a snapshot taken every 7-8 years)
   
Moving discussions away from the mechanics of data collection to a culture of continuous program enhancement

List of Pre-Formatted Data Accessible Through the Control Panel

All data and crosstabs below are by course code letters (e.g. PHT)
                                                     OR
By course code letters AND 1st number of course code numbers (1, 2, 3, 4 and so on)
Descriptive (1 variable)
Count/Frequencies:
Crosstabs (2 variables):
Count/Frequencies:
List of courses in uoSyllabus (Letters, numbers, section) by semester (e.g. 20149)
Number of times each PLO is covered by courses
Teaching strategies
Methods of assessment
Value of assessments
Week of assessment
Course learning outcomes x PLO
For each PLO, crosstab with:
              Teaching strategies
              Type of assessment
              Level of inclusion (Principal or Secondary)         OR (Introduction, Reinforcement or Advanced)
Development (7 combinations of taught practiced and assessed= NT-NP-A, NT-P-A, NT-P-NA, T-NP-NA, T-NP-A, T-P-NA, T-P-A)
PLOs (count) x Level of inclusion
PLOs (count) x Development
Value of assessment  x Type of assessments
Value of assessment  x Week of assessments
Crosstabs (3 variables):
 
PLOs (count) x type of assessment x week         
PLOs (count) x type of assessment x percentage
PLOs (count) x week x percentage 
PLOs (count) x Level of inclusion x Development
 

Upcoming Development of uoSyllabus

  • All information regarding PLOs will be centralised under one tab (Curriculum Tab) allowing for the collection of more refined data about how they are taught and evaluated.
  • Ability for faculty to enter the average grade for each course assessment.
  • Faculty members will be able to enter the average result obtained for each evaluation (giving us a first look at which PLOs are most or least achieved by students).
  • Through a central system, we will be able to link the mean course grade to PLOs
  • Manager access for administrative staff in departments will allow one to:
  • Add user / prof
  • Ability to see and edit all course syllabi for their programs
  • Add program learning outcomes
  • Add instructional approaches
  • Add assessment methods
  • Two types of access to uoSyllabus: Regular mode and Course Design mode.       

Resources / uoSyllabus 

Web Page

User Guide

________________________________________________________________

Online SWOC Analysis

The self-evaluation process requires a description of the strengths and weaknesses of the program. The SWOC Analysis (Strengths, Weaknesses, Opportunities and Challenges) has proven to be an effective process for this.

In this two-hour session, participants identify the elements that are most important to their stakeholders and take part in an enriching dialogue amongst peers, discussing the program in a constructive fashion. As well, participants produce concrete suggestions on what could or should be done to improve the program.

In previous SWOC workshops for programs under review, participants indicated that the SWOC :

  • Created a concise vision of the program
  • Brought new ideas to the discussion
  • Generated active participation
  • Encouraged recognition of what was positive about the program
  • Confirmed common goals and aims

By invitation, the CIPDL will facilitate a SWOC analysis discussions with full time/part time faculty or students and provide a written report of findings. A meeting space may also be provided by the Centre for Innovative Pedagogies and Digital Learning (CIPDL).

For certain programs, reaching students/faculty can be challenging.  Increasingly, programs are choosing to move this process online and thus gather even more input from students and Faculty members, using an online SWOC to ask questions that normally are not part of a SWOC analysis.

Resources SWOC / Templates and Examples

Generic SWOC survey

Modified SWOC for a Master's Program

________________________________________________________________

Who We Are

For greater detail or if you have questions or comments, please get in touch with us.


As Curriculum Design and Quality of Learning Specialist, Jovan Groen, Acting Director of the Centre for Innovative Pedagogies and Digital Learning (CIPDL), works closely with the various faculties and departments in different processes of assessment, program development and review at the undergraduate level. In addition to working at the program level, Jovan collaborates with faculty in different professional development activities such as educational workshops, personalized support and guidance, and ongoing research on innovative pedagogical practices. With a background in both the physical sciences and educational psychology, Jovan has worked at various levels within the public school system and as an instructor and consultant within the university setting.

Jovan Groen, Tel.: (613) 562-5800 ext. 2607


Patrick Milot is a Graduate Curriculum Design and Professional Skills Development Specialist at the Centre for Innovative Pedagogies and Digital Learning (CIPDL) and at the Faculty of Graduate and Postdoctoral Studies. He provides support services for graduate programs in evaluation and for the creation and development of masters and doctoral programs. Patrick has over 20 years of experience in education, having taught at the high school, college and university level. He also managed higher education programs and faculty development projects.

Patrick Milot, Tel.: (613) 562-5800 ext. 4310

 

References

Jackson, N. (2000). Programme specification and its role in promoting an outcomes model of learning. Active Learning in Higher Education, 1(2), 132-151.

Kenny, N. (2014). Curriculum Mapping. Retrieved from http://www.uoguelph.ca/vpacademic/avpa/outcomes/curriculummap.php

Kopera-Frye, K., Mahaffy, J. & Svare, G.M. (2008). The map to curriculum alignment and improvement. Collected Essays on Teaching and Learning, 1, 8-14.

Maki, P. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus.

Uchiyama, K.P. & Radin, J.L. (2009). Curriculum mapping in higher education: A vehicle for collaboration. Innovative Higher Education, 33, 271-280.

Wolf, P. (2007). A model for facilitating curriculum development in higher education. New Directions for Teaching and Learning, 112, 15-20.