Academic Quality Review

Introduction

Both the 1991 and 1992 OCUA Fall Hearings have addressed the importance of accountability for universities. Specifically, Council has proposed processes for periodic institutional and provincial level review of undergraduate programs. Nipissing does not currently have in place an Academic Quality Review process, although some academic units have been reviewed on a periodic basis.

OCUA (1991) proposed some of the following as a list of objectives for a systemic/cyclical review of academic programs.

• Academic integrity - the approach adopted must be seen to have integrity in being able to evaluate effectively and address the quality of academic programs.
• Public integrity - the approach adopted must be seen by interested parties external to the universities to be credible and effective in addressing the quality of academic programs.
• Respect university autonomy - an appropriate balance between the need for public accountability and institutional autonomy should be effected.
• Administrative ease - the process chosen should be easy to administer and not require a significant and costly bureaucracy for its administration. 
• Standards reflect fiscal realities - the standard developed and the consequent expectations of the public must reflect the fiscal realities faced by Ontario universities.
• Build an informed public - provide consumers of academic programs (for example, students, parents, employers and government) appropriate information on the quality of academic programs and what is happening in the classroom.
• Institutional ownership - for the process to work effectively in the long-term, institutions themselves must have a sense of ownership in the process and that it meets the unique circumstances found at their university.

After review of the OCUA discussion papers and the program assessment procedures used by other Ontario universities, the following process is proposed for Nipissing University.

Undergraduate Program Assessment

Purposes

The primary purpose of the assessment process at Nipissing is formative. That is, the review is intended to primarily provide data that will lead to improved program quality.

In addition to program improvement, academic assessment may have the following purposes.

• to inform decision makers about strengths and weaknesses of programs
• to help make decisions about modification, expansion or termination of programs
• to provide information to the planning process
• to help make decisions concerning the allocation of resources
• to provide information to prospective students, parents and employers

Assessment Process

Level of Reviews:

Reviews will be conducted at the basic academic unit (BAU) level. The BAU is the lowest reasonable aggregation of individual departments or programs. In some circumstances, it could be determined that the BAU is at the Faculty level (e.g., Education) and for others, the department or discipline level (e.g., Geography).

The basic academic units would be defined by Senate (Academic Planning Committee).

Frequency of Reviews

Defined basic academic units would be reviewed on a cyclical basis at approximately five to seven year intervals. Specifically, the Faculty of Education (as a BAU) should be reviewed through the process every five years. At least one BAU from the Faculty of Arts & Science should be reviewed through this process annually.

Appointment periods of administrators, proposals for program change, and staffing considerations are issues that could influence the timing of BAU reviews.

Administration and Coordination:

Normally the Dean or a designate of the appropriate Faculty would be responsible for administering the review process. For reviews at the Faculty level, normally the President or designate of the appropriate Faculty would be responsible for administering the review process. In this regard, the President, the Dean, or designate is responsible to the Academic Planning Committee for the review process. The Chair of the Academic Planning Committee will carry review reports forward to the Senate and to the Board.

For reviews at the Faculty level, normally the President or designate of the appropriate Faculty would be responsible for administering the review process.

Steps in the Review Process

The following are the steps to be followed in the BAU assessment.

Step 1: Self study exercise at the BAU level (see details under "Document for Reviewers") 

Step 2: Peer Review: From one to four external reviewers per assessment depending upon the size, scope and nature of the review, will report independently to the Review Committee Chair. External reviewers are appointed by the Academic Planning Committee from nominations received from the Review Committee Chair and the BAU.

Step 3: External assessors visit the university and submit separate reports. (see subsequent section on Site Visit.)

Step 4: Unit Response: The unit under review will submit a reply to the reports to the Review Committee Chair.

Step 5: Reporting: Formal report to be received by the Academic Planning Committee, the Senate and the Board. The formal report consists of the executive summary prepared by the Review Committee Chair and will include all relevant reports and documents.

Step 6: Follow up: Within six months of Senate receiving the document outlined in Step 5, a follow-up report to each review would be provided to the Academic Planning Committee, the Senate and the Board identifying action considerations. The report would be prepared by the President or Dean in consultation with the BAU.

Document for Reviewers

External reviewers should receive appropriate documentation from the Chair of the Review Committee.

The following is a suggested list. Other items may be added as appropriate. All items should be classified by the Academic Planning Committee in the context of the size and scope of the review as one of the following:

1. Required core material (this material must be provided to the reviewers)
2. Suggested material (the Academic Planning Committee would suggest that this material be available to the reviewers)
3. Desired material (this material would likely enhance the reviewers' perspective)

List of Possible Materials

1. The most recent BAU academic plan, including a Mission Statement, an outline of BAU decision-making structures, a rationale of degree requirements, an overview of the composition of student body, an explanation of the shape and emphasis of its curriculum.
2. A statement of program objectives.
3. Self study report including specific concerns on which the consultants advice is sought. Minority opinions if any should be included.
4. Curriculum vitae of tenured/tenure track and limited term faculty, and a summary of names and qualifications of temporary or sessional faculty.
5. Undergraduate (and graduate) curricula, with proposals (if any) for new courses.
6. Enrolment data, current, projected and for the preceding seven-year period.
7. A tabulation of current teaching loads, showing the number of courses taught by each faculty member (undergraduate and graduate).
8. Information regarding the quality of teaching in the unit (e.g. teaching awards to faculty, students evaluations of teaching).
9. A grade distribution profile of undergraduate courses.
10. A description of teaching and research space and equipment.
11. The amount and source of research funding over the last seven years, and current and projected developments.
12. Statements on Library and Computer resources (prepared in collaboration with the Library and Computer Centre) commenting on any special features of the collection/service.
13. Employment and career data of graduates of the program, as available.
14. A description of the structures and procedures within the unit for facilitating new initiatives in research, teaching and service (e.g. committee structures, leadership responsibilities).
15. Relationships with other University units (e.g.. interdisciplinary courses, cross-appointments).
16. The unit's impression of how it compares with similar Departments or Schools in Canada and abroad, with the basis of the comparison.
17. Community relationships, liaison and service.
18. Operating funds for the unit (excluding salaries), with a breakdown of major budget categories.
19. Sample schedule of activities during the site visit.
20. Course outlines.
21. Samples of final examinations.

Site Visit:

The reviewers should visit the BAU at a mutually convenient time for at least two days. They should meet with faculty individually and/or in groups and with students and the deans, appropriate Librarians, computer staff and should, if possible, observe teaching. Reviewers should visit at the same time. At least two weeks prior to the scheduled visit, the reviewers would receive the materials prepared by the BAU and guidelines for their report.

Reports of Reviewers:

Within five weeks of the visit, the reviewers will prepare independent reports responding to the items on the various agendas, offering a considered overview of the program's strengths and weaknesses, and suggesting steps that departments might take - with and without additional resources - to improve their programs. Any comments or observations provided regarding individuals in the basic academic unit (BAU) are to be directed under separate cover to the Dean of the Faculty. The Dean will share these documents/comments with the Chair of APC; however, these comments may not necessarily be forwarded to the department or the Academic Planning Committee. The spirit of these reviews should be formative and constructive. All reports come to the Chair of the Review Committee. Copies of the individual reviewers reports are circulated to the BAU.

Responses to Reports

1. Departmental
   BAUs are asked to prepare, within five weeks of the receipt of the reports, written responses to reviews, including a discussion of changes to be implemented and commentary on suggestions from the reports. The response may take the form of commentary, or a revised academic plan. BAU responses will be circulated to the relevant dean and to the Academic Planning Committee.

2. Dean
   After receiving the BAU response, the Dean will respond, normally in writing, with copies to the BAU and to the Academic Planning Committee.

University Follow-Up:

There is no assurance that all of the consultants' recommendations will be implemented, but every effort will be made to address comments/concerns identified by the reviewers for program improvement.

Cost

The expenses associated with the visit by external reviewers will be covered by the Office of the President. This item would be budgeted annually and depend upon the number and scope of the reviews for a particular year.

In addition, the university budget will include a small contingency fund to allow the President to deal with urgent action items identified by the reviewers.

March 1993
Revised: April 2000