Overview 

The Undergraduate Medical Studies (UGMS) Committee, a standing committee of Faculty Council, has responsibility for "the overall policy of evaluation and the planning of the programs of studies leading to the MD degree". 

The Committee on the Accreditation of Canadian Medical Schools (CACMS) Element 9.8 requires that "A medical school has in place a system of fair and timely summative assessment of medical student achievement in each required learning experience of the medical education program." 

Purpose 

Summative assessment is used to document learner performance after a period of instruction. This procedure outlines the approach to summative assessment for all Phase 1-3 courses in the MD curriculum. 

Scope 

Courses in Phases 1-3 of the Undergraduate Medical Education (UGME) Program.

Procedure 

A.0 Assessment Planning 

A.1. The assessment plan for each course must provide the method(s) of assessment, an outline of how each assessment contributes to the final course grade, the examination schedule and due dates for any assignments or other course work.

A.2. The assessment plan must specify the criteria for passing the course or rotation and the conditions for required re-assessment (see Section H). 

B.0 Assessment Oversight 

B.1. Prior to the administration of any written assignments, the relevant Phase Management Team must file a grading rubric with the Student Assessment Subcommittee (SAS) for all written assignments that contribute to a learner's summative mark.

B.2. After the administration of assessment instruments, the relevant Phase Management Team must file an examination blueprint with the SAS for each summative multiple choice question (MCQ) examination. The blueprint specifies the objectives tested and their relative weights on the examination. 

C.0 Administration of Examinations 

C.1. All internal summative MCQ examinations will be processed through the UGME office. This includes administration of each examination, collation of learner scores and final reporting of scores to learners.

C.2. All summative MCQ examinations are conducted under supervised conditions as described in the Exam Invigilation Procedure. 

C.3. All summative MCQ examinations for Phases 1-3 will include an opportunity for learners to submit written requests for review of specific questions (e.g. through the use of 'challenge cards'). 

D.0 Deferral of Assessments 

D.1. Learners seeking to defer a summative MCQ examination or other assessment must follow the Undergraduate Medical Education Deferred Examination Policy.

E.0 Mark/Grade Release 

E.1 Individual assignment marks and grades must be submitted to the Assessment Academic Program Assistant (APA) in the UGME office for release to the learners via one45. Marks and grades are not to be released directly to learners by faculty. 

F.0 Examination Questions 

F.1. Faculty members provide exam questions through the following processes: 

F.1.1. Twenty (20) working days before each block, an email is sent from the d2l@med.mun.ca  account to all faculty members teaching in the block requesting two (2) formative exam questions, two (2) summative exam questions and one (1) reassessment exam question per hour of instruction. Questions may be drawn from the summative and formative item banks or new items may be created. Formative questions may not be used on summative exams. Questions must be linked to session objectives and conform to the guidelines of the Phase Assessment Working Group (PAWG), which are based on the Medical Council of Canada (MCC) and National Board of Medical Examiners (NBME) guidelines.

F.1.2. Five (5) working days after the first notification the Associate Dean, UGME, is notified of those who have not submitted questions and faculty members are re-contacted regarding question submission. 

F.1.3. Seven (7) working days before examination day, individual faculty members are re-contacted (copied to the appropriate PAWG) by the appropriate Phase Assessment Faculty Lead if their questions have not been submitted. 

F.2. Examination questions are reviewed by the appropriate PAWG through the following process:

F.2.1. Submitted questions are reviewed to ensure that they meet recommended guidelines.

F.2.2. Where questions do not meet recommended guidelines, the PAWG contacts faculty members for recommended revisions.

F.2.3. The PAWG selects questions for formative, summative and reassessment examinations.

F.3. Examinations are created electronically and the system is tested for proper functioning prior to delivery.

G.0 Standard setting for examinations;

G.1. The absolute pass score on a summative MCQ examination is 70%. All learners achieving less than 70% on an examination are notified the day of the exam that they might be required to write a reassessment examination.

G.2. After the review of the examination items and scoring is complete and any credits are awarded in the examination, a modified Hofstee method is used to determine the final pass marks for the summative MCQ examinations. 

Using this method, the UGMS has set the following parameters for Phases 1-3: 

1) mark above which all learners will receive a pass is 70% 

2) maximum percentage of learners who can fail an exam is 10% 

3) maximum percentage of learners who can pass an exam is 100% 

4) mark below which a learner will fail, subject to the limit set in #2 is 60% 

In Phases 1-3, the modified Hofstee method determines the final pass mark if any learners achieve a mark less than 70%. This Hofstee pass mark will be between 60 and 70%. The final pass mark is calculated using the Hofstee algorithm developed by Health Sciences Information and Media Services.

G.3. After the final pass score is determined, the following will take place: 

G.3.1. Learners achieving less than the final Hofstee score are notified that they are required to write the reassessment examination. 

G.3.2. Learners achieving less than 70% but higher than the final Hofstee score are notified that they are strongly encouraged to write the reassessment examination for their own learning. In such cases, the learner's original examination score will be counted as the final score. 

H.0 Review of Examination Items and Scoring

H.1. The PAWGs or designate will undertake a post-examination review prior to the final determination of learner scores.

H.1.1. Statistical item analysis (where available) and requests for review of specific questions (e.g. 'challenge cards') are taken into consideration to determine a corrected answer key. 

H.1.2. Final decisions for crediting/re-scoring are made by the PAWGs in consultation, as appropriate, with the faculty member(s) responsible for the construction of the original assessment item(s), other content experts or specialists in construction of examination questions. 

H.1.3. All scoring changes for individual examination questions are applied to the whole class. 

H.2. A learner may review a summative MCQ examination through the following process: 

H.2.1. The examination review request is made through the UGME office.

H.2.1.1. Learners who failed the examination are given priority.  

H.2.2. The examination review occurs in a meeting of the learner with an APA of the UGME office (or delegate). 

H.2.2.1. In this meeting: 

H.2.2.1.1. The learner may request and receive a copy of an individualized Coaching Report that provides a listing of the questions that the learner answered incorrectly on the examination. 

H.2.2.1.1.1. The learner's incorrect response is provided. 

H.2.2.1.1.2. The correct answer is not provided. 

H.2.2.1.2. The learner reviews the Coaching Report in the presence of the APA. 

H.2.2.1.2.1. The learner is not  permitted to record or transmit any information during this review process.

H.2.2.1.2.2. A learner may be allowed to use a personal computer to view a presentation stored on D2L, but this computer use is only permitted under close scrutiny of the APA or delegate. 

H.2.2.1.3. Once the Coaching Report is reviewed, the entire Coaching Report is returned to the APA or delegate. 

H.2.2.1.4. The learner is given a copy of the Coaching Report that contains a listing of the Blueprint ID (e.g., numbered learning objective) of each question incorrectly answered. 

H.2.2.1.4.1. No questions, choices or answers appear on this Coaching Report. 

H.2.2.1.4.2. The learner may retain this Coaching Report. 

H.3. Learners seeking reconsideration of the assessment/reassessment result or a reread of an examination or paper must follow the procedure outlined in the University Calendar. 

I.0 Summative Reassessment 

I.1. Reassessment is required if a learner achieves less than 70% (or the adjusted Hofstee pass mark in the case of summative MCQ examinations, whichever is lower) on any summative assessment.

I.2. A learner may be reassessed only once for any failed assessment. 

I.3. The pass mark for the reassessment will be 70%, or the Hofstee adjusted mark on the original examination in the case of summative MCQ examinations.

I.4. Reassessment for an MCQ examination is a comparable MCQ examination. 

I.4.1. Reassessment examinations include at least 40% new items.  

I.4.2. The reassessment examination is scheduled for at least one week after the original examination. 

I.5. Appropriate reassessment for written assignments and other assessments is determined by the faculty member who set the original assessment. 

I.6. In cases where a learner scores lower on the reassessment than on the original summative assessment, the higher mark will stand and be used in calculations of course marks and grades. 

J.0 Grading 

J.1. Final grades in all courses are determined using the mechanism detailed in the relevant course assessment plan. 

K.0 Program Evaluation 

K.1. At the end of a phase, the SAS will send an assessment evaluation report to the Phase Lead with a copy to the Program Evaluation Subcommittee (PESC).  

K.2. The Phase Lead or designate will respond to the SAS within one month. 

K.3. An assessment evaluation report determined by the SAS to be adverse automatically requires the Phase Management Team to conduct a review of its assessment plan and if required, submit a revised plan for consideration and approval. 

An adverse report would normally consist of at least one of the following: mean scores of <3.5 on assessment-related items reported from Course Evaluation forms; major concerns recorded from student evaluation feedback; >10% of test items from a summative exam identified as problematic from statistical item analysis.

_________________________________

1 The exam review process described here is distinct from (and not intended to replace or contravene) General University Regulations6.8.3 and 6.8.4 governing access to and request for rereading of final examination scripts.

Related Links 

Summative Assessment Policy Phases 1-4

Examination Invigilation Procedure

Undergraduate Medical Education Deferred Examination Policy 

University Calendar: Examinations