- How exam committees work
- Chief Examiner
- Examination development
- Review and evaluation
- Quality assurance in HSC examination development
An Examination Committee is appointed to develop the examination paper. Examination committees are structured to have, as far as possible, equal numbers of tertiary and secondary members, representation from non-metropolitan areas, gender balance and representation from both government and non-government schools. Examination Committee members are nominated by a selection panel and are approved by the Chief Executive on behalf of the Board.
Examination Committee members are provided with training to support them in their work to develop examinations and marking guidelines. The Chief Examiner and committee members, as well as being given opportunities to familiarise themselves with the course syllabus and previous examination papers, are provided with training in the nature of assessment in a standards-referenced framework; the standards-setting process in the Higher School Certificate; the examination-setting process and the roles and responsibilities of the people involved; and elements of examination setting (the examination brief, marking guidelines, rubrics, the Board’s principles, achievement bands, glossary of terms).
All members of the Examination Committee, including the Chief Examiner, are provided with a booklet of guidelines on the writing of examination questions and take part in training sessions covering: the outcomes-based approach of HSC syllabuses; principles and processes for setting examinations and developing marking guidelines; roles and responsibilities; and item design.
Adapted from Masters, G.N. (2002) Fair and Meaningful Measures? A review of examination procedures in the NSW Higher School Certificate, p15.
How examination committees work
For each HSC course, the Chief Examiner (CE) chairs the Examination Committee.
The process by which CEs are appointed includes a call for expressions of interest circulated to current Chief Examiners, members of examination committees and academics. Nominations of appropriate people also are sought from Board Inspectors. Senior Project Officers of the Board evaluate applicants against a set of criteria, and the appointment is made on the recommendation of a selection panel chaired by the President of the Board and including representatives of school systems, the university sector and senior Board officers.
The CE provides leadership to the committee in the preparation of the examination paper and the guidelines for marking the examination. It is the CE’s responsibility to ensure that the final examination is accurate and error-free, conforms to the examination specifications, and addresses an appropriate range of content and intended learning outcomes in the course syllabus. The CE also responds to any issues raised about the examination and is a spokesperson, when required, for the Examination Committee.
Following the sitting of the examination, the CE participates in the standards-setting process for the examination and provides a report to the Board’s Consultative Committee.
The Examination Committee meets over a number of days between October and April. The CE leads the committee in the drafting of questions and guidelines consistent with the examination brief and the Board’s operational requirements.
A Board Senior Project Officer works with the CE to plan meetings of the committee to ensure that the draft paper is prepared by the due date and provides guidance to committee members in the writing of questions and marking guidelines.
Board Inspectors or, where appropriate, the Curriculum Officer review drafts at various stages of development to ensure that questions are interpreting the syllabus correctly and that marking guidelines describe standards of performance consistent with the intentions of the syllabus.
Senior Project Officers ensure that the draft paper is consistent with the examination specifications, complies with the Board’s principles, uses key terms in accordance with the Board’s glossary, and is formatted in accordance with Board style.
Review and evaluation
Once the draft examination paper and accompanying marking guidelines have been developed, they are subjected to a variety of reviews.
Assessor Review – A practising teacher (in some cases more than one) provides answers/worked solutions to all questions except extended-response questions. For extended-response questions, the assessor is asked to indicate the characteristics they would expect a high-quality response to have. They also are asked to comment on such features as the wording of questions, clarity of instructions, accuracy of content, level of difficulty, space provided for answers, and the extent to which the paper is likely to be accessible to candidates at all levels of ability.
Curriculum Review – The Board Inspector or delegate reviews each examination paper to verify its appropriateness to the syllabus, including whether the examination tests a representative sampling of content and outcomes, whether the level of difficulty is such that it allows students to demonstrate performance at all levels on the achievement scale, and to verify that all questions have been correctly mapped against syllabus outcomes, content and band descriptions. In addition to these major reviews, the examination paper is subjected to a number of other reviews including:
Copy Editor Review – An independent professional copy editor checks the paper against the Test Development Style manual, checking spelling, punctuation, grammar and clarity of expression.
Vision/Hearing Impairment Review – Specialists in visual and hearing impairment provide advice on amendments to questions to assist candidates with visual and/or hearing loss where appropriate.
Examination Operations Branch Review – The Examination Operations Branch reviews instructions and layout for clarity and ease of use for candidates, presiding officers and markers.
The Supervisor of Marking reviews the draft marking guidelines for consistency with the Board’s principles for marking guidelines and to ensure that they are appropriate for the marking operation. The Supervisor of Marking signs-off that the final guidelines are consistent with the Board’s requirements for effective and efficient marking.
The Examination Committee considers and responds to issues raised through this set of reviews, making changes to questions and marking guidelines where necessary. A review group checks that all issues raised in the review phase have been adequately addressed by the Examination Committee. Any unresolved issues at this stage are referred to the Manager, Examining and Testing for negotiation and resolution. A Senior Project Officer ensures that all agreed changes are made.
The examination paper is then proofread by the CE and one other committee member. The final examination is signed off by the CE and countersigned by one other committee member. The marking guidelines are signed off by the CE and countersigned by the Supervisor of Marking.
The final stage is to evaluate each year’s paper following the examination and to forward all feedback on the paper to the CE for consideration in developing the following year’s paper.
Adapted from Masters, G.N. (2002) Fair and Meaningful Measures? A review of examination procedures in the NSW Higher School Certificate, p14–16.
Quality assurance in HSC examination development
The Board ensures the quality of the HSC examinations through the use of rigorous quality assurance procedures throughout the examination development process right up until the actual examinations begin.
The quality assurance procedures used to develop the Board’s HSC examination papers can be grouped into two categories:
- developing quality items and examinations (including marking guidelines)
- ensuring that the examinations are free of errors – for example, making sure there are no typographical errors, the layout and numbering are correct and comply with the Board’s style, and that marks are correct and instructions are clear.
Developing quality items and examinations
Appointing examination committee personnel
To be appointed to an examination committee or to be an HSC examination assessor, applicants must meet a set of selection criteria, including:
- appropriate qualifications
- experience teaching HSC students or first year university students
- knowledge of the relevant HSC syllabus
- high-level written and oral communication skills.
Chief examiners, who lead the examination committees, are also expected to demonstrate leadership, organisational and interpersonal skills.
When people are appointed as chief examiners, committee members or assessors, they sign an acceptance form that outlines all the conditions of appointment. They also sign a statutory declaration to say they will keep all examination material confidential.
Before work on the examinations starts, all chief examiners and committee members receive training about both the examination development process and writing quality examinations. Chief examiners and assessors are also briefed, before marking begins, about their role during marking.
Feedback from the previous year’s examination
After the examination has been conducted and marked, supervisors of marking provide feedback to the examination committee about questions that worked well and those that were problematic. In addition, statistical information on individual questions and the examination as a whole is provided to the committees to help them plan for the next examination.
Each year, a number of letters and emails are received about the examinations from teachers, students and parents. A summary of the issues raised is given to the appropriate examination committee for their consideration in developing the next year’s examination. The issues are also considered by Board officers in their evaluation of the year’s program.
Developing the examination
Throughout the examination development process, the examination is reviewed by a number of people – an assessment expert, a syllabus expert, the supervisor of marking, experienced teachers of the course, a literacy specialist and an editor. Committees are required to respond to every issue raised about the examination by the different reviewers. Before the draft examination is finalised, an audit is carried out by Board officers to ensure that all the issues raised by the reviewers have been addressed appropriately. The draft examination is also checked again to ensure that it still complies with the syllabus.
Ensuring that the examination is free of errors
During the development of the examination
As each draft of the examination is submitted and typed, there are a number of procedural checks made by the desk-topping officer. During the review stage, the draft examination is checked by an editor, and recommended changes are incorporated into the draft.
Committees are provided with a checklist to help them ensure that their multiple-choice questions meet all assessment requirements. The checklist covers technical aspects such as ensuring that the options are grammatically consistent with the sentence stem, that the questions are expressed in clear language, and that the options have been ordered logically. It also requires committees to ensure that there is one – and only one – correct or best answer to the question. In examination papers that include multiple-choice questions, the teacher assessor, as part of their review of the examination, is required to provide answers to the questions, and advise if there are any issues with the questions or distractors.
Examination committees must provide details of any sources used in the examination, and ensure they have been cited accurately and are labelled correctly. The details provided are given an independent editorial check.
Finalising the examinations
Prior to printing, the examination is proofread and checked by the chief examiner and members of the examination committee, and is reviewed again by an editor. It is also checked by Board officers for typographical errors, layout and numbering, marks and instructions, and the use of stimulus material.
The paper is checked again at the printers by Board officers who sign off on the printer proofs.
After the examination has been printed
After the examination paper has been printed, chief examiners check several copies of the printed examination to ensure that it has been printed and collated correctly. If an error is identified at this stage, Board officers determine a course of action to deal with the error. This may involve reprinting the examination, printing an errata notice that is given to each student, or having presiding officers at the examination centre make an announcement before the examination starts.