Setting standards for the New HSC
Published in Board Bulletin Vol 9 No 6, September 2000
From 2001 students’ achievement in each New HSC course will be reported in terms of standards.
The marks that are awarded will be given meaning by relating them to performance bands that describe the knowledge, skills and understanding of the course typically demonstrated by students whose marks place them in a particular band. Six bands will be used to report achievement. A student who is awarded a mark between 90 and 100 will be placed in band 6. Similarly, a mark between 80 and 89 will correspond to band 5, between 70 and 79 will correspond to band 4, between 60 and 69 will correspond to band 3, and between 50 to 59 will correspond to band 2. A mark below 50 will correspond to band 1, where band 1 is referred to as below the minimum standard expected.
Work on setting the standards has already begun. During the development of the new syllabuses, outcome statements were written. The outcomes, along with the course content, give clear guidance to teachers as to the knowledge, skills and understanding students will develop through the study of that course. Teams of experienced teachers also considered student responses, statistical data and other materials from past HSC examinations and prepared statements that summarised different levels of performance in the course. These statements correspond to the performance bands 2 to 6 and are referred to as band descriptions. No statement was prepared for band 1.
The process of setting the standards for each course is essentially a matter of determining the marks in the 2001 HSC examination that correspond to the borderline between each of the performance bands. The procedure used is based on the professional judgment of subject experts informed by statistical data and student responses. It is multi-staged, allowing for review and refinement of earlier decisions and is similar to that used successfully since 1998 for the School Certificate.
The Procedure
For each course in the New HSC a team of experienced markers is created. These markers, who are referred to as judges during this activity, are given special training for this task. They are also given a copy of the band descriptions for their course, a copy of the examination paper and specially designed recording sheets.
Step 1
Working independently of his or her colleagues, each judge reads the band descriptions carefully. They develop an ‘image’ of the knowledge and skills of students whose achievement would place them in each performance band in that course. The judges, using this information, then develop images of students whose achievements would place them on the borderline between two bands.
Having done this, each judge records the mark for each examination question (or task) that a border-line band 5/band 6 student would receive. Adding up these individual marks gives the total examination mark that the judge believes corresponds to the borderline (or cut-off mark) between band 5/band 6. Averaging the cut-off marks proposed by all the judges produces the first estimate of the examination mark that will represent the borderline between band 5 and band 6. The judges follow the same procedure for the other band borderlines.
Step 2
The judges then meet and discuss the decisions they have made individually. At the same time they are given special statistical reports that are very effective in showing how students of different abilities performed on each question in the examination. The judges work through and discuss this information. During this process each has the opportunity to modify any of the decisions he or she recorded during the first step. Through this step the team starts to develop a common image of students who would be at the borderlines between bands. The judges recording sheets are again collected and processed as in Step 1. This results in a new set of band cut-off marks.
Step 3
The examination responses of samples of students whose marks were equal to each of the band cut-off marks are collected. The judges then meet again and review and discuss these examination responses. They are asked to confirm that the responses are typical of what they would expect of students placed at the borderline between bands. The judges also review student works above and below the cut-off marks. During this process the judges have the opportunity to further refine their band cut-off marks.
Step 4
The judges next review the band descriptions in light of the information they have gained from the procedure to ensure that they correctly and appropriately encapsulate the knowledge, skills and understanding typically possessed by students who achieve each band.
What next?
At the end of this procedure the team recommends to the HSC Consultative Committee a set of band cut-off marks that are appropriate for the performance standards that will be used in reporting achievement in the course. The Consultative Committee meets with the judges and discusses the processes and issues that arose during the application of the procedure. Finally, the Consultative Committee gives its approval to the cut-off marks to be used for that course for the 2001 HSC examinations. The Consultative Committee also approves the maximum and minimum mark in the course.
Once this is done the Board’s computer is programmed so that the band 5/band 6 cut-off mark will be mapped to 90, the band 4/band 5 cut-off mark will be mapped to 80, and so on. Marks in between these key values are simply adjusted in a linear manner.
School assessment marks are then moderated using a similar process to that currently in use. In this way both the examination marks and the assessment marks are aligned to the performance standards.
Finalising the standards
After the examinations, the final version of the band descriptions, the 2001 examination paper and samples of responses of students at each borderline and other statistical information are collected and incorporated into a standards package. The material is presented in such a way that teachers, students and others can most effectively develop a clear understanding of the standards that have been developed for each course. The standards packages developed for the School Certificate are an example of this material.
These standards packages are also an essential part of the standard setting procedure for the following examinations. The teams of judges become thoroughly familiar with the material and apply the same standards in determining the band cut-off marks in subsequent years. In this way, while the actual cut-off marks may vary from year to year for a number of reasons, the standards used to report students’ achievement will not vary.
Further information
Further information on the setting of standards for the New HSC
can be found in Occasional Paper 1 - Setting Standards for Public
Examinations, which is published in the Reading Room area of the
Board’s website (www.boardofstudies.nsw.edu.au) or by contacting
the Director, Information Services (ph 02 9367 8192).