2002 HSC Update Newsletter 6
ALIGNING HSC RESULTS TO STANDARDS OF ACHIEVEMENT
Standards-setting
A major part of the new HSC is the description of standards of achievement in performance bands that accompany students’ HSC marks.
Each performance band for a course includes a description that summarises the knowledge, skills and understanding typically demonstrated by students whose achievement meets that standard.
The standards-based marks that are reported are determined through a standards-setting process in which students’ raw examination marks are aligned (or transferred) to the HSC performance scale. (See Figure 1.)
Experienced teachers (working as judges) make decisions about which raw marks equate with the cut-off points between bands on the performance scale.
Once a correspondence is established between particular raw marks and the cut-off points between bands at 50, 60, 70, 80 and 90 on the performance scale, it is possible to align (transfer) all marks to the scale.
Making judgements
The judging process involves three stages, giving each group of judges several opportunities to review and refine their earlier decisions.
Judges are trained for this task prior to the commencement of the HSC examinations.
As part of their training for the course, the judges study the 2001 HSC standards package and develop an ‘image’ of the kinds of knowledge and skills typically characterising students on the borderline between each band. This helps them to ensure that student achievement is reported in terms of the same standards each year.
Stage 1
Working independently, each judge considers each examination question and records the mark that they believe a student at the borderline between Bands 5 and 6 is likely to receive on that question.
They decide what mark in each question on the 2002 examination paper would be likely to be scored by the students who produced the responses in the 2001 HSC standards package.
Each judge’s marks are then added across all examination questions to give a total examination mark corresponding to the borderline (or cut-off) between Band 5 and Band 6 for that judge.
The cut-off marks between Bands 5 and 6 determined by all the judges in the team are then averaged to provide the first estimate of the minimum examination mark required by a student to be aligned to Band 6.
The judges follow these same procedures to determine the cut-offs for each of the other bands.
Stage 2
Once these first estimates for band cut-offs have been made, judges meet to discuss the individual judgements they have made. To assist their discussion, judges are given a statistical report showing how students at different levels of attainment in the course performed on each question in the examination.
The judges work through and discuss this information. Judges also receive a report showing how students responded to the various options of each multiple-choice question.
During this process judges can modify their decisions made during Stage 1. In this way the members of the team share their images of students at the borderlines between bands.
The judges’ recording sheets are collected and processed, resulting in a new set of band cut-off marks.
Stage 3
Judges review and discuss samples of student examination scripts awarded marks equal to or near each cut-off mark the judges are proposing.
The judges read and discuss these scripts with a view to determining whether or not these works are typical of what they would expect of students at that level. In essence, they need to satisfy themselves that the scripts scoring the cut-off marks can be said to match the standards demonstrated by those student works in the standards packages.
During this process each judge has the opportunity to further refine their band cut-off marks.
At the completion of this process, judges recommend to the HSC Consultative Committee a set of examination marks corresponding to the cut-offs between bands.
The Standards-Setting Process
Role of the Consultative Committee
The HSC Consultative Committee is appointed by the Board of Studies to review the outcomes of the standards-setting processes in each course. The committee, which has operated since the early 1980s, is an expert technical committee made up of leading professionals in educational measurement in NSW.
The role of the Consultative Committee in the new Higher School Certificate is to ensure the integrity of the standards-setting process and the integrity of the final decisions.
Following the standards-setting process, the Consultative Committee meets with the judges for each course, the chair of the Examination Committee and the Supervisor of Marking. The role of the committee is to:
- ensure through discussion with the judges that the standards-setting process followed by each team of judges was in accordance with the Board’s requirements
- identify any issues that may have impacted on the effectiveness of the standards-setting process
- receive from the judges their recommended band cut-off marks
- where necessary, make minor adjustments within the natural variability of judges’ decisions taking into account their consultation with the judges
- confirm the cut-off marks.
On behalf of the Board, the Consultative Committee also determines appropriate action if any anomalies in the examination processes are identified.
Finalising examination marks
Once the cut-off marks are approved by the HSC Consultative Committee, the cut-off examination mark between Band 5 and Band 6 is assigned a value of 90. The cut-off mark between Band 4 and Band 5 is assigned a value of 80; between Band 3 and Band 4, a value of 70; and so on. A raw mark of 100 is assigned a value of 100, and a mark of 0 a value of 0. Examination marks between the cut-off marks are then allocated in a linear manner.
This approach to assigning marks to HSC examination performances is very different from the past practice of assigning marks so that, in each course, the median mark was 60 and approximately the same percentage of students achieved the same marks (eg between 1% and 2% of students were assigned marks above 90 in each course).
Moderation of school assessment marks
The process used to moderate school assessment marks in the new HSC is very similar to the process used in previous years.
To ensure comparability of assessment marks from each school, assessment marks are moderated to match the performance of each school group on the common external examination using the raw examination marks obtained by the group.
The moderation process first sets the mean of a school’s assessment marks in a course equal to the mean examination mark of those students. The top assessment is set equal to the top examination mark and, where possible, the lowest assessment is set equal to the lowest examination mark for that school group.
Finally, the students’ moderated assessment marks are aligned to the performance scale using the same adjustments as for examination marks.
In all cases the assessment mark submitted to the Board will still need to reflect the rank order and relative differences between the achievement of students.
The HSC mark
The reported HSC mark is the average of the aligned school assessment mark and examination mark. It is reported together with the description of the standard achieved.
Percentages achieving standards
An obvious consequence of this changed basis for assigning marks is that there is no longer a reason for the percentage of students achieving particular marks to be the same from course to course. For example, the percentage of students scoring 90 or better in a course depends on the number of students who demonstrate achievements of the kind described by the highest standard (Band 6) for the course.
Differences across courses, and within a course from year to year, in the percentages of students achieving Band 6 are to be expected in a standards-referenced system. Achievement standards are developed independently for each course and reflect what subject experts consider to be desirable standards of achievement in that course against the standards required by the syllabus.
Amended Stage 6 Science syllabuses to be implemented for 2003 HSCThis replaces earlier advice in 2002 HSC Update, Newsletter 4 about implementation of the amended syllabuses. At its meeting on 17 September, the Board of Studies determined that the amended Stage 6 Science syllabuses should now be implemented simultaneously for the 2003 HSC cohort and the 2003 Preliminary cohort. The outcomes and content of the amended syllabuses will form the basis of the HSC examinations from 2003. To assist teachers in identifying the content that has been deleted both the amended syllabuses and a version of each syllabus showing the deleted text will be posted on the Board’s website in the week commencing 7 October 2002. The amended syllabuses will be reprinted and distributed to schools in early December 2002. |
Keeping you informed ...
This is the final newsletter in the series of 2002 HSC Updates. To access the complete set of newsletters for teachers and parents as well as information and links relating to the 2002 Higher School Certificate, go to the 2002 HSC Update page on the Board’s website: www.boardofstudies.nsw.edu.au/hsc
Previous 2002 HSC Update Newsletters
| Newsletter 1 |
|
| Newsletter 2 |
|
| Ancient History |
|
| Newsletter 3 |
|
| Newsletter 4 |
|
| Newsletter 5 |
|
Information for Parents and Carers
| Newsletter 1 |
|
| Newsletter 2 |
|
Further information about the 2002 Higher School Certificate examination program can be obtained by contacting Board of Studies Liaison Officers:
|
|
|
|
|
|
|
|
|
|