2002 HSC Update Newsletter 4
HSC Marks and the UAI Rank
Last year there were many questions about the relationship between New HSC marks and the Universities Admission Index (UAI) rank after students received UAI scores which they considered to be lower than expected, given their HSC marks.
There is, however, no direct relationship between HSC marks and the UAI. The reporting of Higher School Certificate achievement and the calculation of UAI rankings have two quite distinct purposes.
The reporting of achievement in the Higher School Certificate is the responsibility of the Board of Studies. Reported HSC marks tell us about the specific achievements of HSC candidates in each of their chosen courses.
The UAI is calculated by the universities’ Technical Committee on Scaling, independently of the Board of Studies. The UAI ranks HSC university applicants based on the statistical scaling of their raw examination marks and moderated assessment marks. This provides universities with a convenient and straightforward selection device for university entrance.
It is important that students understand that the UAI rank is not a mark or an average of their HSC marks.
The UAI does not claim to provide the sort of content information that can be obtained by a considered review of a student’s achievements in each course report.
HSC Reported Marks
HSC Course reports show achievement in relation to standards established for each course described on a performance scale from 0–100. The reports provide detailed information about the standard of knowledge, skills and understanding students have achieved.
The standard of achievement that is reported with a student’s HSC mark is determined through a standards setting process where students’ raw examination marks are aligned (or transferred) to the HSC performance scale. (See Figure 1.)
Experienced teacher judges make decisions about which raw marks equate with the cut-off points between each band on the performance scale.
Once a correspondence is established between particular raw marks and the cut-off points at 50, 60, 70, 80 and 90 on the performance scale, the full range of raw marks is then aligned (transferred) to the scale.
It is this aligned mark that is reported together with the description of the standard achieved. The reports also indicate a student’s statewide position in relation to the cohort for each course.
The proportion of students in bands in a course has no effect on the calculation of the UAI. Where, for example, there is a relatively low percentage of students placed in the higher bands in a course, this has no effect on the calculation of the UAI for students in that course.
Clear reporting
Figure 2 indicates the percentages of students who take recognised pathways following completion of the HSC.
Most Year 12 school leavers move into training and work. Typically around 30% of students enter university courses immediately on completion of Year 12.
Standards-referenced reporting sets out to provide users of HSC results – post-secondary education and training institutions, employers, teachers and students themselves – with useful information about achievement in specific courses.
The Universities Admission Index
The UAI is calculated from raw HSC examination marks and moderated assessment marks to rank students for university selection purposes. (See Figure 3.)
The universities ‘process’ the raw marks and moderated asssessment marks and rank students on a scale from 0–100 as if they have all been examined on the same group of courses.
Each student’s raw HSC marks for each course are scaled and then aggregated across ten units of study.
Students are then allocated a position in relation to others in the cohort – not the aggregate score itself. The position is reported in the form of a percentile rank referred to as the UAI.
The highest-ranking group of students receives a UAI of 100.00, the next ranking group 99.95, then 99.90, 99.85, 99.80 and so on.
The middle-ranking group receives a UAI of approximately 66.00 because the cohort also includes those students who did not complete Year 12. This explains why the middle-ranking students receive a UAI in the 60s – higher than the 50.00 that many would naturally assume.
Reported HSC marks and the UAI in 2001
In 2001 a number of students were disappointed at what appeared to be a discrepancy between their reported HSC mark and their UAI rank.
This was a direct effect of the shift to standards-referenced reporting of HSC marks while the UAI calculation process remained unchanged.
Under the former HSC, exam marks for middle-placed students were arranged around 60 on a fixed distribution scale – coincidentally a similar number to the middle-level UAI rank, which averaged around 66.
On the face of it there appeared to be some correspondence between the two scales. Middle-placed students with HSC marks in the 60s often achieved corresponding middle-level UAI ranks in the 60s.
Many students, teachers and parents continued to have this expectation of correspondence in 2001.
However, the removal in 2001 of the fixed distribution with its consequent ‘ceiling’ on HSC marks resulted in higher average marks in each New HSC course than previously. Students were awarded the marks that matched their performance.
Middle-placed students in 2001 achieved HSC exam marks in the 70–80 range in contrast to the previous median mark of 60.
Many middle-placed students in 2001 who achieved average HSC marks in the 70s expected to receive a corresponding UAI in the 70s.
However, despite their higher marks, these students still occupied middle-ranking positions in their courses. Consequently they generally received a middle-ranking UAI, which continues to be a ranking in the 60s.
The higher HSC marks achieved by middle-placed students in 2001 did not alter the fact that they were middle-placed students who generally speaking can expect a middle-ranking UAI.
Using HSC Reports
There are advantages for students, employers and further education and training authorities in having detailed information about achievement provided in HSC Course Reports.
HSC/TAFE Credit Transfer
TAFE NSW has already taken advantage of the fact that HSC results now provide information about the standard of performance at different band levels.
Through the HSC/TAFE Credit Transfer provisions negotiated between the Board of Studies and TAFE, HSC students are able to gain credit on the basis of having completed particular HSC courses and/or having reached specified performance bands in particular courses.
For example, students who are placed in Band 3 or above in the Geography HSC course can receive recognition for the TAFE modules Climate, Introduction to Soils, Maps and Mapping, Ecosystem Management and Investigating Ecosystems and thereby have credit in any of the TAFE courses in which these modules currently appear.
Many of the subjects offered in the HSC provide these opportunities for credit transfer into TAFE courses. The recently updated HSC/TAFE Credit Transfer website, www.det.nsw.edu.au/hsctafe, provides information about credit transfer opportunities.
University entrance
The UAI is used as the basis of selection into most university courses.
However, increasingly universities are including in their selection procedures other measures, prerequisites and expectations in addition to the UAI to screen students for entry.
For example, from 2005 the University of NSW will expect Band 4 or higher achievement in HSC courses listed as Assumed Knowledge for course entry.
Where students who have achieved at a high level at the HSC may be seeking advanced standing at a university, the Course Report can be used to provide details of student achievement through the band descriptions.
It is also increasingly the case that interviews, portfolios, principals’ recommendations and special tests where appropriate are used to make decisions about course entry.
In this context students can draw on and highlight the knowledge, skills and understanding provided in their course reports to support and supplement their case for admission.
Employers
HSC Course Reports provide clear descriptions of what students know, understand and can do. The descriptions can be used by employers to help judge how well applicants may be suited to particular job requirements.
In addition, the course reports provide explicit statewide standards which assist in making comparisons between prospective employees.
Students preparing applications for employment can draw from the band descriptions of their standard of achievement to address the particular skills and knowledge demands as well as to describe their overall abilities.
Year 12 Advisers and Careers AdvisersTo support discussions with Year 12 students about the different nature and purposes of the HSC reported marks and the UAI rank a set of PowerPoint slides is available on the Board’s website at www.boardofstudies.nsw.edu.au/hsc/ |
In Brief: |
HSC Standards PackagesThe Board of Studies will produce a full set of standards
packages following this year’s HSC. |
Stage 6 Science syllabusesAmended Stage 6 Science syllabuses will be available to schools
at the beginning of Term 1, 2003 for implementation of the
Preliminary course in 2003 and the HSC course in 2004. |
Cognitive ScienceThe timeline for the development and implementation of the Board Developed (Year 12 only) Cognitive Science course has been revised as follows:
|
School-based assessment for 2003For 2003 schools are advised to continue to use the full range of marks from 0–100 when providing school-based assessment marks to the Board. Final assessment marks reported to the Board should not be adjusted to conform to a 50–100 scale. |
2001 HSC Courses With Relatively Low Percentage of Students in Bands
5 and 6
In a standards-referenced system there is no reason why the percentages of students achieving each standard should be the same across courses.
When discussing the proportion of students in the higher bands for courses it is important to remember that no student’s UAI is at all affected by a course having a relatively low percentage of students placed in the higher bands. The UAI is a number between 0 and 100 that shows the student’s rank in the state. The universities continue to base the UAI on the raw examination marks and the school assessment marks and use their traditional scaling process for each subject to achieve a single rank based on the student’s best performance in ten units.
In Table 5.1 of his report on the 2001 HSC examination program Professor Geoff Masters listed eight courses which the Board could decide to look at more closely as perhaps having unusually low percentages of students in Bands 5 and 6. These courses were Studies of Religion I and II, Chemistry, Physics, Engineering Studies, Software Design and Development, Legal Studies and English (Advanced).
The Board adopted his recommendation and set up a project to help it understand possible reasons for the lower band proportions in these courses. A separate project is looking at English (Advanced).
In undertaking the project the Board was assisted by six teachers for each course. The teachers formed the view that more students should have been awarded Band 6 in their course.
They then worked with Board officers to explore the various reasons as to why the proportions might have been lower than they might have expected.
Performance descriptions
Two sets of performance descriptions were considered not to be appropriate to the higher ranges. As a result the performance descriptions for Software Design and Development will be reworked and workshopped prior to their use in this year’s standards setting process.
As well, descriptions related to communications skills will be added to the Engineering Studies performance descriptions for this year’s judging so that judges are more able to delineate between bands and reward higher achievement.
Syllabuses
While syllabus-related factors were not considered by the group to be as significant as other factors, it was recommended that the Board investigate a possible review of the Studies of Religion syllabus. The Board’s decision to reduce the level of content in Physics and Chemistry syllabuses was noted with approval.
Examination setting
It was felt that the examination questions might not have provided enough opportunities for students to demonstrate higher band characteristics. Also, in the first year of the New HSC, students were not necessarily familiar with the nature of responses expected for various types of questions and the papers could have provided more guidance in this regard. Masters made several recommendations about improvements to the process of setting HSC examination papers and all of these were put in place throughout the setting of all 2002 HSC examination papers.
Marking
The marking process was identified as a likely contributing factor. The group felt that in some instances the marking guidelines and their application did not award the range of marks that matched the quality of responses, perhaps due to some overly-rigid application of the marking guidelines or else because the guidelines were not fully consistent with the question.
Once again, significant steps have been taken this year to address these issues in the marking process. In particular, the Examination Committee Chair will be fully involved in the pilot marking phase and any changes necessary to the marking guidelines following pilot marking will be made at the marking centre by the Exam Chair and the Supervisor of Marking.
Standards setting
It was suggested that there should be more consultation between the judges and key marking personnel about the responses that questions were actually eliciting during marking and issues that were arising in the marking process. This would help judges to be fully aware of aspects of the performance scales and the aspects that would be best shown in a written examination. These views are being incorporated into the judging process for 2002.
Each of the adjustments to the 2002 HSC process is expected to assist students to be appropriately recognised for performance at the highest standards. It is also likely that the impact of many issues will diminish as students and teachers become familiar with the New HSC and as ongoing developments, including those associated with the Masters Review, are embedded into processes.