an article by Lucy Simpson (University of Bristol, UK) and Jo-Anne Baird (University of Oxford, UK) published in Oxford Review of Education Volume 39 Issue 1 (February 2013)
Abstract
Over recent years, the credibility of public examinations in England has increasingly come to the fore.
Government agencies have invested time and money into researching public perceptions of the reliability and validity of examinations. Whilst such research overlaps into the conceptual domain of trust, trust in examinations remains an elusive concept. Little is known about what it means to trust in examinations and on what individuals base their trust judgements.
This paper reports the findings of research into key stakeholders’ perceptions of trust in the English A-level examination system through a series of focus groups comprising (separately) of examination board staff, students, higher education admissions tutors, teachers and the general public.
Analysis of the data revealed a number of factors that are integral to the concept of trust in A-levels, including examination standards, the marking of examination papers, the syllabus, communication and provision of information, whether A-levels are fit for purpose and the mode of assessment.
The fact stakeholders have different expectations and requirements of the A-level system, which are subject to change, means that it is difficult for the A-level system to engender the trust of all of its users all of the time.
Showing posts with label examination_standards. Show all posts
Showing posts with label examination_standards. Show all posts
Monday, 8 April 2013
Wednesday, 14 November 2012
Setting and maintaining GCSE and GCE grading standards: the case for contextualised cohort-referencing
an article by Neil Simon Stringer (Centre for Education Research and Policy, Assessment and Qualifications Alliance (AQA), Guildford, UK) published in Research Papers in Education Volume 27 Issue 5 (November 2012)
Abstract
General Certificate of Secondary Education (GCSE) and General Certificate of Education (GCE) grading standards are determined by Awarding Bodies using procedures that adhere to the Code of Practice published by the regulator, Ofqual.
Grade boundary marks (cut scores) are set using subject experts’ (senior examiners’) judgement of the quality of candidates’ work, informed by statistics about the candidature and the mark distribution of the examination.
This model has been called weak criterion-referencing: the requirement of (strong) criterion-referencing for evidence of specific knowledge, skills and understanding is relaxed to allow for variations in examination difficulty, requiring maintenance of only the general quality of examination performance.
This paper considers the major conceptual flaw in this model – the fact that the examiners making the judgements have insufficient information to estimate quantitatively the relative difficulty of two successive years’ examinations – as well as evidence demonstrating that experienced examiners are unable to distinguish between candidates’ work within a small range of marks, as is required to set grade boundaries.
Furthermore, examiners appear biased toward giving candidates the benefit of doubt when deciding grade boundary marks.
Combined with their imprecision, this is a recipe for ‘grade inflation’ – the lowering of the quality of work required for a particular grade – of which the steadily increasing national GCSE and GCE results, and the consequent need to introduce grade As, could be a symptom.
It is proposed that a form of cohort-referencing that is sensitive to changes in examination entry patterns be introduced. The proposed system would shift the weight of evidence toward statistics, using qualitative judgements as part of a check on the veracity of the statistics, at the same time extinguishing the annual debate as to whether increasing examination outcomes are an indicator of improvement in education or a decline in examination ‘standards’, and safeguard against the need to introduce further higher grades in the future.
Abstract
General Certificate of Secondary Education (GCSE) and General Certificate of Education (GCE) grading standards are determined by Awarding Bodies using procedures that adhere to the Code of Practice published by the regulator, Ofqual.
Grade boundary marks (cut scores) are set using subject experts’ (senior examiners’) judgement of the quality of candidates’ work, informed by statistics about the candidature and the mark distribution of the examination.
This model has been called weak criterion-referencing: the requirement of (strong) criterion-referencing for evidence of specific knowledge, skills and understanding is relaxed to allow for variations in examination difficulty, requiring maintenance of only the general quality of examination performance.
This paper considers the major conceptual flaw in this model – the fact that the examiners making the judgements have insufficient information to estimate quantitatively the relative difficulty of two successive years’ examinations – as well as evidence demonstrating that experienced examiners are unable to distinguish between candidates’ work within a small range of marks, as is required to set grade boundaries.
Furthermore, examiners appear biased toward giving candidates the benefit of doubt when deciding grade boundary marks.
Combined with their imprecision, this is a recipe for ‘grade inflation’ – the lowering of the quality of work required for a particular grade – of which the steadily increasing national GCSE and GCE results, and the consequent need to introduce grade As, could be a symptom.
It is proposed that a form of cohort-referencing that is sensitive to changes in examination entry patterns be introduced. The proposed system would shift the weight of evidence toward statistics, using qualitative judgements as part of a check on the veracity of the statistics, at the same time extinguishing the annual debate as to whether increasing examination outcomes are an indicator of improvement in education or a decline in examination ‘standards’, and safeguard against the need to introduce further higher grades in the future.
Subscribe to:
Posts (Atom)
