Problems in estimating composite reliability of ‘unitised’ assessments

an article by Tom Bramley and Vikas Dhawan (ARD Research Division, Cambridge Assessment, Cambridge, UK) published in Research Papers in Education Volume 28 Issue 1 (February 2013)

Abstract

This paper discusses the issues involved in calculating indices of composite reliability for ‘modular’ or ‘unitised’ assessments of the kind used in GCSEs, AS- and A-level examinations in England. The increasingly widespread use of on-screen marking has meant that the item-level data required for calculating indices of reliability is now routinely available for most (but not all) units of unitised assessments.

Whilst it is relatively straightforward to obtain indices of reliability at unit level, it is far more complex to obtain indices at overall assessment level because of problems created by:

1. the number of different possible ‘routes’ to the final assessment;
2. the different knowledge, skills and understanding assessed in different units;
3. the wide variety in item type and size within and across units;
4. the fact that the item-level data required for calculating reliability indices is not available (or does not exist) for certain units; and
5. the different intended weighting of different units in the composite total and the possible distortion of these weights by use of the Uniform Mark Scale.

We derive and compare several indices of composite reliability for, as an example, a three-unit assessment using both classical test theory and item response theory. We conclude that while it is desirable to derive and report indices of reliability at unit level, it is less appropriate at the level of the whole assessment.

---

Gender, parental education, and ability: their interacting roles in predicting GCSE success

an article by Judith Glaesser and Barry Cooper (Durham University, UK) published in Cambridge Journal of Education Volume 42 Issue 4 (December 2012)

Abstract

We investigate the relations between gender, parental education, ability, and educational achievement in Britain, focusing on the way in which gender and parental education interact with ability to contribute to a pupil’s obtaining secondary school qualifications. This allows us to provide evidence relevant to claims concerning the effects of differences in the way in which working- and middle-class familial cultures interact with gender-specific behaviour in school.

Given the configurational nature of the processes likely to be involved, we employ Ragin’s Qualitative Comparative Analysis as our method.

We find that, in both academically selective and non-selective schools, high ability is a quasi-sufficient condition for obtaining certain levels of qualification, but that at lower levels of ability, either being female or having highly educated parents (or both) have to be present, too.

Boys without highly educated parents perform less well than girls from a similar background.

---

Subject progression from GCSE to AS Level and continuation to A Level

a research report (Reference: DFE-RR195)by Education Standards Analysis and Research Division Published: February 2012

Summary

The aim of this topic paper is to provide an analysis of the proportion of pupils taking GCSE and progressing to AS Level in the same subject (i.e. progression rates), focusing on English Baccalaureate subject areas, and then to follow the same cohort through from AS to full A Level achievement.

Data from the National Pupil Database was used to produce descriptive analyses regarding progression rates from GCSE (grades A*-C individually) to AS level for the cohort who completed Key Stage 4 in 2008 for a set of chosen subjects. Subjects assessed are biology, chemistry, physics, maths, English language, history, geography, French, German and Spanish. The publication also shows progression by KS4 institution type (independent schools, grammar schools, academies etc.).

The analysis then follows this cohort through to look at the progression rates for AS to A level and for GCSE to A Level (again by subject and institution type), and examines progression rates directly from GCSE to A Level by GCSE grade, including a comparison of GCSE grade to A Level grade achieved.

Includes:

• Executive summary
• Introduction
• Chapter 1: GCSE to AS level progression
• Chapter 2: AS to A level progression
• Chapter 3: GCSE to A Level progression
• Further work
• Data and methodology

Full report (PDF 36pp)

---

Kids with no qualifications need more help says charity

via WalesOnline - News - Education News on 20 August

While many pupils will celebrate record-breaking achievements in their GCSE results today a number of young people will be leaving the education system with nothing.

Read the full story

---