Showing posts with label assessment. Show all posts
Showing posts with label assessment. Show all posts

Sunday, 11 February 2018

Can markers detect contract cheating? Results from a pilot study

an article by Phillip Dawson (Deakin University, Geelong, Australia) and Wendy Sutherland-Smith (Deakin University, Burwood, Australia) published in Assessment & Evaluation in Higher Education Volume 43 Issue 2 (2018)

Abstract

Contract cheating is the purchasing of custom-made university assignments with the intention of submitting them. Websites providing contract cheating services often claim this form of cheating is undetectable, and no published research has examined this claim.

This paper documents a pilot study where markers were paid to mark a mixture of real student work and contract cheating assignments, to establish their accuracy at detecting contract cheating.

Seven experienced markers individually blind marked the same bundle of 20 second-year psychology assignments, which included 6 that were purchased from contract cheating websites.

Sensitivity analyses showed markers detected contract cheating 62% of the time. Specificity analyses showed markers correctly identified real student work 96% of the time.

Our results contrast with contract cheating sites’ claims that contract cheating is undetectable. However, they should be taken with caution as they are from one course unit in one discipline.


Thursday, 28 May 2015

External examining: fit for purpose?

an article by Sue Bloxham (University of Cumbria) and Margaret Price (Oxford Brookes University) published in Studies in Higher Education Volume 40 Issue 2 (March 2015)

Abstract

In a context of international concern about academic standards, the practice of external examining is widely admired for its role in defending standards. Yet a contradiction exists between this faith in examining and continuing concerns about standards.

This article argues that external examining rests on assumptions about standards which are significantly open to challenge.

Six assumptions relating to the conceptual context, the operation and the nature of examiners themselves are analysed drawing on a review of the available evidence. The analysis challenges the notion of a consensus on standards and the potential to vest in individuals the ability to represent that consensus when judging the comparability of academic standards in a stable and appropriate way.

The issues raised have relevance to the UK and to other national systems using external examiners or seeking to guarantee academic standards by, in some cases, adopting quality assurance approaches developed in the UK.


Monday, 8 July 2013

The literacy practices for assessment in the vocational curriculum – the case of Hospitality

an article by Richard Edwards and Sarah Minty (University of Stirling, UK) and Kate Miller (University of West of Scotland, Ayr, UK) published in Journal of Vocational Education & Training Volume 65 Issue 2 (June 2013)

Abstract

This article explores two case studies of the literacy practices of assessment in the vocational curriculum. Previous studies have identified learning in the vocational curriculum as being assessment-driven and that subjects often associated with limited levels of literacy actually require multiple literacy practices from students.

Drawing upon studies of the literacy practices of assessment in the teaching of Hospitality, the study explores the academic and occupational forms of reading and writing required of students.

These literacy practices are often invisible to both students and staff and yet are central to the success of students in relation to attainment.

The article suggests that vocational subjects may actually be more demanding in the range of literacies required for assessment than is often considered to be the case.


Wednesday, 3 July 2013

The diploma qualification in England: an avoidable failure?

an article by Tina Isaacs (Institute of Education, Department of Curriculum, Pedagogy and Assessment, London, UK) published in Journal of Vocational Education & Training Volume 65 Issue 2 (June 2013)

Abstract

In 2005, the UK government introduced new vocationally-related diploma qualifications.

Two-year programmes were examined in the summers of 2010 and 2011; initial government hopes for candidate numbers proved largely ephemeral and results were disappointing.

This article explores what happened to the diploma’s lofty ambitions using the summer of 2010 and 2011 results and a 2006 government policy analysis that went largely unheeded, in order to draw out lessons from this centrally developed programme.

Critical insights from earlier vocationally related, centrally mandated qualifications such as General National Vocational Qualifications were generally ignored in policy-makers’ misguided attempts to create something ‘new’. The analysis centres on the neglected aspects of the qualification’s complex structure, hurdles-based assessment model and insistence that candidates had to pass Functional Skills qualifications, all of which were major contributors to its demise in the autumn of 2011.


Tuesday, 21 May 2013

Early intervention to support the academic recovery of first-year students at risk of non-continuation

an article by Alf Lizzio and Keithia Wilson (Griffith Institute for Higher Education, Queensland, Australia) published in Innovations in Education and Teaching International Volume 50 Issue 2 (May 2013)

Abstract

The widening participation agenda and related concerns about student retention require a more systematic focus on supporting student success.

This paper describes a process designed with the dual goals of supporting the short-term academic recovery of students at risk of non-continuation due to early difficulties with assessment and developing their ongoing capabilities for self-regulation around challenging assessment tasks.

Commencing students who failed or marginally passed their first piece of university assessment were invited to participate in a two-stage process: independently completing a reflective workbook designed to help them understand the reasons for their assessment performance, followed by a structured consultation with their tutor to identify improvement goals and strategies.

Students undertaking the academic recovery process achieved higher pass rates for the second assessment item and for the course overall than a comparative group of students who did not participate.

Findings indicate that a self-regulation-based intervention can contribute to students’ academic persistence and success. Importantly, students experienced the intervention as providing insight into their underperformance on assessment and developing their capacity for meta-learning.


Monday, 4 March 2013

Qualification users’ perceptions and experiences of assessment reliability

an article by Suzanne Chamberlain (Assessment & Qualifications Alliance, Manchester, UK) published in Research Papers in Education Volume 28 Issue 1 (February 2013)

Abstract

This paper presents the findings of a study designed to explore qualification users’ perceptions and experiences of reliability in the context of national assessment outcomes in England. The study consisted of 17 focus groups conducted across six sectors of qualification users:
  • students,
  • teachers,
  • trainee teachers,
  • job-seekers,
  • employers, and
  • employees.
Each qualification user group discussed a series of issues concerned with the reliability of grades awarded at the completion of compulsory (age 16) and post-compulsory (age 18) education.

Findings suggested that there were some notable differences between the participating groups in relation to their experiences of assessment reliability and their inclinations to trust the reliability of assessment outcomes. For example, teachers shared a number of experiences of suspected and actual error in the production of students’ grades, and of securing subsequent grade changes for their students.

In the light of these experiences, teachers were disinclined to accept at face value the reliability of assessment outcomes. Among other groups, assessment outcomes were generally regarded as trustworthy and reliable measures of academic ability.

However, without access to information about assessment reliability, participants were unaware that reliability can be undermined by random measurement error. Participants discussed measurement errors in relation to student well-being, facets of the test and the test situation and environment, but such errors were deemed to be an inevitable part of test-taking and as having limited impact on the reliability of assessment outcomes.

It is concluded that efforts to enhance public understanding of assessment reliability, therefore, may be complicated by tensions between traditional, specialist conceptions of assessment reliability and what is perceived as relevant and useful by qualification users.


Wednesday, 19 December 2012

Qualifications, examinations and assessment: views and perspectives of students in the 14–19 phase on policy and practice

an article by Jannette Elwood (Queen’s University Belfast, Northern Ireland) published in Cambridge Journal of Education Volume 42 Issue 4 (December 2012)

Abstract

This paper brings to the forefront students’ views on one of the most significant aspects of educational reform in the 14–19 phase in England, specifically qualifications, examinations and assessment reform.

In this respect, the paper foregrounds students as ‘policy actors’, they are significant players in the mediation of national qualification systems rather than just subjects in their implementation. Data from a national dataset of focus groups with 243 students are presented.

Key themes are highlighted relating to young people’s experiences of qualifications, examinations and assessment at this stage of education in a context of continuous initiatives and change as well as the impact on students of qualifications reform in situ which can be confusing, unsettling and ultimately detrimental to future success.


Monday, 26 November 2012

The Career and Work Adaptability Questionnaire (CWAQ): A first contribution to its validation

an article by Laura Nota, Salvatore Soresi and Maria Cristina Ginevra (University of Padua, Italy) published in Journal of Adolescence Volume 35 Issue 6 (December 2012)

Abstract

Over the last decade, occupational changes have the rapidly changing job market has begun to demand that people more actively construct their professional lives and acquire career adaptability.

The aim of the present study was to develop a specific, new instrument, “Career and Work Adaptability”, to assess degree of adaptability in adolescents planning their futures.

We conducted three studies, the first of which aimed to formulate the instrument’s items and to verify its factor structure; the second study confirmed the instrument’s multidimensional structure and evaluated its discriminant validity; the third study was conducted to verify the factorial structure’s across-gender invariance and to evaluate its stability over time.

Our results showed that the instrument is an effective and multidimensional instrument for accurately measuring career adaptability. Specifically, it can serve as a useful vocational guidance tool in analysing adolescents’ career adaptability.


Tuesday, 20 November 2012

Career Thoughts, Indecision, and Depression: Implications for Mental Health Assessment in Career Counseling

an article by Jerry V. Walker III and Gary W. Peterson (Florida State University, Tallahassee, USA) published in Journal of Career Assessment Volume 20 Number 4 (November 2012)

Abstract

This study investigated the relationships among dysfunctional career thoughts and career indecision with respect to symptoms of depression.

Such information could be useful to counsellors in identifying individuals at the outset of career counselling who may be experiencing emotional distress from life stressors in addition to career stress.

One-hundred fifty-eight college students enrolled in a career development course completed measures of dysfunctional career thoughts, an occupational alternative question, and a measure of depression symptoms.

Results indicated that dysfunctional career thoughts and occupational indecision were related to depression symptoms, with decision-making confusion being the best predictor.

Implications of the findings for practice and research are discussed.


Wednesday, 14 November 2012

Career Decision-Making Profiles of Italian Adolescents

Maria Cristina Ginevra (University of Padua, Italy), Laura Nota and Salvatore Soresi (University of Padova, Padua, Italy) and Itamar Gati (Hebrew University of Jerusalem, Israel) published in Journal of Career Assessment Volume 20 Number 4 (November 2012)

Abstract

The goal of the present study was to test the psychometric properties of the Italian version of the Career Decision-Making Profile (CDMP) questionnaire with a sample of 1,835 adolescents.

Gati, Landman, Davidovitch, Asulin-Peretz, and Gadassi suggested that the way individuals make career decisions should not be described by a single decision-making style but rather by a multidimensional profile based on a consideration of 11 dimensions.

The results showed that the Italian version of the CDMP has adequate psychometric properties and structural validity.

As hypothesised, the scores of the Problem-Solving Inventory were correlated with the information-related dimensions of the CDMP. Decided adolescents had more adaptive CDMP profiles than undecided adolescents, supporting the concurrent validity of the CDMP.

Female adolescents were more likely to consult with and depend on others, invest greater effort, and, consequently, take more time to make a decision.

Theoretical and counselling implications are discussed.


Monday, 3 September 2012

Eligibility for statutory learning disability services in the north-west of England. Right or luxury? Findings from a pilot study

an article by Erica E McInnis (NHS Manchester, Manchester Community Health, Manchester City Council), Alan Hills (Pennine Care (Stockport), Community Learning Disability Team) and Melanie J Chapman (Manchester Learning Disability Partnership) published in British Journal of Learning Disabilities Volume 40 Issue 3 (September 2012)

Accessible summary
  • There are often rules about who can use learning disability services.
  • We used a questionnaire to ask clinical psychologists in the north-west of England about the rules where they work. We also asked them what psychologists do when it comes to deciding about who can use learning disability services.
  • We found out that different places had some things in common in how they defined learning disabilities and how they decided who used services. However, there were also differences between different places.
  • Differences between different areas can make it difficult for people with learning disabilities if they move.
Summary

Access to learning disability services in England is often governed by eligibility criteria. A semistructured questionnaire was completed by clinical psychologists in the north-west of England about service eligibility criteria and psychologists’ role within the referral process to learning disability services.

The survey highlighted both variations and similarities across the region in terms of definitions of learning disabilities, eligibility for accessing local services and the psychometric measures used to assess functioning.

Differences in how services assess eligibility can lead to difficulties, for example when people move area. The use of standard eligibility criteria could be one means of overcoming this.


Wednesday, 22 August 2012

Critical thinking in E-learning environments

an article by Raafat George Saadé and Danielle Morin (Concordia University, John Molson School of Business, Montreal, Canada) and Jennifer D.E. Thomas (Pace University, Ivan Seidenberg School of CSIS, New York) published in Computers in Human Behavior Volume 28 Issue 5 (September 2012)

Abstract

One of the primary aims of higher education in today’s information technology enabled classroom is to make students more active in the learning process.

The intended outcome of this increased IT-facilitated student engagement is to foster important skills such as critical thinking used in both academia and workplace environments.

Critical thinking (CT) skills entails the ability(ies) of mental processes of discernment, analysis and evaluation to achieve a logical understanding. Critical thinking in the classroom as well as in the workplace is a central theme; however, with the dramatic increase of IT usage the mechanisms by which critical thinking is fostered and used has changed.

This article presents the work and results of critical thinking in a virtual learning environment.

We therefore present a web-based course and we assess in which parts of the course, and to what extent, critical thinking was perceived to occur. The course contained two categories of learning modules namely resources and interactive components. Critical thinking was measured subjectively using the ART scale. Results indicate the significance of “interactivity” in what students perceived to be critical-thinking-oriented versus online material as a resource.

Results and opportunities that virtual environments present to foster critical thinking are discussed.

Highlights

► We investigated critical thinking in an online learning context (higher education).
► We examined the kinds of resources and activities that fosters/require critical thinking skills.
► Online interactive learning components are perceived to foster critical thinking.
► Design factors of interactive learning environments need to be addressed in online courses.
► Designers and teachers need to integrate more interactive components into the course activities.


Tuesday, 17 April 2012

Ebook collection analysis: subject and publisher trends

an article by James Cory Tucker, (University of Nevada, Las Vegas) published in Collection Building Volume 31 Issue 2 (2012)

Abstract

Purpose
This paper aims to provide an assessment of an ebook collection in an academic library, and attempts to locate usage trends by subject and publisher.
Design/methodology/approach
The research was based on: three years of usage data from two e-book packages: NetLibrary and Ebrary; two methods of purchase: NetLibrary was a one-time purchase; Ebrary is purchased on a subscription basis. The research evaluated usage difference over time between the two packages and analysed subject and publisher usage.
Findings
The research found that Ebrary showed increased usage over time; NetLibrary demonstrated decreased usage; subject analysis showed ebooks in health sciences and hotel had highest usage; publisher analysis results illustrated the fact that five publishers had highest usage in both ebook collections.
Research limitations/implications
For circulation rate of each ebook package, two years of data were not available for Ebrary, resulting in incomplete comparison over three-year period between the two packages.
Practical implications
The paper assists in identifying usage patterns of ebooks across publishers and subjects; compares two different business models of obtaining ebooks; and helps with effective selection of ebooks to support teaching and learning.
Originality/value
Usage data over three years provided evidence to help libraries select a business model for acquiring ebooks; the research provides assessment of ebook collections to identify trends across publishers and subjects.


Wednesday, 14 March 2012

Developing a collaborative faculty-librarian information literacy assessment project

an article by Jackie Belanger and Rebecca Bliquez, (Campus Library, University of Washington Bothell and Cascadia Community College, USA) and Sharleen Mondal, (Department of English, Ashland University, Ohio) published in Library Review Volume 61 Issue 2 (2012)

Abstract

Purpose
The purpose of this paper is to describe the process of developing an information literacy assessment project, and to discuss key findings from the project.
Design/methodology/approach
A variety of assessment tools were used to gather information about student learning and information literacy instruction: pre- and post-surveys, student feedback surveys, faculty feedback to librarians, librarian self-reflection, library worksheets, student research journals, and citation analysis of students’ final research paper bibliographies.
Findings
It was found that the authors’ initial suite of assessment tools did not provide the information wanted about students’ research processes, so the authors’ “assessment toolkit” was modified. It was found that more meaningful information could be gathered about students’ research processes when the authors worked closely with faculty to embed information literacy assessments into course assignments. From the authors’ analysis of student work, it was discovered that, for many students, library instruction was most valuable in helping them refine and explore research topics.
Originality/value
This paper will be useful to librarians and faculty seeking to implement an information literacy assessment project. The authors provide ideas for ways for faculty and librarians to collaborate on information literacy assessment, as well as on assignment and course design.


Monday, 21 November 2011

The assessment of enterprise education in the secondary education sector: …

A new approach?

an article by Matthew C. Draycott, (Glyndwr University, Wrexham), David Rae, (Lincoln Business School, University of Lincoln) and Katie Vause, (Lincoln Castle Academy, Lincoln) published in Education + Training Volume 53 Issue 8/9 (2011)

Abstract

Purpose
Although the assessment of enterprise education activities has been widely highlighted as a key area of concern, it continues to be under represented in the literature. Questions remain as to how educators seeking to monitor student progression can capture quality data and measure relevant aspects of development, often leading enterprise education to be monitored rather than assessed. This article seeks to explore the challenges of assessing enterprise education in the secondary education sector. It aims to provide useful insights to help practitioners understand how to evidence the impact of enterprise learning by students.
Design/methodology/approach
The paper first presents a critical review of the existing literature with insights from specialist practitioners sourced through an online survey and a seminar. This provides a broad review of the field from a practitioner standpoint focusing on current assessment techniques and standards. Using these data a conceptual pedagogy is proposed for the delivery of enterprise education and a methodology for its assessment, to be developed in future work.
Findings
A critical review of the assessment of enterprise education is presented. This exposes challenges of a confused field, with pockets of good practice in schools often not shared or understood out of context. The development of a novel pedagogical model for teaching enterprise education is proposed, linked to a prototype assessment methodology which presents a new approach for enterprise teaching and learning.
Research limitations/implications
The work is limited at this stage since participants in the research were drawn from one geographic area in the East of England, and examples of qualifications reviewed were not exhaustive, but these limitations can be addressed in future research.
Practical implications
The paper provides a conceptual model for structuring enterprise education which may have relevance across the secondary sector and beyond.
Originality/value
The article investigates the problems of assessing enterprise in secondary education, examining what does and does not work, and providing practitioners with useful guidance. In this important topic it is vital that new approaches are developed which can create a broader debate especially at a time of such great change in the educational landscape. This paper provides a platform for further development in the field.


Tuesday, 18 October 2011

Academic libraries – measuring up: …

assessment and collaboration for student success

 an article by Rachel Besara and Kirsten Kinsley (Florida State University Libraries) published in New Library World Volume 112 Issue 9/10 (2011)

Abstract

Purpose
This paper aims to describe how the Florida State University Libraries used assessment data with other campus partners to gain funding and resources for new initiatives. When general funding sources were threatened, alternative funding sources from these campus partners were used to jump-start new initiatives designed to enhance student success.
Design/methodology/approach
This paper is a case study of how assessment data fuelled the creation of a new late-night peer-tutoring program at the Florida State University Libraries. The three main data conduits that inspired a new tutoring program were: an ethnographic study of undergraduate students, undergraduate courses with high failure/high enrolment/high drop rates, and an environmental scan of existing campus tutoring.
Findings
Sharing assessment data with key partners can leverage funding and resources for new initiatives.
Social implications
In hard budgetary times, opportunities for funding and resources may arise when shared values between campus constituencies are met with assessment data. Libraries need to take a leadership role in gathering and sharing those data with other campus constituents in order to place libraries in a strategic position to receive alternative funding for shared initiatives.
Originality/value
Other libraries may use this case as a model, sharing their assessment results with the campus community, especially with those campus constituencies where there is a relationship already in place, to garner further support for piloting innovative services.