Collusion Detection
Statistics of Dishonest Behaviour
MCQ [multiple-choice question] examinations continue to be popular for both formative and summative assessments. They may be rapidly, consistently and accurately marked electronically. Moreover, recent advances in computer, web and network technology make online and/or remote administering of MCQ exams an increasingly attractive and feasible option. In any event, some method for automatically detecting cheating would be valuable.
We have studied the behaviour of students sitting online MCQ examinations, both with and without invigilation [see Reference]. We developed a novel statistical method for comparing the answers submitted by pairs of students to detect suspicious correlations that were suggestive of collusion. It proved possible to corroborate such statistical evidence of cheating with additional temporal and spatial data, mined from server log files.
The Bayesian statistical framework employed is advantageous in that it naturally represents student behaviour in terms of best estimates for ability, question difficulty and [where appropriate] risk aversion. The interpretation of these parameters is particularly transparent and additionally provides information on the validity of the examination itself.
Further info
- Team leader: Ari Ercole [Research Associate]
- Team members: Dave Melvin [Postdoctoral Research Fellow], Kim Whittlestone
- Technologies: ASP [Active Server Pages], ODBC [Open Database Connectivity],
MS Access database - Methodologies: Bayes’ theorem, non-parametric statistical analysis
- Funding: SIFT [Service Increment For Teaching]
- Reference: A. Ercole, K.D. Whittlestone, D.G. Melvin and J. Rashbass [2002]
Collusion detection in multiple-choice examinations. Medical Education
36 [2], 166–173