Singular Choices for Multiple Choice

We revisit and evolve Frandsen and Schwartzbach's axiomatic scoring strategy for multiple-choice exams that credits partial knowledge and levels out guessing.
The evolved scoring strategy equalizes the implicit weight of each question by default and makes this weight an optional parameter for each question.
Partial credit can also be modulated, which provides a measure of the spread of knowledge of the examinee.
Based on a decade of experience in a first-year university course, we find the evolved exams to be more understandable and predictible both for the examiner and for the examinees.
Finally, we present a family of scoring functions that fit the model.

Lucy R. Betts, Tracey J. Elder, James Hartley, and Mark Trueman. Does correction for guessing reduce students’ performance on multiple-choice examinations? yes? no? sometimes? Assessment & Evaluation in Higher Education, 34(1):1–15, 2009.

Richard F. Burton. Multiple-choice and true/false tests: myths and misapprehensions. Assessment & Evaluation in Higher Education, 30(1):65–72, February 2005.

Richard F. Burton. Reducing the need for guesswork in multiple-choice tests. Assessment & Evaluation in Higher Education, 40(2):218–231, 2015.

Martin Bush. A multiple choice test that rewards partial knowledge. Journal of Further and Higher Education, 25(2):157–163, 2001.

Gudmund S. Frandsen and Michael I. Schwartzbach. A singular choice for multiple choice. ACM SIGCSE Bulletin, 32(4):34–38, December 2006.

Richard B. Gunderman and Joseph M. Ladowski. Inherent limitations of multiple-choice testing. Academic Radiology, 20(10):1319–1321, October 2013.

David R. Krathwohl. A revision of Bloom’s taxonomy: an overview. Theory into Practice, 41(4):212–218, 2002.

Ellen Lesage, Martin Valcke, and Elien Sabbe. Scoring methods for multiple choice assessment in higher education – is it still a matter of number right scoring or negative marking? Studies in Educational Evaluation, 39:188–193, 2013.

Simon McCallum. Game design for computer science education. In Erik Hjelmås, editor, Proceedings of the 2010 Norsk Informatikkonferanse (NIK 2010), Gjøvik, Norway, November 2010. Session 3: Pedagogikk og utdanning.

Lene Mejlby. Why are there so few female students in computer science? Master’s thesis, Department of Computer Science, Aarhus University, Aarhus, Denmark, June 2010.

Andrew Petersen, Michelle Craig, and Paul Denny. Employing multiple-answer multiple choice questions. In Janet Carter and Yvan Tupac, editors, ITiCSE ’16: Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, pages 252–253, Arequipa, Peru, July 2016. ACM.

Martin E. K. Rasmussen. A flat approach to syntax-checking, shuffling, and correcting multiple-choice tests. Master’s thesis, Department of Computer Science, Aarhus University, Aarhus, Denmark, June 2016.

Tim S. Roberts. The use of multiple choice tests for formative and summative assessment. In Denise Tolhurst and Samuel Mann, editors, Eighth Australasian Computing Education Conference (ACE 2006), volume 52 of Conferences in Research in Practice in Information Technology, pages 175–180, Hobart, Tasmania, Australia, January 2006. Australian Computer Society.

Michael Schwartzbach. Multiple choice tool. http://users-cs.au.dk/mis/Multiple/, November 2006.

Simon. Wrong is a relative concept: part marks for multiple-choice questions. In John Hamer and Michael de Raadt, editors, Thirteenth Australasian Computing Education Conference (ACE 2011), volume 114 of Conferences in Research in Practice in Information Technology, pages 47–53, Perth, Australia, 2011. Australian Computer Society.

Louis Leon Thurstone. A scoring method for mental tests. Psychological Bulletin, 17:235–240, 1919.

Jon Warwick, Martin Bush, and Sylvia Jennings. Analysis and evaluation of liberal (free-choice) multiple-choice tests. Innovation in Teaching and Learning in Information and Computer Sciences, 9(2):1–12, 2010.

Andriy Zapechelnyuk. An axiomatization of multiple-choice test scoring. Economics Letters, 132:24–27, 2015.