T76 Metacognition in the Trenches

Monday, 16 October 2023
8:00 AM - 12:00 PM
328

SOME TYPES OF QUESTIONS ARE HARDER THAN OTHERS: ANALYZING STUDENT PERFORMANCE ON 1200 QUIZ QUESTIONS FOR AN INTRODUCTORY GEOSCIENCE COURSE

Paper number: T76-14
Program: Technical Sessions
Time: 11:10 AM – 11:25 AM

Authors

Abstract

Geoscience instructors routinely assign a variety of forced-choice questions for students as part of either formative assessments in class (e.g., Conceptests) or summative quizzes and/or exams. Relatively little research has been conducted to compare and contrast student success between different question types. We analyzed thousands of student answers to approximately 1200 questions using a quizzing tool (CLASS) that was deployed in a Physical Geology course. CLASS (Confidence-based Learning Accuracy Support System) allows us to measure both student performance (% correct responses) and student confidence in their answer choice. The questions included in this study were reviewed by colleagues for content and construct validity and edited accordingly before use.

We examined five different types of questions that were used frequently across a dozen quizzes. The question formats included: 1) True/False; 2) Multiple choice; 3) Multiple choice with an image; 4) Double True/False; and, 5) Select All. The average number of correct answers per question declined in the same order that the question formats are listed. Student performance was highest for True/False (averaging ~80% correct) and lowest for Select All questions (~60% correct). Scores for all question types ranged from ~20% to 100%. In all cases, students’ average confidence for each question type was greatest for higher scoring questions. However, the average student confidence per question type showed little difference despite the significant gap in performance. Students became progressively more overconfident as the difficulty of the question format increased. Several low scoring questions in the “Select All” category were edited, often changing format to “Select X” where X represented the number of correct responses. These changes resulted in an average improvement in scores (~9%) but no change in student confidence for the modified questions in a subsequent iteration of the course.

Geological Society of America Abstracts with Programs. Vol. 55, No. 6

doi: 10.1130/abs/2023AM-394789

© Copyright 2023 The Geological Society of America (GSA), all rights reserved. Permission is hereby granted to the author(s) of this abstract to reproduce and distribute it freely, for noncommercial purposes. Permission is hereby granted to any individual scientist to download a single copy of this electronic file and reproduce up to 20 paper copies for noncommercial purposes advancing science and education, including classroom use, providing all reproductions include the complete content shown here, including the author information. All other forms of reproduction and/or transmittal are prohibited without written permission from GSA Copyright Permissions.