T183 Measuring Learning in Geoscience Education

Tuesday, 12 October 2021
1:30 PM - 5:30 PM
B113/B114 (Hybrid Room)

ASSESSING UNDERGRADUATE RESEARCH EXPERIENCES USING KNOWLEDGE SURVEYS

Paper number: T183-13
Program: Technical Sessions
Time: 4:15 PM – 4:30 PM

Authors

Abstract

Undergraduate research (UR) experiences are widely valued for recruiting and developing a vigorous and diverse STEM workforce. Program assessments commonly employ participant surveys due to their ease of use in UR settings. However, such survey results frequently rely on generalized descriptions of learning tasks, ambiguous or unequal Likert scales, and self-reports of learning gains. Here, we describe an adaptation of the well-studied knowledge survey instrument for assessing UR. This instrument is easy to administer and adapt for different purposes and is ideally suited for assessing UR experiences.

Our knowledge survey addresses twenty-two knowledge and skill outcomes central to UR. Each item begins with the root “I can” followed by a detailed description of a research-related task (e.g., maintaining field notebooks, reading scientific articles, preparing poster presentations). Students respond to the items using four Likert-scale options that describe their ability to complete the task and their understanding of professional level knowledge. The format and structure of the knowledge survey offers the context of an exam that promotes more informed and intentional self-assessment.

We used the knowledge survey as a pre/post measure in two different research experiences for undergraduates (REU). Prior to UR, rising sophomores (Gateway REU) reported lower mean confidence (1.9 on scale of 1-4) to complete the tasks in the research knowledge and skills sub-construct compared with Advanced REU rising seniors (2.6). After the summer field experience, mean responses rose to 2.8 and 3.2, respectively, with both cohorts reporting significant gains (Cohen’s d = 1.70 and 1.50, respectively). After attending a professional conference, the mean responses increased again (x = 3.1 and 3.3, respectively); the effect sizes for those in the Gateway (Cohen’s d = 1.96) were higher than the Advanced REU (Cohen’s d = 1.64). Control groups, consisting of Year 1-2 and Year 3-4 geoscience majors, had mean responses (2.0 and 2.6, respectively; Cohen’s d = 1.29) similar to the pre-REU Gateway and Advanced cohorts, respectively. These results confirm the value of UR experiences, document the powerful impacts these can have on younger students, and demonstrate the promise of knowledge surveys for assessing undergraduate research.

Geological Society of America Abstracts with Programs. Vol. 53, No. 6

doi: 10.1130/abs/2021AM-369940

© Copyright 2021 The Geological Society of America (GSA), all rights reserved. Permission is hereby granted to the author(s) of this abstract to reproduce and distribute it freely, for noncommercial purposes. Permission is hereby granted to any individual scientist to download a single copy of this electronic file and reproduce up to 20 paper copies for noncommercial purposes advancing science and education, including classroom use, providing all reproductions include the complete content shown here, including the author information. All other forms of reproduction and/or transmittal are prohibited without written permission from GSA Copyright Permissions.