T97 Methodological Decision Making in Geoscience Education Research

Monday, 26 September 2016
8:00 AM - 12:00 PM
Mile High Ballroom 4EF

HOW IMPORTANT IS THAT STATISTICALLY SIGNIFICANT RESULT? BEING MINDFUL OF P-VALUES IN SCIENCE EDUCATION RESEARCH

Paper number: T97-12
Program: Technical Sessions
Time: 10:45 AM – 11:00 AM

Authors

Abstract

Discipline based education research projects, especially within the STEM fields, have evolved from studies within a single classroom to large efforts that can involve multiple semesters, faculty and even institutions. With the increase in the size and duration of educational studies and the increasing complexity of the resulting datasets, many science education researchers have turned to quantitative and mixed methods for data analysis and interpretation. Moreover the call for reproducible results and educational decisions to be made using evidence based and validated best practices have added to the need for proof that science education efforts are effective. The use of p-values to validate ideas and research through null hypothesis significance testing (NHST) has become prevelant in education research, impacting interpretation and even if some results are deemed publishable.

However the concept of the p-value, and how to properly use and interpret them, has been highly contested within statistics and educational research for over a hundred years. Within the past five year alone multiple high profile stories from biomedical to political science have highlighted studies that are significant using NHST, yet the interpretations are faulty in context, p-values can change when studies are reproduced, and others are “p-hacked” into getting what are viewed as statistically significant results. These issues with NHST are also found within discipline based education research, especially as each discipline has its own statistical culture. This does not mean that NHST is a faulty approach that should be avoided. Instead discipline education researchers need to be more mindful about the design of the project, the specific hypothesis and/or statistical model being tested, wary of ad hoc multiple testing, incorporate other statistics such as effect size and confidence intervals to support the effectiveness of their study, and not overemphasize or over interpret p-values.

Geological Society of America Abstracts with Programs. Vol. 48, No. 7

doi: 10.1130/abs/2016AM-287894

© Copyright 2016 The Geological Society of America (GSA), all rights reserved. Permission is hereby granted to the author(s) of this abstract to reproduce and distribute it freely, for noncommercial purposes. Permission is hereby granted to any individual scientist to download a single copy of this electronic file and reproduce up to 20 paper copies for noncommercial purposes advancing science and education, including classroom use, providing all reproductions include the complete content shown here, including the author information. All other forms of reproduction and/or transmittal are prohibited without written permission from GSA Copyright Permissions.