A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science
Authors:
Clayton Cohn,
Nicole Hutchins,
Tuan Le,
Gautam Biswas
Abstract:
This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning w…
▽ More
This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning with chain-of-thought reasoning. Using a human-in-the-loop approach, we successfully score and provide meaningful explanations for formative assessment responses. A systematic analysis of our method's pros and cons sheds light on the potential for human-in-the-loop techniques to enhance automated grading for open-ended science assessments.
△ Less
Submitted 21 March, 2024;
originally announced March 2024.
Identifying regions of importance in wall-bounded turbulence through explainable deep learning
Authors:
Andres Cremades,
Sergio Hoyas,
Rahul Deshpande,
Pedro Quintero,
Martin Lellep,
Will Junghoon Lee,
Jason Monty,
Nicholas Hutchins,
Moritz Linkmann,
Ivan Marusic,
Ricardo Vinuesa
Abstract:
Despite its great scientific and technological importance, wall-bounded turbulence is an unresolved problem in classical physics that requires new perspectives to be tackled. One of the key strategies has been to study interactions among the energy-containing coherent structures in the flow. Such interactions are explored in this study for the first time using an explainable deep-learning method.…
▽ More
Despite its great scientific and technological importance, wall-bounded turbulence is an unresolved problem in classical physics that requires new perspectives to be tackled. One of the key strategies has been to study interactions among the energy-containing coherent structures in the flow. Such interactions are explored in this study for the first time using an explainable deep-learning method. The instantaneous velocity field obtained from a turbulent channel flow simulation is used to predict the velocity field in time through a U-net architecture. Based on the predicted flow, we assess the importance of each structure for this prediction using the game-theoretic algorithm of SHapley Additive exPlanations (SHAP). This work provides results in agreement with previous observations in the literature and extends them by revealing that the most important structures in the flow are not necessarily the ones with the highest contribution to the Reynolds shear stress. We also apply the method to an experimental database, where we can identify completely new structures based on their importance score. This framework has the potential to shed light on numerous fundamental phenomena of wall-bounded turbulence, including novel strategies for flow control.
△ Less
Submitted 19 February, 2024; v1 submitted 2 February, 2023;
originally announced February 2023.
