Edu-Snippets
The impact of GenAI on learning, the use of fidget spinners with ADHD students, and the relationship between timed tests and math anxiety.
Highlights:
Unrestricted use of GenAI may give teachers the false impression that students are learning, when in fact, it is hindering skills acquisition.
Fidget spinners negatively impact the attention and performance of ADHD students. They interfere with learning.
An experiment found no evidence that timed math tests lead to more math-anxious students. Math anxiety is more likely caused by other factors, such as repeated failures, math-anxious teachers, negative stereotypes, poor-quality math instruction, and/or critical feedback from teachers or peers.
Does GenAI help or hinder student learning?
Generative AI is changing the way humans work. GenAI claims to enhance productivity by automating or supporting human activities such as content creation, data analysis, coding assistance, creative work, and more. Amid these claims, GenAI is also influencing the way people learn. In addition to being a tool that can help humans complete tasks, it also serves as an ever-accessible source of knowledge for learners (Bastani et al., 2024). This access to vast amounts of knowledge can impact one’s learning and performance. Thus, this raises an important question for researchers - how does GenAI affect how humans learn new skills? To better understand the impact GenAI can have on learning, Bastani and colleagues conducted a large-scale randomized controlled trial (RCT) involving nearly 1,000 high school students in Turkey. Specifically focusing on mathematics, they studied the impact of GPT-4 on students’ learning.
Participants in the study were randomly assigned to one of the following three groups:
GPT-Base group: This group had access to GPT-4 with a regular chat interface.
GPT-Tutor group: This group had access to a modified GPT-4 that functioned like a tutor.
Control group: The third group had no access to any generative AI tools.
Each participant completed the following three phases of the study:
Phase 1 - In the first phase, teachers reviewed a math topic previously covered in the course and solved one or two problems on the board.
Phase 2 - The next phase was the assisted-practice phase. Students were asked to solve a set of practice questions to reinforce the covered concept. For this phase, students in each group were allowed to use their assigned tools for support. The GPT-Base group used GPT-4, the GPT-Tutor group used GPT-Tutor, while the Control group could only use their books and notes, with no access to generative artificial intelligence.
Phase 3 - The third phase consisted of an unassisted evaluation, where students took a closed-book, closed-laptop exam.
Note that phases 1 and 3 were identical across all groups and the randomized intervention only affected phase 2, i.e., the assisted practice phase.
Results of the assisted practice phase (Phase 2): Students who were in the GPT-Base group performed 48% better than the students in the Control group on the practice problems. Students in GPT-Tutor group performed 127% better than the students in the Control group.
Results of the unassisted evaluation phase (Phase 3): In the final closed-book exam, students in the GPT-Base group performed significantly worse (by 17%) than the students in the Control group. Meanwhile, students in the Tutor group did no better than the Control students. The students who didn’t use artificial intelligence scored as well as, or better than, the students who were supported by ChatGPT.
One of the major implications of this study for educators and parents is the finding that some uses of generative AI can substantially inhibit learning. Bastani and colleagues suggest that the students used it as a crutch rather than a learning aid. A follow-up analysis of students’ interactions with ChatGPT in the GPT-Base group revealed that students were simply asking for answers, rather than building skills to solve those problems. Therefore, it is not surprising that they performed worse than the students who completed the questions on their own. Researchers of this study caution us that while AI tools can be useful and productive, “they must be deployed with appropriate guardrails when learning is important.”
Do fidget spinners help ADHD students concentrate?
Fidget spinners are small hand-held toys that became popular with children five or six years ago and are still commonly seen today.
When fidget spinners first came out, some people promoted them as devices that can help students who have trouble focusing or concentrating.
Educators have been divided on the value of fidget spinners. Some teachers allow students to bring these toys to class, believing these devices might help them better focus on their schoolwork. Other teachers don't allow children to bring these toys into their classrooms, worried that the devices will distract them.
In recent years, several studies have been published that examine the educational impact of fidget spinners. One study, entitled, To fidget or not to fidget: That is the question, conducted a systematic classroom evaluation of fidget spinners among young children with ADHD. The study reported:
"Children's use of fidget spinners was associated with poorer attention across both phases of the treatment. Fidget spinners negatively influence young children with ADHD's attentional functioning."
A second study, entitled, A multisite study of the effect of fidget spinners on academic performance found that student performance was lower when they played with fidget spinners. Their performance increased when the spinner was taken away. The paper concludes, "The current study suggests that fidget spinners may cause a deficit in student performance."
Similar findings were reported in a third paper. This time, researchers examined the use of fidget spinners with college-age students as they watched a lecture. The paper, Putting a negative spin on it: Using a fidget spinner can impair memory for a video lecture, reported that "using a fidget spinner was associated with increased reports of attentional lapses, diminished judgments of learning, and impaired performance on a memory test for the material covered in the video."
Are there any situations in which fidget spinners might have educational value?
Maybe.
One study examined the use of fidget spinners with a narrow subset of ADHD students: those who regularly exhibit disruptive behaviours. This was an extremely small study, involving only three students. The study found that the incidence of disruptive behaviour decreased when students played with spinners. The paper also reported the three students appeared to be more on-task (i.e., they looked at their work more often). However, the teacher didn't like the intervention:
"[The teacher] stated that although the students may have looked toward their work and were quieter when using the fidget spinners, they did not actually complete the work. She indicated that although she was glad the students were not as rowdy and loud, her main goal was for them to complete the assignments."
Conclusions: On balance, research strongly suggests that fidget spinners negatively impact student attention and performance. They interfere with learning.
Do timed math tests cause math anxiety?
One of the major debates among math teaching experts is the relationship between timed math tests and math anxiety.
Does the use of timed math tests lead to more math-anxious students?
This is a tricky question because a subtle difference exists between something being the origin of math anxiety versus the trigger of math anxiety. It’s true that timed tests can sometimes trigger the anxiety of math-anxious students. However, that’s different from claiming that timed tests are the origin of math anxiety (i.e., administering lots of timed-tests somehow turns non-math-anxious students into math-anxious students).
What does previous research say about this question? So far, there has been little evidence either way. Some popular education consultants, like Jo Boaler, have told teachers that giving students timed tests can promote math anxiety. However, educational researchers have repeatedly pointed out that this claim has no basis in evidence.
One of the reasons why this discussion is so important is because timed math tests often play an important role in the development of math proficiency. Low-stakes, timed quizzes help children in the elementary grades learn their math facts (e.g., the multiplication tables). With practice, students become faster and faster at answering math questions, as their recall of math facts becomes automatic.
An exciting new study published in the peer-reviewed Journal of School Psychology is one of the first efforts to directly study whether timed math tasks produce math anxiety. The study involved 113 junior-level students spread across three U.S. schools. It compared students who took timed math tests to students who took tests in which there was no time pressure. Results? The research failed to find any evidence that timed tests produced higher levels of math anxiety.
So, if timed tests don’t cause math anxiety, then what does cause it? The authors suggest that a large number of individual and environmental factors may be at play. They also observe that there is “growing evidence suggesting poor math performance affects the development of math anxiety.” In other words, children may develop math anxiety when they realize they don’t understand the math concepts taught in class and are unable to solve the related math problems. If this is the case, one part of the solution might involve teaching in such a way that the math-anxious student can succeed at doing math and can see their performance improving.
Given the results of their study, the authors express concern that teachers who avoid timed tests may be harming their children by making it more difficult for them to master their math facts. They write:
"The findings from this study support educators' use of overtly timed tasks as part of their tiered math curriculum because it can help improve elementary students' math fact fluency and overall math performance."
References:
Aspiranti, K., & Hula, D. (2022). Using fidget spinners to improve on-task classroom behavior for students with ADHD. Behavior Analysis in Practice, 15, 454-465. https://doi.org/10.1007/s40617-021-00588-2
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, O., & Mariman, R. (2024). Generative AI can harm learning. Wharton School Research Paper. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4895486
Graziano, P., Garcia, A., & Landis, T. (2020). To fidget or not to fidget, that is the question: A systematic classroom evaluation of fidget spinners among young children with ADHD. Journal of Attentional Disorders, 24(1), 163-171. https://doi.org/10.1177/1087054718770009
Hulac, D., Aspiranti, K., Kriescher, S., Briesch, A., & Athanasiou, M. (2021). A multisite study of the effect of fidget spinners on academic performance. Contemporary School Psychology, 25, 582-588. https://doi.org/10.1007/s40688-020-00292-y
Maki, K., Zaslofsky, A., Codding, R., & Woods, B. (2024). Math anxiety in elementary students: Examining the role of timing and task complexity. Journal of School Psychology, 106, Article 101316. https://doi.org/10.1016/j.jsp.2024.101316
Soares, J., & Storm, B. (2020). Putting a negative spin on it: Using a fidget spinner can impair memory for a video lecture. Applied Cognitive Psychology, 34, 277-284. https://doi.org/10.1002/acp.3610
Great edu-snippets. Thanks for covering the math anxiety article. Robin Codding and her co-authors are doing amazing work!
Thoughts about “Does GenAI help or hinder student learning?”
As mentioned in this study, students were concentrating on answer-finding rather than skill-building. According to my experience both as a student and as an instructor, the root of this issue is that students feel that building skills in math is a painful process. To improve the helpfulness of Generative AI, a redesign incorporating incentives for students to go through the learning activities can potentially provide better results. In other words, if students are doing what they are supposed to do, positive outcomes will follow.
Another thought relating to this study and my real-life teaching experience is that in education, the same pattern observed in the law of conservation of energy applies: the amount of effective input determines the extent of improvements in output. Instead of creating a magical process, effective teaching and learning seem to make the brain work harder within a specific time frame, resulting in a better learning outcome. The students using books and notes in this study did not have the option to get quick answers, so they had to work harder. This may contribute to the more desirable results. My recent teaching experience also confirms this to some extent: the class I teach performed significantly better in the midterm exam this semester compared to other classes. One factor might be the critical questions I asked during class time, which increased their thinking effort. The other factor might be the larger amount of work I assigned – instead of 15 questions provided in a typical quiz, I usually provide around 40 questions.
In short, there is no free lunch. Hence, the direction of education research should probably focus on inducing more intense brain work without cognitive overload.