Exploring Human Behavior During Abstract Rule Inference and Problem Solving with the Cognitive Abstraction and Reasoning Corpus
arXiv:2602.22408v1 Announce Type: new Abstract: Humans exhibit remarkable flexibility in abstract reasoning, and can rapidly learn and apply rules from sparse examples. To investigate the cognitive strategies underlying this ability, we introduce the Cognitive Abstraction and Reasoning Corpus (CogARC), a diverse human-adapted subset of the Abstraction and Reasoning Corpus (ARC) which was originally developed to benchmark abstract reasoning in artificial intelligence. Across two experiments, CogARC was administered to a total of 260 human participants who freely generated solutions to 75 abstract visual reasoning problems. Success required inferring input-output rules from a small number of examples to transform the test input into one correct test output. Participants' behavior was recorded at high temporal resolution, including example viewing, edit sequences, and multi-attempt submissions. Participants were generally successful (mean accuracy ~90% for experiment 1 (n=40), ~80% for e
arXiv:2602.22408v1 Announce Type: new Abstract: Humans exhibit remarkable flexibility in abstract reasoning, and can rapidly learn and apply rules from sparse examples. To investigate the cognitive strategies underlying this ability, we introduce the Cognitive Abstraction and Reasoning Corpus (CogARC), a diverse human-adapted subset of the Abstraction and Reasoning Corpus (ARC) which was originally developed to benchmark abstract reasoning in artificial intelligence. Across two experiments, CogARC was administered to a total of 260 human participants who freely generated solutions to 75 abstract visual reasoning problems. Success required inferring input-output rules from a small number of examples to transform the test input into one correct test output. Participants' behavior was recorded at high temporal resolution, including example viewing, edit sequences, and multi-attempt submissions. Participants were generally successful (mean accuracy ~90% for experiment 1 (n=40), ~80% for experiment 2 (n=220) across problems), but performance varied widely across problems and participants. Harder problems elicited longer deliberation times and greater divergence in solution strategies. Over the course of the task, participants initiated responses more quickly but showed a slight decline in accuracy, suggesting increased familiarity with the task structure rather than improved rule-learning ability. Importantly, even incorrect solutions were often highly convergent, even when the problem-solving trajectories differed in length and smoothness. Some trajectories progressed directly and efficiently toward a stable outcome, whereas others involved extended exploration or partial restarts before converging. Together, these findings highlight CogARC as a rich behavioral environment for studying human abstract reasoning, providing insight into how people generalize, misgeneralize, and adapt their strategies under uncertainty.
Executive Summary
This article introduces the Cognitive Abstraction and Reasoning Corpus (CogARC), a collection of 75 abstract visual reasoning problems designed to investigate human cognitive strategies in abstract rule inference and problem-solving. Using a dataset of 260 human participants, the study reveals a high level of success in solving these problems, with participants exhibiting diverse solution strategies and a slight decline in accuracy over time. The findings highlight the potential of CogARC as a behavioral environment for studying human abstract reasoning and provide insight into how people generalize, misgeneralize, and adapt their strategies under uncertainty. This study contributes to our understanding of human cognition and has implications for the development of artificial intelligence and cognitive models.
Key Points
- ▸ CogARC is a diverse and comprehensive dataset for studying human abstract reasoning
- ▸ Human participants exhibit high success rates in solving abstract visual reasoning problems
- ▸ Solution strategies vary widely across problems and participants, with some participants exhibiting efficient and others extended exploration
Merits
Strength in methodology
The study employs a robust methodology, with a large dataset of human participants and high temporal resolution recordings of behavior. This provides a rich and detailed understanding of human cognitive strategies in abstract rule inference and problem-solving.
Insight into human cognition
The study provides new insights into how people generalize, misgeneralize, and adapt their strategies under uncertainty, shedding light on the complex and dynamic nature of human cognition.
Demerits
Limited generalizability
The study's findings may not be generalizable to other populations or contexts, as the dataset is specifically designed for abstract visual reasoning problems. Further research is needed to explore the applicability of CogARC to other domains.
Limited consideration of cognitive biases
The study does not explicitly consider the impact of cognitive biases on human performance in abstract rule inference and problem-solving. Future research could explore the role of biases in shaping human cognition and behavior.
Expert Commentary
The study's methodology and findings are robust and well-executed, providing new insights into human cognition and strategy adaptation. However, the study's limitations, such as limited generalizability and consideration of cognitive biases, should be acknowledged and addressed in future research. The study's implications for artificial intelligence and cognitive models are significant, as it highlights the importance of developing more effective models that can adapt to changing contexts and uncertainty. Furthermore, the study's findings have important policy implications for education and training programs, as they emphasize the need to develop adaptive and flexible problem-solving strategies in humans.
Recommendations
- ✓ Future research should explore the applicability of CogARC to other domains and populations.
- ✓ The study's limitations, such as limited generalizability and consideration of cognitive biases, should be addressed in future research.
