Vibe Researching as Wolf Coming: Can AI Agents with Skills Replace or Augment Social Scientists?
arXiv:2602.22401v1 Announce Type: new Abstract: AI agents -- systems that execute multi-step reasoning workflows with persistent state, tool access, and specialist skills -- represent a qualitative shift from prior automation technologies in social science. Unlike chatbots that respond to isolated queries, AI agents can now read files, run code, query databases, search the web, and invoke domain-specific skills to execute entire research pipelines autonomously. This paper introduces the concept of vibe researching -- the AI-era parallel to ``vibe coding'' (Karpathy, 2025) -- and uses scholar-skill, a 21-skill plugin for Claude Code covering the full research pipeline from idea to submission, as an illustrative case. I develop a cognitive task framework that classifies research activities along two dimensions -- codifiability and tacit knowledge requirement -- to identify a delegation boundary that is cognitive, not sequential: it cuts through every stage of the research pipeline, not
arXiv:2602.22401v1 Announce Type: new Abstract: AI agents -- systems that execute multi-step reasoning workflows with persistent state, tool access, and specialist skills -- represent a qualitative shift from prior automation technologies in social science. Unlike chatbots that respond to isolated queries, AI agents can now read files, run code, query databases, search the web, and invoke domain-specific skills to execute entire research pipelines autonomously. This paper introduces the concept of vibe researching -- the AI-era parallel to ``vibe coding'' (Karpathy, 2025) -- and uses scholar-skill, a 21-skill plugin for Claude Code covering the full research pipeline from idea to submission, as an illustrative case. I develop a cognitive task framework that classifies research activities along two dimensions -- codifiability and tacit knowledge requirement -- to identify a delegation boundary that is cognitive, not sequential: it cuts through every stage of the research pipeline, not between stages. I argue that AI agents excel at speed, coverage, and methodological scaffolding but struggle with theoretical originality and tacit field knowledge. The paper concludes with an analysis of three implications for the profession -- augmentation with fragile conditions, stratification risk, and a pedagogical crisis -- and proposes five principles for responsible vibe researching.
Executive Summary
The article explores the potential of AI agents to replace or augment social scientists by executing entire research pipelines autonomously. Introducing the concept of 'vibe researching,' the author uses a cognitive task framework to classify research activities and identifies a cognitive delegation boundary that cuts through every stage of the research pipeline. The paper argues that while AI agents excel in speed, coverage, and methodological scaffolding, they struggle with theoretical originality and tacit field knowledge. It concludes by discussing implications for the profession, including augmentation with fragile conditions, stratification risk, and a pedagogical crisis, and proposes principles for responsible vibe researching.
Key Points
- ▸ AI agents represent a qualitative shift in automation technologies for social science.
- ▸ Vibe researching is introduced as the AI-era parallel to 'vibe coding.'
- ▸ A cognitive task framework classifies research activities along codifiability and tacit knowledge requirements.
- ▸ AI agents excel in speed, coverage, and methodological scaffolding but struggle with theoretical originality and tacit field knowledge.
- ▸ The paper discusses implications for the profession and proposes principles for responsible vibe researching.
Merits
Innovative Framework
The cognitive task framework provides a nuanced classification of research activities, offering a clear delineation of where AI agents can and cannot effectively intervene.
Comprehensive Analysis
The paper thoroughly examines the capabilities and limitations of AI agents in social science research, providing a balanced view of their potential impact.
Practical Implications
The discussion on augmentation, stratification risk, and pedagogical crisis offers actionable insights for practitioners and policymakers.
Demerits
Limited Empirical Evidence
The paper relies heavily on theoretical frameworks and hypothetical scenarios, lacking empirical data to support its claims.
Generalizability
The findings are based on a single case study (scholar-skill plugin), which may not be representative of all AI agents or research contexts.
Ethical Considerations
While the paper touches on ethical implications, it does not delve deeply into the ethical dilemmas and potential biases that AI agents might introduce.
Expert Commentary
The article presents a timely and thought-provoking analysis of the role of AI agents in social science research. The introduction of the cognitive task framework is a significant contribution, as it provides a structured way to understand the delegation boundary between human researchers and AI agents. The paper's discussion on the implications for the profession is particularly insightful, highlighting the potential for augmentation under fragile conditions and the risk of stratification. However, the lack of empirical evidence and the reliance on a single case study limit the generalizability of the findings. Future research should aim to provide more robust empirical data and explore the ethical considerations in greater depth. Overall, the article offers a valuable starting point for further discussion and research on the integration of AI agents in social science research.
Recommendations
- ✓ Conduct empirical studies to validate the theoretical frameworks and hypotheses presented in the paper.
- ✓ Explore the ethical implications of AI agents in social science research, including potential biases and ethical dilemmas.
- ✓ Develop guidelines and best practices for the responsible use of AI agents in research, considering the cognitive delegation boundary identified in the paper.
