LAMMI-Pathology: A Tool-Centric Bottom-Up LVLM-Agent Framework for Molecularly Informed Medical Intelligence in Pathology
arXiv:2602.18773v1 Announce Type: new Abstract: The emergence of tool-calling-based agent systems introduces a more evidence-driven paradigm for pathology image analysis in contrast to the coarse-grained text-image diagnostic approaches. With the recent large-scale experimental adoption of spatial transcriptomics technologies, molecularly validated pathological diagnosis is becoming increasingly open and accessible. In this work, we propose LAMMI-Pathology (LVLM-Agent System for Molecularly Informed Medical Intelligence in Pathology), a scalable agent framework for domain-specific agent tool-calling. LAMMI-Pathology adopts a tool-centric, bottom-up architecture in which customized domain-adaptive tools serve as the foundation. These tools are clustered by domain style to form component agents, which are then coordinated through a top-level planner hierarchically, avoiding excessively long context lengths that could induce task drift. Based on that, we introduce a novel trajectory cons
arXiv:2602.18773v1 Announce Type: new Abstract: The emergence of tool-calling-based agent systems introduces a more evidence-driven paradigm for pathology image analysis in contrast to the coarse-grained text-image diagnostic approaches. With the recent large-scale experimental adoption of spatial transcriptomics technologies, molecularly validated pathological diagnosis is becoming increasingly open and accessible. In this work, we propose LAMMI-Pathology (LVLM-Agent System for Molecularly Informed Medical Intelligence in Pathology), a scalable agent framework for domain-specific agent tool-calling. LAMMI-Pathology adopts a tool-centric, bottom-up architecture in which customized domain-adaptive tools serve as the foundation. These tools are clustered by domain style to form component agents, which are then coordinated through a top-level planner hierarchically, avoiding excessively long context lengths that could induce task drift. Based on that, we introduce a novel trajectory construction mechanism based on Atomic Execution Nodes (AENs), which serve as reliable and composable units for building semi-simulated reasoning trajectories that capture credible agent-tool interactions. Building on this foundation, we develop a trajectory-aware fine-tuning strategy that aligns the planner's decision-making process with these multi-step reasoning trajectories, thereby enhancing inference robustness in pathology understanding and its adaptive use of the customized toolset.
Executive Summary
This article introduces LAMMI-Pathology, a novel agent framework for molecularly informed medical intelligence in pathology. The framework adopts a tool-centric, bottom-up architecture, using customized domain-adaptive tools to form component agents. A novel trajectory construction mechanism is proposed, based on Atomic Execution Nodes (AENs), to capture credible agent-tool interactions. The framework is designed to enhance inference robustness in pathology understanding and its adaptive use of the customized toolset. The proposed approach has the potential to improve the accuracy and efficiency of molecularly validated pathological diagnosis.
Key Points
- ▸ LAMMI-Pathology is a tool-centric, bottom-up agent framework for molecularly informed medical intelligence in pathology.
- ▸ The framework uses customized domain-adaptive tools to form component agents.
- ▸ A novel trajectory construction mechanism is proposed, based on Atomic Execution Nodes (AENs).
Merits
Strength
The proposed framework is designed to enhance inference robustness in pathology understanding and its adaptive use of the customized toolset.
Strength
The use of Atomic Execution Nodes (AENs) provides a reliable and composable unit for building semi-simulated reasoning trajectories.
Demerits
Limitation
The framework may require significant computational resources to implement and deploy.
Limitation
The accuracy of the framework may depend heavily on the quality and availability of the customized domain-adaptive tools.
Expert Commentary
The proposed framework is a significant contribution to the field of medical imaging and pathology diagnosis. The use of tool-centric, bottom-up architecture and Atomic Execution Nodes (AENs) provides a novel and robust approach to molecularly informed medical intelligence. However, the framework may require significant computational resources and may depend heavily on the quality and availability of the customized domain-adaptive tools. Nevertheless, the potential benefits of the framework make it an exciting area of research and development.
Recommendations
- ✓ Further research is needed to evaluate the performance and scalability of the proposed framework in real-world settings.
- ✓ The development of standardized protocols and benchmarks for evaluating the framework is essential to ensure its reproducibility and comparability with other approaches.