Retrieval Augmented (Knowledge Graph), and Large Language Model-Driven Design Structure Matrix (DSM) Generation of Cyber-Physical Systems
arXiv:2602.16715v1 Announce Type: new Abstract: We explore the potential of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Graph-based RAG (GraphRAG) for generating Design Structure Matrices (DSMs). We test these methods on two distinct use cases -- a power screwdriver and a CubeSat with known architectural references -- evaluating their performance on two key tasks: determining relationships between predefined components, and the more complex challenge of identifying components and their subsequent relationships. We measure the performance by assessing each element of the DSM and overall architecture. Despite design and computational challenges, we identify opportunities for automated DSM generation, with all code publicly available for reproducibility and further feedback from the domain experts.
arXiv:2602.16715v1 Announce Type: new Abstract: We explore the potential of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Graph-based RAG (GraphRAG) for generating Design Structure Matrices (DSMs). We test these methods on two distinct use cases -- a power screwdriver and a CubeSat with known architectural references -- evaluating their performance on two key tasks: determining relationships between predefined components, and the more complex challenge of identifying components and their subsequent relationships. We measure the performance by assessing each element of the DSM and overall architecture. Despite design and computational challenges, we identify opportunities for automated DSM generation, with all code publicly available for reproducibility and further feedback from the domain experts.
Executive Summary
This article explores the application of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Graph-based RAG (GraphRAG) in generating Design Structure Matrices (DSMs) for Cyber-Physical Systems (CPS). The authors evaluate the performance of these methods on two distinct use cases, demonstrating their potential in automating DSM generation. While the study identifies opportunities for automated DSM generation, it also highlights design and computational challenges. The publicly available code enables reproducibility and invites feedback from domain experts. The findings have significant implications for CPS design and could improve efficiency and accuracy in complex system development. Overall, the research showcases the potential of LLMs in CPS design, but highlights the need for further refinement and testing.
Key Points
- ▸ Application of LLMs, RAG, and GraphRAG in generating DSMs for CPS
- ▸ Evaluation of the methods on two distinct use cases: power screwdriver and CubeSat
- ▸ Identification of opportunities for automated DSM generation
- ▸ Design and computational challenges in the application of LLMs
Merits
Strength in innovative application of LLMs
The study demonstrates the potential of LLMs in CPS design, introducing a novel approach to DSM generation. This strength lies in the innovative application of LLMs, which may lead to improved efficiency and accuracy in complex system development.
Demerits
Limitation in addressing design and computational challenges
The study highlights the need for further refinement and testing to address design and computational challenges in the application of LLMs. This limitation may hinder the widespread adoption of LLMs in CPS design.
Limited evaluation of the methods
The study evaluates the methods on only two distinct use cases, which may not be representative of the broader range of CPS designs. This limitation may limit the generalizability of the findings.
Expert Commentary
The study demonstrates the potential of LLMs in CPS design, introducing a novel approach to DSM generation. However, the need for further refinement and testing to address design and computational challenges is a significant limitation. The study highlights the importance of evaluating the methods on a broader range of CPS designs. Furthermore, the potential for increased adoption of CPS design automation tools is an exciting prospect. Nevertheless, the development of more robust and reliable LLMs is crucial for widespread adoption. Overall, the study is a step in the right direction, and its findings have significant implications for CPS design and development.
Recommendations
- ✓ Further refinement and testing of the LLM-based DSM generation method
- ✓ Evaluation of the method on a broader range of CPS designs
- ✓ Development of more robust and reliable LLMs for CPS design automation
