MIDAS: Mosaic Input-Specific Differentiable Architecture Search
arXiv:2602.17700v1 Announce Type: cross Abstract: Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, a novel approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models node connectivity and simplifies selecting the two incoming edges per node. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it consistently finds globally optimal architectures. In RDARTS, it sets the state of the art on two of four search spaces on CIFAR-10. We
arXiv:2602.17700v1 Announce Type: cross Abstract: Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, a novel approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models node connectivity and simplifies selecting the two incoming edges per node. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it consistently finds globally optimal architectures. In RDARTS, it sets the state of the art on two of four search spaces on CIFAR-10. We further analyze why MIDAS works, showing that patchwise attention improves discrimination among candidate operations, and the resulting input-specific parameter distributions are class-aware and predominantly unimodal, providing reliable guidance for decoding.
Executive Summary
The article presents MIDAS, a novel Differentiable Neural Architecture Search (NAS) approach that modernizes DARTS by incorporating dynamic, input-specific parameters computed via self-attention. MIDAS localizes architecture selection by computing it separately for each spatial patch of the activation map and introduces a parameter-free, topology-aware search space. The authors evaluate MIDAS on three search spaces, demonstrating its effectiveness in finding globally optimal architectures and achieving state-of-the-art performance on CIFAR-10 and CIFAR-100. MIDAS's input-specific parameter distributions are class-aware and predominantly unimodal, providing reliable guidance for decoding. This work contributes to the development of more efficient and robust NAS methods, enabling the discovery of better-performing neural networks. The proposed approach has significant implications for the field of deep learning and may lead to breakthroughs in various applications.
Key Points
- ▸ MIDAS modernizes DARTS by incorporating dynamic, input-specific parameters
- ▸ MIDAS localizes architecture selection by computing it separately for each spatial patch
- ▸ MIDAS introduces a parameter-free, topology-aware search space
Merits
Strength in Architecture Design
MIDAS's ability to adapt architecture parameters to input-specific requirements enables the discovery of more effective neural networks.
Robustness and Efficiency
MIDAS's localized architecture selection and parameter-free search space improve the robustness and efficiency of the NAS process.
Demerits
Computational Complexity
The increased complexity of MIDAS's architecture design and search process may lead to higher computational requirements.
Interpretability
The class-aware and unimodal nature of MIDAS's parameter distributions may reduce interpretability of the resulting architectures.
Expert Commentary
The article presents a novel and innovative approach to NAS, addressing the limitations of existing methods. MIDAS's ability to adapt architecture parameters to input-specific requirements and its localized architecture selection process make it a robust and efficient NAS method. However, the increased complexity of the architecture design and search process may lead to higher computational requirements. Furthermore, the class-aware and unimodal nature of MIDAS's parameter distributions may reduce interpretability of the resulting architectures. Nevertheless, the proposed approach has significant implications for the field of deep learning and may lead to breakthroughs in various applications.
Recommendations
- ✓ Further research is needed to investigate the computational complexity and interpretability of MIDAS's architecture design and search process.
- ✓ The proposed approach should be applied to a wider range of applications to demonstrate its effectiveness and versatility.