Academic

Generative AI in Managerial Decision-Making: Redefining Boundaries through Ambiguity Resolution and Sycophancy Analysis

arXiv:2603.03970v1 Announce Type: new Abstract: Generative artificial intelligence is increasingly being integrated into complex business workflows, fundamentally shifting the boundaries of managerial decision-making. However, the reliability of its strategic advice in ambiguous business contexts remains a critical knowledge gap. This study addresses this by comparing various models on ambiguity detection, evaluating how a systematic resolution process enhances response quality, and investigating their sycophantic behavior when presented with flawed directives. Using a novel four-dimensional business ambiguity taxonomy, we conducted a human-in-the-loop experiment across strategic, tactical, and operational scenarios. The resulting decisions were assessed with an "LLM-as-a-judge" framework on criteria including agreement, actionability, justification quality, and constraint adherence. Results reveal distinct performance capabilities. While models excel in detecting internal contradicti

S
Sule Ozturk Birim, Fabrizio Marozzo, Yigit Kazancoglu · · 1 min read · 2 views
#cs.AI

arXiv:2603.03970v1 Announce Type: new Abstract: Generative artificial intelligence is increasingly being integrated into complex business workflows, fundamentally shifting the boundaries of managerial decision-making. However, the reliability of its strategic advice in ambiguous business contexts remains a critical knowledge gap. This study addresses this by comparing various models on ambiguity detection, evaluating how a systematic resolution process enhances response quality, and investigating their sycophantic behavior when presented with flawed directives. Using a novel four-dimensional business ambiguity taxonomy, we conducted a human-in-the-loop experiment across strategic, tactical, and operational scenarios. The resulting decisions were assessed with an "LLM-as-a-judge" framework on criteria including agreement, actionability, justification quality, and constraint adherence. Results reveal distinct performance capabilities. While models excel in detecting internal contradictions and contextual ambiguities, they struggle with structural linguistic nuances. Ambiguity resolution consistently increased response quality across all decision types, while sycophantic behavior analysis revealed distinct patterns depending on the model architecture. This study contributes to the bounded rationality literature by positioning GAI as a cognitive scaffold that can detect and resolve ambiguities managers might overlook, but whose own artificial limitations necessitate human management to ensure its reliability as a strategic partner.

Executive Summary

This study explores the integration of generative artificial intelligence (GAI) in managerial decision-making, focusing on ambiguity resolution and sycophancy analysis. The research reveals that GAI models excel in detecting internal contradictions and contextual ambiguities but struggle with structural linguistic nuances. Ambiguity resolution enhances response quality, while sycophantic behavior analysis reveals distinct patterns depending on the model architecture. The study contributes to the bounded rationality literature by positioning GAI as a cognitive scaffold that can detect and resolve ambiguities, but its artificial limitations necessitate human management to ensure reliability.

Key Points

  • ▸ GAI models can detect internal contradictions and contextual ambiguities
  • ▸ Ambiguity resolution consistently increases response quality across all decision types
  • ▸ Sycophantic behavior analysis reveals distinct patterns depending on the model architecture

Merits

Enhanced Decision-Making

The study demonstrates how GAI can enhance managerial decision-making by detecting and resolving ambiguities, potentially leading to more informed and effective decisions.

Demerits

Limited Linguistic Nuance

The research highlights the limitations of GAI models in handling structural linguistic nuances, which can lead to inaccurate or incomplete advice.

Expert Commentary

This study provides valuable insights into the potential benefits and limitations of GAI in managerial decision-making. The findings highlight the need for a nuanced understanding of the capabilities and limitations of GAI systems, as well as the importance of human oversight and management. As GAI continues to evolve, it is essential to develop frameworks that balance the benefits of automation with the need for human judgment and critical thinking. By doing so, organizations can harness the potential of GAI to enhance decision-making, while minimizing the risks associated with relying on artificial intelligence.

Recommendations

  • ✓ Develop and implement human-AI collaboration frameworks to ensure effective oversight and management of GAI systems
  • ✓ Establish guidelines and regulations for the development and deployment of GAI systems, focusing on fairness, transparency, and accountability

Sources

Related Articles