Operationalising AI governance through ethics-based auditing: an industry case study
AbstractEthics-based auditing (EBA) is a structured process whereby an entity’s past or present behaviour is assessed for consistency with moral principles or norms. Recently, EBA has attracted much attention as a governance mechanism that may help to bridge the gap between principles and practice in AI ethics. However, important aspects of EBA—such as the feasibility and effectiveness of different auditing procedures—have yet to be substantiated by empirical research. In this article, we address this knowledge gap by providing insights from a longitudinal industry case study. Over 12 months, we observed and analysed the internal activities of AstraZeneca, a biopharmaceutical company, as it prepared for and underwent an ethics-based AI audit. While previous literature concerning EBA has focussed on proposing or analysing evaluation metrics or visualisation techniques, our findings suggest that the main difficulties large multinational organisations face when conducting EBA mirror class
AbstractEthics-based auditing (EBA) is a structured process whereby an entity’s past or present behaviour is assessed for consistency with moral principles or norms. Recently, EBA has attracted much attention as a governance mechanism that may help to bridge the gap between principles and practice in AI ethics. However, important aspects of EBA—such as the feasibility and effectiveness of different auditing procedures—have yet to be substantiated by empirical research. In this article, we address this knowledge gap by providing insights from a longitudinal industry case study. Over 12 months, we observed and analysed the internal activities of AstraZeneca, a biopharmaceutical company, as it prepared for and underwent an ethics-based AI audit. While previous literature concerning EBA has focussed on proposing or analysing evaluation metrics or visualisation techniques, our findings suggest that the main difficulties large multinational organisations face when conducting EBA mirror classical governance challenges. These include ensuring harmonised standards across decentralised organisations, demarcating the scope of the audit, driving internal communication and change management, and measuring actual outcomes. The case study presented in this article contributes to the existing literature by providing a detailed description of the organisational context in which EBA procedures must be integrated to be feasible and effective.
Executive Summary
The article 'Operationalising AI governance through ethics-based auditing: an industry case study' explores the practical implementation of ethics-based auditing (EBA) within a large multinational organisation, AstraZeneca. The study provides a longitudinal analysis over 12 months, highlighting the challenges and governance issues encountered during the preparation and execution of an AI ethics audit. The findings suggest that the primary difficulties in conducting EBA are similar to classical governance challenges, such as harmonising standards across decentralised structures, defining audit scope, managing internal communication, and measuring outcomes. The article contributes to the literature by offering a detailed organisational context for integrating EBA procedures effectively.
Key Points
- ▸ EBA is proposed as a governance mechanism to bridge the gap between AI ethics principles and practice.
- ▸ The study identifies classical governance challenges as the main difficulties in implementing EBA.
- ▸ The case study provides empirical insights from AstraZeneca's experience with AI ethics auditing.
Merits
Empirical Contribution
The article provides valuable empirical data from a real-world case study, which is a significant contribution to the field of AI governance.
Practical Insights
The study offers practical insights into the challenges of implementing EBA, which can guide other organisations in their AI governance efforts.
Organisational Context
The detailed description of the organisational context in which EBA procedures must be integrated adds depth to the understanding of EBA feasibility and effectiveness.
Demerits
Limited Generalisability
The findings are based on a single case study, which may limit the generalisability of the results to other organisations.
Focus on Large Multinationals
The study focuses on a large multinational organisation, which may not fully capture the experiences of smaller or less decentralised entities.
Longitudinal Scope
The 12-month study period, while comprehensive, may not account for long-term impacts and sustainability of EBA procedures.
Expert Commentary
The article 'Operationalising AI governance through ethics-based auditing: an industry case study' provides a timely and much-needed empirical examination of the practical challenges associated with implementing ethics-based auditing in a large multinational organisation. The study's focus on AstraZeneca offers a rich case study that illuminates the complexities of integrating EBA procedures within an existing governance framework. The findings are particularly noteworthy as they reveal that the primary obstacles to EBA are not unique to AI governance but are, in fact, classic governance challenges. This observation is crucial as it suggests that established governance strategies may be adapted to enhance the effectiveness of EBA. However, the study's reliance on a single case study limits the generalisability of its findings. Future research should aim to include a broader range of organisations to provide a more comprehensive understanding of EBA implementation. Additionally, the study's emphasis on large multinationals may overlook the unique challenges faced by smaller entities. Despite these limitations, the article makes a significant contribution to the field by bridging the gap between theoretical principles and practical application in AI ethics. The insights provided can guide both practitioners and policymakers in developing more robust and effective AI governance frameworks.
Recommendations
- ✓ Conduct further empirical studies involving a diverse range of organisations to enhance the generalisability of findings on EBA implementation.
- ✓ Develop guidelines and best practices for harmonising standards and managing communication within decentralised organisations to support EBA procedures.