Leaders at a loss: Fragmented data frustrates executives’ use of AI

AI’s promise of productivity faces a harsh reality check as CEOs discover their data systems are simply not ready to support advanced AI initiatives. The IBM study of 2,000 CEOs across 30 countries reveals a critical disconnect between AI ambitions and the foundational data architecture needed to realize meaningful returns, highlighting the growing urgency for companies to address their fragmented information systems before expecting AI to deliver on its potential.

The big picture: While 61% of CEOs are actively deploying AI agents and planning their expansion, half admit their rush to adopt new technologies has created disconnected systems that undermine AI effectiveness.

• Only 25% of AI initiatives have delivered expected ROI in recent years, a sobering statistic that points to fundamental implementation problems.
• CEOs face a dual challenge: showing short-term AI returns while simultaneously investing in the long-term data infrastructure needed for sustainable AI success.

Behind the numbers: Fragmented data systems, accumulated through years of piecemeal technology adoption, are preventing AI models from accessing the comprehensive, high-quality information they need to function optimally.

• 68% of CEOs recognize that an integrated, enterprise-wide data architecture is critical for enabling cross-functional collaboration.
• 72% view their proprietary data as the key to unlocking generative AI’s value, yet most lack the unified data foundation to properly leverage it.

What they’re saying: “CEOs are balancing the pressures of short-term ROI and investing in long-term innovation when it comes to adopting AI,” explains Mohamad Ali, senior vice president and head of IBM Consulting.

• “There’s no long-term AI ROI in layering models over broken foundations,” warns Amandeep Singh, practice director at QKS Group, highlighting how surface-level AI integrations merely add to growing technical debt.

Important stats: The study reveals widespread challenges in AI implementation and realistic expectations about returns.

• 64% of CEOs admit to investing in AI without fully understanding its value.
• Chief AI Officers reported a modest average ROI of 14% in 2025.
• 85% of executives expect positive ROI from scaled AI efficiency by 2027, suggesting optimism about long-term returns despite current challenges.

Key recommendations: CIOs need to systematically address data foundation issues before expanding AI initiatives.

• Data virtualization and system integration should be prioritized to create coherent information flows.
• Organizations should build data fabrics connecting disparate systems while training AI on clean, enterprise-specific information.
• Business rules, ethics, and security must be embedded in data architecture from the ground up.

The bottom line: “No AI model should hit production without plugging into real business workflows,” Singh noted, emphasizing that properly addressing data challenges can transform organizational chaos into a competitive advantage.