A Gartner survey of 451 senior technology leaders in the second quarter of 2024, found that 57% of CIOs said they are tasked with leading an AI strategy in their enterprise. However, four emerging challenges are making it difficult for CIOs to deliver value with AI.
“Because of the relentless innovation happening in the tech vendor race, CIOs feel like they are always living the hype, while the reality of their AI outcomes race – how tough it is to get value – makes it feel like they are also in the trough,” said Shubhangi Vashisth, Director Analyst at Gartner.
“However, CIOs can set the pace in their AI outcomes race,” said Mbula Schoen, Sr Director Analyst at Gartner. “If you have modest AI ambitions, in an industry that isn’t being remastered by AI yet, you can afford to go at a more measured pace. This is an AI-steady pace. For those organisations with bigger AI ambitions, or in an industry that’s being reinvented by AI, the pace will be faster. This is an AI-accelerated pace. Whether you’re moving at an AI-steady or AI-accelerated pace, you have to deliver value and outcomes.”
During the opening keynote of Gartner IT Symposium/Xpo, which is taking place here through Wednesday, Gartner analysts explained to the audience of over 1,000 CIOs and IT executives how to overcome four emerging challenges to deliver value from AI safely and at scale.
The business benefits of using AI don’t always materialise
To generate business value with generative AI (GenAI), people must consistently use GenAI tools in their workflow. In a second quarter 2024 Gartner survey of over 5,000 digital workers in the U.S. UK, India, Australia and China, employees said that they saved an average of 3.6 hours per week by using GenAI. But not all employees get the same degree of benefit from using GenAI.
“Here’s the real challenge with AI productivity,” said Schoen. “Productivity gains from GenAI are not equally distributed. Gains vary by employee, not just because of their personal interest and levels of adoptions, but according to complexity of job and level of experience.”
AI-accelerated organisations are also looking at benefits beyond productivity – operations and process-level improvements, such as automating key business processes or redesigning roles to work with chatbots; and business-level, game-changing improvements, such as outcomes that create new revenue streams or redesign the enterprise value proposition.
“In these cases, CIOs should manage AI benefits like a portfolio. Determine the size of your bet in each benefit area, and manage risks and rewards across this portfolio,” said Schoen.
The cost of AI can quickly spiral out of control
More than 90% of CIOs said that managing cost limits their ability to get value from AI for their enterprise, according to a Gartner survey of over 300 CIOs in June and July 2024. In fact, Gartner believes that cost is as big an AI risk as security or hallucinations.
If CIOs don’t understand how their GenAI costs scale, Gartner estimates that they could make a 500%-1,000% error in their cost calculations.
“As a CIO, you need to understand your AI bill,” said Vashisth. “You must understand the cost components and pricing model options, and you need to know how to reduce these costs and negotiate with vendors. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”
Data and AI everywhere creates new challenges and risks
With AI and data proliferating everywhere in the enterprise, AI and data are no longer centralised assets that IT directly controls. The Gartner survey of over 300 CIOs found that on average, only 35% of their AI capabilities will be built by their IT teams. This means that new approaches are needed to manage and protect data access and govern AI inputs and outputs and safely deliver AI value.
“This is where the concept of a ‘tech sandwich’ comes in,” said Schoen in describing the AI tech stack of the future (see Figure 1). “On the bottom of the sandwich is all the data and AI from IT, typically centralised. On the top is all the data and AI coming from everywhere, typically decentralised. And the middle contains the trust, risk, and security management (TRiSM) technologies that make it all safe. It’s what you need to create to accommodate AI and data coming from everywhere.”
“As CIO, your job is to design a tech sandwich that can handle the messiness of AI, but still keeps you open to new opportunities,” said Schoen. “AI-steady organisations (ten AI initiatives or fewer) will govern their tech sandwiches using human teams and committees. AI-accelerated organisations will add TRiSM technologies – a set of technologies designed to create trust, monitor risk and manage security for safe AI at scale.”
Using AI can both positively and negatively impact employees’ performance and well-being
Some employees may feel a strong affinity for AI. Others may feel threatened or resentful. These intense reactions to AI can lead to unintended behavioural outcomes that negatively impact employees’ work performance, such as jealousy of those using AI and overdependence on AI tools.
However, few organisations are actively managing these behavioural outcomes. In the June/July Gartner survey, only 20% of CIOs said they focus on mitigating potential negative impacts of GenAI on employee well-being.
“Most enterprises aren’t curious enough about how AI makes their employees feel. This matters because AI can lead to all sorts of unintended behavioural outcomes,” said Vashisth. “The critical point is that if you use change management to manage this, be intentional about who owns which behavioural outcomes. Organisations must manage behavioural outcomes with the same rigour as technology and business outcomes.”