Future of Work
EFFECTIVE USE OF AI
potential, there are still many unanswered questions about AI’s role in the workplace and how firms can best harness its rapidly evolving capabilities. The pioneering research I conducted with colleagues from Harvard Business School, the Wharton School, the MIT Sloan School of Management, and Boston Consulting Group (BCG) offers some important insights into how AI might both help and hinder knowledge workers. It also highlights the benefits and risks this could create for organisations and their leaders. We recruited 758 BCG consultants, who were randomly assigned to work on one of two tasks. Both were designed to be realistic tasks that consultants might face in their regular work. One was an analytic, business strategy task that asked consultants to analyse the brand performance of a company and provide its chief executive with clear recommendations about which brand to focus on. The second was a creative task, whereby consultants needed to develop a new footwear product for a fashion company to target an under-served part of the market. We divided the consultants working on each task into three groups. The first group used AI without any guidance. The second group used AI after being given a very brief training video on how to utilise it. The final group had no access to the technology. Those using AI completed both tasks more quickly. However, there was a stark difference in the quality of their work across the two tasks. In the creative footwear task, those using AI outperformed those who had no access to AI, by a margin of 40 per cent. And while all AI users benefited, the lower-performing candidates
benefited most, closing the gap with their stronger peers. However, on the more strategic decision-making task, those who used ChatGPT-4 performed worse than their counterparts. Consultants who used AI were far less likely to produce correct solutions, by a margin of 20 per cent. Yet their recommendations were evaluated to be of a higher quality because they were so persuasive and well written. In other words, the professional consultants who used generative AI were more likely to be wrong, but still managed to sound more convincing. That presents businesses with a significant problem. The divergence in results highlights one of the main problems around AI adoption. It is extremely hard to know where the limits of its capabilities lie. You can rely on AI to help with “There is no simple answer to the question, for which tasks should companies use AI?” tasks inside the ‘jagged frontier’ of its abilities and produce high quality results. However, if you adopt it for tasks beyond the frontier, you are more likely to make mistakes. This is complicated by the fact that Large Language Models (LLMs) remain fundamentally opaque. Sometimes they produce incorrect results that nonetheless appear plausible and highly convincing. That makes it difficult to predict where they might fall short. Even if you manage to accurately
TO THE CORE
1. Consultants who used AI on a creative task outperformed those who did not by 40 per cent. However, their ideas were less diverse. Human creativity can still help firms stand out. 2. Those who used AI to analyse brand performance and make strategic recommendations performed worse, but their answers were so well written they were evaluated as more persuasive. 3. It difficult for managers to identify when AI is a help or hindrance. AI tools should be tested carefully at regular intervals, as rapid technological progress means this ‘jagged frontier’ is constantly shifting.
FRONTIER by Hila Lifshitz
I t is now three decades since Jurassic Park stormed the box office, gripping audiences around the globe with its cautionary tale about the reckless pursuit of progress. Scientists may be no closer to recreating dinosaurs. However, the startling capabilities of generative AI have prompted a similar spectrum of wonder, excitement, and concern. Managers have raced to embrace AI, fearful that they too could face extinction if they fail to keep pace with the rapid rate of technological progress. Two-thirds of large companies in the UK now report that they use AI, as do more than a third of medium-sized businesses. But to paraphrase Jeff Goldblum, bosses have become so preoccupied with whether or not they could use AI, they didn’t stop to think if they should. Or, more pertinently, how they should. For all its undoubted
Warwick Business School | wbs.ac.uk
wbs.ac.uk | Warwick Business School
52
53
Made with FlippingBook Learn more on our blog