Play podcast loading duration...
Imagine saving 4,500 years of developer work in just one year—an achievement so astonishing it sounds improbable. Yet, in a LinkedIn article from Amazon’s CEO, that’s exactly what the company claims to have achieved by integrating AI into their code modernization efforts. This isn’t a distant dream; it’s a strong example of AI's impact on software production.
Currently, American enterprises are pouring nearly half of their IT budgets into application development and support, as highlighted in a recent study on IT spending intentions. These applications are more than just operational efficiency enablers—they’re often the main way businesses connect with customers and drive differentiation. The ability to consistently deliver the right digital products, quickly and with high quality, is a significant competitive weapon.
Despite Marc Andreessen's 2011 prediction that "software is eating the world," many enterprises are still grappling with the gap between their digital transformation ambitions and the harsh realities of a compressed macroeconomic environment. This tension has left executives in a difficult position. AI-enabled software development could finally bridge this gap, offering a pathway to both reduce decades of accumulated technical debt and spark a wave of digital innovation—all without the need for increased budgets.
Importantly, software development doesn’t just encompass coding and technical tasks, it covers product engineering and digital business transformation as a whole. As enterprise leaders explore the potential of artificial intelligence (AI) in the software development lifecycle (SDLC), including the full scope of activities involved in crafting digital solutions that deliver tangible value, they're faced with a landscape rich in opportunity but also rife with complex questions:
We've invested $325 million in a platform we call “CoreAI” to transform our parent company, Publicis Groupe. These efforts have equipped us with unique insights and practical strategies to help enterprises navigate the AI revolution in software development.
The concept of AI generating code isn’t new. Since Microsoft launched GitHub Copilot in November 2021, developers have been receiving real-time code suggestions and assistance, seamlessly integrated into their workspace tools. With over 1.3 million paid subscribers and 50,000 enterprise customers, GitHub Copilot has quickly become the most widely adopted AI-powered developer assistant.
But AI-assisted software development is more than just a code suggestion tool in the developer’s toolkit or a conversational chatbot like ChatGPT. Its potential is far more profound—by eliminating complexity and human toil, enterprises can slash technical debt and accelerate transformation.
Consider the recent release of Anthropic’s latest large language model (LLM), Claude 3.5 Sonnet, on June 20, 2024. Claude 3.5 Sonnet made headlines for its industry-leading performance across a range of benchmarks, particularly in its ability to write software code. One standout evaluation involved the model’s ability to understand an open-source codebase and implement a pull request for a bug fix or new feature based solely on a natural language description. The model was then tested on whether all the codebase tests—kept hidden from the model—would pass for the completed submission. This wasn’t just a trivial exercise; the problems were based on real pull requests, requiring the model to edit multiple files, write, run and iteratively correct code in a secure, sandboxed environment without internet access. Achieving a new high-water score on this benchmark is a remarkable feat. It demonstrates AI’s growing ability to autonomously manage complex, multistep software development tasks, from understanding requirements to coding and testing, all without human guidance.
The buzz surrounding Claude’s release was immediate. Social media lit up with excited posts from developers and enthusiasts alike. A technical founder on Reddit claimed a 10x productivity boost using the model compared to pre-LLM days. Another individual shared their experience in a post on X of deploying a web app using Claude, Replit and Google Firebase after just six days of learning to code.
Or, consider the more recent release of OpenAI’s “o1-preview” enhanced reasoning system, formerly known as “Project Strawberry/Q,” on September 12, 2024. This new model approaches problem-solving with a more deliberate, reflective process. Early tests show that o1-preview can analyze, backtrack and weigh different options before arriving at a solution, significantly reducing the likelihood of errors or inaccurate responses. This makes it well suited for complex tasks that require planning and iteration—such as enterprise-level code development or large-scale program planning. OpenAI reports that o1 has dramatically improved its Codeforces Elo benchmark performance, jumping from the 11th percentile, “Newbie,” to the 89th percentile, or “Expert” level, compared to human developers.
These stories are thrilling, and it’s easy to get swept up in the idea of a future where the SDLC is fully automated by AI. However, as exciting as these developments are, in an enterprise context it’s crucial to separate the reality from the hype. Understanding where AI is genuinely transforming the SDLC—and where it falls short—requires a balanced perspective.
AI-assisted software development refers to the integration of AI technologies, particularly LLMs, into the software development process to enhance and accelerate various aspects of business and systems analysis, design, coding, testing, deployment and maintenance. This approach leverages AI to augment labor-intensive tasks, such as analyzing existing artifacts, ideating design alternatives, identifying potential bugs, optimizing performance and generating code, test cases and documentation based on natural language descriptions and existing work products.
Traditional software engineering relies heavily on human developers. It’s true that any modern SDLC already uses automation extensively: DevOps tooling automates repetitive tasks like code integration, regression testing and deployment, while static code analyzers use predetermined rulesets to analyze and grade code for compliance. Yet, even with these tools, traditional development demands deep domain and technical expertise and involves a significant amount of manual effort throughout the lifecycle.
AI-assisted software development, on the other hand, integrates AI, especially LLMs, to more efficiently and accurately perform or assist with tasks that were formerly resistant to automation. LLMs are particularly powerful tools for software tasks because they have been pretrained on a vast corpus of preexisting software engineering data. Leveraging their trained models, AI tools like GitHub Copilot can suggest lines of code or even entire functions as developers type, acting as an AI-powered pair programmer. More advanced AI systems, such as Claude Sonnet 3.5 and Cognition’s Devin, go a step further by understanding complex codebases and making sophisticated changes. These systems can implement new features or fix bugs based on high-level instructions by iteratively writing, modifying and testing the code.
The main difference: Traditional DevOps and other SDLC tools, like code analyzers, are rule-based and deterministic, meaning they always produce the same outputs for a given set of inputs. AI tools are probabilistic, with outputs that can vary depending on the data they were trained on and the specific inputs, such as the prompts and context they receive.
The implication: While AI-generated code can speed up development, it may not always meet the cleanliness or standards expected in an enterprise environment due to its training or the quality of inputs.
What does this mean for IT leaders? A skilled developer with a deep understanding of the business domain and development frameworks can use AI as a true force multiplier in improving efficiency and quality. Without this expert human steering the AI and critically examining its outputs, the gains are quickly lost, and either productivity or quality, or both, are compromised. Therefore, integrating AI into the SDLC must be done with careful consideration and continuous skilled human oversight.
According to a recent Stack Overflow survey, the top three AI use cases for developers are:
In fact, while AI tools are often used by software developers for quick answers and assistance with small code sections, the same survey also reported that 38 percent of developers report code assistants provide inaccurate information half of the time or more. There are new code assistant tools being released every day, though ChatGPT and GitHub Copilot dominate the space, with the survey showing GitHub Copilot and Anthropic’s Claude rapidly growing in usage. However, as we will discuss next, the potential for AI-assisted software development is much larger than developer assistants and code completion.
Many organizations may already be using AI tools for code suggestions, defect detection and bug fixing. However, the opportunity space for value creation is much larger. Currently, software development is a labor-intensive process fraught with human toil. Consider the following challenges:
Carefully designed and implemented AI interventions can dramatically reduce the human toil in coping with every one of these challenges. By doing so, enterprises can maximize the value they realize from their application development and maintenance efforts—activities that consume an average of 45 percent of IT budgets, according to Gartner 2023 research.
This emphasis on code completion and the developer role alone obscures a larger opportunity. At Publicis Sapient, we've discovered that leveraging AI across the entire SDLC, as well as across Strategy, Product, Experience, Engineering and Data & AI (SPEED) disciplines, can drive up to a 40 percent increase in productivity. Yet, less than half of this gain comes from developer activities. Where does it come from? Significant potential lies in areas like strategic planning, design and release management—often overlooked due to the spotlight on tools like GitHub Copilot.
By focusing on the full SDLC from concept to production, we’re not just speeding up code development; we’re accelerating the entire process of delivering customer and business value, adapting swiftly to feedback and maximizing impact by trimming low-value features. This holistic approach is crucial for truly agile and effective software development:
These gains present an opportunity to nearly double the rate of digital innovation while keeping budgets flat.
To capture economic value, we apply lean thinking across the complete SDLC, combining value stream mapping and a catalog of targeted AI-enabled interventions. These are the same techniques that made the Toyota Production System famous—applied to the SDLC process. When prioritizing where to start introducing AI into the workflows, it’s essential to prioritize, plan and measure the impact of the interventions. For example, in one set of measurements with GitHub Copilot, we see the highest gains with AI assistance during code creation in Java and React, with approximately 50 percent improvements, while with code optimization and defect fixing, gains are more modest, varying between 15 to 25 percent. Finally, it is important to plan for some activities that take more time. For example, generating code might be a lot faster but we might spend a bit more time on ensuring explainability, tests and verification.
In our experience, diligent measurement of speed and quality is essential for optimizing AI-enabled workflows. Relying solely on AI tool analytics is insufficient; it’s important to track detailed metrics across the entire SDLC to understand where productivity and quality improvements are truly occurring. For example, at this time, the REST API for GitHub Copilot usage metrics provides only aggregated data, such as accepted code suggestions, which is insufficient to examine individual trends or those for each scrum team. It provides insight only into tool usage and not the workflow in which it is being used. PS KnowHOW is our open-source metrics platform used to diligently measure throughout the SDLC. It gives us real-time visibility into cycle time, quality and value metrics, including the ability to measure the impact of AI interventions.
AI benefits more than just developers. A transformative benefit of AI is its capability to connect, cross-analyze and generate artifacts throughout the SDLC for different roles, including strategists, product managers, agile program managers, experience designers and data scientists, which can further strengthen collaboration and innovation. AI can help maintain alignment between strategy, design and implementation, ensuring that the final product adheres closely to original business and architectural intentions—a critical advantage in complex systems where manual oversight often falls short.
AI can also offer real-time feedback and mentoring, help practitioners refine their skills and avoid common mistakes, create first drafts and review work. For example, solution product managers can leverage AI to generate or critique backlogs and test cases, while developers can utilize AI as a peer programmer, enhancing both individual and team productivity.
It's not just new application development and routine maintenance that stand to gain. Many large enterprises, particularly those with decades of history, struggle with outdated legacy systems. Simply lifting and shifting to the cloud is not a viable option for many of these systems. Modernizing them to cloud-native technology stacks has, until now, been anything but trivial. We've conducted over 100 end-to-end legacy transformation experiments using our PS Monarch AI-powered modernization accelerator, achieving:
The potential for AI to transform software development is immense, but how can enterprises truly harness this power? Is it as simple as deploying a code completion tool or offering a secure version of a leading LLM chatbot with some user training? While those tools are logical to start with, our experience is that they only scratch the surface and fail to maximize the achievable gains in value, productivity and quality. Why? Special Forces teams, such as Navy SEALs or Green Berets, don’t use standard-issue equipment when embarking on critical missions. They rely on customized, mission-specific tools—whether it’s modified weapons, specialized communication devices or advanced surveillance technology. These tools are meticulously designed and adapted to meet the exact demands of their mission, allowing them to operate with precision and effectiveness in high-stakes environments.
To appreciate why this relates to AI in the SDLC, it's essential to grasp when LLMs like GPT-4 benefit knowledge workers and when they are harmful. The concept of the "jagged frontier," introduced in a Harvard Business School (HBS) and Boston Consulting Group (BCG) paper, offers a vivid metaphor for understanding what happens when chat-based AI tools are offered to skilled professionals.
This jagged frontier serves as a cautionary tale for those relying on out-of-the-box LLM chatbots or code completion tools. The metaphor reminds us that while AI can greatly assist with tasks it’s well suited for, it can also be harmful when applied to tasks beyond its current capabilities.
Unfortunately, for the highest-value software development use cases, general-purpose LLMs lack the specific domain and enterprise context required to be genuinely effective in enterprise environments. In effect, many valuable SDLC use cases are outside the castle walls—beyond the jagged frontier. In software development, the real power of AI emerges when LLMs are fine-tuned with unique enterprise knowledge, use case-specific context and by chaining custom task-specific agents and guardrails. That fine-tuning brings the high-value use cases within the castle walls.
Fine-tuning LLMs with solution patterns, enterprise context and guardrails isn’t just a nice-to-have—it’s crucial for generating enterprise-ready AI outputs. This is simply not possible today with general-purpose LLMs. Some may argue that well-crafted AI prompts can contain all the enterprise context needed to allow the AI to make a useful response. Our experience has shown that while large context windows in modern LLMs allow for generously long prompts, attempting to include all the context in the prompt alone can lead to surprisingly inconsistent results, more frequent hallucinations and diluted relevance. Simply put, relying solely on prompt engineering or vanilla code completion without deeper customization will not, as of this writing, yield the best AI outputs for enterprise software development.
To address this gap, we’ve adopted a new approach: creating task-specific, workflow-enhanced acceleration products that integrate fine-tuned models enriched with domain, use case, and enterprise context, alongside curated prompt libraries and pre-built guardrails. Enterprise software development challenges demand more than generic, off-the-shelf solutions. While general-purpose LLMs are powerful, they lack the necessary task-specific and workflow context, which can’t be addressed through prompt engineering alone. AI-assisted software development requires tailored acceleration products, much like how Special Forces customize gear for specific missions. These products leverage fine-tuned AI models enriched with relevant context and include built-in safeguards. This ensures teams can confidently produce outputs that align with best practices for their enterprise solutions.
For example, an AI trained on your visual design system can be a game changer for experience designers and front-end developers, ensuring that AI outputs align perfectly with your brand and front-end code standards. Similarly, an AI model familiar with your enterprise architecture and APIs can ensure that its outputs make correct use of your IT foundation, driving efficiency and consistency as it produces and inspects code and test cases. An AI model trained on the nuances of moving an Adobe Experience Manager implementation to Adobe’s cloud service can now assist with such migrations repeatedly and accurately, ensuring its outputs correctly use the architectural patterns consistent with enterprise standards. Case in point: Amazon used a tuned version of its AI product called Q to upgrade their foundation software applications to Java 17. They observed that by curating the migration pattern and tools with Q, the average time to upgrade an application plummeted from what’s typically 50 developer days to just a few hours.
In both the battlefield and the business world, adaptability to the environment and mission context is crucial. The key to building the right AI tools for enterprise software development is identifying the different archetypes or solution patterns where the AI-powered toolset must be precision-built to ensure maximum productivity and quality. It’s crucial to invest in expert AI toolmakers to build the tooling specific for each archetype. The toolmakers ensure that the domain expertise and nuances of the archetype and technology stack are built directly into the AI toolset, thus ensuring the relevance and safety of AI outputs. Each archetype has its own AI toolset, which is managed as a product, to maximize the productivity and quality of the software development teams building solutions of that type.
Many books will be written over the next few years on the risks of generative AI. When using AI for software development, we can narrow our risk surface area with a product-driven approach to AI tools. The product teams responsible for engineering the precision tools must make sound decisions to address the following issues and safeguard the enterprise:
As we navigate the future of AI-assisted software development, the role of enterprise IT is more critical than ever. The opportunity is not just about reducing costs or replacing human effort but about unlocking new opportunities for growth and innovation. To prepare for the future of AI-assisted software development, business and technology leaders must invent the future.
In the future, we can assume that humans will build more software. This article on Jevons paradox and its implications for software development explains why this is inevitable. It describes the counterintuitive situation where technological advancements that make a resource more efficient actually lead to greater overall consumption of that resource, driven by increased demand, rather than a reduction in its use. For example, improvements in fuel efficiency led to more driving and thus higher overall fuel consumption. Advances in storage technology lower the cost per gigabyte, but the ease of storing large amounts of data encourages more data creation and retention, leading to increased storage infrastructure and resources to power and manage it. If Jevons paradox holds for AI-enabled software development, as it has with countless other technologies, those businesses that redouble their investments in digital business transformation will have far better odds of maintaining and growing their competitive edge. Source: Matt Rickard via Substack
This brings us full circle to Marc Andreessen’s famous observation that “software is eating the world.” While his idea captures the imperative for digital business transformation, it raises a crucial question: If software is indeed taking over, why does it seem to take so long to reach its full potential? More importantly, if software were truly to dominate, what would drive this transformation, and what would our world look like? The opportunity to answer these questions is finally available to every business executive, not just those with the resources of digital natives. Taking a leadership position will require a focus on:
The greatest lift in creating differentiated transformation capabilities lies in skills, talent and organizational transformation. The professionals in our organizations who conceive and build software across Strategy, Product, Experience, Engineering and Data & AI (SPEED) must embrace AI as part of how they work—transforming into cyborgs that seamlessly blend their own intelligence with AI to multiply their efficiency and effectiveness. This is not just an individual endeavor, but rather one that leaders thoughtfully design, incorporating new roles to support evolving workflows. For example, the myriad technical specialist skill sets in IT evolve into a pool of versatile AI engineers who seamlessly use AI to build solutions across multiple technologies. These AI engineers are supported by a much smaller cohort of deep AI software development product specialists who design workflows and build the tools that the multiskilled AI engineers use. IT leadership must reimagine and redesign their capabilities, starting with workflow, organization and technology enablers to build this future.
IT needs to focus on more than just creating, testing and launching software, which is often referred to as moving from “backlog to live.” With the speed at which software can now be developed, the real challenge in creating unique value will shift. Instead of just focusing on writing and deploying code, companies will need to pay more attention to how they come up with new ideas, create strategies and design experiences. Just as a chain is only as strong as its weakest link, the software development process is only as fast and efficient as its slowest parts. IT leaders must rethink and improve the entire process, from coming up with ideas to managing them and finally getting them into production. In essence, this is about realizing the full promise of digital business transformation through product engineering, of which software development is a part.
In May 2023, an internal document from a Google employee was leaked and published online in a memo discussing the lack of competitive advantage in LLMs. The memo argued that neither Google nor OpenAI had a sustainable competitive advantage (or “moat”) in LLMs compared to open-source efforts using internet datasets. It was controversial when published but has been seen by some as prescient given subsequent developments in open-source AI models. We can already see the lack of sustained differentiation play out in the rapid race for model supremacy amongst OpenAI, Google, Meta, Anthropic and others.In contrast, large enterprises have a significant moat—proprietary corporate data, knowledge assets and expertise. These can be used to build specific tools fine-tuned for creating accretive enterprise value. Investing in data curation and model fine-tuning will yield significant acceleration compared to using public models alone. Training employees on prompting for the unique capabilities of these fine-tuned models will yield further differentiation.
As we look ahead to the evolving landscape of software development, it's clear that AI-assisted innovation is set to redefine the way we work. The strategies we've discussed highlight how AI can drive efficiency, improve quality and support growth. Rather than being just another tool, AI is becoming a crucial element in digital transformation. Now is the time for leaders to explore this potential, guide their organizations through change and unlock the new possibilities that AI brings to the future of software development.
To maximize the potential of AI, businesses first need to embrace digital business transformation (DBT). At Publicis Sapient, we specialize in guiding companies through DBT, so that they can apply AI across all areas of their business. Recognized as a 2023 Market Leader in Generative Enterprise Services by HFS Research, our teams use AI to transform business models, accelerate time to market and reimagine customer and employee experiences, among many other objectives of our clients’ DBT journeys.
Sapient Slingshot is our proprietary AI platform designed to accelerate software development across all stages of the software development lifecycle (SDLC). The platform supports code generation, testing, deployment and more, powered by agentic AI and an enterprise code library with industry context. When paired with our expert engineers, the platform delivers faster, higher-quality solutions.
Leverage AI to transition your legacy systems to modern, scalable architectures. Using a blend of human expertise and AI-assisted collaboration, we accelerate code migration, streamline documentation and automate testing. This solution enables faster, more reliable transitions, and sets your business up for growth with a flexible and future-ready system.
Develop custom applications with AI assistance every step of the way. Code, test and deploy at an accelerated rate, allowing you to focus on identifying and delivering the features your end users want.
Streamline your MarTech transformation by migrating your platforms, like Adobe Experience Manager, to the cloud with an AI-powered, agile approach. Seamlessly transition your code and configurations for enhanced performance and scalability.
Improve your testing speed, coverage and accuracy while reducing defects. We’ll create a modular testing framework that adapts to your quality and development engineering processes.
Sheldon Monteiro
EVP, Chief Product Officer
Let's connect
Download Report