SPEAKER: The Executive Guide to AI-Assisted Software Development. Why software may finally eat the world. Imagine saving 4,500 years of developer work in just one year. An achievement so astonishing it sounds improbable. Yet, in a LinkedIn article from Amazon's CEO, that's exactly what the company claims to have achieved by integrating AI into their code modernization efforts. This isn't a distant dream. It's a strong example of AI's impact on software production. Currently, American enterprises are pouring nearly half of their IT budgets into application development and support. As highlighted in a recent study on IT spending intentions, these applications are more than just operational efficiency enablers. They're often the main way businesses connect with customers and drive differentiation. The ability to consistently deliver the right digital products quickly and with high quality is a significant competitive weapon. Despite Marc Andreessen's 2011 prediction that software is eating the world, many enterprises are still grappling with the gap between their digital transformation ambitions and the harsh realities of a compressed macroeconomic environment. This tension has left executives in a difficult position. AI-enabled software development could finally bridge this gap, offering a pathway to both reduce decades of accumulated technical debt and spark a wave of digital innovation, all without the need for increased budgets. Importantly, software development doesn't just encompass coding and technical tasks. It covers product engineering and digital business transformation as a whole. As enterprise leaders explore the potential of artificial intelligence, AI, in the software development lifecycle, SDLC, including the full scope of activities involved in crafting digital solutions that deliver tangible value, they're faced with a landscape rich in opportunity, but also rife with complex questions. How can I use AI to supercharge the way digital products are made? How do I safeguard intellectual property and ensure security in this new era? Which tools, methods, and talents should I prioritize as things continue to evolve? How do I measure progress? At Publicis Sapient, we lead large-scale digital transformations for our clients, drawing on our deep, hands-on experience. We've engaged in dozens of enterprise AI deployments to speed application delivery and modernization. We've invested $325 million to transform our parent company, Publicis Group, with AI-enabled software we call Core AI. We are using our Core AI to rigorously test, learn, and distinguish what truly works in our AI-enabled SDLC from theoretical possibilities. These efforts have equipped us with unique insights and practical strategies to help enterprises navigate the AI revolution in software development. In this article, we'll explore the transformative potential of AI in software development, share our most valuable lessons, and answer the critical questions that business and technology executives are asking today. Welcome to the future of software development. Are you ready to lead?
SPEAKER: Understanding AI-Assisted Software Development The concept of AI generating code isn't new. Since Microsoft launched GitHub Copilot in November 2021, developers have been receiving real-time code suggestions and assistance seamlessly integrated into their workspace tools. With over 1.3 million paid subscribers and 50,000 enterprise customers, GitHub Copilot has quickly become the most widely adopted AI-powered developer assistant. But AI-assisted software development is more than just a code suggestion tool in the developer's toolkit or a conversational chatbot like ChatGPT. Its potential is far more profound. By eliminating complexity and human toil, enterprises can slash technical debt and accelerate transformation. Claude 3.5 Sonnet, an OpenAI 01, leading the way in coding. Consider the recent release of Anthropic's latest large language model, LLM, on June 20, 2024. Claude 3.5 Sonnet made headlines for its industry-leading performance across a range of benchmarks, particularly in its ability to write software code. One standout evaluation involved the model's ability to understand an open-source code base and implement a pull request for a bug fix or new feature based solely on a natural language description. The model was then tested on whether all the code base tests, kept hidden from the model, would pass for the completed submission. This wasn't just a trivial exercise. The problems were based on real pull requests, requiring the model to edit multiple files, write, run, and iteratively correct code in a secure, sandboxed environment without internet access. Achieving a new, high-water score on this benchmark is a remarkable feat. It demonstrates AI's growing ability to autonomously manage complex, multi-step software development tasks, from understanding requirements to coding and testing, all without human guidance. The buzz surrounding Claude's release was immediate. Social media lit up with excited posts from developers and enthusiasts alike. A technical founder on Reddit claimed a 10x productivity boost using the model compared to pre-LLM days. Another individual shared their experience in a post on X of deploying a web app using Claude, Repl.it, and Google Firebase after just six days of learning to code. Or, consider the more recent release of OpenAI's 01Preview enhanced reasoning system, formerly known as Project Strawberry or Q-Star, on September 12, 2024. This new model approaches problem solving with a more deliberate, reflective process. Early tests show that 01Preview can analyze, backtrack, and weigh different options before arriving at a solution, significantly reducing the likelihood of errors or inaccurate responses. This makes it well-suited for complex tasks that require planning and iteration, such as enterprise-level code development or large-scale program planning. OpenAI reports that 01 has dramatically improved its code-forces ELO benchmark performance, jumping from the 11th percentile, newbie, to the 89th percentile, or expert level, compared to human developers. These stories are thrilling, and it's easy to get swept up in the idea of a future where the SDLC is fully automated by AI. However, as exciting as these developments are, in an enterprise context it's crucial to separate the reality from the hype. Understanding where AI is genuinely transforming the SDLC and where it falls short requires a balanced perspective.
SPEAKER: Definition and Explanation of AI-Assisted Software Development AI-assisted software development refers to the integration of AI technologies, particularly LLMs, into the software development process to enhance and accelerate various aspects of business and systems analysis, design, coding, testing, deployment, and maintenance. This approach leverages AI to augment labor-intensive tasks, such as analyzing existing artifacts, ideating design alternatives, identifying potential bugs, optimizing performance, and generating code, test cases, and documentation based on natural language descriptions and existing work products.
SPEAKER: Differentiation between AI-assisted and traditional development methods Traditional software engineering relies heavily on human developers. It's true that any modern SDLC already uses automation extensively. DevOps tooling automates repetitive tasks like code integration, regression testing, and deployment, while static code analyzers use predetermined rule sets to analyze and grade code for compliance. Yet, even with these tools, traditional development demands deep domain and technical expertise and involves a significant amount of manual effort throughout the lifecycle. AI-assisted software development, on the other hand, integrates AI, especially LLMs, to more efficiently and accurately perform or assist with tasks that were formerly resistant to automation. LLMs are particularly powerful tools for software tasks because they have been pre-trained on a vast corpus of pre-existing software engineering data. Leveraging their trained models, AI tools like GitHub Copilot can suggest lines of code or even entire functions as developers type, acting as an AI-powered pair programmer. More advanced AI systems such as Claude Sonnet 3.5 and Cognition's Devon go a step further by understanding complex code bases and making sophisticated changes. These systems can implement new features or fix bugs based on high-level instructions by iteratively writing, modifying, and testing the code. The main difference: traditional DevOps and other SDLC tools like code analyzers are rule-based and deterministic, meaning they always produce the same outputs for a given set of inputs. AI tools are probabilistic, with outputs that can vary depending on the data they were trained on and the specific inputs, such as the prompts and context they receive. The implication: while AI-generated code can speed up development, it may not always meet the cleanliness or standards expected in an enterprise environment due to its training or the quality of inputs. What does this mean for IT leaders? A skilled developer with a deep understanding of the business domain and development frameworks can use AI as a true force multiplier in improving efficiency and quality. Without this expert human steering the AI and critically examining its outputs, the gains are quickly lost, and either productivity or quality, or both, are compromised. Therefore, integrating AI into the SDLC must be done with careful consideration and continuous skilled human oversight.
SPEAKER: The top AI use cases for developers. According to a recent Stack Overflow survey, the top three AI use cases for developers are writing code, searching for answers, debugging. In fact, while AI tools are often used by software developers for quick answers and assistance with small code sections, the same survey also reported that 38% of developers report code assistance, provide inaccurate information half of the time or more. There are new code assistant tools being released every day, though ChatGPT and GitHub Copilot dominate the space, with the survey showing GitHub Copilot and Anthropic's Claude rapidly growing in usage. Figure 1 shows that the majority of developers currently use ChatGPT, but plan to use GitHub Copilot long-term. However, as we will discuss next, the potential for AI-assisted software development is much larger than developer assistance and code completion.
SPEAKER: Capturing Value from AI-Assisted Software Development The Key Benefits of AI-Assisted Software Development Many organizations may already be using AI tools for code suggestions, defect detection, and bug fixing. However, the opportunity space for value creation is much larger. Currently, software development is a labor-intensive process fraught with human toil. Consider the following challenges:
SPEAKER: The need for specialized tools and platforms for AI-assisted software development. The potential for AI to transform software development is immense, but how can enterprises truly harness this power? Is it as simple as deploying a code completion tool or offering a secure version of a leading LLM chatbot with some user training? While those tools are logical to start with, our experience is that they only scratch the surface and fail to maximize the achievable gains in value, productivity, and quality. Why? Special forces teams such as Navy SEALs or Green Berets don't use standard-issue equipment when embarking on critical missions. They rely on customized, mission-specific tools, whether it's modified weapons, specialized communication devices, or advanced surveillance technology. These tools are meticulously designed and adapted to meet the exact demands of their mission, allowing them to operate with precision and effectiveness in high-stakes environments.
SPEAKER: The Jagged Frontier of AI Capabilities To appreciate why this relates to AI in the SDLC, it's essential to grasp when LLMs like GPT-4 benefit knowledge workers and when they are harmful. The concept of the Jagged Frontier, introduced in a Harvard Business School, HBS, and Boston Consulting Group, BCG, paper, offers a vivid metaphor for understanding what happens when chat-based AI tools are offered to skilled professionals. Picture this. A fortress wall with towers and battlements of varying heights, representing AI's uneven proficiency across different tasks. Within the fortress walls, AI excels, leading to significant productivity and quality gains for the BCG consultants who were studied. Beyond those walls, however, AI struggles and can provide incorrect or misleading information, leading to the BCG consultants being 19% less likely to produce correct solutions compared to those not using AI. This Jagged Frontier serves as a cautionary tale for those relying on out-of-the-box LLM chatbots or code completion tools. The metaphor reminds us that while AI can greatly assist with tasks it's well-suited for, it can also be harmful when applied to tasks beyond its current capabilities.
SPEAKER: The power of fine-tuned AI Unfortunately, for the highest-value software development use cases, general-purpose LLMs lack the specific domain and enterprise context required to be genuinely effective in enterprise environments. In effect, many valuable SDLC use cases are outside the castle walls, beyond the jagged frontier. In software development, the real power of AI emerges when LLMs are fine-tuned with unique enterprise knowledge, use-case-specific context, and by chaining custom task-specific agents and guardrails. That fine-tuning brings the high-value use cases within the castle walls.
SPEAKER: Why Prompt Engineering is Not Enough Fine-tuning LLMs with solution patterns, enterprise context, and guardrails isn't just a nice-to-have, it's crucial for generating enterprise-ready AI outputs. This is simply not possible today with general-purpose LLMs. Some may argue that well-crafted AI prompts can contain all the enterprise context needed to allow the AI to make a useful response. Our experience has shown that, while large context windows in modern LLMs allow for generously long prompts, attempting to include all the context in the prompt alone can lead to surprisingly inconsistent results, more frequent hallucinations, and diluted relevance. Simply put, relying solely on prompt engineering or vanilla code completion without deeper customization will not, as of this writing, yield the best AI outputs for enterprise software development.
SPEAKER: How Specialized Tools and Platforms Help To address this gap, we've adopted a new approach, creating task-specific, workflow-enhanced acceleration products that integrate fine-tuned models, enriched with domain, use case, and enterprise context, alongside curated prompt libraries and pre-built guardrails. Enterprise software development challenges demand more than generic, off-the-shelf solutions. While general-purpose LLMs are powerful, they lack the necessary task-specific and workflow context, which can't be addressed through prompt engineering alone. AI-assisted software development requires tailored acceleration products, much like how special forces customize gear for specific missions. These products leverage fine-tuned AI models, enriched with relevant context, and include built-in safeguards. This ensures teams can confidently produce outputs that align with best practices for their enterprise solutions.
SPEAKER: Examples of fine-tuned AI in action. For example, an AI trained on your visual design system can be a game-changer for experienced designers and front-end developers, ensuring that AI outputs align perfectly with your brand and front-end code standards. Similarly, an AI model familiar with your enterprise architecture and APIs can ensure that its outputs make correct use of your IT foundation, driving efficiency and consistency as it produces and inspects code and test cases. An AI model trained on the nuances of moving an Adobe Experience Manager implementation to Adobe's cloud service can now assist with such migrations repeatedly and accurately, ensuring its outputs correctly use the architectural patterns consistent with enterprise standards. Case in point, Amazon used a tuned version of its AI product called Q to upgrade their foundation software applications to Java 17. They observed that by curating the migration pattern and tools with Q, the average time to upgrade an application plummeted from what's typically 50 developer days to just a few hours. Adaptability is the key to success. In both the battlefield and the business world, adaptability to the environment and mission context is crucial. The key to building the right AI tools for enterprise software development is identifying the different archetypes or solution patterns where the AI-powered toolset must be precision-built. To ensure maximum productivity and quality, it's crucial to invest in expert AI toolmakers to build the tooling specific for each archetype. The toolmakers ensure that the domain expertise and nuances of the archetype and technology stack are built directly into the AI toolset, thus ensuring the relevance and safety of AI outputs. Each archetype has its own AI toolset, which is managed as a product to maximize the productivity and quality of the software development teams building solutions of that type.
SPEAKER: Issues, risks, and mitigations with AI-assisted software development. Many books will be written over the next few years on the risks of generative AI. When using AI for software development, we can narrow our risk surface area with a product-driven approach to AI tools. The product teams responsible for engineering the precision tools must make sound decisions to address the following issues and safeguard the enterprise. Are AI outputs explainable? Explainable AI is critical because the less we make generative AI a black box and the more we understand not just the output but why it chose that output, the better. This is especially true when we use AI for critical enterprise applications. For example, AI may produce code that works, but developers can't fully understand how or why it was generated that way. AI tools might flag potential bugs without clearly explaining the reasoning behind the detection. AI-assisted design tools could make choices that designers can't easily justify or explain to stakeholders. Similarly, AI-generated requirements may not provide clear rationales to product managers for why certain scenarios were chosen. This risk can be mitigated by various techniques, such as asking the AI to explain different sections of its outputs, comparing the output of one AI model with another, and asking for explanations of key differences, or using chain-of-thought prompting. Chain-of-thought prompting encourages AI models to break down complex problems into a series of intermediate steps, mimicking human-like reasoning. This approach involves providing examples that demonstrate step-by-step reasoning, thus guiding the model to generate similar reasoning chains for new tasks. This enables the model to articulate its thought process, leading to more transparent and interpretable outputs.
SPEAKER: Advancing AI Explainability How OpenAI's O1 Preview Model Enhances Reasoning and Accuracy OpenAI's latest O1 Preview Model achieves superior reasoning through a combination of reinforcement learning and chain-of-thought reasoning. Reinforcement learning enables the model to continuously refine its problem-solving strategies as it explores constraints, learning from mistakes, and adjusting its approach to deliver more accurate and logical outcomes. By explicitly mapping out its reasoning process, O1 not only explains its output but also identifies its own potential errors and addresses them, significantly improving the likelihood of arriving at the correct solution, much like how humans reduce mistakes when we carefully outline our thinking. We believe O1 is a significant step forward in improving AI explainability, particularly for our SDLC use cases.
SPEAKER: Security and Disclosure All enterprises want their secrets to be, well, secret. For many enterprises, it is sufficient to ensure that the enterprise legal agreements with the LLM API provider, like Microsoft, Google, OpenAI1. Transforming IT capabilities. The greatest lift in creating differentiated transformation capabilities lies in skills, talent, and organizational transformation. The professionals in our organizations who conceive and build software across strategy, product experience, engineering, and data, and AI speed must embrace AI as part of how they work, transforming into cyborgs that seamlessly blend their own intelligence with AI to multiply their efficiency and effectiveness. This is not just an individual endeavor, but rather one that leaders thoughtfully design, incorporating new roles to support evolving workflows. For example, the myriad technical specialist skill sets in IT evolve into a pool of versatile AI engineers who seamlessly use AI to build solutions across multiple technologies. These AI engineers are supported by a much smaller cohort of deep AI software development product specialists who design workflows and build the tools that the multi-skilled AI engineers use. IT leadership must reimagine and redesign their capabilities, starting with workflow, organization, and technology enablers to build this future.
SPEAKER: Moving from general contractors to inventors. IT needs to focus on more than just creating, testing, and launching software, which is often referred to as moving from backlog to live. With the speed at which software can now be developed, the real challenge in creating unique value will shift. Instead of just focusing on writing and deploying code, companies will need to pay more attention to how they come up with new ideas, create strategies, and design experiences. Just as a chain is only as strong as its weakest link, the software development process is only as fast and efficient as its slowest parts. IT leaders must rethink and improve the entire process, from coming up with ideas to managing them and finally getting them into production. In essence, this is about realizing the full promise of digital business transformation through product engineering, of which software development is a part.
SPEAKER: Exploiting proprietary assets. In May 2023, an internal document from a Google employee was leaked and published online in a memo discussing the lack of competitive advantage in LLMs. The memo argued that neither Google nor OpenAI had a sustainable competitive advantage, or moat, in LLMs compared to open-source efforts using internet datasets. It was controversial when published, but has been seen by some as prescient given subsequent developments in open-source AI models. We can already see the lack of sustained differentiation play out in the rapid race for model supremacy amongst OpenAI, Google, Meta, Anthropic, and others. In contrast, large enterprises have a significant moat, proprietary corporate data, knowledge assets, and expertise. These can be used to build specific tools fine-tuned for creating accretive enterprise value. Investing in data curation and model fine-tuning will yield significant acceleration compared to using public models alone. Training employees on prompting for the unique capabilities of these fine-tuned models will yield further differentiation.
SPEAKER: Conclusion. The new AI frontier in software development. As we look ahead to the evolving landscape of software development, it's clear that AI-assisted innovation is set to redefine the way we work. The strategies we've discussed highlight how AI can drive efficiency, improve quality, and support growth. Rather than being just another tool, AI is becoming a crucial element in digital transformation. Now is the time for leaders to explore this potential, guide their organizations through change, and unlock the new possibilities that AI brings to the future of software development.
SPEAKER: How Publicis Sapient can help. A successful AI approach is built on digital business transformation, and DBT is at our core. Our team helps prepare for AI and generative AI implementation, evaluate and prioritize use cases, and execute the right strategy for your business. Recognized as a 2023 market leader in generative enterprise services by HFS Research, our teams have experience designing and building innovative AI solutions. Together, we can unlock the full value of AI with an ethical and sustainable approach. Discover Sapient AI, Publicis Sapient's ethos for purpose-driven and scalable AI solutions. Sapient Slingshot. Sapient Slingshot is our proprietary AI toolset designed to enhance software development across all SDLC stages. It provides tools for code generation, testing, deployment, and more, backed by our industry-specific data and prompt library. Beyond the tools, our experienced team supports implementation and customization to meet your needs, helping engineers, product managers, scrum masters, and designers deliver better, faster, and more cost-effective solutions. AI Application Modernization. Boost operational efficiency and enable innovation by modernizing your legacy systems with the help of AI. Accelerate developer productivity, enabling faster code transition, documentation, and automated testing for scalable growth. AI Custom Application Development. Create differentiated and dynamic experiences through application development using AI. Code, test, and deploy at an accelerated rate, allowing you to focus on identifying the features your customers want, while continuously optimizing and innovating. AI Martech Transformation. Drive richer customer connections and efficiency within your marketing organization enabled by a leading Martech stack infused with AI. AI Test Automation. Enhance software quality and accelerate release cycles with AI-powered test automation. Streamline test creation, execution, and maintenance, enabling rapid identification and resolution of issues. This article was written by Sheldon Monteiro, Executive Vice President and Chief Product Officer at Publicis Sapient. To learn more about Publicis Sapient's approach to AI-assisted software development, check out Sapient AI for Applications.