Chevron Corporation’s (CVX), global refining system manufactures fuels and other products sold by the company’s marketing, lubricants, and supply and trading organizations under the Chevron®, Texaco® and Caltex® brands. These refining systems are capable of processing heavy crude oils and producing a variety of high-value products, including transportation fuels, chemicals, and lubricants. This value chain supports 2.4 million barrels/day worth of annual products sold by seven refineries across a network of 8,000 retail stations.
The Commodity Supply Chain Management (CSCM) Data and Insights team is responsible for managing more than 200 data pipelines to ingest data from various internal and external sources and standardize it in the Supply Chain Data Foundation (SCDF). SCDF is central to driving data integration and related initiatives across the many functions responsible for managing the flow of crude oil and refined products, including:
The immediate priority of the SCDF project was to replace the existing legacy on-premise data platform with an equivalent data platform deployed in Microsoft Azure®. The Chevron team envisioned that the migration to Azure could go beyond the Company’s existing data capabilities to enable aspirational and advanced capabilities, such as
The Publicis Sapient Product & Data Engineering team partnered with Chevron CSCM to build a new Platform as a Service (PaaS) solution in Azure to deliver on this vision.
Migrating the data foundation to Azure has substantially:
It gave more than 400 manufacturing and value chain optimization users access to:
Traditional on-premise systems incur significant legacy costs due to:
The new PaaS SCDF platform has enabled significant reduction of these legacy costs. The only costs incurred in the cloud are related to resource consumption. Some cloud services have licensing costs, but those services can be rapidly turned on or off as needed enabling low-stakes, fit-gap analysis as new requirements are detected. The commercial costs and barriers of identifying, assessing, and integrating new technologies are eliminated with the availability of the wide-ranging Azure cloud services.
Before the project, Chevron used an on-premise data warehouse solution. Providing direct user access was cumbersome due to network connectivity restrictions and client software setup issues. The project team deployed Synapse, Azure’s data warehousing and analytics solution, to securely share datasets, provision access to those with data needs, and enable the connection of datasets otherwise impossible.
Queries generally complete 45 percent faster on Synapse compared with the previous on-premise solution with queries frequently timing out or failing. For fixed resource on-premise systems, too many simultaneous or resource intensive queries could cause system-wide performance issues. Dynamic cloud resource allocation allows minimization of disruption costs by scaling the platform to the resource level needs at the moment.
The previous solution required a full-time (Database Administrator) DBA team on staff to oversee 24x7 operational support for database issues, including:
With Synapse, these support activities are covered by Microsoft cloud support, with a smaller local support team focusing on the functional nature of the platform.
Agile work processes are enabled by removing infrastructure/administrative dependencies for simple tasks, such as taking database back-ups for testing purposes. Improved self-sufficiency of developers enables rapid development and deployment cycles, minimizing development cost and time.
Enhancements to data management services such as Azure Data Factory and Synapse are continually deployed by Microsoft and don’t require large-scale planning efforts or costly outages associated with comparable upgrades to on-premise systems.
It gave more than 400 manufacturing and value chain optimization users access to:
More than 200 data integration jobs were converted from the previous solution to Azure Data Factory (ADF). The sequencing of the delivery of these had to be carefully managed due to the internal dependencies of the data platform. However, to enable delivery and a consistent pattern, multiple frameworks were developed early in the project to facilitate:
While the requirements of the timeliness of data depend on the part of the supply chain being supported, critical 24/7 operations like refinery, blending, and scheduling have more strict data needs. In addition, proper design and management of resources in the cloud is crucial to ensure excess Azure costs are not incurred. Major components include:
In addition to storing the data, the data furthermore needed to be transformed and served to the business function needing the data. Streamlined access and proper preparation of the data are crucial to keep the data consumers from experiencing disruption as the system migrates from on-premise to the cloud. Major components include:
In less than one year, the Publicis Sapient delivery and Chevron CSCM Platform teams were tasked to migrate the following components to Azure:
Conversion of these statements to end-state architecture was facilitated by Microsoft Tradurre, a Microsoft product not yet publically available.
With traditional project management techniques, the SCDF project would have taken years. However, the Publicis Sapient team, partnering with the larger Chevron platform team leveraging Microsoft workflow and integration tools like Azure DevOps (ADO), combined the following practices to enable a successful project completion, all in less than one year.
Given the size and scope of the project, the coordination required between the Publicis Sapient delivery team and the larger Chevron CSCM platform team was a significant endeavor. Azure DevOps was used to facilitate both internal iteration planning, in addition to managing cross-team dependencies. More than 20 supply chain application teams were needed to test, plan, and migrate their integrations to the new platform simultaneous with the decommissioning of the legacy on-premise platform. The delivery team embraced Chevron’s newly deployed Scaled Agile process within the CSCM platform in ADO to plan out quarterly PI increments, map dependencies, and monitor execution via ADO dashboards and progress reports to ensure delivery adhered to the schedule.
Agile Program Management and modern DevOps practices go hand-in-hand; one is essential to enable the other and neither are successful independently. While many believe DevOps is a technology or a skillset, the Publicis Sapient delivery team embraced DevOps as a mindset to enable the rapid development, integration, testing, and deployment cycle necessary to meet tight timelines. Without this DevOps mindset, enabled by ADO, Azure Repos, and Azure Automated Deployment Pipelines, the delivery of the platform in less than a year would have been unrealistic.
The new platform was deployed in production using Microsoft Project to coordinate the cutover,” a carefully sequenced set of steps to ensure that the project is deployed simultaneously with all data publishers and consumers reconnecting to the new platform. Remarkably, the team was able to deliver the platform on time at high quality, despite a surge in the Delta variant of COVID-19 during 2021 which, at times, reduced total team capacity by 30 percent as team members took off for illness and to care for their families .
Business Consulting Manager
Let's connect
Senior Client Partner
Let's connect