Advances in analytics and predictive modelling are helping to maximise company performance.
Big Data has emerged as a powerful disruptor across industries as diverse as healthcare, media, and retail. In biology, decoding and sequencing the human genome once took over a decade of research; by 2020, innovations in data analysis have cut this down to barely a day. Big Data is now so well integrated into social media giants that hundreds of terabytes are collected and mined every day.
For oil and gas, data is the sector’s lifeblood. Huge volumes are generated across the entire value chain, and the trend is only accelerating. However, with ever-increasing amounts of data, oil and gas companies are often ill-equipped to leverage the benefits. Legacy analysis tools and storage capabilities are often no longer fit for purpose, unable to produce meaningful results or meet the challenges of scalability.
“Most of these companies have been sitting on huge amounts of data in silos, whether exploration, operations, finance, or marketing data on the downstream side,” says Mukul Kapur, Senior Vice President Digitalisation and Innovation at S&P Global. “There was no incentive for companies to go back and drive insights before. But with digitalisation, the need for speed and more pressure on margins, what was on the periphery is now in full focus.”
Merging Big Data analytics effectively into company operations can deliver powerful new business insights, improve net margins, and drive efficiencies. While Big Data may have been a niche tool for oil and gas in the past, volatile market conditions have made the technology ever more critical to capture operational insights.
Companies are increasingly realising the benefits of adopting Big Data solutions. The technology was ranked number one for planned investment over the next 18 months, according to the results of our digitalisation survey. Respondents also expected the technology to generate the best returns over the same time period. Big Data and analytics are ranked highest for both investment and benefit, followed by mobile devices, digital marketing and consumer insights, AI/machine learning, robotics and automation, blockchain, and wearable technology.
The Middle East and Africa was the sole regional exception, with respondents citing digital marketing as their main investment focus and Big Data in third position. However, even this could be slightly misleading given the growing importance of Big Data within digital marketing. In the Middle East, where lifting costs are low, the demand side is often a core focus. “Oil companies are talking about the upstream side, where you want your core analytics to become more cost efficient, as well as the Big Data from the consumer to be able to personalise the marketing,” says Musad Haq, Senior Vice President at Publicis Sapient. “For companies in the Middle East and Africa, the bigger focus is the demand side and Big Data will also be behind their marketing.”
Many oil and gas firms are already harnessing the benefits of Big Data to drive company performance. US major ExxonMobil applied Big Data in the development of its offshore Guyana project by merging data workflows with AI. The producer collaborated with IBM to turn seismic data into a workable model to test the commercial payoff of hydrocarbon fields. ExxonMobil achieved first oil at its Liza project, offshore Guyana, ahead of schedule in December and is now moving towards deploying more floating production storage and offloading vessels at the field. The company’s leading use of data has seen 18 major oil discoveries in the basin since 2015 and will continue to drive startup of other fields in the basin. The AI project saved 40% in data processing time at Liza, significantly accelerating the project.
IBM has partnered with many other upstream companies to leverage their data. Australian independent Woodside Energy collaborated with IBM to extract meaningful insights from 30 years of complex drilling data and unstructured documentation. The producer realised that, with the sheer volume of documentation from each project, trawling through the documentation was becoming arduous and wasteful. Each new project relied on the experience and expertise of senior engineers who had accumulated decades of knowledge. But as staff retired or moved on, the knowledge accrued was at risk of being lost. To fast-track projects and remain industry competitive, Woodside needed to tap into its engineering treasure trove and become proactive rather than reactive.
Using machine learning and text analysis, 33,000 technical documents from past projects were made easily accessible to staff, allowing new insights to be gleaned. The result was $7 million in savings from lowered employee costs and a 75% reduction in time spent trawling through extensive documentation.
Low-cost development and improved agility are driving companies to third-party providers.
Advancements in cloud-based computing are transforming the capabilities of oil and gas companies. Legacy storage systems unable to cope with accelerating data collection are no longer major barriers to uncovering valuable business insights. Cloud solutions are allowing firms to tap into powerful performance tools like Big Data, AI, and machine learning.
Scalability and flexibility are among the clearest benefits to the oil and gas sector. Cloud computing lets firms build custom-built solutions at pace and without the heavy expense of designing internal infrastructure. As data demand increases, companies can scale their models at ease or run multiple projects in parallel, benefitting from the enormous computing power of the cloud.
“In a cloud platform, you are able to easily bring in data and spin up an analytics platform for a few hours,” says Musad Haq, Senior Vice President at Publicis Sapient. “You would not be able to do this from your own infrastructure without spending a huge amount of money. Cloud infrastructure facilitates that in a very cost-effective way.”
Cloud-based services form the nucleus of technology adopted in the oil and gas sector, no matter the phase of digital business transformation. In our digitalisation survey, 79% of respondents revealed that they used cloud-based services in their current operations. The highest use of cloud services was in respondents who viewed their digital business transformation as being on schedule, followed by those behind schedule with around half the number.
The Americas region ranked cloud services as their number one most-used technology, considerably higher than in any other region. The disparity could be partly explained by tightening margins on US shale and the Canadian oil sands compared with many other regions. Technology solutions have been critical to sustaining the shale production boom in the US.
The Middle East and Africa also identified cloud services as the technology offering the highest benefits, narrowly beating Big Data to the number one spot. “Cloud technologies give you the ability to be agile and resilient,” says Kaveh Pourteymour, Chief Information Officer at Neptune Energy. “If you want to scale up, down or add more horsepower it allows you to respond faster. When we look at the Covid-19 situation, organisations that were in the cloud had a distinct edge.”
Rather than build systems entirely from scratch, cloud-based services allow companies to move quickly and at a fraction of the cost. Last year, UK major BP collaborated with Microsoft to integrate the Azure platform into its operations. The company aimed to capitalise on AI and machine-learning capabilities rather than develop solely in-house. BP used the cloud service to improve safety at its drilling rig platforms, creating autonomous platforms and allowing employees to manage operations remotely using AI. The producer also used Azure’s machine-learning resources to improve forecasting the recovery factor at oil and gas reservoirs. Using historical data, BP’s scientists built predictive models which slashed development time and improved legacy processes. Modelling the data revealed surprising results at some reservoirs, highlighting errors in the source data and likely inherent human error. BP’s internal data store was subsequently upgraded and the predictive models further improved.
US firm Google has also partnered with many oil and gas companies, including French major Total and US oilfield services firm Schlumberger, to improve subsurface data modelling. Drawing on rig and seismic data, together with AI and machine learning, the companies removed unnecessary drill time and improved the ability to explore and drill new prospects.
Companies are also leveraging the data capabilities of the cloud to mitigate cybersecurity risk. European major Shell utilised Amazon’s AWS cloud service to sift through collected data to look for vulnerabilities in its operations. The firm had reached the limits of what its legacy data system could process on-premises. Shell was able to use the AWS service cloud to integrate historical and collected data, processing several terabytes each day. Moving to the cloud offered a scalable and cost-effective solution to the company’s legacy data problem. The inherent flexibility allowed Shell to keep up to date with a constantly shifting cybersecurity landscape.
Digital Technologies Used by Organisations:Avoiding asset downtime is the core priority for firms looking to adopt new technology solutions.
Digitalisation is accelerating the capabilities of the upstream. Seismic, drilling, and production data amassed by producers is increasingly being applied to a diverse range of new technologies to help boost asset performance, drive down costs, and lower risk.
In the downstream, capturing and exploiting data buried within silos can also deliver important new insights. Merging valuable company data with predictive analytics, automation, and other digital solutions can help streamline operations, leverage competitive advantage, and provide greater profitability for companies.
Asset utilisation is a prime example of how downstream companies think about and prioritise digitalisation. High utilisation ensures stronger revenues, while scheduled downtime and maintenance significantly lowers company profitability. Downstream firms value technologies that can help optimise assets and reduce the need for lengthy turnarounds.
This was showcased in the results of our digitalisation survey. Improving maintenance activities was by far the core driver of digitalisation among midstream and downstream executives. 71% of respondents expected maintenance to benefit from greater adoption of digital technologies, ahead of optimising production at 68%. The Middle East and Africa were the only regions that had a different preference, citing training and knowledge transfer as their number one priority.
Asset maintenance is determined by long- and short-term drivers that can both benefit from digitalisation, particularly data analytics and predictive solutions. In the short-term, feedstock price and product margins can greatly impact asset utilisation. Knowing when feedstock pricing is likely to be high allows downstream firms to schedule upgrading or maintenance work with lower impact to asset profitability.
Modelling consumer demand can also give downstream firms a better understanding of which products are most sought-after and which will likely maximise revenues, both in the short-term and the long-term. Repurposing facilities to produce a different share of products can take time but can also boost company performance.
Looking long-term, tracking demand for refined products can give firms a better understanding of when facilities might need maintenance. Similarly, analysing facility performance means companies have a better understanding of asset health and when operations need upgrading or downtime.
By using sensors to track and store valuable data, and then a range of technologies to exploit the insights buried within, companies can create a robust maintenance strategy. Optimising asset efficiency means greater revenues for firms but can also improve sustainability. Predictive technologies help reduce emissions and ensure firms can meet stricter regulations and standards. “Companies are becoming a lot more conscious of their carbon footprint,” says Mukul Kapur, Senior Vice President of Digital Transformation from Publicis Sapient.
Many companies are already merging digital technologies into their downstream operations to leverage operational benefits. Spanish energy firm Repsol moved to a predictive maintenance programme as part of its digitalisation strategy. Using analytics tools, the company built a platform to better track the health and reliability of assets, optimising both short and mid-term maintenance plans.
“Maintenance is a process that can be converted into data very easily,” says Enrique Fernandez Puertas, Programme and Products Digital Director at Repsol. “Avoiding production losses or unexpected failure can be huge, so with a little investment and adoption of current processes the outcomes are very high.”
Optimising maintenance is crucial across the value chain, not just in the downstream. Technology firm Cognite partnered with Norwegian oil company Aker BP to automate the maintenance programme at the producer’s offshore Valhall oil field. Previously, engineers were forced to examine every valve in the system before scheduling for maintenance. The laborious inspection meant engineers had to scrutinise valves in good condition as well as those in need of maintenance, adding to downtime at the facility.
Using Cognite’s analytics platform, data from all the valves was captured and then visualised through a dashboard for engineers to use, on either computers or handheld devices. The digital upgrade was estimated to have cut down maintenance at the field by 80% per year and reduced annual maintenance checks by two-thirds. The length of an average maintenance session was shortened by half and the money saved from the process was reinvested back into critical production equipment.
Regional Priorities for Digitalisation in the Midstream and Downstream:The great diversity of technological options available to firms is helping accelerate digitalisation. But executives are still wary of the challenging economic conditions ahead.
The Covid-19 pandemic could be a defining moment for the oil and gas sector. Not only have global markets and energy demand been drastically upended, but businesses have been forced to embrace the digital frontier at a dizzying pace.
For much of the world, virtual work has been the biggest change. The move to blanket homeworking has been sustained by a variety of technological options. The rewards of automation have been evident in a world where physical access to assets has been restricted and safety is paramount.
Technologically literate companies have been among those best placed to shift to the new digitally dependent way of working. While many oil and gas companies have been hit hard by the economic downturn, those at the forefront of digital adoption have often gained a valuable competitive advantage.
While many speculate that the pandemic could be a landmark moment for the oil and gas sector, not everyone believes it will trigger wider technological adoption. In our survey, 53% of respondents said the pandemic would have a negative impact on digital appetite or net effect over the next six-to-nine months.
There was also no change in perspective when executives considered the short or long-term positive impact of Covid-19 on digital adoption. The only difference is when we examine negative sentiment. Executives viewed the long-term effect of the pandemic as being five percentage points less negative for future digitalisation than the short-term effect.
This is likely partly explained by the challenging economic conditions this year. The dramatic fall in oil revenues and already sluggish demand caused by the pandemic has forced much of the sector to cut capex and non-essential spending. In the survey, 50% of respondents said that low oil prices would negatively affect their businesses’ digitalisation schedule in the short-term, while 54% cited uncertain future oil and gas demand. This was by far the greatest concern for executives when they thought about digital implementation.
This trend is slightly different, though, when we look at the long-term impact. If oil prices and demand were to remain lower for longer—or never fully recover—the importance of maximising efficiencies and cutting costs will no doubt accelerate. If this were to occur, companies may change their digitalisation spending strategy to address asset efficiencies and lower expenditure, seeing little other alternative.
This perspective is shared by Repsol. “Because of the uncertainties of the pandemic, keeping a strong balance sheet and focusing on lowering costs is a necessary condition to staying competitive,” says Enrique Fernandez Puertas, Programme and Products Digital Director at Repsol. “But we firmly believe that the digitalisation programme will contribute to strengthening Repsol’s resilience in the long-term.”
Companies may also be forced to invest more in new technologies to offset the impact of a changing world. If governments and investors accelerate their support for sustainability, and likewise the industry shift to low-carbon energy gathers pace, digitalisation could help play a decisive role in this transition. The speed of change in energy transition was the third-highest positive driver for digitalisation when executives thought long-term.
There was other evidence that digitalisation could accelerate under this scenario. The positive impact of low oil prices and digitalisation improved by ten percentage points when viewed long-term. Uncertain demand also rose by two percentage points. If economic conditions permanently change, firms could be forced to seek alternative ways of boosting revenue streams.
This pattern is even more noticeable when the negative impact of these economic factors on digital adoption is viewed. Low oil prices had the biggest swing of any category between short-term and long-term, falling by 13 percentage points. Uncertain future demand fell by seven percentage points but was still the greatest concern for executives among all the categories.
Technology adoption is accelerating throughout the hydrocarbons industry, but there are still some major hurdles to greater implementation.
Cutting-edge digital technologies are gradually penetrating the oil and gas sector. Companies increasingly recognise the business value of integrating digital solutions into their operations, narrowing the gap with those at the forefront of the technology revolution.
But while there are many companies leading the digital vanguard, there are still some key barriers preventing a wider shift. A conservative culture that relies on guaranteed returns on investment over experimentation and innovation has often stymied the progress of digitalisation in oil and gas.
Lack of long-term vision was perceived as the greatest aggregate barrier to the wider adoption of technologies among the respondents to our digitalisation survey. In the Americas and MENA regions, it was the core reason for delaying implementation, against a lack of mindset for change in Europe. Both answers point to the pervasive use of legacy practices in the oil and gas sector. Many companies continue to take a reactive stance to the growth of digital technologies rather than prioritising agility and the ability to adapt to rapidly changing conditions.
Barriers to Digital Business Transformation by Region:In Asia-Pacific, the biggest concern was a lack of understanding of how to execute digital transformation. With so many technology options available, many firms are unable to discern which would work most effectively for their businesses and how best to apply those solutions. Asia-Pacific ranked problems created by siloed organisations as the region’s second-highest concern, illustrating the challenge of overhauling legacy practices. Huge volumes of often unstructured data are generated throughout the sector every day, with little attention given to the possible insights buried within and lack of knowledge of how to process the material.
Lack of standardisation of data was also considered a significant risk to digital business transformation among oil and gas executives. Around 38% of business leaders ranked it the highest barrier, behind only data privacy and cybersecurity fears, at 46%.
Leveraging the benefits of technological tools without the expertise in how to integrate them into existing infrastructure appropriately can stall adoption and cause companies to miss out on the potential operational and revenue gains.
Equally, securing a skilled workforce needed is a recurring problem within the industry. “The attractiveness of the sector is very relevant as there is a talent gap,” says Enrique Fernandez Puertas, Programme and Products Digital Director at Repsol. “Young workers need to be proactively drawn into the industry. The sector needs to evolve its attractiveness where sustainability is a core focus.”
With increasing pressures to combat climate change, companies risk losing young talent to other industries if they ignore these sustainability concerns. Firms must lower their carbon footprints so they are seen as part of the solution rather than the problem.
To meet the growing demand for digitally literate personnel, many companies are turning to retraining. “We have more than 1,000 employees involved in digitalisation initiatives, mostly in data analytics,” adds Puertas. “We as an industry need to focus on empowering that workforce through upskilling and increasing the digital literacy.”
Ranking Risks to Digital Business Transformation:Cybersecurity is another core concern across the energy sector. Historically, companies isolated their facilities and control systems from public networks like the internet, reducing the potential security risk. But today, the scale of data collection and benefits from automating operations have made this mostly impractical.
Companies want to benefit from the cloud and other third-party solutions, as well as integrate mobile technology on-site for field engineers. The entire value chain is increasingly being digitised, creating more potential vulnerabilities for cyber criminals to target.
Firms can be deterred from shifting towards greater digitalisation because of the challenges of ensuring robust safety measures. Notably, Italian cloud services firm Saipem was hit by a malware attack in 2018 that impacted company data and IT infrastructure.
Cyber-attacks can have financial and reputational repercussions, but even more importantly endanger human life. “Security is a big concern for everyone in our sector,” says Kaveh Pourteymour, Chief Information Officer at Neptune Energy. “It has to be at the forefront of our mindset. The evolution of the threat landscape is increasing all the time.”
Companies are starting to showcase their digitalisation credentials but have some way to go to emerge from the shadows.
Profound digital change is underway across the global oil and gas sector. Companies increasingly look towards technological solutions to drive performance, maximise safety, and adapt their businesses to shifting markets and opportunities.
While some companies are clearly further ahead of the digitalisation curve than others, perception can be a powerful force. Oil and gas firms that showcase their digital prowess and long-term strategy stand to profit in a volatile market where foresight and resilience are critical.
European major Shell is a case in point of successful marketing. It was named the most digitally transformed company in our research survey, significantly ahead of fellow European major BP in second place. Shell was viewed as the industry leader in the space regardless of the competitiveness of its peers or its actual digital standing.
Results for Companies in the Oil & Gas Sector That Are the Most Digitally Transformed:The core element of Shell’s strategy has been to combine its technological advancements with its aspirations to pivot towards low-carbon energy. Like other European majors, Shell has set itself the ambitious goal of reaching zero emissions by 2050 at the latest. Digitalisation is presented as an important bridge in meeting this low-carbon pledge. Harnessing new technologies is aimed at improving facility efficiency and drastically lowering emissions ahead of the pivot towards more low-carbon assets. Shell estimates that digital technologies have the potential to cut global CO2 emissions by around 20% by 2030.
While Shell has made strong efforts to link digitalisation to its low-carbon drive, it has also stressed the importance of cooperating with others. Last year, the firm launched more than 200 research collaborations with universities and institutions. One notable collaboration with Leiden University in the Netherlands aims to explore the potential use of quantum computing to model complex chemical reactions. The research could reduce the carbon intensity of several industrial processes, as well as enhance low-carbon processes such as solar energy storage, electrolysis to make green hydrogen, and oxidation of methane. Success could accelerate the growth and adoption of many low-carbon options.
Beyond the large-scale oil and gas companies, our digitalisation survey also highlighted the lack of recognised digital leaders across the sector. 16% of respondents struggled to think of any clear examples, and among small and oil and gas organisations no one stood out as exceptional or notable.
Regardless of what companies have actually achieved in integrating digital technologies into their operations, the message is clearly often failing to reach a wider audience external to the core business. “As an industry we are beginning to talk about digitalisation and the real examples that deliver value,” says Kaveh Pourteymour, Chief Information Officer at Neptune Energy. “But there are other industries that we can really learn from to advance our agenda and technologies need to go beyond a proof of concept to really drive value and business outcomes.”
For more information: