Friday, December 1, 2023

How should the Oil and Gas Industry adopt Data and Analytics?

Oil and gas are crucial to the global economy, impacting everything from transportation to electricity to manufacturing. In this backdrop, the key characteristics of a typical oil and gas firm from a Data and Analytics perspective are:

  1. More emphasis on market commoditization or fungibility over personalization. This means there is little granular data of individual customers and consumers.
  2. Operations and regulatory compliance more than data and analytics. This means the data strategy of an oil and gas firm is typically more on risk mitigation, data security, and regulatory compliance, and so on.
  3. Aggregation over granularization. This means, most oil and gas firms have focused on data categories or taxonomies over individual products and services.

Overall, oil and gas companies have been traditionally dependent on scale and achieving scale requires large capital. But when the crude oil prices fell in 2015, oil and gas firms were forced to focus on margins and efficiency for survival.  They started looking at the best practices in margin centric industries such as Retail, Airlines, CPG (Consumer packaged goods), and so on and one of the key areas the companies in these industries focused to get most from the margins was Data and Analytics. Given that oil and gas companies have enormous amount of data through sensors, well logs, CAD drawings, ERPs and more, they too decided to leverage data for improved business performance. We often say – Oil and gas firms are data rich, but insight poor. According to a report by McKinsey, Data and Analytics when implemented effectively in an oil and gas company, can yield returns amounting 30-50 times the investment within a few months. This presented an enormous untapped opportunity for the oil and gas industry [1].

So how can Data and Analytics help oil and gas companies? Fundamentally Data and Analytics is needed to measure and improve business performance. If the culture of measurement and performance improvement doesn’t exist in a company, then Data and Analytics solutions will not go very far. Basically, Data and Analytics in any firm including an oil and gas firm is needed for three main reasons [2].

  1. To measure and improve performance
  2. To support data driven decision making
  3. To continue grow and improve

In this backdrop, here are some use-cases oil and gas industry – both dos and don’ts. Based on my experience consulting in over 20 oil and gas companies, below are the two key use-cases that have the highest return on investment (ROI).

  • Leverage Predictive Analytics. Historically oil and gas firms have focused on historical performance with reports (transactional and Business Intelligence) and dashboards. But one cannot go very far with historical performance reports coming from descriptive analytics. For example, which is insight more valuable? The weather report in April 2019 (past) v/s weather report for next month (future). Obviously, it is the future insights that matter because you can plan for those events and perhaps even influence and control the outcome.
  • Integrate predictive and prescriptive analytics. Predictive analytics is using historical data to make predictions about future and prescriptive analytics is using data to determine an optimal course of action. But many oil and gas firms are not exploring the opportunities predictive and prescriptive analytics offer. For example, many refineries or upgraders have not updated their linear programming models for many yeas. Many procurement departments do not use predictive and prescriptive analytics models to optimize their costs and services.

On the other hand, what are some of the common mistakes oil and gas companies do in Data and Analytics? Here are the two use cases they have a low ROI for Oil and gas companies from Data and Analytics solutions.

  • Today, many oil and gas firms are directly exploring Machine Learning (ML) solutions without exploring predictive or preceptive analytics solutions. ML is about automation of insights and automation depends on standardization of rules, and these rules have to be formulated manually. At the core, deploying Data and Analytics solutions is about managing change. Hence companies should harness predictive and perspective analytics solutions first, understand the mechanics and rules of standardization and insight consumption, before embarking on ML.
  • Digitization always doesn’t lead to better ESG (Environmental, Social, and Governance) or net-zero performance. While data is definitely a business asset, unplanned digitalization can result in data can become a liability resulting in increased cost, risk and carbon footprint [2]. Data takes up vast amounts of energy to store, secure and process. In 2018, data centers consumed roughly 1%of total global electricity. By 2025, the energy consumption of data centers is expected to consume 20% of global electricity. According to Stanford University, storing and processing 100 GB of data per year will cause a carbon footprint of about 200 kilograms [2].

So, what is the best way to capture or harness opportunities from Data and Analytics for an oil and gas company? Research says that a vast majority of the Data and Analytics programs fail. The most common mistake many oil and gas companies make is focusing too much on technology and tools. Actually, getting business results is not about technology; it is not even about quality data. Technology and quality data is needed, but they come later. To start, you need a strong data culture and data literacy. These two are the necessary conditions needed to derive value from data and analytics. Once you have addressed the issue of culture and literacy, you can start addressing other technical aspects such as data quality, governance, building a SoR (System of Record), leverage the Semantic Layer for Analytics, and more.  But again, the foundation for all Data and Analytics initiatives is data culture and data literacy.


  2. Southekal, Prashanth, “Analytics Best Practices”, Technics Publications, 2020

Author Bio

Dr. Prashanth Southekal is the Managing Principal of DBP Institute (, data and analytics consulting, research and education firm. He is a Consultant, Author, and Professor. He has consulted for over 80 organizations including P&G, GE, Shell, Apple, and SAP. Within the energy space he has consulted for firms such as Shell, TCPL, Husky Energy, Cenovus Energy, Suncor Energy, AER, NEB, Petrinex, Precision Drilling, Parkland Fuels, Mansfield Oil, Payload Technologies, and more. Dr. Southekal is the author of two books — “Data for Business Performance” and “Analytics Best Practices” — and writes regularly on data, analytics, and machine learning in, FP&A Trends, and CFO. University. His second book, ANALYTICS BEST PRACTICES was ranked the #1analytics books of all time in May 2022 by Book Authority. Apart from his consulting pursuits, he has trained over 3,000 professionals worldwide in Data and Analytics. Dr. Southekal is also an Adjunct Professor of Data and Analytics at IE Business School (Madrid, Spain). CDO Magazine included him in the top 75 global academic data leaders of 2022.  He holds a PhD. from ESC Lille (FR) and an MBA from Kellogg School of Management (U.S.). He lives in Calgary, Canada with his wife, two children, and a high-energy Goldendoodle dog. Outside work, he loves juggling and cricket.



Translate »