Management of proliferating data
90% of the world’s data has been generated over the last two years. As a result, data management has become increasingly complicated. Nowhere is this truer than in the upstream industry, where the pace of Shale drilling has more than doubled in recent years.
In Woody Bendle’s blog Brainzooming, a recent article called, “Strategic Insights – Much Ado about Big Data Analytics,” features a discussion around the rise of big data. Specifically, in the last two years, as the physical amount of data has mushroomed, so has interest in big data.
Bendle shares this image comparing 2011 to now to demonstrate the jump in interest.
The main point of the article is that there is a lot of confusion around what big data is, and what to do with it. As Bendle states, “Big Data encapsulates A LOT of different data types ranging from good (accurate and reliable) to bad (wrong and inconsistent).”
Getting to the Quality Data
Part of the reason there is so much data in the first place is because of “advancements in computing capacity and speed, data capture and storage in conjunction with rapidly decreasing costs.” In a nutshell, until recently, it was not even possible to store as much data as we do now.
But is the problem really with not having enough data?
Anyone in the oil and gas industry who has been involved with data management at all knows that quantity is not always the problem. For example, you might have SCADA system data for the past quarter. But if the data was never processed, and no alerts were set up to notify the team of issues, this data is not useful.
In addition to lack of significance, upstream companies deal with a variety of data quality issues, including:
Wrong data
i.e. production numbers for a given well were inaccurate
Outdated data
i.e. 30 day data lag between the drilling and land department
Too much time spent turning data into meaningful information
All of these quality issues have a direct cost on the line of business as a result of time wasted, uninformed decisions, or taking too long to react to needed change.
More data isn’t the answer
While it might not make sense, many decision makers have associated data quality issues with the need for more data. As Bendle points out, “For years, the lack of data (or the costs associated with obtaining data) has been used as an excuse for not knowing (or being able to answer) something.”
As I discussed above, the availability of more data does not automatically solve the quality problem. Data is cheap, explains Bendle, so bosses, “naïvely believe since Big Data is now relatively easy to capture and house, it also should be relatively easy and inexpensive to analyze.”
The fact is that data is much harder to process than it is to capture. Bendle gives two good tips for beginning to conquer the data road block that I think apply very well to the oil and gas industry.
1. Give some thought to the questions you’d like to answer with data. This should help narrow the problem from gathering data for its own sake. Instead you can focus on ways of making data meaningful for optimizing production, or reducing safety incidents.
2. Understand how your data could be optimally configured. Specifically, what structure would enable data to “answer all of these yet-to-be-defined business questions”? Perhaps a specific table needs to instead be visualized as graph. Or maybe a dashboard could give you the fifty thousand foot view your team needs to make forecasting more accurate.
Taking Data Management Action
Data management projects can seem daunting because they are not always straight forward, and the initial investment seems expensive. In a world where successful Shale drilling means drilling less wells more efficiently, access to the right data is critical!
For more, read this blog on becoming involved with data management by learning to be a good data consumer…