Big oil and gas companies are finding new ways to extract data streams, combine them, merge them and gain insights that can optimize their businesses, increase profits and save the environment.

We're talking about big oil meeting big data.

Probably one of the most important tech revolutions after the Internet, big data is about the fact that there is much more data than we ever had before. Imagine saving all the digital information that we have on CDs and making large stacks. Today, we have about five large stacks that would reach the moon. More than 10 years ago, there would have been only one stack.

Yet in order to understand what role big data plays in big oil, there are three core elements to this phenomenon, most of which are referenced from Viktor Mayer-Schoenberg and Kenneth Neil Cukier, the authors of "Big Data: A Revolution That Will Transform How We Live, Work, and Think":

1. Big data is about the fact that we have much more data available, that we are able to save that data and have found new ways to turn into data information that has never been considered as data before — something that is called "datafication."

2. Big data is about statistics turning away from samples and well-curated, ordered data. Turning away from samples means allowing messiness, working with data that might contain errors, but because we have so much data available, it does not matter whether a hand full of data points are wrong.

3. Big data means that we will turn away from searching for the cause of things. It is not about making a hypothesis and testing whether the hypothesis is true. The way big data investigations work is taking large data streams and finding correlations between different data streams — but not carrying about finding a story about the reason for these correlations.

How is big data used in the oil and gas industry?

To some extent, the oil and gas industry has always been a big data industry. Large amounts of data are inherent in the process of finding and extracting oil and gas from more than 5,000 to 35,000 feet below the Earth's surface.

On a regular basis, the industry uses seismic data (2-D and 3-D), spatial and GPS coordinates and other geological information. As some experts say, the origin of the shale revolution is not horizontal drilling and fracturing, it's smart drilling — fed by innovation and data.

What is new is that the industry has found ways to extract data where it has not been possible to extract data before, and to use that data and combine it with other data streams that it would not have dared before.

For instance, Houston-based company Pacific Drilling has started to monitor thousands of pieces of equipment on its Nigerian offshore rigs in real time. For example, the company tracks data points for the generator that helps power the ship — the vibrations coming out of the engine, the amount of heat it produces, how fast it's running and its oil pressure.

This has allowed Pacific Drilling to cut down on its monitoring costs and to estimate when a new part, i.e. of an engine, has to be ordered and replaced.

Yet, including big data in maintenance is only the start — for the exploration phase, microseismic data could play an increasingly important role. In order to decrease environmental risks, data collected during the extraction process can be combined with weather and other relevant data, just to mention some.

What are the drawbacks of big data, and what is its future?

One of the biggest drawbacks of big data has been pointed out by Nate Silver, statistician and author of "The Signal and the Noise":

"If the quantity of information is increasing by 2.5 quintillion bytes per day, the amount of useful information almost certainly isn't," Silver said. "Most of it is just noise, and the noise is increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine — but a relatively constant amount of objective truth."

What this means is that not every correlation in the data has a meaning. Big data runs the risk of predicting something where there is actually nothing, just noise.

However, given the fact that the oil and gas industry has always been and will be a big data industry, this risk will be much lower. Having more data compared to less is a gain in a game of huge uncertainties.

So it's safe to say, big data and big oil are a perfect marriage.