Open data in the oil and gas industry — How far have we come?
Friday, March 27, 2015
The oil and gas industry, more than most, has a reputation for opacity and secrecy — deals made on a handshake behind closed doors, secretive contract negotiations and revenues moving in complicated cross-streams among hundreds of corporate subsidiaries.
In 2011, global campaign group PWYP estimated that 10 of the world's most powerful oil, gas and mining companies own 6,038 subsidiaries and more than a third of those subsidiaries are based in secrecy jurisdictions in Caribbean islands, the Netherlands and Delaware.
So where does the oil and gas sector fit into the emerging global momentum toward open data, which is no longer being championed only by techie revolutionaries sitting in co-working spaces in Berlin or London, but at the highest level among government officials?
"Open data," Washington's new favorite buzzword, is rarely fully understood outside of specialist circles. It has been defined most succinctly as data that "can be freely used, modified and shared by anyone for any purpose." More simply, the movement wants to change the situation we currently face — where too much data about how we live, interact and are governed is locked up in proprietary documents and databases.
The rationale behind opening up these data sets is that when citizens are better informed, we are more able to hold our governments and representatives accountable. The place of open data on the agenda of the G8, G20 and the Open Government Partnership — which has grown from eight to 65 countries in four years — has escalated the movement beyond the niche "open" community to the highest political level as its dividends are being better understood.
In the world's many oil-exporting countries, there is no more important sector than energy to better understand. Open oil and gas data policies allow people to ask questions such as: Who owns the oil field on the edge of my town? What does it produce? What are its revenues? And what are the details of the concession or contract that has been signed with my local government or the landowner?
Once partnered with open budget data, that allows local people to assess whether they are seeing the benefit in public investment and services, to compensate for the environmental consequences they have to put up with.
But the sector presents a set of unique challenges. Most of the data we are talking about already exists, but confidentiality requirements are pushed by a strong sector lobby. Tim Berners-Lee, the inventor of the Internet and passionate open data advocate, talks of the extraordinary excuses that government agencies and corporations come up with to hoard their data rather than unlock it to the public.
Transparency of contracts, for example, is crucial. But even where there is political will for a government to openly publish its nation's oil contracts, this usually requires the consent of all contracting parties, which is often not forthcoming.
In the Open Data Barometer — a global ranking of countries by their "open" credential launched in 2013 — oil-producing countries make a poor showing. With the exception of perennial outlier Norway (which ranks seventh) and North American countries, no major oil-producing country features in the top 20 of the 86 countries assessed. Looking at the world's largest oil and gas exporters, Russia comes in at 26th, Saudi Arabia at 59th, and Iran and Iraq don't even show enough commitment to feature.
Much of this comes down to the unfortunate coincidence between oil and gas wealth and a weak institutional framework. Globally, the Extractive Industries Transparency Initiative (EITI) has tried to address this by opening up data on oil and gas revenues, and global transparency legislation such as the Dodd-Frank amendments have given legally binding weight to transparency efforts.
But the latter has been battled fiercely by the industry lobby. There is also a problem of scope, for as long as budget data remains closed, there is no clear picture of where the revenues go once they have been registered.
Another problem is the structure of the multinational oil companies that still rule the sector, registered in jurisdictions of varying opacity and confounding citizen researchers with their complexity. OpenCorporates, a globally accessible database of company financial and director information and cross-national connections, has gone some way to unpicking those networks.
The U.S., on its way to dominating the global oil industry again, fares well in the Open Data stakes. It trails only the U.K. in its provision of governmental data sets in the public domain.
In May 2014, President Barack Obama signed off on the country's first open data law, the Digital Accountability and Transparency Act, which builds on memorandums and executive orders issued since 2009 mandating federal agencies to make spending data available in a standardized format at USASpending.gov and other portals.
At data portal Data.gov, I found 2,321 oil-related data sets at state, federal and city level. For each data set, the metadata ("data about the data" such as where, when and by whom the data was compiled) is provided, as is an email contact. Anyone can freely access and download data on anything from the location of wells in Arkansas to usage statistics in the city-owned buildings of New York.
Data dumping is not enough. A key demand of Open Data advocates is the provision of data in an accessible, machine-readable format. The Energy Information Administration (EIA) — most journalists' first stop for industry data — responded in 2012 to the White House's call-to-action to provide truly open data with its own application programming interface (API).
The API allows computers to more easily access its huge data repository on production, consumption, inventory, import, export and sales data. Downstream, the Green Button initiative allows U.S. citizens to securely access their own personal consumption data in standard, machine-readable format from major utilities companies.
That is all well and good, but the key stumbling block in opening up data is ensuring it is used, and used enthusiastically. While initiatives like "Code For America" and the prioritization of tech skills in school curricula is helping, the world of CSV datasets remains a maze for most U.S. citizens, let alone citizens of far less-developed oil-exporting countries. These citizens will need to understand why to engage with this data and will need support in understanding how to process it.
Well-designed infographics, such as this award-winning example tracking the global arms trade, can provide inspiration. The Department of Energy also runs an Energy Data Challenge that offers cash prizes of $17,500 for innovative manipulation of their data resources to make them more usable.
In an era where unprecedented disillusion with political processes collides with a resurgent oil and gas boom in the U.S., opening up oil and gas sector data is key. Open data empowers us to access raw data and draw independent conclusions based on the facts.
A low-end personal computer today has the processing power of one of the supercomputers of the 1980s. Using this democratization of computing power as a springboard, open data policies can be more than geek-fodder. They are an important tool in making our democratic process a more vibrant one.
- The stress of 911 call-takers and emergency dispatchers
- 9 steps to more concise business writing
- Can solar energy compete with fossil fuels?
- Nurses rally in DC to address staffing issues with Congress
- US vs. Europe: Comparing different approaches to renewable energy
- Modern slavery and the hidden world of human trafficking
- Big winners in California’s new healthcare plan: Households and small businesses
- Nurses to rally in DC again to promote safe staffing
- Enamel: The body part that lasts a lifetime and doesn’t wear out
- Mocktails, low-alcohol cocktails bring innovation to beverages
- Infographic: Understanding marijuana laws, state-by-state
- Podcast: The economics of starting a practice
- The UK grapples with its low-carbon heating future
See your work in future editions
Your content, Your Expertise,
Your Industry Needs YOUR Expert Voice & We've got the platform you needFind Out How