Open Data – a quiet revolution

There is a quiet revolution taking place in some of the corridors of power, and it gathers under the banner of Open Data. Its leaders are an unlikely global coalition of technology evangelists, software developers and forward-looking civil servants, which together have been crafting a new model of engagement for public information with the potential to transform the relationship between citizen and the state.

The fundamental premise of Open Data is that “unlocking” — making freely available without constrains on re-use – the reams of data and information produced by government officials could immensely enhance the quality of the services received by citizenship. As a bonus, this would also improve the efficiency of the public administration, as the main user of government data is government itself.

Of course, this concept is not new. In the UK for example we’ve had The Guardian’s initiative to Free Our Data, which campaigned for several years for government to stop charging large amounts of money for public data. The best known example is probably mapping data from Ordinance Survey, which eventually was released after a pincer manoeuvre from citizen campaigns and businesses that felt unfairly treated by OS.

There is indeed a strong business lobby for the release and re-use of Public Sector Information (PSI) as the basis for the creation of a vibrant economic sector. The central argument here is the contrast between the flourishing business based on PSI in the USA – weather, transport, mapping, registries, etc. — with their European laggard counterparts. In simple terms the main difference adduced is that in USA PSI is free and funded by taxation on the value it generates, while generally in Europe the producers of PSI try to recover the full costs of production, thus restricting the potential market and even abusing monopoly positions.

There is growing agreement on the superiority of providing data for free, or more precisely at “marginal cost”, and a current EU consultation hints on this direction, although questions remain. The main issues centre around commitment to the sustained production of quality data when funding depends on the political priorities of the day. As we have seen in the recent Comprehensive Spending Review, these are very real concerns. However, they do not justify maintaining the status quo. There is also no evidence – please correct me – that in the USA there is any attempt to link public funding for data production to the economic value generated. It seems to be a public duty that has worked well so far, but without hard causal evidence. While there is no doubt that changes are overdue in opening up public data, there is no guarantee that it will become the promised golden egg. The case for Open Data cannot solely rely on economic arguments.

Organisations such as MySociety have been developing successful tools for civic engagement that build on public data. These range from petitions to the prime minister to the stunning Mapumental tool that helps you choose where to live based on house prices and transport times. There are similar initiatives in many other countries powered by a fast growing network of “civic hackers”, geeks and programmers channelling their skills and imagination in this area. The film below looks at creative re-use of transport data.

A Case for Open Data in Transit from Streetfilms on Vimeo.

Those tools initially relied on technological tricks to access data that was publicly available but not in a usable form that allowed you to build an automated application. In many cases these clever reuses of public data are frowned upon, or even persecuted However, in the past couple of years the gates of government opened to the geeks, or at least a decently-sized side door.

In the UK, a stream of reports gathered increased momentum and turned the policy machine in this direction, which has given more room to open minded civil servants for external collaboration outside the stifling Government IT procurement frameworks. A good example of this are the hack days organised by Rewired State , “where developers show government what is possible, and government shows developers what is needed”. These developments were initiated by the Labour government and carried over with gusto by the Coalition in a rare show of “tri-partisan” agreement.

In North America local and regional governments also started experiments in collaboration with the civic hacker community. The best known of those is the competition Apps for Democracy ’09 in Washington, DC, where a near economic miracle was performed by producing “47 web, iPhone and Facebook apps in 30 days – a $2,300,000 value to the city at a cost of $50,000”. Notwithstanding the possibility of a bubble valuation and the complex ethics of free labour, the event showed that opening up the public sector to the community could bring amazing benefits both economic and in terms of engagement. There have now been dozens of attempts to emulate this with varying degrees of success.

In the USA, the arrival of Obama to the White House marked a turning point in terms of openness. As a senator, Barack Obama had already showed support for transparency, and he campaigned as the New Media candidate. On January 21, 2009, on his first full day in office, President Obama issued a transparency and open government memo. He also appointed Vivek Kundra, the administrator responsible for the apps miracle, as Federal Chief Information Officer. The apps competition had been enabled by a one-stop-shop data catalogue for all D.C.’s public data, and Vivek went to replicate this approach at the federal level, creating Again this approach is now being implemented in several – mostly Anglo-Saxon – countries, and watched intently by others.

Back in UK things have developed along similar lines, with the creation of the UK data portal, developed in close collaboration with the community using open source software. Cameron also issued his own call for open government:

Greater transparency across Government is at the heart of our shared commitment to enable the public to hold politicians and public bodies to account; to reduce the deficit and deliver better value for money in public spending; and to realise significant economic benefits by enabling businesses and non-profit organisations to build innovative applications and websites using public data.

Although Cameron is mainly appealing at the economic benefits – forcing savings and value generation, this unprecedented situation has rekindled ideas about the transformational power of ICT for radical democratic upheaval. There is widespread talk of Open Government, and even Government 2.0, a fundamental realignment of citizen and state.

However, it may take more than technology and engagement to bring such a radical change. Campaigning groups that have been promoting government transparency and right to information are now converging with open data activists. This is leading to interesting insights about both the universal applicability of an unadulterated Open Data agenda, and the obstacles OD promoters may find as they move towards politically charged datasets.

ORG will be developing an approach to Open Data that proactively promotes data openness as an enabler for enhanced citizenship in a digital age, while being consistent with our wider concerns on the protection of rights and freedoms. The situation is positively encouraging and it presents a unique opportunity from ORG’s perspective.