Credit: "Open government data," by Justin Grimes

Is the open data movement a ‘joke‘?

Canadian blogger Tom Slee published a post yesterday that made that claim, writing that “it’s not a movement, at least in any reasonable political or cultural sense of the word” and that “it’s doing nothing for transparency and accountability in government. Slee followed by a second post that highlighted some reactions to the first —  including my own, driven by a rather heated dialogue on Twitter with author Evgeny Morozov.

Slee makes one assertion that will be of particular interest to Govfresh readers, who may be surprised to find that civic hacking and a movement to put open government data online don’t exist:

Who, after all, is the Open Data Movement? Well it turns out there isn’t one really, at least when it comes to “open data” in the sense of “open government data”, which along with “open scientific data” is one of the two most common uses of the term.

“Open Data Movement” is a phrase dragged out by media-oriented personalities to cloak a private-sector initiative in the mantle of progressive politics. Along with other cyberculture terms (“hacktivism”, “unconferences”, “hackathons”) the word “movement” suggests a countercultural grass-roots initiative for social change, but there isn’t anything of the sort that I can see.

After a weekend at Transparency Camp in DC where an international open government (counter) culture was everywhere and a trip to Brasilia to see  representatives from civil society meet at the Open Government Partnership conference, along with years of reporting, I can honestly say that Slee simply must not be looking very hard. Sunlight Labs director Tom Lee rung in this afternoon with his assessment, defending open data’s role in accountability and activism:

But in a larger sense, there can be no question that open data is an important tool for creating accountability. Consider what FOIA means for corruption. Consider what OpenCorporates is doing. Consider our work on lobbying reform or tax expenditure transparency or the understanding of the interplay between the two that open data makes possible. Or look at Revenue Watch’s work to get better data on international royalty payments from mineral and petroleum companies. Or our push for better information about Congressional activity and political advertising.

Slee’s notion that open data is simply obfuscation for private sector initiatives was also more or less turned on its ear by TechPresident’s summary of Transparency Camp:

….an international group of transparency and open government activists got together for TransparencyCamp. Among the folks represented there: Open Data Albania, a collaborative that collects and analyzes data about the government of Albania and partners with journalists to build context to explain how that country works; Global Integrity, which partnered with others to build a 50-state corruption report card for the United States; and LittleSis, which looks to map connections and influence at the highest levels of American society.

Are there commercial interests that will use open data in their products or services? Absolutely. Look to consumer finance startups like Hello Wallet or BrightScope or Mint.com or bigger players in healthcare, like Aetna, that stand to benefit from smart disclosure initiatives.

There’s little doubt that smart entrepreneurs, big and small, are going to mashup data from the rapidly expanding new sources — social data, geolocation data, mobile data, financial data, transit data, health data, etc — and build new businesses on it or improve their existing services, like Zillow or Google Maps or Consumer Reports or Bloomberg Government. In a time when job creation is critical, using public sector information to create jobs isn’t an aim to dismiss lightly, although the terms and conditions under which such activity occurs must be clear to all actors involved, to avoid the creation of new monopolies based upon artificial scarcity.

We are in such early days in this new century, and the role of civic entrepreneurs in putting data to work in the marketplace shouldn’t be discounted or dismissed out of hand, although the licensing and extraction of value of public data must be closely watched. (And yes, that includes the progress of civic startups coming out of Code for America’s civic accelerator or any funded by O’Reilly’s Alpha Tech ventures, like SeeClickFix. If you see the latter in my coverage, expect a disclosure and know I personally do not have an interest in the success of such startups and maintain a rigorous “church and state” relationship with that entity. My investments currently consist of a 401(k), and not one that’s weathered the economic downturn as well as I’d like.)

Here’s the Sunlight Foundation’s Tom Lee again, on the issue that Slee raised:

… the core of Tom’s complaint isn’t about episodic failures of activism. Rather, he seems to be bothered by open data enthusiasts’ adoption of language and an aesthetic that traditionally belong to projects with more expressly political (and progressive) aims. He seems suspicious that a self-described nonpartisan activist movement could be anything but a cynical lobbying ploy for private interests. Indeed, there’s a clear strain of hostility toward business that runs through Tom’s critique. Fair enough: more than a few such “movements” have turned out to be astroturfing operations, and it’s certainly true that on some open data issues I expect a less than enthusiastic response from the corporate world.

But I think it’s flatly wrong to consider private actors’ interest in public data to be uniformly problematic. We should be clear: we won’t tolerate those interests’ occasional attempts to lock public data into exclusive monopolies. I think our community has done a pretty good job lately of identifying such situations and stopping them, and of course people like Carl Malamud have been doing important work on this question since well before most of us ever heard of “open data.” But if commercial activity is enabled by data, that’s all to the good—the great thing about digital information is that scarcity doesn’t have to be a concern. Google Maps’ uses of Census TIGER data, for instance, is proprietary, motivated by profit, and unquestionably a huge boon to human welfare. And the source data remains free for anyone else to use! Cutting off those kinds of uses with noncommercial licensing would be nothing more than a destructive act of pique.

Open government godfather Carl Malamud offered a blunter assessment in a comment:

…In my world, the commercial sector is raping and pillaging the public treasury, getting exclusive deals on data that not only keeps out other companies, but researchers, public interest groups, and everybody else who make up “the public.” In many cases, the government data is so tightly behind a cash register that even government workers enforcing the law can’t afford to buy copies of the data they produce or the rules they promulgated.

I have no idea who Whimsley is and don’t usually bother to comment on random blogs by armchair quarterbacks, and I have no idea what is going on in Canada, but this one seems so far off the mark it seemed worth a few words. The post is backwards in the analysis, but it is also lacking a bit of reality.

I don’t give a hoot if something is a movement, but I’m not sure that making lists of who gets to use data and who doesn’t get to use public data makes any sense (many nonprofits are intensely commercial and many commercial operations seem to avoid the evilness of many of the beltway bandits). As far as Code for America’s program and their sponsors, or Tim O’Reilly and his talks, I’ve observed all of those at first hand and it is pretty clear the pseudonymous blogger doesn’t have a clue what either group does or what they think.

Carl

P.S. I’ve watched many tens of millions of people access and use government data that wasn’t available before from my servers. Maybe not a movement, but definitely a really big crowd.

Canadian open government data advocate and analyst David Eaves responded to Slee today on his own blog, where he argues (at considerable length) that open data movement is not a joke. The whole post is worth reading but I’ll quite Eaves on one point in particular:

…to be clear, I would never equate open government data as being tantamount to solving the problems of a restrictive or closed government (and have argued as much here). Just as an authoritarian regime can run on open-source software, so too might it engage in open data. Open data is not the solution for Open Government (I don’t believe there is a single solution, or that Open Government is an achievable state of being – just a goal to pursue consistently), and I don’t believe anyone has made the case that it is. I know I haven’t. But I do believe open data can help. Like many others, I believe access to government information can lead to better informed public policy debates and hopefully some improved services for citizens (such as access to transit information). I’m not deluded into thinking that open data is going to provide a steady stream of obvious “gotcha moments” where government malfeasance is discovered, but I am hopeful that government data can arm citizens with information that the government is using to inform its decisions so that they can better challenge, and ultimately help hold accountable, said government.

Expect more reactions to emerge. For my part, I’d reiterate what I’ve written about open government data before: Simply opening up data is not a replacement for a Constitution that enforces a rule of law, free and fair elections, an effective judiciary, decent schools, basic regulatory bodies or civil society, particularly if the data does not relate to meaningful aspects of society.

I’ll go a bit further, extending a comment I left on Slee’s second post. Morozov’s points about raising critical questions are legitimate and my comments on Twitter could easily be read as being dismissive of their pertinence. That was not my goal and to do so would be antithetical to my belief in free speech and robust public discourse about the serious issues that confront us as a country and humans on this small blue dot.

I did think — and still do — that the latter half of Slee’s post needed to include some basic research that would have improved its discussion of the issues involved, particularly with respect to the tension between open data and open government. The comments he received on that post particularly those of Carl Malamud, as quoted above, and the posts that I’ve linked, should serve as a bellwhether.

Readers curious about the tensions here (perhaps including Slee) would benefit from reading a recent research paper by Yu and Robinson on the ambiguity of open government and open data and from delving into the broader discussion in civil society in South America and Europe in the lead up to the Open Government Partnership, where this dynamic was the subject of much concern — and not just in a Canadian or UK context.

One reason that Slee’s post may have received attention — and continues to do so — is that it does not read as “The problem with open data versus open government in Canada.” Instead, it’s an indictment of what’s happening in the U.S. or around the world. I suspect that if he had stopped after the first half of what he’d written about Canada’s open government record, most people from civil society and NGOs would have nodded along. One reason those legitimate concerns he raised may not be receiving the attention that Slee might like is that they were coupled them with a headline and analysis that distract from them. In his followup, he walks that back a bit, but not much.

As the conversation in the public sphere over the past few years has demonstrated, there are a lot of different perspectives on what purposes “open data” should serve, often informed by what the watcher intends or the organization’s goals. That’s to be expected in a new and rapidly evolving space.

There are people who want to see legislatures open their data, to provide more insight into deliberative processes. In the U.S., for instance, Govtrack.us has been making government legislative data open (and more useful!) by scraping it from government websites.

There are constituencies who wish to see campaign finance data open, like the Sunlight Foundation, thereby showing where influence and power lies in the political system.

There are entrepreneurs and civic activists who wish to see transit data or health data become more open, in the service of more civic utility or patient empowerment — one can look at the efforts of various cities or the Veterans Administration in the US on that count. When you consider that such data can include ratings or malpractice information about hospitals or doctors, or fees for insurance companies, transparency and accountability does crop up as a goal, which in turn does have political implications.

One could spend quite a bit of time listing organizations or individuals who are putting data online, including open government activists in Brazil, Africa or yes, Canada, and then listing consumers of that data.

Whether Slee wish to describe those activities as a “movement” is up to him — but it is indisputable that 3 years ago, a researchers would be hard-pressed to find a open government data platform nor downstream consumers.

Now there are dozens of such platforms at the national, state and city level and even more services that use that data. What matters more than their existence, however, is what goes onto them. In that respect, civil society and media must to be extremely careful about giving governments credit for just putting a “portal” online. In March, Sunlight Foundation policy director John Wonderlich wrote a post about the “missing open data policy” that every government standing up a platform could benefit from reading:

Most newly implemented open data policies, much like the Open Government Directive, are announced along alongside a package of newly released datasets, and often new data portals, like Data.gov. In a sense, these pieces have become the standard parts of the government data transparency structure.  There’s a policy that says data should generally be open and usefully released, a central site for accessing it, some set of new data, and perhaps a few apps that demonstrate the data’s value.

Unfortunately, this is not the anatomy of an open government.  Instead, this is the anatomy of the popular open government data initiatives that are currently in favor. Governments have learned to say that data will be open, provide a place to find it, release some selected datasets, and point to its reuse.

What gets left out of these initiatives, however, is often the most important part — the decisions as to what gets released, and how.  Many open government data discussions skip over the question of whether governments are deciding appropriately what gets released and what doesn’t.  Instead of making complex decisions about what should be released, central governments suggest that those decisions are hard, and that as long as there’s always some new information, then we’re making progress that deserves praise.

There are also, notably, many civil society and media organizations that are collecting and sharing open data, from OpenCorporates to OpenCongress to ProPublica, and startups as well, like Brightscope.

There are a lot of different voices in this space. Asking hard questions is important and useful, particularly given that motivations and context will differ from country to country and from industry to civil society. In the United States, there is a sizable group of people that believe that data created using public funds should in turn be made available to the public — and that the Internet is a highly effective place to make such data available. (See the Sunlight Foundation’s support for POIA, or “Public = Online.”) Such thinking extends to research or code now too. Support for open data also extends across the political spectrum here, as evidenced by the recent passage of the DATA Act with bipartisan support.

Whether one agrees with that perspective or not is, of course, something that free thinkers in democratic societies to decide for themselves, including my neighbors in Canada. Given the pervasive tendency towards more secrecy in governments, not less, and my experience in open government over the past few years, my tendency is towards making public information open by public by default as opposed to its inverse. If the trend towards over-classification is not reversed, the nation will be less informed.

Finally, since Slee brought the source of his frustration up in his second post, I want to be clear: the issues he cited with respect to Canada’s open government record are not founded in speculation, as his links and points demonstrate. The Harper administration has received the dubious distinction of a secrecy award from a journalism association. It did cancel Canada’s long-form census, prompting the resignation of the head of the Statscan service. And journalists have been confronted with limited access to government scientists, much in the same vein of open government issues in the United States.

Highlighting the difference between rhetoric and actions is a crucial role for civil society and independent media in any open government context. To the extent Slee has done so with these posts, I applaud his actions. To the extent he questions my motivations or those of my publisher and friend, Tim O’Reilly, I reject his conclusions.

I agreed to collaborate with Tim on focusing attention on open government and technology because he wanted to see government work better, become smarter, and be more accountable to the people whom it serves. I did so with the understanding that I’d be able to pursue storytelling with editorial independence and adherence to truth, fairness, accuracy and reason. If I ever fail in that goal, I expect the open government community to continue to hold me as accountable as they would an elected official or public servant.

Should Canada’s open government advisory panel be convened again, I will have specific issues to raise concerns about with the other members and commit to doing so.