Half empty or half full? Mixed reactions to Pew research on open data and open government

Yesterday, I wrote up 15 key insights from the Pew Internet and Life Project’s new research on the American public’s attitude towards open data and open government. If you missed it, what people think about government data and the potential impact of releasing it is heavily influenced by the prevailing low trust in government and their politics.

mixed-hopes-open-data-improve-pew

Media coverage of the survey reflected the skepticism of the reporters (“Most Americans don’t think government transparency matters a damn“) or of the public (“Who cares about open data” and “Americans not impressed by open government initiatives“). This photo by Pete Souza below might be an apt image for this feeling:

not-impressed-souza-obama

Other stories pulled out individual elements of the research (“Open data on criminals and teachers is a-okay, say most US citizens” or mixed results (“People Like U.S. Open Data Initiatives, But Think Government Could Do More” and “Sorry, open data: Americans just aren’t that into you“) or general doubts about an unfamiliar topic (“Many Americans Doubt Government Open Data Efforts“). At least one editor’s headline suggested that the results were an indictment of everything government does online: (“Americans view government’s online services and public data sharing as a resounding ‘meh’.) Meh, indeed.

As usual, keep a salt shaker handy as you browse the headlines and read the original source. The research itself is more nuanced than those headlines suggest, as my interview with the lead researcher on the survey, John Horrigan, hopefully made clear.

Over at TechPresident, editor-in-chief Micah Sifry saw a glass half full:

  • Digging deeper into the Pew report, it’s interesting to find that beyond the “ardent optimists” (17% of adults) who embrace the benefit of open government data and use it often, and the “committed cynics” (20%) who use online government resources but think they aren’t improving government performance much, there’s a big group of “buoyant bystanders” (27%) who like the idea that open data can improve government’s performance but themselves aren’t using the internet much to engage with government. (Heads up Kate Krontiris, who’s been studying the “interested bystander.”)
  • It’s not clear how much of the bystander problem is also an access problem. According to a different new analysis done by the Pew Research Center, about five million American households with school-age children–nearly one in five–do not have high-speed internet access at home. This “broadband gap” is worst among households with incomes under $50,000 a year.

Reaction from foundations that have advocated, funded or otherwise supported open government data efforts went deeper. Writing for the Sunlight Foundation, communications director Gabriela Schneider saw the results from the survey in a rosy (sun)light, seeing public optimism about open government and open data.

People are optimistic that open data initiatives can make government more accountable. But, many surveyed by Pew are less sure open data will improve government performance. Relatedly, Americans have not quite engaged very deeply with government data to monitor performance, so it remains to be seen if changes in engagement will affect public attitudes.

That’s something we at Sunlight hope to positively affect, particularly as we make new inroads in setting new standards for how the federal government discloses its work online. And as Americans shift their attention away from Congress and more toward their own backyards, we know our newly expanded work as part of the What Works Cities initiative will better engage the public, make government more effective and improve people’s lives.

Jonathan Sotsky, director of strategy and assessment for the Knight Foundation, saw a trust conundrum for government in the results:

Undoubtedly, a greater focus is needed on explaining to the public how increasing the accessibility and utility of government data can drive accountability, improve government service delivery and even provide the grist for new startup businesses. The short-term conundrum government data initiatives face is that while they ultimately seek to increase government trustworthiness, they may struggle to gain structure because the present lack of trust in government undermines their perceived impact.

Steven Clift, the founder of e-democracy.org, views this survey as a wakeup call for open data advocates.

One reason I love services like CityGram, GovDelivery, etc. is that they deliver government information (often in a timely way) to the public based on their preferences/subscriptions. As someone who worked in “e-government” for the State of Minnesota, I think most people just want the “information” that matters to them and the public has no particular attachment to the idea of “open data” allowing third parties to innovate or make this data available. I view this survey as a huge wake up call to #opengov advocates on the #opendata side that the field needs to provide far more useful stuff to the general public and care a lot more about outreach and marketing to reach people with the good stuff already available.

Mark Headd, former chief data officer for the City of Philadelphia and current developer evangelist for Accela software, saw the results as a huge opportunity to win hearts and minds:

The modern open data and civic hacking movements were largely born out of the experience of cities. Washington DC, New York City and Chicago were among the first governments to actively recruit outside software developers to build solutions on top of their open data. And the first governments to partner with Code for America – and the majority over the life of the organization’s history – have been cities.

How do school closings impact individual neighborhoods? How do construction permit approvals change the character of communities? How is green space distributed across neighborhoods in a city? Where are vacant properties in a neighborhood – who owns them and are there opportunities for reuse?

These are all the kinds of questions we need people living and working in neighborhoods to help us answer. And we need more open data from local governments to do this.

If you see other blog posts or media coverage that’s not linked above, please let me know. I storified some reactions on Twitter but I’m certain that I missed conversations or opinions.

few-think-govt-data-sharing-effective-pew

There are two additional insights from Pew that I didn’t write about yesterday that are worth keeping in mind with respect to how how Americans are thinking about the release of public data back to the public. First, it’s unclear whether the public realizes they’re using apps and services built upon government data, despite sizable majorities doing so.

Second, John Horrigan told me that survey respondents universally are not simply asking for governments to make the data easier to understand so that they can figure out what I want to figure out: what people really want is intermediaries to help them make sense of the data.

“We saw a fair number of people pleading in comments for better apps to make the data make sense,” said Horrigan. “When they went online, they couldn’t get budget data to work. When the found traffic data, couldn’t make it work. There were comments on both sides of the ledger. Those that think government did an ok job wish they did this. Those that thin government is doing a horrible job also wish they did this.”

This is the opportunity that Headd referred to, and the reason that data journalism is the critical capacity that democratic governments which genuinely want to see returns on accountability and transparency must ensure can flourish in civil society.

If a Republican is elected as the next President of the United States, we’ll see if public views shift on other fronts.

[STAT] State Department employees made .004% of email sent in 2013 into public records

Window_and_Hillary_Clinton_Not_Alone_in_Using_Private_Emails_to_Govern_-_Tech_-_GovExec_com

According to a new report from U.S. Department of State’s Office of the Inspector General, agency employees sent more than 1 billion emails, of which they made just 41,649 of them into public records.

That’s about 0.004% of them, by my rough calculation.

It’s a minuscule number, which probably why The Daily Beast ran a post reporting “only .00006% of State Department emails are preserved.”

While their calculation looks off by orders of magnitude, this tiny percentage still translates into members of the civil and foreign service entering almost none of their emails into archiving systems.

While the story hardly need it, this adds more interesting context to former Secretary of State Hillary Clinton’s decision to designate roughly 50% of her personal email as public records.

As Sunlight Foundation policy director John Wonderlich commented in Politico, this IG report undermines her argument that her emails with State Department workers were preserved on their end.

“Her justification around FOIA requests and around preservation became that most of the documents were cc’d or sent to .gov or state.gov addresses used by employees and therefore were preserved and accessible to requests, ” said Wonderlich “This [report] suggests that is not reliable at all.”

For more, read Josh Gerstein report exploring the broader ramifcations of the watchdog report on Clinton’s defense at greater length.

In a win for open government advocacy, DC removes flaws in its municipal open data policy

Update:

dcgov_logoIt’s a good day for open government in the District of Columbia. Today, DC’s Office of the Chief Technology Officer (OCTO) has updated the Terms and Conditions for DC.gov and the city’s new open data platform, addressing some of the concerns that the Sunlight Foundation and Code for DC expressed about the new open data policy introduced in July. The updated terms and conditions rolling out onto the city’s digital civic architecture this afternoon. “Today’s changes are really focused on aligning DC.Gov’s Terms and Conditions of Use with the new open data and transparency policy released this summer,” explained Mike Rupert, the communications director for OCTO,” in an interview. “The site’s T&C hadn’t been updated in many years,” according to Rupert. The new T&C will apply to DC.gov, the open data platform and other city websites. “It is encouraging that DC is taking steps toward considering feedback and improving its Terms and Conditions, but there is still room for improvement in the broader scope of DC’s policies,”said Alisha Green, a policy associate with Sunlight Foundation’s local policy team.  “We hope those implementing DC’s new open data policy will actively seek stakeholder input to improve upon what the policy requires. The strength of the policy will be in its implementation, and we hope DC will take every opportunity to make that process as open, collaborative and impactful as possible.” So, OCTO both heard and welcomed the feedback from open government advocates regarding the policy and agreed that the policy implications of the terms and conditions were problematic. Certain elements of the previous Terms and Conditions of Use (Indemnity, Limitation of Liability) could have chilled the understanding of the public’s right to access and have been eliminated,” said Rupert. Those were the sections that prompted civic hacker Josh Tauberer to wonder whether he needed a lawyer to hack in DC are simply gone, specifically that Indemnity and Liability Section. Other sections, however, still remain. The revised policy I obtained before the updated terms and conditions went online differs in a couple of ways from the one that just that went online. First, the Registration section remains, as does the Conduct section, although DC eliminated the 11 specific examples. That said, it’s better, and that’s a win. District officials remain cautious about how and where reuse might occur, they’re going to at least let the data flow without a deeply flawed policy prescription. “While we want to be mindful of and address the potential for harm to or misuse of District government information and data, the Terms and Conditions of Use should promote the new open data and transparency philosophy in a more positive manner,” said Rupert. Sharp-eyed readers of the new policy, however, will note that DC’s open data and online information has now been released to the public under a Creative Commons license, specifically Attribution 3.0 United States. That means that anyone who uses DC’s open data is welcome to “Share — copy and redistribute the material in any medium or format and Adapt — remix, transform, and build upon the material — for any purpose, even commercially,” as long as they provide “Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.” When asked about the CC license choice, Rupert said that “The new copyright language from Creative Commons – which as you is know is becoming the international standard – better states the overriding principle of the public’s right to web content and data. ” That did not sit entirely well with open government advocates who hold that making open data license free is a best practice. Asked for comment, Tauberer emailed the following statement in a response to the draft of the revision, welcoming the District’s responsiveness but questioning the premise of the District of Columbia having any “terms and conditions” for the public using open government data at all.

The new terms drop the most egregious problems, but these terms still don’t count as “open.” Should I expect a lawsuit if I don’t tip my hat and credit the mayor every time I use the data we taxpayers paid to create? Until the attribution requirement is dropped, I will recommend that District residents get District data through Freedom of Information Act requests. It might take longer, but it will be on the people’s terms, not the mayor’s. It’s not that the District shouldn’t get credit, but the District shouldn’t demand it and hold civil and possibly criminal penalties over our heads to get it. For instance, yesterday Data.gov turned their attribution requirement into a suggestion. That’s the right way to encourage innovation. All that said, I appreciate their responsiveness to our feedback. Tim from DC GIS spent time at Code for DC to talk about it a few weeks ago, and I appreciated that. It is a step in the right direction, albeit one deaf to our repeated explanation that “open” does not mean “terms of use.

The good news is that DC’s OCTO is listening and has committed to being responsive to future concerns about how it’s handling DC’s online presences and policies. “Several of your questions allude to the overall open data policy and we will definitely be reaching out to you and all other interested stakeholders as we begin implement various elements of that policy,” said Rupert.

Update: On October 29th, DC updated its Terms and Conditions again, further improving them. Tauberer commented on the changes to the open data policy on his blog. In his view, the update represents a step forward and a step back:

In a new update to the terms posted today, which followed additional conversations with OCTO, there were two more great improvements. These terms were finally dropped:

  • agreeing to follow all “rules”, a very ambiguous term
  • the requirement to attribute the data to the District in all uses of the data (it’s now merely a suggestion)

The removal of these two requirements, in combination with the two removed in September, makes this a very important step forward.

One of my original concerns remains, however, and that is that the District has not granted anyone a copyright license to use District datasets. Data per se isn’t protected by copyright law, but the way a dataset is presented may be. The District has claimed copyright over its things before, and it remains risky to use District datasets without a copyright license. Both the September update and today’s update attempted to address this concern but each created more confusion that there was before.

Although today’s update mentions the CC0 public domain dedication, which would be the correct way to make the District data available, it also explicitly says that the District retains copyright:

  • The terms say, at the top, that they “apply only to . . . non-copyrightable information.” The whole point is that we need a license to use the aspects of the datasets that are copyrighted by the District.
  • Later on, the terms read: “Any copyrighted or trademarked content included on these Sites retains that copyright or trademark protection.” Again, this says that the District retains copyright.
  • And: “You must secure permission for reuse of copyrighted … content,” which, as written (but probably not intended), seems to say that to the extent the District datasets are copyrighted, data users must seek permission to use it first. (Among other problems, like side-stepping “fair use” in copyright law.)

With respect to the copyright question, the new terms document is a step backward because it may confuse data users into thinking the datasets have been dedicated to the public domain when in fact they haven’t been.

This post has been updated with comments from Tauberer and the Sunlight Foundation.

Chris Gates will be the new president of the Sunlight Foundation

As reported by Politico, Chris Gates will be the next president of the Sunlight Foundation, the Washington, DC-based nonprofit that advocates for open government and creates tools that empower people to improve government transparency and accountability around the world.

“I couldn’t be more excited to join the team at Sunlight to help advance their work to bring more accountability and transparency to our politics and our government,” said Gates, in a statement. “For those of us who care deeply about the health of our democracy, these are perilous times. Our political system is swimming in anonymous money and influence, and our federal government is paralyzed and unable to respond to the challenges of our times. Our hope is that the new tools, data and information generated by Sunlight will help break through this impasse. We look forward to working with others in the reform field to fix a system that clearly isn’t working.”

Gates is currently the executive director of Philanthropy for Active Civic Engagement, part of the Council on Foundations. Previously, he served as president of America’s oldest good government organization, the National Civic League.

“Chris, who is a thought leader in the fields of democratic theory and practice and political and civic engagement, has, for the past decade, been a leading voice for strengthening democratic processes and structures and developing new approaches to both engagement and decision-making,” wrote Ellen Miller, co-founder and current president of the Sunlight Foundation, at the organization’s blog. “He and I have been colleagues for the past several years — we’ve worked together on numerous occasions — and I am truly thrilled that he will become Sunlight’s president.”

Miller went on to say that Gates will “bring a breadth of experience and style of leadership that will take us to new levels.” This fall, the Sunlight Foundation will undergo the biggest transition of leadership since its founding: last week, Sunlight Labs director Tom Lee announced that he was leaving to work at DC-based Mapbox, with James Turk stepping up to assume responsibility for the nonprofit’s powerful technology resources and development team. I look forward to seeing how both men build out the civic infrastructure, reporting group, and advocacy shop that Sunlight has established since the organization opened its doors in 2006.

[Image Credit: Sunlight Foundation]

Sunlight Foundation highlights benefits of open data beyond the business case

Given the considerable attention that the economic outcomes of open data releases has received over the past year, with trillions of dollars in potential value flowing across headlines, it’s worth reminding everyone of the impacts of open data beyond the bottom line. Thankfully, Emily Shaw, the national policy manager at the Sunlight Foundation, did exactly that in a blog post today, including a handy briefing document that I have embedded below. She credited her colleagues for the brief:

“Democratic governance improves when people have data that helps them see how officials are doing relative to past or promised performance,” she wrote.

As Shaw highlighted in her post, open data can increase the transparency of governments, corporations, journalism or academia. Its release and analysis can and does hold those same entities accountable. Open data can enable efficiencies in information search, access and retrieval, supporting the case of those looking for the return on investment in these kinds of open government initiatives. And open data can support and enhance civic engagement and participation between the people and their government.