Fauxpen data, open data and bridging the data divide

My Ignite talk from the Strata Conference in NYC is online.

Comments welcome, as ever.

Update: In the context of fauxpen data, beware “openwashing:” Simply opening up data is not a replacement for a Constitution that enforces a rule of law, free and fair elections, an effective judiciary, decent schools, basic regulatory bodies or civil society — particularly if the data does not relate to meaningful aspects of society. Adopting open data and digital government reforms is not quite the same thing as good government, although they certainly can be and are related, in some cases.

If a country launches an open data platform but deprecates freedom of the press or assembly, questions freedom of information laws or restricts the ability of government scientists to speak to the public, is it adopting “open government” — or doing something else?

This is the ambiguity of open government and open data that Harlan Yu and David Robinson wrote about in 2012. Expect it to be the subject of more “takedowns” in the 2013.

Code for America Summit showcases civic innovation [LIVEBLOG]

What began as an idea just over two years ago is now a reality: a citizen corps of designers and programmers who committed to a year-long fellowship to Code for America. Today in San Francisco, the inaugural Code for America Summit will highlight the year past, look to the year ahead and convene a conversation around four core topics that will be familiar to people who have been following the story of Gov 2.0: citizen participation, data-driven decision making, co-Creation and co-stewardship, government as a platform. The SPUR Center is packed with civic innovators from all around the country and buzzing with energy. My liveblog is below.

Day 2

Code for America Summit (Day 2)

Day 1

Code for America Summit

Chicagobuildings.org maps vacant and abandoned buildings using open government data

One of the minds behind the Look at Cook open government data visualization app is at it again. Derek Eder wrote in this week to share another Web app he just launched (ChicagoBuildings.org) and a reminder about what’s happening in Chicago in this space.

This Web app takes 311 reports about vacant and abandoned buildings from the Chicago and visualizes them onto a searchable map. “It’s specifically set up to pull data from Chicago’s data portal,” said Eder, linking to the 311 service requests of vacant and abandoned buildings dataset.

Eder shared more about how mapping Chicago’s vacant buildings in a blog post earlier this week. The results are unsurprising: there are many more vacant buildings in areas with high poverty rates.

Eder said that the app could be used by other cities, depending on how they store or format their data. The code for
Chicago Buildings is on Github. On that front, he says that Chicago “isn’t using Open 311 yet, so this site isn’t either. That being said, it wouldn’t be too hard to hook up the same interface to a different data source.” Code for America will help Chicago to implement Open311 in 2012. Eder shared that he wrote a script that converts Socrata to Google Fusion Tables that could be modified for this purpose.

ChicagoBuildings.org is one of a growing number of civic applications that have come out of Chicago’s open government initiative. As Eder made sure to point out, his app is a finalist in the Apps for Metro Chicago contest, along with 9 other apps, including iFindItChicago and Techno Finder.

In the video below, Elizabeth Park, the creator of IFindit Chicago, talks about how she was inspired to build the team that created an Android app to help homeless and lower income citizens find resources like as shelters, medical clinics,and food pantries.

Voting for the winners ends this Friday, October 14th, so check out the community round entries and weigh in.

As a reminder: If you have open government news to share, you can always find me at @digiphile on Twitter, where I share my email address, alex@oreilly.com.

NASA to launch inexpensive Android ‘phonesats’ into space

Android….in….space! This morning, Will Marshall of NASA showed the Android Open Conference plans for a sub-$10,000 Nexus One “phonesat.” Given that the cost of satellites usually measure in the tens or hundreds of millions or so, that’s a rather spectacular cost savings.

Marshall says that this will be the fastest processor to govern a satellite. For reference, Mars Explorer used a 33MHz processor. It sends signals back via amateur radio packet system, rather than ground tracking. The launch video is embedded below:

Great anecdote: when a launch failed, the entire payload fell without parachute into desert. The data was left intact.

William launching 3 Android phonesats in December, in space for 3 weeks. As O’Reilly Media’s Gina Blaber pointed out, they’re “iterating Silicon Valley-style.”

For more, check out the short documentary below about the PhoneSat suborbital test launch in the Black Rock desert:

PhoneSat Rocket Launch Documentary from Ben Howard on Vimeo.

You can follow @NASA_Phonesat on Twitter — there’s no official website yet – and, according to Marshall, eventually check out code on Github, where NASA is open sourcing some software behind it. (And yes, that’s a big deal.)

Open data and maps tell the local story of unemployment and recovery spending

Washington-based DevelopmentSeed continues to tell dazzling data stories with open source mapping tools. This week, they’ve posted a map of the local impact of unemployment and recovery spending. The map visualizes unemployment rate changes at a county level and folds in total economic recovery spending by the government under the American Recovery and Reinvestment Act of 2009. In the map embedded below, red corresponds to an increased unemployment rate and green corresponds to a lower unemployment rate or job growth. Counties that received less than $10 million dollars in recovery spending have a white pattern.

David Cole explains more in a post at DevelopmentSeed.org:

Over the last year, we see that unemployment dropped in 58% of counties by an average of 0.25 percentage points. On average the Recovery Act funded 31 projects at a total of $24,131,582.47 per county. Nationally this works out to about $282.66 in recovery spending per person.

Overall, it’s impossible to tell for sure how much recovery spending improved the economic situation, because we just don’t know how bad things could have been. It may be the case that without spending, this map would have a lot more red. Or maybe not. What’s interesting here is the local impact and information we are able to see from processing a few sets of open data. Check out how your county is doing compared to its surroundings. How about compared to a more or less urban county nearby?


Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively. Save 20% on registration with the code STN11RAD

White House offers “We the People” online petitions at WhiteHouse.gov

With We the People,” the White House has added a new page to WhiteHouse.gov and has announced a potentially disruptive feature for an American public that increasingly turning online for government information and political action: online petitions.

“When I ran for this office, I pledged to make government more open and accountable to its citizens,” reads a statement by President Barack Obama at WhiteHouse.gov. “That’s what the new We the People feature on WhiteHouse.gov is all about – giving Americans a direct line to the White House on the issues and concerns that matter most to them.”

There’s a big idea embedded in this launch, going back to the original compact between the American people and its government. The First Amendment of United States Constitution gives citizens the right to petition their government. In the 21st century, the Internet provides a new means for such petitions to be made.

“With We the People, we’re offering a new way to submit an online petition on a range of issues — and get an official response,” writes Macon Phillips, White House director of digital in a WhiteHouse.gov blog post announcing We the People.

He explains more in the video embedded below and invites people to sign up for email updates when We The People goes live.

Phillips explained the basics of how the White House e-petitions will work on the White House blog. Here’s the key takeaways:

  • Citizens can create or sign e-petitions on a “range of issues” — it’s not clear yet whether citizens can define their own issues or will have to choose from a list.
  • If an e-petition gathers more than 5,000 signatures in 30 days, White House officials will review and answer it.
  • Initially, an e-petition will have a unique URL that only its creator knows. “It’s up to that person to share it in their network to gather an initial amount of signatures — initially 150 — before it is searchable on WhiteHouse.gov. ” It’s not clear what a “network” means but it likely refers to Twitter or Facebook, like the way act.ly works.

There are still many questions that remain in terms of how this is going to work or how it’s going to fit into a 21st century e-democracy. As Phillips recognized, the United States isn’t the first to try this: the United Kingdom offers e-petitions, and according to Phillips, “this work was very helpful as we developed our own.”

The sticky e-widget there is that the UK dropped e-petitions late last year as the new prime minister came into office, due to negative publicity and other issues. Reasonably, we can expect there to be similar challenges with the White House version. The UK has since relaunched its e-petitions site, as Phillips points out, and sharedplans to release the e-petitions code on Github.While it’s not clear yet who built the White House version, it’s possible that they used this code, given the support for open source that Philips has demonstrated over the past three years. The White House built the system in house, according to Phillips.

The initial response online ranges from celebration, including a “high five from PopVox,” to extreme skepticism.

Open government godfather Carl Malamud the long view: “Nice job on We The People,” he tweeted. “Treading in the footsteps of the Founders, petitions have a long and honorable history in our republic!”

“What difference do they make?” tweeted FutureGov Dominic Campbell. “None. Just a distraction technique to pacify the masses. Need new politics not gimmicks. Backbenchers are generally as influential over govt policy as my gran. And she’s dead. Petition / precise tech tool is irrelevant, it’s all about political culture. Petitions are lame. All power is in the hands of govt. Not game changing. More make u feel better/doing *something*.” While the UK petitions have come back, “You’d be hard pushed to find anyone in UK speak +vely of them. Waste of space… think they just reinforce status quo and reward loudest/best organised. Not democracy. ”

Former Sunlight Foundationer Jake Brewer dug into some of the structural issues that exist with this approach. The “only reason “We the People” would [be] useful vs other tools is if @WhiteHouse can convince all they are listening & meaningfully responding,” he tweeted.

“It strikes me though that “giving people a voice” is not at all the problem in gov. Many ways to talk AT gov. Few ways to do so usefully.We simply don’t need more ways to send petitions or gather ideas. We need better ways to listen & operationalize good ideas. What will be an agency’s incentive to take any action based on a petition? Will Whitehouse pressure? Petitions to Congress (theoretically) work because Reps want to be responsive/re-elected. Exec not the same, so how to handle? Guess I’m having a hard time seeing “We the People” as anything more than gov 2.0 theater, and I’d like to be wrong. We simply don’t need more ways to send petitions or gather ideas. We need better ways to listen and operationalize good ideas.”

Questions for We the People

The White House is taking questions on We the People using the feedback form at White House.gov and on Twitter, using the hashtag #WHWeb, where Phillips is listening as @macon44.

Why do petitions at all? “Online petitions are commonly understood, and petitions have been part of our democracy since the beginning,” he tweeted.

When asked by Nancy Scola whether the thinking with We the People is to “have @whitehouse act as [a] clearinghouse for petitions directed towards agencies,” Phillips replied: “People shouldn’t have to decipher how the executive branch is organized in order to speak out about an issue. Processing incoming petitions handled by WH, but relevant petitions will be coordinated w/others as needed, including Agencies.”

In response to a question by @abc4all, Phillips tweeted that “participation in We the People is open to the general public (13yrs+) & requires a valid email address.”

When Alex Rose asked if “WH have a profile of citizens based on petitions we support on We the People? Who can access aggregated data?,” Phillips replied that “only a small group of wh staff will have access to administrative data We the People will be subject to a public privacy policy.”

Here are the questions I’ve tweeted out and their answers:

Who built the e-petitions function? Is it the the same code as the UK tool?

Answer: “System design and development of We the People was developed in house,” tweeted Phillips.

How will identity be handled? How will the White House authenticate citizens to e-petitions government?

Answer: “Lightweight – participation will require an email verification step,” tweeted Phillips. “For now we are using first party WH accounts that verify an email address. Plan to incorporate NSTIC rec’s in future http://1.usa.gov/p7n8HR ”

Do you have to be a citizen?

Answer: “Right now the system only requires valid email and does not verify citizenship,” tweeted Phillips.

How will social media be integrated? 

Answer: “when you create a petition you get a unique link. How you share that is up to you. Will have @facebook & @twitter share [buttons],” tweeted Phillips. yes, just like other content on wh.gov

Can citizens ask questions using We The People on whatever topic they wish or will these be predefined? The screenshot below implies the latter categorization: taxonomy, not folksonomy.

Answer: “there will be a defined set of topic people can choose from but its a wide range, and there will also be ad hoc tags,” tweeted Phillips.

Will there be an API so that civic developers can visualize and analyze them to see if there are duplicates or emerging themes?

Answer: “Not now; API’s for analysis & extending petition functionality on a long list of features we we are considering for future. With [federal CIO] Steve upstairs now, thinking through how that can best work is both a priority & more informed.”

Why build this when services like PopVox, Votizen and Change exist to create social e-petitions?

Answer: “Developing We the People ourselves […] offers the flexibility to adapt to the public response to improve engagement,” tweeted Phillips. “It’s a false choice to say _either_ We the People _or_ others – there’s lots of collaboration ahead, this space is still young.”

There’s another key detail: these e-petitions would go to the executive branch, whereas Votizen and PopVox are targeted at Congress and constituent communications.

The creator of act.ly, Jim Gilliam, offered some of his own perspective and questions. “I built a petition/priority tool White House 2 back in 2008. I learned a lot, happy to share,” he tweeted to Phillips, linking to his post on White House 2.0. On this count, the White House was listening: Phillips asked Gilliam to “dm him his email address.” Here’s a look back at “imagining White House 2.0” from the 2009 Personal Democracy Forum:

“I figured out all the problems, except for one. getting the white house to pay attention. (or maybe it just took 3yrs),” tweeted Gilliam.

He highlighted two issues, one for advocates and one for White House technologists: “”How will the white house use all the email addresses it collects with new petition tool? Advocacy groups will have to decide whether to send their people to whitehouse.gov at the expense of their own list building,” he tweeted. “White house will need some serious anti-spam jujitsu to knock back the tools that scrape congressional forms.”

Perhaps most important, how will citizens know that they’re being heard by the White House, that these e-petitions matter, and that this will not be a public relations exercise that ends with a thank you letter from staff?

This goes to the issue of connecting e-petition action to results. “OpenGov has the equivalent of a “last mile” problem: a culture+digital-infrastructure gap at the workgroup level,” tweeted Dan Latorre, leader of Digital Placemaking and creator ofFixCity.org.

For instance, if enough people sign e-petitions on withdrawing from Afghanistan, supporting gay marriage, legalizing marijuana or opposing ICE takedowns of websites without judicial review, will the White House change its policy?

Stay tuned for answers [See above] and upon launch, outcomes.

US DNI releases government open source handbook at MILOSS 2011

The United States Director of National Intelligence released “GOSS for Govies” during this year’s U.S. military open source software 2011 Working Group 3 in Atlanta, GA. The livestream for the MIL OSS conference is embedded below:

The handbook, which is embedded below, offers a guide to doing open source projects inside of government, based upon the DNI’s experience working on the Ozone Widget Framework.

US DNI Government Open Source Handbook (function() { var scribd = document.createElement(“script”); scribd.type = “text/javascript”; scribd.async = true; scribd.src = “http://www.scribd.com/javascripts/embed_code/inject.js”; var s = document.getElementsByTagName(“script”)[0]; s.parentNode.insertBefore(scribd, s); })();

MySociety launches FixMyTransport to solve transit problems with microactivism

Over in the United Kingdom, MySociety had launched FixMyTransport, a Web application to help citizens solve persistent public transit issues.

According to Tom Steinberg, FixMyTransport is the biggest project for MySociety since they launched WhatDoTheyKnow in 2008.

“This is a huge accomplishment — a nationwide UK system for individuals to document and report problems with any kind of public transportation system,” wrote CivicCommons executive director Andrew McLaughlin this morning. “MySociety has figured out how to route every kind of report to the responsible agency (or even person) — “the service works everywhere in Great Britain, our database has over 300,000 stops and routes for train, tube, tram, bus, coach and ferry.” Great design and interface. Congratulations, +Tom Steinberg and team!”

“We’ve never before launched a site that took so much work to build, or that contained so much data,” writes Steinberg at the MySociety blog, where he explained more about what it’s for. (The emphasis below is mine.)

FixMyTransport has two goals – one in your face, and the other more subtle.

The first goal, as the site’s name suggests, is to help people get common public transport problems resolved. We’re talking broken ticket machines, gates that should be open and stations without stair-free access. We’ll help by dramatically lowering the barrier to working out who’s responsible, and getting a problem report sent to them – a task that would have been impossible without the help of volunteers who gathered a huge number of operator email addresses for us. Consequently the service works everywhere in Great Britain, our database has over 300,000 stops and routes for train, tube, tram, bus, coach and ferry.

The second goal – the subtle one – is to see if it is possible to use the internet to coax non-activist, non-political people into their first taste of micro-activism. Whilst the site intentionally doesn’t contain any language about campaigning or democracy, we encourage and provide tools to facilitate the gathering of supporters, the emailing of local media, the posting of photos of problems, and the general application of pressure where it is needed. We also make problem reports and correspondence between operators and users public, which we have frequently seen create positive pressure when used on sister sites FixMyStreet and WhatDoTheyKnow.

Steinburg goes much deeper into the thinking and process behind FixMyTransport over at Radar today, where he writes about how to create sustainable open data projects with purpose:

I’m not saying it is impossible to hack brilliant things without piles of VC gold. But if you are going to hack something really, genuinely valuable in just a couple of weeks, and you want it to thrive and survive in the real Internet, you need to have an idea that is as simple as it is brilliant. Matthew Somerville’s accessible Traintimes fits into this category, as does FlyOnTime.us, E.ggtimer.com and doodle.ch. But ideas like this are super rare — they’re so simple and powerful that really polished sites can be built and sustained on volunteer-level time contributions. I salute the geniuses who gave us the four sites I just mentioned. They make me feel small and stupid.

If your civic hack idea is more complicated than this, then you should really go hunting for funding before you set about coding. Because the Internet is a savagely competitive place, and if your site isn’t pretty spanking, nobody is going to come except the robots and spammers.

To be clear — FixMyTransport is not an example of a super-simple genius idea. I wish it were. Rather it’s our response to the questions “What’s missing in the civic web?” and “What’s still too hard to get done online?”

As we say here on the Internet, go read the whole thing. If you’re interested in working on stuff making stuff that matters to citizens and make the world a better place, instead of, say, getting them to click on ads, you’ll be glad you did.

ExpertLabs: The future of open government is citizen-focused

On Monday, the White House asked for ideas on the National Plan for open government in the Open Government Partnership. (For background on the initiative, read this digest on Open Government Partnership analysis for context.) Specifically, the White House asked for feedback on ideas related to two of the key challenges from the OGP: improving public services and increasing public integrity.

  • How can regulations.gov, one of the primary mechanisms for government transparency and public participation, be made more useful to the public rulemaking process?
  • OMB is beginning the process of reviewing and potentially updating its Federal Web Policy. What policy updates should be included in this revision to make Federal websites more user-friendly and pertinent to the needs of the public?
  • How can we build on the success of Data.Gov and encourage the use of democratized data to build new consumer-oriented products and services?

Today, Clay Johnson offered the White House a deep set of recommendations for open government in response to the three questions it posed, including better ways to use open data, social media, improving regulations, public comment, and the developer community better. If you’re interested in open government, it’s a must-read.

We believe the future of open government is citizen focused — to be open to engagement on the terms that citizens are used to in the venues they’re accustomed to.

The growth of social media since the delivery of the initial open government directive, and its adoption by agencies’ communications departments requires the next step: for social media to also be used in giving citizens a voice in regulatory decisions. We believe the future of open government is about discoverability. Moving data or regulations from print publications to the online world results in a net loss if there are less people viewing them. Government should work hard to make sure that all publicly available information is discoverable by search engines, and via social media.

We believe that the future of open government is through the engagement of open source communities, and that agencies should begin to open up to their participation. Government treats lawyers as experts in the field of law, why not treat developers as experts in the field of processing data? Take the next step and participate with them directly, rather than through independent vessels.

– Clay Johnson (@cjoh), Recommendations for Open Government

For more on the decision to use email in the consultation, read Tiago Peixoto’s post on Google Plus.