The Financial Times opinion page weighed in on the state of open government in the United States and the United Kingdom today. The gist of the comment is that President Obama’s open data agenda has slightly stalled. A key excerpt, below, focused on potential sources of delay.
Barack Obama, the US president, and David Cameron, the UK premier, took power with a more mundane vision of transparency than that of Julian Assange – that governments should keep as many secrets only as truly necessary. Both were excited by the dry business of putting public data online. But they were right to be: signs that Mr Obama, in particular, has seen his efforts stall are a shame.
WikiLeaks releases highly sensitive diplomatic material. But governments collect reams of less delicate data, from exam results and hospital inspections to maps and weather reports. This can often be reused on the web, creating profitable businesses, helpful advice to citizens or tools that hold leaders to account. Not all such data are useful, just as not all scientific discoveries lead to new drugs. And not all should be published, if they break data protection rules. But more should be than at present.
It would be irresponsible to deny a state’s right to protect its interests, and those of its citizens, by keeping some secrets. Open government never meant the real-time disclosure of all state activity. But the reasons for hiding public information too often stem from fear of embarrassment, force of habit or politicians going cold on previous ideals – not the public interest.
Here Mr Obama began strongly, speedily unveiling schemes to unlock new data. Yet his progress has slowed as departments delay and fudge and the White House’s attention is diverted elsewhere. Mr Cameron has done better, in part by being more focused — and thus picking fewer fights with often recalcitrant civil servants.
The contest is reasonably straightforward, at least as a proposal; challenge developers to create applications that test networks that inform consumers about broadband Internet connections. Specifically:
The Open Internet Challenge is designed to encourage the development of creative, innovative and functional Internet software tools for fixed or mobile broadband that provide users with real-time data about their Internet connection and, when aggregated, can show Internet-wide patterns and trends.
Apps that can detect, aggregate and analyze such information might be of use to both the agency and consumers alike, in terms of making policy or consumption decisions:
The Open Internet Challenge seeks to encourage the development of creative, innovative and functional applications that provide users with information about the extent to which their fixed or mobile broadband Internet services are consistent with open Internet principles. These software tools could, for example, detect whether a broadband provider is interfering with DNS responses, application packet headers, or content.
These applications should also collect anonoymized data that is useful for network research and analysis that enables the discovery of patterns and trends in Internet openness.
One popular platform for Internet software tools is Measurement Lab (M-Lab), which “is an open, distributed server platform for researchers to deploy Internet measurement tools.” Those interested in running their software tools on the M-Lab platform should contact the M-Lab steering committee, which coordinates research on the M-Lab platform.
The commission is also soliciting research as part of the challenge:
In addition to measurement tools, this challenge also seeks research papers that analyze relevant Internet openness measurement techniques, approaches, and data. The Challenge is designed to encourage and reward the creation of novel, innovative and useful research. The research must be new or recent and directly involve open Internet principles. For example such research may illuminate how widely fixed and mobile networks observe the FCC’s open Internet principles or how advanced network services can be provided in a way that adheres to the spirit of the open Internet. Such research papers need to have been peer-reviewed by a recognized scientific conference or journal and must have been published since January 2007. (Dissertations, white papers and technical reports are not acceptable, but may be referenced for further details within the paper.) Research on Internet openness can improve policy making and advance Internet transparency, which helps to sustain a healthy Internet.
Will it take off? Another challenge, so to speak, might be the incentives. The winners will earn a free (up $500/person in travel expenses) trip to FCC headquarters in Washington, DC, where they’ll go to an FCC Chairman’s reception, present their work to the Commission, receive plaque and “have their apps and research featured on the FCC’s website and social media outlets.”
It’s also not clear how the development community will feel about the FCC after today’s hearing on somewhat controversial net neutrality rules, for which a public document still hasn’t been published online. Geeks and government have converged at the FCC before. If this challenge is going to take off, they’ll need to do a lot of outreach to encourage the development community to participate, which in turn will likely also mean exactly what the new open Internet principles will mean in practice. Stay tuned.
Today at George Washington University, GW Office of Entrepreneurship & iStrategyLabs is hosting a “Gov 2.0 Startup Lab” to highlight the opportunities around civic entrepreneurship. According to the event organizers, the event “will stimulate new ideas and bring together collaborators for the 2011 GW Business Plan Competition with $50,000 in prizes, and further establish GW as a hotspot for innovation and entrepreneurship.” The livestream is embedded below:
“The single best thing we could do in open government is to get the American people engaged in the question of what high value data is,” said Aneesh Chopra, the first United States chief technology officer, speaking at this morning’s Politico “What’s Next in Tech” forum in Union Station. Video is below:
In an interview with Politico’s technology editor, Kim Hart, Chopra looked back at the lessons learned from his first two years on the job and ahead, appropriately, to what to expect in tech policy from the Obama administration. They covered a lot of ground, from open government successes to what’s next in Congress (hint: watch the push to open up spectrum for first responders) to supporting entrepreneurial growth.
Lessons learned
What were Chopra’s lessons learned? He offered up three examples.
First, with support from the President, Chopra said that they’ve been able to open up discussion and build trusted relationships across the federal government, which has been “critical” to improving the way technology could be used and the long term policy posture.
Second, with that support, he’s been surprised on seeing the pace of response become fast. There’s a “lesson on balance of getting long term balance, versus getting results in 90 days,” he said, referring to the turnaround on projects like HealthCare.gov.
Third, Chopra emphasized the role of “government as a convener,” where the administration can use its influence to bring people together to accomplish goals with technology without new regulations or legislation.
Working tech policy levers
What are the levers that the first US CTO has worked to try to galvanize action on the administration’s priorities?
First, a commitment to openness. From Manor, Texas, to inner cities, “people have found ways to tap into info in ways that helps them do something different,” said Chopra, speaking to the phenomenon of Gov 2.0 going local. “85 to 90% of that activity is happening in places we wouldn’t have imagined,” not gathering in Washington.
Second, Chopra cited the White House’s work towards “voluntary, consensus-driven standards,” noting that he was ” very proud of the work on NHIN Direct.”
Finally, Chopra noted that there’s some $150 billion spent on research and development every year, which offers a number of ways to push forward with innovation in priorities like healthcare IT, energy, smart grid or communications.
Making meaningful use modular
Given the new Congress coming in to Washington, Chopra’s description on the bipartisan agreement on tech policy from his time in Virginia under Republican leadership has to be more than a little strategic. He talked about “getting to the right answer,” referring back to an former manager, David Bradley, and his management strategy of “True North.”
That approach will be rested in the next Congress, on rulemaking. and in moving forward with the tech policy decisions. Outside of the healthcare bill that President Obama signed into law, which continues to meet with significant opposition in Congress, Chopra noted that “healthcare is signature part of President’s agenda,” specifically advanced by more than 20 billion dollars in Recovery Act spending on healthcare IT.
Chopra looked back at two decisions related to approaching technology policy a bit differently. “Rather than walking into Best Buy and buying software, we created more flexible standards for meaningful use,” he said. As a result, “entrepreneurs that never thought of themselves as EMR companies are entering the market.”
The decision to make meaningful use more modular was also significant, asserted Chopra. “We opened up the regulatory regime so you could certify each and every regulatory module.”
In aggregate, Chopra associated that R&D investment, work to convene conversations, open up data and create more flexible regulatory regimes with a better outcomes: venture capital investment in HIT going up by 39%, citing a statistic from the National Venture Capital Association.
How did Chopra respond? He asked for more criticism, responding that you “must listen to people who are frustrated” and consider that much of the tech platform is in the space “where the plane is yet to land.” If you go through campaign promises, and look at executive ability to move the needle on different areas, Chopra asserted that the
biggest part of that – open government – has gone ahead. “It’s not ‘mom and apple pie perfect’,” he said, but they’re proud of delivering on 90 day deliverables like standards, or websites.
Part of the challenge of delivering on campaign promises is that budgetary or legislative action requires different stakeholders, observed Chopra, a reality that will become even more sharply defined in the next Congress. “The Recovery Act is a unique moment in time,” which, as he argued is “overwhelmingly the vehicle for campaign promises” in health IT and clean tech.
What’s next in United States technology policy?
First, it’s clear that Chopra and the Obama administration is thinking about online privacy, with the recently announced Internet privacy committee. There are open questions about how much portfolio, budget, subpoena power or other authority any new position would hold, but it’s an area to watch. Chopra said that he had met with Representative Bob Goodlatte (R-VA) and had found him supportive of privacy policy.
Chopra also met with Representative Darrell Issa (R-CA), who is very supportive of increased government transparency through technology. Issa, a successful technology entrepreneur, is one of the most knowledgeable members of Congress when it comes to technology. Whatever comes out of his his legislative staff, or the new House Oversight committee, which he will chair, could represent a step forward for open government after the 2010 election.
Chopra also emphasized “modest but significant actions” that could improve the conditions for tech entrepreneurs in the United Stats, from open government data to regulatory action to smart grid or support for new learning technologies. On that count, Chopra offered up a “scoop” to Kim Hart, observing that the next area where he will focus on driving innovation will be into learning technologies, with more news coming at a Brookings Institute event in December.
The top opportunities that Chopra sees for entrepreneurs are in healthcare and energy, the former of which is already becoming hot with more healthcare apps provisioned with open healthcare data
“One policy lever is the role of public-private partnerships,” observed Chopra, highlighting the growth in STEM education, with over half a billion dollars in investment. “It’s not the money, it’s the platforms,” he said.
Chopra fielded a question Congressman Wu (D-OR), the current chairman of the House technology and innovation committee. After a discursion into what went wrong for the Democratic Party in the midterm, Wu asked what the next priority will be for Congress and Chopra to work together upon. His answer was simple: spectrum policy, emphasizing voluntary processes for formulating solution. The priority, he said, was to get a broadband network for public safety that’s interoperable for first responders.
Finally, Chopra talked about the story of the Alfred brothers, who founded Brightscope in California in 2008. The story of Brightscope is important: data driving the innovation economy. They knew about key data on 401(k) plan fees at the Department of Labor, worked hard to liberate it and now have a successful, growing startup as a result.
Is sunlight the best disinfectant, as Supreme Court Justice Brandeis famously said?
This week in Washington, D.C., hundreds of experts have come together at the International Open Government Data Conference (IOGDC) to explore how data can also help citizens to make better decisions and underpin new economic growth. The IOGDC agenda is online, along with the presenters.
“Since the United Kingdom and United States movement started, lots of other countries have followed,” said Tim Berners-Lee, the inventor of the World Wide Web. Canada, New Zealand, Australia, France and Finland are all working on open data initiatives.
As he noted with a smile, the “beautiful race” between the U.S. and U.K. on the Data.gov and Data.gov.uk websites was healthy for both countries, as open data practitioners were able to learn from one another and share ideas. That race was corked off when former British Prime Minister Gordon Brown asked Tim Berners-Lee how the United Kingdom could make the best use of the Internet. When Berners-Lee responded to “put government data on the Web,” Brown assented, and Data.gov.uk was born.
Berners-Lee explored the principles of open linked data that underpin data.gov.uk and open government. Specifically, he emphasized his support for open standards and formats over proprietary versions of either, inviting everyone present to join the W3C open government data working group.
1 Stars for putting data on the Web at all, with an open license. PDFs get 1 star.
2 Stars if it’s machine-readable. Excel qualifies, though Berners-Lee prefers XML or CSVs.
3 Stars for machine-readable, non-proprietary formats
4 Stars if the data is converted into open linked data standards like RDF or SPARQL
5 Stars when people have gone through the trouble of linking it
“The more transparency there is, there more likely there is to be external investment,” said Berners-Lee, highlighting the potential for open government data to make countries more attractive to the global electronic herd.
Will open data spread to more cities, states and countries, as HTML did in the 1990s? If the open standards and technologies that Berners-Lee advocates for are adopted, perhaps. “The Web spread quickly because it was distributed,” said Berners-Lee. “The fact that people could put up Web servers themselves without asking meant it spread more quickly without a centralized mandate.”
Putting open government data to work
Following Berners-Lee, federal CIO Vivek Kundra highlighted how far the open government data movement has come in the short time since President issued his open government memorandum in January 2009.
Kundra remarked that he’s “seeing more and more companies come online” in the 7 countries have embarked on an open government movement that involves democratizing data. He also reeled off a list of statistics to highlight the growth of the Data.gov platfrom.
Within the boundaries of the United States, Kundra observed that 16 states and 9 cities have stood up open data platforms
256 applications have now been developed on top of the Data.gov platform
There are now 305,692 data sets available on Data.gov
Since Data.gov was launched in 2009, it has received 139 million hits.
The rapid growth of open government data initiatives globally suggests that there’s still more to come. “When I look at Data.gov platform and where we are as a global community, we’re still in the very early days of what’s possible,” said Kundra.
He emphasized that releasing open data is not just a means of holding government accountable, focusing three lenses on its release:
Accountability, both inside of government and to citizens
Utility to citizens, where, as Kundra said, “data is used in the lives of everyday people to improve the decisions they makes or services they receive on a daily basis
Economic opportunities created as a results of open data.
“The simple act of opening up data has had a profound impact on the lives of ordinary people,” said Kundra, who pointed to the impact of the Veterans Administration’s Blue Button. Over 100,000 veterans have now downloaded their personal health records, which tundra said has stimulated innovation in blue button readers to connect systems from Google or Microsoft.
“I predict that we’ll have an industry around data curation and lightweight applications,” said Kundra. “The intersection of multiple data sets are where true value lies.” The question he posed to the audience is to consider how the government will move to towards an API-centric architecture that allows services to access data sets on a real-time basis.
When asked about that API strategy and the opportunity costs of pursuing it by open government advocate Harlan Yu, Kundra said that he follows an “80/20” rule when it comes to the government building apps vs third parties. “Do we want to be a grocery store or a restaurant when it comes to the Data.gov platform and movement?” he asked.
As a means of answering that question, Jeanne Holm, the former chief knowledge architect at the NASA Jet Propulsion and current Data.gov evangelist, announced a new open government open data community at Data.gov that will host conversations about the future of the platform.
A United States-United Kingdom partnership on open government, which will include an open government data camp later this week
The release of a concept of operations for Data.gov, embedded below, which includes strategic goals for the site, an operation overview and a site architecture.
What do the two technology leaders see as a vision for success for open government data?
For Berners-Lee, it was to be able to directly access data from a dashboard on laptop, rather than indexes and catalogs on Data.gov and data.gov.uk. He talked about accessing open government data that wasn’t just machine-readable or linked to other sets but directly accessed from his local machine, called through powerful Python scribts.
In contrast, Kundra talked about being able to go to a store like Brookstone and “in the same way you can buy alarm clocks with data in the weather channel,” how data from federal agencies had been employed to provision objects from everyday life.
To be fair, there’s a long way to go yet before that vision becomes reality. As Andrew Odewahn pointed out at Radar, earthquakes are HUGE on Data.gov, consistently bringing in the most downloads, even ahead of those product recall data sets. While provisioning recurring visualization in the Popular Mechanics iPad App might be useful to the publisher, it’s also a reminder that the full vision for delivering utility to citizens through open data that Kundra hopes for hasn’t come to fruition as a result of Data.gov – yet.
As Anil Dash blogged earlier today, Andy Baio has joined Expert Labs as a project director for ThinkUp App. According to Dash, Baio will be working as a “Director of Hacks,” focusing on making better use of the data collected by apps like ThinkUp.
Andy’s new role marks the beginning of a whole new phase for Expert Labs; We’re now very tightly focused on working with agencies that want to get crowdsourced feedback, and the biggest request those agencies make is to better understand the ideas that people submit. We’re answering those requests with smart tools for presenting, visualizing, sorting and filtering ideas and suggestions that come in through Facebook and Twitter.
In short, we’ve already got the ability to collect responses to policy questions through a social network, and now we’ll be able to turn those responses into real insights.
In his own blog post about on joining Expert Labs, Baio elaborated further on the mission he’s taking on:
Our goal’s to help government make better decisions about policy by listening to citizens in the places they already are: social networks like Twitter and Facebook.
Our first project is ThinkUp, an open-source tool for archiving and visualizing conversations on social networks. It started with Gina scratching a personal itch, a way to parse and filter @replies. But it’s grown to be something more: a tool for policy makers to harness the collective intelligence of experts.
There’s tons to do, but I’m particularly excited to tackle ThinkUp’s ability to separate signal from noise, making it easier to derive meaning from hundreds or thousands of responses, using visualization, clustering, sentiment analysis, and robotic hamsters. I’m planning on building some fun hacks on top of ThinkUp, as well as keeping an eye open for other vectors to tackle our core mission.
And his motivations for going to work in open government:
So, why would I go to a Gov 2.0 non-profit? For three main reasons:
It’s important. To tackle our most serious national issues, we need better communication between government and citizens. I want my son to grow up in a world where he doesn’t feel disconnected and disillusioned by government, and I want government to meet the needs of the people, rather than favoring those with the most money or the loudest voices.
It’s exciting. Technology is quite possibly our best hope of breaking down that divide, using social tools to disrupt the way that governments are run and policy is made. I love designing and building tools that use social connections to tackle difficult problems, and it feels like government is an area ripe for disruption.
I love the team. I’ve known Anil and Gina for years and have long admired their work. They’re both extraordinarily talented and creative people, and I feel lucky to call them both friends. The opportunity to work with them was too hard to pass up.
Welcome to the Gov 2.0 community, Andy. I saw him hard at work at the first FCC Open Developer Day in Washington this week, where we talked more about what it means to have members of the technology community work on technology to make government function more effectively.
When high profile members of the Web 2.0 community pitch in, their networks will learn more about what’s going on. Call it “digital diffusion.” That attention, scrutiny and tinkering is likely to be a good outcome for everyone.
For more on that count, I interviewed Expert Labs’ Gina Trapani at the FCC about ThinkUp app, the apps that came out of the developer day and this issue: what does it mean when geeks try to help government work better. It was a great conversation and I encourage readers of this blog to embed it elsewhere.
Yesterday, the San Francisco Board of Overseers voted unanimously to approve the first municipal an open data law* in the United States. November 9, 2010 is a milestone for open government.
That said, the moment comes with major caveats on open data appetite vs reality, as John Wonderlich points out at the Sunlight Foundation.
Here are the changes that came out of the committee, though:
AMENDED on Page 2, Lines 2-6 by adding ‘make reasonable efforts to’ after ‘shall’; deleting ‘all’ after ‘available’; adding ‘and with applicable law, including laws related to privacy’ after ‘(“COIT”)’; Page 3, Line 2 by deleting ‘all’ after ‘accounting of’; and on Page 3, Line 5 by replacing ‘would’ with ‘could’.
Those changes are comical. The ordinance now mandates that agencies have to try to follow the standards set by an IT oversight body, to release some information based on an audit of some subset of public data.
This is the language of the low-hanging fruit — the kind of aspirational mandate that isn’t really a mandate at all, but more of a statement of goals and principles, lofty rhetoric with a roadmap made up of other road maps, and plans for other plans.
Again, this declaration, and the others like it, aren’t inappropriate. Their effects probably vary based on the context, based on the actual commitment of everyone involved, from government officials to citizens.
If all these declarations do is to win some of the easy fights, then they’re well worth the effort, because those obvious decisions (like open local transit data) have been gotten wrong far too often in the past, and can have significant positive effects when they’re gotten right.
We need to avoid, however, thinking that these top-level political declarations are something they aren’t. Governments have a vast stores of information, and most of it won’t be reached by these pronouncements. There’s a whole world between the initial urge for government to “put all its data online” and the “please try to put some data online better when you remember to” that that urge decays into. At the first sign of trouble, “all” disappears, “must” becomes “should,” and a mandate becomes a suggestion. The San Francisco ordinance demonstrates just how far a vision for transparency can be from the kind of nuance and structure that makes it possible.
*CORRECTION: As Philip Ashlock pointed out, “Portland’s open data & open source legislation passed a year ago in Oregon, though it wasn’t as explicit as San Francisco’s. See open data policy [at CivicCommons].” In his comment to FreeGovInfo’s story, he notes that Portland’s “law was largely influenced by the legislation passed in Vancouver about six months before that. The major difference (and a very important one) with San Francisco’s new legislation is that it is more explicit about using open licenses with the open data.”
I regret the error; the tweet from the White House @OpenGov account (“San Francisco passes first municipal open data law”) led me astray.
The text of the law is embedded below, courtesy of Chris Dorobek at Federal News Radio.
E Pluribus Unum 