The VA is being neither open nor transparent about its missing open government plans or policies. On March 29, 2018, I made a Freedom of Information Act request to the United States Department of Veterans Affairs (VA) in which I … Continue reading
Yesterday, I wrote up 15 key insights from the Pew Internet and Life Project’s new research on the American public’s attitude towards open data and open government. If you missed it, what people think about government data and the potential impact of releasing it is heavily influenced by the prevailing low trust in government and their politics.
Media coverage of the survey reflected the skepticism of the reporters (“Most Americans don’t think government transparency matters a damn“) or of the public (“Who cares about open data” and “Americans not impressed by open government initiatives“). This photo by Pete Souza below might be an apt image for this feeling:
Other stories pulled out individual elements of the research (“Open data on criminals and teachers is a-okay, say most US citizens” or mixed results (“People Like U.S. Open Data Initiatives, But Think Government Could Do More” and “Sorry, open data: Americans just aren’t that into you“) or general doubts about an unfamiliar topic (“Many Americans Doubt Government Open Data Efforts“). At least one editor’s headline suggested that the results were an indictment of everything government does online: (“Americans view government’s online services and public data sharing as a resounding ‘meh’.) Meh, indeed.
As usual, keep a salt shaker handy as you browse the headlines and read the original source. The research itself is more nuanced than those headlines suggest, as my interview with the lead researcher on the survey, John Horrigan, hopefully made clear.
Over at TechPresident, editor-in-chief Micah Sifry saw a glass half full:
- Digging deeper into the Pew report, it’s interesting to find that beyond the “ardent optimists” (17% of adults) who embrace the benefit of open government data and use it often, and the “committed cynics” (20%) who use online government resources but think they aren’t improving government performance much, there’s a big group of “buoyant bystanders” (27%) who like the idea that open data can improve government’s performance but themselves aren’t using the internet much to engage with government. (Heads up Kate Krontiris, who’s been studying the “interested bystander.”)
- It’s not clear how much of the bystander problem is also an access problem. According to a different new analysis done by the Pew Research Center, about five million American households with school-age children–nearly one in five–do not have high-speed internet access at home. This “broadband gap” is worst among households with incomes under $50,000 a year.
Reaction from foundations that have advocated, funded or otherwise supported open government data efforts went deeper. Writing for the Sunlight Foundation, communications director Gabriela Schneider saw the results from the survey in a rosy (sun)light, seeing public optimism about open government and open data.
People are optimistic that open data initiatives can make government more accountable. But, many surveyed by Pew are less sure open data will improve government performance. Relatedly, Americans have not quite engaged very deeply with government data to monitor performance, so it remains to be seen if changes in engagement will affect public attitudes.
That’s something we at Sunlight hope to positively affect, particularly as we make new inroads in setting new standards for how the federal government discloses its work online. And as Americans shift their attention away from Congress and more toward their own backyards, we know our newly expanded work as part of the What Works Cities initiative will better engage the public, make government more effective and improve people’s lives.
Jonathan Sotsky, director of strategy and assessment for the Knight Foundation, saw a trust conundrum for government in the results:
Undoubtedly, a greater focus is needed on explaining to the public how increasing the accessibility and utility of government data can drive accountability, improve government service delivery and even provide the grist for new startup businesses. The short-term conundrum government data initiatives face is that while they ultimately seek to increase government trustworthiness, they may struggle to gain structure because the present lack of trust in government undermines their perceived impact.
Steven Clift, the founder of e-democracy.org, views this survey as a wakeup call for open data advocates.
One reason I love services like CityGram, GovDelivery, etc. is that they deliver government information (often in a timely way) to the public based on their preferences/subscriptions. As someone who worked in “e-government” for the State of Minnesota, I think most people just want the “information” that matters to them and the public has no particular attachment to the idea of “open data” allowing third parties to innovate or make this data available. I view this survey as a huge wake up call to #opengov advocates on the #opendata side that the field needs to provide far more useful stuff to the general public and care a lot more about outreach and marketing to reach people with the good stuff already available.
Mark Headd, former chief data officer for the City of Philadelphia and current developer evangelist for Accela software, saw the results as a huge opportunity to win hearts and minds:
The modern open data and civic hacking movements were largely born out of the experience of cities. Washington DC, New York City and Chicago were among the first governments to actively recruit outside software developers to build solutions on top of their open data. And the first governments to partner with Code for America – and the majority over the life of the organization’s history – have been cities.
How do school closings impact individual neighborhoods? How do construction permit approvals change the character of communities? How is green space distributed across neighborhoods in a city? Where are vacant properties in a neighborhood – who owns them and are there opportunities for reuse?
These are all the kinds of questions we need people living and working in neighborhoods to help us answer. And we need more open data from local governments to do this.
If you see other blog posts or media coverage that’s not linked above, please let me know. I storified some reactions on Twitter but I’m certain that I missed conversations or opinions.
There are two additional insights from Pew that I didn’t write about yesterday that are worth keeping in mind with respect to how how Americans are thinking about the release of public data back to the public. First, it’s unclear whether the public realizes they’re using apps and services built upon government data, despite sizable majorities doing so.
— Alex Howard (@digiphile) April 21, 2015
Second, John Horrigan told me that survey respondents universally are not simply asking for governments to make the data easier to understand so that they can figure out what I want to figure out: what people really want is intermediaries to help them make sense of the data.
“We saw a fair number of people pleading in comments for better apps to make the data make sense,” said Horrigan. “When they went online, they couldn’t get budget data to work. When the found traffic data, couldn’t make it work. There were comments on both sides of the ledger. Those that think government did an ok job wish they did this. Those that thin government is doing a horrible job also wish they did this.”
This is the opportunity that Headd referred to, and the reason that data journalism is the critical capacity that democratic governments which genuinely want to see returns on accountability and transparency must ensure can flourish in civil society.
If a Republican is elected as the next President of the United States, we’ll see if public views shift on other fronts.
On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.
A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.
Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.
In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:
The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.
This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.
Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.
Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.
Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.
If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.
[Image Credit: Govfresh, via USASpending.gov]
Last year, a successful Freedom of Information request for the United States enterprise data inventory by the Sunlight Foundation was a big win for open government, nudging Uncle Sam towards a better information policy through some creative legal arguments. Today, the federal government started releasing its enterprise indices at data.gov. You can browse the data for individual agencies, like the feed for the Office for Personnel Management, using a JSON viewer like this one.
“Access to this data will empower journalists, government officials, civic technologists, innovators and the public to better hold government accountable,” said Sunlight Foundation president Chris Gates, in a statement. “Previously, it was next to impossible to know what and how much data the government has, and this is an unprecedented window into its internal workings. Transparency is a bedrock principle for democracy, and the federal government’s response to Sunlight’s Freedom of Information request shows a strong commitment to open data. We expect to see each of these agencies continue to proactively release their data inventories.”
Understanding what data an organization holds is a critical first step in deciding how it should be stored, analyzed or published, shifting towards thinking about data as an asset. That’s why President Barack Obama’s executive order requiring federal agencies to catalog the data they have was a big deal. When that organization is a democratic government and the data in question was created using taxpayer funds, releasing the inventory of the data sets that it holds is a basic expression of open and accountable government.
In a fascinating turn of events, rainy Oregon is embracing sunlight online after a scandal that led to the resignation of its governor. After governor Kate Brown was sworn in as the 38th governor of the state of Oregon, replacing fellow Democrat John Kitzhaber, her administration chose to try to restore public trust by not only posting public records requests online but including the authors, status and a downloadable link to the records themselves, once fulfilled. The records only go back to January 15th, 2015, with a note that requests made prior to that date are “still being processed.”
The City of Oakland’s public records system, built by Code for America, does the same thing but this appears to set a new bar for state government that’s unmatched in the United States of America. As has been reported elsewhere, exemptions to Oregon’s public records laws mean that this website will be no panacea, but it looks like progress from 3000 miles away in snowy Cambridge.
As Kirk Johnson reported for the New York Times, Brown’s record includes open government work while she was a state legislator, where, as Senate Majority Leader, she worked to reform Oregon’s ethics law and helped to enact legislation that created an online campaign finance database.
“…throughout my 24 years in public service, I have also sought to promote transparency and trust in government, working to build confidence that our public dollars are spent wisely,” she said, in her inaugural speech.
Later in her remarks, Governor Brown said that “we must seize this moment to work across party lines to restore the public’s trust. That means passing meaningful legislation that strengthens the capacity and independence of the Government Ethics Commission. We also must strengthen laws to ensure timely release of public documents.”
On that count, it’s notable that two of the records requests that have been posted for download involve Cynthia Hayes, the fiancee of former Governor Kitzhaber who was at the center of the scandals that led to his resignation. One comes from Margaret Olney, who is quite likely the same Margaret Olney who served in Oregon’s Department of Justice. The other requester was Alejandra Lazo, who co-authored a Wall Street Journal article on former Governor Kitzhaber’s resignation. In an interesting sidenote, the records for both responses were uploaded to Dropbox.
If you know of another state that meet or exceeds this standard for digital transparency, or have experience or feedback regarding the quality or importance of the public records posted by Oregon, please let us know in a comment.
Emily Shaw, the national policy manager at the Sunlight Foundation, said on Twitter that she has not seen any other state’s public records system exceed this standard of transparency.
Today, Federal Communications Commission Commissioner Ajit Pai released a statement & fact sheet on the non-public FCC’s proposal for Open Internet Rules, which would be applied to regulating broadband Internet service providers. He tweeted out a picture of himself holding the document (above) several days ago. In his statement today, Pai said that the American people are being misled about what’s in the rules and broke out an outline of the 300-page document. Reporting and analysis of Pai’s objections to the proposal may be found at the New York Times and National Journal. Subsequent to Pai’s statement and press conference, FCC Special Counsel for External Affairs Gigi Sohn tweeted out a series of rebuttals, including a clarification regarding the size of the document.
Here, I’ll focus on something else: the commissioner took FCC Chairman Tom Wheeler to task for not releasing them ahead of the scheduled vote on February 26th, in DC, although Pai did note that doing so would be “unprecedented.” In fact, FCC commissioner Mike O’Rielly tweeted much the same thing, stating that “all FCC meeting items should be made public when circulated to Commissioners. ”
In August, I wrote a blog post urging the Commission to post on its website the actual text of the items to be considered at our Open Meetings at the same time they are provided to Commissioners. I made the suggestion because the inability of the public to obtain a complete picture of what is in a pending notice of proposed rulemaking or order routinely leads to confusion over what exactly is at stake. Making matters worse, Commissioners are not allowed to reveal the substantive details to outside parties. We can’t even correct inaccurate impressions that stakeholders may have received, and we are barred from discussing what changes we are seeking. This barrier to a fulsome exchange can be extremely frustrating for all involved. Despite positive feedback from people at the FCC, outside parties, Members of Congress,  and the general public, four months later, we have yet to post a single meeting item in advance. Moreover, the lack of full disclosure and transparency has continued to be a problem as some parties have not been fully briefed on recent items, such as the recently adopted 911 Reliability NPRM, while others are not briefed at all. The reason that nothing has happened, I am told, is that there are two basic concerns with the proposal: 1) that it could be harder to comply with the Administrative Procedure Act (APA); and 2) that it could be more difficult to withhold documents under the Freedom of Information Act (FOIA). I do not find either argument persuasive or insurmountable
Wheeler took a question on this issue in a press conference last month: “The precedent here, through Democratic chairmen and Republican chairmen, has always been that just like, you know, in the court system, and elsewhere that when you’re handling cases like this, you have an internal discussion and then you release what the result of that vote is. And you don’t change that decades of precedent overnight without following the procedures to review questions like that.”
Asked for comment, FCC press secretary Kim Hart Jonson said that “Chairman Wheeler circulated his proposal to his fellow Commissioners three weeks before the planned vote in accordance with long-standing FCC process. We are confident the other Commissioners will give the proposal an exhaustive review. The Chairman looks forward to receiving their input and releasing it to the public after the February 26 vote.”
What this boils down to is that publishing proposed rules before a vote is technically legal but has never been done before. As Pai himself noted, “FCC rules prohibit disclosure of nonpublic information except as authorized by the Chairman.” Then-FCC chairman Julius Genachowski didn’t pull the full version of the 2010 Open Internet rules up until right before Christmas, well after the vote. The standard FCC procedure for rulemaking is to circulate the draft rules at least 3 weeks prior to a vote, incorporate edits received from FCC commissioners, then finalize everything. In practice, however, some commissioners have complained that edits may be made right up until a vote. Once rules have been voted upon, they’re published in the federal register.
As long-time readers know, I was critical of that decision at the time, and remain so. To reiterate what I said them, Genachowski made a commitment to a more open, transparent and data-driven F.C.C. under President Obama’s Open Government Directive. In many respects, in its first year of open government, the agency made commendable progress, with strides towards taking public comment through e-rulemaking at OpenInternet.gov, Broadband.gov and Reboot.FCC.gov. The sites were deployed by an able new media team that has used online communications in unprecedented ways. The chairman and his managing director, Steven Van Roeckel, both deserved credit for their plans to reboot FCC.gov as a platform for government including the use of APIs and open source technologies like Drupal.
When it comes to the question of whether the public could see the proposed Open Internet rules before the commissioners vote upon it, the agency fell short of its transparency pledge. I have not found a legal precedent that explicitly gives the agency authority to keep the text of a proposed rule secret until it is voted upon by the Commission. While it is true that the FCC does not appear to be under no legal obligation to do so, given that the members of the commission presumably had to negotiate on the details of the final rules for vote, the decision not to share a version publicly may have made such discussion more flexible.
The choice not to post the proposed rules online before the most reced vote is an example of the same level of government transparency in the creation of important regulation as before, not more. It’s possible that the legal context for how the FCC operates might change in the future, if a bill by Senator Dean Heller (R-NV) were to be passed. Per his office, the FCC Reform Act would:
“provide more opportunities for the public to see pending FCC action by publishing the exact rule or amendment to a rule for at least 21 days and by allowing any Commissioner to ask for a vote on any order issued by a bureau. *Enhance consistency and transparency in the Commission’s operations by requiring the FCC to establish procedures for handling extensive new data submitted towards the end of a comment period, adequate review and deliberation regarding pending orders, providing the status of open rulemaking proceedings, minimum public review periods for statistical reports and ex parte communications. *Empower the Commission to operate more efficiently through reform of the “sunshine” rules, allowing a bipartisan majority of Commissioners to meet for collaborative discussions subject to transparency safeguards.
A provision that the FCC must “identify a market failure, consumer harm, or regulatory barrier to investment before adopting economically significant rules,” however, may well mean that Democrats in the Senate won’t support its advance. As always with these net neutrality stories, to be continued.
UPDATE: Commissioner Pai tweeted another picture of a revised plan February 21, including his intention to vote against it and again noting that the public cannot see it before the vote. In the tweet, he says the plan is now 317 pages, which means it has been trimmed by 15 pages.
Here is Pres. Obama’s revised 317-page plan to regulate the Internet. The public still can’t see it. I’m voting no. pic.twitter.com/RirBVPChmV
— Ajit Pai (@AjitPaiFCC) February 25, 2015
On Monday, I delivered a short talk on data journalism, networked transparency, algorithmic transparency and the public interest at the Data & Society Research Institute’s workshop on the social, cultural & ethical dimensions of “big data”. The forum was convened by the Data & Society Research Institute and hosted at New York University’s Information Law Institute at the White House Office of Science and Technology Policy, as part of an ongoing review on big data and privacy ordered by President Barack Obama.
Video of the talk is below, along with the slides I used. You can view all of the videos from the workshop, along with the public plenary on Monday evening, on YouTube or at the workshop page.
Here’s the presentation, with embedded hyperlinks to the organizations, projects and examples discussed:
Today, the Center for Effective Government released a scorecard for access to information from the 15 United States federal government agencies that received the most Freedom of Information Act (FOIA) requests, focusing upon an analysis of their performance in 2013.
The results of the report (PDF) for the agencies weren’t pretty: if you computed a grade point average from this open government report card (and I did) the federal government would receive a D for its performance. 7 agencies outright failed, with the State Department receiving the worst grade (37%).
The grades were based upon:
- How well agencies processed FOIA requests, including the rate of disclosure, fullness of information provided, and timeliness of the response
- How well the agencies established rules of information access, including the effectiveness of agency polices on withholding information and communications with requestors
- Creating user-friendly websites, including features that facilitate the flow of information to citizens, associated online services, and up-to-date reading rooms
The report is released at an interesting historic moment for the United States, with Sunshine Week just around the corner. The United States House of Representatives just unanimously passed a FOIA Reform Act that is substantially modeled upon the Obama administration’s proposals for FOIA reforms, advanced as part of the second National Open Government Action Plan. If the Senate takes up that bill and passes it, it would be one of the most important, substantive achievements in institutionalizing open government beyond this administration.
The Citizens for Responsibility and Ethics in Washington have disputed the accuracy of this scorecard, based upon the high rating for the Department of Justice. CREW counsel Anne Weismann:
It is appropriate and fair to recognize agencies that are fulfilling their obligations under the FOIA. But CEG’s latest report does a huge disservice to all requesters by falsely inflating DOJ’s performance, and ignoring the myriad ways in which that agency — a supposed leader on the FOIA front — ignores, if not flouts, its obligations under the statute.
Last Friday, I spoke with Sean Moulton, the director of open government policy at the Center for Effective Government, about the contents of the report and the state of FOIA in the federal government, from the status quo to what needs to be done. Our interview, lightly edited for content and clarity, follows.
What was the methodology behind the report?
Moulton: Our goal was to keep this very quantifiable, very exact, and to try and lay out some specifics. We thought about what the components were necessary for a successful FOIA program. The processing numbers that come out each year are a very rich area for data. They’re extremely important: if you’re not processing quickly and releasing information, you can’t be successful, regardless of other components.
We did think that there are two other areas that are important. First, online services. Let’s face it, the majority of us live online in a big way. It’s a requirement now for agencies to be living there as well. Then, the rules. They’re explained to the agencies and the public, in how they’re going to do things when they get a request. A lot of the agencies have outdated rules. Their current practices may be different, and they may be doing things that the rules don’t say they have to, but without them, they may stop. Consistent rules are essential for consistent long term performance.
A few months back, we released a report that laid out what we felt were best practices for FOIA regulations. We went through a review of dozens of agencies, in terms of their FOIA regulations, and identified key issues, such as communicating with the requester, how you manage confidential business information, how you handle appeals, and how you handle timelines. Then we found inside existing regulations the best ways this was being handled. It really helped us here, when we got to the rules. We used that as our roadmap. We knew agencies were already doing these things, and making that commitment. The main thing we measured under the rules were the items from that best practices report that were common already. If things were universal, we didn’t want to call a best practice, but a normal practice.
Is FOIA compliance better under the Obama administration, more than 4 years after the Open Government Directive?
Moulton: In general, I think FOIA is improving in this administration. Certainly, the administration itself is investing a great deal of energy and resources in trying to make greater improvements in FOIA, but it’s challenging. None of this has penetrated into national security issues.
I think it’s more of a challenge than the administration thought it would be. It’s different from other things, like open data or better websites. The FOIA process has become entrenched. The biggest open government wins were in areas where they were breaking new ground. There wasn’t a culture or way of doing this or problems that were inherited. They were building from the beginning. With FOIA, there was a long history. Some agencies may see FOIA as some sort of burden, and not part of their mission. They may think of it as a distraction from their mission, in fact. When the Department of Transportation puts out information, it usually gets used in the service of their mission. Many agencies haven’t internalized that.
There’s also the issue of backlogs, bureaucracy, lack of technology or technology that doesn’t work that well — but they’re locked into it.
What about redaction issues? Can you be FOIA compliant without actually honoring the intent of the request?
Moulton: We’re very aware of this as well. The data is just not there to evaluate that. We wish it was. The most you get right now is “fully granted” or “partly granted.” That’s incredibly vague. You can redact 99% or 1% and claim it’s partially redacted, either way. We have no indicator and no data on how much is being released. It’s frustrating, because something like that would help us get a better sense on whether agencies would benefit would new policies
We do know that the percentage of full grants has dropped every year, for 12 years, from the Clinton administration all the way through the Bush administration to today. It’s such a gray area. It’s hard to say whether it’s a terrible thing or a modest change.
Has the Obama administration’s focus on open government made any difference?
Moulton: I think it has. There were a couple of agencies that got together on FOIA reform. The EPA led the team, with the U.S. National Archives and the Commerce Department, to build a new FOIA tool. The outward-facing part of the tool enables a user to go to a single spot, request and track it. Other people could come and search FOIA’ed documents. Behind the scenes, federal workers could use the tool to forward requests back and forth. This fits into what the administration has been trying to do, using technology better in government
Another example, again at the EPA, is where they’ve put together a proactive disclosure website. They got a lot of requests, like if there are inquiries about properties, environmental history, like leaks and spills, and set up a site where you could look up real estate. They did this because they went to FOIA requests and see what people wanted. That has cut down their requests to a certain percentage.
Has there been increasing FOIA demand in recent years, affecting compliance?
Moulton: I do think FOIA requests have been increasing. We’ll see what this next year of data shows. We have seen a pretty significant increase, after a significant decrease in the Bush administration. That may be because this administration keeps speaking about open government, which leads to more hopeful requestors. We fully expect that in 2013, there will be more requests than the prior year.
DHS gets the biggest number of all, but that’s not surprising when we look at the size of it. It’s second biggest agency, after Defense, and the biggest domestic facing agency. when you start talking about things like immigration and FEMA, which go deep into communities and people’s lives, in ways that have a lot impact, that makes sense.
What about the Department of Justice’s record?
Moulton: Well, DoJ got the second highest rating, but we know they have a mixed record. There are things you can’t measure and quantify, in terms of culture and attitude. I do know there were concerns about the online portal, in terms of the turf war between agencies. There were concerns about whether the tech was flexible, in terms of meeting all agency needs. If you want to build a government-wide tool, it needs to have real flexibility. The portal changed the dialogue entirely
Is FOIA performance a sufficient metric to analyze any administration’s performance on open government?
Moulton: We should step back further and look at the broader picture, if we’re going to talk about open government. This administration has done things, outside of FOIA, to try to open up records and data. They’ve built better online tools for people to get information. You have to consider all of those things.
Does that include efforts like the Intelligence Community Tumblr?
Moulton: That’s a good example. One thing this administration did early on is to identify social media outlets. We should be going there. We can’t make citizens come to us. We should go to where people are. The administration pushed early on that agencies should be able to use Tumblr and Twitter and Facebook and Flickr and so on.
Is this social media use “propaganda,” as some members of the media have suggested?
Moulton: That’s really hard to decide. I think it can result in that. It has the potential to be misused to sidestep the media, and not have good interaction with the media, which is another important outlet. People get a lot of their information from the media. Government needs to have good relationship.
I don’t think that’s the intention, though, just as under Clinton, when they started setting up websites for the first time. That’s what the Internet is for: sharing information. That’s what social media can be used for, so let’s use what’s there.
— For Effective Gov (@ForEffectiveGov) March 10, 2014
One of the more interesting aspects of Dave Eggers’ dystopic new novel, “The Circle,” is the introduction of the “SeaChange,” a small, powerful camera that can transmit wireless images to a networked global audience. The SeaChange is adopted by politicians who “go transparent,” broadcasting all of their interactions to the public all day long.
Regardless of whether that degree of radical transparency in beneficial for elected representatives or not, in early 2014, we’ve now seen many early glimpses of what a more networked world full of inexpensive cameras looks like when United States politicians are online and on camera more often, from scandals to threats to slurs to charged comments that may have changed a presidential election. Most of that video has been captured by small video cameras or, increasing, powerful smartphones. Over the next year, more people will be wearing Google Glass, Google’s powerful facial computing device. Even if Google Glass has led to a backlash, the next wave of mobile devices will be wearable, integrated into clothing, wristbands, shoes and other gear. This vision of the future is fast approaching, which means that looking for early signals of various aspects of it is crucial.
One such signal came across my desktop earlier this week, in the form of a new app for Google Glass from RedEdge, a digital advocacy consultancy based in Arlington, Virginia. Their new “augmented advocacy” application for Google Glass is a proof of concept that demonstrates how government data can be served to someone wearing glass as she moves around the world. It’s not in the GDJ Store but people interested in testing it can request the Glass application file (android 1) from RedEdge, its maker.
“While we don’t expect widespread deployment of this app, though that would be cool, this is a window into what’s possible with wearable computing just using federal department data,” said Ian Spencer, chief technology officer of RedEdge, in an interview. “The data we used to launch this app and populate the database was all sourced from publicly available information. We primarily used publications from the Office of Management and Budget for budget figures, as well as the president’s own budget, for monetary data. Location data on federal buildings was sourced from Google Maps.”
The app leverages Google Glass’s ability to detect the wearer’s location, feeding a government data through RedEdge’s API to populate a relevant card. It pulls in from open data, formatted as JSON, and provides a list of all locations.
“You can just walk around with the app running in background,” said Spencer. “It doesn’t take up a ton of battery life. With geofencing, Glass knows when you’re near a building and triggers the app, which pops in a card that shows you a phone number and budget information. You can then tap to get more information and it loads up public contact information. Eventually the GDK [Glass Developer Kit] will let you make calls and emails.”
The RedEdge app is currently limited by the amount of time and investment RedEdge has put into it, along with the technology of Glass itself. “Once we add more data points, we will need a more complicated API,” said Spencer. “User experience was our focus, not massive complete sets. Even if we were using a government API, which would be ideal at some point, we would need a hashing layer so that we don’t overwhelm their servers.”
The only data the developers are feeding into it is the total federal budget for a given agency, not more granular details concerning how it related to programs, their performance or who is in charge of them. It’s very much a “proof of concept.”
“We’re looking at it as a trial balloon,” said Spencer. “It started with our tech team. We haven’t had researchers go over tons of entries. If there is interest in it, we then may do more, like adding more federal data and state-level data.”
One potentially interesting application of augmented advocacy might seem to be Congress, where data from the Sunlight Foundation’s Influence Explorer or Open Congress could be integrated as the Glass wearer walked around. The technical limitations of Glass, however, mean that citizens will need to keep downloading Sunlight’s popular Congress app for smartphones.
“The problem is the precision of the GPS,” said Spencer. “If you’re wearing Glass in the Hart building, you don’t have enough accuracy. You can get building-to-building precision, but not more. There are technical problems with trying to use satellites for this, whether it’s GPS or GLONASS, the Russian version.”
That doesn’t mean such precision might not be possible in the future. As Spencer highlighted, app developers can determine “micropositioning” through wifi or Bluetooth, enabling triangulation within a room. “A classic example comes from marketing in a store –” I see you’re looking at X,” he said.
That technology is already live, as Brian Fung reported in the Washington Post: stores are using cellphones to track shopping habits. In Washington, a more palatable example might be around the Mall, where geofences and tracking trigger information about Smithsonian paintings, trees, statuary, or monuments.
The limitation on facial recognition capabilities in Glass also means that the most interesting and disturbing potential application of its gaze is still far away: looking at someone in a lobby, bar, hearing or conference and learning not only who the person is but what role he or she may play in DC’s complicated ecosystem of lobbyists, journalists, Congressional staffers, politicians, media, officials, public advocates and campaign operatives. (For now, the role of the trusted aide, whispering brief identifiers into the ears of the powerful is safe.)
When more apps like this go live in more devices, expect some fireworks to ensure around the United States and the world, as more private and semi-public spaces become recorded. Glass and its descendents will provide evidence of misbehavior by law enforcement, just as cellphones have in recent years. The cameras will be on the faces of officers, as well. While some studies suggest that police wearing cameras may improve the quality of their policing — and civil liberties advocates support their introduction — such devices aren’t popular with the New York City Police Department.
As with the dashboard cameras that supply much of the footage for “Cops” in the United States and offer some protection against corrupt police and fraud in Russia, wearable cameras look likely to end up on the helmets, glasses, lapels or shoulders of many officers in the future, from Los Angeles to London.
The aspirational view of this demo is that it will show how it’s possible to integrate more public data into the life of a citizen without requiring her to pull out a phone.
“There’s a lot of potential for this app to get people to care about an issue and take action,” said Spencer. “It’s about getting people aware. The cool thing about this is its passive nature. You start it once and it tells you when you’re near something.”
A more dystopian view is that people will see a huge budget number and call the switchboard of a given agency to angrily complain, as opposed to the constituent relations staff of their representatives in Congress.
Given the challenges that Congress already faces with the tidal wave of social media and email that has swelled up over the last decade, that would be unhelpful at best. If future digital advocates want to make the most of such tools, they’ll need to provide users with context for the data they’re being fed, from sources to more information about the issues themselves the progress of existing campaigns.
This initial foray is, after all, just a demo. More integration may be coming in the next generation of wearables.
— Mark Silverberg (@skram) January 30, 2014
— Mark Silverberg (@skram) January 30, 2014
One of the most important open government data efforts in United States history came into being in 1993, when citizen archivist Carl Malamud used a small planning grant from the National Science Foundation to license data from the Securities and Exchange Commission, published the SEC data on the Internet and then operated it for two years. At the end of the grant, the SEC decided to make the EDGAR data available itself — albeit not without some significant prodding — and has continued to do so ever since. You can read the history behind putting periodic reports of public corporations online at Malamud’s website, public.resource.org.
Two decades later, Malamud is working to make the law public, reform copyright, and free up government data again, buying, processing and publishing millions of public tax filings from nonprofits to the Internal Revenue Service. He has made the bulk data from these efforts available to the public and anyone else who wants to use it.
“This is exactly analogous to the SEC and the EDGAR database,” Malamud told me, in an phone interview last year. The trouble is that data has been deliberately dumbed down, he said. “If you make the data available, you will get innovation.”
November Form 990s now ready. http://t.co/HDoMzPjpY0 We have 7,335,804 Form 990s available. *STILL* no word from the IRS.
— Carl Malamud (@carlmalamud) January 4, 2014
Making millions of Form 990 returns free online is not a minor public service. Despite many nonprofits file their Form 990s electronically, the IRS does not publish the data. Rather, the government agency releases images of millions of returns formatted as .TIFF files onto multiple DVDs to people and companies willing and able to pay thousands of dollars for them. Services like Guidestar, for instance, acquire the data, convert it to PDFs and use it to provide information about nonprofits. (Registered users view the returns on their website.)
As Sam Roudman reported at TechPresident, Luke Rosiak, a senior watchdog reporter for the Washington Examiner, took the files Malamud published and made them more useful. Specifically, he used credits for processing that Amazon donated to participants in the 2013 National Day of Civic Hacking to make the .TIFF files text-searchable. Rosiak then set up CItizenAudit.org a new website that makes nonprofit transparency easy.
“This is useful information to track lobbying,” Malamud told me. “A state attorney general could just search for all nonprofits that received funds from a donor.”
Malamud estimates nearly 9% of jobs in the U.S. are in this sector. “This is an issue of capital allocation and market efficiency,” he said. “Who are the most efficient players? This is more than a CEO making too much money — it’s about ensuring that investments in nonprofits get a return.
Malamud’s open data is acting as a platform for innovation, much as legislation.gov.uk is the United Kingdom. The difference is that it’s the effort of a citizen that’s providing the open data, not the agency: Form 990 data is not on Data.gov.
Opening Form 990 data should be a no-brainer for an Obama administration that has taken historic steps to open government data. Liberating nonprofit sector data would provide useful transparency into a $1.6 trillion dollar sector for the U.S. economy.
After many letters to the White House and discussions with the IRS, however, Malamud filed suit against the IRS to release Form 990 data online this summer.
“I think inertia is behind the delay,” he told me, in our interview. “These are not the expense accounts of government employees. This is something much more fundamental about a $1.6 trillion dollar marketplace. It’s not about who gave money to a politician.”
If I order these IRS DVDs, my cost is $2910. Media and gov get them free, but none of them lifting a finger to help. http://t.co/B6m5VECV1O
— Carl Malamud (@carlmalamud) January 22, 2014
When asked for comment, a spokesperson for the White House Office of Management and Budget said that the IRS “has been engaging on this topic with interested stakeholders” and that “the Administration’s Fiscal Year 2014 revenue proposals would let the IRS receive all Form 990 information electronically, allowing us to make all such data available in machine readable format.”
Today, Malamud sent a letter of complaint to Howard Shelanski, administrator of the Office of Information and Regulatory Affairs in the White House Office of Management and Budget, asking for a review of the pricing policies of the IRS after a significant increase year-over-year. Specifically, Malamud wrote that the IRS is violating the requirements of President Obama’s executive order on open data:
The current method of distribution is a clear violation of the President’s instructions to
move towards more open data formats, including the requirements of the May 9, 2013
Executive Order making “open and machine readable the new default for government
I believe the current pricing policies do not make any sense for a government
information dissemination service in this century, hence my request for your review.
There are also signiﬁcant additional issues that the IRS refuses to address, including
substantial privacy problems with their database and a ﬂat-our refusal to even
consider release of the Form 990 E-File data, a format that would greatly increase the
transparency and effectiveness of our non-proﬁt marketplace and is required by law.
It’s not clear at all whether the continued pressure from Malamud, the obvious utility of CitizenAudit.org or the bipartisan budget deal that President Obama signed in December will push the IRS to freely release open government data about the nonprofit sector,
The furor last summer over the IRS investigating the status of conservative groups claimed tax-exempt status, however, could carry over into political pressure to reform. If political groups were tax-exempt and nonprofit e-file data were published about them, it would be possible for auditors, journalists and Congressional investigators to detect patterns. The IRS would need to be careful about scrubbing the data of personal information: last year, the IRS mistakenly exposed thousands of Social Security numbers when it posted 527 forms online — an issue that Malamud, as it turns out, discovered in an audit.
“This data is up there with EDGAR, in terms of its potential,” said Malamud. “There are lots of databases. Few are as vital to government at large. This is not just about jobs. It’s like not releasing patent data.”
If the IRS were to modernize its audit system, inspector generals could use automated predictive data analysis to find aberrations to flag for a human to examine, enabling government watchdogs and investigative journalists to potentially detect similar issues much earlier.
That level of data-driven transparency remains in the future. In the meantime, CitizenAudit.org is currently running on a server in Rosiak’s apartment.
Whether the IRS adopts it as the SEC did EDGAR remains to be seen.
[Image Credit: Meals on Wheels]