Tag Archives: transparency

USASpending.gov addresses some data issues, adds Github issues tracker for feedback

Posted on by

2

usaspending

On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.

A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.

Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.

In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:

The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.

This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.

Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.

Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.

Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.

If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.

[Image Credit: Govfresh, via USASpending.gov]

In a step towards sunlight, United States begins to publish a national data inventory

Posted on by

1

20130929-142228.jpg
Last year, a successful Freedom of Information request for the United States enterprise data inventory by the Sunlight Foundation was a big win for open government, nudging Uncle Sam towards a better information policy through some creative legal arguments. Today, the federal government started releasing its enterprise indices at data.gov. You can browse the data for individual agencies, like the feed for the Office for Personnel Management, using a JSON viewer like this one.

“Access to this data will empower journalists, government officials, civic technologists, innovators and the public to better hold government accountable,” said Sunlight Foundation president Chris Gates, in a statement. “Previously, it was next to impossible to know what and how much data the government has, and this is an unprecedented window into its internal workings. Transparency is a bedrock principle for democracy, and the federal government’s response to Sunlight’s Freedom of Information request shows a strong commitment to open data. We expect to see each of these agencies continue to proactively release their data inventories.”

Understanding what data an organization holds is a critical first step in deciding how it should be stored, analyzed or published, shifting towards thinking about data as an asset. That’s why President Barack Obama’s executive order requiring federal agencies to catalog the data they have was a big deal. When that organization is a democratic government and the data in question was created using taxpayer funds, releasing the inventory of the data sets that it holds is a basic expression of open and accountable government.

In the wake of scandal, the State of Oregon seeks to restore trust through publishing public records

Posted on by

3

or-state-sealIn a fascinating turn of events, rainy Oregon is embracing sunlight online after a scandal that led to the resignation of its governor. After governor Kate Brown was sworn in as the 38th governor of the state of Oregon, replacing fellow Democrat John Kitzhaber, her administration chose to try to restore public trust by not only posting public records requests online but including the authors, status and a downloadable link to the records themselves, once fulfilled. The records only go back to January 15th, 2015, with a note that requests made prior to that date are “still being processed.”

The City of Oakland’s public records system, built by Code for America, does the same thing but this appears to set a new bar for state government that’s unmatched in the United States of America. As has been reported elsewhere, exemptions to Oregon’s public records laws mean that this website will be no panacea, but it looks like progress from 3000 miles away in snowy Cambridge.

As Kirk Johnson reported for the New York Times, Brown’s record includes open government work while she was a state legislator, where, as Senate Majority Leader, she worked to reform Oregon’s ethics law and helped to enact legislation that created an online campaign finance database.

“…throughout my 24 years in public service, I have also sought to promote transparency and trust in government, working to build confidence that our public dollars are spent wisely,” she said, in her inaugural speech.

Later in her remarks, Governor Brown said that “we must seize this moment to work across party lines to restore the public’s trust. That means passing meaningful legislation that strengthens the capacity and independence of the Government Ethics Commission. We also must strengthen laws to ensure timely release of public documents.”

On that count, it’s notable that two of the records requests that have been posted for download involve Cynthia Hayes, the fiancee of former Governor Kitzhaber who was at the center of the scandals that led to his resignation. One comes from Margaret Olney, who is quite likely the same Margaret Olney who served in Oregon’s Department of Justice. The other requester was Alejandra Lazo, who co-authored a Wall Street Journal article on former Governor Kitzhaber’s resignation. In an interesting sidenote, the records for both responses were uploaded to Dropbox.

If you know of another state that meet or exceeds this standard for digital transparency, or have experience or feedback regarding the quality or importance of the public records posted by Oregon, please let us know in a comment.

Emily Shaw, the national policy manager at the Sunlight Foundation, said on Twitter that she has not seen any other state’s public records system exceed this standard of transparency.

FCC Commissioners take agency to task for opacity around rulemaking

Posted on by

0

Today, Federal Communications Commission​ Commissioner Ajit Pai​ released a statement & fact sheet on the non-public FCC’s proposal for Open Internet Rules, which would be applied to regulating broadband Internet service providers. He tweeted out a picture of himself holding the document (above) several days ago. In his statement today, Pai said that the American people are being misled about what’s in the rules and broke out an outline of the 300-page document. Reporting and analysis of Pai’s objections to the proposal may be found at the New York Times and National Journal. Subsequent to Pai’s statement and press conference, FCC Special Counsel for External Affairs Gigi Sohn tweeted out a series of rebuttals, including a clarification regarding the size of the document.

Here, I’ll focus on something else: the commissioner took FCC Chairman Tom Wheeler​ to task for not releasing them ahead of the scheduled vote on February 26th, in DC, although Pai did note that doing so would be “unprecedented.” In fact, FCC commissioner Mike O’Rielly tweeted much the same thing, stating that “all FCC meeting items should be made public when circulated to Commissioners. ”

O’Rielly published a blog post on advanced posting of Commission items back in January, following a previous post in August 2014.

In August, I wrote a blog post urging the Commission to post on its website the actual text of the items to be considered at our Open Meetings at the same time they are provided to Commissioners. I made the suggestion because the inability of the public to obtain a complete picture of what is in a pending notice of proposed rulemaking or order routinely leads to confusion over what exactly is at stake. Making matters worse, Commissioners are not allowed to reveal the substantive details to outside parties. We can’t even correct inaccurate impressions that stakeholders may have received, and we are barred from discussing what changes we are seeking. This barrier to a fulsome exchange can be extremely frustrating for all involved. Despite positive feedback from people at the FCC, outside parties, Members of Congress, [1] and the general public, four months later, we have yet to post a single meeting item in advance. Moreover, the lack of full disclosure and transparency has continued to be a problem as some parties have not been fully briefed on recent items, such as the recently adopted 911 Reliability NPRM, while others are not briefed at all. The reason that nothing has happened, I am told, is that there are two basic concerns with the proposal: 1) that it could be harder to comply with the Administrative Procedure Act (APA); and 2) that it could be more difficult to withhold documents under the Freedom of Information Act (FOIA). I do not find either argument persuasive or insurmountable

Wheeler took a question on this issue in a press conference last month: “The precedent here, through Democratic chairmen and Republican chairmen, has always been that just like, you know, in the court system, and elsewhere that when you’re handling cases like this, you have an internal discussion and then you release what the result of that vote is. And you don’t change that decades of precedent overnight without following the procedures to review questions like that.”

Asked for comment, FCC press secretary Kim Hart Jonson​ said that “Chairman Wheeler circulated his proposal to his fellow Commissioners three weeks before the planned vote in accordance with long-standing FCC process. We are confident the other Commissioners will give the proposal an exhaustive review. The Chairman looks forward to receiving their input and releasing it to the public after the February 26 vote.”

What this boils down to is that publishing proposed rules before a vote is technically legal but has never been done before. As Pai himself noted, “FCC rules prohibit disclosure of nonpublic information except as authorized by the Chairman.” Then-FCC chairman Julius Genachowski didn’t pull the full version of the 2010 Open Internet rules up until right before Christmas, well after the vote. The standard FCC procedure for rulemaking is to circulate the draft rules at least 3 weeks prior to a vote, incorporate edits received from FCC commissioners, then finalize everything. In practice, however, some commissioners have complained that edits may be made right up until a vote. Once rules have been voted upon, they’re published in the federal register.

As long-time readers know, I was critical of that decision at the time, and remain so. To reiterate what I said them, Genachowski made a commitment to a more open, transparent and data-driven F.C.C. under President Obama’s Open Government Directive. In many respects, in its first year of open government, the agency made commendable progress, with strides towards taking public comment through e-rulemaking at OpenInternet.gov, Broadband.gov and Reboot.FCC.gov. The sites were deployed by an able new media team that has used online communications in unprecedented ways. The chairman and his managing director, Steven Van Roeckel, both deserved credit for their plans to reboot FCC.gov as a platform for government including the use of APIs and open source technologies like Drupal.

When it comes to the question of whether the public could see the proposed Open Internet rules before the commissioners vote upon it, the agency fell short of its transparency pledge. I have not found a legal precedent that explicitly gives the agency authority to keep the text of a proposed rule secret until it is voted upon by the Commission. While it is true that the FCC does not appear to be under no legal obligation to do so, given that the members of the commission presumably had to negotiate on the details of the final rules for vote, the decision not to share a version publicly may have made such discussion more flexible.

The choice not to post the proposed rules online before the most reced vote is an example of the same level of government transparency in the creation of important regulation as before, not more. It’s possible that the legal context for how the FCC operates might change in the future, if a bill by Senator Dean Heller (R-NV) were to be passed. Per his office, the FCC Reform Act would:

“provide more opportunities for the public to see pending FCC action by publishing the exact rule or amendment to a rule for at least 21 days and by allowing any Commissioner to ask for a vote on any order issued by a bureau. *Enhance consistency and transparency in the Commission’s operations by requiring the FCC to establish procedures for handling extensive new data submitted towards the end of a comment period, adequate review and deliberation regarding pending orders, providing the status of open rulemaking proceedings, minimum public review periods for statistical reports and ex parte communications. *Empower the Commission to operate more efficiently through reform of the “sunshine” rules, allowing a bipartisan majority of Commissioners to meet for collaborative discussions subject to transparency safeguards.

A provision that the FCC must “identify a market failure, consumer harm, or regulatory barrier to investment before adopting economically significant rules,” however, may well mean that Democrats in the Senate won’t support its advance. As always with these net neutrality stories, to be continued.

UPDATE: Commissioner Pai tweeted another picture of a revised plan February 21, including his intention to vote against it and again noting that the public cannot see it before the vote. In the tweet, he says the plan is now 317 pages, which means it has been trimmed by 15 pages.

On data journalism, accountability and society in the Second Machine Age

Posted on by

1

On Monday, I delivered a short talk on data journalism, networked transparency, algorithmic transparency and the public interest at the Data & Society Research Institute’s workshop on the social, cultural & ethical dimensions of “big data”. The forum was convened by the Data & Society Research Institute and hosted at New York University’s Information Law Institute at the White House Office of Science and Technology Policy, as part of an ongoing review on big data and privacy ordered by President Barack Obama.

Video of the talk is below, along with the slides I used. You can view all of the videos from the workshop, along with the public plenary on Monday evening, on YouTube or at the workshop page.

Here’s the presentation, with embedded hyperlinks to the organizations, projects and examples discussed:

For more on the “Second Machine Age” referenced in the title, read the new book by Erik Brynjolfsson and Andrew McAfee.

Federal government agencies receive .91 GPA in FOIA compliance from Center for Effective Government

Posted on by

3

Today, the Center for Effective Government released a scorecard for access to information from the 15 United States federal government agencies that received the most Freedom of Information Act (FOIA) requests, focusing upon an analysis of their performance in 2013.

The results of the report (PDF) for the agencies weren’t pretty: if you computed a grade point average from this open government report card (and I did) the federal government would receive a D for its performance. 7 agencies outright failed, with the State Department receiving the worst grade (37%).

The grades were based upon:

  1. How well agencies processed FOIA requests, including the rate of disclosure, fullness of information provided, and timeliness of the response
  2. How well the agencies established rules of information access, including the effectiveness of agency polices on withholding information and communications with requestors
  3. Creating user-friendly websites, including features that facilitate the flow of information to citizens, associated online services, and up-to-date reading rooms

The report is released at an interesting historic moment for the United States, with Sunshine Week just around the corner. The United States House of Representatives just unanimously passed a FOIA Reform Act that is substantially modeled upon the Obama administration’s proposals for FOIA reforms, advanced as part of the second National Open Government Action Plan. If the Senate takes up that bill and passes it, it would be one of the most important, substantive achievements in institutionalizing open government beyond this administration.

The Citizens for Responsibility and Ethics in Washington have disputed the accuracy of this scorecard, based upon the high rating for the Department of Justice. CREW counsel Anne Weismann:

It is appropriate and fair to recognize agencies that are fulfilling their obligations under the FOIA. But CEG’s latest report does a huge disservice to all requesters by falsely inflating DOJ’s performance, and ignoring the myriad ways in which that agency — a supposed leader on the FOIA front — ignores, if not flouts, its obligations under the statute.

Last Friday, I spoke with Sean Moulton, the director of open government policy at the Center for Effective Government, about the contents of the report and the state of FOIA in the federal government, from the status quo to what needs to be done. Our interview, lightly edited for content and clarity, follows.

What was the methodology behind the report?

Moulton: Our goal was to keep this very quantifiable, very exact, and to try and lay out some specifics. We thought about what the components were necessary for a successful FOIA program. The processing numbers that come out each year are a very rich area for data. They’re extremely important: if you’re not processing quickly and releasing information, you can’t be successful, regardless of other components.

We did think that there are two other areas that are important. First, online services. Let’s face it, the majority of us live online in a big way. It’s a requirement now for agencies to be living there as well. Then, the rules. They’re explained to the agencies and the public, in how they’re going to do things when they get a request. A lot of the agencies have outdated rules. Their current practices may be different, and they may be doing things that the rules don’t say they have to, but without them, they may stop. Consistent rules are essential for consistent long term performance.

A few months back, we released a report that laid out what we felt were best practices for FOIA regulations. We went through a review of dozens of agencies, in terms of their FOIA regulations, and identified key issues, such as communicating with the requester, how you manage confidential business information, how you handle appeals, and how you handle timelines. Then we found inside existing regulations the best ways this was being handled. It really helped us here, when we got to the rules. We used that as our roadmap. We knew agencies were already doing these things, and making that commitment. The main thing we measured under the rules were the items from that best practices report that were common already. If things were universal, we didn’t want to call a best practice, but a normal practice.

Is FOIA compliance better under the Obama administration, more than 4 years after the Open Government Directive?

Moulton: In general, I think FOIA is improving in this administration. Certainly, the administration itself is investing a great deal of energy and resources in trying to make greater improvements in FOIA, but it’s challenging. None of this has penetrated into national security issues.

I think it’s more of a challenge than the administration thought it would be. It’s different from other things, like open data or better websites. The FOIA process has become entrenched. The biggest open government wins were in areas where they were breaking new ground. There wasn’t a culture or way of doing this or problems that were inherited. They were building from the beginning. With FOIA, there was a long history. Some agencies may see FOIA as some sort of burden, and not part of their mission. They may think of it as a distraction from their mission, in fact. When the Department of Transportation puts out information, it usually gets used in the service of their mission. Many agencies haven’t internalized that.

There’s also the issue of backlogs, bureaucracy, lack of technology or technology that doesn’t work that well — but they’re locked into it.

What about redaction issues? Can you be FOIA compliant without actually honoring the intent of the request?

Moulton: We’re very aware of this as well. The data is just not there to evaluate that. We wish it was. The most you get right now is “fully granted” or “partly granted.” That’s incredibly vague. You can redact 99% or 1% and claim it’s partially redacted, either way. We have no indicator and no data on how much is being released. It’s frustrating, because something like that would help us get a better sense on whether agencies would benefit would new policies

We do know that the percentage of full grants has dropped every year, for 12 years, from the Clinton administration all the way through the Bush administration to today. It’s such a gray area. It’s hard to say whether it’s a terrible thing or a modest change.

Has the Obama administration’s focus on open government made any difference?

Moulton: I think it has. There were a couple of agencies that got together on FOIA reform. The EPA led the team, with the U.S. National Archives and the Commerce Department, to build a new FOIA tool. The outward-facing part of the tool enables a user to go to a single spot, request and track it. Other people could come and search FOIA’ed documents. Behind the scenes, federal workers could use the tool to forward requests back and forth. This fits into what the administration has been trying to do, using technology better in government

Another example, again at the EPA, is where they’ve put together a proactive disclosure website. They got a lot of requests, like if there are inquiries about properties, environmental history, like leaks and spills, and set up a site where you could look up real estate. They did this because they went to FOIA requests and see what people wanted. That has cut down their requests to a certain percentage.

Has there been increasing FOIA demand in recent years, affecting compliance?

Moulton: I do think FOIA requests have been increasing. We’ll see what this next year of data shows. We have seen a pretty significant increase, after a significant decrease in the Bush administration. That may be because this administration keeps speaking about open government, which leads to more hopeful requestors. We fully expect that in 2013, there will be more requests than the prior year.

DHS gets the biggest number of all, but that’s not surprising when we look at the size of it. It’s second biggest agency, after Defense, and the biggest domestic facing agency. when you start talking about things like immigration and FEMA, which go deep into communities and people’s lives, in ways that have a lot impact, that makes sense.

What about the Department of Justice’s record?

Moulton: Well, DoJ got the second highest rating, but we know they have a mixed record. There are things you can’t measure and quantify, in terms of culture and attitude. I do know there were concerns about the online portal, in terms of the turf war between agencies. There were concerns about whether the tech was flexible, in terms of meeting all agency needs. If you want to build a government-wide tool, it needs to have real flexibility. The portal changed the dialogue entirely

Is FOIA performance a sufficient metric to analyze any administration’s performance on open government?

Moulton: We should step back further and look at the broader picture, if we’re going to talk about open government. This administration has done things, outside of FOIA, to try to open up records and data. They’ve built better online tools for people to get information. You have to consider all of those things.

Does that include efforts like the Intelligence Community Tumblr?

Moulton: That’s a good example. One thing this administration did early on is to identify social media outlets. We should be going there. We can’t make citizens come to us. We should go to where people are. The administration pushed early on that agencies should be able to use Tumblr and Twitter and Facebook and Flickr and so on.

Is this social media use “propaganda,” as some members of the media have suggested?

Moulton: That’s really hard to decide. I think it can result in that. It has the potential to be misused to sidestep the media, and not have good interaction with the media, which is another important outlet. People get a lot of their information from the media. Government needs to have good relationship.

I don’t think that’s the intention, though, just as under Clinton, when they started setting up websites for the first time. That’s what the Internet is for: sharing information. That’s what social media can be used for, so let’s use what’s there.

Will Google Glass enable “augmented advocacy” in a more transparent society?

Posted on by

4

One of the more interesting aspects of Dave Eggers’ dystopic new novel, “The Circle,” is the introduction of the “SeaChange,” a small, powerful camera that can transmit wireless images to a networked global audience. The SeaChange is adopted by politicians who “go transparent,” broadcasting all of their interactions to the public all day long.

Regardless of whether that degree of radical transparency in beneficial for elected representatives or not, in early 2014, we’ve now seen many early glimpses of what a more networked world full of inexpensive cameras looks like when United States politicians are online and on camera more often, from scandals to threats to slurs to charged comments that may have changed a presidential election. Most of that video has been captured by small video cameras or, increasing, powerful smartphones. Over the next year, more people will be wearing Google Glass, Google’s powerful facial computing device. Even if Google Glass has led to a backlash, the next wave of mobile devices will be wearable, integrated into clothing, wristbands, shoes and other gear. This vision of the future is fast approaching, which means that looking for early signals of various aspects of it is crucial.

glass_promotions-01 (1)

One such signal came across my desktop earlier this week, in the form of a new app for Google Glass from RedEdge, a digital advocacy consultancy based in Arlington, Virginia. Their new “augmented advocacy” application for Google Glass is a proof of concept that demonstrates how government data can be served to someone wearing glass as she moves around the world. It’s not in the GDJ Store but people interested in testing it can request the Glass application file (android 1) from RedEdge, its maker.

“While we don’t expect widespread deployment of this app, though that would be cool, this is a window into what’s possible with wearable computing just using federal department data,” said Ian Spencer, chief technology officer of RedEdge, in an interview. “The data we used to launch this app and populate the database was all sourced from publicly available information. We primarily used publications from the Office of Management and Budget for budget figures, as well as the president’s own budget, for monetary data. Location data on federal buildings was sourced from Google Maps.”

The app leverages Google Glass’s ability to detect the wearer’s location, feeding a government data through RedEdge’s API to populate a relevant card. It pulls in from open data, formatted as JSON, and provides a list of all locations.

“You can just walk around with the app running in background,” said Spencer. “It doesn’t take up a ton of battery life. With geofencing, Glass knows when you’re near a building and triggers the app, which pops in a card that shows you a phone number and budget information. You can then tap to get more information and it loads up public contact information. Eventually the GDK [Glass Developer Kit] will let you make calls and emails.”

Visitors to the White House with this app, for instance, could call the White House switchboard, though they would be unlikely to get President Obama on the phone.
Whitehouse

The RedEdge app is currently limited by the amount of time and investment RedEdge has put into it, along with the technology of Glass itself. “Once we add more data points, we will need a more complicated API,” said Spencer. “User experience was our focus, not massive complete sets. Even if we were using a government API, which would be ideal at some point, we would need a hashing layer so that we don’t overwhelm their servers.”

The only data the developers are feeding into it is the total federal budget for a given agency, not more granular details concerning how it related to programs, their performance or who is in charge of them. It’s very much a “proof of concept.”

“We’re looking at it as a trial balloon,” said Spencer. “It started with our tech team. We haven’t had researchers go over tons of entries. If there is interest in it, we then may do more, like adding more federal data and state-level data.”

One potentially interesting application of augmented advocacy might seem to be Congress, where data from the Sunlight Foundation’s Influence Explorer or Open Congress could be integrated as the Glass wearer walked around. The technical limitations of Glass, however, mean that citizens will need to keep downloading Sunlight’s popular Congress app for smartphones.

“The problem is the precision of the GPS,” said Spencer. “If you’re wearing Glass in the Hart building, you don’t have enough accuracy. You can get building-to-building precision, but not more. There are technical problems with trying to use satellites for this, whether it’s GPS or GLONASS, the Russian version.”

That doesn’t mean such precision might not be possible in the future. As Spencer highlighted, app developers can determine “micropositioning” through wifi or Bluetooth, enabling triangulation within a room. “A classic example comes from marketing in a store –” I see you’re looking at X,” he said.

That technology is already live, as Brian Fung reported in the Washington Post: stores are using cellphones to track shopping habits. In Washington, a more palatable  example might be around the Mall, where geofences and tracking trigger information about Smithsonian paintings, trees, statuary, or monuments.
IRS
The limitation on facial recognition capabilities in Glass also means that the most interesting and disturbing potential application of its gaze is still far away: looking at someone in a lobby, bar, hearing or conference and learning not only who the person is but what role he or she may play in DC’s complicated ecosystem of lobbyists, journalists, Congressional staffers, politicians, media, officials, public advocates and campaign operatives. (For now, the role of the trusted aide, whispering brief identifiers into the ears of the powerful is safe.)

When more apps like this go live in more devices, expect some fireworks to ensure around the United States and the world, as more private and semi-public spaces become recorded. Glass and its descendents will provide evidence of misbehavior by law enforcement, just as cellphones have in recent years. The cameras will be on the faces of officers, as well. While some studies suggest that police wearing cameras may improve the quality of their policing — and civil liberties advocates support their introduction — such devices aren’t popular with the New York City Police Department.

As with the dashboard cameras that supply much of the footage for “Cops” in the United States and offer some protection against corrupt police and fraud in Russia, wearable cameras look likely to end up on the helmets, glasses, lapels or shoulders of many officers in the future, from Los Angeles to London.

The aspirational view of this demo is that it will show how it’s possible to integrate more public data into the life of a citizen without requiring her to pull out a phone.

“There’s a lot of potential for this app to get people to care about an issue and take action,” said Spencer. “It’s about getting people aware. The cool thing about this is its passive nature. You start it once and it tells you when you’re near something.”
Treasury
A more dystopian view is that people will see a huge budget number and call the switchboard of a given agency to angrily complain, as opposed to the constituent relations staff of their representatives in Congress.

Given the challenges that Congress already faces with the tidal wave of social media and email that has swelled up over the last decade, that would be unhelpful at best. If future digital advocates want to make the most of such tools, they’ll need to provide users with context for the data they’re being fed, from sources to more information about the issues themselves the progress of existing campaigns.

This initial foray is, after all, just a demo. More integration may be coming in the next generation of wearables.

Opening IRS e-file data would add innovation and transparency to $1.6 trillion U.S. nonprofit sector

Posted on by

4

One of the most important open government data efforts in United States history came into being in 1993, when citizen archivist Carl Malamud used a small planning grant from the National Science Foundation to license data from the Securities and Exchange Commission, published the SEC data on the Internet and then operated it for two years. At the end of the grant, the SEC decided to make the EDGAR data available itself — albeit not without some significant prodding — and has continued to do so ever since. You can read the history behind putting periodic reports of public corporations online at Malamud’s website, public.resource.org.

Meals-on-Wheels-Reports

Two decades later, Malamud is working to make the law public, reform copyright, and free up government data again, buying, processing and publishing millions of public tax filings from nonprofits to the Internal Revenue Service. He has made the bulk data from these efforts available to the public and anyone else who wants to use it.

“This is exactly analogous to the SEC and the EDGAR database,” Malamud told me, in an phone interview last year. The trouble is that data has been deliberately dumbed down, he said. “If you make the data available, you will get innovation.”

Making millions of Form 990 returns free online is not a minor public service. Despite many nonprofits file their Form 990s electronically, the IRS does not publish the data. Rather, the government agency releases images of millions of returns formatted as .TIFF files onto multiple DVDs to people and companies willing and able to pay thousands of dollars for them. Services like Guidestar, for instance, acquire the data, convert it to PDFs and use it to provide information about nonprofits. (Registered users view the returns on their website.)

As Sam Roudman reported at TechPresident, Luke Rosiak, a senior watchdog reporter for the Washington Examiner, took the files Malamud published and made them more useful. Specifically, he used credits for processing that Amazon donated to participants in the 2013 National Day of Civic Hacking to make the .TIFF files text-searchable. Rosiak then set up CItizenAudit.org a new website that makes nonprofit transparency easy.

“This is useful information to track lobbying,” Malamud told me. “A state attorney general could just search for all nonprofits that received funds from a donor.”

Malamud estimates nearly 9% of jobs in the U.S. are in this sector. “This is an issue of capital allocation and market efficiency,” he said. “Who are the most efficient players? This is more than a CEO making too much money — it’s about ensuring that investments in nonprofits get a return.

Malamud’s open data is acting as a platform for innovation, much as legislation.gov.uk is the United Kingdom. The difference is that it’s the effort of a citizen that’s providing the open data, not the agency: Form 990 data is not on Data.gov.

Opening Form 990 data should be a no-brainer for an Obama administration that has taken historic steps to open government data. Liberating nonprofit sector data would provide useful transparency into a $1.6 trillion dollar sector for the U.S. economy.

After many letters to the White House and discussions with the IRS, however, Malamud filed suit against the IRS to release Form 990 data online this summer.

“I think inertia is behind the delay,” he told me, in our interview. “These are not the expense accounts of government employees. This is something much more fundamental about a $1.6 trillion dollar marketplace. It’s not about who gave money to a politician.”

When asked for comment, a spokesperson for the White House Office of Management and Budget said that the IRS “has been engaging on this topic with interested stakeholders” and that “the Administration’s Fiscal Year 2014 revenue proposals would let the IRS receive all Form 990 information electronically, allowing us to make all such data available in machine readable format.”

Today, Malamud sent a letter of complaint to Howard Shelanski, administrator of the Office of Information and Regulatory Affairs in the White House Office of Management and Budget, asking for a review of the pricing policies of the IRS after a significant increase year-over-year. Specifically, Malamud wrote that the IRS is violating the requirements of President Obama’s executive order on open data:

The current method of distribution is a clear violation of the President’s instructions to
move towards more open data formats, including the requirements of the May 9, 2013
Executive Order making “open and machine readable the new default for government
information.”

I believe the current pricing policies do not make any sense for a government
information dissemination service in this century, hence my request for your review.
There are also significant additional issues that the IRS refuses to address, including
substantial privacy problems with their database and a flat-our refusal to even
consider release of the Form 990 E-File data, a format that would greatly increase the
transparency and effectiveness of our non-profit marketplace and is required by law.

It’s not clear at all whether the continued pressure from Malamud, the obvious utility of CitizenAudit.org or the bipartisan budget deal that President Obama signed in December will push the IRS to freely release open government data about the nonprofit sector,

The furor last summer over the IRS investigating the status of conservative groups claimed tax-exempt status, however, could carry over into political pressure to reform. If political groups were tax-exempt and nonprofit e-file data were published about them, it would be possible for auditors, journalists and Congressional investigators to detect patterns. The IRS would need to be careful about scrubbing the data of personal information: last year, the IRS mistakenly exposed thousands of Social Security numbers when it posted 527 forms online — an issue that Malamud, as it turns out, discovered in an audit.

“This data is up there with EDGAR, in terms of its potential,” said Malamud. “There are lots of databases. Few are as vital to government at large. This is not just about jobs. It’s like not releasing patent data.”

If the IRS were to modernize its audit system, inspector generals could use automated predictive data analysis to find aberrations to flag for a human to examine, enabling government watchdogs and investigative journalists to potentially detect similar issues much earlier.

That level of data-driven transparency remains in the future. In the meantime, CitizenAudit.org is currently running on a server in Rosiak’s apartment.

Whether the IRS adopts it as the SEC did EDGAR remains to be seen.

[Image Credit: Meals on Wheels]

Intelligence community turns to Tumblr and Twitter to provide more transparency on NSA surveillance programs

Posted on by

3


Yesterday afternoon, the Office of the Director of National Intelligence began tumbling towards something resembling more transparency regarding the National Security Agency’s electronic surveillance programs.

The new tumblog, “Intelligence Community on the Record,” is a collection of  statements, declassified documents, congressional testimony by officials, speeches & media, interviews, fact sheets, details of oversight & legal compliance, and video. It’s a slick, slim new media vehicle, at least as compared to many government websites, although much of the content itself consists of redacted PDFs and images. (More on that later.) It’s unclear why ODNI chose Tumblr as its platform, though the lack of hosting costs, youthful user demographics and easy publishing have to have factored in.

In the context of the global furor over electronic surveillance that began this summer when the Washington Post and Guardian began publishing stories based upon the “NSA Files” leaked by former NSA contractor Edward Snowden, the new tumblr has been met with a rather …skeptical reception online.

Despite its reception, the new site does represent a followthrough on President Obama’s commitment to set up a website to share information with the American people about these programs. While some people in the federal technology sector are hopeful:

…the site won’t be enough, on its own. The considerable challenge that it and the intelligence community faces is the global climate of anger, fear and distrust that have been engendered by a summer of fiery headlines. Despite falling trust in institutions, people still trust the media more than the intelligence community, particularly with respect to its role as a watchdog.

Some three hours after it went online, a series of new documents went online and were tweeted out through the new Twitter account, @IConTheRecord:

The launch of the website came with notable context.

First, as the Associated Press reported, some of the documents released were made public after a lawsuit by the Electronic Frontier Foundation (EFF). In a significant court victory, the EFF succeeded in prompting the release of a 2011 secret court opinion finding NSA surveillance unconstitutional. It’s embedded below, along with a release on DNI.gov linked through the new tumblr.

The opinion showed that the NSA gathered thousands of Americans’ emails before the court struck down the program, causing the agency to recalibrate its practices.

Second, Jennifer Valentino and Siobhan Gorman Carpenter reported at The Wall Street Journal that the National Security Agency can reach 75% of Internet traffic in the United States. Using various programs, the NSA applies algorithms to filter and gather specific information from a dozen locations at major Internet junctions around North America. The NSA defended these programs as both legal and “respectful of Americans’ privacy,” according to Gorman and Valentino: According to NSA spokeswoman Vanee Vines, if American communications are “incidentally collected during NSA’s lawful signals intelligence activities,” the agency follows “minimization procedures that are approved by the U.S. attorney general and designed to protect the privacy of United States persons.”

The story, which added more reporting to confirm what has been published in the Guardian and Washington Post, included a handy FAQ with a welcome section detailed what was “new” in the Journal’s report. The FAQ also has clear, concise summaries of fun questions you might still have about these NSA programs after a summer of headlines, like “What privacy issues does this system raise?” or “Is this legal?”

The NSA subsequently released a statement disputing aspects of the Journal’s reporting, specifically the “the impression” that NSA is sifting through 75% of U.S. Internet communications, which the agency stated is “just not true.” The WSJ has not run a correction, however, standing by its reporting that the NSA possesses the capability to access and filter a majority of communications flowing over the Internet backbone.

Reaction to the disclosures has fallen along pre-existing fault lines: critical lawmakers and privacy groups are rattled, while analysts point to a rate of legal compliance well above 99%, with now-public audits showing most violations of the rules and laws that govern the NSA coming when “roamers” from outside of the U.S.A. traveled to the country.

Thousands of violations a year, however, even if they’re out of more than 240,000,000 made, is still significant, and the extent of surveillance reported and acknowledged clearly has the potential to have a chilling effect on free speech and press freedom, from self-censorship to investigative national security journalism. The debates ahead of the country, now more informed by disclosures, leaks and reporting, will range from increased oversight of programs to legislative proposals to update laws for collection and analysis to calls to significantly curtail or outright dissolve these surveillance programs all together.

Given reports of NSA analysts intentionally abusing their powers, some reforms to the laws that govern surveillance are in order, starting with making relevant jurisprudence public. Secret laws have no place in a democracy.

Setting all of that aside for a moment — it’s fair to say that this debate will continue playing out on national television, the front pages of major newspapers and online outlets and in the halls and boardrooms of power around the country — it’s worth taking a brief look at this new website that President Obama said will deliver more transparency into surveillance programs, along with the NSA’s broader approach to “transparency”. To be blunt, all too often it’s looked like this:

…so heavily redacted that media outlets can create mad libs based upon them.

That’s the sort of thing that leads people to suggest that the NSA has no idea what ‘transparency’ means. Whether that’s a fair criticism or not, the approach taken to disclosing documents as images and PDFs does suggest that the nation’s spy agency has not been following how other federal agencies are approaching releasing government information.

As Matt Stoller highlighted on Twitter, heavily redacted, unsearchable images make it extremely difficult to find or quote information.

Unfortunately, that failing highlights the disconnect between the laudable efforts the Obama administration has made to release open government data from federal agencies and regulators and the sprawling, largely unaccountable national security state aptly described as Top Secret America.”

Along with leak investigations and prosecution of whistleblowers, drones and surveillance programs have been a glaring exception to federal open government efforts, giving ample ammunition to those who criticize or outright mock President Obama’s stated aspiration to be the “most transparent administration in history.” As ProPublica reported this spring, the administration’s open government record has been mixed. Genuine progress on opening up data for services, efforts to leverage new opportunities afforded by technology to enable citizen participation or collaboration, and other goals set out by civil society has been overshadowed with failures on other counts, from the creation of the Affordable Care Act to poor compliance with the Freedom of Information Act and obfuscation of the extend of domestic surveillance.

In that context, here’s some polite suggestions to the folks behind the new ODNI tumblr regarding using the Web to communicate:

  • Post all documents as plaintext, not images and PDFs that defy easy digestion, reporting or replication. While the intelligence budget is classified, surely some of those untold billions could be allotted to persons taking time to release information in both human- and machine-readable formats.
  • Put up a series of Frequently Asked Questions, like the Wall Street Journal’s. Format them in HTML. Specifically address that reporting and provide evidence of what differs. Posting the joint statement on the WSJ stories as text is a start but doesn’t go far enough.
  • Post audio and plaintext transcripts of conference calls and all other press briefings with “senior officials.” Please stop making the latter “on background.” (The transcript of the briefing with NSA director of compliance John DeLong is a promising start, although getting it out of a PDF would be welcome.
  • Take questions on Twitter and at questions@nsa.gov or something similar. If people ask about programs, point them to that FAQ or write a new answer. The intelligence community is starting behind here, in terms of trust, but being responsive to the public would be a step in the right direction.
  • Link out to media reports that verify statements. After DNI Clapper gave his “least untruthful answer” to Senator Ron Wyden in a Congressional hearing, these “on the record” statements are received with a great deal of skepticism by many Americans. Simply saying something is true or untrue is unlikely to be received as gospel by all.
  • Use animated GIFs to communicate with a younger demographic. Actually, scratch that idea.

Fung outlines principles for democratic transparency and open government

Posted on by

0

Archon Fung has published a new paper” [PDF] on open government, information and democracy. The abstract includes a useful breakdown of the components of democratic transparency:

In Infotopia, citizens enjoy a wide range of information about the organizations
upon which they rely for the satisfaction of their vital interests. The provision of
that information is governed by principles of democratic transparency. Democratic
transparency both extends and critiques current enthusiasms about transparency. It
urges us to conceptualize information politically, as a resource to turn the behavior of
large organizations in socially beneficial ways. Transparency efforts have targets, and we
should think of those targets as large organizations: public and civic, but especially private
and corporate. Democratic transparency consists of four principles. First, information
about the operations and actions of large organizations that affect citizens’ interests
should be rich, deep, and readily available to the public. Second, the amount of available
information should be proportionate to the extent to which those organizations
jeopardize citizens’ interests. Third, information should be organized and provided in
ways that are accessible to individuals and groups that use that information. Finally, the
social, political, and economic structures of society should be organized in ways that
allow individuals and groups to take action based on Infotopia’s public disclosures.

Fung’s paper focuses on focus upon “information about the activities of
large organizations—especially corporations and governments—rather than individuals” and “the important, defensive, face of the informational problem: information that people need to protect themselves against the actions of large organizations and to navigate the terrain created by such organizations,” as opposed to the myriad positive uses of open government data.