National Security Archive finds 40% E-FOIA compliance rate in federal government agencies

underConstruction

For Sunshine Week 2015, the National Security Archive​ conducted an audit of how well 165 federal government agencies in the United States of America comply with the E-FOIA Act of 1996. They found that only 67 of them had online libraries that were regularly updated with a significant number of documents released under the Freedom of Information Act. The criteria for the 165 agencies were that they had to have a chief Freedom of Information Officer and components that handled more than 500 FOIA requests annually.

Almost a decade after the E-FOIA Act, that’s about a 40% compliance rate. I wonder if the next U.S. Attorney General or the next presidential administration will make improving on this poor performance priority. It’s important for The United States Department of Justice​ to not only lead by example but push agencies into the 21st century when it comes to the Freedom of Information Act.

It would certainly help if Congress passed FOIA reform.

On that count, the Archive highlights a relevant issue in the current House and Senate FOIA reform bills in Congress: the FOIA statute states that documents that are “likely to become the subject of subsequent requests” should be published electronic reading rooms:

“The Department of Justice’s Office of Information Policy defines these records as “frequently requested records… or those which have been released three or more times to FOIA requesters.” Of course, it is time-consuming for agencies to develop a system that keeps track of how often a record has been released, which is in part why agencies rarely do so and are often in breach of the law. Troublingly, both the current House and Senate FOIA bills include language that codifies the instructions from the Department of Justice.

The National Security Archive believes the addition of this “three or more times” language actually harms the intent of the Freedom of Information Act as it will give agencies an easy excuse (“not requested three times yet!”) not to proactively post documents that agency FOIA offices have already spent time, money, and energy processing. We have formally suggested alternate language requiring that agencies generally post “all records, regardless of form or format that have been released in response to a FOIA request.”

This is a point that Members of Congress should think through carefully as they take another swing at reform. As I’ve highlighted elsewhere, FOIA requests that industry make are an important demand signal to show where data with economic value lies. (It’s also where the public interest tends to lie, with respect to FOIA requests from the media.)

While it’s true that it would take time and resources to build and maintain a system that tracks such requests by industry, there should already be a money trail from the fees paid to the agency. If FOIA reform leads to modernizing how it’s implemented, perhaps tying FOIA.gov to Data.gov might finally take place. The datasets are the subject of the most FOIA requests are the ones that should be prioritized for proactive disclosure online.

Adding a component that identifies which data sets are frequently requested, particularly periodically, should be a priority across the board for any administration that seeks to “manage information as an asset.” Adding the volume and periodicity of requests to the expanding national enterprise data inventory might naturally follow. It’s worth noting, too, that reform of the FOIA statute may not be necessary to achieve this end, if the 18F team working on modernizing FOIA software worked on it.

Could Hillary Clinton’s email account galvanize Congress to pass FOIA reform?

IMG_1992It’d be swell if the flap over former Secretary of State Hillary Clinton’s personal email account catalyzed the passage of Freedom of Information Act reform in Congress. Trevor Timm, executive director of the Freedom of the Press Foundation, laid out a strong case in the Guardian today for why both sides of the aisle should support reform:

Instead of both parties competing over who can be more secretive, like they did in the 2012 presidential campaign, this is also a great opportunity for 2016 presidential candidates to debate about who can deliver the most transparent White House. That doesn’t mean just voluntarily releasing emails you want the public to see – though that’s a start – but implementing lasting policy changes and laws that will change the trajectory of US secrecy law, which has grown out of control in the past decade.

The challenge is that the interests that didn’t want that reform to happen, both inside and outside of government, aren’t going to go away, from the financial industry to government agencies.

As readers no doubt recall, FOIA reform bills passed the U.S. Senate and House *unanimously* last year and yet failed to become law.

The pushback is already (quietly) happening in Congress, as reported last week in E&E publishing:

“I think a number of the agencies are probably concerned. This is the impression that I get: They think that you shouldn’t have this presumption that things should be revealed. In other words, there should be more of a screening process,” [Representative Elijah] Cummings said. “It’s hard for them to just come outright and say, ‘No, we don’t like that, we’re not going to do it.’ But I get that impression that they don’t feel that people need to have access to every record.”

Asked whether he or other lawmakers have heard from agencies regarding his bill, Cummings said their concerns about FOIA are more subtly made to Congress.

“In general, in general. But I don’t think it’s a big push, but that’s just the impression I get,” said the ranking member on the House Oversight and Government Reform Committee.

That doesn’t mean that reform won’t happen, or that it couldn’t be a political winner for members of both parties, particularly Republican Senators who aspire to higher office. This year, editorial boards are more outspoken on the issue and transparency could, once again, be a campaign issue. Here’s hoping that’s enough to lead to Congress enacting FOIA reform the country needs, not a watered down bill.

In the wake of scandal, the State of Oregon seeks to restore trust through publishing public records

or-state-sealIn a fascinating turn of events, rainy Oregon is embracing sunlight online after a scandal that led to the resignation of its governor. After governor Kate Brown was sworn in as the 38th governor of the state of Oregon, replacing fellow Democrat John Kitzhaber, her administration chose to try to restore public trust by not only posting public records requests online but including the authors, status and a downloadable link to the records themselves, once fulfilled. The records only go back to January 15th, 2015, with a note that requests made prior to that date are “still being processed.”

The City of Oakland’s public records system, built by Code for America, does the same thing but this appears to set a new bar for state government that’s unmatched in the United States of America. As has been reported elsewhere, exemptions to Oregon’s public records laws mean that this website will be no panacea, but it looks like progress from 3000 miles away in snowy Cambridge.

As Kirk Johnson reported for the New York Times, Brown’s record includes open government work while she was a state legislator, where, as Senate Majority Leader, she worked to reform Oregon’s ethics law and helped to enact legislation that created an online campaign finance database.

“…throughout my 24 years in public service, I have also sought to promote transparency and trust in government, working to build confidence that our public dollars are spent wisely,” she said, in her inaugural speech.

Later in her remarks, Governor Brown said that “we must seize this moment to work across party lines to restore the public’s trust. That means passing meaningful legislation that strengthens the capacity and independence of the Government Ethics Commission. We also must strengthen laws to ensure timely release of public documents.”

On that count, it’s notable that two of the records requests that have been posted for download involve Cynthia Hayes, the fiancee of former Governor Kitzhaber who was at the center of the scandals that led to his resignation. One comes from Margaret Olney, who is quite likely the same Margaret Olney who served in Oregon’s Department of Justice. The other requester was Alejandra Lazo, who co-authored a Wall Street Journal article on former Governor Kitzhaber’s resignation. In an interesting sidenote, the records for both responses were uploaded to Dropbox.

If you know of another state that meet or exceeds this standard for digital transparency, or have experience or feedback regarding the quality or importance of the public records posted by Oregon, please let us know in a comment.

Emily Shaw, the national policy manager at the Sunlight Foundation, said on Twitter that she has not seen any other state’s public records system exceed this standard of transparency.

https://twitter.com/emilydshaw/status/568469416406618112

Federal government agencies receive .91 GPA in FOIA compliance from Center for Effective Government

Today, the Center for Effective Government released a scorecard for access to information from the 15 United States federal government agencies that received the most Freedom of Information Act (FOIA) requests, focusing upon an analysis of their performance in 2013.

The results of the report (PDF) for the agencies weren’t pretty: if you computed a grade point average from this open government report card (and I did) the federal government would receive a D for its performance. 7 agencies outright failed, with the State Department receiving the worst grade (37%).

The grades were based upon:

  1. How well agencies processed FOIA requests, including the rate of disclosure, fullness of information provided, and timeliness of the response
  2. How well the agencies established rules of information access, including the effectiveness of agency polices on withholding information and communications with requestors
  3. Creating user-friendly websites, including features that facilitate the flow of information to citizens, associated online services, and up-to-date reading rooms

The report is released at an interesting historic moment for the United States, with Sunshine Week just around the corner. The United States House of Representatives just unanimously passed a FOIA Reform Act that is substantially modeled upon the Obama administration’s proposals for FOIA reforms, advanced as part of the second National Open Government Action Plan. If the Senate takes up that bill and passes it, it would be one of the most important, substantive achievements in institutionalizing open government beyond this administration.

The Citizens for Responsibility and Ethics in Washington have disputed the accuracy of this scorecard, based upon the high rating for the Department of Justice. CREW counsel Anne Weismann:

It is appropriate and fair to recognize agencies that are fulfilling their obligations under the FOIA. But CEG’s latest report does a huge disservice to all requesters by falsely inflating DOJ’s performance, and ignoring the myriad ways in which that agency — a supposed leader on the FOIA front — ignores, if not flouts, its obligations under the statute.

Last Friday, I spoke with Sean Moulton, the director of open government policy at the Center for Effective Government, about the contents of the report and the state of FOIA in the federal government, from the status quo to what needs to be done. Our interview, lightly edited for content and clarity, follows.

What was the methodology behind the report?

Moulton: Our goal was to keep this very quantifiable, very exact, and to try and lay out some specifics. We thought about what the components were necessary for a successful FOIA program. The processing numbers that come out each year are a very rich area for data. They’re extremely important: if you’re not processing quickly and releasing information, you can’t be successful, regardless of other components.

We did think that there are two other areas that are important. First, online services. Let’s face it, the majority of us live online in a big way. It’s a requirement now for agencies to be living there as well. Then, the rules. They’re explained to the agencies and the public, in how they’re going to do things when they get a request. A lot of the agencies have outdated rules. Their current practices may be different, and they may be doing things that the rules don’t say they have to, but without them, they may stop. Consistent rules are essential for consistent long term performance.

A few months back, we released a report that laid out what we felt were best practices for FOIA regulations. We went through a review of dozens of agencies, in terms of their FOIA regulations, and identified key issues, such as communicating with the requester, how you manage confidential business information, how you handle appeals, and how you handle timelines. Then we found inside existing regulations the best ways this was being handled. It really helped us here, when we got to the rules. We used that as our roadmap. We knew agencies were already doing these things, and making that commitment. The main thing we measured under the rules were the items from that best practices report that were common already. If things were universal, we didn’t want to call a best practice, but a normal practice.

Is FOIA compliance better under the Obama administration, more than 4 years after the Open Government Directive?

Moulton: In general, I think FOIA is improving in this administration. Certainly, the administration itself is investing a great deal of energy and resources in trying to make greater improvements in FOIA, but it’s challenging. None of this has penetrated into national security issues.

I think it’s more of a challenge than the administration thought it would be. It’s different from other things, like open data or better websites. The FOIA process has become entrenched. The biggest open government wins were in areas where they were breaking new ground. There wasn’t a culture or way of doing this or problems that were inherited. They were building from the beginning. With FOIA, there was a long history. Some agencies may see FOIA as some sort of burden, and not part of their mission. They may think of it as a distraction from their mission, in fact. When the Department of Transportation puts out information, it usually gets used in the service of their mission. Many agencies haven’t internalized that.

There’s also the issue of backlogs, bureaucracy, lack of technology or technology that doesn’t work that well — but they’re locked into it.

What about redaction issues? Can you be FOIA compliant without actually honoring the intent of the request?

Moulton: We’re very aware of this as well. The data is just not there to evaluate that. We wish it was. The most you get right now is “fully granted” or “partly granted.” That’s incredibly vague. You can redact 99% or 1% and claim it’s partially redacted, either way. We have no indicator and no data on how much is being released. It’s frustrating, because something like that would help us get a better sense on whether agencies would benefit would new policies

We do know that the percentage of full grants has dropped every year, for 12 years, from the Clinton administration all the way through the Bush administration to today. It’s such a gray area. It’s hard to say whether it’s a terrible thing or a modest change.

Has the Obama administration’s focus on open government made any difference?

Moulton: I think it has. There were a couple of agencies that got together on FOIA reform. The EPA led the team, with the U.S. National Archives and the Commerce Department, to build a new FOIA tool. The outward-facing part of the tool enables a user to go to a single spot, request and track it. Other people could come and search FOIA’ed documents. Behind the scenes, federal workers could use the tool to forward requests back and forth. This fits into what the administration has been trying to do, using technology better in government

Another example, again at the EPA, is where they’ve put together a proactive disclosure website. They got a lot of requests, like if there are inquiries about properties, environmental history, like leaks and spills, and set up a site where you could look up real estate. They did this because they went to FOIA requests and see what people wanted. That has cut down their requests to a certain percentage.

Has there been increasing FOIA demand in recent years, affecting compliance?

Moulton: I do think FOIA requests have been increasing. We’ll see what this next year of data shows. We have seen a pretty significant increase, after a significant decrease in the Bush administration. That may be because this administration keeps speaking about open government, which leads to more hopeful requestors. We fully expect that in 2013, there will be more requests than the prior year.

DHS gets the biggest number of all, but that’s not surprising when we look at the size of it. It’s second biggest agency, after Defense, and the biggest domestic facing agency. when you start talking about things like immigration and FEMA, which go deep into communities and people’s lives, in ways that have a lot impact, that makes sense.

What about the Department of Justice’s record?

Moulton: Well, DoJ got the second highest rating, but we know they have a mixed record. There are things you can’t measure and quantify, in terms of culture and attitude. I do know there were concerns about the online portal, in terms of the turf war between agencies. There were concerns about whether the tech was flexible, in terms of meeting all agency needs. If you want to build a government-wide tool, it needs to have real flexibility. The portal changed the dialogue entirely

Is FOIA performance a sufficient metric to analyze any administration’s performance on open government?

Moulton: We should step back further and look at the broader picture, if we’re going to talk about open government. This administration has done things, outside of FOIA, to try to open up records and data. They’ve built better online tools for people to get information. You have to consider all of those things.

Does that include efforts like the Intelligence Community Tumblr?

Moulton: That’s a good example. One thing this administration did early on is to identify social media outlets. We should be going there. We can’t make citizens come to us. We should go to where people are. The administration pushed early on that agencies should be able to use Tumblr and Twitter and Facebook and Flickr and so on.

Is this social media use “propaganda,” as some members of the media have suggested?

Moulton: That’s really hard to decide. I think it can result in that. It has the potential to be misused to sidestep the media, and not have good interaction with the media, which is another important outlet. People get a lot of their information from the media. Government needs to have good relationship.

I don’t think that’s the intention, though, just as under Clinton, when they started setting up websites for the first time. That’s what the Internet is for: sharing information. That’s what social media can be used for, so let’s use what’s there.

U.S. House unanimously votes in favor of FOIA reform and a more open government

Earlier tonight, The United States House of Representatives voted 410-0 to pass the FOIA Oversight and Implementation Act. If the FOIA Act passes through the Senate, the bill would represent the most important update to United States access to information laws in generations.

“Transparency in government is a critical part of restoring trust and the House will continue to work to make government more transparent and accessible to all Americans,” said House Majority Leader Eric Cantor (R-VI). “By expanding the FOIA process online, the FOIA Oversight and Implementation Act creates greater transparency and continues our open government efforts in the House.”

The FOIA Oversight and Implementation Act (FOIA), ‪‎H.R.1211‬, is one of the best opportunities to institutionalize open government in the 113th Congress, along with the DATA Act, which passed the House of Representatives 388-1 last November.

The FOIA reform bill now moves to the Senate, which passed unanimous FOIA reform legislation in the last Congress.

As Nate Jones detailed at the National Security Archive, the Senate’s own legislative effort to reform FOIA, the so-called the “Faster FOIA Act” (S.627S. 1466), was not picked up by the House: the open government bill was hijacked in service of a 2011 budget deal, where the FOIA provisions in it ultimately met an untimely end. Chairman Darrell Issa (R-CA.), Ranking Member Elijah Cummings (D-MD), and Representative Mike Quigley (D-IL) chose to draft their own bill instead of taking that bill up again.

Open government advocates applauded the unanimous passage of the FOIA Act, although there are some caveats about its provisions for the Senate to consider.

“This vote shows strong congressional support for government transparency and the Freedom of Information Act,” said Sean Moulton, Director of Open Government Policy at the Center for Effective Government, in a statement:

Since its original passage nearly 50 years ago, FOIA has been a cornerstone of the public’s right to know. By modernizing FOIA, H.R. 1211 would improve Americans’ ability to access public information and strengthen our democracy.

We thank the chair and ranking member of the House Committee on Oversight and Government Reform, Reps. Darrell Issa (R-CA) and Elijah Cummings (D-MD), who worked with the open government community to develop this legislation in a bipartisan fashion. We urge the Senate to advance legislation addressing these issues and other pressing FOIA reforms, including the need to rein in secrecy claims under Exemption 5, which restrict access to important information about government operations.

Access to public information is crucial to our democracy and the government’s effectiveness. It allows Americans to actively engage in policymaking in a thoughtful, informed manner and to hold public officials accountable for decisions that impact us all.

The bill represents important incremental, improvements to the FOIA process, but “it doesn’t address some fundamental shortfalls in the way that the FOIA is implemented and viewed within the Federal government,” wrote Matt Rumsey, policy analyst at the Sunlight Foundation:

… A “presumption of openness” and improved online infrastructure are important, but the bigger challenge will be getting agencies to change their posture away from one of non-disclosure and often aggressive litigation that is opposed to openness. … It clearly shows that ensuring public access to government information is not a partisan issue, or even one that should divide the branches of government. We hope to see the Senate take up legislation in the near future so that both chambers can work together to send a strong FOIA reform bill to President Obama’s desk for him to sign.

Passage of the House bill is a good first step but only a first step, wrote Anne Weismann, chief counsel of Citizens for Responsibility and Ethics in Washington:

Without a doubt these are needed reforms. As CREW has long advocated, however, meaningful FOIA reform must include changes in the FOIA’s exemptions to make the statute work as Congress intended.  All too often agencies hide behind Exemption 5 and its protection for privileged material to bar public access to documents that would reveal the rationale behind key government decisions.  For example, the Department of Justice denies every request for a legal opinion issued by DOJ’s Office of Legal Counsel that determines what a law means and what conduct it permits, claiming to reveal these opinions would harm the agency’s deliberative process.  This has led to the creation of a body of secret law — precisely what Congress sought to prevent when it enacted the FOIA.

To address this serious problem, CREW has advocated adding a balancing test to Exemption 5 that would require the agency and any reviewing court to balance the government’s claimed need for secrecy against the public interest in disclosure.  Other needed reforms include a requirement that agencies post online all documents disclosed under the FOIA.  The House bill, however, does not incorporate any of these reforms.

This post has been updated with additional statements over time.