US FOIA Advisory Committee recommends agencies harmonize FOIA and open data programs

Yesterday, the United States Freedom of Information Act Advisory Committee met at the National Archives in Washington and approved a series of recommendations that would, if implemented, dramatically improve public access to public information. And in May, it will consider … Continue reading

US FOIA Advisory Committee considers recommendations to part the rising curtains of secrecy

Government secrecy, as measured by censorship or non-responsiveness under the Freedom of Information Act, is at an all-time high during the Trump administration. The Freedom of Information Act is getting worse under Trump for a variety of reasons. Continued secrecy … Continue reading

US House hearing on transparency misses the open government forest for the FOIA trees

When sunshine is applied to government, what’s revealed can be determined by the eye of the beholder. After the 115th Congress neglected oversight of open government, the U.S. House Oversight Committee held a hearing on the Freedom of Information Act … Continue reading

National Security Archive finds 40% E-FOIA compliance rate in federal government agencies

underConstruction

For Sunshine Week 2015, the National Security Archive​ conducted an audit of how well 165 federal government agencies in the United States of America comply with the E-FOIA Act of 1996. They found that only 67 of them had online libraries that were regularly updated with a significant number of documents released under the Freedom of Information Act. The criteria for the 165 agencies were that they had to have a chief Freedom of Information Officer and components that handled more than 500 FOIA requests annually.

Almost a decade after the E-FOIA Act, that’s about a 40% compliance rate. I wonder if the next U.S. Attorney General or the next presidential administration will make improving on this poor performance priority. It’s important for The United States Department of Justice​ to not only lead by example but push agencies into the 21st century when it comes to the Freedom of Information Act.

It would certainly help if Congress passed FOIA reform.

On that count, the Archive highlights a relevant issue in the current House and Senate FOIA reform bills in Congress: the FOIA statute states that documents that are “likely to become the subject of subsequent requests” should be published electronic reading rooms:

“The Department of Justice’s Office of Information Policy defines these records as “frequently requested records… or those which have been released three or more times to FOIA requesters.” Of course, it is time-consuming for agencies to develop a system that keeps track of how often a record has been released, which is in part why agencies rarely do so and are often in breach of the law. Troublingly, both the current House and Senate FOIA bills include language that codifies the instructions from the Department of Justice.

The National Security Archive believes the addition of this “three or more times” language actually harms the intent of the Freedom of Information Act as it will give agencies an easy excuse (“not requested three times yet!”) not to proactively post documents that agency FOIA offices have already spent time, money, and energy processing. We have formally suggested alternate language requiring that agencies generally post “all records, regardless of form or format that have been released in response to a FOIA request.”

This is a point that Members of Congress should think through carefully as they take another swing at reform. As I’ve highlighted elsewhere, FOIA requests that industry make are an important demand signal to show where data with economic value lies. (It’s also where the public interest tends to lie, with respect to FOIA requests from the media.)

While it’s true that it would take time and resources to build and maintain a system that tracks such requests by industry, there should already be a money trail from the fees paid to the agency. If FOIA reform leads to modernizing how it’s implemented, perhaps tying FOIA.gov to Data.gov might finally take place. The datasets are the subject of the most FOIA requests are the ones that should be prioritized for proactive disclosure online.

Adding a component that identifies which data sets are frequently requested, particularly periodically, should be a priority across the board for any administration that seeks to “manage information as an asset.” Adding the volume and periodicity of requests to the expanding national enterprise data inventory might naturally follow. It’s worth noting, too, that reform of the FOIA statute may not be necessary to achieve this end, if the 18F team working on modernizing FOIA software worked on it.

Could Hillary Clinton’s email account galvanize Congress to pass FOIA reform?

IMG_1992It’d be swell if the flap over former Secretary of State Hillary Clinton’s personal email account catalyzed the passage of Freedom of Information Act reform in Congress. Trevor Timm, executive director of the Freedom of the Press Foundation, laid out a strong case in the Guardian today for why both sides of the aisle should support reform:

Instead of both parties competing over who can be more secretive, like they did in the 2012 presidential campaign, this is also a great opportunity for 2016 presidential candidates to debate about who can deliver the most transparent White House. That doesn’t mean just voluntarily releasing emails you want the public to see – though that’s a start – but implementing lasting policy changes and laws that will change the trajectory of US secrecy law, which has grown out of control in the past decade.

The challenge is that the interests that didn’t want that reform to happen, both inside and outside of government, aren’t going to go away, from the financial industry to government agencies.

As readers no doubt recall, FOIA reform bills passed the U.S. Senate and House *unanimously* last year and yet failed to become law.

The pushback is already (quietly) happening in Congress, as reported last week in E&E publishing:

“I think a number of the agencies are probably concerned. This is the impression that I get: They think that you shouldn’t have this presumption that things should be revealed. In other words, there should be more of a screening process,” [Representative Elijah] Cummings said. “It’s hard for them to just come outright and say, ‘No, we don’t like that, we’re not going to do it.’ But I get that impression that they don’t feel that people need to have access to every record.”

Asked whether he or other lawmakers have heard from agencies regarding his bill, Cummings said their concerns about FOIA are more subtly made to Congress.

“In general, in general. But I don’t think it’s a big push, but that’s just the impression I get,” said the ranking member on the House Oversight and Government Reform Committee.

That doesn’t mean that reform won’t happen, or that it couldn’t be a political winner for members of both parties, particularly Republican Senators who aspire to higher office. This year, editorial boards are more outspoken on the issue and transparency could, once again, be a campaign issue. Here’s hoping that’s enough to lead to Congress enacting FOIA reform the country needs, not a watered down bill.

18F launches alpha foia.gov in a bid to reboot Freedom of Information Act requests for the 21st century

alpha foia gov

18F, the federal government’s new IT development shop, has launched a new look at the Freedom of Information Act (FOIA) in the form of a open source application hosted on Github. Today’s announcement is the most substantive evidence yet that the Obama administration will indeed modernize the Freedom of Information Act, as the United States committed to doing in its second National Action Plan on Open Government. Given how poor some of the “FOIA portals” and underlying software that supports them exists is at all level of government, this is tremendous news for anyone that cares about the use of technology to support open government.

Notably, 18F already has a prototype (pictured above) online that shows what a consolidated request submission hub could look like and plans to iterate upon it.  This is a perfect example of “lean government,” or the application of lean startup principles and agile development to the creation of citizen-centric services in the public sector.  Demonstrating its commitment to developing free and open source software in the open, 18F asked the public to follow the process online at their FOIA software repository on Github, send them feedback or even contribute to the project.

18F has now committed to creating software that improvse how requests made under the Freedom of Information Act can be improved through technology. Specifically that it will develop tools that “improve the FOIA request submission experience,” “create a scalable infrastructure for making requests to federal agencies” and “make it easier for requesters to find records and other information that have already been made available online.”

According to 18F’s blog post, this work is supported and overseen by a “FOIA Task Force,” consisting of representatives from the Department of Justice, Environmental Protection Agency, the Office of Management and Budget, the Office of Science and Technology Policy. The task force will need to focus upon more than technology: while poor software has hindered requests and publishing, that’s not the primary issue that’s hindering the speed or quality of responses.

Despite the U.S. attorney general’s laudable commitment to a new era of open government in 2009, the Obama administration received a .91 GPA in FOIA compliance earlier this year from the Center for Effective Government.

While White House press secretary Josh Earnest may be well correct in stating that the federal government is processing more FOIA requests than ever, As the National Security Archive noted in March, the use of a FOIA exemption (protecting “deliberative processes”) to deny or heavily redact requests has skyrocketed in the past two years.

use of B5 exemptions

[NATIONAL SECURITY ARCHIVE: Chart created by Lauren Harper.]

As with the reduced access to government staff and scientists that a group of 38 journalism and open government advocates decried earlier this year, improving FOIA compliance cannot solely be addressed through technological means. To address endemic government secrecy and outright abuse of exemptions to protect against politically inconvenient disclosures, Obama administration — in particular, the U.S. Justice Department — will need to expend political capital and push agencies to actually shift the cultural default towards openness and release uncomfortable or embarrassing data and documents and not redact them beyond understanding.

That’s admittedly a huge challenge, particularly for an administration facing multiple foreign and domestic conundrums, including a scandal over missing IRS emails and obfuscated records in an election year and the most politically polarized Congress and electorate in the nation’s history, but if President Barack Obama is truly committed to “creating an unprecedented level of openness in government,” it’s one that he and his administration will need to take on.

DC city government issues executive order on open data, FOIA portal and chief data officer

Today, the District of Columbia launched a new online service for Freedom of Information Act requests and Mayor Vincent Gray issued a transparency, open government and open data directive. DC city government has come under harsh criticism from the ACLU for its record on FOIA and transparency and has a spate of recent corruption scandals, albeit not one that appears to be worse than other major American cities.

“This new online FOIA system is a key part of our strategy to improve government transparency and accountability,” said Mayor Gray, in a statement. “In addition, the executive order I am issuing today sends an important message to District government agencies and the public: Everyone wins when we make it easier for the public to understand the workings of the District government. I also look forward to seeing the exciting applications I hope the District’s technology community will develop with the government data we will be putting online.”

Here’s what Mayor Gray has instructed DC government to do:

1) Within 30 days from today, the DC chief technology officer (currently Rob Mancini) must create “a common Web portal” that “will serve as the source for District-wide and agency activities related to this Transparency and Open Data Directive.” Translation: OCTO must create a new website that aggregates information related to this directive.

2) OCTO will publish technical standards for open data by November 1, 2014. DC government could refer to the Sunlight Foundation’s Open Data Guidelines as a useful reference, or the canonical 8 principles of Open Government Data.

3) Within 120 days from today, the DC City Administrator and each deputy mayor must identify at least 3 new high-value datasets to publish to the DC Data Catalog that are either not currently available or not available in an exportable format.

4) Starting on October 1, 2014, and continuing annually, each DC agency will develop and publish an “Open Government Report” that will “describe how the agency has or will enhance and develop transparency, public participation, and collaboration. Each agency shall include in its open government report a description of the information (including data) that will be made available to the public, formats in which information and data will be made.”

Translation: city agencies will report on how they’re doing complying with this mandate. Hopefully, the DC Office of Open Government will be an effective ombudsman on that progress, along with directly engaging on Freedom of Information Act disputes and processes, and will do more public engagement around open government or open data than @OCTONEWS has to date.

Unfortunately, and not a little bit ironically, the directive was published online as a scanned-in PDF that is neither searchable nor accessible to the blind, itself embodying the way not to release text online in the 21st century. Below, I have summarized the main deliverables mandated in the directive and converted the images to plain text. Following the order is criticism from open government advocate, civic hacker, and DC resident Josh Tauberer.


GOVERNMENT OF THE DISTRICT OF COLUMBIA

ADMINISTRATIVE ISSUANCE SYSTEM

Mayor’s Order 2014-170
July 21, 2014

SUBJECT: Transparency, Open Government and Open Data Directive

ORIGINATING AGENCY: Office of the Mayor

By virtue of the authority vested in me as Mayor of the District of Columbia by section 422(2) and (11) of the District of Columbia Home Rule Act, approved December 24, 1973, 87 Stat. 790, Pub. L. No. 93-198, D.C. Official Code § 1-204.22(2) and (11) (2012 Repl.), and section 206 of the District of Columbia Freedom of Information Act, effective March 25, 1977, D.C. Law 1-96, D.C. Official Code § 2-536 (2012 Repl.), it is hereby ORDERED that:

SECTION 1: Introduction.

a. Background. The District of Columbia government (“District”) is committed to creating an unprecedented level of openness in government. Agency heads will work together and with the public to ensure public trust, and an open and effective government by establishing a system of transparency, public participation, collaboration, and accountability that increases the public’s confidence in their government. The goal of this directive is to provide a tool for prescribing and institutionalizing change within all departments and agencies.

The District has been a leader in government transparency and open data policy in the United States. In 2001, the Freedom of Information Act was amended to require that certain public records be published online. Since 2006, the District has been making data publicly available on the Internet. In January 2011, Mayor’s Memorandum 2011-1, entitled Transparency and Open Government Policy, was issued, recognizing that the District government needed to continue to proactively provide information to citizens, and thereby reduce the need for information requests. This directive implements Mayor’s Memorandum 2011-1, to require District government departments and agencies to take the following
steps to achieve the goal of creating a more transparent and open government:

b. Definitions.

  1. “Chief Data Officer” (“CDO”) means the Chief Technology Officer or a Chief Data Officer designated by the Chief Technology Officer.
  2.  “Data” means statistical, or factual, quantitative, or qualitative information that are regularly maintained or created by or on behalf of a District agency, and controlled by such agency in structured formats, including statistical or factual information about image files and geographic information system data.
  3. “Dataset” means a named collection of related records, with the collection containing data organized or formatted in a specific or prescribed way, often in tabular form.
  4. “Open Government Coordinator” means agency personnel designated by an agency head, in coordination with the Office of the Chief Technology Officer (“OCTO”) or the CDO as appropriate, to ensure that the information and data required to be published online is published and updated as required by this Order.
  5. “Protected data” means (i) any dataset or portion thereof to which an agency may deny access pursuant to the District of Columbia Freedom of Information Act, effective March 25, 1977 (D.C. Law 1-96; D.C. Official Code § 2-531 et seq.)(“FOIA”), or any other law or rule or regulation; (ii) any dataset that contains a significant amount of data to which an agency may deny access pursuant to FOIA or any other law or rule or regulation promulgated thereunder, if the removal of such protected data from the dataset would impose an undue financial or administrative burden on the agency; or (iii) any data which, if disclosed on the District of Columbia Data Catalog, could raise privacy, confidentiality or security concerns or jeopardize or have the potential to jeopardize public health, safety or welfare.

C. Scope.

a. The requirements of this Order shall be applied to any District of Columbia department, office, administrative unit, commission, board, advisory committee or other division of the District government (“agency”), including the records of third party agency contractors that create or acquire information, records, or data on behalf of a District agency.

b. Any agency that is not subject to the jurisdiction of the Mayor under the Freedom of Information Act or any other law is strongly encouraged to comply with the requirements of this Order.

SECTION 2: Transparency and Open Government Policy.

a. Publish Government Information Online. To increase accountability and transparency, promote informed public participation, and create economic development opportunities, each District agency shall expand access to information by making it proactively available online, and when practicable, in an open format that can be retrieved, downloaded, indexed, sorted, searched, and reused by commonly used Web search applications and commonly used software to facilitate access to and reuse of information. Examples of open format include HTML, XML, CSV, JSON, RDF or XHTML. The Freedom of Information Act creates a presumption in favor of openness and publication (to the extent permitted by law and subject to valid privacy, confidentiality, security, or other restrictions).

b. Open Government Web Portal: Within 30 days from the date of this Order, the Chief Technology Officer shall establish a common web portal that will serve as the source for District-wide and agency activities related to this Transparency and Open Data Directive. The Chief Technology Officer, in his or her discretion, may build upon an existing web portal, or may establish a new portal. Each agency shall be responsible for ensuring that the information required to be published online is accessible from the agency’s designated Open Government and FOIA webpage. The required information shall include, but is not limited to, where applicable:

  1. Means for the public to submit and track Freedom of Information Act requests online;
  2. The information required to be made public under this Directive and D.C. Official Code § 2-536, including links to:
    A. Employee salary information;
    B. Administrative staff manuals and instructions that affect the public;
    C. Final opinions and orders made in the adjudication of cases;
    D. Statements of policy, interpretations of policy, and rules adopted by the agency;
    E. Correspondence and other materials relating to agency regulatory, supervisory or enforcement responsibilities in which the rights of the public are determined;
    F. Information dealing with the receipt or expenditure of public or other funds;
    G. Budget information;
    H. Minutes of public meetings;
    I. Absentee real property owners and their agent’s names and mailing addresses;
    J. Pending and authorized building permits;
    K. Frequently requested public records; and
    L. An index to the records referred to in this section;
  3. Freedom of Information Act reports;
  4. An organizational chart or statement of the agency’s major components;
  5. Links to high-value datasets (as defined in section 3(a)(4);
  6. Public Meeting Notices and minutes required to be published under the Open Meetings Act and Freedom of Information Act; and
  7. A mechanism for the public to submit feedback on the agency’s Open Government Report or other agency actions.

c. Open Government Report. To institutionalize a culture of transparent and open government, accountability, and to expand opportunities for resident participation and collaboration, beginning October 1, 2014, and each year thereafter, each agency shall develop and publish an Open Government Report that will describe how the agency has or will enhance and develop transparency, public participation, and collaboration. Each agency shall include in its open government report a description of the information (including data) that will be made available to the public, formats in which information and data will be made available, a schedule for making the information available, the dates for which information and datasets will be updated, and contact information for agency Open Government Coordinators. The Open Government Report shall address the following topics, and be transmitted to the Mayor and Director of the Office of Open Government:

  1. Transparency: The Open Government Report shall reference statutes, regulations, policies, legislative records, budget information, geographic data, crime statistics, public health statistics, and other public records and data, and describe steps each agency has taken or will take to:A. Meet its legal information dissemination obligations under Freedom of Information Act and Open Meetings Act;
    B. Create more access to information and opportunities for public participation; and
    C. Conduct its work more openly and publish its information online, including a plan for how each board and commission subject to the Open Meetings Act will ensure that all of its meetings are, where practicable, webcast live on the Internet.
  2. Participation: To create more informed and effective policies, each agency shall enhance and expand opportunities for the public to participate throughout agency decision-making processes. The Open Government Report will include descriptions of or plans to provide:A. Online access to proposed rules and regulations;
    B. Online access to information and resources to keep the public properly informed (such as frequently asked questions, contact information of city officials’ and departments, and other supportive content);
    C. Opportunities for the public to comment through the Web on any proposed rule, ordinance, or other regulation;
    D. Methods of identifying stakeholders and other affected parties and inviting their participation;
    E. Proposed changes to internal management and administrative policies to improve participation;
    F. Links to appropriate websites where the public can engage in the District government’s existing participatory processes;
    G. Proposals for new feedback mechanisms, including innovative tools and practices that create new and more accessible methods for public participation; and
    H. A plan that provides a timetable for ensuring that all meetings of boardsand commissions that are subject to the Open Meetings Act are webcast live and archived on the Internet.
  3. Collaboration: The Open Government Report will describe steps the agency will take or has taken to enhance and expand its practices to further cooperation among departments, other governmental agencies, the public, and non-profit and private entities in fulfilling its obligations. The Report will include specific details about:A. Proposed changes to internal management and administrative policies to improve collaboration;
    B. Proposals to use technology platforms to improve collaboration among District employees and the public;
    C. Descriptions of and links to appropriate websites where the public can learn about existing collaboration efforts; and
    D. Innovative methods, such as prizes and competitions, to obtain ideas from and to increase collaboration with those in the private sector, non-profit, and academic communities.

SECTION 3: Open Data Policy.

a. Agency Requirements.

  1. Each agency shall, in collaboration with the Chief Data Officer and OCTO, make available through the online District of Columbia Data Catalog all appropriate datasets, associated extensible metadata, and associated documented agency business processes under the agency’s control. Each agency, in collaboration with OCTO, shall determine the frequency for updates to a dataset, and the mechanism to be utilized. To the extent possible, datasets shall be updated through an automated process to limit the additional burden on agency resources. The publication of an agency’s datasets shall exclude protected data.
  2. Datasets under paragraph (4) shall be made available in accordance with technical standards published by OCTO not later than November 1, 2014 that ensure that data is published in a format that is machine readable, and fully accessible to the broadest range of users, for varying purposes. Datasets shall be made available to the public on an open license basis. An open license on a dataset signifies there are no restrictions on copying, publishing, further distributing, modifying or using the data for a non-commercial or commercial purpose.
  3. For the purposes of identifying datasets for inclusion on the District of Columbia Data Catalog, each agency shall consider whether the information embodied in the dataset is (i) reliable and accurate; (ii) frequently the subject of a written request for public records of the type that a public body is required to make available for inspection or copying under FOIA; (iii) increases agency accountability, efficiency, responsiveness or delivery of services; (iv) improves public knowledge of the agency and its operations; (v) furthers the mission of the agency; or (vi) creates economic opportunity.
  4. Within 120 days of the date of this Order, the City Administrator and each Deputy Mayor shall, collaborating with their cluster agencies, and OCTO, identify at least 3 new high-value datasets to publish to the Data Catalog, in accordance with OCTO’s open data standards. The identified high-value datasets will not be currently available, or not available in an exportable format. For the purposes of this section, “high-value dataset” includes agency outcome data, agency caseload data, data reported to the federal government outcome data, agency caseload data, data reported to the federal government by the agency, agency data reported as part of the performance measurement process, and any data that is tracked by the agency that is not protected data.

b. Chief Data Officer.

  1. The Chief Technology Officer shall designate a Chief Data Officer (“CDO”) for the District of Columbia to coordinate implementation, compliance and expansion of the District’s Open Data Program, to facilitate the sharing of information between departments and agencies, and to coordinate initiatives to improve decision making and management through data analysis. The Chief improve decision making and management through data analysis. The Chief Data Officer shall report to the Chief Technology Officer.
  2. The Chief Data Officer shall:
    A. Identify points of contact, which may include agency open government coordinators within departments, on data related issues who will be responsible for leading intra-departmental open data initiatives;
    B. Emphasize the culture behind open data and the benefits to ensure that opportunities to increase efficiency through open data practices can be obtained from those with the most direct expertise;
    C. Work together with District agencies to develop a methodology and framework that supports the collection, or creation of data in a way that assists in downstream data processing and open data distribution activities;
    D. Identify and overcome challenges with agency proprietary business systems; create and/or leverage opportunities through procurement or other means to upgrade legacy systems to one of an open data architecture; and
    E. Function as a data ombudsman for the public, fielding public feedback and ensuring the policy is included into a long-term data strategy.

c. District of Columbia Open Data Catalog.

  1. A single web portal, or integrated set of websites, shall be established and maintained by or on behalf of the District of Columbia. The Chief Data maintained by or on behalf of the District of Columbia. The Chief Data Officer, in collaboration with OCTO, may build upon previous open data initiatives, or may establish a new portal for managing and delivering open data benefits to constituents.
  2. Any dataset made accessible on the District of Columbia Data Catalog shall use an open format that permits automated processing of such data in a form that can be retrieved via an open application programming interface (API), downloaded, indexed, searched and reused by commonly used web search applications and software; (ii) use appropriate technology to notify the public of updates to the data; and (iii) be accessible to external search capabilities.
  3. OCTO shall (i) post on the portal a list of all datasets available on such portal; and (ii) establish and maintain on the portal an online forum to solicit feedback from the public and to encourage public discussion on open data policies and dataset availability.

d. Open Data Legal Policy.

  1. The District of Columbia Data Catalog and all public data contained on such portal shall be subject to Terms of Use developed by OCTO. Such Terms of Use shall be posted by OCTO in a conspicuous place on the District ofColumbia Data Catalog.
  2. Public data made available on the District of Columbia Data Catalog shall be provided as a public service, on an “as is” basis. Although the District will strive to ensure that such public data are accurate, the District shall make no warranty, representation or guaranty of any type as to the content, accuracy, timeliness, completeness or fitness for any particular purpose or use of any public data provided on such portal; nor shall any such warranty be implied, including, without limitation, the implied warranties of merchantability and fitness for a particular purpose. The District shall assume no liability for any other act identified in any disclaimer of liability or indemnification provision or any other provision set forth in the Terms of Use required under subsection (d)(1) of this section.
  3. The District shall reserve the right to discontinue availability of content on the District of Columbia Data Catalog at any time and for any reason. If a dataset is made accessible by an agency on the District of Columbia Data Catalog and such agency is notified or otherwise learns that any dataset or portion thereof posted on the Data Catalog is factually inaccurate or misleading or is protected data, the agency shall, as appropriate, promptly correct or remove, or cause to be corrected or removed, such data from the Data Catalog and shall so inform the Chief Data Officer.
  4. Nothing in this Order shall be deemed to prohibit OCTO or any agency or any third party that establishes or maintains the District of Columbia Data Catalog on behalf of the District from adopting or implementing measures necessary or appropriate to (1) ensure access to public datasets housed on the Data Catalog; (ii) protect the Data Catalog from unlawful use or from attempts to impair or damage the use of the portal; (iii) analyze the types of public data on the Data Catalog being used by the public in order to improve service delivery or for any other lawful purpose; (iv) terminate any and all display, distribution or other use of any or all of the public data provided on the Data Catalog for violation of any of the Terms of Use posted on the Data Catalog pursuant to subsection (d)(1) of this section; or (v) require a third party providing the District’s public data (or applications based on public data) to the public to explicitly identify the source and version of the public dataset, and describe any modifications made to the public dataset.
  5. Nothing in this Order shall be construed to create a private right of action to enforce any provision of this Order. Failure to comply with any provision of this Order shall not result in any liability to the District, including, but not limited to, OCTO or any agency or third party that establishes or maintains on behalf of the District the Open Data Services Portal required under this Order.

Section 4. Open Government Advisory Group.

a. The Mayor shall convene an Open Government Advisory Group to be chaired and convened by the Mayor’s designee, CDO, and the Director of the Office of Open Government within the Board of Ethics and Government Accountability.

b. The Open Government Advisory Group shall:

  1. Evaluate the District’s progress towards meeting the requirements of this Order and make specific recommendations for improvement; and
  2. Assist the Mayor and CDO in creating policy establishing specific criteria for agency identification of protected data in accordance with FOIA, maintenance of existing data, and the creation of data in open formats.

c. The CDO shall publish the evaluation and recommendations on the Open Government Web Portal or create an Open Government Dashboard that will provide the public with both graphic and narrative evaluation information.

Section 5: EFFECTIVE DATE:

This Order shall be effective immediately.

VINCENT C. GRAY
MAYOR

ATTEST:
CYNTHIA BR CIS-SMITH
SECRETARY OF THE DISTRICT OF COLUMBIA


After the order was published online, GovTrack.us founder Josh Tauberer issued a series of critical tweets and extended his thoughts into a blog post, holding that DC city government adopted the mistakes made by the White House:

There is a strong American tradition — or at least a core American value — that the government does not get in the way of the dissemination of ideas. We don’t always live up to that ideal, but we strive for it. Access to information about the government that comes with restrictions on what we can say when we use it (e.g. attribution & explanation), a waiver of rights or a commitment to indemnify, etc. are all an anathema to accountability and transparency and respect for the public.

Federal government agencies receive .91 GPA in FOIA compliance from Center for Effective Government

Today, the Center for Effective Government released a scorecard for access to information from the 15 United States federal government agencies that received the most Freedom of Information Act (FOIA) requests, focusing upon an analysis of their performance in 2013.

The results of the report (PDF) for the agencies weren’t pretty: if you computed a grade point average from this open government report card (and I did) the federal government would receive a D for its performance. 7 agencies outright failed, with the State Department receiving the worst grade (37%).

The grades were based upon:

  1. How well agencies processed FOIA requests, including the rate of disclosure, fullness of information provided, and timeliness of the response
  2. How well the agencies established rules of information access, including the effectiveness of agency polices on withholding information and communications with requestors
  3. Creating user-friendly websites, including features that facilitate the flow of information to citizens, associated online services, and up-to-date reading rooms

The report is released at an interesting historic moment for the United States, with Sunshine Week just around the corner. The United States House of Representatives just unanimously passed a FOIA Reform Act that is substantially modeled upon the Obama administration’s proposals for FOIA reforms, advanced as part of the second National Open Government Action Plan. If the Senate takes up that bill and passes it, it would be one of the most important, substantive achievements in institutionalizing open government beyond this administration.

The Citizens for Responsibility and Ethics in Washington have disputed the accuracy of this scorecard, based upon the high rating for the Department of Justice. CREW counsel Anne Weismann:

It is appropriate and fair to recognize agencies that are fulfilling their obligations under the FOIA. But CEG’s latest report does a huge disservice to all requesters by falsely inflating DOJ’s performance, and ignoring the myriad ways in which that agency — a supposed leader on the FOIA front — ignores, if not flouts, its obligations under the statute.

Last Friday, I spoke with Sean Moulton, the director of open government policy at the Center for Effective Government, about the contents of the report and the state of FOIA in the federal government, from the status quo to what needs to be done. Our interview, lightly edited for content and clarity, follows.

What was the methodology behind the report?

Moulton: Our goal was to keep this very quantifiable, very exact, and to try and lay out some specifics. We thought about what the components were necessary for a successful FOIA program. The processing numbers that come out each year are a very rich area for data. They’re extremely important: if you’re not processing quickly and releasing information, you can’t be successful, regardless of other components.

We did think that there are two other areas that are important. First, online services. Let’s face it, the majority of us live online in a big way. It’s a requirement now for agencies to be living there as well. Then, the rules. They’re explained to the agencies and the public, in how they’re going to do things when they get a request. A lot of the agencies have outdated rules. Their current practices may be different, and they may be doing things that the rules don’t say they have to, but without them, they may stop. Consistent rules are essential for consistent long term performance.

A few months back, we released a report that laid out what we felt were best practices for FOIA regulations. We went through a review of dozens of agencies, in terms of their FOIA regulations, and identified key issues, such as communicating with the requester, how you manage confidential business information, how you handle appeals, and how you handle timelines. Then we found inside existing regulations the best ways this was being handled. It really helped us here, when we got to the rules. We used that as our roadmap. We knew agencies were already doing these things, and making that commitment. The main thing we measured under the rules were the items from that best practices report that were common already. If things were universal, we didn’t want to call a best practice, but a normal practice.

Is FOIA compliance better under the Obama administration, more than 4 years after the Open Government Directive?

Moulton: In general, I think FOIA is improving in this administration. Certainly, the administration itself is investing a great deal of energy and resources in trying to make greater improvements in FOIA, but it’s challenging. None of this has penetrated into national security issues.

I think it’s more of a challenge than the administration thought it would be. It’s different from other things, like open data or better websites. The FOIA process has become entrenched. The biggest open government wins were in areas where they were breaking new ground. There wasn’t a culture or way of doing this or problems that were inherited. They were building from the beginning. With FOIA, there was a long history. Some agencies may see FOIA as some sort of burden, and not part of their mission. They may think of it as a distraction from their mission, in fact. When the Department of Transportation puts out information, it usually gets used in the service of their mission. Many agencies haven’t internalized that.

There’s also the issue of backlogs, bureaucracy, lack of technology or technology that doesn’t work that well — but they’re locked into it.

What about redaction issues? Can you be FOIA compliant without actually honoring the intent of the request?

Moulton: We’re very aware of this as well. The data is just not there to evaluate that. We wish it was. The most you get right now is “fully granted” or “partly granted.” That’s incredibly vague. You can redact 99% or 1% and claim it’s partially redacted, either way. We have no indicator and no data on how much is being released. It’s frustrating, because something like that would help us get a better sense on whether agencies would benefit would new policies

We do know that the percentage of full grants has dropped every year, for 12 years, from the Clinton administration all the way through the Bush administration to today. It’s such a gray area. It’s hard to say whether it’s a terrible thing or a modest change.

Has the Obama administration’s focus on open government made any difference?

Moulton: I think it has. There were a couple of agencies that got together on FOIA reform. The EPA led the team, with the U.S. National Archives and the Commerce Department, to build a new FOIA tool. The outward-facing part of the tool enables a user to go to a single spot, request and track it. Other people could come and search FOIA’ed documents. Behind the scenes, federal workers could use the tool to forward requests back and forth. This fits into what the administration has been trying to do, using technology better in government

Another example, again at the EPA, is where they’ve put together a proactive disclosure website. They got a lot of requests, like if there are inquiries about properties, environmental history, like leaks and spills, and set up a site where you could look up real estate. They did this because they went to FOIA requests and see what people wanted. That has cut down their requests to a certain percentage.

Has there been increasing FOIA demand in recent years, affecting compliance?

Moulton: I do think FOIA requests have been increasing. We’ll see what this next year of data shows. We have seen a pretty significant increase, after a significant decrease in the Bush administration. That may be because this administration keeps speaking about open government, which leads to more hopeful requestors. We fully expect that in 2013, there will be more requests than the prior year.

DHS gets the biggest number of all, but that’s not surprising when we look at the size of it. It’s second biggest agency, after Defense, and the biggest domestic facing agency. when you start talking about things like immigration and FEMA, which go deep into communities and people’s lives, in ways that have a lot impact, that makes sense.

What about the Department of Justice’s record?

Moulton: Well, DoJ got the second highest rating, but we know they have a mixed record. There are things you can’t measure and quantify, in terms of culture and attitude. I do know there were concerns about the online portal, in terms of the turf war between agencies. There were concerns about whether the tech was flexible, in terms of meeting all agency needs. If you want to build a government-wide tool, it needs to have real flexibility. The portal changed the dialogue entirely

Is FOIA performance a sufficient metric to analyze any administration’s performance on open government?

Moulton: We should step back further and look at the broader picture, if we’re going to talk about open government. This administration has done things, outside of FOIA, to try to open up records and data. They’ve built better online tools for people to get information. You have to consider all of those things.

Does that include efforts like the Intelligence Community Tumblr?

Moulton: That’s a good example. One thing this administration did early on is to identify social media outlets. We should be going there. We can’t make citizens come to us. We should go to where people are. The administration pushed early on that agencies should be able to use Tumblr and Twitter and Facebook and Flickr and so on.

Is this social media use “propaganda,” as some members of the media have suggested?

Moulton: That’s really hard to decide. I think it can result in that. It has the potential to be misused to sidestep the media, and not have good interaction with the media, which is another important outlet. People get a lot of their information from the media. Government needs to have good relationship.

I don’t think that’s the intention, though, just as under Clinton, when they started setting up websites for the first time. That’s what the Internet is for: sharing information. That’s what social media can be used for, so let’s use what’s there.

FOIA bill in the U.S. House is one of the best opportunities to institutionalize open government

b368b5d878ffa594d2_qtm6bxsz7
U.S. House unanimously voted 410-0 in favor of FOIA reform.

Unless the Congress passes legislation to codify reforms and policies proposed or promulgated under a given administration, the next President of the United States can simply revoke the executive orders and memoranda passed by his or her predecessor.

Today, almost a year after its introduction, the FOIA Oversight and Implementation Act (FOIA), H.R. 1211, will go before the U.S. House for a vote. If enacted*, it would commit the reforms to the Freedom of Information Act that the Obama administration has proposed but go further, placing the burden on agencies to justify withholding information from requestors, codifying the creation of a pilot to enable requestors to submit requests in one place, creating a FOIA Council, and directing federal agencies to automatically publish records responsive to requests online.

While these actions were proposed by the administration in its National Open Government Action Plan, Congressional action would make them permanent.

If it passed both houses of Congress and is signed into law, the FOIA Reform Act would carry into law the spirit of President Barack Obama’s Open Government Memorandum of January 21, 2009 and subsequent Open Government Directive, along with Attorney General Eric Holder’s FOIA memorandum: “The Freedom of Information Act should be administered with a clear presumption: In the face of doubt, openness prevails.”

The bipartisan bill, cosponsored by House Oversight and Government Reform Chairman Darrell Issa (R-CA.), Ranking Member Elijah Cummings (D-MD), and Representative Mike Quigley (D-IL), has received support from every major open government advocacy group in Washington, DC. The released a letter to Congress this week urging the passage of the FOIA Reform Act. The Sunshine in Government and Small Business and Entrepreneurship Council also published letters in support of the bill. It has not, however, picked up a sponsor in the Senate yet.

Currently, 97% of POPVOX users support HR1211. While the bill may not be perfect, very few pieces of legislation are.

“Requests through the Freedom of Information Act remain the principal vehicle through which the American people can access information generated by their government,” said Issa, in a statement last March. “The draft bill is designed to strengthen transparency by ensuring that legislative and executive action to improve FOIA over the past two decades is fully implemented by federal agencies.”

“This bill strengthens FOIA, our most important open government law, and makes clear that the government should operate with a presumption of openness and not one of secrecy,” said Cummings, in a statement.

Given the continued importance of the Freedom of Information Act to journalists and its relevance to holding the federal government accountable, I would urge any readers to find your Representative in Congress and urge him or her to vote for passage of the bill. Improving open government oversight through FOIA reform has been a long time coming, but change should come.

[Image Credit: CREW]