Category Archives: FOIA

Threatening legacy, Senator Jay Rockefeller stands alone holding back historic FOIA reform in the USA

Posted on by

4

15542551330_098542fd36_b

Multiple sources have confirmed that retiring U.S. Senator Jay Rockefeller (D-WV) has put a hold on the FOIA Improvement Act, the bill that would enact historic reforms to the Freedom of Information Act, the core open government law in the United States of America that gives every citizen the right to information about their government. While the senator may wish to tout his legacy of service online, if he does not release his hold by Monday, the bill will be dead, and the responsibility for that failure will lie squarely upon his shoulders.

Without naming Senator Rockefeller specifically, Senator Patrick Leahy (D-VT), the chairman of the Senate Judiciary Committee, issued the following statement into the Congressional Record last night:

The Freedom of Information Act is one of our Nation’s most important laws. James Madison said the people “must arm themselves with the power knowledge gives.” For nearly 50 years, FOIA has given Americans a way to access government information ensuring their right to know what their government doing. The FOIA Improvement Act advances this fundamental democratic principle. It is why I urge all Senators to support the FOIA Improvement Act of 2014, without delay. This legislation builds on what the President laid out in his historic executive order in 2009 by requiring Federal agencies to adopt a “Presumption of Openness” when considering the release of government information under FOIA. Prioritizing the people’s interest in what their government is doing, our bill will reduce the overuse of exemptions to withhold information where there is no foreseeable harm. It will make information available for public inspection and frequently requested documents available online. It will provide the Office of Government Information Services (OGIS), with additional independence and authority to carry out its work. I believe this legislation reaffirms the fundamental premise of FOIA, that government information belongs to all Americans. Supporting these commonsense reforms will help open the government to the 300 million Americans it serves. The bill is supported by more than 70 public interest groups that advocate for government transparency. The Sunshine in Government Initiative, said the Leahy-Cornyn bill “strengthens government transparency by limiting the ability of agencies to hide decades old documents from the public.” At the Judiciary Committee’s business meeting to consider this legislation, which was reported to the full Senate with unanimous support, Ranking Member Grassley said the FOIA Improvement Act “opens wide the curtains and provides more sunlight on the Federal government.” Senator Cornyn, my partner for many years on government transparency, noted our bipartisan efforts “to open up the government and make it more consumer and customer friendly.” I thank both Senators for their work on this legislation. We often talk about the need for government transparency, and many also note how rare it is that Democrats and Republicans can come together on any legislation. We have accomplished both with the FOIA Improvement Act. It was drafted in a bipartisan fashion after a long and thoughtful process of consultation. This week, we can pass this bill in the Senate and send it over to the House, where I am confident that it will pass, and send it to the President to sign before the end of the year. There is no reason to delay this legislation, which has broad support from a range of stakeholders, costs very little to implement and will improve access to government for all Americans. I urge the Senate to pass the FOIA Improvement Act now, without delay.

In his farewell address, Senator Rockefeller bemoaned the “hyper-partisan politics” in today’s Washington, offering memories of the Senate where compromises led to good policy.

Respectfully, Mr. Rockefeller, to opine thus and then block a bipartisan bill that passed the U.S. House of Representatives unanimously, 410-0, before it was amended, introduced by Senator Leahy and Senator John Cornyn (R-TX), and then unanimously passed by the Senate Judiciary Committee is nothing less than rank hypocrisy.

It’s hard to believe that Senator Rockefeller wants to be known for blocking open government reform at a time of historic lows in trust in government and abysmal public perceptions of the U.S, but that’s exactly what will happen if he doesn’t release this hold.

Update: on Friday night, Senator Rockefeller made the following statement on the FOIA Improvement Act, confirming the hold:

“I have a long record of support for open government and the FOIA process. I am concerned that provisions in this bill will have the unintended consequence of harming our ability to enforce the many important federal laws that protect American consumers from financial fraud and other abuses. According to experts across the federal government, these provisions would make it harder for federal agency attorneys to prepare their cases, and they would potentially give defendants new ways to obstruct and delay investigations into their conduct. I hope there is a way to address these concerns and pass the bill.”

In response, David Plazas, editorial board chair of the Tennessean, maintained that Senator Rockefeller should lift his hold.

“The bill specifically states that all records are presumed to be open unless there’s a law that would exempt it,” he wrote. “Clearly, the concerns raised in the senator’s statement should be assuaged. That House members voted unanimously for the companion bill and that a bipartisan group of senators are backing it has us questioning the real motives behind this hold.”

So do I, for the same reason. For 509 other members of Congress, these concerns were not enough to halt progress of much-needed reforms. It’s not clear which provisions the Senator is referring to, or what experts across the federal government he is referring to, because this short statement, issued at 6:30 PM on Friday after a full day of advocates, journalists and citizens asking for an explanation, doesn’t explain.

Currently, these same federal agencies are failing to comply with FOIA requests, overusing exemptions and delaying responses for years. Vague concerns about delaying investigations or harming enforcement of financial fraud, perhaps referring to actions by federal attorneys at the Federal Trade Commission or Securities and Exchange Commission, don’t hold water when balanced against the documented resistance to the public’s right to know what’s being done in their name. Raising these issues at this point in the legislative calendar very well may scuttle the bill, which would still need to go back to the House and then to the White House. If that’s the outcome, Senator Rockefeller’s “long record of support for open government and the FOIA process” will be forever cast into shadow.

Patrice McDermott, executive director of OpenTheGovernment.org, made the following statement:

“We encourage Senator Rockefeller to reconsider his hold on the bipartisan FOIA Improvement Act, S. 2520, and release the bill as soon as possible. The benefits of this critical reform bill far outweigh any nebulous concerns about unintended consequences, all of which can be addressed by the Senate as they pass the bill. S. 2520 was passed unanimously by the Judiciary Committee and is set to be taken up by the House as soon as it clears the Senate. It is also widely supported by a range of groups that cross the political spectrum and represent a wide range of interests. Senator Rockefeller shuld not remain the sole holdout that stops our ability to make the federal government more open and accountable.”

Update: On Friday, The Hill (accurately!) reported that Senator Rockefeller is being blamed for blocking the Freedom of Information reform bill. Roll Call covered the fact that Senator Rockefeller opposes the FOIA bill, advancing the story in reporting that his objections were as a surprise to Senator Leahy:

“Yesterday was the first we have heard of these concerns,” a Leahy aide said. “The FOIA Improvement Act, which was introduced in June, was approved unanimously by the Judiciary Committee on November 20. It has the support of more than 70 government transparency groups and is the result of months of consultation with the administration and a wide range of stakeholders.”

If wanted FOIA reform to address his concerns, why didn’t he raise them earlier? As always, apply Occam’s Razor: the most likely explanation is that he knew putting a hold on it this late in the legislative calendar would keep it from passing. What might motivate that action?

As reporter C.J. Ciaramella noted in his FOIA newsletter, FreedomInfo.org reported that Senator Rockefeller’s “doubts were identified by one bill supporter as being stimulated by the Federal Trade Commission, an independent agency. An advocate for the bill said the FTC was concerned about the administrative burden and judicial review of the foreseeable harm standard.”

The emerging consensus among the open government advocates in DC that I’ve talked to over the past 24 hours is that the FTC was actively lobbying against this bill, and appears to have found a receptive ear. Neither the FTC or Rockefeller’s is taking questions. As Ciaramella ironically observed, “it’s pretty cool that the federal agency and U.S. Senator who are single-handedly holding up a transparency bill that has the support of 99 other Senators won’t answer questions about it.”

Update: Josh Gerstein, writing about the Senate standoff over the FOIA bill, reported more on the concerns that led Senator Rockefeller to put the hold on it:

…sources said the agencies’ concerns are that the legislation would allow companies to pierce the attorney-client and attorney work-product privileges, potentially giving targets of enforcement actions a roadmap detailing what kind or level of misconduct will trigger action and what kind is likely to be ignored.

“The bill would statutorily require government law enforcement agencies to withhold documents from a FOIA request only if they first establish that ‘the agency reasonably foresees that disclosure would harm an interest protected by’ the exemption invoked,” said a Rockefeller aide who asked not to be named. “Consequently, the bill could expose law enforcement agencies to needless litigation and drain their already limited resources in defending FOIA decisions that have long been invoked for legitimate law enforcement purposes.”

“The chairman believes this new foreseeable harm standard would likely have a chilling effect on internal communications and deliberations and could limit internal debating on law enforcement strategy, deter agency employees from providing candid advice, and lower the overall quality of the government decision-making process – all which are absolutely vital to effective law enforcement,” the aide added.

Since news of Senator Rockefeller’s hold on the bill first broke on Friday, dozens of people on Twitter and Facebook have been asking @SenRockefeller to release the hold.

Tweets are almost certainly not going to be enough. If this is an issue that concerns you or (if you’re in the media, your listeners, readers or viewers, since FOIA applies to everyone), Senator Rockefeller’s office is in at 531 Hart Senate Office Building, Washington, DC 20510. The office number is (202) 224-6472, the fax is (202) 224-7665, and the email is senator@rockefeller.senate.gov. Unfortunately, the DC phone continues to go to voicemail and there’s no guarantee that your email will be read. Concerns citizens or reporters looking for answers can also call the senator’s satellite offices in Beckley, WV (304-253-9704), Charleston (304-347-5372), Fairmont (304-367-0122), or Martinsburg (304 262-9285) or contact other U.S. Senators and ask them about the hold.)

https://twitter.com/openGavin/status/540650243713277952

https://twitter.com/openGavin/status/540650316081795072

https://twitter.com/morisy/status/540699157795975169

This post has been updated over time.

On its 3rd anniversary, opportunities and challenges for the Open Government Partnership

Posted on by

1

opgFrance1

In 2010, President Barack Obama spoke to the United Nations General Assembly about open government. “The common thread of progress is the principle that government is accountable to its citizens,” he said, “and the diversity in this room makes clear — no one country has all the answers, but all of us must answer to our own people.”

In all parts of the world, we see the promise of innovation to make government more open and accountable.  And now, we must build on that progress.  And when we gather back here next year, we should bring specific commitments to promote transparency; to fight corruption; to energize civic engagement; to leverage new technologies so that we strengthen the foundations of freedom in our own countries, while living up to the ideals that can light the world.

Open government, said Samantha Power, now the U.S. ambassador to the United Nations, could have a global impact.

In 2011, a historic Open Government Partnership launched in New York City, hailed as a fresh approach to parting the red tape by the Economist. “The partnership is really the first time that there is a multilateral platform to address these issues,” said Maria Otero, former under secretary of state for democracy and global affairs at the United States State Department. “The partnership could have focused on countries come in and present best practices and exchange ideas and then just go home.”

“The partnership is really focused on first having countries participate that have already demonstrated interest in this area and have already put in place a number of specific things and the material laid out, if you will, the minimum standards that are being requested. What the partnership really looks for is to provide a mechanism by which the countries can each develop their own national plans on ways to expand what they’re doing on transparency, accountability, and civic engagement, or to start new initiatives for them. That is really what is very different and important about this partnership, is that it is very action- and results-oriented.”

In 2012, the Open Government Partnership became a player on the world stage as it hosted a global gathering of national leaders and civil society an annual meeting in Brazil, with the responsibilities and challenges that accompany that role, including pushing participants to submit missing action plans and progress reports, not just letters of commitment.

In January 2013, Power hailed the Open Government Partnership (OGP) as President Obama’s signature governance initiative:

It’s not about the abstraction about ‘fighting corruption’ or ‘promoting transparency’ or ‘harnessing innovation’ — it’s about ‘are the kids getting the textbooks they’re supposed to get’ or does transparency provide a window into whether resources are going where they’re supposed to go and, to the degree to which that window exists, are citizens aware and benefiting from the data and that information such that they can hold their governments accountable. And then, does the government care that citizens care that those discrepancies exist?

In May 2013, a seminal event in the evolution of OGP occurred when Russia withdrew from the Open Government Partnership:

If the dominant binary of the 21st century is between open and closed, Russia looks more interested in opting towards more controllable, technocratic options that involve discretionary data releases instead of an independent judiciary or freedom of assembly or the press. One of the challenges of the Open Government Partnership has always been the criteria that a country had to pass to join and then continue to be a member. Russia’s inclusion in OGP instantly raised eyebrows, doubts and fears last April, given rampant corruption in the public sector and Russia’s terrible record on press freedom. “Russia’s withdrawal from the OGP is an important reminder that open government isn’t easy or politically simple,” said Nathaniel Heller, executive director of Global Integrity. “While we don’t yet fully understand why Russia is leaving OGP, it’s safe to assume that the powers that be in the Kremlin decided that it was untenable to give reformers elsewhere in the Russian government the freedom to advance the open government agenda within the bureaucracy.”

In November 2013, the world may have hit ‘peak open‘ at the OGP annual summit in London, despite the partnerships’ members facing default states of closed.

Swirling underneath the professional glitz of an international summit were strong undercurrents of concern about its impact upon governments reluctant to cede power, reveal corruption or risk embarrassment upon disclosure of simple incompetence. The OGP summit took place at a moment where 21st century technology-fueled optimism has splashed up against the foundations of institutions created in the previous century. While the use of the Internet as a platform for collective action has grown, so too have attendent concerns about privacy and surveillance, in the wake of disclosures by NSA contractor Edward Snowden, where the same technologies that accelerated revolutions across the Middle East and North Africa are being used to capture and track the people advocating for change.

In 2014 the Open Government Partnership has matured and expanded, with France joining earlier in the year and Bosnia and Herzegovina bringing the total number of participating countries to 65 out of about 88 eligible countries worldwide. As OGP turns three, the partnership is celebrating the success of its expansion and looking ahead to its future, with a clearer mission and goals and ambitious four year strategy (PDF). The partnership is finally writing letters to countries that are not living up to their commitments, although the consequences for their continued participation if they do not comply remain to be seen.

The challenges and opportunities ahead for a partnership that provides a platform for civil society to hold government accountable are considerable, given the threats to civil society worldwide and the breathtaking changes brought about through technological innovation. Today, 10 national leaders will speak in New York City to mark OGP’s third anniversary. (I’ll be there to listen and share what I can.)

After the speeches end and the presidents and prime ministers return home, serious questions will remain regarding their willingness to put political capitol behind reforms and take tough stands to ensure that their governments actually open up. Digital government is not open government, just as not all open data supports democratic reforms.  As Mexico prepares to become lead co-chair of OGP, one element that didn’t make it into the challenges listed for the country is the state of press freedom in Mexico. As the Committee to Protect Journalists highlighted, open government is not sustainable without a free press. As long as the murders of journalists go unpunished in Mexico, the commitments and efforts of the Mexican national government will have to be taken in context.

Given this blog’s past stance that as press freedom goes, so too does open government, I’ve signed a petition urging the White House to explicitly support a right to report. Every other country that has committed to open government should do the same. Given OGP’s own challenges around the media and open government (PDF), I would also urge the partnership to make sure that press freedom and freedom of expression occupies a prominent place in its advocacy efforts in the years ahead.

Open government advocates: terms and conditions mean DC open data is fauxpen data

Posted on by

1

500px-WilsonbldgEarlier this summer, this blog covered the launch of District of Columbia’s executive order on open government, open data policy, open data platform and online FOIA portal. Last week, the Sunlight Foundation laid out what DC should have done differently with its open data policy.

“The evolution of open data policies since 2006 provides a chance for stakeholders to learn from and build on what’s been accomplished so far,” wrote policy associate Alisha Green. “This summer, a new executive directive from Mayor Vincent Gray’s office could have taken advantage of that opportunity for growth, It fell far short, however. The scope, level of detail, and enforceability of the policy seem to reveal a lack of seriousness about making a significant improvement on DC’s 2006 memorandum.”

Green says that DC’s robust legal, technology and advocacy community’s input should have helped shape more of the policy and that “the policy should have been passed through the legislative, not executive, process.” Opportunities, missed.

Yesterday, civic hacker and Govtrack.us founder Joshua Tauberer took the critique one step further, crying foul over the terms of use in the DC data catalog.

“The specter of a lawsuit hanging over the heads of civic hackers has a chilling effect on the creation of projects to benefit the public, even though they make use of public data released for that express purpose,” he wrote. “How does this happen? Through terms of service, terms of use, and copyright law.”

The bottom line, in Tauberer’s analysis, is that the District oF Columbia’s open data isn’t truly open. To put it another way, it’s fauxpen data.

“Giving up the right to take legal action and being required to follow extremely vague rules in order to use public data are not hallmarks of an open society,” writes Tauberer. “These terms are a threat that there will be a lawsuit, or even criminal prosecution, if civic hackers build apps that the District doesn’t approve of. It has been a long-standing tenant that open government data must be license-free in order to truly be open to use by the public. If there are capricious rules around the reuse of it, it’s not open government data. Period. Code for DC noted this specifically in our comments to the mayor last year. Data subject to terms of use isn’t open. The Mayor should update his order to direct that the city’s “open data” be made available a) without restriction and b) with an explicit dedication to the public domain.”

In the wake of these strong, constructive critiques, I posted an update in an online open government community wondering what the chances ar that DC public advocates, technologists, lawyers, wonks, librarians and citizens will go log on to the DC government’s open government platform, where the order is hosted, and suggest changes to the problematic policy? So far, few have.

The issue also hasn’t become a serious issue for the outgoing administration of Mayor Vincent Gray, or in the mayoral campaign between Muriel Bowser and David Catania, who both sit on the DC Council.

The issues section of Bowser’s website contains a positive but short, vague commitment to “improved government”: “DC needs a government that works for the people and is open to the people,” it reads. “Muriel will open our government so that DC residents have the ability to discuss their concerns and make suggestions of what we can do better.”

By way of contrast, Catania published a 128 page platform online that includes sections on “democracy for the District” and “accountable government.”(Open data advocates, take note: the document was published on Scribd, not as plaintext or HTML.) The platform includes paragraphs on improving access to government information, presenting information in user-friendly formats, eradicating corruption and rooting out wasteful spending.

Those are all worthy goals, but I wonder whether Catania knows that the city’s current policy and the executive order undermines the ability and incentives for journalists, NGOs, entrepreneurs and the District’s residents to apply the information he advocates disclosing for the purposes intended.

Last week, I asked Bowser and Catania how their administrations would approach open data in the District.

To date, I’ve heard no reply. I’ve also reached out to DC’s Office of Open Government. If I hear from any party, I’ll update this post.

Update: In answer to a question I posed, the Twitter account for DC.gov, which manages DC’s online presence and the open data platform in question as part of the Office of the Chief Technology Officer, indicated that “new terms and conditions [were] coming shortly.” No further details were offered.

The White House (quietly) asks for feedback on the open government section of its website

Posted on by

0

Obama at computer. Official White House Photo by Pete Souza

Official White House Photo by Pete Souza

Over at Govfresh, Luke Fretwell took note of the White House asking for feedback on the open government section of WhiteHouse.gov. Yesterday, Corinna Zarek, senior advisor for open government in the White House Office of Science and Technology Policy (OSTP), where the administration’s Open Government Initiative was originally spawned under former deputy chief technology officer Beth Noveck, published a email to the US Open Government Google Group:

We are working on a refresh of the Open Gov website, found at whitehouse.gov/open, and we’d like your help!

If you’re familiar with the history of the page, you can see we have begun updating it by shifting some of the existing content and adding new tabs and material.

What suggestions do you have for the site? What other efforts might we feature?

Please let us know – reply back to this thread, email us at opengov@ostp.gov, or tweet us at @OpenGov!

Here’s some background on the group and its purpose: The White House’s Open Government Working Group needs to solicit feedback from civil society in the United States on the various initiatives and commitments the administration has made. Such engagement is essential to the providing feedback from governance experts, advocates and the public on the development of new agency open government plans and discuss progress on the national open government action plan.

As a result of a discussion at the working group this spring, OSTP created the US Open Government discussion group to connect White House staff and agency officials who work on open government to people outside of the federal government. According to the group’s description, the goal of this group is to “provide a safe and welcoming arena for government-focused collaboration and news-sharing around Open Government efforts of the United States government.” That “safe and welcoming” language is notable: the group is moderated by OpenTheGovernment.org with an eye on constructive, on-topic feedback, as opposed to, say, the much more open-ended freewheeling posts and threads on the (long-since closed) Open Government Dialog of 2009 or Change.gov.

After almost six months, the open government group, which can be accessed through a Web browser or using an email listserv, has 177 members and 37 posts. By almost any measure, these are extremely low levels of participation and engagement, although the quality of feedback from those members remains extremely high. By way of contrast, a open government and civic tech group on Facebook now has over 1900 members and an open government community on Google+ has over 1400 members, with both enjoying almost daily contributions. Low participation rates on this US Open Government Google Group are likely due in part to lack of promotion by other White House staff to the media or using the various social media platforms has joined, which cumulatively have millions of followers, and, more broadly, the historic lows of public trust in government which have created icy headwinds for open government initiatives in recent years.

So far, Zarek’s solicitation has received two responses. One comes from Daniel Schuman, policy director for Citizens for Responsibility and Ethics (CREW) in Washington, who made great suggestions, like adding a link to ethics.data.gov, a list of staff working on openness in the White House and their areas of responsibility, a link to 18f and the USDS.

“Finally, there are many great ideas about how to make government more open and transparent,” wrote Schuman. “Consider including a way for people to submit ideas where those submissions are also visible to the public (assuming they do not violate TOS). Consider how agencies or the government could respond to these suggestions. Perhaps a miniature version of “We the People,” but without the voting requiring a response.”

The other idea comes from open government consultant Lucas Cioffi, who suggested adding a link to a “community-powered open government phone hotline” like the experiment he recently created.

To those ideas, I’ll add eight quick suggestions in the spirit of open government:

1) Reinstate the open government dashboard that was removed and update it to the current state of affairs and compliance, with links to each. The Sunlight Foundation and CREW have already audited agency compliance with the Open Government Directive. By keeping an updated scorecard in a prominent place, the Obama administration could both increase transparency to members of the public wondering about what has been done and by whom, and put more pressure on agencies to be accountable for the commitments they have made.

2) Re-integrate individual case studies from the “Innovator’s Toolkit,” which was also removed, under participation and collaboration

3) Create a Transparency tab and link to the “IC on the Record” tumblr and other public repositories for formerly secret laws, policies or documents that have been released.

4) Blog and tweet more about what’s happening in the open government world outside of the White House. Multiple open government advocates do daily digests and there’s a steady stream of news and ideas on the #opengov and #opendata hashtags on Twitter. Link to what’s happening and show the public that you’re reading and responding to feedback.

5) Link to the White House account and open government projects on Github under both the new participation and collaboration tabs, like Project Open Data.

6) Highlight 18F’s effort to reboot the Freedom of Information Act.

7) Publish the second national action plan on open government as HTML on the site, and post and link to a version on Github where people can comment on it.

8)  Create a FAQ under “participation” that lists replies to questions sent to @OpenGov

If you have ideas for what should be wh.gov/open, well, now you know who to tell, and where.

18F launches alpha foia.gov in a bid to reboot Freedom of Information Act requests for the 21st century

Posted on by

4

alpha foia gov

18F, the federal government’s new IT development shop, has launched a new look at the Freedom of Information Act (FOIA) in the form of a open source application hosted on Github. Today’s announcement is the most substantive evidence yet that the Obama administration will indeed modernize the Freedom of Information Act, as the United States committed to doing in its second National Action Plan on Open Government. Given how poor some of the “FOIA portals” and underlying software that supports them exists is at all level of government, this is tremendous news for anyone that cares about the use of technology to support open government.

Notably, 18F already has a prototype (pictured above) online that shows what a consolidated request submission hub could look like and plans to iterate upon it.  This is a perfect example of “lean government,” or the application of lean startup principles and agile development to the creation of citizen-centric services in the public sector.  Demonstrating its commitment to developing free and open source software in the open, 18F asked the public to follow the process online at their FOIA software repository on Github, send them feedback or even contribute to the project.

18F has now committed to creating software that improvse how requests made under the Freedom of Information Act can be improved through technology. Specifically that it will develop tools that “improve the FOIA request submission experience,” “create a scalable infrastructure for making requests to federal agencies” and “make it easier for requesters to find records and other information that have already been made available online.”

According to 18F’s blog post, this work is supported and overseen by a “FOIA Task Force,” consisting of representatives from the Department of Justice, Environmental Protection Agency, the Office of Management and Budget, the Office of Science and Technology Policy. The task force will need to focus upon more than technology: while poor software has hindered requests and publishing, that’s not the primary issue that’s hindering the speed or quality of responses.

Despite the U.S. attorney general’s laudable commitment to a new era of open government in 2009, the Obama administration received a .91 GPA in FOIA compliance earlier this year from the Center for Effective Government.

While White House press secretary Josh Earnest may be well correct in stating that the federal government is processing more FOIA requests than ever, As the National Security Archive noted in March, the use of a FOIA exemption (protecting “deliberative processes”) to deny or heavily redact requests has skyrocketed in the past two years.

use of B5 exemptions

[NATIONAL SECURITY ARCHIVE: Chart created by Lauren Harper.]

As with the reduced access to government staff and scientists that a group of 38 journalism and open government advocates decried earlier this year, improving FOIA compliance cannot solely be addressed through technological means. To address endemic government secrecy and outright abuse of exemptions to protect against politically inconvenient disclosures, Obama administration — in particular, the U.S. Justice Department — will need to expend political capital and push agencies to actually shift the cultural default towards openness and release uncomfortable or embarrassing data and documents and not redact them beyond understanding.

That’s admittedly a huge challenge, particularly for an administration facing multiple foreign and domestic conundrums, including a scandal over missing IRS emails and obfuscated records in an election year and the most politically polarized Congress and electorate in the nation’s history, but if President Barack Obama is truly committed to “creating an unprecedented level of openness in government,” it’s one that he and his administration will need to take on.

Data journalism and the changing landscape for policy making in the age of networked transparency

Posted on by

0

This morning, I gave a short talk on data journalism and the changing landscape for policy making in the age of networked transparency at the Woodrow Wilson Center in DC, hosted by the Commons Lab.

Video from the event is online at the Wilson Center website. Unfortunately, I found that I didn’t edit my presentation down enough for my allotted time. I made it to slide 84 of 98 in 20 minutes and had to skip the 14 predictions and recommendations section. While many of the themes I describe in those 14 slides came out during the roundtable question and answer period, they’re worth resharing here, in the presentation I’ve embedded below:

DC city government issues executive order on open data, FOIA portal and chief data officer

Posted on by

6

Today, the District of Columbia launched a new online service for Freedom of Information Act requests and Mayor Vincent Gray issued a transparency, open government and open data directive. DC city government has come under harsh criticism from the ACLU for its record on FOIA and transparency and has a spate of recent corruption scandals, albeit not one that appears to be worse than other major American cities.

“This new online FOIA system is a key part of our strategy to improve government transparency and accountability,” said Mayor Gray, in a statement. “In addition, the executive order I am issuing today sends an important message to District government agencies and the public: Everyone wins when we make it easier for the public to understand the workings of the District government. I also look forward to seeing the exciting applications I hope the District’s technology community will develop with the government data we will be putting online.”

Here’s what Mayor Gray has instructed DC government to do:

1) Within 30 days from today, the DC chief technology officer (currently Rob Mancini) must create “a common Web portal” that “will serve as the source for District-wide and agency activities related to this Transparency and Open Data Directive.” Translation: OCTO must create a new website that aggregates information related to this directive.

2) OCTO will publish technical standards for open data by November 1, 2014. DC government could refer to the Sunlight Foundation’s Open Data Guidelines as a useful reference, or the canonical 8 principles of Open Government Data.

3) Within 120 days from today, the DC City Administrator and each deputy mayor must identify at least 3 new high-value datasets to publish to the DC Data Catalog that are either not currently available or not available in an exportable format.

4) Starting on October 1, 2014, and continuing annually, each DC agency will develop and publish an “Open Government Report” that will “describe how the agency has or will enhance and develop transparency, public participation, and collaboration. Each agency shall include in its open government report a description of the information (including data) that will be made available to the public, formats in which information and data will be made.”

Translation: city agencies will report on how they’re doing complying with this mandate. Hopefully, the DC Office of Open Government will be an effective ombudsman on that progress, along with directly engaging on Freedom of Information Act disputes and processes, and will do more public engagement around open government or open data than @OCTONEWS has to date.

Unfortunately, and not a little bit ironically, the directive was published online as a scanned-in PDF that is neither searchable nor accessible to the blind, itself embodying the way not to release text online in the 21st century. Below, I have summarized the main deliverables mandated in the directive and converted the images to plain text. Following the order is criticism from open government advocate, civic hacker, and DC resident Josh Tauberer.

GOVERNMENT OF THE DISTRICT OF COLUMBIA

ADMINISTRATIVE ISSUANCE SYSTEM

Mayor’s Order 2014-170
July 21, 2014

SUBJECT: Transparency, Open Government and Open Data Directive

ORIGINATING AGENCY: Office of the Mayor

By virtue of the authority vested in me as Mayor of the District of Columbia by section 422(2) and (11) of the District of Columbia Home Rule Act, approved December 24, 1973, 87 Stat. 790, Pub. L. No. 93-198, D.C. Official Code § 1-204.22(2) and (11) (2012 Repl.), and section 206 of the District of Columbia Freedom of Information Act, effective March 25, 1977, D.C. Law 1-96, D.C. Official Code § 2-536 (2012 Repl.), it is hereby ORDERED that:

SECTION 1: Introduction.

a. Background. The District of Columbia government (“District”) is committed to creating an unprecedented level of openness in government. Agency heads will work together and with the public to ensure public trust, and an open and effective government by establishing a system of transparency, public participation, collaboration, and accountability that increases the public’s confidence in their government. The goal of this directive is to provide a tool for prescribing and institutionalizing change within all departments and agencies.

The District has been a leader in government transparency and open data policy in the United States. In 2001, the Freedom of Information Act was amended to require that certain public records be published online. Since 2006, the District has been making data publicly available on the Internet. In January 2011, Mayor’s Memorandum 2011-1, entitled Transparency and Open Government Policy, was issued, recognizing that the District government needed to continue to proactively provide information to citizens, and thereby reduce the need for information requests. This directive implements Mayor’s Memorandum 2011-1, to require District government departments and agencies to take the following
steps to achieve the goal of creating a more transparent and open government:

b. Definitions.

  1. “Chief Data Officer” (“CDO”) means the Chief Technology Officer or a Chief Data Officer designated by the Chief Technology Officer.
  2.  “Data” means statistical, or factual, quantitative, or qualitative information that are regularly maintained or created by or on behalf of a District agency, and controlled by such agency in structured formats, including statistical or factual information about image files and geographic information system data.
  3. “Dataset” means a named collection of related records, with the collection containing data organized or formatted in a specific or prescribed way, often in tabular form.
  4. “Open Government Coordinator” means agency personnel designated by an agency head, in coordination with the Office of the Chief Technology Officer (“OCTO”) or the CDO as appropriate, to ensure that the information and data required to be published online is published and updated as required by this Order.
  5. “Protected data” means (i) any dataset or portion thereof to which an agency may deny access pursuant to the District of Columbia Freedom of Information Act, effective March 25, 1977 (D.C. Law 1-96; D.C. Official Code § 2-531 et seq.)(“FOIA”), or any other law or rule or regulation; (ii) any dataset that contains a significant amount of data to which an agency may deny access pursuant to FOIA or any other law or rule or regulation promulgated thereunder, if the removal of such protected data from the dataset would impose an undue financial or administrative burden on the agency; or (iii) any data which, if disclosed on the District of Columbia Data Catalog, could raise privacy, confidentiality or security concerns or jeopardize or have the potential to jeopardize public health, safety or welfare.

C. Scope.

a. The requirements of this Order shall be applied to any District of Columbia department, office, administrative unit, commission, board, advisory committee or other division of the District government (“agency”), including the records of third party agency contractors that create or acquire information, records, or data on behalf of a District agency.

b. Any agency that is not subject to the jurisdiction of the Mayor under the Freedom of Information Act or any other law is strongly encouraged to comply with the requirements of this Order.

SECTION 2: Transparency and Open Government Policy.

a. Publish Government Information Online. To increase accountability and transparency, promote informed public participation, and create economic development opportunities, each District agency shall expand access to information by making it proactively available online, and when practicable, in an open format that can be retrieved, downloaded, indexed, sorted, searched, and reused by commonly used Web search applications and commonly used software to facilitate access to and reuse of information. Examples of open format include HTML, XML, CSV, JSON, RDF or XHTML. The Freedom of Information Act creates a presumption in favor of openness and publication (to the extent permitted by law and subject to valid privacy, confidentiality, security, or other restrictions).

b. Open Government Web Portal: Within 30 days from the date of this Order, the Chief Technology Officer shall establish a common web portal that will serve as the source for District-wide and agency activities related to this Transparency and Open Data Directive. The Chief Technology Officer, in his or her discretion, may build upon an existing web portal, or may establish a new portal. Each agency shall be responsible for ensuring that the information required to be published online is accessible from the agency’s designated Open Government and FOIA webpage. The required information shall include, but is not limited to, where applicable:

  1. Means for the public to submit and track Freedom of Information Act requests online;
  2. The information required to be made public under this Directive and D.C. Official Code § 2-536, including links to:
    A. Employee salary information;
    B. Administrative staff manuals and instructions that affect the public;
    C. Final opinions and orders made in the adjudication of cases;
    D. Statements of policy, interpretations of policy, and rules adopted by the agency;
    E. Correspondence and other materials relating to agency regulatory, supervisory or enforcement responsibilities in which the rights of the public are determined;
    F. Information dealing with the receipt or expenditure of public or other funds;
    G. Budget information;
    H. Minutes of public meetings;
    I. Absentee real property owners and their agent’s names and mailing addresses;
    J. Pending and authorized building permits;
    K. Frequently requested public records; and
    L. An index to the records referred to in this section;
  3. Freedom of Information Act reports;
  4. An organizational chart or statement of the agency’s major components;
  5. Links to high-value datasets (as defined in section 3(a)(4);
  6. Public Meeting Notices and minutes required to be published under the Open Meetings Act and Freedom of Information Act; and
  7. A mechanism for the public to submit feedback on the agency’s Open Government Report or other agency actions.

c. Open Government Report. To institutionalize a culture of transparent and open government, accountability, and to expand opportunities for resident participation and collaboration, beginning October 1, 2014, and each year thereafter, each agency shall develop and publish an Open Government Report that will describe how the agency has or will enhance and develop transparency, public participation, and collaboration. Each agency shall include in its open government report a description of the information (including data) that will be made available to the public, formats in which information and data will be made available, a schedule for making the information available, the dates for which information and datasets will be updated, and contact information for agency Open Government Coordinators. The Open Government Report shall address the following topics, and be transmitted to the Mayor and Director of the Office of Open Government:

  1. Transparency: The Open Government Report shall reference statutes, regulations, policies, legislative records, budget information, geographic data, crime statistics, public health statistics, and other public records and data, and describe steps each agency has taken or will take to:A. Meet its legal information dissemination obligations under Freedom of Information Act and Open Meetings Act;
    B. Create more access to information and opportunities for public participation; and
    C. Conduct its work more openly and publish its information online, including a plan for how each board and commission subject to the Open Meetings Act will ensure that all of its meetings are, where practicable, webcast live on the Internet.
  2. Participation: To create more informed and effective policies, each agency shall enhance and expand opportunities for the public to participate throughout agency decision-making processes. The Open Government Report will include descriptions of or plans to provide:A. Online access to proposed rules and regulations;
    B. Online access to information and resources to keep the public properly informed (such as frequently asked questions, contact information of city officials’ and departments, and other supportive content);
    C. Opportunities for the public to comment through the Web on any proposed rule, ordinance, or other regulation;
    D. Methods of identifying stakeholders and other affected parties and inviting their participation;
    E. Proposed changes to internal management and administrative policies to improve participation;
    F. Links to appropriate websites where the public can engage in the District government’s existing participatory processes;
    G. Proposals for new feedback mechanisms, including innovative tools and practices that create new and more accessible methods for public participation; and
    H. A plan that provides a timetable for ensuring that all meetings of boardsand commissions that are subject to the Open Meetings Act are webcast live and archived on the Internet.
  3. Collaboration: The Open Government Report will describe steps the agency will take or has taken to enhance and expand its practices to further cooperation among departments, other governmental agencies, the public, and non-profit and private entities in fulfilling its obligations. The Report will include specific details about:A. Proposed changes to internal management and administrative policies to improve collaboration;
    B. Proposals to use technology platforms to improve collaboration among District employees and the public;
    C. Descriptions of and links to appropriate websites where the public can learn about existing collaboration efforts; and
    D. Innovative methods, such as prizes and competitions, to obtain ideas from and to increase collaboration with those in the private sector, non-profit, and academic communities.

SECTION 3: Open Data Policy.

a. Agency Requirements.

  1. Each agency shall, in collaboration with the Chief Data Officer and OCTO, make available through the online District of Columbia Data Catalog all appropriate datasets, associated extensible metadata, and associated documented agency business processes under the agency’s control. Each agency, in collaboration with OCTO, shall determine the frequency for updates to a dataset, and the mechanism to be utilized. To the extent possible, datasets shall be updated through an automated process to limit the additional burden on agency resources. The publication of an agency’s datasets shall exclude protected data.
  2. Datasets under paragraph (4) shall be made available in accordance with technical standards published by OCTO not later than November 1, 2014 that ensure that data is published in a format that is machine readable, and fully accessible to the broadest range of users, for varying purposes. Datasets shall be made available to the public on an open license basis. An open license on a dataset signifies there are no restrictions on copying, publishing, further distributing, modifying or using the data for a non-commercial or commercial purpose.
  3. For the purposes of identifying datasets for inclusion on the District of Columbia Data Catalog, each agency shall consider whether the information embodied in the dataset is (i) reliable and accurate; (ii) frequently the subject of a written request for public records of the type that a public body is required to make available for inspection or copying under FOIA; (iii) increases agency accountability, efficiency, responsiveness or delivery of services; (iv) improves public knowledge of the agency and its operations; (v) furthers the mission of the agency; or (vi) creates economic opportunity.
  4. Within 120 days of the date of this Order, the City Administrator and each Deputy Mayor shall, collaborating with their cluster agencies, and OCTO, identify at least 3 new high-value datasets to publish to the Data Catalog, in accordance with OCTO’s open data standards. The identified high-value datasets will not be currently available, or not available in an exportable format. For the purposes of this section, “high-value dataset” includes agency outcome data, agency caseload data, data reported to the federal government outcome data, agency caseload data, data reported to the federal government by the agency, agency data reported as part of the performance measurement process, and any data that is tracked by the agency that is not protected data.

b. Chief Data Officer.

  1. The Chief Technology Officer shall designate a Chief Data Officer (“CDO”) for the District of Columbia to coordinate implementation, compliance and expansion of the District’s Open Data Program, to facilitate the sharing of information between departments and agencies, and to coordinate initiatives to improve decision making and management through data analysis. The Chief improve decision making and management through data analysis. The Chief Data Officer shall report to the Chief Technology Officer.
  2. The Chief Data Officer shall:
    A. Identify points of contact, which may include agency open government coordinators within departments, on data related issues who will be responsible for leading intra-departmental open data initiatives;
    B. Emphasize the culture behind open data and the benefits to ensure that opportunities to increase efficiency through open data practices can be obtained from those with the most direct expertise;
    C. Work together with District agencies to develop a methodology and framework that supports the collection, or creation of data in a way that assists in downstream data processing and open data distribution activities;
    D. Identify and overcome challenges with agency proprietary business systems; create and/or leverage opportunities through procurement or other means to upgrade legacy systems to one of an open data architecture; and
    E. Function as a data ombudsman for the public, fielding public feedback and ensuring the policy is included into a long-term data strategy.

c. District of Columbia Open Data Catalog.

  1. A single web portal, or integrated set of websites, shall be established and maintained by or on behalf of the District of Columbia. The Chief Data maintained by or on behalf of the District of Columbia. The Chief Data Officer, in collaboration with OCTO, may build upon previous open data initiatives, or may establish a new portal for managing and delivering open data benefits to constituents.
  2. Any dataset made accessible on the District of Columbia Data Catalog shall use an open format that permits automated processing of such data in a form that can be retrieved via an open application programming interface (API), downloaded, indexed, searched and reused by commonly used web search applications and software; (ii) use appropriate technology to notify the public of updates to the data; and (iii) be accessible to external search capabilities.
  3. OCTO shall (i) post on the portal a list of all datasets available on such portal; and (ii) establish and maintain on the portal an online forum to solicit feedback from the public and to encourage public discussion on open data policies and dataset availability.

d. Open Data Legal Policy.

  1. The District of Columbia Data Catalog and all public data contained on such portal shall be subject to Terms of Use developed by OCTO. Such Terms of Use shall be posted by OCTO in a conspicuous place on the District ofColumbia Data Catalog.
  2. Public data made available on the District of Columbia Data Catalog shall be provided as a public service, on an “as is” basis. Although the District will strive to ensure that such public data are accurate, the District shall make no warranty, representation or guaranty of any type as to the content, accuracy, timeliness, completeness or fitness for any particular purpose or use of any public data provided on such portal; nor shall any such warranty be implied, including, without limitation, the implied warranties of merchantability and fitness for a particular purpose. The District shall assume no liability for any other act identified in any disclaimer of liability or indemnification provision or any other provision set forth in the Terms of Use required under subsection (d)(1) of this section.
  3. The District shall reserve the right to discontinue availability of content on the District of Columbia Data Catalog at any time and for any reason. If a dataset is made accessible by an agency on the District of Columbia Data Catalog and such agency is notified or otherwise learns that any dataset or portion thereof posted on the Data Catalog is factually inaccurate or misleading or is protected data, the agency shall, as appropriate, promptly correct or remove, or cause to be corrected or removed, such data from the Data Catalog and shall so inform the Chief Data Officer.
  4. Nothing in this Order shall be deemed to prohibit OCTO or any agency or any third party that establishes or maintains the District of Columbia Data Catalog on behalf of the District from adopting or implementing measures necessary or appropriate to (1) ensure access to public datasets housed on the Data Catalog; (ii) protect the Data Catalog from unlawful use or from attempts to impair or damage the use of the portal; (iii) analyze the types of public data on the Data Catalog being used by the public in order to improve service delivery or for any other lawful purpose; (iv) terminate any and all display, distribution or other use of any or all of the public data provided on the Data Catalog for violation of any of the Terms of Use posted on the Data Catalog pursuant to subsection (d)(1) of this section; or (v) require a third party providing the District’s public data (or applications based on public data) to the public to explicitly identify the source and version of the public dataset, and describe any modifications made to the public dataset.
  5. Nothing in this Order shall be construed to create a private right of action to enforce any provision of this Order. Failure to comply with any provision of this Order shall not result in any liability to the District, including, but not limited to, OCTO or any agency or third party that establishes or maintains on behalf of the District the Open Data Services Portal required under this Order.

Section 4. Open Government Advisory Group.

a. The Mayor shall convene an Open Government Advisory Group to be chaired and convened by the Mayor’s designee, CDO, and the Director of the Office of Open Government within the Board of Ethics and Government Accountability.

b. The Open Government Advisory Group shall:

  1. Evaluate the District’s progress towards meeting the requirements of this Order and make specific recommendations for improvement; and
  2. Assist the Mayor and CDO in creating policy establishing specific criteria for agency identification of protected data in accordance with FOIA, maintenance of existing data, and the creation of data in open formats.

c. The CDO shall publish the evaluation and recommendations on the Open Government Web Portal or create an Open Government Dashboard that will provide the public with both graphic and narrative evaluation information.

Section 5: EFFECTIVE DATE:

This Order shall be effective immediately.

VINCENT C. GRAY
MAYOR

ATTEST:
CYNTHIA BR CIS-SMITH
SECRETARY OF THE DISTRICT OF COLUMBIA

After the order was published online, GovTrack.us founder Josh Tauberer issued a series of critical tweets and extended his thoughts into a blog post, holding that DC city government adopted the mistakes made by the White House:

There is a strong American tradition — or at least a core American value — that the government does not get in the way of the dissemination of ideas. We don’t always live up to that ideal, but we strive for it. Access to information about the government that comes with restrictions on what we can say when we use it (e.g. attribution & explanation), a waiver of rights or a commitment to indemnify, etc. are all an anathema to accountability and transparency and respect for the public.

[REPORT] On data journalism, democracy, open government and press freedom

Posted on by

1

On May 30, I gave a keynote talk on my research on the art and science of data journalism at the first Tow Center research conference at Columbia Journalism School in New York City. I’ve embedded the video below:

My presentation is embedded below, if you want to follow along or visit the sites and services I described.

Here’s an observation drawn from an extensive section on open government that should be of interest to readers of this blog:

“Proactive, selective open data initiatives by government focused on services that are not balanced by support for press freedoms and improved access can fairly be criticized as “openwashing” or “fauxpen government.”

Data journalists who are frequently faced with heavily redacted document releases or reams of blurry PDFs are particularly well placed to make those critiques.”

My contribution was only one part of the proceedings for “Quantifying Journalism: Metrics, Data and Computation,” which you can catch up through the Tow Center’s live blog or TechPresident’s coverage of measuring the impact of journalism.

Esri’s new ArcGIS feature is live. Will terabytes of new open data follow?

Posted on by

0

esri-open-data

Back in February, I reported that Esri would enable governments to open their data to the public.Today, the geographic information systems (GIS) software giant pushed ArcGIS Open Data live, instantly enabling thousands of its local, state and federal government users to open up the public data in their systems to the public, in just a few minutes.

open-data-esri

“Starting today any ArcGIS Online organization can enable open data, specify open data groups and create and publicize their open data through a simple, hosted and best practices web application,” wrote Andrew Turner, chief technology officer of Esri’s Research and Development Center in D.C., in a blog post about the public beta of Open Data ArcGIS. “Originally previewed at FedGIS ArcGIS Open Data is now public beta where we will be working with the community on feedback, ideas, improvements and integrations to ensure that it exemplifies the opportunity of true open sharing of data.”

Turner highlighted what this would mean for both sides of the open data equation: supply and demand.

Data providers can create open data groups within their organizations, designating data to be open for download and re-use, hosting the data on the ArcGIS site. They can also create public microsites for the public to explore. (Example below.) Turner also highlighted the code for Esri’s open-source GeoPortal Server on Github as a means to add metadata to data sets.

Data users, from media to developers to nonprofits to schools to businesses to other government entities, will be able to download data in common open formats, including KML, Spreadsheet (CSV), Shapefile, GeoJSON and GeoServices.

“As the US Open Data Institute recently noted, [imagine] the impact to opening government data if software had ‘Export as JSON’ by default,” wrote Turner.

“That’s what you now have. Users can also subscribe to the RSS feed of updates and comments about any dataset in order to keep up with new releases or relevant supporting information. As many of you are likely aware, the reality of these two perspectives are not far apart. It is often easiest for organizations to collaborate with one another by sharing data to the public. In government, making data openly available means departments within the organization can also easily find and access this data just as much as public users can.”

EmploymentLaborForces_B230251

Turner highlighted what an open data site would look like in the wild:

“Data Driven Detroit a great example of organizations sharing data. They were able to leverage their existing data to quickly publish open data such as census, education or housing. As someone who lived near Detroit, I can attest to the particular local love and passion the people have for their city and state – and how open data empowers citizens and businesses to be part of the solution to local issues.

In sum, this feature could, as I noted in February, could mean a lot more data is suddenly available for re-use. When considered in concert with Esri’s involvement in the White House’s Climate Data initiative, 2014 looks set to be a historic year for the mapping giant.

It also could be a banner year for open data in general, if governments follow through on their promises to release more of it in reusable forms. By making it easy to upload data, hosting it for free and publishing it in the open formats developers commonly use in 2014, Esri is removing three major roadblocks governments face after a mandate to “open up” come from a legislature, city council, or executive order from the governor or mayor’s office.

“The processes in use to publish open data are unreasonably complicated,” said Waldo Jacquith, director of the U.S. Open Data Institute, in an email. 

“As technologist Dave Guarino recently wrote, basically inherent to the process of opening data is ETL: “extract-transform-load” operations. This means creating a lot of fragile, custom code, and the prospect of doing that for every dataset housed by every federal agency, 50 states, and 90,000 local governments is wildly impractical.

Esri is blazing the trail to the sustainable way to open data, which is to open it up where it’s already housed as closed data. When opening data is as simple as toggling an “open/closed” selector, there’s going to be a lot more of it. (To be fair, there are many types of data that contain personally identifiable information, sensitive information, etc. The mere flipping of a switch doesn’t address those problems.)

Esri is a gold mine of geodata, and the prospect of even a small percentage of that being released as open data is very exciting.”