USASpending.gov addresses some data issues, adds Github issues tracker for feedback

usaspending

On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.

A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.

Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.

In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:

The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.

This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.

Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.

Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.

Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.

If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.

[Image Credit: Govfresh, via USASpending.gov]

Hillary Clinton hires Google product manager for civic innovation and social impact as campaign CTO

In hiring Stephanie Hannon to be her campaign’s chief technology officer, former Secretary of State Hillary Clinton is not just bringing on a former Cisco software engineer with product management experience at Google and Facebook and time in the trenches at tech startups: she’s added a woman who’s deeply immersed in the civic technology movement and knowledgeable about open data. Just watch her talk at the 2014 Code for America Summit:

So, here’s the bad news: Hannon was a product manager for Gmail and Google Wave, so steel yourself for a lot of bad jokes in the days and months ahead in the media, given her new boss’ questionable choices about email as Secretary of State and Google Wave’s demise. While a B.A. and an M.S. in electrical engineering from Stanford and an MBA from Harvard will insulate her from the some of the lamer slings and arrows, get ready for unsubtle, tortured metaphors and misogyny in the comment sections.

Here’s the mixed news: Hannon’s focus on open data on Google appears to have been on standards, services and civic impact, not accountability and transparency. Take a look at her presentation, embedded below:

Given the record of the Obama administration, it remains to be seen whether Clinton will proactively adopt an ambitious agenda on open government if elected, from implementing and resource FOIA reforms, whistleblower support or nominating inspector generals for all federal agencies.

And here’s the good news: while political observers will (and should!) no doubt focus upon her ability to duplicate the success of Harper Reed, the CTO for Barack Obama’s 2012 presidential campaign, Clinton just brought someone who’s tapped into what’s happening in the civic tech space into orbit. If that experience elevates those issues into the campaign and national conversation, it will stand to benefit everyone working towards improving civic life in America. If Clinton starts talking about “building better governance with the people, not for them” on the campaign trail, you’ll know something important has occurred.

FOIA reform faces higher odds in 114th Congress

Federal financial regulators and the industry that they regulate are fretting over Freedom of Informatiom Act reform in Congress, per The Hill.

At least the concerns about sensitive info are being aired in public this time, albeit not on the record: the regulators aren’t commenting, and neither is industry. (It was their lobbying that scuttled FOIA reform become law last December, despite bills passing both houses of Congress unanimously.) 

Behind this story is a deeper one about how power and influence are used. The odds against strong FOIA reform being passed in the 114th Congress look longer today. 

Obama names top Facebook engineer director of White House IT, creates Presidential IT Committee

Davidrecordon

In its search for technology talent, the White House has been recruiting heavily from Google of late, including U.S. chief technology officer Megan Smith. Today, President Barack Obama showed that his administration also likes Facebook, announcing that engineer David Recordon would upgrade the White House’s technology infrastructure. The news was first reported by Yahoo.

“In our continued efforts to serve our citizens better, we’re bringing in top tech leaders to support our teams across the federal government,” said President Obama, in a statement. “Today, I’m pleased to welcome David Recordon as the Director of White House Information Technology. His considerable private sector experience and ability to deploy the latest collaborative and communication technologies will be a great asset to our work on behalf of the American people.”

On the one hand, it’s terrific to see The White House attract top tech talent. Getting David Recordon into public service should be a win for the American people. Based upon a somewhat cryptic hint he posted on Facebook last August, it appeared that he was involved in helping to fix Heathcare.gov and another unnamed important project. The blog post that went up at WhiteHouse.gov confirmed that Recordon was “one of those engineers.” Bringing the best engineers the administration can find into the U.S. Digital Service will help the nation avoid more IT catastrophes, and Recordon, a notable open standard advocate who helped develop OpenID, is clearly one of them.  That’s good news.

On the other hand, while being the first “Director of White House Information Technology” is clearly great copy for the tech press, working to “ensure that the technology utilized by the White House is efficient, effective, and secure” sounds more or less what the White House chief information officer should be — and has been – doing for years.

Just look at the responsibilities for the Office of the CIOPer Federal News Radio, the White House CIO for the past two years, Karen Britton, stepped down in January 2015, without any announced replacement since. Michael Hornsby, the director of engineering and operations within OCIO, served as acting CIO. This all leads me to hypothesize that Recordon has effectively been named the new White House CIO but doesn’t have that title.

Regardless, here’s hoping Recordon’s considerable expertise leads to improvements in an information technology infrastructure that has come a long way since 2009 (read this) but still lags the private sector.

President Obama signed an official presidential memorandum today creating the role and establishing an “Executive Committee for Presidential Information Technology” made up of the “Assistant to the president for Management and Administration, the Executive Secretary of the National Security Council, the Director of the Office of Administration, the Director of the United States Secret Service, and the Director of the White House Military Office.”

According to the memorandum, which is embedded beneath and reproduced in plaintext below (it’s not online at WhiteHouse.gov yet), this committee will “shall advise and make policy recommendations to the Deputy Chief of Staff for Operations and the Director with respect to operational and procurement decisions necessary to achieve secure, seamless, reliable, and integrated information resources and information systems for the President, Vice President, and EOP.”

In other words, these folks will advise the director on how to by, build and run tech for the White House.

Presidential Memorandum White House IT:

https://www.scribd.com/embeds/259313174/content?start_page=1&view_mode=scroll&show_recommendations=true

[Photo Credit: Brian Solis]

THE WHITE HOUSE
Office of the Press Secretary

For Immediate Release March 19, 2015
March 19, 2015
MEMORANDUM FOR THE SECRETARY OF DEFENSE
THE SECRETARY OF HOMELAND SECURITY
THE DIRECTOR OF THE OFFICE OF MANAGEMENT AND
BUDGET
THE NATIONAL SECURITY ADVISOR
THE DIRECTOR OF THE OFFICE OF ADMINISTRATION
SUBJECT: Establishing the Director of White House
Information Technology and the Executive
Committee for Presidential Information Technology
By the authority vested in me as President by the Constitution
and the laws of the United States of America, and in order to
improve the information resources and information systems
provided to the President, Vice President, and Executive Office
of the President (EOP), I hereby direct the following:
Section 1. Policy. The purposes of this memorandum are to
ensure that the information resources and information systems
provided to the President, Vice President, and EOP are
efficient, secure, and resilient; establish a model for
Government information technology management efforts; reduce
operating costs through the elimination of duplication and
overlapping services; and accomplish the goal of converging
disparate information resources and information systems for the
EOP.
This memorandum is intended to maintain the President’s
exclusive control of the information resources and information
systems provided to the President, Vice President, and EOP.
High-quality, efficient, interoperable, and safe information
systems and information resources are required in order for the
President to discharge the duties of his office with the support
of those who advise and assist him, and with the additional
assistance of all EOP components. The responsibilities that
this memorandum vests in the Director of White House Information
Technology, as described below, have been performed historically
within the EOP, and it is the intent of this memorandum to
continue this practice.
The Director of White House Information Technology, on
behalf of the President, shall have the primary authority to
establish and coordinate the necessary policies and procedures
for operating and maintaining the information resources and
information systems provided to the President, Vice President,
and EOP. Nothing in this memorandum may be construed to
delegate the ownership, or any rights associated with ownership, 2
of any information resources or information systems, nor of any
record, to any entity outside of the EOP.
Sec. 2. Director of White House Information Technology.
(a) There is hereby established the Director of White House
Information Technology (Director). The Director shall be the
senior officer responsible for the information resources and
information systems provided to the President, Vice President,
and EOP by the Presidential Information Technology Community
(Community). The Director shall:
(i) be designated by the President;
(ii) have the rank and status of a commissioned
officer in the White House Office; and
(iii) have sufficient seniority, education, training,
and expertise to provide the necessary advice,
coordination, and guidance to the Community.
(b) The Deputy Chief of Staff for Operations shall provide
the Director with necessary direction and supervision.
(c) The Director shall ensure the effective use of
information resources and information systems provided to the
President, Vice President, and EOP in order to improve mission
performance, and shall have the appropriate authority to
promulgate all necessary procedures and rules governing these
resources and systems. The Director shall provide policy
coordination and guidance for, and periodically review, all
activities relating to the information resources and information
systems provided to the President, Vice President, and EOP by
the Community, including expenditures for, and procurement of,
information resources and information systems by the Community.
Such activities shall be subject to the Director’s coordination,
guidance, and review in order to ensure consistency with the
Director’s strategy and to strengthen the quality of the
Community’s decisions through integrated analysis, planning,
budgeting, and evaluation processes.
(d) The Director may advise and confer with appropriate
executive departments and agencies, individuals, and other
entities as necessary to perform the Director’s duties under
this memorandum.
Sec. 3. Executive Committee for Presidential Information
Technology. There is hereby established an Executive Committee
for Presidential Information Technology (Committee). The
Committee consists of the following officials or their
designees: the Assistant to the President for Management and
Administration; the Executive Secretary of the National Security
Council; the Director of the Office of Administration; the
Director of the United States Secret Service; and the Director
of the White House Military Office.
Sec. 4. Administration. (a) The President or the Deputy
Chief of Staff for Operations may assign the Director and the
Committee any additional functions necessary to advance the
mission set forth in this memorandum.
(b) The Committee shall advise and make policy
recommendations to the Deputy Chief of Staff for Operations and
the Director with respect to operational and procurement 3
decisions necessary to achieve secure, seamless, reliable, and
integrated information resources and information systems for the
President, Vice President, and EOP. The Director shall update
the Committee on both strategy and execution, as requested,
including collaboration efforts with the Federal Chief
Information Officer, with other government agencies, and by
participating in the Chief Information Officers Council.
(c) The Secretary of Defense shall designate or appoint a
White House Technology Liaison for the White House
Communications Agency and the Secretary of Homeland Security
shall designate or appoint a White House Technology Liaison for
the United States Secret Service. Any entity that becomes a
part of the Community after the issuance of this memorandum
shall designate or appoint a White House Technology Liaison for
that entity. The designation or appointment of a White House
Technology Liaison is subject to the review of, and shall be
made in consultation with, the President or his designee. The
Chief Information Officer of the Office of Administration and
the Chief Information Officer of the National Security Council,
and their successors in function, are designated as White House
Technology Liaisons for their respective components. In
coordination with the Director, the White House Technology
Liaisons shall ensure that the day-to-day operation of and
long-term strategy for information resources and information
systems provided to the President, Vice President, and EOP are
interoperable and effectively function as a single, modern, and
high-quality enterprise that reduces duplication, inefficiency,
and waste.
(d) The President or his designee shall retain the
authority to specify the application of operating policies and
procedures, including security measures, which are used in the
construction, operation, and maintenance of any information
resources or information system provided to the President, Vice
President, and EOP.
(e) Presidential Information Technology Community entities
shall:
(i) assist and provide information to the Deputy
Chief of Staff for Operations and the Director,
consistent with applicable law, as may be necessary to
implement this memorandum; and
(ii) as soon as practicable after the issuance of
this memorandum, enter into any memoranda of
understanding as necessary to give effect to the
provisions of this memorandum.
(f) As soon as practicable after the issuance of this
memorandum, EOP components shall take all necessary steps,
either individually or collectively, to ensure the proper
creation, storage, and transmission of EOP information on any
information systems and information resources provided to the
President, Vice President, and EOP.
Sec. 5. Definitions. As used in this memorandum:
(a) “Information resources,” “information systems,”
and “information technology” have the meanings assigned by
section 3502 of title 44, United States Code.4
(b) “Presidential Information Technology Community” means
the entities that provide information resources and information
systems to the President, Vice President, and EOP, including:
(i) the National Security Council;
(ii) the Office of Administration;
(iii) the United States Secret Service;
(iv) the White House Military Office; and
(v) the White House Communications Agency.
(c) “Executive Office of the President” means:
(i) each component of the EOP as is or may
hereafter be established;
(ii) any successor in function to an EOP component
that has been abolished and of which the function is
retained in the EOP; and
(iii) the President’s Commission on White House
Fellowships, the President’s Intelligence Advisory
Board, the Residence of the Vice President, and such
other entities as the President from time to time may
determine.
Sec. 6. General Provisions. (a) Nothing in this
memorandum shall be construed to impair or otherwise affect:
(i) the authority granted by law to an executive
department, agency, entity, office, or the head
thereof; or
(ii) the functions of the Director of the Office of
Management and Budget relating to budgetary,
administrative, or legislative proposals.
(b) This memorandum shall be implemented consistent with
applicable law and appropriate protections for privacy and civil
liberties, and subject to the availability of appropriations.
(c) This memorandum is not intended to, and does not,
create any right or benefit, substantive or procedural,
enforceable at law or in equity by any party against the
United States, its departments, agencies, or entities, its
officers, employees, or agents, or any other person.
BARACK OBAMA
# # #

U.S. government launches online traffic analytics dashboard for federal websites

There are roughly 1,361 .gov domains* operated by the executive branch of the United States federal government, 700-800 of which are live and in active use. Today, for the first time, the public can see how many people are visiting 300 executive branch government domains in real-time, including every cabinet department, by visiting analytics.usa.gov.

According to a post on the White House blog, the United States Digital Service “will use the data from the Digital Analytics Program to focus our digital service teams on the services that matter most to the American people, and analyze how much progress we are making. The Dashboard will help government agencies understand how people find, access, and use government services online to better serve the public – all while protecting privacy.  The program does not track individuals. It anonymizes the IP addresses of all visitors and then uses the resulting information in the aggregate.”

On Thursday morning, March 19th, tax-related services, weather, and immigration status are all popular. Notably, there’s an e-petition on the White House WeThePeople platform listed as well, adding data-driven transparency to what’s popular there right now.
analytics_usa_gov___The_US_government_s_web_traffic_

Former United States deputy chief technology officer Nick Sinai is excited about seeing the Web analytics data opened up online. Writing for the Harvard Shorenstein Center, where he is currently a fellow, Sinai adds some context for the new feature:

“Making government web performance open follows the digital services playbook from the new U.S. Digital Services,” he wrote. “Using data to drive decisions and defaulting to open are important strategies for building simple and useful citizen-facing digital services. Teal-time and historical government web performance is another example of how open government data holds the promise of improving government accountability and rebuilding trust in government.”

Here’s what the U.S. digital services team says they’ve already learned from analyzing this data:

Here’s what we’ve already learned from the data:

  • Our services must work well on all devices. Over the past 90 days, 33% all traffic to our sites came from people using phones and tablets. Over the same period last year, the number was 24%. Most of this growth came from an increase in mobile traffic. Every year, building digital services that work well on small screens becomes more important.
  • Seasonal services and unexpected events can cause surges in traffic. As you might expect, tax season is a busy time for the IRS. This is reflected in visits to pages on IRS.gov, which have more than tripled in the past 90 days compared with the previous quarter. Other jumps in traffic are less easy to predict. For example, a recently-announced settlement between AT&T and the Federal Trade Commissiongenerated a large increase in visits to the FTC’s website. Shortly after the settlement was announced, FTC.gov had four times more visitors than the same period in the previous year. These fluctuations underscore the importance of flexibility in the way we deploy our services so that we can scale our web hosting to support surges in traffic as well as save money when our sites are less busy.
  • Most people access our sites using newer web browsers. How do we improve digital services for everyone when not all web browsers work the same way? The data tells us that the percentage of people accessing our sites using outdated browsers is declining steadily. As users adopt newer web browsers, we can build services that use modern features and spend less time and money building services that work on outdated browsers. This change will also allow us to take advantage of features found in modern browsers that make it easier to build services that work well for Americans with disabilities, who access digital services using specialized devices such as screen readers.

If you have ideas, feedback or questions, the team behind the dashboard is working in the open on Github.

Over the coming months, we will encourage more sites to join the Digital Analytics Program, and we’ll include more information and insights about traffic to government sites with the same open source development process we used to create the Dashboard. If you have ideas for the project, or want to help improve it, let us know by contributing to the project on GitHub or emailing digitalgov@gsa.gov.

That last bit is notable; as its true all of the projects that 18F works on, this analytics dashboard is open source software.

There are some interesting additional details in 18F’s blog post on how the analytics dashbard was built, including the estimate that it took place “over the course of 2-3 weeks” with usability testing at a “local civic hacking meetup.”

First, that big number is made from HTML and D3, a Javascript library, that downloads and render the data. Using open standards means it renders well across browsers and mobile devices.

Second, 18F made an open source tool to manage the data reporting process called “analytics-reporter” that downloads Google Analytics reports and transforms that data into JSON.

Hopefully, in the years ahead, the American people will see more than the traffic to .gov websites: they’ll see concrete performance metrics like those displayed for the digital services the United Kingdom’s Government Digital Services team publishes at gov.uk/performance, including uptime, completion rate and satisfaction rate.

In the future, if the public can see the performance of Heathcare.gov, including glitches, or other government digital services, perhaps the people building and operating them will have more accountability for uptime and quality of service.

White House hosts “Open Government Workshop” during Sunshine Week

Yesterday, the White House hosted an “Open Government Workshop” in Washington, DC, a portion of which was livestreamed at though whitehouse.gov. The workshop was the kickoff event for planning the third United States Open Government National Action Plan for the Open Government Partnership.

Archived video is embedded below, including remarks from Megan Smith, the U.S. chief technology officer, Gayle Smith, a special assistant to the President and senior director at the National Security Council, and Tom Malinowski, the assistant secretary of state for democracy, human rights and labor. T

Some of the participants in the workshop shared pictures of the event coupled with brief observations on Twitter, but little of substance regarding the participants or the outcomes of their discussions has been released to the public to date.

Editor’s Note: Where social media falls short of sunshine

Shinning a light today on public participation in government thru social media! #opengov

A photo posted by Laura Cohen (@lauraandotis) on Mar 17, 2015 at 2:13pm PDT

Ironically, given that the event took place during Sunshine Week, the open government workshop was not open to the public or the press. While a user of the White House open government Twitter account encouraged its followers to “share ideas” and “keep the dialogue going,” the choice to use the #SunshineWeek hashtag effectively meant that the backchannel for the event was swamped with news of the White House’s decision to officially remove a regulation that subjected its Office of Administration to the Freedom of Information Act, the news of which broke on Freedom of Information Day in the United States. The administration’s legal reasoning is based upon a 2009 federal court decision that ruled the office was not subject to FOIA. In the Federal Register notice of the final rule, the administration hold that “The Office of Administration, as an entity whose sole function is to advise and assist the President of the United States, is not an agency under the Freedom of Information Act or the Privacy Act of 1974, nor does its implementation of Executive Order 13526 affect members of the public.”

The White House indicated that they will “absolutely” share more info about the workshop in the future.

UPDATE: OpenTheGovernment.org is helping to coordinate the public-facing aspect of the civil society consultation. They’re asking the public to contribute to a model National Action Plan. You can learn more and, after reading the guidelines, submit your own commitment online.

UCS: Progress on public access to U.S. government scientists, but serious issues remain

A new report (PDF) from the Union of Concerned Scientists found some improvements on the freedom of government scientists to speak, including their use of social media, but that significant impediments to unimpeded access also remain. The report, which included the scorecard pictured below, was published during Sunshine Week, the annual celebration of the People’s right to know what government does on their behalf. According to the report:

“Progress has continued since the 2013 report, with a majority of agency policies now including key provisions such as the right to state personal views, whistleblower provisions, and a dispute resolution process. On the social media front, where five agencies in the 2013 analysis had no social media policy at all, that number in the 2015 report has shrunk to just one.

However, most agency policies still lack important provisions such as right of last review and access to drafts and revisions. And while nearly all the agencies now have social media policies, some of those policies are still vague or incomplete. Thus, there is still significant work to do.”

csd-transparency-ggt2015-scorecard-full

The accessibility of government scientists to journalists and the public has been a significant issue in the United States in recent years (and north of the border, in Canada), particularly in the context of climate science and other environmental issues. In September 2011, Columbia Journalism Review (CJR) published an extensive feature that found that, despite high hopes, President Barack Obama’s administration had failed to make science accessible. By 2013, there was some measurable progress in the relationship between the scientific agencies and the press, at least as measured by the 2013 version of the UCS report.

UCS made several recommendations to federal agencies improve further:

Federal agency media policies need to be stronger to offer scientists clear guidance and protections against political interference. More broadly, agencies need to put free and open communication ahead of political considerations.

  • Federal agencies should develop strong media and social media policies that grant scientists the fundamental right of scientific free speech.
  • The Office of Science and Technology Policy should assess agency progress and speak forcefully on the importance of strong and effective media and social media policies.
  • Congress should hold agency heads accountable for encouraging the free flow of scientific information to the public.
  • The president should make strong and effective agency policies on media and social media a priority.
  • Journalists should call out those agencies that block the free flow of information to the public.

The importance of media and public access to government scientists will only grow in the years ahead as more government data is released online. It’s crucial for the press and the public to be able to contact the people who create, maintain and understand these databases when they create acts of journalism based upon them.

National Security Archive finds 40% E-FOIA compliance rate in federal government agencies

underConstruction

For Sunshine Week 2015, the National Security Archive​ conducted an audit of how well 165 federal government agencies in the United States of America comply with the E-FOIA Act of 1996. They found that only 67 of them had online libraries that were regularly updated with a significant number of documents released under the Freedom of Information Act. The criteria for the 165 agencies were that they had to have a chief Freedom of Information Officer and components that handled more than 500 FOIA requests annually.

Almost a decade after the E-FOIA Act, that’s about a 40% compliance rate. I wonder if the next U.S. Attorney General or the next presidential administration will make improving on this poor performance priority. It’s important for The United States Department of Justice​ to not only lead by example but push agencies into the 21st century when it comes to the Freedom of Information Act.

It would certainly help if Congress passed FOIA reform.

On that count, the Archive highlights a relevant issue in the current House and Senate FOIA reform bills in Congress: the FOIA statute states that documents that are “likely to become the subject of subsequent requests” should be published electronic reading rooms:

“The Department of Justice’s Office of Information Policy defines these records as “frequently requested records… or those which have been released three or more times to FOIA requesters.” Of course, it is time-consuming for agencies to develop a system that keeps track of how often a record has been released, which is in part why agencies rarely do so and are often in breach of the law. Troublingly, both the current House and Senate FOIA bills include language that codifies the instructions from the Department of Justice.

The National Security Archive believes the addition of this “three or more times” language actually harms the intent of the Freedom of Information Act as it will give agencies an easy excuse (“not requested three times yet!”) not to proactively post documents that agency FOIA offices have already spent time, money, and energy processing. We have formally suggested alternate language requiring that agencies generally post “all records, regardless of form or format that have been released in response to a FOIA request.”

This is a point that Members of Congress should think through carefully as they take another swing at reform. As I’ve highlighted elsewhere, FOIA requests that industry make are an important demand signal to show where data with economic value lies. (It’s also where the public interest tends to lie, with respect to FOIA requests from the media.)

While it’s true that it would take time and resources to build and maintain a system that tracks such requests by industry, there should already be a money trail from the fees paid to the agency. If FOIA reform leads to modernizing how it’s implemented, perhaps tying FOIA.gov to Data.gov might finally take place. The datasets are the subject of the most FOIA requests are the ones that should be prioritized for proactive disclosure online.

Adding a component that identifies which data sets are frequently requested, particularly periodically, should be a priority across the board for any administration that seeks to “manage information as an asset.” Adding the volume and periodicity of requests to the expanding national enterprise data inventory might naturally follow. It’s worth noting, too, that reform of the FOIA statute may not be necessary to achieve this end, if the 18F team working on modernizing FOIA software worked on it.

[STAT] State Department employees made .004% of email sent in 2013 into public records

Window_and_Hillary_Clinton_Not_Alone_in_Using_Private_Emails_to_Govern_-_Tech_-_GovExec_com

According to a new report from U.S. Department of State’s Office of the Inspector General, agency employees sent more than 1 billion emails, of which they made just 41,649 of them into public records.

That’s about 0.004% of them, by my rough calculation.

It’s a minuscule number, which probably why The Daily Beast ran a post reporting “only .00006% of State Department emails are preserved.”

While their calculation looks off by orders of magnitude, this tiny percentage still translates into members of the civil and foreign service entering almost none of their emails into archiving systems.

While the story hardly need it, this adds more interesting context to former Secretary of State Hillary Clinton’s decision to designate roughly 50% of her personal email as public records.

As Sunlight Foundation policy director John Wonderlich commented in Politico, this IG report undermines her argument that her emails with State Department workers were preserved on their end.

“Her justification around FOIA requests and around preservation became that most of the documents were cc’d or sent to .gov or state.gov addresses used by employees and therefore were preserved and accessible to requests, ” said Wonderlich “This [report] suggests that is not reliable at all.”

For more, read Josh Gerstein report exploring the broader ramifcations of the watchdog report on Clinton’s defense at greater length.

White House moves WhiteHouse.gov to HTTPS by default, tying privacy to security

The_White_House-https

A .gov website that uses HTTPS encryption by default for its visitors is a superb example of “privacy by design.” On March 6th, the Federal Trade Commission enabled encryption for FTC.gov. When I visited whitehouse.gov tonight, I found that the White House digital team had flipped the site for what’s likely the most prominent government website in the world. The White House Web team confirmed the change just after midnight.

According to Leigh Heyman, director of new media technologies at the White House, over the next few days, the team be migrating other domains, like the bare domain name, whitehouse.gov, and m.whitehouse.gov, over to HTTPS as well, joining http://www.whitehouse.gov.

“Americans care about their privacy, and that’s what the White House’s move to HTTPS by default is about,” said Eric Mill, an open government software engineer at 18F. “The White House’s use of HTTPS protects visitors’ personal information and browsing activity when they connect to whitehouse.gov across the vast, unpredictable network of computers that is the internet.”

If you’re unfamiliar with HTTPS, it’s a way of encrypting the way you connect to a Web server online. Specifically, HTTPS refers to layering the Hypertext Transfer Protocol (HTTP) on top of the Secure Sockets Layer (SSL) or Transport Layer Security (TLS). What that means in practice is that your requests to the Web server and the pages results from it are encrypted and decrypted. Why does that matter? Consider, for instance, if someone is looking up sensitive health information online and visits a government website without HTTPS that also has data collection.

“Use of https is generally considered to be good practice, however, as opposed to unencrypted, regular http, although it adds a small amount of extra processing and delay to do the encryption,” commented Eugene Spafford, a Purdue University computer science professor and founder and executive director of the Center for Education and Research in Information Assurance and Security.

“HTTPS primarily provides three things: greater authentication, stream privacy, and message integrity. A quick look at the site doesn’t reveal (to me) anything that would likely require privacy or heightened message integrity. The most immediate consequence is that parties connecting to the website can have increased confidence of the site’s authenticity because a signed certificate will be employed. Of course, most people don’t actually verify certificates and their roots (cf. Superfish), so this isn’t an ironclad identification.”

Why does this matter?

“This immediately creates a strong baseline of privacy and security for anyone in the world, American or otherwise, who visits the White House website — whether to read their blog, learn more about the President, download official policies, or anything else inside whitehouse.gov,” said Mill.

“At a basic level, what a person sees and does on whitehouse.gov should be between them and the White House. When someone reads official policies published on whitehouse.gov, they should be confident that policy is real and authentic. The White House’s use of HTTPS by default means those promises just got a lot stronger.”

Ashkan Soltani, the FTC’s chief technologist, explained why that federal agency shifted at the Tech@FTC blog:

As a quick primer, HTTPS encryption secures your communications while in transit with websites so that only you and the website are able to view the content. The lock icon now appearing in your browser represents that the communication is encrypted and eavesdroppers are unable to look in. At this time, secure browsing is generally not a requirement for federal websites, but it is considered an industry best practice. Transit encryption is an important safeguard against eavesdroppers and has been the subject of previous investigations where we alleged companies failed to live up to their security promises when collecting personal information. It’s an important step when websites or apps collect personal information, and is a great best practice even if they don’t.

What broader trends does this tap into?

The White House moving to HTTPS is part of a larger move to lead by example in promoting privacy and security best practices, related Soltani, over email.

“I believe we’ll see a slow shift over the next few years of websites and services moving to HTTPS by default,” he said, “something a number of standards bodies including ISOC, IETF, and IAB have also called for.”

Along with FTC.gov, Mill highlighted the Privacy and Civil Liberties Oversight Board (PCLOB), the independent agency charged with balancing the rights of American citizens against the security steps taken in the wake of the terrorist attacks of 9/11, to HTTPS.

They’re far from alone: “Last month, 18F worked with 19 other .gov domains to go the distance to ensure browsers would always connect to them over HTTPS,” said Mill.

“Tt’s important to understand that what’s happening now in the federal government is what the broader internet has been working on for a while: making privacy the default.

The standards bodies that guide the internet’s development are recommending that the internet be encrypted by default, instructing their working groups to prioritize encryption in new protocol development, and declaring a more secure future for the web. The fastest versions of HTTP today already require encryption in major browsers, and it’s becoming easier to imagine a future where web browsers proactively warn users about unencrypted websites.

This is also why every .gov that 18F builds with its partner agencies uses HTTPS, full stop. We work hard to demonstrate that HTTPS can be fast, inexpensive, and easy. It’s a better future, and a practical one.”

The kind of privacy and security the White House is offering its visitors is what we should come to expect from the entire web, not just websites someone thinks are “sensitive”. All Web browsing is sensitive, and the White House’s leadership here reinforces that.”

It looks like Chris Soghoian, the principal technologist at the Speech, Privacy and Technology Project in the American Civil Liberties Union, is going to have a good day tomorrow.

While the Obama administration has taken its lumps on digital privacy after revelations of bulk surveillance of the Internet backbone by the National Security Agency, this is undeniably an important step towards securing the traffic of millions of people who visit whitehouse.gov every month.

Now that the White House is leading by example, hopefully other federal, state and local government entities will also adopt the standard.

“Everyone should want a simple feeling of privacy as they use the web, and confidence that they’re at the real and exact website they meant to visit,” said Mill. “While not everyone is highly attuned to watching for that padlock in their browser, the more websites that add it — especially high profile ones like the White House — the more that people can depend on that promise being met.”