USASpending.gov addresses some data issues, adds Github issues tracker for feedback

usaspending

On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.

A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.

Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.

In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:

The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.

This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.

Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.

Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.

Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.

If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.

[Image Credit: Govfresh, via USASpending.gov]

18F launches alpha foia.gov in a bid to reboot Freedom of Information Act requests for the 21st century

alpha foia gov

18F, the federal government’s new IT development shop, has launched a new look at the Freedom of Information Act (FOIA) in the form of a open source application hosted on Github. Today’s announcement is the most substantive evidence yet that the Obama administration will indeed modernize the Freedom of Information Act, as the United States committed to doing in its second National Action Plan on Open Government. Given how poor some of the “FOIA portals” and underlying software that supports them exists is at all level of government, this is tremendous news for anyone that cares about the use of technology to support open government.

Notably, 18F already has a prototype (pictured above) online that shows what a consolidated request submission hub could look like and plans to iterate upon it.  This is a perfect example of “lean government,” or the application of lean startup principles and agile development to the creation of citizen-centric services in the public sector.  Demonstrating its commitment to developing free and open source software in the open, 18F asked the public to follow the process online at their FOIA software repository on Github, send them feedback or even contribute to the project.

18F has now committed to creating software that improvse how requests made under the Freedom of Information Act can be improved through technology. Specifically that it will develop tools that “improve the FOIA request submission experience,” “create a scalable infrastructure for making requests to federal agencies” and “make it easier for requesters to find records and other information that have already been made available online.”

According to 18F’s blog post, this work is supported and overseen by a “FOIA Task Force,” consisting of representatives from the Department of Justice, Environmental Protection Agency, the Office of Management and Budget, the Office of Science and Technology Policy. The task force will need to focus upon more than technology: while poor software has hindered requests and publishing, that’s not the primary issue that’s hindering the speed or quality of responses.

Despite the U.S. attorney general’s laudable commitment to a new era of open government in 2009, the Obama administration received a .91 GPA in FOIA compliance earlier this year from the Center for Effective Government.

While White House press secretary Josh Earnest may be well correct in stating that the federal government is processing more FOIA requests than ever, As the National Security Archive noted in March, the use of a FOIA exemption (protecting “deliberative processes”) to deny or heavily redact requests has skyrocketed in the past two years.

use of B5 exemptions

[NATIONAL SECURITY ARCHIVE: Chart created by Lauren Harper.]

As with the reduced access to government staff and scientists that a group of 38 journalism and open government advocates decried earlier this year, improving FOIA compliance cannot solely be addressed through technological means. To address endemic government secrecy and outright abuse of exemptions to protect against politically inconvenient disclosures, Obama administration — in particular, the U.S. Justice Department — will need to expend political capital and push agencies to actually shift the cultural default towards openness and release uncomfortable or embarrassing data and documents and not redact them beyond understanding.

That’s admittedly a huge challenge, particularly for an administration facing multiple foreign and domestic conundrums, including a scandal over missing IRS emails and obfuscated records in an election year and the most politically polarized Congress and electorate in the nation’s history, but if President Barack Obama is truly committed to “creating an unprecedented level of openness in government,” it’s one that he and his administration will need to take on.

18F commits to developing free and open source software by default for Uncle Sam

5682524083_4c81641ce1_oAt 18F, Uncle Sam is hoping to tap the success of the U.K.’s Government Digital Services. If the new digital government team housed with the U.S. General Services Administration gets it right, they’ll succeed in building 21st century citizen services by failing fast instead of failing big, as the Center for Medicare and Medicaid Services memorably did last year with Healthcare.gov through poor planning and oversight and Social Security has this summer. One of the lessons learned from the Consumer Financial Protection Bureau‘s successful use of technology is to align open source policy with mission. This week, 18F has done just that, publishing an open source policy on Github that makes open source the default in development:

The default position of 18F when developing new projects is to:
1. Use Free and Open Source Software (FOSS), which is software that does not charge users a purchase or licensing fee for modifying or redistributing the source code, in our projects and contribute back to the open source community.
2. Create an environment where any project can be developed in the open.
3. Publish publicly all source code created or modified by 18F, whether developed in-house by government staff or through contracts negotiated by 18F.

Eric Mill and Raphael Majma published a post on Tumblr that explained what FOSS, the policy, 18F’s open source team, approach and teased forthcoming guidelines for reuse:

FOSS is software that does not charge users a purchase or licensing fee for modifying or redistributing the source code. There are many benefits to using FOSS, including allowing for product customization and better interoperability between products. Citizen and consumer needs can change rapidly. FOSS allows us to modify software iteratively and to quickly change or experiment as needed.

Similarly, openly publishing our code creates cost-savings for the American people by producing a more secure, reusable product. Code that is available online for the public to inspect is open to a more rigorous review process that can assist in identifying flaws in the source code. Developing in the open, when appropriate, opens the project up to that review process earlier and allows for discussions to guide the direction of a products development. This creates a distinct advantage over proprietary software that undergoes a less diverse review and provides 18F with an opportunity to engage our stakeholders in ways that strengthen our work.

The use of open source software is not new in the Federal Government. Agencies have been using open source software for many years to great effect. What fewer agencies do is publish developed source code or develop in the open. When the Food and Drug Administration built out openFDA, an API that lets you query adverse drug events, they did so in the open. Because the source code was being published online to the public, a volunteer was able to review the code and find an issue. The volunteer not only identified the issue, but provided a solution to the team that was accepted as a part of the final product. Our policy hopes to recreate these kinds of public interactions and we look forward to other offices within the Federal Government joining us in working on FOSS projects.

In the next few days, we’re excited to publish a contributor’s guide about reuse and sharing of our code and some advice on working in the open from day one.

IMAGE CREDIT: mil-oss.org

At 18F in GSA, U.S. seeks to tap the success of the U.K.’s Government Digital Services

bridge-21st-centuryThe question of how the United States can avoid another Healthcare.gov debacle has been on the mind of many officials, from Congress to the man in the Oval Office.

Last November, I speculated about the potential of a” kernel of a United States Digital Services team built around the DNA of the CFPB: digital by default, open by nature,” incorporating the skills of Presidential Innovation Fellows.

As I wrote last week, after a successful big fix to Healthcare.gov by a trauma team got the trouble marketplace for health insurance working, the Obama administration has been moving forward on information technology reforms, including a new development unit within the U.S. General Services Administration.

This week, that new unit became a real entity online, at “18F.

As with the United Kingdom’s Government Digital Services Team, 18F is focused on delivery, an area that the UK’s executive director of digital, Mike Bracken, has been relentless in pushing. Here’s how 18F introduced itself:

18F builds effective, user-centric digital services focused on the interaction between government and the people and businesses it serves. We help agencies deliver on their mission through the development of digital and web services. Our newly formed organization, within the General Services Administration, encompasses the Presidential Innovation Fellows program and an in-house digital delivery team.

18F is a startup within GSA — the agency responsible for government procurement — giving us the power to make small changes with big effect. We’re doers, recruited from industry and the most innovative corners of public service, who are passionate about “hacking” bureaucracy to drive efficiency, transparency, and savings for government agencies and the American people. We make easy things easy, and hard things possible.

The 18F team, amongst other things, has some intriguing, geeky, and even funny titles for government workers, all focused around “agents.” API Agent. Counter Agent. Free Agent. Service Agent. Change Agent. User Agent. Agent Schmagent. Reagent. Agent onGover(). It’s fair to say that their branding, at minimum, sets this “startup in government” apart.

So does their initial foray into social media, now basic building block of digital engagement for government: 18F is on Twitter, Tumblr and Github at launch.

Looks like their office suite is pretty sweet, too.

18F-floorplan

This effort won’t be a panacea for federal IT ills, nor will a U.S. Government Digital Office nor the role of a U.S. chief technology officer be institutionalized until Congress acts. That said, 18F looks like a bonafide effort to take the approaches to buying, building and maintaining digital and Web services that worked in the Presidential Innovation Fellows program and the Consumer Financial Protection Bureau and trying to scale them around the federal government. The team explained more at their Tumblr blog about how they’ll approach their sizable remit:

  • Partner with agencies to deliver high quality in-house digital services using agile methodologies pioneered by top technology startups.
  • Rapidly deploy working prototypes using Lean Startup principles to get a desired product into a customer’s hands faster.
  • Offer digital tools and services that result in governmentwide reuse and savings, allowing agencies to reinvest in their core missions.
  • We’re transparent about our work, develop in the open, and commit to continuous improvement.

More than five years ago, Anil Dash wrote that the most interesting startup of 2009 was the United States government. Maybe, just maybe, that’s become true again, given the potential impact that the intelligent application of modern development practices could have on the digital government services that hundreds of millions of Americans increasingly expect and depend upon. What I’ve seen so far is promising, from the website itself to an initial pilot project, FBopen, that provides a simple, clean, mobile-friendly interface for small businesses to “search for opportunities to work with the U.S. government.”

fbopen

Clay Johnson, a member of the inaugural class of Presidential Innovation Fellows and founder of a startup focused on improving government IT procurement, offered measured praise for the launch of 18F:

Is it a complete solution to government’s IT woes? No. But, like RFP-IT and FITARA, it’s a component to a larger solution. Much of these problems stem from a faulty way of mitigating risk. The assumption is that by erecting barriers to entry – making it so that the only bets to be made are safe ones – then you can never fail. But evidence shows us something different: by increasing the barriers to competition, you not only increase risk, you also get mediocre results.

The best way for government to mitigate risk is to increase competition, and ensure that companies doing work for the citizen are transparently evaluated based on the merits of their work. Hopefully, 18F can position itself not only as a group of talented people who can deliver, but also an organization that connects agencies to great talent outside of its own walls. To change the mindset of the IT implementation, and convince people inside of government that not only can small teams like 18F do the job, but there are dozens of other small teams that are here to help.

Given the current nation-wide malaise about the U.S. government’s ability to execute on technology project, the only approach that will win 18F accolades after the launch of these modern websites will be the unit’s ability to deliver more of them, along with services to support others. Good luck, team.