USASpending.gov addresses some data issues, adds Github issues tracker for feedback

usaspending

On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.

A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.

Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.

In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:

The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.

This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.

Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.

Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.

Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.

If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.

[Image Credit: Govfresh, via USASpending.gov]

Farewell, Thomas.gov. Hello, Congress.gov.

THOMAS-redirecting-to-Congress.gov_

On November 19th, Thomas.gov, the venerable website of the United States Congress, will begin to redirect visitors to Congress.gov. The new site, which launched in beta in September 2012, will become the primary governmental resource for the text of legislation, past, present and future, along with reports from committees, speeches from the floor of Congress and cost estimates from the Congressional Budget Office.

While the official announcement was made today by the Library of Congress, Thomas.gov’s custodian, leading headlines about Congress trading in the new Congress.gov and a note in Roll Call, the transition from THOMAS.gov to Congress.gov has been going on all fall, including updates to the new site and launching the Constitution Annotated and associated app.

THOMAS is centuries old, at least as measured in terms of Internet time. Launched in January of 1995, Thomas.gov was one of the first 23,000 websites to go online. When it went live the Internet had a worldwide user base of less than 40 million people, the majority of whom surfed the young World Wide Web using Mosaic and Netscape, checked their email on Eudora and dialed in on America Online. Watch the video below to get a sense of what life was like online nearly two decades ago.

Today, Thomas.gov receives, on average, 10 million visits every year, although I suspect many of those visits come from wonky repeat customers in or around the District of Columbia. I have no servers logs to prove that one way or another, but THOMAS has long been alternately beloved of or bemoaned by Congressional staffers and correspondents, all of whom have had to rely upon its increasingly creaky infrastructure for nearly two decades as the national repository of legislation and reports. So, too, have millions of Americans around the rest of the country who want to read proposed bills.

While incremental improvements to search and sharing in recent years have improved the site, for a decade people interested in tracking Congress have increasingly turned to sites like Govtrack or the New York Times for data created by scraping THOMAS. What does that mean, in practice? While Congress.gov will be official source of information, until its operators move to act as a platform for legislative data instead of a portal for legislative information. Open government advocates have been calling for the release of bulk legislative data for many years, culminating in frustration this September when a Library of Congress cost estimate acknowledged that Congress.gov “was not designed specifically to facilitate the extraction of the data as XML documents for bulk download.”

Putting the issue of bulk data aside, the new Congress.gov is an immense improvement on THOMAS in every way, as I reported last year:

Tapping into a growing trend in government new media, the new Congress.gov features responsive design, adapting to desktop, tablet or smartphone screens. It’s also search-centric, with Boolean search and, in an acknowledgement that most of its visitors show up looking for information, puts a search field front and center in the interface. The site includes member profiles for U.S. Senators and Representatives, with associated legislative work. In a nod to a mainstay of social media and media websites, the new Congress.gov also has a “most viewed bills” list that lets visitors see at a glance what laws or proposals are gathering interest online.

Since September 2012 digital staff at the Law Library of Congress have been busy since the Congress.gov launched in beta, adding new features and context at a steady pace, including adding the Congressional Recordcommittee reports, standing committee pages, and the ability to “Search within results.

On November 19th, when THOMAS is retired, the social media outposts of the site will also transition. @THOMASDotGov will transition its more than 15,500 followers to a new identity.

In a press release, the Library of Congress indicated that the old site will remain accessible from the Congress.gov homepage through late 2014. After that, historians may have to hope that the National Archives adopts whatever code or data retains historical interest into its servers, lest it moulder and succumb to bitrot — unfortunately, the configuration of the robots.txt file for Thomas.gov appears to have prevented the Internet Archive from preserving its iterations over the years.

If you’re interested in learning how to use the new Congress.gov, you can register at beta.congress.gov/help for training sessions scheduled for November 14, January 16, March 11 and March 16.