U.K. National Archives makes ‘good law’ online, builds upon open data as a platform

uk-justice-ministry

This September, I visited the United Kingdom’s Ministry of Justice and looked at the last remaining section of the Magna Carta that remains in effect. I was not, however, in a climate-controlled reading room, looking at a parchment or sheepskin.

uk-justice-ministry-glass

Rather, I was sitting in the Ministry’s sunny atrium, where John Sheridan was showing me the latest version of the seminal legal document, now living on online, on his laptop screen. The remaining section that is in force is rather important to Western civilization and the rule of law as many citizens in democracies now experience it:

NO Freeman shall be taken or imprisoned, or be disseised of his Freehold, or Liberties, or free Customs, or be outlawed, or exiled, or any other wise destroyed; nor will We not pass upon him, nor [X1condemn him,] but by lawful judgment of his Peers, or by the Law of the Land. We will sell to no man, we will not deny or defer to any man either Justice or Right.

9919241003_ffd39b552d_b

From due process to eminent domain to a right to a jury trial, many of the rights that American or British citizens take as a given today have their basis in the English common law that stems from this document.

I’d first met Sheridan virtually, back in August 2010, when I talked with the head of e-services and strategy at the United Kingdom’s National Archives about how linked data was opening up eight hundred years of legal history. That month, the National Archives launched legislation.gov.uk to provide public access to more than eight centuries of the legal history in England, Scotland, Wales and Northern Ireland. Just over three years later, I stepped off the Tube at the St. James Park Station and walked over to meet him in person and learn how his aspirations for legislation.gov.uk had met up with reality.

Over a cup of tea, Sheridan caught me up on the progress that his team has made in digitizing documents and improving the laws of the land. There are now 2 million monthly unique visitors to legislation.gov.uk every month, with 500+ million page views annually. People really are reading Parliament’s output, he observed, and increasingly doing so on tablets and mobile devices. The amount of content flowing into the site is considerable: according to Sheridan, the United Kingdom is passing laws at an estimated rate of 100,000 words every month, or twice as much as the complete works of Shakespeare.

Notable improvements over the years include the ability to compare the original text of legislation versus the latest version (as we did with the Magna Carta) and view a timeline of changes using a slider for navigation, exploring any given moment in time. Sheridan was particularly proud of the site’s rendering of legislation in HTML, include human-readable permanent uniform resource locators (URLS) and the capacity to produce on-demand PDFs of a given document. (This isn’t universally true: I found some orders appear still as PDFs).

More specifically, Sheridan highlighted a “good law” project, wherein the Office of the Parliamentary Counsel (OPC) of Britain is working to help develop plain language laws that are “necessary, clear, coherent, effective and accessible.” A notable component of this good law project is an effort to apply a tool used in online publishing, software development and advertising — A/B testing — to testing different versions of legislation for usability.

The video of a TedX talk embedded below by Richard Heaton, the permanent secretary of the United Kingdom’s Cabinet Office and first parliamentary counsel, explores the idea of “good law” at more length:

Sheridan went on to describe one of the more ambitious online collaborations between a government and its citizens I had heard of to date, a novel cross-Atlantic challenge co-sponsored by the UK and US governments, and a hairy legal technology challenge bearing down upon societies everywhere: what happens when software interprets the law?

For instance, he suggested, consider the increasing use of Oracle software around legislation. “As statutes are interpreted by software, what’s introduced by the code? What about quality testing?”

As this becomes a data problem, “you need information to contextualize it,” said Sheridan. “If you’re thinking about legislation as code, and as data, it raises huge questions for the rule of law.”

Open data as a platform

In the video below, John Sheridan talks about the benefits of opening up government data using application programming interfaces:

Sheridan has been one of the world’s foremost proponents of publishing legislative data through APIs, an approach that has come under criticism by open government data advocates after the government shutdown in the United States. (In 2014, forward-thinking governments publishing open data might consider provide basic visualization tools to site visitors, API access for third-party developers and internal users, and bulk data downloads.) One key difference between the approach of his team and other government entities might be that the National Archives are “dogfooding,” or consuming the same data through the same interface that they expect third-parties to use, as Sheridan wrote last March:

“We developed the API and then built the legislation.gov.uk website on top of it. The API isn’t a bolt-on or additional feature, it is the beating heart of the service. Thanks to this approach it is very easy to access legislation data – just add /data.xml or /data.rdf to any web page containing legislation, or /data.feed, to any list or search results. One benefit of this approach is that the website, in a way, also documents the API for developers, helping them understand this complex data.”

Perhaps because of that perspective, Sheridan, was as supportive of an APIs when we talked this September as he had been in 2012:

The legislation.gov.uk API has changed everything for us. It powers our website. It has enabled us to move to an open data business model, securing the editorial effort we need from the private sector for this important source of public data. It allows us to deliver information and services across channels and platforms through third party applications. We are developing other tools that use the API, using Linked Data – from recording the provenance of new legislation as it is converted from one format to another, to a suite of web based editorial tools for legislation, including a natural language processing capability that automatically identifies the legislative effects. Everything we do is underpinned by the API and Linked Data. With the foundations in place, the possibilities of what can be done with legislation data are now almost limitless.

Sheridan noted to me that the United Kingdom’s legislative open government data efforts are now acting as a platform for large commercial legal publishers and new entrants, like mobile legislative app, iLegal.

ilegal-launch-website_05_indexThe iLegal app content is derived from the legislation.gov.uk API and offers handy features, like offline access to all items of legislation. iLegal currently costs £49.99/$74.99 annually or £149.99/$219.99 for a lifetime subscription, which might seem steep but is a fraction of the cost of of Halsbury’s Statutes, currently listed at £9,360.00 from Lexis-Nexis.

This approach to publishing the laws of the land online, in structured form under an open license, is an instantiation of the vision for Law.gov that citizen archivist Carl Malamud has been advocating for in the United States. 2013 saw some progress in that vein when the U.S. House of Representatives publishes U.S. Code as open government data.)

What’s notable about the United Kingdom’s example, however, is that less then a decade ago, none of this could have been possible. Why? As ScraperWiki founder Francis Irving explained, the UK’s database of laws was proprietary data until December 2006. Now, however, the law of the land is released back to the people as it is updated, a living code available in digital form to any member of the public that wishes to read or reuse it.

The United Kingdom, however, has moved beyond simply publishing legislation as open data: they’re actively soliciting civic participation in its maintenance and improvement. For the last year, the National Archives has been guiding the world’s leading commercial open data curation project.

“We are using open data as business model for fulfilling public services,” said Sheridan, in our interview. “We train people to do editorial work. They are paid to improve data. The outputs are public.”

In other words, the open government data always remains free to the people through legislation.gov.uk but any academic, nonprofit or commercial entity can act to add value to it and sell access to the resulting applications, analyses or interfaces.

As far as Sheridan could recall, this was the only such example in the government of the United Kingdom where such a feedback loop exist. The closest parallels in the United States is the U.S. Agency for International Development crowdsourcing geocoding 117,000 loan records with the help of online volunteers [Case Study] or the citizen archivist program of the U.S. National Archives.

Since the start of the UK project, they have doubled the number of people working on their open data, Sheridan told me. “The bottleneck is training,” he said. “We have almost unlimited editorial expertise available through our website. We define the process and rules, and then let anyone contribute. For example, we’re now working on revising legislation, identifying changes, researching it — when it comes in, what it affects — and then working with editor. Previous to this effort, government hasn’t been able to revise secondary legislation.”

Sheridan said that the next step is feedback for other editorial values.

“We’re looking for more experts,” he said. “They’re generally paid for by someone. It’s very close to open source software model. They must be able to demonstrate competence. There’s a 45-minute test, which we’re now given to thousands of people.”

If this continues to work, distributed online collaboration is a “brilliant way to help improve the quality of law,” said Sheridan.

“It’s a way to get the work done — and the work is really hard. You have to invest time and energy, and you must protect the reputation of the Archive. This is somewhat radical for the nation’s statute book. We have redesigned the process so people can work with us. It’s not a wiki, but participation is open. It’s peer production.”

A trans-Atlantic challenge to map legislative data

large_opaque

In September, Sheridan also told me about an unusual challenge that has just gone live at Challenge.gov, the United States’ flagship prizes and competitions platform: a contest to assess the compatibility of Akoma Ntoso with U.S. Congress and U.K. Parliament markup languages.

The U.K. National Archives and U.S. Library of Congress have asked for help mapping elements from bills to the most recent Akoma Ntoso schema. (Akoma Ntoso is an emerging global standard for machine-readable data describing parliamentary, legislative and judiciary documents.) The best algorithm that maps U.S. bill XML or UK bill XML to Akoma Ntoso XML, including necessary data files and supporting documentation, will win $10,000.

If you have both skills and interest, get cracking: the challenge closes on December 31, 2013.

Regulations.gov relaunches with APIs, integrates social media, hopes for public participation

President Barack Obama signs H.R. 2751, the “FDA Food Safety Modernization Act,” in the Oval Office, Jan. 4, 2011. (Official White House Photo by Pete Souza)

President Barack Obama in the Oval Office, Jan. 4, 2011. (Official White House Photo by Pete Souza)

On January 18, 2011, President Obama issued an executive order directing that regulations shall be adopted through a process that involves participation. 13 months later, the nation’s primary online regulatory website received an overdue redesign and, significantly, a commitment from the administrator of the White House Office of Information and Regulatory Affairs (OIRA) to make regulatory data available to the public.

Today, the White House announced the relaunch of Regulations.gov in a post on remaking public participation by Cass Sunstein, the administrator of the OIRA:

…the President issues Executive Order 13563, in which he directed regulatory agencies to base regulations on an “open exchange of information and perspectives” and to promote public participation in Federal rulemaking.  The President identified Regulations.gov as the centralized portal for timely public access to regulatory content online.

In response to the President’s direction, Regulations.gov has launched a major redesign, including innovative new search tools, social media connections, and better access to regulatory data.  The result is a significantly improved website that will help members of the public to engage with agencies and ultimately to improve the content of rules.

The redesign of Regulations.gov also fulfills the President’s commitment in The Open Government Partnership National Action Plan to “improve public services,” including to “expand public participation in the development of regulations.” This step is just one of many, consistent with the National Action Plan, designed to make our Federal Government more transparent, participatory, and collaborative.

I’ve embedded the video that Regulations.gov released about the launch below:

The relaunch includes the following changes:

  • New Regulations.gov and Web design.
  • A new “Browse” tab that groups regulations into 10 categories, sorted by industry
  • A new “Learn” tab that describes the regulatory process
  • Improved search
  • Integrated social media tools (Twitter, Facebook, YouTube and Regulations.gov Exchange)
  • New Application Programming Interfaces (APIs) and standard, Federal Register-specific URLs.

That last detail will be of particular interest to the open government and open data community. Sunstein explained the thinking behind the role of APIs at the WhiteHouse.gov blog:

Application Programming Interfaces (APIs) are technical interfaces/tools that allow people to pull regulatory content from Regulations.gov. For most of us, the addition of “APIs” on Regulations.gov doesn’t mean much, but for web managers and experts in the applications community, providing APIs will fundamentally change the way people will be able to interact with public federal regulatory data and content.

The initial APIs will enable developers to pull data out of Regulations.gov, and in future releases, the site will include APIs for receiving comment submissions from other sites. With the addition of APIs, other web sites – ranging from other Government sites to industry associations to public interest groups – will now be able to repurpose publicly-available regulatory information on Regulations.gov, and format this information in unique ways such as mobile apps, analytical tools, “widgets” and “mashups.” We don’t know exactly where this will lead us – technological advances are full of surprises – but we are likely to see major improvements in public understanding and participation in rulemaking.

While the APIs will need to be explored and the data behind them assessed for quality, releasing regulatory data through APIs could in theory underpin a wide variety of new consumer-facing services. If you’re interested in the APIs, click on “Developers – Beta” at Regulations.gov to download a PDF with that contains API directions, URLs and information about an API Key.

A time for e-rulemaking

This move comes as part of a larger effort towards e-rulemaking by this White House that will almost certainly be carried over into future administrations, regardless of the political persuasion of the incumbent of the Oval Office. In the 21st century, the country desperately needs a smarter approach to regulations.

As the Wall Street Journal reported last year, the ongoing regulatory review by OIRA is a nod to serious, long-standing concerns in the business community about excessive regulation hampering investment and job creation as citizens struggle to recover from the effects of the Great Recession.

As the cover story of this month’s issue of The Economist highlights, concerns about an over-regulated America are cresting in this election year, with headlines from that same magazine decrying “excessive environmental regulation” and calling for more accurate measurement of the cost of regulations. Deleting regulations is far from easy to do but there does appear to be a political tailwind behind doing so.

We’ll see if an upgraded online portal that is being touted as a means to include the public in participating in rulemaking makes any difference in regulatory outcomes. Rulemaking and regulatory review are, virtually by their nature, wonky and involve esoteric processes that rely upon knowledge of existing laws and regulations.

While the Internet could involve many more people in the process, improved outcomes will depend upon an digitally literate populace that’s willing to spend some of its civic surplus on public participation.

To put it another way, getting to “Regulations 2.0” will require “Citizen 2.0” — and we’ll need the combined efforts of all our schools, universities, libraries, non-profits and open government advocates to have a hope of successfully making that upgrade.

Visualizing the future of programmable cities

Technology is fueling new visions for the future of cities. Today at the South by Southwest Interactive festival, a panel considered “Web Mashup Platforms and Future Programmable Cities. NYC chief digital officer Rachel Sterne (@RachelSterne) joined Christine Outram (@cityinnovation), Vlad Trifa (@vladounet) and Dominique Guinard (@domguinard) in exploring how open data, mobile platforms and citizen engagement will shape what comes next in urban life.

Below, visual notes by OgilvyNotes and ImageThink capture the conversation.

n Web Mashup Platforms and Future Programmable Cities

For more on how cities are embracing new platforms and technologies, learn about citizensourcing smarter government in New York City.

[Hat Tip: Rachel Sterne]

Rebooting FCC.gov: An interview with FCC managing director Steven VanRoekel

Earlier today, my newest Radar post explained what to expect as FCC.gov is poised for an overdue overhaul featured FCC managing director Steven VanRoekel on participation and building platforms. For the full perspective on what’s next in “FCC 2.0,” check out Radar. Our video interview is embedded below.

FCC Chairman Julius Genachowski and Managing Director Steven VanRoekel will discuss their experiences turning FCC.gov into a 21st-century consumer resource at the Gov 2.0 Summit in Washington, D.C. (Sept. 7-8). Request an invitation.