UPDATE: The refresh of Healthcare.gov in June went well. On October 1st, when the marketplace for health insurance went live at the site.gov, millions of users flocked to the website and clicked “apply now.” For days, however, virtually none of them were able to create accounts, much less complete the rest of the process and enroll for insurance. By the end of the week, however, it was clear that the problems at Healthcare.gov were not just a function of high traffic but the result of the failure of software written by private contractors, with deeper issues that may extend beyond account creation into other areas of the site. On October 9th, as prospective enrollees continued to be frustrated by error-plagued websites around the country, I joined Washington Post TV to give a preliminary post-mortem on why the HealthCare.gov relaunch went so poorly.

The article that follows, which was extended and published at The Atlantic, describes the team and process that collaborated on launch of the new site in June, not the officials or contractors that created the botched enterprise software application that went live on October 1st. In the Atlantic, I cautioned that “…the site is just one component of the insurance exchanges. Others may not be ready by the October deadline.”  The part of the site I lauded continues to work well, although the Github repository for it was taken offline. The rest has …not. I’ve taken some heat in the articles’ comments and elsewhere online for being so positive, in light of recent events, but the reporting holds up: using Jekyll is working. Both versions of the story, however, should have included a clearer caveat that the software behind the website had yet to go live — and that reports that the government was behind on testing Healthcare.gov security suggested other issues might be present at launch. If readers were misled by either article, I apologize. –Alex

---

Healthcare.gov already occupies an unusual place in history, as the first website to be demonstrated by a sitting President of the United States. In October, it will take on an even more important historic role, guiding millions of Americans through the process of choosing health insurance.

How a website is built or designed may seem mundane to many people in 2013, but when the site in question is focused upon such a function, it matters. Yesterday, the United States Department of Health and Human Services (HHS) relaunched Healthcare.gov with a new look, feel and cutting edge underlying architecture that is beyond rare in federal government. The new site has been built in public for months, iteratively created by a team of designers and engineers using cutting edge open source technologies. This site is the rarest of birds: a next-generation website that happens to be a .gov.

“It’s fast, built in static HTML, completely scalable and secure,” said Bryan Sivak, chief technology officer of HHS, in an interview. “It’s basically setting up a Web server. That’s the beauty of it.”

The people building the new Healthcare.gov are unusual: instead of an obscure sub-contractor in a nameless office park in northern Virginia, a by a multidisciplinary team at HHS worked with Development Seed, a scrappy startup in a garage in the District of Columbia that made its mark in the DC tech scene deploying Drupal, an open source content management system that has become popular in the federal government over the past several years.

“This is our ultimate dogfooding experience,” said Eric Gundersen, the co-founder of Development Seed. “We’re going to build it and then buy insurance through it.”

“The work that they’re doing is amazing,” said Sivak, “like how they organize their sprints and code. It’s incredible what can happen when you give a team of talented developers and managers room to work and let them go.”

What makes this ambitious experiment in social coding unusual is that the larger political and health care policy context that they’re working within is more fraught with tension and scrutiny than any other arena in the federal government. The implementation and outcomes of the Patient Protection and Affordable Care Act — AKA “Obamacare” — will affect millions of people, from the premiums they pay to the incentives for the health care they receive.

“The goal is get people enrolled,” said Sivak. “A step to that goal is to build a health insurance marketplace. It is so much better to build it in a way that’s open, transparent and enables updates. This is better than a big block of proprietary code locked up in CMS.”

The new Healthcare.gov will fill a yawning gap in the technology infrastructure deployed to support the mammoth law, providing a federal choice engine for the more than thirty different states that did not develop their own health insurance exchanges. The new website, however modern, is just one component of the healthcare insurance exchanges. Others may not be ready by the October deadline. According to a recent report from the Government Accountability Office, the Department of Health and Human Services’ Centers for Medicare & Medicaid Services (CMS) is behind in implementing key aspects of the law, from training workers to help people navigate the process to certifying plans that will sold on the exchanges to determining the eligibility of consumers for federal subsidies. HHS has expressed confidence to the GAO that exchanges will be open and functioning in every state on October 1.

On that day, Healthcare.gov will be the primary interface for Americans to learn about and shop for health insurance, as Dave Cole, a developer at Development Seed, wrote in a blog post this March. Cole, who served as a senior advisor to the United States chief information officer and deputy director of new media at the White House, was a key part of the team that moved WhiteHouse.gov to Drupal. As he explained, the code will be open in two important ways:

First, Bryan pledged, “everything we do will be published on GitHub,” meaning the entire code-base will be available for reuse. This is incredibly valuable because some states will set up their own state-based health insurance marketplaces. They can easily check out and build upon the work being done at the federal level. GitHub is the new standard for sharing and collaborating on all sorts of projects, from city geographic data and laws to home renovation projects and even wedding planning, as well as traditional software projects.

Moreover, all content will be available through a JSON API, for even simpler reusability. Other government or private sector websites will be able to use the API to embed content from healthcare.gov. As official content gets updated on healthcare.gov, the updates will reflect through the API on all other websites. The White House has taken the lead in defining clear best practices for web APIs.

Thinking differently about a .gov

According to Sivak, his team didn’t get directly involved in the new Healthcare.gov until November 2012. After that “we facilitated the right conversations around what to build and how to build it, emphasizing the consumer-facing aspects of it,” he said. “The other part was to figure out what the right infrastructure was going to be to build this thing.”

That decision is where this story gets interesting, if you’re interested in how government uses technology to deliver information to the people it serves. Government websites have not, historically, been sterling examples of design or usability. Unfortunately, in many cases, they’ve also been built at great expense, given the dependence of government agencies on contractors and systems integrators, and use technologies that are years behind the rest of the Web. Healthcare.gov could have gone in the same direction, but for the influence of its young chief technology officer, an “entrepreneur-in-residence” who had successfully navigated the bureaucracies of the District of Columbia and state of Maryland.

“Our first plan was to leverage Percussion, a commercial CMS that we’d been using for a long time,” said Sivak. “The problem I had with that plan was that it wasn’t going to be easy to update the code. The process was complicated. Simple changes to navigation were going to take a month.”

At that point, Sivak did what most people do in this new millennium when making a technology choice: he reached out to his social networks and went online.

“We started talking to people about a better way, including people who had just come off the Obama campaign,” he said. “I learned about the ground they had broken in the political space, from A/B testing to lightweight infrastructure, and started reading about where all that came from. We started thinking about Jekyll as a platform and using Prose.io.”

After Sivak and his team read about Development Seed’s work with Jekyll online, they contacted the startup directly. After a little convincing, Development Seed agreed to do one more big .gov project.

“A Presidential Innovation Fellow used same tech we’re using for several of their projects,” said Cole. “Bryan heard about it and talked to us. He asked where we would go. We wanted to be on Github. We knew there were performance and reliability benefits from building the stack on HTML.”

Jekyll, for those who are unfamiliar with Web development trends, is a way for developers to build a static website from dynamic components. Instead of running a traditional website with a relational database and server-side code, using Jekyll enables programmers to create content like they create code. The end result of this approach is a site that loads faster for users, a crucial performance issue, particularly on mobile devices.

“Instead of farms of application servers to handle a massive load, you’re basically slimming down to two,” said Sivak. “You’re just using HTML5, CSS, and Javascript, all being done in responsive design. The way it’s being built matters. You could, in theory, do the same with application servers and a CMS, but it would be much more complex. What we’re doing here is giving anyone with basic skills the ability to do basic changes on the fly. You don’t need expensive consultants.”

That adds up to cost savings. Sites that are heavily trafficked — as Healthcare.gov can reasonably be expected to be – normally have to use a caching layer to serve static content and add more server capacity as demand increases.

“When we worked with the World Bank, they chose a plan from Rackspace for 16 servers,” said Gundersen. “That added tens of thousands of dollars, with a huge hosting bill every month.”

HHS had similar strategic plans for the new site, at least at first.

“They were planning 32 servers, between staging, production and disaster recovery, with application servers for different environments,” said Cole. “You’re just talking about content. There just needs to be one server. We’re going to have 2, with one for backup. That’s a deduction of 30 servers.”

While Jekyll eliminates the need for a full-blown content management system on the backend of Healthcare.gov (and with it, related costs), the people managing the site still need to be able to update it. That’s where Prose.io comes in. Prose.io is an open source content editor created by Development Seed that gives non-programmers a clean user interface to update pages.

“If you create content and run Jekyll, it requires content editors to know code,” said Cole. “Prose is the next piece. You can run it on your on own servers or use a hosted version. It gives access to content in a CMS-like interface, basically adding a WYSIWYG skin, giving you a text editor in the browser.”

In addition to that standard “what you see is what you get” interface, familiar from WordPress or Microsoft Word, Prose.io offers a couple of bells and whistles, like mobile editing.

“You can basically preview live,” said Cole. “You usually don’t get a full in-browser preview. The difference is that you have that with no backend CMS. It’s just a directory and text files, with a Web interface that exposes it. There are no servers, no infrastructure, and no monthly costs. All you need is a free Web app and Github. If you don’t want to use that, use Git and Github Enterprise.” Update: Cole wrote more about launching Healthcare.gov on the DevelopmentSeed blog on Tuesday.

Putting open source to work

Performance and content management aside, there’s a deeper importance to how Healthcare.gov is being built that will remain relevant for years to come, perhaps even setting a new standard for federal government as a whole: updates to the code repository on Github can be adopted for every health insurance exchange using the infrastructure. (The only difference between different state sites is a skin with the state logo.)

“We have been working in the .gov space for a while,” said Gundersen. “Government people want to make the right decisions. What’s nice about what Bryan is doing is that he’s trying to make sure that everyone can learn from what HHS is doing, in real-time. From a process standpoint, what Bryan is doing is going to change how tech is built. FCC is watching the repository on Github. When agencies can collaborate around code, what will happen? The amount of money we have the opportunity to save agencies is huge.”

Collaboration and cascading updates aren’t an extra, in this context: they’re mission-critical. Sivak said that he expects the new site to be improved iteratively over time, in response to how people are actually using it. He’s a fan of the agile development methodology that has become core to startup development everywhere, including using analytics tools to track usage and design accordingly.

“We’re going to be collecting all kinds of data,” said Sivak. “We will be using tools like Optimizely to do A/B and multivariate testing, seeing what works on the fly and adapting from there. We’re trying to treat this like a consumer website. The goal of this is to get people enrolled in health care coverage and get insurance. It’s not simple. It’s a relatively complex process. We need to provide a lot of information to help people make decisions. The more this site can act in a consumer-friendly fashion, surfacing information, helping people in simple ways, tracking how people are using it and where they’re getting stuck, the more we can improve.”

Using Jekyll and Prose.io to build the new Healthcare.gov is only the latest chapter in government IT’s quiet open source evolution. Across the federal government, judicious adoption of open source is slowly but surely leading to leaner, more interoperable systems.

“The thing that Git is all about is social coding,” said Sivak, “leveraging the community to help build projects in a better way. It’s the embodiment of the open source movement, in many ways: it allows for truly democratic coding, sharing, modifications and updates in a nice interface that a lot of people use.”

Open by design

Sivak has high aspirations, hoping that publishing the code for Healthcare.gov will lead to a different kind of citizen engagement.

“I have this idea that when we release this code, there may be people out there who will help us to make improvements, maybe fork the repository, and suggest changes we can choose to add,” he said. “Instead of just internal consultants who help build this, we will suddenly have legions of developers.”

Not everything is innovative in the new Healthcare.gov, as Nick Judd reported at TechPresident in March: the procurement process behind the new site is complicated and the policy and administrative processes that undergird it aren’t finished yet, by any account.

The end result, however, is a small startup in a garage rebuilding one of the most important federal websites of the 21st century in a decidedly 21st century way: cheaper, faster and scalable, using open source tools and open standards.

“Open by design, open by default,” said Sivak. “That’s what we’re doing. It just makes a lot of sense. If you think about what should happen after this year, all of the states that didn’t implement their systems, would it make sense for them to have code to use as their own? Or add to it? Think about the amount of money and effort that would save.”

That’s a huge win for the American people. While the vast majority of visitors to Healthcare.gov this fall will never know or perhaps care about how the site was built or served, the delivery of better service at lowered cost to taxpayers is an outcome that matters to all.