Congress releases open data on bill status

us-capitol-dome-sun

Imagine searching Facebook, Google or Twitter for the status of a bill before Congress and getting an instant result. That future is now here, but it’s not evenly implemented yet.

When the Library of Congress launched Congress.gov in 2012, they failed to release the data behind it. Yesterday, that changed when the United States Congress started releasing data online about the status of bills.

For the open government advocates, activists and civic hackers that have been working for over a decade for this moment, seeing Congress turn on the data tap was a historic shift.

//platform.twitter.com/widgets.js

Congressional leaders from both sides of the aisle applauded the release of House and Senate bill status information by the U.S. Government Printing Office and Library of Congress.

“Today’s release of bill status information via bulk download is a watershed moment for Congressional transparency,” said House Majority Leader Kevin McCarthy (R-CA), in a statement. “By modernizing our approach to government and increasing public access to information, we can begin to repair the relationship between the people and their democratic institutions. The entire Congressional community applauds the dedication of the Legislative Branch Bulk Data Task Force, the Office of the Clerk, the House Appropriations Committee, GPO, and the Library of Congress, which worked together to make this progress possible.”

“Building off previous releases of bills and summaries, today’s release of bill status information largely completes the overarching goal of providing bulk access to all the legislative data that traditionally has been housed on Thomas.gov and now also resides on Congress.gov,” said Democratic Whip Steny Hoyer (D-MD). “This is a major accomplishment that has been many years in the making. It goes a long way toward making Congress more transparent and accessible to innovation through third party apps and systems. I applaud the dedicated civil servants who made this possible at the Legislative Branch service agencies, and I want to thank the Bulk Data Task Force for their leadership in this effort. While this largely completes a major goal of the Task Force, I look forward to continuing to workwith them to further modernize the U.S. Congress.”

The impact of open government data releases depend upon publicy and political agency. Releasing the states of bills before Congress in a way that can be baked in by third party apps and services is a critical, laudable step in that direction, but much more remains to be done in making the data more open and putting it to use and re-use. If the Library of Congress opens up an application programming interface for the data that supplies both Congress.gov and the public, it would help to reduce the asynchrony of legislative information between the public and elites who can afford to pay for Politico’s Legislative Compass or Quorum Analytics that is the status quo today.

In an era when Congress job approval ratings and trust in government are at historic lows, the shift didn’t make news beyond the Beltway. Govtrack.us, which is based upon data scraped from the Library of Congress, has been online for years. Until this XML data is used by media and technology companies in ways that provide the public with more understanding of what Congress is doing on their behalf and give them more influence in that legislative process, that’s unlikely to change quickly.

15 key insights from the Pew Internet and Life Project on the American public, open data and open government

Today, a new survey released by the Pew Research Internet and Life Project provided one of the most comprehensive snapshots into the attitudes of the American public towards open data and open government to date. In general, more people surveyed are guardedly optimistic about the outcomes and release of open data, although that belief does vary with their political views, trust in government, and specific areas.  (Full disclosure: I was consulted by Pew researchers regarding useful survey questions to pose.)

mixed-hopes-open-data-improve-pew

“Trust in government is the reference that people bring to their answers on open government and open data,” said John Horrigan, the principal researcher on the survey, in an interview. “That’s the frame of reference people bring. A lot of people still aren’t familiar with the notion, and because they don’t have a framework about open data, trust dominates, and you get the response that we got.”

more-trust-gov-more-benefits-open-data

While majorities of the American public use applications and services that use government data, from GPS to weather to transit to health apps, relatively few are aware that data produced and released by government drives them.

“The challenge for activists or advocates in this space will be to try to make the link between government data and service delivery outcomes,” said Horrigan. “If the goals are to make government perform better and maybe reverse the historic tide of lowered trust, then the goal is to make improvements real in delivery. If this is framed just as argument over data quality, it would go into an irresolvable back and forth into the quality of government data collection. If you can cast it beyond whether unemployment statistics are correct or not but instead of how government services improve or saved money, you have a chance of speaking to wether government data makes things better.”

The public knowledge gap regarding this connection is one of the most important points that proponents, advocates, journalists and publishers who wish to see funding for open data initiatives be maintained or Freedom of Information Act reforms pass.

“I think a key implication of the findings is that – if advocates of government data initiatives hope that data will improve people’s views about government’s efficacy – efforts by intermediaries or governments to tie the open data/open government to the government’s collection of data may be worthwhile,” said Horrigan. “Such public awareness efforts might introduce a new “mental model” for the public about what these initiatives are all about. Right now, at least as the data for this report suggests, people do not have a clear sense of government data initiatives. And that means the context for how they think about them has a lot to do with their baseline level of trust in the government – particularly the federal government.”

Horrigan suggested thinking about this using a metaphor familiar to anyone who’s attended a middle school dance.

“Because people do engage with the government online, just through services, it’s like getting them on a big dance floor,” he suggested. “They’re on the floor, where you want them, but they’re on the other part of it. They don’t know that there’s another part of the dance that they’d like to see or be drawn to that they’d want to be in. There’s an opportunity to draw them. The good news that they’re on the dance floor, the bad news is they don’t know about all of it. Someone might want to go over and talk to them an explain that if you go over here you might have a better experience.”

Following are 13 more key insights about the public’s views regarding the Internet, open data and government. For more, make sure to read the full report on open government data, which is full of useful discussion of its findings.

One additional worth noting before you dive in: this survey is representative of American adults, not just the attitudes of people who are online. “The Americans Trends Panel was recruited to be nationally representative, and is weighted in such a way (as nearly all surveys are) to ensure responses reflect the general population,” said Horrigan. “The overall rate of internet use is a bit higher than we typically record, but within the margin of error. So we are comfortable that the sample is representative of the general population.”

Growing number of Americans adults are using the Internet to get information and data

While Pew cautions that the questions posed in this survey are different from another conducted in 2010, the trend is clear: the way citizens communicate with government now includes the Internet, and the way government communicates with citizens increasingly includes digital channels. That use now includes getting information or data about federal, state and local government.

internet-use-info-government-pew

College-educated Americans and millennials are more hopeful about open data releases

college-grad-millennials-more-hopeful-open-data-pew

Despite disparities in trust and belief in outcomes, there is no difference in online activities between members of political parties

no-difference-trust-parties-in-online-activities

Wealthier Americans are comfortable with open data about real estate transactions but not individual mortgages

wealthy-comfortable-transaction-data-not-mortgages-pew

This attitude is generally true across all income levels.

people-comfortable-data-sharing-not-mortgage-pew

College graduates, millennials and higher-income adults are more likely to use data to monitor government performance

About a third of college grads, young people and wealthy Americans have checked out performance data or government contracting data, or about 50% more than other age groups, lower income or non-college grads.

college-grads-higher-income-monitor-performance-pew

The ways American adults interact with government services and data digitally are expanding

ways-people-interact-government-info-pew

But very few American adults think government data sharing is currently very effective:

few-think-govt-data-sharing-effective-pew

A small minority of Americans, however, have a great deal of trust in federal government at all:

majorities-low-trust-government-pew

In fact, increasing individual use of data isn’t necessarily correlated with belief in positive outcomes:

Pew grouped the 3,212 respondents into four quadrants, seen below, with a vertical axis ranging from optimism to skepticism and a horizontal axis that described use. Notably, more use of data doesn’t correlate to more belief in positive outcomes.

“In my mind, you have to get to the part of the story where you show government ran better as a result,” said Horrigan. “You have to get to a position where these stories are being told. Then, at least, while you’re opening up new possibilities for cynicism or skepticism, you’re at least focused on the data as opposed to trust in government.”

quadrants-belief-open-data

tech-profiles-quadrants-open-data-pew

Instead…

Belief in positive outcomes from the release of open data is correlated with a belief that your voice matters in this republic:

belief-voice-matters-believe-open-data-improves-pew

If you trust the federal government, you’re more likely to see the benefit in open data:

trust-fed-govt-more-likely-to-see-benefit

But belief in positive outcomes from the release of open data is related to political party affiliation:

democrats-more-likely-to-believe-open-data-pay-off-pew

Put simply, Democrats trust the federal government more, and that relates to how people feel about open data released by that government.

democrats-trust-fed-govt-more

 

Political party has an impact upon the view of open data in the federal government

One challenge is that if President Barack Obama says “open data” again, he may further associate the release of government data with Democratic policies, despite bipartisan support for open government data in Congress. If a Republican is elected President in November 2016, however, this particular attitude may well shift.

“That’s definitely the historic pattern, tracked over time, dating to 1958,” said Horrigan, citing a Pew study. “If if holds and a Republican wins the White House, you’d expect it to flip. Let’s say that we get a Republican president and he continues some of these initiatives to make government perform better, which I expect to be the case. The Bush administration invested in e-government, and used the tools available to them at the time. The Obama administration picked it up, used the new tools available, and got better. President [X] could say this stuff works.”

democrats-more-upbeat-view-open-data

The unresolved question that we won’t know the answer to until well into 2017, if then, is whether today’s era of hyper-partisanship will change this historic pattern.

There’s bipartisan agreement on the need to use government data better in government. Democratss want to improve efficiency and effectiveness, Republicans want to do the same, but often in the context of demonstrating that programs or policies are ineffective and thereby shrink government. If the country can rise about partisan politics to innovate government, awareness of the utility of releases will grow, along with support for open data will grow.

“Many Americans are not much attuned to government data initiatives, which is why they think about them (in the attitudinal questions) through the lens of whether they trust government,” said Horrigan. “Even the positive part of the attitudinal questions (i.e., the data initiatives can improve accountability) has a dollop of concern, in that even the positive findings can be seen as people saying: ‘These government data initiatives might be good because they will shine more light on government – which really needs it because government doesn’t perform well enough.’ That is an opportunity of course – especially for intermediaries that might, through use of data, help the public understand how/whether government is being accountable to citizens.”

That opportunity is cause for hope.

“Whether it is ‘traditional’ online access for doing transactions/info searches with respect to government, or using mobile apps that rely on government data, people engage with government online, “said Horrigan. “That creates the opportunity for advocates of government data initiatives to draw citizens further down the path of understanding (and perhaps better appreciating) the possible impacts of such initiatives.”

In a step towards sunlight, United States begins to publish a national data inventory

20130929-142228.jpg
Last year, a successful Freedom of Information request for the United States enterprise data inventory by the Sunlight Foundation was a big win for open government, nudging Uncle Sam towards a better information policy through some creative legal arguments. Today, the federal government started releasing its enterprise indices at data.gov. You can browse the data for individual agencies, like the feed for the Office for Personnel Management, using a JSON viewer like this one.

“Access to this data will empower journalists, government officials, civic technologists, innovators and the public to better hold government accountable,” said Sunlight Foundation president Chris Gates, in a statement. “Previously, it was next to impossible to know what and how much data the government has, and this is an unprecedented window into its internal workings. Transparency is a bedrock principle for democracy, and the federal government’s response to Sunlight’s Freedom of Information request shows a strong commitment to open data. We expect to see each of these agencies continue to proactively release their data inventories.”

Understanding what data an organization holds is a critical first step in deciding how it should be stored, analyzed or published, shifting towards thinking about data as an asset. That’s why President Barack Obama’s executive order requiring federal agencies to catalog the data they have was a big deal. When that organization is a democratic government and the data in question was created using taxpayer funds, releasing the inventory of the data sets that it holds is a basic expression of open and accountable government.

FCC Commissioners take agency to task for opacity around rulemaking

Today, Federal Communications Commission​ Commissioner Ajit Pai​ released a statement & fact sheet on the non-public FCC’s proposal for Open Internet Rules, which would be applied to regulating broadband Internet service providers. He tweeted out a picture of himself holding the document (above) several days ago. In his statement today, Pai said that the American people are being misled about what’s in the rules and broke out an outline of the 300-page document. Reporting and analysis of Pai’s objections to the proposal may be found at the New York Times and National Journal. Subsequent to Pai’s statement and press conference, FCC Special Counsel for External Affairs Gigi Sohn tweeted out a series of rebuttals, including a clarification regarding the size of the document.

Here, I’ll focus on something else: the commissioner took FCC Chairman Tom Wheeler​ to task for not releasing them ahead of the scheduled vote on February 26th, in DC, although Pai did note that doing so would be “unprecedented.” In fact, FCC commissioner Mike O’Rielly tweeted much the same thing, stating that “all FCC meeting items should be made public when circulated to Commissioners. ”

O’Rielly published a blog post on advanced posting of Commission items back in January, following a previous post in August 2014.

In August, I wrote a blog post urging the Commission to post on its website the actual text of the items to be considered at our Open Meetings at the same time they are provided to Commissioners. I made the suggestion because the inability of the public to obtain a complete picture of what is in a pending notice of proposed rulemaking or order routinely leads to confusion over what exactly is at stake. Making matters worse, Commissioners are not allowed to reveal the substantive details to outside parties. We can’t even correct inaccurate impressions that stakeholders may have received, and we are barred from discussing what changes we are seeking. This barrier to a fulsome exchange can be extremely frustrating for all involved. Despite positive feedback from people at the FCC, outside parties, Members of Congress, [1] and the general public, four months later, we have yet to post a single meeting item in advance. Moreover, the lack of full disclosure and transparency has continued to be a problem as some parties have not been fully briefed on recent items, such as the recently adopted 911 Reliability NPRM, while others are not briefed at all. The reason that nothing has happened, I am told, is that there are two basic concerns with the proposal: 1) that it could be harder to comply with the Administrative Procedure Act (APA); and 2) that it could be more difficult to withhold documents under the Freedom of Information Act (FOIA). I do not find either argument persuasive or insurmountable

Wheeler took a question on this issue in a press conference last month: “The precedent here, through Democratic chairmen and Republican chairmen, has always been that just like, you know, in the court system, and elsewhere that when you’re handling cases like this, you have an internal discussion and then you release what the result of that vote is. And you don’t change that decades of precedent overnight without following the procedures to review questions like that.”

Asked for comment, FCC press secretary Kim Hart Jonson​ said that “Chairman Wheeler circulated his proposal to his fellow Commissioners three weeks before the planned vote in accordance with long-standing FCC process. We are confident the other Commissioners will give the proposal an exhaustive review. The Chairman looks forward to receiving their input and releasing it to the public after the February 26 vote.”

What this boils down to is that publishing proposed rules before a vote is technically legal but has never been done before. As Pai himself noted, “FCC rules prohibit disclosure of nonpublic information except as authorized by the Chairman.” Then-FCC chairman Julius Genachowski didn’t pull the full version of the 2010 Open Internet rules up until right before Christmas, well after the vote. The standard FCC procedure for rulemaking is to circulate the draft rules at least 3 weeks prior to a vote, incorporate edits received from FCC commissioners, then finalize everything. In practice, however, some commissioners have complained that edits may be made right up until a vote. Once rules have been voted upon, they’re published in the federal register.

As long-time readers know, I was critical of that decision at the time, and remain so. To reiterate what I said them, Genachowski made a commitment to a more open, transparent and data-driven F.C.C. under President Obama’s Open Government Directive. In many respects, in its first year of open government, the agency made commendable progress, with strides towards taking public comment through e-rulemaking at OpenInternet.gov, Broadband.gov and Reboot.FCC.gov. The sites were deployed by an able new media team that has used online communications in unprecedented ways. The chairman and his managing director, Steven Van Roeckel, both deserved credit for their plans to reboot FCC.gov as a platform for government including the use of APIs and open source technologies like Drupal.

When it comes to the question of whether the public could see the proposed Open Internet rules before the commissioners vote upon it, the agency fell short of its transparency pledge. I have not found a legal precedent that explicitly gives the agency authority to keep the text of a proposed rule secret until it is voted upon by the Commission. While it is true that the FCC does not appear to be under no legal obligation to do so, given that the members of the commission presumably had to negotiate on the details of the final rules for vote, the decision not to share a version publicly may have made such discussion more flexible.

The choice not to post the proposed rules online before the most reced vote is an example of the same level of government transparency in the creation of important regulation as before, not more. It’s possible that the legal context for how the FCC operates might change in the future, if a bill by Senator Dean Heller (R-NV) were to be passed. Per his office, the FCC Reform Act would:

“provide more opportunities for the public to see pending FCC action by publishing the exact rule or amendment to a rule for at least 21 days and by allowing any Commissioner to ask for a vote on any order issued by a bureau. *Enhance consistency and transparency in the Commission’s operations by requiring the FCC to establish procedures for handling extensive new data submitted towards the end of a comment period, adequate review and deliberation regarding pending orders, providing the status of open rulemaking proceedings, minimum public review periods for statistical reports and ex parte communications. *Empower the Commission to operate more efficiently through reform of the “sunshine” rules, allowing a bipartisan majority of Commissioners to meet for collaborative discussions subject to transparency safeguards.

A provision that the FCC must “identify a market failure, consumer harm, or regulatory barrier to investment before adopting economically significant rules,” however, may well mean that Democrats in the Senate won’t support its advance. As always with these net neutrality stories, to be continued.

UPDATE: Commissioner Pai tweeted another picture of a revised plan February 21, including his intention to vote against it and again noting that the public cannot see it before the vote. In the tweet, he says the plan is now 317 pages, which means it has been trimmed by 15 pages.