What’s next for open data in the United States? That was the open question posed at the Center for Data Innovation (CDI) last week, where a panel of industry analysts and experts gathered to discuss the historic open government data … Continue reading
Livestreaming is not new to American politics in 2019. presidential candidates started using uStream in 2008. But it is much easier than ever before. Recent advances in smartphones and wirelesss Internet access have radically improved the capacity of politicians and … Continue reading
On December 21, 2018, the United States House of Representatives voted to enact H.R. 4174, the Foundations for Evidence-Based Policymaking Act of 2017, in a historic win for open government in the United States of America.
The Open, Public, Electronic, and Necessary Government Data Act (AKA the OPEN Government Data Act) is about to become law as a result. This codifies two canonical principles for democracy in the 21st century:
- public information should be open by default to the public in a machine-readable format, where such publication doesn’t harm privacy or security
- federal agencies should use evidence when they make public policy
For the full backstory on what’s in the bill and how it came to pass, read yesterday’s feature.
It’s worth noting that last minute objection did result in two amendments that the Senate had to act upon. Thankfully, on Saturday, December 22nd, the Senate acted, passing the resolution required to send the bill onwards to the president’s desk.
Here’s what changed: First, the text of Title I was amended so that it only applied to CFO Act agencies, not the Federal Reserve or smaller agencies. Title II (the Open Government Act) still applies to all federal agencies.
Second, there was a carve out in Title I “for data that does not concern monetary policy,” which relates to the Federal Reserve, among others.
While the shift weakened the first title of the bill a bit, this was still a historic moment: Congress has passed a law to make open data part of of the US Code.
While the United States is not the first or even the second democracy to pass an open data law – France and Germany have that distinction – this is a welcome advance, codifying the open government data policies, practices, roles and websites (looking at you, Data.gov) that the federal government had adopted over the past decade.
Open government activists, advocates and champions continue to celebrate, online and off.
Victory! Last night the @DataCoalition got the #OPENGovData Act through the Senate, as part of H.R. 4174. Expected to sail through the House TODAY. Sets a presumption, in law, that govt info ought to be published as #opendata, using data standards. https://t.co/jTdM1rIVTW
— Hudson Hollister (@hudsonhollister) December 20, 2018
#Opendata will soon be the way our government publishes information by default!! What a great day for #opendata #opengov #transparency! This would not have been possible w/o @SpeakerRyan @PattyMurray @SenBrianSchatz @SenSasse @RepDerekKilmer. Thank you! #OPENGovDataAct (HR 4174)
— Data Coalition (@DataCoalition) December 21, 2018
Merry Christmas to me! The U.S. Congress has passed an open data law. pic.twitter.com/tDwo8j8UCB
— Rebecca Williams (@internetrebecca) December 21, 2018
Excited to see the Open Government Data Act pass the House today! H.R. 4174 will enable libraries to provide businesses, researchers and students with valuable data that fuels innovation and economic growth. #OPENGovData #opendata
— Gavin Baker (@OpenGavin) December 21, 2018
“The bipartisan passage of the Foundations for Evidence-Based Policymaking Act is a significant step toward a more efficient, more effective government that uses evidence and data to improve results for the American people,” said Michele Jolin, CEO and co-founder of Results for America, in a statement. “We commend Speaker Ryan, Senator Murray and their bipartisan colleagues in both chambers for advancing legislation that will help build evidence about the federally-funded practices, policies and programs that deliver the best outcomes. By ensuring that each federal agency has an evaluation officer, an evaluation policy and evidence-building plans, we can maximize the impact of public investments.”
“The OPEN Government Data Act will ensure that the federal government releases valuable data sets, follows best practices in data management, and commits to making data available to the public in a non-proprietary and electronic format,” said Daniel Castro, in a statement. “Today’s vote marks a major bipartisan victory for open data. This legislation will generate substantial returns for the public and private sectors alike in the years to come.”
“The passage of the OPEN Government Data Act is a win for the open data community”, said Sarah Joy Hays, Acting Executive Director of the Data Coalition, in a statement. “The Data Coalition has proudly supported this legislation for over three years, along with dozens of other organizations. The bill sets a presumption that all government information should be open data by default: machine-readable and freely-reusable. Ultimately, it will improve the way our government runs and serves its citizens. This would not have been possible without the support of Speaker Paul Ryan (WI-1-R), Senators Patty Murray (WA-D), Brian Schatz (HI-D), Ben Sasse (NE-R), and Rep. Derek Kilmer (WA-6-D). Our Coalition urges the President to promptly sign this open data bill into law.”
Congratulations to everyone who has pushed for this outcome for years.
[Image Credit: Sunlight Foundation]
This post has been updated, and corrected: France was ahead of Germany in enacting an open data law.
The arc of open government in United States is long, but perhaps it will bend towards transparency and accountability as 2018 comes to a close, in an unlikely moment in our history. After years of dedicated effort by advocates and bipartisan leadership in both houses of Congress, the Open, Public, Electronic, and Necessary Government Data Act (AKA the OPEN Government Data Act) is about to become law after the United States Senate passed the bill as part of H.R. 4174 on December 19.
Senator Brian Schatz (D-HI) shared the news in a tweet last night:
We just passed a bill that requires all data the govt collects (that isn’t secret or private) to be machine readable and interoperable. It’s data that taxpayers paid for and they deserve access – weather, traffic, census, budget numbers – it’s your info and you should have it.
— Brian Schatz (@brianschatz) December 19, 2018
…followed by Representative Derek Killmer (D-
Last night the Senate passed a bill I introduced in the House called the Open Government Data Act! There are so many possibilities for our economy when folks can access data they paid to develop and fund through the nation’s largest angel investor, Unce Sam. https://t.co/BvMX54mcCY
— Rep. Derek Kilmer (@RepDerekKilmer) December 20, 2018
Here’s Schatz speaking about the bill at a Data Coalition event last winter:
Two steps remain: passage of the bill in the House and President Donald J. Trump signing it into law. Barring a scheduling issue or unexpected change (keep an eye out for shenanigans on the House floor today), the nation is close to a historic codification of two powerful principles:
- public information should be open by default to the public in a machine-readable format, where such publication doesn’t harm privacy or security
- federal agencies should use evidence to make public policy
Along with making open government data the default in U.S. government and requiring the White House Office of Management and Budget to oversee enterprise data inventories for every agency, the bill would require federal agencies to do the following, as listed in the summary from the Law Library of Congress:
This bill requires departments and agencies identified in the Chief Financial Officers Act to submit annually to the Office of Management and Budget (OMB) and Congress a plan for identifying and addressing policy questions relevant to the programs, policies, and regulations of such departments and agencies.
The plan must include: (1) a list of policy-relevant questions for developing evidence to support policymaking, and (2) a list of data for facilitating the use of evidence in policymaking.
The OMB shall consolidate such plans into a unified evidence building plan.
The bill establishes an Interagency Council on Evaluation Policy to assist the OMB in supporting government-wide evaluation activities and policies. The bill defines “evaluation” to mean an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency.
Each department or agency shall designate a Chief Evaluation Officer to coordinate evidence-building activities and an official with statistical expertise to advise on statistical policy, techniques, and procedures.
The OMB shall establish an Advisory Committee on Data for Evidence Building to advise on expanding access to and use of federal data for evidence building.
Open, Public, Electronic, and Necessary Government Data Act or the OPEN Government Data Act
This bill requires open government data assets to be published as machine-readable data.
Each agency shall: (1) develop and maintain a comprehensive data inventory for all data assets created by or collected by the agency, and (2) designate a Chief Data Officer who shall be responsible for lifecycle data management and other specified functions.
The bill establishes in the OMB a Chief Data Officer Council for establishing government-wide best practices for the use, protection, dissemination, and generation of data and for promoting data sharing agreements among agencies.
While the United States would not be not the first democracy to pass such a law, it would be a welcome advance, codifying many aspects of the open government data policies that have been developed and promulgated in the federal government over the past decade.
How did open government data get into the US Code?
This was no accident of fate or circumstance: This bill, which was previously passed by the House last month, was sponsored by the Speaker of the House, Paul Ryan. It’s an important element of his legislative legacy, and one that can and should earn praise – unlike other aspects of his time with the gavel.
It’s taken years of advocacy and activism by a broad coalition to get here, including the Sunlight Foundation, the EFF, Business Software Alliance, Center for Data Innovation, the Scholarly Publishing and Academic Resources Coalition, American Library Association, the R Street Institute, among many others, and bipartisan efforts on both sides of the aisle. Senator Ben Sasse (R-NE) co-sponsored the OGDA in the Senate, with 5 others, and former Representative Blake Farenthold (R-TX) cosponsored it in House, with 12 others.
That original bill almost made it into law in 2016, when the Senate passed OGDA by unanimous consent, but the House didn’t move it before the end of the 115th Congress. In September 2017, when it was poised to pass Congress as part of the National Defense Authorization Act., before it was stripped from the final version.
In October 2017, the text of the Open Government Data Act, however, was incorporated into H.R. 4174, the Foundations for Evidence-Based Policymaking Act of 2017,
The new bipartisan, bicameral companion legislation was introduced on October 31, 2017 by Speaker of the House Paul Ryan (R-WI) and Senator Patty Murray (D-WA) to enact the recommendations contained in the final report from the Commission of Evidenced-Based Policy.
While it has been watered down a bit, what I argued then is still true today: the bill “offers a genuine opportunity to not only improve how the nation makes decisions but embed more openness into how the federal government conducts the public’s business.”
The addition of OGDA into that bill was “an important endorsement of open government data by one of the most powerful politicians in the world” and “a milestone for the open movement, an important validation of this way of making public policy, and the fundamental principles of data-driven 21st century governance.”
The OGDA was one of the primary legislative priorities for me during my years as a senior analyst and then deputy director at the Sunlight Foundation, along with Freedom of Information Act Reform and Honest Ads Act.
I picked up the transparency baton on OGDA from former Sunlight analyst Matt Rumsey, Sunlight federal policy manager Sean Vitka, OpenGov Foundation founder Seamus Kraft, and Data Coalition founder Hudson Hollister, who drafted the original open data bill, working to make the principle that “public data created with taxpayer dollars should be available to the public in open, machine-readable forms when doing so does not damage privacy or national security” the law.
This is a huge win for public access to public information that every American can and should celebrate today. Special thanks for this victory are due to Christian Hoehner, policy director for the Data Coalition, who did extraordinary yeoman’s work getting this through Congress, Sasha Moss of the R Street Institute, Hollister, Daniel Schuman of Demand Progress, Daniel Castro and Joshua New at the Center for Data Innovation, and Gavin Baker from the American Library Association, some with whom I went to Congress with me to meet with staff over the years and advocated for the bill on and offline.
The passage of this bill won’t mean that the scanned images of spreadsheets that agencies still send in response to FOIA requests will magically go away tomorrow, but journalists, watchdogs and the public can now tell civil servants that they’re now behind the times: open government data is now the default in the USA! Please publish our data on the agency website in a structured format and let the public know.
[Image Credit: Sunlight Foundation]
This morning, James Grimaldi reported that Ajit Pai, chairman of the Federal Communications Commission, has told two U.S. Senators that he has proposed “to rebuild and re-engineer” the agency’s online electronic comment system “to institute appropriate safeguards against abusive conduct.” … Continue reading
When I asked whether when or if it is acceptable for the United States government to charge companies, journalists and the public for government data, citing the example of paywalled immigration data, the chief information officer of the United States told me that “it’s part of the commercial equation” and that it was “actually a discussion point for the strategy” in her office in the White House Office of Management and Budget.
“I don’t have a specific answer,” Suzette Kent went on. “That is something that we’re looking at because there’s many tenets of it. There’s some data the government collects & document on behalf of the american public that may have the mode. There’s other types of data, that people are asking for. It’s a broad spectrum and one we are going to continue to explore.” Kent was speaking at the Data Coalition’s Data Demo Day on June 6 in Washington, DC. Video of the keynote speech she gave on data is embedded below:
When asked about the continued disclosure of data in PDFs and non-machine readable forms by federal agencies, despite President Barack Obama’s 2013 executive order, Kent said simply that she advocates compliance with every executive order and law and cited a Cross-Agency Priority goal to remove paper from agency systems.
Charging for public data is not a new topic or debate, but it has continued to be relevant during the Trump administration, when new concerns have grown about government data access, collection, and quality.
As I wrote back in 2014, local, state and national governments across the United States and around the world can and do charge for access to government data.
While some developers in Europe advocate charging for public sector information (PSI) as a way to ensure higher service levels and quality, adding fees does effectively charge the public for public access and has frequently been used as a barrier to press requests:
A city hall, state house or government agency charging the press or general public to access or download data that they have already paid for with their tax revenues, however, remains problematic.
It may make more sense for policy makers to pursue a course where they always make bulk government data available for free to the general public and look to third parties to stand up and maintain high quality APIs, based upon those datasets, with service level agreements for uptime for high-volume commercial customers.
Instead of exploring a well-trodden path, the United States government should follow the money and determine which data is agencies are currently charging for under public records requests or other means, using FOIA demand to drive open data disclosure.
This past week, Facebook launched a new political ad transparency website. Facebook believes that “shining a light on ads” will increase transparency, which in turn “will lead to increased accountability & responsibility over time – not just for Facebook but advertisers as well.“
I think they’re right — which should be no surprise given my focus on advocating for more political transparency in Washington over the two years I spent at the Sunlight Foundation — but reviewing reports of unlabeled political ads is going to be hard.
Overall, this site is a welcome step towards more transparency, but misses the mark. The site only “exceeds expectations” if you think a search interface that exposes no underlying data is sufficient to inform the public and regulators.
In my initial assessment, I concur with journalists who found Facebook’s new political ad system is missing a lot, as ProPublica reported. (Please install ProPublica’s political ad collector so they can inform the public about how well Facebook’s tool actually works.)
It was easy to use @Facebook‘s new archive of “ads with political content” – essentially a simple search tool for paid political ads that have run since May 7, 2018 – once I got on my laptop and logged in. I found recent ads that matched Trump, Clinton, gun control & corruption. pic.twitter.com/Fhx0lrMzBE
— Alex Howard (@digiphile) May 24, 2018
On the one hand, it was easy to use Facebook’s new archive of “ads with political content” – essentially a simple search tool for paid political ads that have run since May 7, 2018 – at least once I got on my laptop and logged into Facebook. I found recent ads that matched Trump, Clinton, gun control and corruption.
If you click on “see ad performance,” you can learn more about each ad.
If you click “see ad performance,” you see the ad content, who paid, when it was active, how many impressions it received, total spent, & breakdown of audience by age, gender & location.
But clicking “view all ads” brings you to aggregate search results, NOT the page or a profile pic.twitter.com/8XtzmWqdYy
— Alex Howard (@digiphile) May 24, 2018
If you click on the username, you arrive at the Page behind the ads. Unfortunately, there’s no tab for political ads or link to this archive. It’s hard to see how folks will find them, without it.
If you click on the username – in this case, Donald Trump, @realDonaldTrump‘s campaign account on @Facebook – you arrive at the Page behind the ads. Unfortunately, there’s no tab for political ads or link to this archive. It’s hard to see how folks will find them, without it. pic.twitter.com/EASlccVAhF
— Alex Howard (@digiphile) May 24, 2018
As I noted on Twitter, however, there’s one more critical wrinkle: you can’t get to the page unless you’re logged into Facebook!
This would be hilariously ironic, if it weren’t for the context of Russian interference and how Facebook handled it. Self-regulation is not enough.
As sociology professor Zeynep Tufecki noted, no one — whether member of the public, the press, watchdog, academic, regulator or legislator – should have to agree to Facebook’s Terms of Service and become a user to access political data.
😱 You shouldn’t have to agree to Facebook TOS in order to access information about political reports. In fact, that is a core problem. I’ve seen examples where schools put *emergency* information on Facebook and people have to agree to FB TOS to learn whether children are safe. https://t.co/6kmsOXgYgu
— zeynep tufekci (@zeynep) May 24, 2018
To Facebook’s credit, the director of product at Facebook, Rob Leathern, responded publicly to Tufecki on Twittter, stating that this page is a first step:
“More ways are coming to make the ads with political content and information more accessible to people. One of those is an API, another is exploring opening the archive to people not on Facebook. We started with the Facebook community to see how they use the tool and gain feedback from third parties, including our newly-formed Election Commission. We’ll continue to update on our progress.”
If Facebook started with open data with no log-in, they could have gotten feedback from third parties like the Center for Responsive Politics or the public. No one should have to be part of Facebook’s “community” to understand who is buying electioneering on the platform, for whom, and what’s being shown.
As I commented to Leathern, if Facebook is only “exploring” making this archive open to people not on Facebook, then it is not implementing the Honest Ads Act, as its staff has claimed to Congress and the public. I asked Facebook to post a public ad file as bulk open data on the open Web.
Leathern told me that “we have prioritized getting the archive in the hands of people to use (as of today) + will follow up soon with an archive API. Thank you for the feedback, we are definitely listening.”
That’s good news, but not good enough.
Real transparency at Facebook will look like a public file of all paid political ads that are disclosed on a public website with bulk open data downloads and an API, none of which require the public to log into the site.
The good news is that I think Facebook understands this page as a start, not an end. In a post that closes matches what he told me, Leathern wrote that they’re “working closely” with a new “Election Commission” to launch an API for the archives.
It’s good news, but no deadline cited.
It’s hard for me not to be happy that Facebook is finally explicitly embracing political ad transparency in words and (some) deeds, including public soul searching about what constitutes a political ad and a policy.
It’s just long overdue. Ultimately, elected representatives should be the ones to enact standards for transparency for political ads online after debate, not tech company executives.
Until Congress and other legislatures around the world empower regulators like Federal Election Commission by updating electioneering rules and enacting standards for disclaimers and disclosures, however, I’m glad to see positive actions.
I hope Facebook, its founder and its staff deliver on its most recent promises and their public obligations. Given past, current or predictable interference, opacity is unpatriotic.
When the president or others with access to his Twitter account block American citizens from following @realDonaldTrump based upon the viewpoints they express, it violates their First Amendment rights.
In a historic decision, a federal judge ruled today that it is unconstitutional for President Donald J. Trump to block his critics on Twitter, as portions of @realDonaldTrump account constitute a public forum, which means blocking them based on their political speech violates the First Amendment:
We hold that portions of the @realDonaldTrump account — the “interactive space” where Twitter users may directly engage with the content of the President’s tweets — are properly analyzed under the “public forum” doctrines set forth by the Supreme Court, that such space is a designated public forum, and that the blocking of the plaintiffs based on their political speech constitutes viewpoint discrimination that violates the First Amendment. In so holding, we reject the defendants’ contentions that the First Amendment does not apply in this case and that the President’s personal First Amendment interests supersede those of plaintiffs.
This is a historic win for the First Amendment and the public’s right to access official statements and participate in public discourse regarding those statements.
As I highlighted last year, tweets by @realDonaldTrump are official statements from the President, which means that the public has a right to equal access and participation around them, even when their speech is hosted on a private platform. The public interest argument was clear then:
“A president’s statements are not just made for people who voted for him or support his policies or politics.
Unfortunately, Trump is not alone: other local, state and federal politicians are also blocking their constituents on Twitter.
Doing so sends the wrong message to the public about whom they serve. Listening and responding to members of the public that they represent is a minimum expectation for public servants in any democratic state, whether those voices are raised in protest, petition, email, send letters or reply on social media. While there are practical challenges to making sense of millions of emails, tweets, call or letters, blocks are not the solution to filter failure.”
No President should block Americans from reading his official statements, replying or interacting with others here.
No other public servant should block constituents, either, from city councilors and alderman to judges, governors and mayors.
On Twitter, officials and politicians who have blocked constituents now consider policies to Mute accounts if someone is being vile or abusive, with transparency about guidelines and use. Users who abuse one another are already subject to accountability for violations of @Twitter rules, which could be reported by officials or civil society.
As the judge noted, addressing President Trump blocking people is legally tricky.
While we reject defendants’ categorical assertion that injunctive relief cannot ever be awarded against the President, we nonetheless conclude that it is unnecessary to enter that legal thicket at this time. A declaratory judgment should be sufficient, as no government official — including the President — is above the law, and all government officials are presumed to follow the law as has been declared.
President Trump should acknowledge the ruling and follow the law, unblocking everyone. Whether he’ll embrace such a change on his social platform of choice isn’r clear at all — especially given his refusal to follow security protocols for his iPhone, despite the risk of nation states spying on him.
In the wake of this ruling, the president should acknowledge the ruling in a video & tweeted post, work with Twitter to unblock everyone, and apologize for engaging in viewpoint-based discrimination and chilling the speech of his constituents, the American public.
But I doubt Trump will.
So, here’s a different idea. It would be an unprecedented move for an unprecedented presidency, but I hope Jack Dorsey and his board will seriously consider removing the Block feature from all official government accounts verified by Twitter.
If code is law and law is now encoded, one way for Twitter to embrace its DNA as a 21st century platform for free speech and make open, public access to official statements the default, putting pressure on Facebook, Google and others to follow.
The Bot Wars, begun they have. Over the past two years, automated social media accounts and fraudulent regulatory filings have been used by anonymous parties to obscure public opinion, distort public discourse, and corrupt the integrity of rulemaking in the … Continue reading
The VA is being neither open nor transparent about its missing open government plans or policies. On March 29, 2018, I made a Freedom of Information Act request to the United States Department of Veterans Affairs (VA) in which I … Continue reading