Government secrecy, as measured by censorship or non-responsiveness under the Freedom of Information Act, is at an all-time high during the Trump administration. The Freedom of Information Act is getting worse under Trump for a variety of reasons. Continued secrecy … Continue reading
Like Google, Facebook and Twitter, Snapchat now has an online political ad library. That’s good news: every technology company that accept money to run issues and campaign ads should have a political ad file – particularly one that has aspirations … Continue reading
For a decade, I’ve tracked the contours of open data, digital journalism, open source software and open government, publishing research on the art and science of data journalism that explored and tied together those threads.
This week, I’m proud to announce that a new book chapter on open data, journalists, and the media that I co-authored with data journalism advisor Eva Constantaras has been published online as part of “The State of Open Data: Histories and Horizons!”
We joined 43 other authors in a 18-month project that reflected on “10 years of community action and review the capacity of open data to address social and economic challenges across a variety of sectors, regions, and communities.”
The publication and the printed review copy I now possess is the end of a long road.
Back in January 2018, I asked a global community of data journalists, open government advocates, watchdogs and the public for help documenting the “State of Open Data” and journalism for a new project for Open Data for Development. While I was at the Sunlight Foundation, I seeded the initial network scan on journalists, media and open data.
Over the past year, people and organizations from around the world weighed in – & Eva Constantaras joined me as co-editor & lead author, creating a “network scan” of the space. The writing project traveled with me, after I left Sunlight, and over this winter we edited and synthesized the scan into a polished chapter.
The book was originally going to be introduced at the 5th International Open Government Data Conference in Buenos Aires, in the fall of 2018, but will instead be officially launched at the Open Government Partnership’s global summit in Ottawa, Canada at the end of May 2019.
The book, which was funded by the International Research Centre and supported by the Open Data for Development (OD4D) Network. As a result of that support, “The State of Open Data” was also published in print by African Minds, from whom it may also be purchased. (While I was paid to edit and contribute, I don’t receive any residuals.)
Last night, I joined Tim Davies and other authors at the OpenGov Hub in DC to talk about the book & our chapters. Video of Davies, who managed to cover a tremendous amount of ground in ten minutes, is embedded below.
Thank you to everyone who contributed, commented and collaborated in this project, which will inform the public, press and governments around the world.
On May 10, the Select Committee on the Modernization of Congress in the United States House of Representatives held a hearing on “opening up the process,” at which 4 different experts talked with Congress about making legislative information more transparent,” from ongoing efforts to proposed reforms to the effect of sunshine laws passed decades ago.
If you’re not up to speed on this committee, it was established on January 4, 2019 the House voted in favor of establishing the Select Committee by an overwhelming margin (418-12) by adopting of Title II of H.Res.6, the Rules of the House of Representatives for the One Hundred Sixteenth Congress, with a sole authority of investigating, studying, holding public hearings, making finding, and developing recommendations to modernize Congress – but no legislative jurisdiction nor ability to take legislative action. The committee has has been fairly described by IssueOne as “the best opportunity in decades” for Congress to improve itself, by looking inward.
The two hours of discussion on May 10 mostly added up to good news for good government, with useful summaries of progress opening up Congress to date from the deputy clerk of the House, proposed recommendations and reforms from GovTrack founder Josh Tauberer and DemandProgress policy director Daniel Schuman that would build on that progress, and some skepticism of sunshine in the legislative process from University of Maryland professor Frances Lee.
I attended the hearing and tweeted from it, as has been my practice for nearly a decade in DC:
The thread of tweets above, however is not meant to be comprehensive, nor could it be fully contextualized in the moment. For that, watch the hearing on YouTube, in the video embedded below:
…and read a summary of the hearing from the Congressional Institute, reporting from Federal Computer Week on “transparency through technology” and Roll Call on ongoing development of an “artificial intelligence engine” (applying machine-learning to structured legislative data), and the latest edition of Demand Progress’ First Branch Forecast, in which Schuman summarizes his testimony and aspects of the hearing.
He and Tauberer recommended from appointing a legislative branch Chief Data Officer, releasing structured data that would feed into the clerk’s tool to show how proposed amendments would change bills, and enact a mandate and open standards for a unique identifier for lobbyists across the U.S. government. Whether those ideas make it into the committee’s recommendations remains to be seen, but they’re worth weighing – along with further studying of the value or risk of increasing or decreasing public access to various aspects of the deliberative processes that constitute legislative and oversight activities.
On a meta note, the process on display at this forum was notable for comity between witnesses and members, openness to the public and press, and engagement online.
The medium is still the message, when it comes showing (not telling) how open a given institution is.
While paying attention to the digital component has become downright mundane in 2019, the Committee demonstrated admirable competence, streaming the hearing online at YouTube, publicized it on social media prior to the event, engaged with the public during the hearing, and published testimony on its website afterwards. (Unfortunately, there’s no text transcription of the hearing on the hearing page. Given the committee’s acknowledgement of the importance of accessibility, it should make sure to get transcripts online.)
As at the most recent “Congressional hackathons,” Members of Congress were able to show good government, ethics and transparency, far away from partisan rancor over white hot political issues or Congressional attempts to conduct constitutional oversight of a corrupt administration.
If you’re interested in following the activities of the committee or providing feedback, visit Modernizecongress.House.gov and scrub in: the People’s House will only be as open, accessible, accountable, and effective as we make it – or demand it to be. There’s no one else coming to help.
In 2019, journalists, politicians and pundits shouldn’t be asking whether White House officials should using WhatsApp. If a given encrypted or ephemeral app does not have archiving built in, public servants should not use it for public business, much less … Continue reading
This past week, Facebook launched a new political ad transparency website. Facebook believes that “shining a light on ads” will increase transparency, which in turn “will lead to increased accountability & responsibility over time – not just for Facebook but advertisers as well.“
I think they’re right — which should be no surprise given my focus on advocating for more political transparency in Washington over the two years I spent at the Sunlight Foundation — but reviewing reports of unlabeled political ads is going to be hard.
Overall, this site is a welcome step towards more transparency, but misses the mark. The site only “exceeds expectations” if you think a search interface that exposes no underlying data is sufficient to inform the public and regulators.
In my initial assessment, I concur with journalists who found Facebook’s new political ad system is missing a lot, as ProPublica reported. (Please install ProPublica’s political ad collector so they can inform the public about how well Facebook’s tool actually works.)
It was easy to use @Facebook‘s new archive of “ads with political content” – essentially a simple search tool for paid political ads that have run since May 7, 2018 – once I got on my laptop and logged in. I found recent ads that matched Trump, Clinton, gun control & corruption. pic.twitter.com/Fhx0lrMzBE
— Alex Howard (@digiphile) May 24, 2018
On the one hand, it was easy to use Facebook’s new archive of “ads with political content” – essentially a simple search tool for paid political ads that have run since May 7, 2018 – at least once I got on my laptop and logged into Facebook. I found recent ads that matched Trump, Clinton, gun control and corruption.
If you click on “see ad performance,” you can learn more about each ad.
If you click “see ad performance,” you see the ad content, who paid, when it was active, how many impressions it received, total spent, & breakdown of audience by age, gender & location.
But clicking “view all ads” brings you to aggregate search results, NOT the page or a profile pic.twitter.com/8XtzmWqdYy
— Alex Howard (@digiphile) May 24, 2018
If you click on the username, you arrive at the Page behind the ads. Unfortunately, there’s no tab for political ads or link to this archive. It’s hard to see how folks will find them, without it.
If you click on the username – in this case, Donald Trump, @realDonaldTrump‘s campaign account on @Facebook – you arrive at the Page behind the ads. Unfortunately, there’s no tab for political ads or link to this archive. It’s hard to see how folks will find them, without it. pic.twitter.com/EASlccVAhF
— Alex Howard (@digiphile) May 24, 2018
As I noted on Twitter, however, there’s one more critical wrinkle: you can’t get to the page unless you’re logged into Facebook!
This would be hilariously ironic, if it weren’t for the context of Russian interference and how Facebook handled it. Self-regulation is not enough.
As sociology professor Zeynep Tufecki noted, no one — whether member of the public, the press, watchdog, academic, regulator or legislator – should have to agree to Facebook’s Terms of Service and become a user to access political data.
😱 You shouldn’t have to agree to Facebook TOS in order to access information about political reports. In fact, that is a core problem. I’ve seen examples where schools put *emergency* information on Facebook and people have to agree to FB TOS to learn whether children are safe. https://t.co/6kmsOXgYgu
— zeynep tufekci (@zeynep) May 24, 2018
To Facebook’s credit, the director of product at Facebook, Rob Leathern, responded publicly to Tufecki on Twittter, stating that this page is a first step:
“More ways are coming to make the ads with political content and information more accessible to people. One of those is an API, another is exploring opening the archive to people not on Facebook. We started with the Facebook community to see how they use the tool and gain feedback from third parties, including our newly-formed Election Commission. We’ll continue to update on our progress.”
If Facebook started with open data with no log-in, they could have gotten feedback from third parties like the Center for Responsive Politics or the public. No one should have to be part of Facebook’s “community” to understand who is buying electioneering on the platform, for whom, and what’s being shown.
As I commented to Leathern, if Facebook is only “exploring” making this archive open to people not on Facebook, then it is not implementing the Honest Ads Act, as its staff has claimed to Congress and the public. I asked Facebook to post a public ad file as bulk open data on the open Web.
Leathern told me that “we have prioritized getting the archive in the hands of people to use (as of today) + will follow up soon with an archive API. Thank you for the feedback, we are definitely listening.”
That’s good news, but not good enough.
Real transparency at Facebook will look like a public file of all paid political ads that are disclosed on a public website with bulk open data downloads and an API, none of which require the public to log into the site.
The good news is that I think Facebook understands this page as a start, not an end. In a post that closes matches what he told me, Leathern wrote that they’re “working closely” with a new “Election Commission” to launch an API for the archives.
It’s good news, but no deadline cited.
It’s hard for me not to be happy that Facebook is finally explicitly embracing political ad transparency in words and (some) deeds, including public soul searching about what constitutes a political ad and a policy.
It’s just long overdue. Ultimately, elected representatives should be the ones to enact standards for transparency for political ads online after debate, not tech company executives.
Until Congress and other legislatures around the world empower regulators like Federal Election Commission by updating electioneering rules and enacting standards for disclaimers and disclosures, however, I’m glad to see positive actions.
I hope Facebook, its founder and its staff deliver on its most recent promises and their public obligations. Given past, current or predictable interference, opacity is unpatriotic.
The VA is being neither open nor transparent about its missing open government plans or policies. On March 29, 2018, I made a Freedom of Information Act request to the United States Department of Veterans Affairs (VA) in which I … Continue reading
Yesterday, I wrote up 15 key insights from the Pew Internet and Life Project’s new research on the American public’s attitude towards open data and open government. If you missed it, what people think about government data and the potential impact of releasing it is heavily influenced by the prevailing low trust in government and their politics.
Media coverage of the survey reflected the skepticism of the reporters (“Most Americans don’t think government transparency matters a damn“) or of the public (“Who cares about open data” and “Americans not impressed by open government initiatives“). This photo by Pete Souza below might be an apt image for this feeling:
Other stories pulled out individual elements of the research (“Open data on criminals and teachers is a-okay, say most US citizens” or mixed results (“People Like U.S. Open Data Initiatives, But Think Government Could Do More” and “Sorry, open data: Americans just aren’t that into you“) or general doubts about an unfamiliar topic (“Many Americans Doubt Government Open Data Efforts“). At least one editor’s headline suggested that the results were an indictment of everything government does online: (“Americans view government’s online services and public data sharing as a resounding ‘meh’.) Meh, indeed.
As usual, keep a salt shaker handy as you browse the headlines and read the original source. The research itself is more nuanced than those headlines suggest, as my interview with the lead researcher on the survey, John Horrigan, hopefully made clear.
Over at TechPresident, editor-in-chief Micah Sifry saw a glass half full:
- Digging deeper into the Pew report, it’s interesting to find that beyond the “ardent optimists” (17% of adults) who embrace the benefit of open government data and use it often, and the “committed cynics” (20%) who use online government resources but think they aren’t improving government performance much, there’s a big group of “buoyant bystanders” (27%) who like the idea that open data can improve government’s performance but themselves aren’t using the internet much to engage with government. (Heads up Kate Krontiris, who’s been studying the “interested bystander.”)
- It’s not clear how much of the bystander problem is also an access problem. According to a different new analysis done by the Pew Research Center, about five million American households with school-age children–nearly one in five–do not have high-speed internet access at home. This “broadband gap” is worst among households with incomes under $50,000 a year.
Reaction from foundations that have advocated, funded or otherwise supported open government data efforts went deeper. Writing for the Sunlight Foundation, communications director Gabriela Schneider saw the results from the survey in a rosy (sun)light, seeing public optimism about open government and open data.
People are optimistic that open data initiatives can make government more accountable. But, many surveyed by Pew are less sure open data will improve government performance. Relatedly, Americans have not quite engaged very deeply with government data to monitor performance, so it remains to be seen if changes in engagement will affect public attitudes.
That’s something we at Sunlight hope to positively affect, particularly as we make new inroads in setting new standards for how the federal government discloses its work online. And as Americans shift their attention away from Congress and more toward their own backyards, we know our newly expanded work as part of the What Works Cities initiative will better engage the public, make government more effective and improve people’s lives.
Jonathan Sotsky, director of strategy and assessment for the Knight Foundation, saw a trust conundrum for government in the results:
Undoubtedly, a greater focus is needed on explaining to the public how increasing the accessibility and utility of government data can drive accountability, improve government service delivery and even provide the grist for new startup businesses. The short-term conundrum government data initiatives face is that while they ultimately seek to increase government trustworthiness, they may struggle to gain structure because the present lack of trust in government undermines their perceived impact.
Steven Clift, the founder of e-democracy.org, views this survey as a wakeup call for open data advocates.
One reason I love services like CityGram, GovDelivery, etc. is that they deliver government information (often in a timely way) to the public based on their preferences/subscriptions. As someone who worked in “e-government” for the State of Minnesota, I think most people just want the “information” that matters to them and the public has no particular attachment to the idea of “open data” allowing third parties to innovate or make this data available. I view this survey as a huge wake up call to #opengov advocates on the #opendata side that the field needs to provide far more useful stuff to the general public and care a lot more about outreach and marketing to reach people with the good stuff already available.
Mark Headd, former chief data officer for the City of Philadelphia and current developer evangelist for Accela software, saw the results as a huge opportunity to win hearts and minds:
The modern open data and civic hacking movements were largely born out of the experience of cities. Washington DC, New York City and Chicago were among the first governments to actively recruit outside software developers to build solutions on top of their open data. And the first governments to partner with Code for America – and the majority over the life of the organization’s history – have been cities.
How do school closings impact individual neighborhoods? How do construction permit approvals change the character of communities? How is green space distributed across neighborhoods in a city? Where are vacant properties in a neighborhood – who owns them and are there opportunities for reuse?
These are all the kinds of questions we need people living and working in neighborhoods to help us answer. And we need more open data from local governments to do this.
If you see other blog posts or media coverage that’s not linked above, please let me know. I storified some reactions on Twitter but I’m certain that I missed conversations or opinions.
There are two additional insights from Pew that I didn’t write about yesterday that are worth keeping in mind with respect to how how Americans are thinking about the release of public data back to the public. First, it’s unclear whether the public realizes they’re using apps and services built upon government data, despite sizable majorities doing so.
— Alex Howard (@digiphile) April 21, 2015
Second, John Horrigan told me that survey respondents universally are not simply asking for governments to make the data easier to understand so that they can figure out what I want to figure out: what people really want is intermediaries to help them make sense of the data.
“We saw a fair number of people pleading in comments for better apps to make the data make sense,” said Horrigan. “When they went online, they couldn’t get budget data to work. When the found traffic data, couldn’t make it work. There were comments on both sides of the ledger. Those that think government did an ok job wish they did this. Those that thin government is doing a horrible job also wish they did this.”
This is the opportunity that Headd referred to, and the reason that data journalism is the critical capacity that democratic governments which genuinely want to see returns on accountability and transparency must ensure can flourish in civil society.
If a Republican is elected as the next President of the United States, we’ll see if public views shift on other fronts.
On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, an official statement about the USASpending.gov redesign at the U.S. Treasury’s blog confirmed that the redesign was real. Analysts, media and businesses that rely on the contracting data on the site were loudly decried the decreased functionality of USASpending.gov.
A week later, there’s a still no evidence of deliberate intent on the part of Treasury not to publish accurate spending data or break the tool, despite headlines about rolling back transparency. Rather, it looks more likely that there were been a number of mistakes or even unavoidable errors made in the transitioning the site and data from a bankrupt federal contractor. There was certainly poor communication with the business community and advocates who use the site, a reality that Luke Fretwell helpfully suggested at Govfresh that other government agencies work to avoid next time.
Today, as Fretwell first reported, the federal government launched a new repository for tracking issues on USASpending.gov on Github, the social coding site that’s become an increasingly important platform for 18F, which committed to developing free and open source software by default last year.
In an email to the White House’s open government Google Group, Corinna Zarek, the senior advisor for open government in the Obama administration, followed up on earlier concerns about the redesign:
The USAspending team has been working to improve the usability of the site and has made some great strides to make it easier for average citizens to navigate information. But at the same time, we all understand that some of our expert users (like a lot of you) seek more technical information and the team is striving to meet your needs as well.
This is definitely a work in progress so please keep working with the team as it iterates on the best ways to improve function of the site while maintaining the content you seek. Your initial comments have been really helpful and the USAspending team is already working to address some of them.
Zarek also said that several of the problems with data that people have reported been addressed, including the capacity to download larger data sets and define specific dates in search, and asked for more feedback.
Specifically, this week the team addressed data export issues to allow the ability to specify date ranges to download data, added the bulk file format API, and modified the download capability so larger datasets can be downloaded. Additionally, data archives are being added continually. This week, they loaded the 2014 and 2015 delta files that show the new transactions in the last month. You can keep track of the ongoing improvements on the “What’s new” page.
Please keep sharing your feedback and continue working with the USAspending team as it makes improvements to the site. You can do this through the site’s contact page or on the new Github page where you can report issues and track them in the open.
If you find bugs, let the feds know about them on Github so that everyone can see the issues and how they’re addressed. As Mollie Walker reported for FierceGovernmentIT, there’s still missing functionality yet to be restored.
[Image Credit: Govfresh, via USASpending.gov]
Last year, a successful Freedom of Information request for the United States enterprise data inventory by the Sunlight Foundation was a big win for open government, nudging Uncle Sam towards a better information policy through some creative legal arguments. Today, the federal government started releasing its enterprise indices at data.gov. You can browse the data for individual agencies, like the feed for the Office for Personnel Management, using a JSON viewer like this one.
“Access to this data will empower journalists, government officials, civic technologists, innovators and the public to better hold government accountable,” said Sunlight Foundation president Chris Gates, in a statement. “Previously, it was next to impossible to know what and how much data the government has, and this is an unprecedented window into its internal workings. Transparency is a bedrock principle for democracy, and the federal government’s response to Sunlight’s Freedom of Information request shows a strong commitment to open data. We expect to see each of these agencies continue to proactively release their data inventories.”
Understanding what data an organization holds is a critical first step in deciding how it should be stored, analyzed or published, shifting towards thinking about data as an asset. That’s why President Barack Obama’s executive order requiring federal agencies to catalog the data they have was a big deal. When that organization is a democratic government and the data in question was created using taxpayer funds, releasing the inventory of the data sets that it holds is a basic expression of open and accountable government.