Yearly Archives: 2012

Do newspapers need to adopt data science to cover campaigns?

Posted on by

0

Last October, New York Times elections developer Derek Willis was worried about what we don’t know about elections:

While campaigns have a public presence that is mostly recorded and observed, the stuff that goes on behind the scenes is so much more sophisticated than it has been. In 2008 we were fascinated by the Obama campaign’s use of iPhones for data collection; now we’re entering an age where campaigns don’t just collect information by hand, but harvest it and learn from it. An “information arms race,” as GOP consultant Alex Gage puts it.

For most news organizations, the standard approach to campaign coverage is tantamount to bringing a knife to a gun fight. How many data scientists work for news organizations? We are falling behind, and we risk not being able to explain to our readers and users how their representatives get elected or defeated.

Writing for the New York Times today, Slate columnist Sasha Issenberg revisited that theme, arguing that campaign reporters are behind the curve in understanding, analyzing or being able to capably replicate what political campaigns are now doing with data. Whether you’re new to the reality of the role of big data in this campaign or fascinated by it, a recent online conference on the data-driven politics of 2012 will be of interest. I’ve embedded it below:

Issenberg’s post has stirred online debate amongst journalists, academics and at least one open government technologist. I’ve embedded a storify of them below.

http://storify.com/digiphile/should-newspapers-adopt-data-science-to-cover-poli.js[View the story “Should newspapers adopt data science to cover political campaigns?” on Storify]

Can government innovation rise above partisan politics?

Posted on by

5

Earlier today, the White House announced the first class of Presidential Innovation Fellows. Following is the story you might have missed on the Twitter backchannel, followed by a NodeX graph of the tweets around #InnovateGov.

http://storify.com/digiphile/when-innovategov-gets-hot-will-policy-debates-foll.js

[View the story “When #innovategov gets hot, will policy debates follow?” on Storify]

In the network graph below, you’ll see there are 3 discreet groups around the White House, Tim O’Reilly and me, and Michelle Malkin. The lines between the nodes show replies.

Are “Commons 2.0” and participatory urbanism hype or hope?

Posted on by

2

“…armed with low-cost phones and an Internet connection, people are using civic-minded apps like ForageCity to tackle everything from public safety to potholes. The question is whether the technology will have the long-term effect that some foresee, or whether the “commons 2.0” and “participatory urbanism” will become empty marketing slogans.”

-Angela Woodall, writing in the Oakland Tribune about a new mobile application from Oakland’s Youth Radio that is designed to help people redistribute extra fruit and vegetables to people in need.

Forage City app

[Image Credit: Susan Mernit]

Woodal asks good questions and, as it happens, posed them to me last week in a phone interview. (I’m quoted in the article.)

Here’s a couple of thoughts that didn’t make it in. Mobile applications that civic developers are creating around the world — like ForageCity — are making it increasingly possible for more people to interact more easily and for less cost where ever and whenever they wish. That does lead to giving more power to more people to connect to one another and solve problems, or at least discuss them.

The potential for such apps to connect and, crucially, scale is particularly significant when there is a shared standard for the open government data that fuels, as with the standard for transit data (GTFS) that now exists in 450 different cities. Around the U.S., cities are slowly working with one another to define more such standards — but it’s a complicated process that doesn’t happen overnight, or even years.

The question is whether the technology will have the long-term effect that Code for America founder Jen Pahlka described to Woodall. On that count, I tend to give Pahlka — and my publisher, Tim O’Reilly — the benefit of the doubt.

As I said to the reporter, the potential for civic apps is enormous — but these the tools are only as good as the people who use them and adapt them. The tools can be quite good on their own — full stop — but many network effects will only take place with broad, mainstream adoption.

Smartphones can now be used for finding shelter, improving medical care and documenting riots — but the same devices are also used for gaming, pornography, celebrity gossip and shopping. While the apps used to find city services are generally not the ones used to surveil citizens, in practice the mobile device itself may be an agent of both actions.

Working out how to both protect the rights of citizens and empower citizens using mobile devices will be a difficult and crucial need in the years ahead.

It’s not immediately clear, at least to this observer, that state governments, Congress, regulators and law enforcement are up to the challenge, but it’s hard not to hope that they rise to the challenge.

Jay Nath on how San Franscisco is working to get its Gov 2.0 groove back

Posted on by

0

Back in January, Govfresh founder wrote about how San Francisco can “get its Gov 2.0 groove back,” offering six recommendations to the city government to use technology better.

[Image Credit: Fog City Journal]

When asked for comment, San Francisco chief innovation officer Jay Nath (@Jay_Nath) responded to Fretwell’s suggestions via email. While I’ll be sharing more from Nath and SF CIO Jon Walton over at the O’Reilly Radar civic innovation channel, in the meantime I’m publishing his specific responses to those recommendations below.

Build the best mayoral website in the world

Nath: We can always improve how we communicate with our constituents. If we were to undertake an effort to redesign the Mayor’s site, we should take a holistic approach and not just focus on the Mayor’s site. The approach NYC took to invite their design community is one that I think is very smart and something that SF should consider.

Use “Built in SF” technology

Nath: We agree and launched our City Hall iZone concept where we pilot great local technologies and services. We frequently meet with great companies like Square, Twitter, Uber, Yammer and invite each of them to work with the City. Specifically, we’re actively exploring Yammer, Zendesk, Get Satisfaction, Cozybit and 802.11s mesh, Google+ hangouts, and others. Additionally, we’re already using local tech like WordPress (which powers our innovation site), Twitter via Open311API, and Instagram.

Go back to the (data) fundamentals

Nath: We have an open data roadmap to strengthen our leadership in this area. It’s in our 2012 innovation portfolio as well. Our goal is to structurally change how we share data so that our default position is one of sharing. One idea is to require that all software purchased that stores structured data to have a public API. As we secure staffing for this effort, we will invite the community to help us shape the final form and execute.

Leverage the civic surplus

Nath: I would argue that we’ve done a great job in this area. Last summer, we partnered with Gray Area Foundation for the Arts (GAFFTA) to produce the “Summer of Smart.” This series of hackathons produced over 20 prototypes, 500 participants and 10,000 hours of civic engagement. We’ve continued our efforts this year with the City’s first unhackathon around taxi dispatch and real-time mass communication. Our Mayor and transit director both attended the event and thanked our community for their efforts to make SF a better city.

Additionally, we launched our citizen engagement platform, ImproveSF, in a very big way in April.

Open source the infrastructure

Nath: While we can do more to increase open source software adoption, I want to
recognize our efforts to date:

  • open source policy
  • SFPark Android/iPhone app
  • Enterprise Addressing System
  • SmartPDF
  • LAMP as an option for internal customers
  • Pligg (DataSF)
  • Several Drupal applications

Additionally, the idea of moving our City from the existing CMS (Vision) to WordPress is not just about open source technology. We, as a City, made the decision to utilize Vision CMS a couple of years ago and the switching costs to migrate to WordPress currently outweigh the benefits. I will encourage the City to strongly consider WordPress, Drupal, etc for consideration when Vision no longer meets our needs.

Give citizens a dashboard

Nath: This is more than just adopting the IT Dashboard. We have to implement the governance and project management model to ensure that the data is accurate. This is something we need to do but requires time and culture change. I agree that we need to increase access to high value datasets like expenditures. This is part of our open data roadmap and will receive renewed focus in 2012.

SahelResponse.org showcases the power of open data and neocartography

Posted on by

0

On January 9th, I wondered whether 2012 would be “the year of the open map.” I started reporting on digital maps made with powerful new software and open data last winter, in the context of open government.

In the months since, I’ve seen many more maps emerge from the work of data journalists and government, including a beautiful one made with TileMill and open data from aid agencies at SahelResponse.org. You can explore the map in the embed below:

Nate Smith, who works at DevelopmentSeed, the makers of MapBox and TileMill, blogged about SahelReponse.org at PBS Mediashift.

To bring key aid agencies together and help drive international response, the SahelResponse.org data-sharing initiative maps information about the ongoing food crisis in the Sahel region of West Africa. More than 18 million people across the Sahel are at risk and in need of food assistance in the coming months, according to the United Nations. Recent drought, population movements, and conflict have created a rapidly changing emergency situation. As in any crisis, multiple agencies need to respond and ramp up their coordination, and access to data is critical for effective collaboration. In a large region like the Sahel, the band of mostly arid land below the Sahara Desert stretching across the continent, effective coordination and collaboration are critical for responding effectively.

Thanks to new technologies like TileMill, and an increased adoption of open data, it was possible to put all the key data about the crisis — from relief access routes to drought conditions and population movements — in one place, openly available and mapped to give it further context.

More than half a year later, on other words, I think the prediction that 2012 will be the year of the open map is being born out. The adoption of OpenStreetMap by Foursquare was a notable data point, as was StreetEast moving to OpenStreetMap from Google Maps. In response to the challenge, Google slashed its price for using the Google Maps API by 88%. In an ideal world, the new competition will result in better products and more informed citizens.

Social citizenship: CNN and Facebook to partner on “I’m Voting” app in 2012 election

Posted on by

1

Two years ago, I wondered whether “social voting” on Foursquare would increase voter participation.

That experiment is about to be writ much larger. In a release today, first reported (as far as I can tell) by Mike Allen in Politico Playbook, CNN and Facebook announced that they will be partnering on a “I’m Voting” Facebook app that will display commitments to vote on timelines, newsfeeds and the “real-time ticker” in Facebook.

“Each campaign cycle brings new technologies that enhance the way that important connections between citizens and their elected representatives are made. Though the mediums have changed, the critical linkages between candidates and voters­ remain,” said Joel Kaplan, Facebook Vice President-U.S. Public Policy, in a prepared statement. “Innovations like Facebook can help transform this informational experience into a social one for the American people.”

“By allowing citizens to connect in an authentic and meaningful way with presidential candidates and discuss critical issues facing the country, we hope more voters than ever will get involved with issues that matter most to them,” said Joe Lockhart, Facebook Vice President Corporate Communications, in a prepared statement. “Facebook is pleased to partner with CNN on this uniquely participatory experience.”

“We fundamentally changed the way people consume live event coverage, setting a record for the most-watched live video event in Internet history, when we teamed up with Facebook for the 2009 Inauguration of President Obama,” said KC Estenson, SVP CNN Digital, in a prepared statement. “By again harnessing the power of the Facebook platform and coupling it with the best of our journalism, we will redefine how people engage in the democratic process and advance the way a news organization covers a national election.”

“This partnership doubles down on CNN’s mission to provide the most engaging coverage of the 2012 election season,” said Sam Feist, CNN Washington bureau chief, in a prepared statement. “CNN’s unparalleled political reporting combined with Facebook’s social connectivity will empower more American voters in this critical election season.”

What will ‘social citizenship’ mean?

There’s also a larger question about the effect of these technologies on society: Will social networks encouraging people to share their voting behavior lead to more engagement throughout the year? After all, people are citizens 365 days a year, not just every two years on election day. Will “social citizenship” play a role in Election 2012?

In 2010, Foursquare founder Dennis Crowley said yes. As has often been the case (Dodgeball, anyone?), Crowley may well have been ahead of his time.

“One of the things that we’re finding is that when people send their Foursquare checkins out to Twitter and to Facebook, it can drive behaviors,” said Crowley in 2010. “If I check into a coffee shop all the time, my friends are going to be like, hey, I want to go to that coffee shop. We’re thinking the same thing could happen en masse if you start checking into these polling stations, if you start broadcasting that you voted, it may encourage other friends to go out there and do something.”

The early evidence, at least from healthcare in 2010, was that social sharing can lead to more awareness and promote health. Whether civic health improves, at least as measured in voter participation, is another matter. How you voted used to be a question that each registered citizen could choose to keep to him or herself. In 2012 and the age of social media, that social norm may be shifting.

One clear winner in Election 2012, however, will almost certainly be Facebook, which will be collecting a lot of data about users that participate in this app and associated surveys — and that data will be of great interest to political scientists and future campaigns alike.

“Since both CNN and [Facebook] are commercial entities, and since data collection/tracking practices in these apps are increasingly invasive, I am curious to see how these developments impact the evolution of the currently outdated US privacy regime,” commented Vivian Tero, an IDC analyst focused on governance, risk and compliance.

UPDATE: The Poynter Institute picked up this story and connected it in a tweet with a recent AdWeek interview with CNN digital senior vice president and general manager KC Estenson on “CNN’s digital power play.

Estenson, whose network has been suffering from lower ratings of late, notes that online, CNN is now “regularly getting 60 million unique users,” with an “average 20 million minutes a month across the platforms” and CNN Digital generating 110 million video streams per month.

That kind of traffic could power a lot of Likes.

Full release by Facebook on U.S. Politics over on Facebook.

This post has been updated as more information became available, via Facebook spokesman Andrew Noyes.

U.S. cities form working group to share predictive data analytics skills

Posted on by

2

Yesterday, I published an interview with Michael Flowers, New York City’s director of analytics for the Office of Policy and Strategic Planning in Mayor Bloomberg’s office. In the interview, “Predictive data analytics is saving lives and taxpayer dollars in New York City,” Flowers talks about how his team of 5 is applying data analysis on the behalf of citizens to improve the efficiency of processes and more effectively detection of crimes, from financial fraud to cigarette bootlegging.

After our interview, Flowers followed up over email to tell me about a new working group on data analytics between New York City, Boston, Chicago and Philadelphia. The working group, which recently launched a website at www.g-analytics.org, is sharing methodologies, ideas and strategies,

“Ultimately we want the group to grow and support as many cities interested in pursuing this approach as possible,” wrote Flowers. “It can get pretty lonely when you pursue something asymmetrical or untraditional in the government space, so we felt it was important to make it as simple as possible for like-minded cities to get started. There’s a great guy I work closely with out in Chicago on this effort – [Chicago chief data officer] Brett Goldstein; we talk at least twice a week.”

The City of Quebec launched an open data site

Posted on by

1

Up in the currently not-so-frozen north, the City of Quebec has stood up an open data directory online. There are currently 26 datasets listed, spanning a variety of data formats, from .CSV to .XML to .XLS to to .KML to .SHP. (The latter two are GIS files, of interest to folks who like to make maps.)

The city published the video embedded below last night, in addition to a “demarche” (or statement) on the open data website about the project.

ExpoTI-GVQ – Projets étudiants, CÉGEP Limoilou from E-Gouv Québec on Vimeo.

Hat tip @Data_BC

UPDATE: As Richard Ackerman pointed out on Twitter, this open data site went live in February. While the video is new, the site is not.

https://twitter.com/scilib/status/217758569792155648

Google Trends shares data on the U.S. “Veepstakes”

Posted on by

0

If you think that search trends, Google News mentions and YouTube video views offer insight into the selection of a vice presidential nominee, Google Trends has you covered with a new VEEP Stakes” dashboard. Below, I’ve embedded the data around the candidates that the Washington Post has deemed most likely to be chosen by former Massachusetts governor Mitt Romney to round out the top of the Republican ticket this fall. (Due to the limited width of this blog template, you can’t see Virginia governor Bob McConnell or Puerto Rico governor Luis Fortuno on the far right.)

Here’s how the Google Politics and Elections team introduced the dashboard in an update on Google Plus:

The summer before a Presidential election typically brings unending speculation about potential Vice Presidential picks. Veepstakes, as the process has commonly been referred to since 1988, has become a favorite topic of discussion among journalists and politicos.

We are excited to partner with +Washington Post’s +Chris Cillizza and The Fix to launch our first Veepstakes Trends Dashboard to track the buzz around GOP presidential candidate Mitt Romney’s potential VP pick. The Veepstakes dashboard allows you to take the web’s real-time political pulse by comparing potential VP candidates’ YouTube video views, search traffic, and Google News mentions. You can even drill down and check out which potential nominee has been searched the most over the the last day, week or month.

While this is an interesting use of Google data, I find it of limited use in guessing who the Romney campaign will choose. The InTrade prediction for the 2012 Republican VP nominee ranked probabilities offer a much better instant insight into where the smart money from the collective wisdom of observers is pooled (Senator Rob Portman (R-OH) currently leads, at 21%) or, for that matter, Nate Silver’s statistical analyses of the vice presidential nominee’s effect on the race. All that being said, there’s a great deal of interest in who the next potential VP of the United States might be — and this data from Google reflect that wave. Make of it what you will.

Open City launches new civic app to map crime in Chicago

Posted on by

2

Last year, Chicago-based open data wranglers Open City set a high bar for open government data visualizations and transparency websites. Today, Open City launched a new civic Web app at CrimeInChicago.org, adding to their growing portfolio of projects.

Crime in Chicago Poster

Derek Eder, one of the co-founders of Open City, emailed in this morning to share news of Crime in Chicago. “The website offers an interactive data visualization of the 4.8 million crimes reported in Chicago over the last decade,” he wrote. “It lets citizens see crime trends around them, compare crime levels over the years and across city wards, and explore each ward’s homicides, robberies, assaults and dozens of other crimes.”

The site also includes an interesting wrinkle on creating value from open data: selling high quality print posters ranking the incidence of crime in Chicago’s 50 wards.

As Eder pointed out, CrimeInChicago.com is possible because the Emanuel administration and the Chicago Police Department (CPD) are now publishing a open data online that includes local crime trends. In 2012, working to open government in Chicago means developers collaborating with the city to give citizens more understanding of their city.

This post has been updated to reflect an error in the Web address given for the project, if not the link underneath it. As Open City co-founder Juan-Pablo Velez pointed out via email, chicagocrime.org is “Adrian’s Holovaty’s old project, the one that gave birth to Everyblock. You could maybe see this project as the spiritual successor of chicagocrime.org, one that focuses on crime trends instead of crime incidents, but we don’t own that domain.”