NEW YORK—Many pedestrians have noted dangerous streets in their neighborhoods as the ones they are reluctant to cross. But when it comes to community members or advocates trying to convince city agencies to take action, it’s difficult to express just how dangerous a street or intersection is.
“We get requests many times a week from people with family members or people they know involved in crashes who want to know how dangerous their streets are,” said Alana Miller, policy coordinator for Transportation Alternatives.
Miller works with the New York Police Department’s crash data, and one of the organization’s projects has been mapping the city’s traffic incidents on CrashStat.org.
However, there are many holes in the NYPD data, and by one account, nearly a fifth of the NYPD’s location records are inaccurate. Despite that, civic hackers have worked to turn the mess of data into useful information for communities.
Soon, this may no longer be an issue.
According to a department spokesperson, there are agency-wide efforts underway to revamp how officers collect crash data.
Data Scraping and Mapping
When John Krauss read that the NYPD was reluctant to release crash data in a machine-readable format as a way to keep people from manipulating it “to make a point of some sort,” he wrote a script to scrape the data from the NYPD’s PDFs into a readable spreadsheet.
“Data is essential in allowing NYPD, [Department of Transportation], and other city agencies to know where to make improvements,” Krauss said. “Right now there’s a huge lag time between when collisions happen and when the DOT can review and analyze it.”
In late 2012, Krauss created the NYPD Crash Data Band-aid built off work David Turner did in converting NYPD’s PDFs into text files. The script went live in 2013, and since then, many interested groups have created maps visualizing the data in useful ways for communities.
Pediacities, an interactive encyclopedia of cities, created a Vision Zero map to track the initiative soon after Mayor Bill de Blasio’s announcement. Vision Zero is a set of policies intended to reduce pedestrian deaths to zero. A key recommendation involves redesigning dangerous city streets.
“Data is the accountability part of Vision Zero, while the real work is doing the street redesign or changing enforcement priorities,” said Nathan Storey of Pediacities.
Pediacities has talked to members of community boards and City Council who want more information. By opening up access to the information, more and more people will help analyze the data, Storey said.
Statistician Ben Wellington, who teaches at Pratt Institute, created “The Terrifying Cycling Injury Map” of the city recently with help from Krauss’s script, and different groups have used it to center in on their neighborhoods to point out dangerous streets.
2013 traffic fatalities (Ben Wellington)
On his blog IQuantNY, Wellington posted heatmaps as well, showing neighborhoods with the highest numbers of injuries or fatalities. He notes that the maps don’t show density, however, and do not completely represent which areas are most dangerous.
For example, if bicyclists feel a street is highly dangerous, they may avoid it, thus lowering bicycle traffic and potential injuries per bicyclist.
“This is ensuring community groups are empowered to have that dialogue with DOT, NYPD, or even their families,” said Noel Hidalgo, co-founder of BetaNYC and a charter member of the group that pushed for the city’s open data law. “Families with kids who walk to school want to know their kids are safe, cyclists want to know the route [they] take is the safest route.”
“There’s no reason the public shouldn’t have that information,” Hidalgo said.
Open Data Law
The location of a crash, time of day, individuals involved, review measures taken, previous requests for street redesigns, and density are all pieces of information the city has, and a small part of what is available to analyze in creating safer streets.
“The only way to accomplish Vision Zero is a discussion about transparency,” Hidalgo says.
The city’s open data policy makes over 1,100 datasets available for free to the public, from the locations of all city-owned vacant lots, to property valuations, to Wi-Fi hotspots. Some of this information, now free, used to only be obtainable for hundreds of dollars.
The purpose is to let people understand what is going on in government, and also to ensure oversight, Hidalgo says, but so few people understand the value of it.
Inaccuracies and Change
The NYPD had started releasing non-readable PDFs of crash data after the law required the agency to do so, but it has been lagging in complying with requests from officials, community groups, and advocacy groups on releasing data in machine-readable formats. The agency also replaced the released data monthly, rather than making past records available.
After much prodding, NYPD started releasing the data in spreadsheet format, but the cells are joined in a way that makes it essentially useless, Krauss said.
According to an NYPD spokesperson, the department is looking to unlock the data into an Excel or other similar format. There is no time-frame set yet.
Because the locations aren’t recorded in a longitude/latitude format, some errors get introduced during conversion.
A second layer of errors are introduced because of the way collision information gets recorded. Police officers use a New York State Department of Motor Vehicles form that requires listing the nearest cross street of the incident, which can be misleading because it then seems like all accidents happen at intersections, Miller said.
Together, this results in mapping errors on almost a fifth of reported incidents, according to Krauss. Occasionally there are street names misspelled, and sometimes the cross streets listed don’t intersect at all.
NYPD is also in the process of changing this system for recording accidents, a spokesperson confirmed.
NYPD Commissioner Bill Bratton announced Tuesday a pilot program to arm officers with tablets. The goal is to allow officers access to crime records, but it could also provide the GPS information needed for recording location data with greater accuracy.
While there is no time frame, civic hacking, and advocacy groups are optimistic. Open data is beneficial to both communities and government agencies, they say, as it only ensures the government has more informed partners.
