As open data becomes a priority in our governments and data communities nationally, it’s crucial that each category of data — from commuting, to environmental, to demographic — finds its home and its purpose. One of these primary categories is that of crime data. Throughout the country, justice departments, governments, and data lovers alike are using crime data to identify sociological patterns, evaluate neighborhoods, and more.
As a community partner for Code for San Francisco, Microsoft is honored to work alongside San Francisco’s brigade with such data. Using data to improve communities aligns directly with Microsoft’s Cloud for Global Good Initiative, and we’re thrilled to support a local team that has built a powerhouse based on crime data.
— Scott Mauvais, Director of Technology & Civic Innovation, Microsoft
San Francisco has been publishing police incident data for about as long as the open data portal has been operating. After joining as the City’s first Chief Data Officer in 2014, Joy Bonaguro discovered staff at the San Francisco Police Department (SFPD) spent hours performing custom geographic queries on incident data to respond to public records requests. They wanted to give the public the ability to geographically query incident data to save some time and increase access.
While the portal didn’t offer a user interface to query data geographically, it did offer a way to do that with code. Inspired by this simple challenge, Jason Lally, the City’s Open Data Program Manager, hacked together a prototype user interface that eventually became sfcrimedata.org. It was able to query data around an address, covering a span of time. And it simply returned a map of incident points and a table the user could download.
The prototype was good enough, but Jason wanted some help defining it and carrying the work forward.
Enter Code for San Francisco, a group of local volunteers regularly coming together to make San Francisco a better place through the use of technology. Jesse Biroscak, the Captain of Code for San Francisco at the time, saw some immediate potential for usability upgrades. Jesse decided to take on management of the project, beginning with some in-depth user research and testing to validate the prototype’s assumptions.
He wanted to be clear about what the SFPD really needed. During Jesse’s interviews with the SFPD Crime Analysis Unit, he discovered the primary users of the crime data weren’t the SFPD at all; universities, it turns out, were using the data to fulfill their annual Clery Act reporting requirements. The Clery Act is a consumer protection law that aims to provide transparency around campus crime policy and statistics.
Jesse connected with Jason Heil, Clery Act Coordinator for UCSF and an expert in understanding the Act’s reporting requirements. Jason helped to clarify what UCSF wants with crime data and shared details on UCSF’s use cases. The conversation helped hone what SF Crime Data needed to do to satisfy a major university’s needs: coordinating SFPD’s internal data transfer processes, cleaning up huge amounts of inconsistent data, and adding natural language processing.
Jesse knew this was beyond the scope of what the volunteers could deliver and support so he decided to pivot toward something else. He presented what he learned — a great example of why user research is critical to the product development process — to Code for San Francisco.
Paul Hennings, one of the many volunteers at Code for San Francisco, had been attending Wednesday hack nights off and on through 2015 and early 2016. The mission to support open data, to reduce public records requests at SFPD and to support campus safety struck Paul as worthy goals and the technology stack for the project seemed very accessible: all front-end, HTML, CSS, Javascript and some API calls.
Paul spent the summer coming up to speed on the project and learning about campus safety reporting requirements and the Clery Act. He recruited developers from the volunteers in the Code for San Francisco Brigade to continue adding enhancements to the product through the last half of 2016.
One of those volunteers was Sergey Litvinenko, of Xtreet, an urban mobility and location data company. Sergey saw an opportunity to give back to the local community. Xtreet uses many types of data and crime is one of the categories of interest since it is directly connected to transportation safety. Sergey and Xtreet contribute to the engineering side of the project by donating the URL, DNS and other infrastructure services.
In early 2017, Code for San Francisco released the beta version of sfcrimedata.org. A marketing push alerted individuals responsible for safety reporting at local San Francisco colleges and universities on the day that a full year of 2016 SFPD Incidents was available on the city’s open data portal and could now be accessed easily through sfcrimedata.org.
In February, the team released a significant improvement in search capability by offering polygon search — the ability to draw an irregular shape around a local campus area — to focus a search for crime incidents on a specific campus and the immediate surrounding areas only.
Upcoming enhancements will deliver the ability to report on campus safety-related crime incidents. The team is also working on Search Engine Optimization (SEO), so that prospective students and parents can find the site and have the ability to do their own searches and comparative evaluation of campus safety statistics.
The Code for San Francisco team plans to enhance the site through 2017 and complete the first official release by the time the 2017 SFPD Crime Data is available in mid-January 2018.
