This guest post was written by San Francisco Mayor Gavin Newsom, who was elected to the position in 2003 and reelected in 2007. Newsom is also running for governor of California in the upcoming 2010 election. In this guest post, Mayor Newsom details the launch of DataSF.org, a new website that will offer convenient access to city data that is relevant to the community.
San Francisco has a long history of innovation. We are home to hundreds of technology companies that are changing the way the world operates from Twitter to WordPress to Kiva.
In an effort to engage our highly skilled workforce we are launching DataSF.org, an initiative designed to increase access to city data.
The new web site will provide a clearinghouse of structured, raw and machine-readable government data to the public in an easily downloadable format. For example, there will be updated crime incident data from the police department and restaurant inspection data from the Department of Public Health. The initial phase of the web site includes more than 100 datasets, from a range of city departments, including Police, Public Works, and the Municipal Transportation Agency.
We imagine creative developers taking apartment listings and city crime data and mashing it up to help renters find their next home or an iPhone application that shows restaurant ratings based on health code violations.
The idea behind the site is to open up San Francisco government and tap into the creative expertise of our greatest resource – our residents. We hope DataSF.org will create a torrent of innovation similar to when the developer community was given access to the platforms behind popular technologies and devices like Facebook and Apple’s iPhone.
Our effort to improve access to city data has already led to the creation of new services never imagined within the walls of government. Earlier this summer, our Department of Environment released recycling data that was used by a third party to develop EcoFinder, an iPhone application that helps residents recycle based on their location.
By bringing city data and communities together in one location, we hope to stimulate local industry, create jobs and highlight San Francisco’s creative culture and attractiveness as a place to live and work.
As we look to deepen and broaden citizen engagement we will face common challenges: resistance to change, political will, and sustaining data streams from government sources to name a few. Collaboration with citizens, non-profits, vendors, academia, and our peers in government will be critical to overcoming these barriers. It will also take leadership as we’ve seen from President Obama and his CIO, Vivek Kundra to establish our ideals and set forth a shared vision for a more transparent and open government.
This sounds like a campaign PR document. Lot’s of rhetoric and little substance.
No kidding, Funny how it’s written by the mayor himself & not one of TC’s bloggers.
It’s not the job of a Mayor to create substance from a tech initiative like this. His job is to grant permission on create incentives for citizens to participate in city innovation initiatives.
It is the job of the SF tech community to pick up th…e ball and run with it – to make their own substance.
In DC we have a huge data catalog http://data.octo.dc.gov but it was useless to citizens prior to Apps for Democracy http://www.apps...ordemocracy.org
We now have -
311 iPhone/Facebook Apps, Tourism Apps, Parking apps, Roadkill reporting apps (love that one) and about 50 others.
Congrats Mayor Newsom and all the talented citizens behind this effort… Show the world what you can do with open city data!
hmm you’re a tool. this is an awesome effort and EVERY city should do this. how pathetic to bash something like this, when we’re finally being granted access to open data which will help usher in a new era of transparency. get it together man! some things can be good just because they are good.
so sad to see how cynical everyone has become.
Yes. Let’s promote honesty by publishing an article from a guy who banged his best friends wife.
He’s hands down much better than Meg Whitman. The last thing California needs is another “superstar” Governor to get us in a hole.
Haha nice… no reference to honesty in the post, but I agree on his fucked up personal life.
however, discounting the benefits of transparent government due to a mayor’s infidelities is highly non sequitur.
people are always fucking each other’s spouses. let’s not confuse that with a new policy not being good for citizens. similarly, everyone postures to better their own situation (gavin postures to get himself elected to higher power) – and let’s not confuse that with selfless good.
the end.
Get this guy off here. You just slimed TechCrunch. Didn’t he SLEEP with his best friend’s wife? Nice.
Ya look into it before you claim I’m making an irrational statement. Gavin is a joker.
Better than Meg Whitman (former CEO of eBay). Find a better contender and people mmight care to listen to your ramblings.
Let’s not forget that this is the same guy who slept with his campaign manager’s wife behind his back.
And he still got elected to be the Mayor of San Francisco with 72% of the votes. Clearly, no one really cares about that as much as you do.
Johnnie Taylor said it best:
“Whose makin’ love
To Your old lady
While you were out
makin’ love?”
Very nice effort. Similar to some efforts on the federal level.
That said Pittsburgh already has it’s own iPhone app.
http://www.cmu....ng-iburgh.shtml
This is a nice efforts to synchonize the technology with the real need of the people
I am not sure if I would want to know what health violations all restaurants have. It would be scary if nearly all the restaurants had even minor violations.
But, I think this is a great idea and very interested to see what mash-up products come from it.
There’s an iPhone app that already uses data like this to give you the health violations of restaurants by your proximity to them. It is indeed scary. I think it’s called Clean Eats or something
Not an iPhone app, but similar idea: http://www.cleanscores.com/
The official SFDPH site is http://dph-extr...401705715589618
I think they need to hire a new designer.
Definitely kudos to Gavin for getting this done.
I think the bigger test will be actually getting third party developers to create tools and getting people to actually use those tools.
Is there a good use case of open gov data being used to create a tool which has gained mainstream acceptance?
I can’t believe people are bashing him.
I would DREAM of something like this for my city. Not the site itself as it is pretty crappy, but the spirit and the intentions of the major.
I would never imagine any of the most important italian politic figures to write on a technology blog about the release of new websites and iPhone applications.
You maybe think it’s as ordinary as any other post and consider my comment stupid, but I assure you that it just blows me, and I guess many other europeans, away.
The question is why it took so long. Other cities and states have had similar sites for a while.
As the post says, he’s running for governor. And this is part of that campaign.
He’s doing town halls all over the state for his campaign, but not in San Francisco which has been having a budget crisis.
veryyy few cities have this, and you fail to cite any.
Seriously. I just looked at the site, and the data available is ridiculously detailed and broad across many different subjects. Maybe this exists for other cities, but I seriously doubt it.
Nice job, San Francisco!
Nice rhetoric, but Newsom has often opposed sunshine legislation and not lived up to them
He vetoed legislation to have video of city meeting online longer (I believe the board overturned it)
http://www.fogc...d-transparency/
He opposed the legislation that created the restaurant inspection data he now mentions
http://www.sfga...RESTAURANTS.TMP
And he has released sparse data on his official calendars
http://www.sfbg...202&catid=4
Smoke and mirror, smoke and mirrors…. All elected people seem to be the same… Even Mr Change his self is turning out to be a fraud…
wow, i didn’t see that one coming…
/sarc
Just like Newsom’s best friend and campaign manager didn’t see him coming on his wife’s chin.
Is that a recent photo? He looks very young there.
Thats where the recently reviewed EveryBlock takes its data from! Their reports of police calls in SF look interesting.
“We are home to hundreds of technology companies that are changing the way the world operates from Twitter to WordPress to Kiva.”
Are these best tech companies to come from San Francisco?
Yes, how about the tech company Cisco, which is named after San FranCisco.
I think this is pretty cool. It should be standard practice for this data to be public.
Kudos to Gavin Newsom and to the city of San Francisco for taking this big bold step into the new world of Open Cities. I’m sure developers with full access to the city’s data will find very imaginative ways to make San Francisco work better for the people living there. Looking forward to seeing what comes of this.
All you other cities out there: take note – the ball is in your court!
Is that Fred Savage from the wonder Years.
It may not be perfect, but the important thing is it’s a step in the right direction for gov’t.
Keep up the good work Gavin!
Then Newsom should get involved in the whole Routesy debacle and say once and for all that the city’s Muni data isn’t proprietary:
http://www.routesy.com/
I think this site looks great esp since this is a govt web site, and I am pretty sure that this is built on open source technology. way to go SF!
Gavin you make me sick. Every time I hear about you you make me want to vomit.
You are the epitome of a morally corrupt politician. You have done more bad than good for for San Francisco and for California and you consistently go against the will of the people.
Really? Sounds like your issue is personal, and not a real issue with Gavin. Everything I’ve heard about him has been overwhelmingly positive, and this is another step in the right direction for the city of SF.
And here, in a nutshell, lies the idiocy of our political debate. “He’s evil!” “No, everything he touches turns to gold”
We don’t care about your knee-jerk tendencies, but we could use a little help with supporting the good ideas that come out of our government.
Jeez dude. As a resident for a little over 2 years now, I think San Francisco is doing pretty well. And it sure runs a heck of a lot smoother than other cities I’ve lived in. I am seriously confused as to why Gavin Newsom gets people so fired up.
You are an idiot. I can not say anything else that can describe you better. Your just a tool here, telling people that he’s done a lot of bad stuff. The bad stuff he has committed are personal issues, nothing that has to do with politics. He won his 2007 re-election with 72% of the votes, which occurred after the details of his scandal became public. Your statements have been proven pointless
He won by a narrow margin in 2003 and won in 2007 with 72% because, even though I like several of the candidates who ran against him, none of them had a chance.
And the bad stuff includes policy and budget issues as well as focusing more on the next step on the ladder rather than dealing with major problems in San Francisco.
It is good San Francisco is making more data available.
It would have been better to have a post by someone who actually worked on the project. Then the focus would have been more on the site instead of the politician.
Although it’s not my favorite site aesthetically, this is a nice first effort by SF. I’m excited to see more data, and more community brought into the site. Good work San Francisco!
“We imagine creative developers taking apartment listings and city crime data and mashing it up…”
As some here already pointed out: many of Gavin’s future vision is already available. Housingmaps & Crimereports have been around for ages, while Gavin just barely “imagine” it to happen.
While the move is applauded, it also sadly shows how oblivious & behind the curve the government is -even one as supposedly techie as SF.
neither of those use actual city data…. but now they can.
Lived in SF my whole life and ditto to what @charleybravo said. This guy is the biggest joke I’ve ever seen. Well, I take that back. Supervisor Chris Daly is the biggest joke in SF, but Newsom comes pretty damn close.
I hope Meg Whitman crushes him in 2010.
It’s so cute that you think Meg Whitman might win…
Way to troll for some votes on TechCrunch Newsom. I will NOT be voting for you. The last think we need is a slimy SF politician as governor.
Nice to see a mayor supporting transparency and promoting entrepreneurship at the same time.
A lot of elected officials should be inspired by this move…
Congratulations San Francisco! This is pretty huge news. Vancouver last May passed a resolution committing the city to open data, open standards, and open-source (where feasible), so it’s exciting to see other cities following suit.
For those interested, you can read a CBC article about Vancouver’s initiative at http://www.cbc....tware-city.html
I just hope the conservatives in Orange County can negate all the votes he’ll get from the lefties in SF whose a$$e$ he’s been kissing since 2003.
This is really cool and very important step forward.
I don’t know the guy policies, but I hope this initiative will give him some points, so other cities, states and countries will decide to follow.
Anyone knows if the data sets provided are in a propriety format or based on a universal standard schema?
This initiative is in the right direction.
This data is collected by government and paid for by taxpayers, it is by definition public information. That Newsom is making it available to the public should not be held up as a remarkable act of public stewardship but rather upholding a responsibility that government to its citizenry.
Now that’s he’s checked that off the list… how about fixing some potholes?
Privacy issues? The accuracy of the GPS data (accurate to a specific street address) in the police reports is somewhat unsettling…
In 3-4 minutes of poking around, I didn’t see their privacy policy.
Hmmm.. Just in time for Google to shut down their free wi-fi? http://www.dslr...-ISP-Huh-104019
Lots of good data but shouldn’t this site be part of sfgov.org e.g. sfdata.sfgov.org? All the data on sfdata.org can be found on sfgov.org, it’s like another directory on top of a directory!! White elephant, serves no purpose, just another political stunt!!!
Talk about confusing readers.
This is a good step in the right direction, however…
The site is currently just a collection of links to existing data on other city sites.
A few of the datasets are complete and in a usable format (Building Permits, Trees)
But most of them are not complete (only the last 30-90 days), cannot be downloaded at once (split into multiple files across neighborhoods), are in unusable formats (like 4MB KML files), or are all of these combined (Health Ratings = just a web page that you couldn’t even scrape).
The data formats should be agreed upon and made easy to find, like on data.gov. The content of each set should have some minimum requirements like downloading in one click, having historic data, having one human and one machine-readable format, and attaching latitude and longitude when possible.
So good first step, long way to go.
typical pr stuff.. thought you guys are better then that.. the mayor has no clue about anything.. in fact I find him rather dumb… can’t believe you go for that
Kudos.
@mb there is a link to their privacy policy at the very bottom of the site, tho I’m not sure if it covers your concern about the granularity of data revealed to the public.
I didn’t know Newsom was running for gov, but if he wins I’ll rejoice for his sanity regarding the drug war. Ditching vestigial drug policies, and focusing on compassionate addiction treatment, and celebrating responsible consciousness modulation, is immensely good for everyone.
How would Meg Whitman be qualified to be the governor of California?
We did a visualization of crime data using the DataSF crime feed from the SF police department. The visualization finds drug dealing offenses (specifically crack, heroin, and meth) that occur during school hours within 1,000 feet of schools in the city. Check out the video in the blog post to see how we filtered down the data:
http://blog.spa...n-safe-schools/