Ben Wellington: How we found the worst place to park in New York City -- using big data
Ben Wellington - Data scientist
Ben Wellington blends his love of statistics, the city, and comedy in his entertaining analysis of the story of New York City, told through data. Full bio
the infrastructure of New York City.
of our infrastructure.
released in reports by city agencies.
of Transportation will probably tell you
of subway track there are.
13,500 taxis here in New York City.
where these numbers came from?
someone at the city agency
that somebody might want want to know.
that our citizens want to know.
will have numbers like this.
all of our questions?
an infinite number of questions
and I think our policymakers realize that,
signed into law what he called
open data legislation in the country.
the city has released 1,000 datasets
the number of cabs,
When is rush hour exactly?
these cabs aren't just numbers,
driving around in our city streets
and I looked at that data,
taxis in New York City throughout the day.
to around 5:18 in the morning,
things turn around,
until about 8:35 in the morning,
11 and a half miles per hour.
miles per hour on our city streets,
there's no rush hour in New York City.
for a couple of reasons.
this might be pretty interesting to know.
4:45 in the morning and you're all set.
just available, it turns out.
a Freedom of Information Law Request,
Taxi and Limousine Commission website.
you need to go get this form,
did exactly that.
down to our office,
we'll copy the data and you take it back."
who wants to make the data public,
and that's where this graph came from.
These GPS recorders -- really cool.
walking around with hard drives
to make it public --
you could get to it,
walking around with hard drives.
is behind a FOIL Request.
dangerous intersections in New York City
the East side of Manhattan,
has more cyclist accidents.
coming off the bridges there.
There's Roosevelt Avenue in Queens.
we need for Vision Zero.
behind this data as well.
and paste data out of a PDF
than knew the logo. I like that.
that you just saw was actually on a PDF.
and hundreds of pages of PDF
you would either have to copy and paste
I'm going to write a program.
and it would download PDFs.
if it found a PDF, it would download it
some PDF-scraping program,
and then people could make maps like that.
the fact that we have access to it --
is a row in this table.
have access to that is great,
write PDF scrapers.
of our citizens' time,
the de Blasio administration
a few months ago,
actually have access to it,
still entombed in PDF.
is still only available in PDF.
our own city budget.
right now in PDF form.
that can't analyze it --
who vote for the budget
the budget that they are voting for.
a little better than that as well.
that's not hidden in PDFs.
in New York City.
of fecal coliform,
in each of our waterways.
the dirtier the water,
the small circles are cleaner.
by the city over the last five years.
in general, dirtier.
And I learned a few things from this.
that ends in "creek" or "canal."
the dirtiest waterway in New York City,
the Coney Island you swim in, luckily.
of samples taken over the last five years
to swim in the water.
that you're going to see
the front page on nyc.gov.
to that data is awesome.
on the open data portal.
a year or a few months.
of Environmental Protection's website.
sheet, and each Excel sheet is different.
you copy, paste, reorganize.
and that's great, but once again,
as a city, we can normalize things.
there's this website that Socrata makes
that don't suffer
and that's great.
be it CSV or PDF or Excel document.
you can download the data that way.
codes their addresses differently.
even when we have this portal,
normalizing our address fields.
of our citizens' time.
we can get more maps like this.
in New York City,
hydrants in terms of parking tickets.
and I really like this map.
on the Upper East Side.
you park, you will get a hydrant ticket.
grossing hydrants in all of New York City,
55,000 dollars a year in parking tickets.
to me when I noticed it,
what you had is a hydrant
a curb extension,
space to walk on,
and the hydrant --
painted there beautifully for them.
disagreed with this designation
who found a parking ticket.
Street View car driving by
on I Quant NY, and the DOT responded,
any complaints about this location,
and make any appropriate alterations."
typical government response,
something incredible happened.
the future of open data,
ticketed, and it was confusing,
they told the city, and within a few weeks
see open data as being a watchdog.
to be better partners for government,
being FOILed over and over again,
a sign that it should be made public.
releasing a PDF,
to post it with the underlying data,
is coming from somewhere.
coming from somewhere,
some open data standards.
here in New York City.
normalizing our addresses.
a leader in open data,
and set an open data standard,
and maybe the federal government,
where you could write one program
We're actually quite close.
empowering with this?
and it's not just Chris Whong.
going on in New York City right now,
attending these meetups.
and on weekends,
to look at open data
released something called citygram.nyc
to 311 complaints
or around your office.
you get local complaints.
that are after these things.
the students I teach at Pratt.
set of backgrounds.
and the ability of our citizens
and make our city even better,
or one parking spot at a time.
About the speaker:Ben Wellington - Data scientist
Ben Wellington blends his love of statistics, the city, and comedy in his entertaining analysis of the story of New York City, told through data.
Why you should listen
Ben Wellington runs the I Quant NY blog, in which he crunches city-released data to find out what's really going on in the Big Apple. To date he has tackled topics such as measles outbreaks in New York City schools, analyzed how companies like Airbnb are really doing in NYC, and asked questions such as "does gentrification cause a reduction in laundromats?" (Answer: inconclusive.)
Ben is a visiting assistant professor in the City & Regional Planning program at the Pratt Institute in Brooklyn; his day job involves working as a quantitative analyst at the investment management firm, Two Sigma. A budding comedian and performer, he also teaches team building workshops through Cherub Improv, a non-profit that uses improv comedy for social good.
Ben Wellington | Speaker | TED.com