Yet Another Update: AOL: “This was a screw up”
Further Update: Sometime after 7 pm the download link went down as well, but there is at least one mirror site. AOL is in damage control mode - the fact that they took the data down shows that someone there had the sense to realize how destructive this was, but it is also an admission of wrongdoing of sorts. Either way, the data is now out there for anyone that wants to use (or abuse) it.
Update: Sometime around 7 pm PST on Sunday, the AOL site referred to below was taken down. The direct link to the data is still live. A cached copy of the page is here.
AOL must have missed the uproar over the DOJ’s demand for “anonymized” search data last year that caused all sorts of pain for Microsoft and Google. That’s the only way to explain their release of data that includes 20 million web queries from 650,000 AOL users.
The data includes all searches from those users for a three month period this year, as well as whether they clicked on a result, what that result was and where it appeared on the result page. It’s a 439 MB compressed download, expanded to just over 2 gigs. The data is available here (this link is directly to the file) and the output is in ten text files, tab delineated.
The utter stupidity of this is staggering. AOL has released very private data about its users without their permission. While the AOL username has been changed to a random ID number, the abilitiy to analyze all searches by a single user will often lead people to easily determine who the user is, and what they are up to. The data includes personal names, addresses, social security numbers and everything else someone might type into a search box.
The most serious problem is the fact that many people often search on their own name, or those of their friends and family, to see what information is available about them on the net. Combine these ego searches with porn queries and you have a serious embarrassment. Combine them with “buy ecstasy” and you have evidence of a crime. Combine it with an address, social security number, etc., and you have an identity theft waiting to happen. The possibilities are endless.
Marketers are going nuts over the possibilities, users are calling for a boycott of AOL, and others are just enraged:
User 491577 searches for “florida cna pca lakeland tampa”, “emt school training florida”, “low calorie meals”, “infant seat”, and “fisher price roller blades”. Among user 39509’s hundreds of searches are: “ford 352″, “oklahoma disciplined pastors”, “oklahoma disciplined doctors”, “home loans”, and some other personally identifying and illegal stuff I’m going to leave out of here. Among user 545605’s searches are “shore hills park mays landing nj”, “frank william sindoni md”, “ceramic ashtrays”, “transfer money to china”, and “capital gains on sale of house”. Compared to some of the data, these examples are on the safe side. I’m leaving out the worst of it - searches for names of specific people, addresses, telephone numbers, illegal drugs, and more. There is no question that law enforcement, employers, or friends could figure out who some of these people are.
There is some really scary stuff in this data.
I am assuming that AOL will take this page and the data down soon, but as of the time of this post it has been downloaded 809 times already. People I’ve spoken with are already building a web interface to the data. If you are an AOL customer, I feel sorry for you.
Note that Microsoft has proposed releasing similar data to researchers, although with an important difference - the data is not associated with a user. Excite released data very similar to what AOL has done here, with user associations, in 1999.
AOL is hitting bottom when it comes to brand image. This story comes on the heels of the recorded phone call with customer service disaster as well as a just-in story about a woman who is unable to cancel her deceased father’s AOL account, nine months after his death.
Comments
Mike,
Good post. I agree this is terrible, but does it make things any better to supply the link to the file? Aren’t you just propogating the privacy infringement even further this way? Unless this is your attempt at taking the file down with a traffic overload.
Hey Noam, the data is out there and blogs are buzzing. Too late. If I found it first I would have called AOL and strongly suggested they take it down.
This will be like dropping a nuke on the affiliate marketing world. Watch the spammers go nuts.
Hey, it’s not like there’s a been a lot of furor over the last year or two about data privacy…
Gone. Anyone with a mirror please post.
Microsoft was offering researchers similar query data earlier this year.
I was just giving you a softball Mike. See how much better your response comment made you look?
Madness. A “what were they thinking?” moment if ever I’ve seen one.
They weren’t. Thinking.
I’m having trouble getting the whole file. It looks like their web server is having as much trouble as their legal team is about to.
Danger: You’re not alone. I’m sure a couple of people have already successfully downloaded the entire file; it’s just a matter of time before a torrent pops up, and the file spreads like wildfire.
You’d think a company the size of AOL had more common sense.
AOL being a mammoth beast has been attempting to cut red tape within its ranks. This very effort might have enabled the tech/research team to publish such data without obtaining clearance through the legal/executive team. Big ‘whoopsie’ here, which is going to cost AOL a bomb.
I can see a couple of heads rolling already.
It’s not gone. I’m still downloading.
im still downloading too…
Yip, the download is stalling. The intro page has been wiped too (not surprising - it featured some pretty irrate customer comments). Story is now on digg and getting some traction, so I suspect there won’t be any hope of getting the file until someone sets up a torrent.
On another note, I’ve been observing TechCrunch’s FeedBurnet stats. Just a week or two ago, TC’s feed had 90k+ readers, but this number has been steadily declining ever since and now stands at 78k. What’s the deal, Michael?
Wow this is insane.
Wow. Just in time for Search Engine Strategies in SJ this week. Should be fun to visit their booth.
Mike - a good time for the shorts to come piling in?
I personally have never used AOL but I find a large number of people hate it and its services. I wonder what kind of uproar this will cause.
Joe Banix: This is pretty common with FeedBurner’s TC stats. They have to guess at a lot of things to measure active readers. Many more people check TC during the work-week, so by sunday evening, FeedBurner shows a lower # of subscribers. By Tuesday, you should see it shoot back up.