Creating email spam lists is a multi-billion dollar business. Most webmail providers long ago closed a number of the more obvious methods spammers used to put together their lists in an automated way. One example – you don’t get bounced email messages from webmail services for emails to address that don’t exist. That way spammers can’t verify if an email address is good unless they get a response (clicking the opt out link is one sinister method to verify an email is good) or include a tracking pixel.
Apple, however, has created a dead simple way for spammers to easily spider their idisk property to retrieve the entire MobileMe user name list. And each of those usernames can be converted to an email address by adding @me.com or @mac.com to the end of it.
Here’s how it works. Every MobileMe user gets a public idisk file sharing site where they can post files for their public or private use. It’s simple to set the page to private, but it still shows the username if you to to the page. An example of a bad username: idisk.mac.com/mehmehmeh-Public. Here’s a good one: idisk.mac.com/steve-Public (That’s Steve Jobs’ account). There is no way as a user to hide or delete your public folder. If you are a MobileMe customer, you have one.
Gathering the entire MobileMe username list, and therefore email list, via a simple dictionary attack is trivial.
Apple knows about the problem but insists it isn’t an issue because no one has complained publicly. An Apple representative said to one of our readers: “We’ve never had a complaint from a customer about people spamming them because of their iDisk public folder name. There is no way to remove your account name from the iDisk folders. I’m very sorry.”
So here’s our public complaint. The bad guys already know about this. Your engineers shouldn’t have designed the product without thinking this through. Please fix it.
Crap, I knew I shouldn’t have let my interns write the service.
mike…
umm.. where’s the issue? have you been spammed this way? has anyone you know been spammed this way?
i’m not arguing if what you say/describe might not be an issue.. it might be. i’m asking if in fact it really is an issue that needs to be worried about..
there are other less problematic ways for the bad guys to get email addresses..
peace
well, think of this another way. Apple has published the username and email address of every MobileMe customer. Is that a problem from your point of view?
How is creating a simple crawler to iterate over web addresses any more trivial than creating a crawler to iterate over email addresses? Surely you aren’t suggesting that this simple crawler fill in random values for the idisk URL and then check to see if the web server responds appropriately when they make a request.
If there are no links to these pages, a crawler will not find them.
lrn2internet, dumbass.
errr, huh? just do a dictionaty attack like he said. easy.
Spammers no longer spider to get email addresses; it’s much simpler to just generate a list of potential addresses at any domain they wish to spam and send them out. There’s no point in spidering these public folders to verify that it’s an actual email address; it’s too much work for no actual gain. In the time spent poking at an address to see if it exists they could have created another 10,000 potential addresses that they can try on any domain.
Anyway, if this was a real issue rather than some trumpted up bag of nothing, Apple would simply make the 404 page for an inactive account look like an active one.
haha!
“dead simple way for spammers to easily spider” I like this.. Let’s see if they are reading TC and how much will they take to FIX this.
Time starts now : Aug 21st 2008. 3.30
Cheers, Nag
You seem to hone in on controversy very well. Great skill! I’m still baffled as to why anyone would pay for such a service. You can get your own domain, with 20GB+ of storage, and email for $70/ yr. This is some prestige thing for the “rich and ignorant” (as Mr. Burns said about Ticketmaster concert attendees).
There is something to be said for the rich and ignorant…. it seems that the knowledagble crowd that knows how to setup a Web host, configure DNS and Web hosting, and maintain the process, thinks that the rest of the world knows what they do.
Keep in mind, someone might invest $1000 in MobileMe and other simplistic services, but they will turn 10’s of thousands an hour in their time running their business, all the while thinking they got a good deal.
Good investment for them.
Dipshit thinking for you.
When you are making over 100K a month, you’ll understand, otherwise…. Shut the Fuck Up (STFU for you wannnabees)
I’m sure you find it of interest to comment on blogs while “making over 100K a month”, but I think you just lost another 5K, dipshit.
I learned to setup all that you described in 4 hrs, sorry it was so difficult for you.
If someone is making that kind of dough and not running something better than mobileme (like exchange), they deserve to part with their duckets and get what is coming to them.
Typical dipshit elitist apple crap for people that think a salary is respectable.
So, does this count as a public complaint?
I guess I’m unclear about how they are publishing the list…. what they are publishing is a method to test for usernames, but that doesn’t mean this is crawl-able. Spammers will shotgun-send emails to every username they think might exist, without any ability to test.
So, I don’t see how Apple is publishing a “list” exactly, or how one could write a ‘crawler’ in the common-sense use of the word.
this is a way to verify an email address with out sending an actual email and hoping for a response.
Yeah, but that is pointless from the spammer’s point of view. They aren’t looking to assemble a golden list of valid email addresses; they’re just interested in pumping out as many emails as possible. They have 2 chances with each email; the fake sender and the recipient. It’s all just about swapping in new domains for each.
@sam, c’mon. this is bad. as if MobileMe didn’t have enough problems, but to make such an easy privacy mistake is just more crap piled on it.
yes, spammers can get emails & most of us get spammed everyday, but this is the kind of newbie security mistake that undermines the MM service.
you gotta ask yourself: if they made this simple of a mistake, what else are they missing — and what else am i exposing myself to if i join? phishing, etc…. i’m definitely less confident in a service that’s been slammed already.
& as for “no one complaining” or blaming Mike for raising a stink — well, that’s what he’s supposed to do & though i don’t agree with MA on everything, this time he nailed it.
apple’s got some great products (i love my iphone) but this is one of those CS101 mistakes that make you go HUH? Letting iphone app vendors jump to the top of the list with special characters was another noobie mistake.
I don’t think you actually attended a CS101 class…
How do you spider/crawl this information? Guess user names and keep hitting the site to check for matches? That’s not crawling. That’s just guessing. It’s simply easier for spammers to generate traditional email.
There’s no way I’m aware of to navigate from user page to user page through me/mac.com. There’s no directory. Is there? Show us. Otherwise, this seems like a silly complaint and doesn’t represent a crawling risk to information.
Bigfoot story. Now this. What’s next?
was gonna answer your dumb question, then realized that everyone else ignored it for a reason.
Clearly you have no idea what you’re talking about. There’s no list. There’s nothing to crawl.
No seriously, you don’t. Do you understand how much the value of an email address increases when you know that there is a user at the other end? I am going to explain this once in the comments here, instead of replying to every person in the thread here who has said the same thing you did and who doesn’t see what has actually happen here.
* In standard spam, the spammer would send to random usernames at a certain domain. Most common webmail providers now detect this easily and block it off.
* For the spam that does make it though, the spammer would include a tracking image (now also blocked by default in most email clients), or an opt-out link. The reason why they do this is because they then know that it is not only a valid email, but a LIVE email address.
* Out of tens of thousands of random emails a spammer would send or harvest, they might get ONE or TWO live addresses out of it.
NOW
this apple vulnerability – which is BASIC DESIGN COMMON SENSE, that hundreds of other applications CLOSED OFF YEARS AGO allows a spammer to PASSIVELY check if the user is valid or not. They dont need to send an email, they dont need a tracking image, they dont even need to worry about the email client. Instead they can HARVEST *EVERY VALID MAC.COM USER* because of this completely ridiculous oversight.
This error IS a privacy leak and no matter how you argue it including a live username in the URL which tells you if the username is live and valid or not is NOT OK.
@Aaron White:
They aren’t publishing a list or anything that you can “crawl”, but they do provide a trivial test to see if an account exists.
Run a brute-force check on every possible account name *once*, compile a list, then spam the list as much as you want.
Yes, but @David Geller points stands: it’d be simpler (even ‘cheaper’) to skip the test, and just fire the email off anyway. So brute-force can skip that step and be better off for it. So again, there’s no ‘risk’ here that is different from any other email service, on any server.
To label something a crawler, there needs to be a directory, which Mike hasn’t shown to exist.
you evidently have no idea how spammers actually operate. A fresh, guaranteed valid list is going to sell for a much higher price than an autogenerated crap list that’s been floating around spam forums for years. Add to that the fact that this particular list is all on one domain, so if you can bypass the spam filter you’re guaranteed 100% delivery, and add to that the fact that these users are demographically similar and you’ve got a much more valuable list than a hit and miss dictionary list.
The way people make money from spam is not simply pumping out as many emails per second as they can – people will pay for fresh, quality lists, guaranteed delivery, etc etc. it’s not just volume.
(disclaimer: I’m not a spammer, I’ve been marginally involved in fighting spammers in the past.)
A simple way to prevent this would be to return an empty iDisk page even for accounts that don’t exist, thus there would be no way to tell whether the account exists and doesn’t have files, or just doesn’t exist.
agree.
Agree. Tom’s fix should be implemented. It’s not a *huge* security issue, but Michael’s right to file a complaint.
Reminds me of the common problem where web sites divulge who is a user of their system by replying “address not recognized” to the “to reset your password, give us your email” web forms.
Tom & Michael
I completely agree with your assessment they should NOT be doing this and the fact they leak data like Steve’s got 100GB of data is really not what you would expect
between this and the awful IMAP implementation/deployment/migration they really need to sort things out how hard would it have been to use syncML and IMAP ? combine this with a bluetooth social network and turn off the friken GPS by default on the iPhone… just some of my surgestions
apple have spread their good engineers rather thin and their admins…
regards
John Jones
http://www.johnjones.me.uk
Actually, returning an empty page wouldn’t help all that much — it’s too easy to filter on that. They should be returning a fake page with seemingly valid information.
Correction:
They should be returning a fake page with seemingly valid (BUT RANDOM) information.
Tom – Run a brute force check on every possible account name once? Have you any idea how long that would take and how impractical that is? Every dictionary word, sure, but every possible account name? Get real.
Michael – “Apple has published the user name of every email customer.” Really? Where?
“Gathering the entire list is trivial.” Really? How?
Mobile Me user names are from 3-20 characters. How long do you suppose it’s going to take to check 26^20 = 19,928,148,895,209,409,152,340,197,376 possible accounts to see if they have a public iDisk page. 20 decillion accounts. That’s a heck of a list.
This is just silly. This same “alleged” test is also a great way for Apple to develop a blacklist, no?
If it’s so trivial, kick Nik off your couch and get him a-codin’.
“Apple knows about the problem but insists it isn’t an issue because no one has complained publicly.”
Starting to sound like Apple policy – I was going to buy a new iPhone 3g until I read about the wireless issues – now this! Way to go Apple!
Sorry, counted wrong; 20 octillion accounts. But’s that’s just the ones with 26 alphabetic characters in their names. There are plenty more with 25 letters. And 24 letters. And 8 letters and 3 digits, etc etc.
You can’t seriously suggest that you can brute-force your way through this to build the entire list.
Well, Gmail has the same issue I guess.
You can run url’s like this one through your favorite brute force crawler:
https://www.goo...chael.Arrington
That should provide way more email addresses…
Haha, thanks Idzard, @jerry. This is a FUD article for sure. Mike, who ‘tipped’ you off to this?
OH SNAP!!!
*settles in with popcorn and waits for Michael’s response*
they give out a captcha after 10 or so requests. sure it would be easy with a hired botnet or tonnes of proxies, but it at least raises the bar.
You also receive a different error page than “Account Error: Inactive” after fewer than 10 invalid MobileMe accounts as well. Not a captcha, but there certainly appears to be some security at work.
The fact remains that this system has been in place for EIGHT YEARS without reports of widespread spamming.
Yes, but with Google, after ~10 times you’ll get a CAPTCHA…
This is not the case with Apple.
Michael – where exactly is the crawlable list? Looks like you jumped the gun a bit here for some sensationalism. let’s see if you have what it takes to say “Ok, Sorry, I take it all back”.
OK my description is still wrong. 26^20 =~ 20 octillion accounts with 20 letters in their names, plus 25^20 accounts with 25 letters, etc.
But the takeaway is: “More possible accounts than you could ever hope enumerate.”
Dang, I wish I could edit a post!
26^20 accounts with 20 alphabetic characters.
26^19 accounts with 19 alphabetic characters.
Etc.
I will shut up now and wait for Michael to clarify his sensationalist article.
This is Classic Michael Arrington. Kicking ass and taking names.
This is a beat-up of epic proportions!
Every email service I know has a “check availability” type function on sign-up. They are all vulnerable to the same thing. It took me 30 seconds to find Google’s name checker for Gmail:
https://www.goo...amp;Email=steve
Replace “steve” with whatever name you want to check and you can see whether that email address exists @gmail.com
Come on Michael!
well yes but everybody fakes it and looks at the IP address/refer to determine if they should allow the address
apple by contrast tell me how hunch storage someone has…
regards
John Jones
http://www.johnjones.me.uk
This is interesting. A few months ago, I made an experiment and created a new account on Yahoo that was a totally random string (length 20) of characters, digits and a a few special chars (.-$). This **username** would be considered a very strong password…
I didn’t use the account, didn’t tell others, didn’t use it on Flickr or other Yahoo properties, just waited. After less than 24 hours the first spam arrived! The account still exists and still accumulates spam, although admittedly not that much.
I first thought that Yahoo employees, or Yahoo itself were selling new usernames, but maybe there are other explanations.
Little too much to drink, Mike? How much intellectual energy do you really spend on any of your “articles” or “investigations”?
Cant the same be said for Gmail users then? If you use products like Picasa, your username is the same as your email. Just crawl picasa, grab usernames, then add @gmail.com to the end.
Same thing, no?
I can’t resist, this is too much fun.
Suppose accounts can include any of 26 letters and 10 digits.
So there are 3^36 possible 3 letter accounts, 4^36 possible 4 letter accounts, etc. And an account can be from 3-20 characters long.
So the number of possible accounts is
( 3^36 + 4^36 + … + 20^36 ) = 13,749,422,954,239,300,000,000,000,000,000
Suppose you can go to the Apple web site and check one million of these accounts per second, which would be a heroic achievement.
Then it’ll take 13,749,422,954,239,300,000,000,000,000,000 / 1,000,000 = 13,749,422,954,239,300,000,000,000 seconds.
Or only 435,991,341,775,725,000 YEARS for Michael to “retrieve the entire MobileMe user Email List.”
Jerry, come on. That’s not how a spammer writes an algorithm… common words and phrases will account for 75% of the names…. so harvesting is an easy natural (no pun intended).
All that fancy, smancy math means jack biatch to a spammer… social engineering is the way to go..
Thank God you don’t work at a spam enforcer…
Oh for crying out loud, math right, explanation wrong. I have trouble describing huge numbers. There are 36^3 3 letter accounts (not 3^36) , 36^4 4 letter accounts, etc.
I think the rest of the math is right (checking my spreadsheet now …) Bottom line, it’s still 435,991,341,775,725,000 years. Michael, we’ll wait until you have that list ready.
No offense, but if you’d kindly post your corrections as replies to your first comment, it would be much easier for the casual reader to skip the umpteenth (oh no, let me correct, bazillionth, no wait, …) instance.
kthxnp
P.S. for n (e.g. 26, 36) possibilities per character, the words that are shorter than the maximum length make up a little less than 1/n of the total, which does not warrant a new posting for n > pi.
P.P.S. I daresay I verifitested that latter mathematicious claim with a spreadsheet of my own.
Definitely a bad idea to tie the idisk file site url to your email address. Just use a search engine to harvest email addresses with site:idisk.mac.com
*.mpg site:idisk.mac.com is a google hack for this, change the file extension to anything you want rather than MPG, no need to create your own spider, google has been there already.
Your quick “hack” returns 10 results. *.jpg only 1. *.doc only 1.
Cool, not too bad, seen worst, did you check pst, or other file types? Xls, and other fun things? I have yet to completely play with it cause I was blogging about it somewhere else. Thanks Tim! You saved me some time.
Starchy – Thanks, but a nerd fight can be a lot of fun, anyone got a video camera? That would be a fun video comment for Seesmic! LOL
You think 10 results is cool? Sad. I expected your “theory” to not hold water, but I didn’t expect it to not hold so few drops.
“site:idisk.mac.com mac”
should work better, but google still only returns 10. maybe these urls don’t get spidered much…?
now now nerds… calm down. just figure out what to do and then tell the rest of us mmmkay?
This is non-news. JeffTheGreat is right. You can scan Picasa accounts in the same way: http://picasa.g...le.com/username.
If Apple’s sysadmins are reasonably bright, a dictionary attack would be easily blocked. Moreover, it’s unclear to me whether this sort of attack is actually any cheaper than an email-based dictionary attack, something which spammers are already set up for and have expertise in.
err.. make that http://picasawe...le.com/username.
I’m just waiting for Jerry to publish his spreadsheet to Google Docs so the rest of us can take a look at his math.
Must be a slow day in the blogosphere.
LOL
Replace “apple” with “microsoft” above and everyone would jump on the hate wagon. but apple can do no wrong. even if they publish your email address.
No, it would be just as stupid which is why everyone is pointing out the same is true of Google, a more neutral target.
two differences:
- a name being unavailable on Google is different than an active account on Apple. When a name goes inactive, Apple disables the idisk page. Gmail addresses never really go inactive.
- the Google tool is far easier for Google to monitor for abuse by a bad guy than all the idisk pages on Apple. both can be abused, but apple is more open to attack.
yeah. google is wrong too. but that doesn’t excuse this ridiculous behavior by apple.
Your quibbling isn’t a difference; it’s ignoring what’s wrong with this: there is virtually zero value to spammers to attempt to verify e-mail addresses and even if they choose to do so, this by no means says you are exposed to spam — there are more effective defenses at the point of attack.
It appears this is an issue that many email providers have. I would think that this would still be an easy way to get addresses but being able to harvest the Gmail emails would be of more significance. I wonder if this is a known risk or if Mike stumbled upon a greater security issue.
Personally, I think there are so many bigger issues with MobileMe this one should be on the bottom of the list. MobileMe sucks. I’m one of the ones that was forced to switch from .Mac.
I never issued a problem with .Mac. But now I can never get MobileMe to work. It’s friggin horrible. I’m about to switch.
it sure does. ruined my entire work life. and i never even signed up for it, it just screwed up the delicate .mac ecosystem that i finally got humming.
Great stuff. doesn’t help the already wonky roll-out of mobile me. Let’s guess how many month of additional free use we get for this one.
In our local newspaper a couple of weeks ago: dead animals were found near the beach, along walking routes, among which an actual cat hung in a tree, in a sack. Public health department and municipal services both replied to the newspaper that ‘they never had any complaints’. There you go.
agreed. not good planning. another complaint is that with the .mac -> .me email address account switch over, Apple could have fixed the problem and allowed users to selectively opt into automatic email redirecting from one to the other. im so tired of my .mac spam i cant use it. that would have been a nice solution.
One difference between Google and MobileMe is Google is free. If I’m paying or a service I’d like a little more security than that.
@Jerry: To assume sheer brute force is the only way to get to those addresses
Even if it is not doable to quickly retrieve an entire list it makes me feel uncomfortable knowing Apple makes it unnecessary easy to connect the dots. Fact: This multi-million industry is getting better every day, their applied knowledge in mashing databases, combining, attacking is very very successful. Your overconfident attitude seems quite inappropriate don’t you agree?
Actually, no I don’t agree – with Michael Arrington’s assertions in his article that it is “easy” to retrieve the entire Mobile Me User email list simply by a dictionary attack that would try every possible address.
Look at the title of the article! This basic premise of his entire article is complete baloney. It would take eons to do attack. (I’m glad he thinks Mobile Me will be around that long though.)
It would take eons to attack according to YOU but hey, look outside, spammers just won another gold medal.
Well, according to basic mathematics it would take eons to retrieve the entire mobileme user email list via the technique Arrington is suggesting, but go ahead and believe in the magic powers of spammers if you like.
Isn’t the definition of a dictionary attack something that *doesn’t* cycle through every single possible permutation of characters, but instead goes through a “dictionary” of common words (or in this case names)? If this is what Mike means then your statistical analysis is irrelevant.
I’m not a hacker or a spammer but I did play Introversion’s Uplink quite a few times.
Arrington’s thesis – see the title! – is that he can collect the ENTIRE Mobile Me user email list easily. That’s my point, which I guess many are missing. He can’t be sure he has the ENTIRE list, unless he tries every possible user name, and that would take a kajillion years.
as i stated.. there are numerous ways for the “bad guy” to access potential email addresses.. what mike has got his panties in a bunch about just ain’t that big of a deal.
by the way, this normally happens when perp tech dudes try to talk about stuff they’ve never really developed/been exposed to…
peace!
The Mobile Web Gallery also has the same issue, regarding the user IDs, I have complained about this since the 1st day of launch, but no action has been taken. You can mask the Web Gallery by using a personal domain but the RSS feed is still consist of the user ID.
Apple needs to fix this, and if they are waiting till the spammers get to it, it is too late!
Back in the day, .Mac (now MobileMe) was the best service available. I stored my senior thesis and research materials on it, and thank God I did!
It also made it super easy to access that and other info while working in the computer lab. No hassle with disks or anything. This was back before there was “cloud” computing that apparently is considerable more secure than MobileMe.
I had been wondering why I was getting so much Spam. German Spam, too. I figured it was because Apple’s Spam filters weren’t up to snuff with MobileMe just yet (though they sucked when it was .Mac too).
But it’s good to know that Apple truly does protect my privacy by publicly displaying a folder with my name on it. Now I have complete confidence in Apple’s ability to protect all of my information from MobileMe all the way to iTunes.
Bravo, Apple. Bravo.
With a little help from EC2 this is definitely a feasible way for a spammer to get access to a ton of emails. I’d bet that the script is already running. Now it’s just time for Apple to shut it off.
The only real benefit here is confirmation that a username exists. For large email spammers, this is pretty easy with top-level domains. Then again I’m not a dedicated hacker so I’ll leave the spamming up to someone else. Just my 2 cents
Here’s a list of 38,000+ to get the spammers started, care of Yahoo:
http://siteexpl...2Fidisk.mac.com
Haha it works. Thanks for sharing.
they aren’t even instructing search engines not to crawl the pages:
http://idisk.ma....com/robots.txt
Why would they? The point is they are public web folders. Many people hosting files in their public directories want them indexed.
Apple is digging a hole, and the hole is getting bigger after this article!!
The simplest fix is for Apple to return the same success result for all possible values of username, regardless of whether that account actually exists.
An easy way to retrive the entier Gmail user List :
> telnet aspmx2.googlemail.com smtp
> ehlo techcrunch.com
> MAIL FROM:
(the server should respond “250 2.1.0 OK”)
> RCPT TO:
If the server repond : 250 2.1.5 OK, the users exists.
If the server respond “550-5.1.1 The email account that you tried to reach does not exist” : the user doesnt exists.
And this trick works on all SMTP servers.
Correction :
> MAIL FROM:
(the server should respond “250 2.1.0 OK”)
> RCPT TO:
(remove the spaces !)
SMTP harvesting doesn’t work on most servers… almost all SMTP servers will now return the same response no matter what email address you enter. The mail servers will then accept the message and then either bounce it or quietly delete if it is spam.
One of the biggest problems with Apple opening up the list of valid (Live) email addresses in this way is that they are also highly targeted addresses.
The spammers for instance know the list of addresses are MobileMe account holders and can then tailor specific Phishing attacks at them (Like the ones that were about a couple of weeks back).
Good catch Mike, the list of addresses on Yahoo is rapidly growing so the addresses are now in out in the wild. Too late for Apple to help these people!
See, this is one of those discussions where non-techies like me have no frikkin idea who is right (despite that impressive math by jerry!).
So many email addresses are already made public on sites and weblogs. I’d say fear the Chinese slave laborers who are taught to harvest the stuff by hand! And, oh yeah, those very tricky Nigerians too!
Last!
I’ve used itools/dotmac/mobileme for almost 10 years. My email address has not changed in that entire time. My iDisk public folder has been the same thing all that time. And I use my REAL full name as my email with no special characters. Pardon the use of Dvorak’s catchphrase but – I DON’T GET SPAM.
If all of these spamming geniuses haven’t been able to figure this out over the span of a decade, using the most kickass algorithms, all this computing power, and the frickin’ INTERNET, I’d have to say Arrington’s Theory is a bucket without a bottom.
You’re an idiot, if you think Apple didn’t think of this, and didn’t take some kind of precaution. That said, it would be nice to partition my iDisk and give each public folder a unique name/URL.