December 28, 2006

Gmail Disaster: Reports Of Mass Email Deletions

Michael Arrington

150 comments »

Just a week after I wrote “Uh Oh, Gmail Just Got Perfect” a number of users started complaining that all of their Gmail emails and contacts were auto deleted.

The first message, posted on the Google Groups forum on December 19, stated “Found my account clean..nothing in Inbox, contacts ,sent mail..How can all these information residing in different folders disappear? ..How to write to gmail help team to restore the account..is it possible?..Where to report this abuse?.Any help ..Welcome..Thanks in advance ps101″

Other Gmail users then added to the conversation, saying that their emails had been deleted as well. Most of the users reported using Firefox 2.0 and that Gmail was open in their browser when the deletions occured.

The cause of the problem isn’t clear. One user wrote that after the deletion they received the following message: “This is not a mistake. All your emails and contacts have been deleted on purpose. This was a malicious attack and not an error. Have a nice day. =)” One user pointed to a known security issue with Firefox 2.0, which was fixed in 2.0.0.1.

On December 22, four days after the initial incident was reported, a Google representative posted this message on the thread:

Thank you all for reporting this issue. We apologize for the scare and
inconvenience that it’s causing. We’re actively investigating as we
speak, and we’ll follow up individually with users in this thread as we
get to the bottom of the problem.

We appreciate your patience and understanding.

Google’s official policy is that once emails are deleted, they are gone forever. And based on the Google Groups thread, no one has been able to have their Gmail accounts restored to pre-deletion status.

Update: A representative from Google just sent the following email:

Hi there TechCrunch folks,

We saw your post today about Gmail and wanted to let you know what was going on.

Regretfully, a small number of our users — around 60 — lost some or all of their email received prior to December 18th. Once we found out about this issue, we worked day and night to confirm that only a few accounts were affected and to do whatever we could to restore as much of the users’ accounts as we could. We’ve also reached out to the people who were affected to apologize and to work with them to restore the email from any personal backup they might have.

We know how important Gmail is to our users – we use it ourselves for our corporate email. We have extensive safeguards in place to protect email stored with Gmail and we are confident that this is a small and isolated incident.

Thanks,
Courtney

  • Sphere It

Trackbacks/Pings (Trackback URL)

Comments

That sucks. All my porn mail is gone.

 

Funny reading all those posts from distraught people who lost all their important messages which they stored on some remote site over which they had no control, information so vital they couldn’t be bothered to keep copies of any of it locally. And me without my tiny invisible violin. Shucks.

Oh, and if it’s not ALL CAPS, it’s not screaming.

 

This is the best reason I’ve heard in a long time to upgrade your browser. Scary.

We might say “google should keep backups.” But then again, for a lot of things, we probably don’t want google to keep backups. The gubment can read backups, afterall. Data security vs. privacy is an intrinsic trade-off that people don’t seem to think of much yet.

 
 

Back are the days where the company’s mission didn’t include storing people’s private and valuable information. Guess you can’t have it both ways. Someone at Redmond may be throwing chairs out of joy.

 

Wish it would’ve happened to me, I just can’t take the time to clean it out myself…a fresh start would be nice.

 

Thats crazy…bad news for google…what happened to users keeping backups on a disk!

 

GMail should allow users to back up all their data themselves.

It should simply allow each user to download a .ZIP archive containing all the user’s e-mails and contacts.

You may say this can be accomplished using POP3. This is not true - using GMail’s POP3, one can only access the last 30 MB or so of e-mail, even if he never picked up e-mail unsing POP3.

So, the POP3 of GMail has some limits.

Other great things GMail could do:

- Allow me to receive .EXE and .RAR files. I really need this because my co-workers are developers and often send me .EXE files to test (and guess what - I have never confused a worm with a legitimate .EXE file, because I am careful and can make the difference)

- Allow me to turn off the spam filter or configure it.

- Allow me to backup my data by downloading a .ZIP file

- Send me a backup DVD with my data every month by postal mail

In fact, why not implement all of the above, call it GMail Pro and sell it to me for $50 a year?

I would pay for this immediately.

 

Yikes. I do use Gmail as my primary email account, but have POP set up on my desktop, and all my old emails are there.

I shudder at the thought of all emails being deleted … I suggest anyone paranoid to set up a desktop client to suck all those emails down, even just once, so you have a backup.

 

Google’s official policy is that once emails are deleted, they are gone forever

Their policy doesn’t state that, it is:
Unfortunately, we’re unable to recover messages or Contact entries that have been deleted from your account.

The “we” in there might refer to Joe Schmo running gmail support that doesn’t have access to it. This policy is very vague, but it does not state that emails are “gone forever”

 

Way to keep backups, Google.

I’d sort of assumed that the “you delete it, it’s gone forever” rule was about them not caring enough or having the manpower to restore your emails, not that it was physically impossible.

Seriously, they have to have backups somewhere, or they are being incredibly lax about their business.

 

I just put this into my “WTF Dept” postings… in addition read the terms:

Indemnification. You agree to hold harmless and indemnify Google, and its subsidiaries, affiliates, officers, agents, and employees from and against any third party claim arising from or in any way related to your use of the Service, including any liability or expense arising from all claims, losses, damages (actual and consequential), suits, judgments, litigation costs and attorneys’ fees, of every kind and nature. In such a case, Google will provide you with written notice of such claim, suit or action.

 

Even with incidents such as this, I’d say GMAIL probably provides better reliability than most other paid services.

Incidents such as this can’t be good for Google’s PR and thus while they might not be paranoid about protecting your INDIVIDUAL account, they HAVE TO care enough to prevent stuff such as this. Being free doesn’t at all give them a reason to slack off (neither do I believe they do–gmail must be bringing ‘em good revenue).

-Zaid

 

re: #10
It says they’re unable to recover messages that have been deleted. How much more clearly do you need them to say that the emails are gone forever?

 

It is a still a beta in the end, gmail has been a great secondary mail account for me. Given the price Google is asking, there’s not much room for complaint… does suck for anyone that may have used it as a primary and lost important data though.

 

This is one of the dangers of the “Software as a Service” revolution that is being eagerly adopted now. It reminds one to be careful where you put your mission critical data! I agree that these types of services should allow end-user initiated backups/zip’d downloads.

 

@8 - easy way to send *.exe or *.rar is to just rename file and add +remove

i.e. windows.exe+remove or windows.rar+remove

you can send them then

 

Probably someone had access to the password and deleted all the messages. Your title is too sensationalistic.

 

That’s some scary crap right there. (setting up desktop client immediately)

 

It’s not that hard to back up your Gmail. I foward selected/important email to my PocoMail on a regular basis and save it .
Easy as falling off a log.

 

Hmmm. Take a look at this:

http://arstechnica.com/news.ar.....-6406.html

A judge ordered Google to produce emails that had been “deleted for ever.” Their terms of service (at the time) stated they could keep backups of such mail at their discretion.

 

If it’s all related to Firefox 2.0, the most probable culprit would be this bug: http://www.mozilla.org/securit.....06-72.html

The user would have to have received an HTML mail with embedded images, probably from an unknown party, and also voluntarily clicked the “Display images” link. If that is the case, it’s not GMail’s fault.

 

Surprised to hear that Firefox has the trouble; But I still wonder how that flaw is related to deletion of gmail mails? I don’t see a possibility of their gmail account being hijacked in this case.

 

I admit that this one really flipped me out, but instead of running around looking for a sandwich board, I just made a backup for myself, and realized that I’m responsible for my data, no mater where it is.

I wrote up how I did this (which I imagine there are variations), for friends and relatives.

 

I always register on the websites that techcrunch finds interesting. Suppose, for instance, that a malicious employee on some malicious website decided to ‘test’ whether users with gmail addresses gave them the same email password on the site. Most likely, that’s what you should have.

Expect this again in the future. It’s human nature, sad to say… –A.

 

Apparently, the bug allows you to specify a javascript: URI as your src attribute. If XSS checking is broken there, you can do all sorts of crazy stuff if you can fool the recipient into displaying your image within an HTML mail.

 

As a test of password security for my security clients, I once looked at one of my databases for all account owners that used yahoo.com for their email address and used their password (stored in my database) to hack into their yahoo account.

4 out of 5 times the user used the same password.

In theory, someone could have hacked into 100+ Gmail accounts and deleted everything *just* to give Google bad PR.

 

The commenting system ate my html tags.

To clarify comment #26: Apparently, the bug allows you to specify a _cross-site_ javascript: URI in your _IMG_ src attribute.

 

@jessup

a flaw with your database is that you could view and or decode the user’s password (unless that was the purpose of the ‘test’). if not, why weren’t the passwords in your database hashed with an asymmetrical key?

on the subject of remote services. there will probably come a time when everyone will own a CPU in a data-center which hosts all of your personal web-centric applications and data. Backups will be automated… -stan

 

Does anyone know of a quick way to back up a gmail account?

 

I have been using Gmail for a long time, so far I never had problems with Gmail. I can say I haven’t lost one e-mail in Gmail for the past 3 years. This is just a few users out of millions, it is better than Outlook and Hotmail combination. I access Gmail with Outlook, so I have them saved in both. I lose more e-mails with Outlook with their constant installing and uninstalling of software. I see none of the users here had problems with their mails deleted in Gmail, so I don’t see what the big deal is. Gosh I can’t even access hotmail in Outlook 2007 it says has problems with the server every time, so I delete hotmail accounts and read it in the browser instead. Even if it is a problem with Gmail, I am sure they will fix it right away. Just relax and enjoy the New Year.

 

Deleted emails? From a free, beta email program? We are surprised…how? Hey! you get what you pay for.

 

it’d be nice if gmail had a method to download compressed backup of the email.

 

I think the Beta issue is a lot of BS. Since 1998 everything is in Beta all the time. My real worry from this is that it confirms what some people have always said about ASPs and about using network-based services for critical functions. Has Salesforce ever lost client data? If the alternative to hosted systems is the Exchange Server we are running in my office - that goes down or has some kind of problem every couple days — then someone bring me a gun.

 

@Gmail Fan

You can download ALL of your gmail using POP. Just make sure to have it configured correctly within gmail.

http://mail.google.com/support.....topic=1557

pEace

 

This is a hard kick in Google’s groin and all those people who thought that Google would bring the next-generation Google OS for purely web-based computing.

 

This is always a risk when all the e-mails are on servers. Use POP and download them all as pointed by Gabriel. But not many people are aware of this or do not take pain to configure POP tools. Google should themselves come up with a desktop program that does all the tricks. I don’t blame google for this. 60 accounts out of millions?? Wait, what is the percentage of error?

 

This is really unacceptable. A data recovery model - with a fee if needed - should be a required feature of a personal data service hosted on the web.

As for “perfect” per the post a week ago, there are so many things Gmail is missing that it’s not even close to being perfect (see my post on CrunchBack.com for a starting list). I do enjoy Gmail, but I’m not as easy to please as you, Michael.

 

We all know that Gmail is not yet become public and its is under TEST and there can be some mistakes that can happen, but the deleting of the mails is too much and something wried… So i personalize mi important mails….

 

You should never lose your email, have your gmail forward to a windows live mail account which sorts it into a gmail folder, instant backups….

 

Someone’s already pointed out that POP3 isn’t a perfect solution to this. However, now that Gmail offers POP3 retrieval, you could theoretically create a backup account that constantly retrieved mail from the other account. Although I don’t know if Google’s POP3 retriever would be able to be configured to retrieve Gmail via POP3.

 

While it is never good to be in a position of having lost data, I think it is also comforting to know that even the best company can make a mistake.

 

Well, look we have here. A google problem with mass deletions of gmail contacts and email through Firefox bugs, and what happens? Apologists come out of the woodwork.

Some guy actually says “Just relax and enjoy the holiday”. WTF?

Imagine this were a problem with Hotmail instead of gmail, or with IE7 instead of Firefox - what would these comments be like?

Stopping giving Google the free pass. Be objective. This is a major fucking problem.

 

Jessup (#27) — DID YOU ORDER THE CODE RED??!!

 

Huh. I know I have two gigs of storage, but it only fills as far as it does out of laziness. I routinely purge my email about once a year. There’s rarely any valuable conversations I really care to go back to… so why keep it around? ‘Sides, something like 2/3rds of my email is just LiveJournal telling me that someone has posted a reply to my account. That’s half of what I use my bloody email for anyway. Wouldn’t have chapped me, much.

 

Somehting is seriously broken if even just a few gmail accounts lose data. It’s only a matter of time before this weakness is exploited if the problem is trivialized.

I hate to think how many people working in gov, military, finance etc use it as a convenient means to send/save data back and forth..

Anyway, Eudora is the way to go imo..zero latency.

 

Bugger….Not a good end to 2006 for Google. I thought GMail could do no wrong. Guess I was wrong.

 

First when Google comes out people are shouting invasion of privacy for keeping emails forever. Now this incident and people are complaining they don’t back up?!

idiots. back. it. up. yourself.

 

This problem existed since early September 2006 as far as I know. How do I know that? Well, I am a victim of this email bug. I contacted google via email several times regarding this matter and they consistently replied that they checked everything and the only place where the fault occurred is on the user side. after several letters I just said: “Whatever”. Well, guess what? a week later, same thing happened to me again - I lost all of my emails!!! Sent them emails again and again received the same reply, which read that it was something I was doing. I think they knew of this problem long time but could not do anything about it so they waited until firefox updates.

So what is the solution I have now? I have a 2nd gmail account for backup purposes.

 

FIREFOX VIRUS, NOT GMAIL FAULT, damn idiots. Gmail is damn good for email, people not using a reputable AV, firewall, anti-malware, updated system are at fault due to a virus!

 

My Firefox 2.0 cannot open my GMail page. It displays “Loading” red label and no more.

 

GmailFan:

- Send me a backup DVD with my data every month by postal mail

You have obviously no idea what the word privacy means.

 

Sixty folks lost their ‘all precious’ communication. With Gmail kind of user base, perhaps more people would have lost such ‘all precious’ communication due to desktop crashes in this period. Do not think storing mail on your desktop is the copper coated solution we look for.

 

Rajesh and Zaid are right.

1. Storing locally remains far less secure than storing remotely on Google’s servers, which are protected by multiple layers of redundancy against theft, fire and hardware breakdown.

2. It is absolutely not in Google’s interest that this glitch spreads. If it does spread, Google will surely introduce a catastrophe backup-restore solution.

Having said that, one user’s mail deleted is one too many. Google should be finding a way to get to the bottom of this problem, case by case.

 

Its interesting that Google didn’t comment on the ‘malicious attack’ note that was left in Google Groups. Was it an attack or not? If so they need to disclose that to the public.

 

Despite what the “official policy” may say, everyone knows that Google has, and will always have, access to the “deleted” messages. I guarantee you, if the gubmint came looking for those emails in their War of Terror, Google would roll over and give them up in a heartbeat. They simply choose not to restore the deleted files so they don’t set a precedent.

 

google does keep “backups” in the sense that if a server hard drive housing your emails dies you won’t lose them because copies are kept on other hard drives. replication, they call it. but if a piece of data is deleted by the system on purpose, as seems to be the case, all copies are deleted.

 

Hey…the world of technology is so intriguing that we often times forget the vulnerabilities. I do honestly believe that these seemingly “isolated incidents” are signs of bigger things to come. For all those custodians out there, be warned……

“Having said that, one user’s mail deleted is one too many. Google should be finding a way to get to the bottom of this problem, case by case.” - Rollo

Rollo - You are absolutely right, but as I said Custodians out there are also just as responsible…

http://jollyjo.org

 

But what I’d like to know is…what happened? Just because it affected only 60 people out of thousands doesn’t mean it won’t happen again, right? I’m not sure about #51’s claim that it is a VIRUS but even if it is a Firefox issue and not a Gmail issue, shouldn’t they be working together to solve it so it doesn’t happen again? For all we know it was a bizarre key combination the 60 people didn’t realize they hit…Google hasn’t said what CAUSED this.

And I definitely remember hearing way back at the beginning of Gmail that they NEVER delete anything. Even if you purge it from your Trash, it’s still somewhere. Now, it may not be programmatically possible to restore it, and if not then I don’t see why they keep it. But unless this policy has changed, I think the missing e-mails and contacts are still out there somewhere.

 

This is one huge reason why the ‘gmail is perfect ‘was always mindless hyperbole. Not because of the f*up but because any app that doesn’t have a *built in* way to export your data into a pliable, portable format shouldn’t be worthy of even being called Web 2.0. That should be a core principle but instead a core principle seems to be starry-eyed cheerleading. I can only imagine the gnashing of teeth if there was no way to backup or export your email out of outlook and entourage. Shame we can’t hold the new wave to the same standard.

 

Thats what happens when you use cheap servers!! This is just the beginning, Google’s honeymoon phase is over.

 

Maybe that’s why they call it beta? Sorry, just use Gmail for not-so-serious stuffs ;)

 

I really don’t see what all the fuss is about. Oh 60 people were affected out of how many Gmail users? Holy Sh*t! The end of the world is near!

1. Isolated
2. Firefox/hacker related

That’s all you need to know. No need to find an alternative email solution. However, using POP3 to download your messages from time to time is smart. No need to lose “faith” in Google. God forbid a FREE web service (beta) isn’t perfect. I understand the importance of email but I hardly call this scary. Maybe if it happens again I would start to question the integrity of their system but come on people…calm the f*ck down.

 

it sounds like BS to me
use pop3 once a month or so and prob solv
trust no1
ganjarules

 

Doesn’t 60 seem like a really low number? I count 10 people who complained about deleted email on the google group message board. Seriously, if there were 1 million gmail accounts, the probability that a user is affected is 0.00006. Assume there are 10000 users on google group, then the probability that a user is affected and is also a member of google group is 0.0006 * 0.01 or 0.000006. The chances of getting 10 users is now about 6.04 * 10^(-53).

The difference between permutation and combination increases the probability, as does blogs and outside factors, but really… this might as well be 1/googol.

 

Yet another reason I’m glad I stay the fuck away from Google.I hate Google.There are many far more useful tools for doing ANYTHING Google is supposed to do.Even the name annoys me.

 

If I walked around a conference floor and handed out 2Gb cards to people, asked them to load them up with critical personal information and give them back to me for safekeeping, they’d call me crazy.

Yet that is exactly what happened with GMail. Google had no brand history with storing people’s critical data, and yet millions swooned over the idea of 2gb free storage, and didn’t think twice about trusting Google with a critical task, despite the fact that it was previously just a search and news engine.

 

All my emails were deleted on Sept. 1st while I was in my gmail account I was watching them disappear in front of my eyes. And I got the same BS from the folks at Google that Edgar (comment#50) got. The weird thing was that it also deleted all the personalized modules I had on my google homepage (google.com/ig).

 

I use my gmail account to store remote backups of some important jobs. I do local backups too, but I always have had the security that my remote backup on gMail was more secure than mine.

Now, thanks to this thread, I have learned that gMail is actually a shoddy piece of work (in Spanish we have a specific word for this: chapuza). As a techie one of my functions is to do a daily backup copy of the important folders of my computer and I always have encoraged my job mates to do it. In my opinion do backups is a basic requirement of a computer profesional.

And now I discover that Google doesn’t do this, they haven’t one of the basic features that it is presumed about a web service.

I like gMail, is easy to use, free and very powerfull. But Google people must do changes until I return to trust my important documents to it.

 

Why is no one joining the archiving idea proposed by a previous commenter
I would also appreciate one such addition to the otherwise simply perfect email reader.
Please let us archive gmail!

 

Now this is scary. I use my gmail account to back up some of my important docs and if my stuff is not backed up then why am I backing up? I like gmail cos it is easy to use and powerful but this has eroded a bit of my trust and I wont recommend it till this mess is cleaned up.

 

this relates to gmail! not the same issue, BUT… i recently purchased the 8703 blackberry (T-mobile) and configured my gmail to it… was working great and now i dont recieve ANY incoming emails, but can send emails…. its driving me crazy. Google needs to start getting there hands out of all the baskets and start focusing on the bread winners.

 

i can’t login. my account is there, i just can’t access it.
yay for google.

i just hope some admin will wake up from a massive hunover and realize he unplugged some servers by mistake.

 

In the letter you got from Google, they stated that they are working with the people to restore their back ups. Ok, from what I know you cannot save or export gmail e-mail. So does that mean google is encouraging to stop using their interface and start POP’ing it to Outlook/Thunderbird? Then if that is the case one of google’s things thet bank on is that interface, so then how would gmail be different from any other e-mail service/provider?

 

That’s way I still have outlook downing all my emails from gmail with pop most times I start up my PC. I don’t trust any single company to look after my emails for every.

However I am now thinking of getting my DNS hoster to forward my emails to two different web email providers as well.

 

I have another question…After installing firefox 2.0, it seems that my gmail account is open across browser sessions. It closes if I “log out” but before 2.0, all I had to do was close the browser and the session would be ended. Could this possibly be a part of the security issue? Also…does anyone else have this “problem”?

 

Tiscali may have the same problem. I do not know if they are a service provider in the USA, I am from the UK. I am out and about in the world and therefore relying on internet cafes and it looks like an old Mozilla browser may have deleted all my tiscali webmail folders, messages and the address book. No message displayed though. I’m off for a lie down.

 

I use the Safari browser on Mac OS X. I had my mail cleared out too, but only from my inbox, not my archived mail…

Of more concern was when gmail was down recently (17th/18th Jan?) for some time, and I lost only the mails that had been received in the last couple of days.

As I travel a lot, I don’t have the leisure of having permanent access to my mail, so saying that mails lost are my problem, and not the problem of the mail service (catch that? mail service) that I connect to, before I’ve had the chance to connect, is ridiculous. Either they can provide a reliable, trustworthy service, or there’s no point in using it at all. Have Google actually said what caused the problem yet, on either occasion? Or are they content to hint that it is the fault of third party software?

The BETA software defense is also pathetic. GMail has been in use for years - why is it still in BETA? Google obviously use the BETA label to avoid the responsibilities that go with properly released products. I don’t understand how they can employ this particular tactic and still maintain that they adhere to their original mantra of “Do No Evil”.

I quit.

 

Lost all my emails from past 2 yrs from Gmail. Worst part, there is no customer service email or website for google where I could atleast report coz from the above aritcle, I’ve already lost my important emails. Cant believe google has acted so irresponsibly using words like ‘Beta’ and ‘Free’ and then not be responsible for losing important emails.
P.S: I would appreciate if anyone has a address where I can report this incident.

 

Well this is a surprising news, will all those who lost their mail be in a position to let other’s know what mistake had been done at their end or say what browser they had been using, as there has to be reason for those 60 persons to have their email get deleted. Thankfully mine is fully intact, rather the only hitch faced is the labeling system as the folder system does look more easy and compact, always wish to keep the inbox light with named folders to keep the emails in them even if it means to be read later on. Now have more than 300 mails and have to go about labeling them and then consider which ones get deleted - wish Gmail could give the folder system to its wonderful mail system that keeps increasing space, mine is 2.8+ gb thanks Gmail do look into the Folder system please.

 

…>>>>>Allow me to receive .EXE and .RAR files. I really need this because my co-workers are developers and often send me .EXE files to test (and guess what - I have never confused a worm with a legitimate .EXE file, because I am careful and can make the difference)

You can do that. Just change the extension of the file. And upload.

 

God dxmmit i have my mails lost too! All mails after 6-Dec-2006 r gone! O god!
I did used a Thunderbird extension called Webmail (http://webmail.mozdev.org/) just last night, havnt check if it downloaded all mails in gmail. Hopefully yes. If not! God! No wanna think about it.

 

http://digg.com/software/Gmail.....t_of_users

This has started again … lets hope all our mails are safe

 

Aaaargh

My mail from 14.12.06 to present day has just dissappeared. What the f+ck am I supposed to do about it? There is so much important information in there that I really need to access. Does anyon have any idea how to help me?

In hope, Simon

 

I lost all my gmail from Dec 26th to March 3rd.
There is no place to file a complain either.
NOT RELIABLE AT ALL!!!!!

I BETTER START USING MICROSOFT OR YAHOO AGAIN. ATLEAST I WON’T HAVE TO GO THROUGH THIS PAIN!

 

Sorry, the comment form is closed at this time.