« Previous post
Next post »

November 25, 2007

Alexa’s Make Believe Internet

Michael Arrington

76 comments »

Amazon’s Alexa traffic reporting service has little credibility left among people who follow traffic trends. Most analytics services, like Comscore, don’t measure small sites well, but they tend to get it right for the larger sites. Alexa seems to get everything wrong, no matter how large or small the site.

Example: In August Alexa said that YouTube passed Google itself in total page views. They were wrong, but their data continues to perpetuate this alternate reality.

Now, another embarrassing error. Alexa says that Facebook, on a steady growth curve for the last two years, now has a larger audience than MySpace. This isn’t as ridiculous as the YouTube/Google error, but it’s still way off. Comscore says that worldwide MySpace uniques are 109 million/month, whereas Facebook is at 86 million. Compete.com, which measures traffic using similar techniques as Alexa, stills says that MySpace is larger than Facebook.

Thanks for the tip Mark.

CrunchBase Information
Alexa
Information provided by CrunchBase
  • Sphere It

This entry was posted on Sunday, November 25th, 2007 at 2:07 pm and is filed under Company & Product Profiles. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Trackbacks/Pings (Trackback URL)

  1. THE CIRCLE OF LIFE

    November 25th, 2007 at 7:47 pm

  2. Alexa’s Make Believe Internet - Soft-Go.Com Blog

    November 26th, 2007 at 12:33 am

  3. Support this story on Stirrdup

    November 26th, 2007 at 10:24 am

  4. Andreessen Responds On Ning Porn: It’s About Being Pro-Freedom

    January 5th, 2008 at 9:41 pm

  5. Our Techno Lyrics » Blog Archive » Andreessen Responds On Ning Porn: It’s About Being Pro-Freedom

    January 7th, 2008 at 2:28 pm

  6. Alexa Overhauls Ranking System

    April 16th, 2008 at 6:03 pm

  7. Veranderingen in Alexa ranking | Jelmer de Jong

    April 17th, 2008 at 12:57 am

  8. D.I.A Free News Online 2008 » Techcrunch: Alexa Overhauls Ranking System

    April 17th, 2008 at 6:31 am

Comments

RSS feed for comments on this post.

  1. Dan Schawbel

    November 25th, 2007 at 2:15 pm

    They don’t even track all websites. Quantcast is much better.

    http://www.quantcast.com ENJOY!

  2. Adam Sharp

    November 25th, 2007 at 2:24 pm

    Interesting, but not too surprising. Facebook users are more likely to be interested in things like web traffic, so they’re more likely to have have Alexa’s toolbar installed. Of the thousands of people building/marketing facebook apps, a good % probably have it installed. 1000s of users frequenting your site with the TB installed will spike the heck out of your Alexa score, even at this level.

    That’s the problem with Alexa data, it’s not anywhere near a random sample. The more internet-marketing & tech-saavy the audience, the more inflated a site’s traffic will appear. Some engineeer from Google did a study, good read:

    http://norvig.com/logs-alexa.html

  3. RBA

    November 25th, 2007 at 2:24 pm

    One problem with Alexa (internally) is that while we all know their data depends on people with the toolbar installed, they don’t seem to be doing anything to get more people to install it. As such, their stats become more and more irrelevant overtime…

  4. Tony

    November 25th, 2007 at 2:27 pm

    http://www.compete.com, and quantcast from above, http://www.ranking.com are other services.

    Alexa also has a redirect problem, where people put redirect to alexa then on to the website to boast scores. Gaming the system

  5. Michael Arrington

    November 25th, 2007 at 2:29 pm

    yeah, i think the problems with Alexa are well know. What I don’t get is why they don’t make any effort to fix those problems.

  6. Kelli

    November 25th, 2007 at 2:31 pm

    Anyone in the SEO business has long known that a diaper wearing monkey throwing darts at random numbers is more accurate than Alexa. If I hear someone bragging about their Alexa rating, to me and many others, their credibility goes right through the floor.

  7. Duncan Riley

    November 25th, 2007 at 2:40 pm

    Alexa may be far from perfect, but consider that comScore gets its figures through downloads of an “internet accelerator” that some places have referred to as spyware. Before the investors jump in defending comScore (Fred Wilson is on the board) they naturally deny this, instead referring to their program as “researchware” but at least consider the distribution model: a program that is marketed as a download accelerator to PC users. Do you really think that PC users who download an internet accelerator make for a sample audience that is that much better? If anything comScore stats weigh against heavy tech and sites that may find favor amongst first adopters…and most certainly against sites favored by Mac users. comScore gets it right only as far as PC users who download internet accelerators get it right, and nothing more. The overall usage figures are then applied on estimates of sample size applied to overall market…which makes it, like any stats program, an educated guess at best. If comScore says a site has 16m users it may only have 100,000 comScore users in the particular market then applied statistically to how many they think it would have based on this…but again, the sample is narrow and not necessarily representative of the broader market.

  8. Allen Stern

    November 25th, 2007 at 2:43 pm

    Alexa has been an issue since way before Web 2.0 explosion. I’ve been dealing with the crap for years - probably since 2000ish. I always enjoyed when someone would try to get me to sign millions of dollars to their service based on Alexa charts. Those days I enjoyed :)

  9. NY Giants Stink

    November 25th, 2007 at 2:50 pm

    Duncan, very informative. Thanks for sharing…

    btw- isn’t Fred Wilson a VC; not a technologist, CTO or an authority in the space?

    Sincerely,
    NY Giants Stink

  10. Mark

    November 25th, 2007 at 2:52 pm

    Thanks for the credit and link Michael!

  11. Akash

    November 25th, 2007 at 2:58 pm

    Compete aggregates data from 10+ sources, including ISP logs, ASP’s, toolbars, Panels, etc to form its 2 million person sample.

    Check out this interview with Guy Kawasaki about Compete for a whole lot more - http://blog.guykawasaki.com/20.....ons-2.html

  12. WinAndFun

    November 25th, 2007 at 2:59 pm

    I can confirm in more than 10 cases for my stats that the stats of Alexa is wrong… :-)

  13. Duncan Riley

    November 25th, 2007 at 3:03 pm

    NY Giants Stink
    thx.
    Fred Wilson (VC) is an investor in comScore and also sits on the board, least he did about 12 months ago unless he has stepped down in the mean time :-)

  14. Hi-Tech-IT

    November 25th, 2007 at 3:03 pm

    It’s ok for most people I guess. I used Compete also. How accurate is Alexa? No idea, but when you dealing with millions of sites who knows.

  15. Michael Arrington

    November 25th, 2007 at 3:03 pm

    Of course it’s wrong. Like a broken clock, it’s correct twice a day, but that’s about it.

  16. Zuckenburg

    November 25th, 2007 at 3:06 pm

    My site is clearly bigger than myspace.

  17. Alex

    November 25th, 2007 at 3:08 pm

    Comparing Alexa stats (international) with Compete (US-only) without even mentioning that discrepancy seems almost as silly as using Alexa for important decisions.

    *walks away from the dead horse*

  18. Oracle

    November 25th, 2007 at 3:14 pm

    The main problem with Alexa is that they deal with irrelevant stats, like overall rank and reach, which say very little.

    And of course the Alexa system is too easy to game to get into the top sites list with very little real traffic.

    Compete and others are more useful due to showing the approximated no of uniques, even if they way underestimate them, but you learn quickly how much they underestimate.

  19. Arthur Chaparyan

    November 25th, 2007 at 3:26 pm

    Not to mention that it shows MySpace traffic as unrealistically constant since May

  20. Mark

    November 25th, 2007 at 3:28 pm

    Did anyone hear of Syntryx? Alexa’s method is simply not capable of dealing with the large mass of websites out there. Compete is definitely better, especially now with their new Search Analytics. Quantcast is a different type of a bird. I don’t think trending is their true focus nor vision. They got something else in mind. There’s are few more alternatives out there, however I think that Syntryx unique technological approach to competitive analysis enables comparison of both Long, Mid, and Front tail websites is more complete.

    I don’t blame Alexa for not being able to properly assess trends for the mid and long tail domains; however it’s disappointing that they are incapable of providing proper trends for the front tail domains where their methodology should have been sufficient.

  21. Andy Beard

    November 25th, 2007 at 3:32 pm

    Compete isn’t any more accurate than Alexa, it all really depends on how many people you have using a compete toolbar.

    Quantcast is about as accurate as most javascript tracking scripts, but there is a major problem, most websites don’t include their code.

    If you went by Compete data, I should be charging the same for advertising as Techcrunch were in June this year….
    http://siteanalytics.compete.c.....?metric=uv

  22. Dallas Freeman

    November 25th, 2007 at 3:34 pm

    I don’t know why Google hasn’t came up with a way to measure Internet traffic, I think they would have a better idea. If only they could partner up with Yahoo, and MS Live Search.

  23. rick

    November 25th, 2007 at 3:45 pm

    I think all of these guys were in make believe statistics class. Compete, alexa, hell even nielsen and arbitron, every ratings company under the sun use such horrible methods that I’m surprised any ad campaign has ever succeeded.

  24. peterlake

    November 25th, 2007 at 3:46 pm

    The counting wars were over a long time ago. All counting has a frame of reference. If you work with the frame then insight may follow.

    Look for consistency in data collection methodology whatever it is. Strongest comparisons will be trends for like with like by collection sample, site audience, site structure, site use and so on for any metric.

    Ranked position is always whatever for any counting methodology - very subject to the what, how, why of the methodology. The tighter the online methodology to get ranked position right - the less useful the data seems to become.

    Alexa is very useful for it’s global coverage and length of service. Be nice to Alexa.

  25. Steve Ballmer

    November 25th, 2007 at 4:09 pm

    Alexa is right on the facebook thing, more people are using it since they found out that I invested in it!

    http://fakesteveballmer.blogspot.com

  26. Chris R.

    November 25th, 2007 at 4:13 pm

    http://www.alexa.com/data/deta.....ftware.com
    Alexa has our 1wk at 195,917 and our US rank at 60,260.

    That’s pretty good for a website that only does outsourcing to a handful of clients.

    BTW, if you haven’t heard, BeerCo is coming to town next year.
    I figure a search released anywhere else will never get used or seen. Getting beat down by 75% tax didn’t help either. See you all next year.

  27. Steve Jobs

    November 25th, 2007 at 4:26 pm

    My website went from 298,987 to 1034 in one day and then back to 490,871 the next.

    It is worthless. The only people who look at it are out-of-touch CEO’s.

  28. ZiZi

    November 25th, 2007 at 4:39 pm

    i DO NOT tamper with my alexas ranking for one reason only.
    I use alexa for myself, i want to know how good my site is doing based on their users. i could care less what people think of my alexas ranking and this is why i dont try to bump it up (and i can easily)
    Alexa although not accurate it is the most accurate of what is available today.

  29. Dan

    November 25th, 2007 at 5:18 pm

    A lot of very good points here - Alexa is easy to game, and biased towards one set of data. As is Comscore, Compete, Hitwise, etc. But this kind of data is absolutely necessary, and nobody seems to have any idea for making it more accurate.

    To make it about 98% accurate, the browsers would have to build in the code across all operating systems - with screams and howls from the data protection lobby.

    Hence, for the foreseeable future, you have to look at all of these sites, and use a bit of common sense mixed with a fistful of salt.

  30. King Tut

    November 25th, 2007 at 5:31 pm

    Alexa is 100% right - every time. Spot on. I set my watch to it every day - twice each day.

  31. Ron J

    November 25th, 2007 at 6:28 pm

    I have worked in the research group of a large publisher for over 8 years. I have been tasked with trying to find out what tracking service is the best and have been working on this for almost 3 months now, without help from any of the services. We want a true independent analysis.

    I have found that you get what you pay for. Compete is terrible, Alexa is worse, Quantcast is bad (though people seem to like it), and the services you pay for are the best. Of course the smaller the site the harder it is to measure and I understand this being a researcher. I just think people forget that free data is usually crap.

  32. Gustav

    November 25th, 2007 at 6:37 pm

    Alexa, Compete, Comscore, they are all way off. The problem with Alexa is that it is so easy to game, so that just one person can really affect a site. As to why they don’t do anything about it- I’m guessing they just can’t come up with anything.

  33. Justin

    November 25th, 2007 at 8:30 pm

    Alexa is for trend watching, not statistics. And the best tool for trend watching there is.

    I develop a site that has grown steadily since I came on board. This growth has been reflected in Alexa at an uncanny accuracy.

    It is not fair that Techcruch continues to monitor Alexa accuracy by focusing on the behemoths of the web. I don’t think we need to use Alexa to gauge these trends. There is a need for something that does it well.

  34. Burgo

    November 25th, 2007 at 9:14 pm

    @21 (Andy)
    Don’t forget us little people who knew you when… :P

  35. Captain Obvious

    November 25th, 2007 at 9:34 pm

    You know how the old saying goes?
    Don’t trust a statistic that you haven’t faked yourself.

    Alexa, Compete, Comscore, it’s all the same garbage.
    They all seem to be using the same flawed tools, relying heavily
    on spyware toolbars and “magic” extrapolation. Keep in mind
    that only the most clueless (”bottom of the barrel” users) install
    those toolbars. So if your site doesn’t attract a reasonable
    portion of mouthbreathers - too bad, you’re off their radar.

    Complaining about alexa is like complaining that there is only
    crap on tv.

  36. Frank

    November 25th, 2007 at 9:35 pm

    None of them are accurate. if you look at stats from Hitwise and comScore for the same website, they have very different numbers.

    Also, many of the services measure things in different ways (daily unique, monthly unique, visits, visitors, pageviews, US-only etc)

    For a comparison of different competitive analystics tools, try this post: http://lifeisaventure.wordpres.....he-biases/ and http://lifeisaventure.wordpres.....fic-stats/

  37. Rajeev

    November 25th, 2007 at 10:10 pm

    Its important to be in business than getting business affectively right.

    http://tekno-world.blogspot.com

  38. 42mb.com

    November 25th, 2007 at 10:15 pm

    I don’t think alexa data is accurate, if Comscore is using similar techniques obviously its stats is going to be inaccurate too.

  39. Raj

    November 25th, 2007 at 10:18 pm

    Very clever mike:) y dont u and duncan write a book..”Hundred ways and more to highlight Facebook”. No publicity is a bad publicity.Especially for a web 2.0 company.

  40. VoipHype

    November 25th, 2007 at 10:25 pm

    Yeah, Alexa is mixed up. I dunno why so many people rely on it. And TextAdsLinks are using it to calculate the value of a link — they should consider using something else ;)

  41. Johnny

    November 25th, 2007 at 10:45 pm

    Alexa is the biggest load of crap since since George bush said Iraq had wmd’s

  42. Johnny

    November 25th, 2007 at 10:46 pm

    I have to agree with RAJ Michael and Duncan have a little soft spot for Facebook.

    But then again so do I

  43. edenstrom.wordpress.com

    November 26th, 2007 at 12:09 am

    Alexa is like dessert to lunch. Nice, but not necessary.

  44. Ed

    November 26th, 2007 at 4:07 am

    I’m sorry, compete is better than alexa? Whaaa?

    I just tried to compare 2 sites of mine that get exactly the same traffic (about a million uniques a month each). Compete told me one site gets 5 times more traffic than the other.

    It is absolutely no better than alexa, not even close.

  45. Misery

    November 26th, 2007 at 5:41 am

    Personally I like Hitwise’s model - they have proxies which sit at ISPs and watch traffic flows. It doesn’t get much better than that unless you’re Narus or the NSA ;-) (I don’t work for any of them).

  46. Chris Sack

    November 26th, 2007 at 8:50 am

    Comscore is not the holy grail of user statistics.. I myself prefer google trends which does indeed show facebook is very close to catching myspace..

    http://www.google.com/trends?q.....amp;sort=0

  47. Matt

    November 26th, 2007 at 9:23 am

    wow… i can’t believe i agree with Chris Sack!!! :-D

    j/k chris. it’s not the first time.

  48. Matt

    November 26th, 2007 at 9:25 am

    sorry for the double post:

    although…. http://www.google.com/trends?q.....amp;sort=0 hehehe. ;)

  49. Searchquant

    November 26th, 2007 at 9:27 am

    OK, so the general consensus here is that *none* of the top measurement firms (Comscore, Nielsen/NetRatings, Compete.com, Quantcast) produce data that can be considered consistently accurate. Do you mean to tell me that Wall St has been making huge investment decisions on companies like Google, Yahoo, EBay and others based on completely useless data?

    Comscore, Compete.com and Nielsen regularly put out search engine marketshare data, and those announcements are picked up by Wall St and send search engine stocks moving. How is it possible that they could be wrong when looking at such huge sites?

  50. Andy Hudson

    November 26th, 2007 at 10:09 am

    How can the myspace usage be flat for so long? To me it implies no data for that period

  51. Richard

    November 26th, 2007 at 10:15 am

    No-one outside North-America uses myspace…

  52. Freelance Guru

    November 26th, 2007 at 10:16 am

    Just my opinion, but I think that to provide reliable stats using Alexa’s model we’d need someone big like Microsoft to start publishing traffic stats based on IE (sorry to all I’m not a huge fan myself but would be a much more reliable way of doing it) or at least Google (through its Toolbar).

    I really don’t see how Alexa can have information that is anywhere near the real deal - I don’t know a single person that has the Alexa toolbar installed (especially in the UK/Europe).

    Agree with the comment that Alexa information is becoming more and more irrelevant over time, using data from ISPs would be another interesting way of doing it!

  53. Tom

    November 26th, 2007 at 10:25 am

    The graph does not represent page views, it shows the site’s Alexa rating. Myspace was constantly ranked at #6 (or whatever it is) for the duration of the flat line.

    “Daily Traffic *Rank* Trend”
    Doesn’t anyone read graph titles anymore??

  54. TJ Mahony

    November 26th, 2007 at 10:32 am

    To be clear, Compete does not rely on Toolbars like Alexa. As a former employee of Compete, I can tell you that a very small percentage of Compete’s panel is acquired through its toolbar. Compete invests heavily in ISP partnerships (similar to Hitwise) and application partnerships (like comScore) to derive it’s 2M member panel. Compete is the only service out their that blends multiple data sources, so its erroneous to bucket its methodology with Alexa.

  55. Niko Bellic

    November 26th, 2007 at 10:44 am

    Actually Quantcast and Compete are garbage as well, they’re way off even on bigger sites. They’re really no better or worse then Alexa. Alexa makes bad generalizations based on that stupid toolbar, but Quantcast and Compete just wing the number based off of an even more arbitrary guess (mix of backlinks in search engines, and just sheer guessing).

  56. Rob L.

    November 26th, 2007 at 10:46 am

    Note that even more acknowledged (and expensive) sources like comScore and NetRatings have their issues too. We pulled together data from Facebook themselves on their targetable users and compared it to comScore’s figures and found some jarring discrepancies, especially in light of the hou-ha about the oldening profile of MySpace and Facebook widely reported a few months ago. It looks like those assertions on their audiences might be divorced from reality:

    http://cpmadvisors.com/2007/co.....k-reality/

  57. Justa Poster

    November 26th, 2007 at 11:01 am

    As someone who dosn’t really follow traffic trends, but finds the article interesting, do you think you could explain why it is that you claim that Alexa is wrong and Comscore is right? Atleast for me, it would not be surprising at all if more people visited Facebook than myspace, and Youtube then Google.

  58. Mark

    November 26th, 2007 at 11:08 am

    I actually don’t find it that hard to believe that Facebook passed MySpace for a couple days before Thanksgiving.

    Michael, you say MySpace is at “109 million/month, whereas Facebook is at 86 million.” Assuming that tracks globals (not just US), thats what, 20% off? Not much… and is that for October? Facebook is rapidly growing, so if we assume Compete is more accurate than Alexa, Facebook is at most 20% below MySpace.

    As a Facebook developer with 2 top 100 apps, I can tell you that Facebook’s traffic peaks on Thursdays and that traffic is a good 20-30% more than on Saturdays.

    Why does traffic peak on Thursday? I’m guessing people are making plans for the weekend and mentally “checking out” for the week. So its not hard to believe that with American Thanksgiving, the only four-day weekend and the biggest travel time of the year, traffic peaked more than ever on Wednesday… and beat out MySpace.

    I agree Alexa isn’t very good for the little sites… but I find it hard to believe it could be very far off here.

  59. David Payne

    November 26th, 2007 at 12:01 pm

    When, oh when, will Google launch an Alexa equivilent? Their toolbar has a much wider install base. I’ve been begging for this from Google for over a year now.

    Do you know anyone that has the Alexa Toolbar installed? Maybe if it gave you a discount at Amazon.com people would install it. Maybe if they promoted the Alexa Toolbar on Amazon it would get some attention. But, right now, the only people that have heard of Alexa are people that care about online marketing.

    Alexa thinks Facebook beat MySpace because people that care about online marketing most likely have a Facebook account (i.e. it’s more professional than MySpace) versus a MySpace account.

    *ending rant*

  60. Devrim Demirel

    November 26th, 2007 at 12:04 pm

    Facebook has great momentum, with great unique traffic worldwide.

    I have also commented your Google/Youtube post, Michael. I think you could not be able to read that, it was one of the last comments.

    I think Alexa results are likely to be more correct than competitors in both situations. Facebook is gaining great popularity in international arena, as Youtube does. I have already stated why i believe Alexa was correct in my comment about Alexa stats on Google/Youtube.

    Alexa is very good in estimates for the first 1.000 web sites, especially when you are comparing traffic relatively.

  61. Marek

    November 26th, 2007 at 1:44 pm

    Michael,

    1. What data do you have to counter Alexa’s claims that YouTube passed Google[.com] in terms of page views internationally? You keep repeating Alexa is wrong in that matter without providing any sensible reason.

    2. Alexa blog has a good post explaining issues you ignored or weren’t aware of when bashing Alexa previously: http://awis.blogspot.com/2007/.....utube.html

    3. In this post you’re confusing audience (visitors) with traffic rank (visitors + page views) which is a lame mistake.

    4. In Poland, there’s company (Gemius) which monitors about 50-60% of all page views on Polish web sites with site-centric scripts. They have exact stats for most large and midsize sites and surprisingly their site-centric stats prove Alexa is very accurate even for hundreds times smaller Polish sites than YouTube and Facebook. Alexa looks wrong only if mix up apples with oranges.

  62. Statsandwich

    November 26th, 2007 at 5:41 pm

    @44

    Narus uses technology from visualsciences.com to analyze the data. collected in concert with webserver receiving|processing requests(for http(s) data). it can also be used on telecom switches.

    the big kids on the block(visual sciences, omniture, webtrends, coremetrics, etc.) track far more accurate than alexa, comScore or any other sampling garbage out there. people just aren’t ready to pay to hear the truth, they’d rather get inflated numbers for free.

  63. Ventrilo Servers

    November 26th, 2007 at 11:11 pm

    I still disagree that Alexa is at all useful in terms of judging how popular a site is — it’s all relative considering tracking is done through those who use the toolbar.

  64. Dario Salvelli

    November 26th, 2007 at 11:52 pm

    I don’t know: we continue to consider Alex an affidable service. They use a toolbar..

  65. methews

    November 27th, 2007 at 5:31 pm

    not many using alexa toolbar in china

  66. methews

    November 27th, 2007 at 5:38 pm

    come here please
    http://www.zjol.com.cn

  67. Christian Rodriguez

    December 10th, 2007 at 8:06 am

    The truth is that none of these sites are accurate. But if I were to pick a winner it would be Google Analytics. Its too bad they can’t release the real trafffic figures…….

  68. Squeaky

    December 24th, 2007 at 7:50 pm

    I am not sure why so many people have brought into the whole Alexa stats to be the real deal about web traffic statics.
    From the directory business ends of things, I believe that of 70% of people who submit to web directories, still base it on Alexa stats and Google PageRank. Most of us know that it is very easy to fake Alexa traffic, just by purchasing traffic as you see on most forums.
    For the most part, I have noticed over a 20% decrease in Alexa traffic in one of my directories and most of this is caused by that there is not Alexa toolbar for Windows Vista Users. While many business sites online attract visitors that are business based and there OS is Vista, it hurts many sites as far as what our true web traffic stats are.

    So, when I think friend or foe. I vote foe for, Alexa…

« Previous post
Next post »