Brand loyalty is a powerful thing, especially when it comes to technology. Consider the battle brewing now between Google and Microsoft’s new search engine, Bing. Even if Bing proves to be just as good as Google, it might not matter because of the strength of Google’s brand. An independent usability and consumer preference study, which we’ve obtained and embedded below, suggests as much. It was conducted by the Catalyst Group, a usability research and design firm located in New York City.
The study was an intense focus group in which 12 subjects were monitored with eye-tracking cameras as they conducted searches. Afterward, they were interviewed and completed a survey. Prior to the test, all the subjects used Google as their main search engine. Following the test, 4 out of the 12, or one third, said that overall they preferred Bing. The other 8 said that they preferred Google because they were already familiar with it, used other Google products, or that Bing’s improvements are simply not enough to make them switch.
What is amazing is that when the test subjects were asked to rate Bing on specific criteria (visual design, organization of features, filtering options, and relevance of results), Bing handily beat Google in everything but result relevance. Arguably, that is the most important criteria, but most of the study participants thought that both search engines tied on result relevance. So even though Bing ranked better on design, and tied on relevance, that was not enough for most of them to switch.
Catalyst CEO Nick Gould concludes that Microsoft “created something as good as Google and that is not good enough.” Overall, the test subjects “were not swayed.” No wonder Microsoft is spending up to $100 million on Bing marketing.
Remember, this is only a dozen people so it is not a statistically valid sample. Consider the survey results anecdotal. What is more conclusive are the eye-tracking results.
About half of the participants found and used the Explore Pane on the left side of Bing’s home page and results pages to aid in refinement and navigation, while all of them ignored the navigation/refinement links along the top left of Google (Web, video, Images, Maps, News).
Also because the Explore Pane on Bing takes up the left hand column and then stops, creating white space underneath, people naturally stop there. Heat maps generated by the eye-tracking data showed that people scroll much farther down Google’s search results pages, perhaps because there is no visual cue telling them to stop or they were not finding what they were looking for. It is not clear they got better results with Bing, but if the result they wanted was not above the fold, then they might use the Explore pane to refine their search.
The way results were displayed also had an effect on how long people looked at the ads along the top. The amount of time spent on the ads varied by search, depending on what kind of additional navigation information was presented just below the sponsored results. For instance, a search on Bing for “digital camera” concentrates attention there with navigational links to filter results by top brands, prices and guides. Participants spent 150 percent more time looking at the ads just above that activity zone than on Google.
A search for a local hotel, however, produces similar results on both search engines in that area just below the top ads (a map with local listings). So there was not much difference between the two in the how much time was spent looking at the top ads related to that search. In all cases, the ads on the right were only noticed about half the time.
I now Bing.
Can users embed bing?
Hrm, I have not yet been made into a binger. Maybe they can change me.
switched already. bing is fun.
I set my default search in Chrome to use Bing
I have the same set up, I love chrome.
Switch to bing
Why doesn’t Google exclude all Bing results
Its great to see a potentially viable competitor to Google. MSFT has always been a bit hated in the valley, but we love competition and innovation. Bing’s also a really good name. Hopefully their differentiated results display will continue to provide better searches in many cases.
I like Bing too..but that could be a problem, a search engine should try to get rid of you ASAP…that’s how they get paid.
I bing too and waiting for equally good bMail and bTalk
http://www.bing...michael+jackson
Google’s results are much better… it realizes that news results are probably of more interest than search results, and puts them up front. Bing gives you Michael Jackson images as its first result (yuck) and no news results until the very end of the page.
And Google’s related searches also kick Bing’s ass for this query:
Related searches: michael jackson cardiac arrest michael jackson heart attack
This my friends is why Bing will never surpass Google. Beyond the glitzy design, its results just aren’t very good.
i just ran the same querry on bing and google and the results are virtually identical:
3 links to news stories about his death, a link to his wikipedia page and a link to his personal home page.
both have at least one image on their search page and both have a whole bevy of ‘related search’ suggestions unrelated to his death.
i kinda have to wonder just wtf you’re talking about.
http://blindsearch.fejus.com/
I found bing constantly better than google in that test. There are some issues (it hides related search), but after two dozen searches, I’ve never voted for google..
http://searchen...ogle-says-21588
Google also has its drawbacks.
Why do you say “never”? If Bing’s related searches aren’t as good as Google, then Microsoft just has to through $$$ at “related search technology” until its as good. Or better.
That’s what Microsoft does.
Mark I think the big weakness of bing is hotmail.. the mistake all these companies keep making is thinking you can’t beat google when you attack one front… but until someone comes up with the big 3.. blog platform which is better than blog (buy posterous), email that is better than gmail..hotmail brand is old.. and ugly.. and not even close to gmail..I like the look of bing better and I should say they need to make ie way way better it sucks
+1
Yahoo does a much better job than both bing and Google on this one:
http://search.y...michael+jackson
The searcher can be in ‘ a quick lookup’ mode or ‘research’ mode. In the former case, any one modern search engine suffices.
For the later, you want to grab as much information possible about a product or a medical condition. It is a good idea to use as many search engines possible starting from AAfter Search [of course, I am biased]
Actually the first thing on that results page is a link that says “News about Michael Jackson” which you have to click but then you just get news stories. I think it works pretty well.
+1
but I think MSFT should make that more pronounced so the user can see that it needs to be clicked, because a user won’t to click the links slike that (including the category links)
I just sent them a feedback note about this. that they should make it so that the user will know to click it
It’s a freakin’ link. It’s blue & underlined and says “News about Michael Jackson”. If the user has been on the internet more than 5 seconds he knows that clicking on it will give him more news results.
Also of note: it’s actually a little larger in Bing than the otherwise identical “News results for Michael Jackson” link that Google puts at the top of its page of search results.
Guess it’s what should be expected. Even being as good as Google isn’t enough when the brand is so entrenched people use it as a verb.
I’m actually fond of the Bing commercials, which is good because they play almost constantly no matter what channel I’m watching.
Nonsense. Bing is a far better product than google and people will switch when they realize it. I switched to bing when its result started saving me so much time. Google forces you to rummage through piles of irrelavent results to find what you want, not with bing.
I launch bing by searching for bing on google, then clicking over to their site. Eventually google will just be a page that auto forwards over to bing.
Forget about this stupid ‘brand’ thing. Like altogether. Some people are ‘readers’ – they read text, some people are ‘viewers’ they WANT pics.
Actually it’s based on physiology – some people use left part of their brain more, some – the right.
Brands… ROFL.
You can’t possibly be serious, laughing about brands.
Clearly, you are not a marketing student. (Disclosure: Neither am I.) :P
Your comment makes no sense.
Subconsciously, branding influences many things we do on a daily basis.
Google has been THE #1 stop for search engine stuff for years now.
Bing is just not very good — results wise or branding wise (I personally hate it. Live Search was a much stronger brand and at least fit into most of MS’s other web offerings).
I’ve switched to Bing both at work and home. I even went as far as getting an extension for Safari to allow me to use Bing, rather than the two options of Yahoo! and Google.
Does anyone know of a way to make Bing the default on IPhone Safari??
I was searching for a way to make bing my defualt search in opera mobile too….but so far I haven’t found a way to
All Google has to do is have an option for a start-up picture of the day, like Bing. That would instantly give it much of the appeal that Bing has.
People love to discover new things. Take a page out of the MS rulebook and copy the front page. Leave the bare google for the geeks in the house, but seriously, implement that one feature … I say bing it! I mean bring it.
if they do that the hardcores and some regulars might see them as a sell out and they would lose some branding power.
however, I think MSFT hit the nail in the right end this time with their search front page. when you look at bing’s and google’s front page side by side it makes google page look flat with no apeal.
by the way, have you seen the Pic of the day today? I wish I could have that pic for my background……it looks really nice….they should have an option to download this pics for your background. I would build a little utility to auto download it and auto change my background when the new image is available.
You can download the image…just right-click’ anywhere on the page and select ’save background as’
There you are !!!
How many times since google was borm have you heard what google should copy? I think it’s just the beginning. Let’s wait for bing mail and bing talk and it will be all B instead of G
Give Bing a bit of time, not everyone knows about it and it takes some time to get people to switch/change their habits. Declaring it not enough when Bing just launched is sorta putting it to bed early, imo.
No, give the MSFT hegemony enough time to make it the default search engine on their largest share web browser and suddenly it’s Bing FTW.
Way to go Bing, Bing, Bing!
When I am searching Google I know I am searching the largest amount of information on the planet. Bing honestly doesn’t even come close to having the same amount of content indexed as Google. I understand joe blow may prefer bing’s niceties but I am capable of using the right search terms to get the information I want on Google, and as such I prefer to stick the more comprehensive source of information.
i think you underestimate how easy it is to robo-index the web these days. i’ve been using bing and google side by side for 2 weeks now and more often than not, the results are so similar that differences amount to ‘oh, look, the wikipedia page is 4th on google and 3rd on bing.’
For regular/common terms yes. But for instance, when I was looking for a review on a doctor I was about to visit (mind you he should have a greater web presence anyway lol), Bing had nothing, Google had three pages with his name, one of those pages had various reviews of him.
When I am looking for information for various essay’s I write at University I use Google from time to time, it often links me to Google Books with information that barely anyone would want, but that is relevant to what I want.
My own websites (and they aren’t spam sites) index much quicker on Google, then either Yahoo or Bing as well, as do updates to those sites, etc, etc. I honestly think Bing is decent for common terms, but it is really really far off Google in terms of being a comprehensive search engine.
I agree Steven – Google’s large collection of cached links and old pages is what keeps me going back. I’m not always looking for the most recent search results — those are easy to find on ANY engine. Usually I’m looking for specific pages or info that is maybe 2-5 years old or older.
people dont know of bing unless your a tech head. I mentioned today to a british 24yr old who lives in the capital the words ‘bing’ , they had never heard of bing.
The third reason “already using other Google apps” probably applies to me too. However, aren’t people using Google Apps because they are solid apps? I mean Gmail, Google Calendar, and Google Docs & Spreadsheets, and Google Contacts and so on are very good in terms of data portability. I use Google Mail because I know I am not chained to it. (MS took a good first step by introducing POP access to Hotmail users. We’ll see.)
Try asking some other companies to open up like that …
This is like allowing google to enter you bed room….be careful K
I like Bing’s interface but I’m so used to using Google that I can’t switch. I guess I’m addicted to Google.
You are not addicted to Google, you are ‘conditioned’ to using Google.
Former search leaders included Alta Vista, Yahoo, Excite, and of course now Google.
It is ultra easy to switch search tools, if it makes sense.
You should try new offerings, including Bing, and then simply switch back if not doing what you want.
It really is very easy.
Thumbs Up from me.
I think it’s just a matter of time before most start binging
Most people are creatures of habit. Until Bing offers a true “killer app” that compels people to switch, basic customary behavior and laziness will prevail.
12 people is a “study”? Is this a joke?
Exactly what I was thinking. Their power has to be way off!
There are different ways to design studies. This study was designed as a controlled experiment. It was not a large opinion poll. That is why I say the survey results are anecdotal and suggestive, but interesting nonetheless.
However, the participants were not just asked their opinion on the phone. They were monitored as they did searches and then asked about their experience afterward. That results in a more informed set of responses, but I agree that the sample size could have been bigger.
This should be an article about the “usability research and design firm located in New York City”, not about the “study” they “performed”.
Techcrunch is turning into valleywag.
Next on TechCrunch: Researchers from crunchanon, a usability research and design firm located in Detroit, release a study suggesting daily visits to crunchbase.com can improve relationships, personal hygiene, and probably get you some kickass Xanax from an unknown relative in Nigeria.
Bing has a good thing going because they focused on trying to give the user a better search experience. Google has always been about simply listing relevant results, and they haven’t changed, or needed to – until now. All google has to do is repackage their results with a better interface/search experience (taking what works about bing), and that will stop Bing’s momentum.
i´ve been “binging” since late may (25th i think) and i have not use google since then, not even once, i´m happy with bing´s results and now google is not evenan option as a default browser, so i hope more people switch to bing because competition is good for the consumer and goog and msft will improve their products.
P.S./ i still use goog finance when im on the road because of real time quotes without refreshing the window but if bings improves msn money (think about Bing money or Bing finance) i´ll sure as hell give it a try
bing will never ever ever win search. Google owns the idea in the mind it doesn’t matter how good bing or any other search gets….or how many dipshits they get to say they switched over… if you think great product win’s you just don’t get it it isn’t first to the market, or best product its first in the mind.
perception trumps reality every single time
Still, if Google’s product quality continues to fall, people will be looking at the competitors. A couple years ago I was pressing the “I feel lucky” button, now days you need to look at a few links to find what you’re looking for.
Why on earth were you pressing “I Feel Lucky”? Are you a grandma computer user?
Google’s only flaw at this point is that it’s making many people uncomfortable about privacy.
The “I feel lucky” button was all it took,
first result was always spot on.
Now days it isn’t, and its your attitude that’s exactly what brings down great brands!
Google simply makes a better product and Microsoft has a large trust gap to fill. It is true that pretty much all of Google’s design work is horrid at best. I mean take a look at the support pages. There web design is stuck in the netscape era but it doesn’t matter their products work great.
I agree to few comments that brand loyalty is also very critical and so is comfort factor. Bing might pick up more with young demographics… but only time will say.
I still find myself taking longer on Bing. That is only downside I see so far.
You’re comparing the Bing brand which is less than 30 days old to the Google brand. It makes sense Google’s brand will win at this point. If Bing, as a product, is getting better ratings among users then it’s reputation will catch up. Let’s see where things stand 6 months from now.
Weird results, limited test subjects. What’s with the staring at the white area below google’s search field?
Interesting. Perhaps in this battle it would be more worthwhile for Microsoft to invest in improving their brand image. They still appear an old dinosaur that’s slow to change and does not represent innovation to a lot of people I know.
Also, when Bing first came out I was confused as to who was behind it. My first thought was that it was just another cheapo no-name search engine that comes up when you accidentally go to a site that doesn’t exist.
It’s actually pretty smart for them to invest in Bing as it’s own brand. think about the Live/MSN Search confusion and it’s pretty easy to understand why they wouldn’t want to be branded as Microsoft search or Bing by Microsoft, or anything that gets murky. Judging from the comments here, there are enough people who have a negative brand association for Microsoft anyway, so no need to make that the first thing people hear or they’d probably be even less likely to try Bing than if they thought it was a no-name search engine
Is the sample size not ridiculously small? There is no way in hell any of the conclusions can be statistically significant. Add to that possible biases in the selection of the 12 users and you probably have a totally worthless study.
I am surprised that this is deemed worthy of a techcrunch post.
No one is saying that the test is statistically significant. Qualitative studies (as opposed to quantitative) are not designed to be statistically significant. The entire purpose of doing an eye tracking study (or any study where you talk to a subject in a one-on-one setting for a longer amount of time) is to gain INSIGHT into their answers. It’s not about “what”, it’s about “why.” I don’t think the results are being presented as actionable facts, just interesting insight.
Bing is more powerful than Google.Google want to stay on top in SE market.So it will follow What the people want in a Search Engine…..
Thanks to everyone who commented regarding our study. I’m glad the results sparked some interest. Just a quick note to emphasize that the study was not intended to be quantitative / statistically significant. This was a qualitative usability study meant to explore users’ initial reactions to Bing versus the more familiar Google interface. Given this context, the sample size is, in our experience, adequate.
Hey Nick,
For the sake of trying to get over my dislike of qualitative studies, can your clarify “explore”? The article seems to denote definite results (Although this may just be the fMRI effect due to the picture-see “Brain Images Make Cognitive Research More Believable”). Can you link an article, journal entry, or something that uses more modern mathematical models and not Nielsen’s curves in usability studies? I have a hard time getting over my notion that your results may only being indicative of your sample group and may not be finding true usability flaws or occurrences. Any literature that you use in your field I would love to look over to better understand. Thanks!
Leon
I know much better than to try to get anyone over their “dislike of qualitative studies”! You either find these studies useful or you don’t. Fortunately, many of our clients do. If you read the study carefully I think you’ll find that we are not claiming that these results extend to the entire population – or even that we have uncovered significant flaws in either site. Google and Bing have both been exhaustively tested, so I would not presume that we could find new problems that Google and MSFT aren’t fully aware of. Our intent was to understand *qualitatively* how people viewed the two experiences comparatively. We thought the results were directionally interesting, but certainly not the end of the story.
Hope this helps.
NG
And one more thing… many of the comments on this piece are echoing the sentiments of our study participants… Coincidence? :-)
A lot of people are saying that Bing is a quick learner, and the results are improving themselves on a daily basis.
Just wait until Microsoft pays Mozilla to use Bing instead of Google for the default homepage. And Dell and HP.
I asked someone who is decent with computers the other day (he knows how to use BitTorrent, WinRar, how to transcode from one video format to another, etc) what web browser he used…
“I use Yahoo and sometimes MSN”
He was using Firefox and the homepage was set to Yahoo and on IE it was msn. People who read TC care about Google vs Bing. The general population use what they are given.
So true Paul. The average person just types their search into whatever box pops up on their screen first and the average person is 99.9% of the market.
99.9%? That’s a bit extreme…
What some people fail to see is that inertia works in both directions. If Bing is “good enough”, then people who start off with Bing as their default browser will have little incentive to switch to Google or Yahoo.
MS does not have to win over current Google users; it just has to hold onto those who start off with Bing through the purchase of new Windows computers. MS does not take over markets via instant knockout, but through gradual, insidious attrition.
BTW, I have been using Bing since its release and have not missed Google at all.
Use it, before you curse it. Bing!
I like Bing and have actually switched whatever I can to default to Bing instead of Google. I have only had a few issues with Bing and when that happens I use Google as a backup. – I think Google still has an advantage over Bing in the News search with breaking news results.
This is the first time in the last few years that we are seeing people saying that oh there is a new search engine that is almost as good as Google.
That itself is phenomenal. Google has built a huge brand equity over the years. Microsoft has taken a giant leap in that direction too and with regular updates, intelligent planning, there is no reason why it can’t make a serious dent in Google’s Market Share.
The game is on
BING …!!!
A few things to try:
An AMBIGUOUS Web search: “turkey” (do you want images, recipes, facts, or a map of the country?
The topic guides in the left explore pane will help you narrow your search).
A TRAVEL search: “SFO to JFK”
VIDEO search: “Simpsons” (hover over the thumbnail to play the video)
IMAGE search: “Rollercoasters” (notice the infinite scroll).
A HEALTH search: “Sore throat”
SHOPPING: “Digital SLR” (sort by price or brand, get average ratings and CashBack).
MAPS: “BBQ” (automatically knows what city you are in and offers up geo-appropriate results).
BING BING BING…!
I hope BING gives a good competition to google. Monopoly are never a nice thing :>
I think the comparison is not quite on the same platform. Google has evolved over a period of time and faced some immense internal issues, what we see today is the final picture of that evolution. Microsoft although tried a several times to match Google they have not been allowed to settle down.
Bing surely is a good thought out product and still has it’s drawbacks but then it’s the closest we could ever think of anyone coming to google. It will improve over period of time due to direct comparisons with google. I personally beleive this product can go a long way with a few improvements. The only thing which I beleive is biasing the surveys and researches is that for almost a decade we have thought “Google= Best Search solution” and it has embedded so deep into us that we not willing to try anything else.
Good or bad is for us to decide but then atleast see what we labelling at.
Hope that adds some value to this discussion :)
Sonal
USourceIT: Riskfree IT outsourcing/ sourcing partner for small and medium businesses.
when a doubt, i always google it, no matter whatever options i have..
“Remember, this is only a dozen people so it is not a statistically valid sample. Consider the survey results anecdotal. What is more conclusive are the eye-tracking results.”
Try putting that higher up in the story before making bold headlines.
Believe me before I was using only Google. Never thought of any alternative. But now I wonder how I switched to ‘Bing’. I guess ‘Bing’ is much smarter than Google. That’s the point here.
Bling will have to out perform google to beat, there is no point being as good as google. It has to bet better to beat google.
Been Binging all month and now main search (decision) tool.
Agree, that mass public will be much slower to try Bing due to ‘indoctrination’ with no. uno.
That being said, if Bing remains as good as it is I predict that in coming months the balance of search will begin to shift.
*** Sheriff Bing-Bing-Bing
Does anyone have a manual for bing…if you do….please share!!!
I think before Google Micrososft was there only not as Bing and they allowed Google come in and have become so significant for so long and Microsoft wants to tell us that ‘they are taking over from Google’. Were they not around when Google was started? What have they done up till now? This would be a failed attempt. All Google needs is repackage to include a fair bit graphics and Bing would be no challenger. Mark my words…..
If I could just add a gmail link to the top of my ‘Bing’ page…
Yes I’m too lazy to type.
I’m one of those addicted to Google. I did try Bing but wasn’t exactly floored by it. It is good to have competition and it’s amazing that Bing has managed to attract so much attention and users in so little time. But then again, Google is not going to sit back and watch MS threaten its reign. I’m sure they will make their search engine even better.
Sample size was probably more than sufficient for the study type (note: i’ve not yet read it) depending on the variables being measured. Generally, a usability study is looking for potential issues or problems or is a comparison (like in this case) of user performance. Sample size in this case is only a “problem” if they are attempting to generalize the results to the entire population.
btw – a focus group is one of many types of usability studies… a focus group consists of multiple people in a room and a moderator, a “standard” usability study (which this probably was) consists of a single person in a room and potentially the test administrator though he or she can be in an adjacent room monitoring via behind a mirror.
It is nice to see people like Microsoft again. All these years all we have heard is one joke after another about how much they suck.
I would like to suggest that the very idea of eye tracking research in this particular case is flawed, since they restricted their research pool to people who were already familiar with Google.
I believe eye movement patterns on the Google search results would be quite different between a consistent Google searcher, and people new to Google.
Regular Google users would already be so familiar with the layout, that I would think it is entirely normal that eye tracking would avoid the top of the page, and mostly stay away from sponsored results.
It would have probably been more relevant to use regular Yahoo! Search users for this study.
Although the fight between Microsoft and Google might be “brewing”, as per this article, the real fight that is being waged right now is between Microsoft and Yahoo. (Before Microsoft can get the title fight, it has to beat the current number one contender.) And I think that Yahoo might be in trouble. More at: http://domusinc...w-thoughts.html
bing cant yet differentiate between x.com and x.com/index.php and has many sites listed twice for the same query
Google has always referred to the fact that the competition was always one click away. I believe Bing is a serious contender. Bing Video in particular is compelling