The following post was written by a well known executive at one of the largest sites on the Internet. The author has requested to remain anonymous – not for dramatic effect, but because of the backlash he would receive from the SEO industry and possibly Google itself. He also doesn’t want his company associated with the post.
He is starting a discussion on the need for government regulation of the organic and paid search policies of Google, which maintains a commanding lead in search market share today. Or at least transparency in how search results are determined. There is clearly growing frustration on the constantly changing “border policies” that are created and enforced by Google and other search engines. It is a fascinating read.
Imagine, if you will, that the entire Internet is contained within a single continent. That continent is filled with countries, states and cities. Each jurisdiction is autonomous, relying on visitors to cross on to their turf to engage in commerce. Now, imagine if the only way to get into this continent involved just two methods: SEO and SEM. Let’s further imagine that the borders to this continent were controlled by a single company. Let’s also hold that the rules for search engine optimization listings and search engine marketing were not only defined but were completely controlled at the whim of this single company. Of course, we all realize that word-of-mouth marketing and viral marketing also contribute to traffic to individual websites. That said, the primary methodology for all users to reach any individual website destination is still search, of either paid or organic listings.
Or suppose the paradigm is the streets of Los Angeles. Let’s imagine that in order to enter the city you had to pass through a single gate. And once you entered that gate, the streets you were or were not allowed to go down — and thus the businesses you were or were not allowed to visit — could be randomly blocked from your access. Blocked to a point where you might not even know they exist; whatever streets were available for you to traverse were in essence the only streets you knew where business could be transacted.
Whatever the scenario, it’s unsettlingly close to the situation that prevails today in search. It’s now conventional wisdom that search engine optimization, representing the organic result sets on any search query, is more voodoo than science. Through an uncontrolled set of factors search engines determine which listings appear at the top and bottom of any individual query. In addition, consumer behavior dictates the top three results on any search page are all that matter. If you happen to own an online business, unless you exist within those top three, the amount of individual traffic you will obtain from organic listings is very, very low. As the proprietor of that business you may hire search engine optimization companies to assist in increasing your rankings on organic results, with or without success. And at any one time, the controller of these borders (that is, the search engine itself) can change and manipulate those rules – and that can substantially decrease or destroy all organic traffic coming to your website, without notice and without your knowledge.
Search engine marketing now faces a similar challenge. Although anyone can open an account to buy paid search listings, the rules on each account are arbitrary. Accounts can be shut down at any time, without notice to the website owner. In other words, if you haven’t successfully obtained enough traffic to your site from organic listings and you decide to rely on paid search, you still face a situation where regardless of how much you bid per click you may or may not show up at the top of the paid results. That’s because paid results that are displayed on any query are not only determined based on the price the buyer is willing to pay. Unlike other auctions that are completely priced-based, these results are determined and sequenced not only on price but also on quality of advertising and click-through volume. For example, if company A was willing to pay $1 per click on a certain term and company B was willing to pay 10 cents but company B’s ad generated ten times as many clicks as company A’s, the yield to the search engine would be identical between the two.
The second factor is that the search engine can, at any time, determine that either company A or company B may or may not buy traffic within its index. And without notifying the company and with no path toward recourse and statement, the search engine can disable the paid search account from either business. Returning to the continent metaphor, this ends up looking quite a bit like free trade. Various businesses (call them sellers), operating within this continent, wish to conduct business with the rest of the world (that is, the population of buyers). The border — which in this case is the search engine — thus has complete control of who can transact and how often. And at its discretion, the search engine can decide to increase, decrease or completely disable access between buyers and sellers. Because search is the dominant methodology for consumers to find what they are looking for, whether a product or a service, the unilateral control that search engines wield enables them to control billions upon billions of dollars of consumer spend every year. It also gives them the ability to completely determine which companies become more successful — or less so.
The situation we face today is unique. Due to Google’s dominance — and the fact that it controls such an enormous amount of consumer behavior through paid and organic search listings – the company in essence governs commerce on the web. And any company that falls out of favor with Google, whether for reasons of bad practice or simple disagreement, can find itself at risk of going out of business.
This system also benefits the few in a host of other ways. Because the rules of organic and paid search change frequently – and remain undefined — agencies and other traffic brokers can win big; through their experience, they’re capable of reverse-engineering these rules. This means that, as this market matures, individual businesses have a diminishing chance to actually compete and gain search market share. That, in turn, puts them in a position of not only needing to hire an agency in order to find any traffic, but also making it more expensive overall to build businesses on the web.
I’ve worked with many businesses who feel they are playing in Google’s world — behaviors from product decisions to marketing strategies rely completely on appeasing these undocumented and often mystical Google desires. I’ve seen companies choose to not work with Google’s competitors for fear that by building those relationships, they’re damaging the ability to be indexed properly on Google and are anxious that result sets will be compromised. Many likewise believe that by having a monetization relationship through Google, they will somehow achieve higher quality listings through organic search. I’ve also witnessed companies who, in addition to using Google for monetization, have preferred relationships with purchasing traffic through Google Adwords. By supporting this dual relationship, they appear to want to live by two sets of rules – those that exist within the Adwords marketplace and those that apply to the Adsense product. And because they’re walking on both sides of the (Google) street, they feel they have a strategic advantage — as though the Adwords product will enable them to acquire traffic at both a lower cost and with a looser rule set than their competitors.
Here’s where the parallel to free trade breaks down. There are no perfect paradigms looking at free trade and import/export laws that exactly define or address this challenge. Neither would a secret relationship between the government and the search engines solve the problem. The only real solution is disclosure. Transparency. Those traffic generators that use rule-based algorithms to determine result sets must publicly disclose their methodologies. That is the means by which all businesses can compete freely in the organic and paid search marketplaces. If we lived in a world where Google didn’t hold sway over such a significant portion of consumer behavior, this kind of regulation wouldn’t be necessary. The market would be self-correcting, and we could trust the individual decisions of a healthy and competitive search industry. Regrettably, due to search dominance, the industry can’t be left to its own devices.
There should be optimal combination of SEO & SEM service but still it’s difficult to find a company who can accomplish both task effectively.
That’s a lot of imagining. I guess you’d like to use that SEO and SEM budget for TV ads?
If a company stop providing relevant products, it dies. Remember Alta Vista, Lycos, HotBot and GO.com ?? In 5 years, the completion can change dramatically.
Great!!
in JAPAN
No VC money. No tricks.
Just good content, good customer service and hard work.
You know us, we are the little guys that don’t usually have a voice except when it comes to google natural search. But with that level playing field we turn up number 1 in our vertical.
My advice? Stop moaning and start working.
Dear anonymous: I am very confused: is it *government* regulate “industry” or regulate Google? Also, are you going to answer questions here? I could take this post seriously if I knew who wrote this post. At this time, seems suspect.
I disagree.
More transparency and disclosure of the ranking algos from any of the SE’s will lead to the rise in “black hatters”. I guess it would be more chaos than any help.
this is the most ridiculous statement I’ve ever read: “they’re capable of reverse-engineering these rules”
who exactly is “reverse-engineering” the Google algo? No firm is capable of that. Granted through certain testing techniques (that may use some type of reverse engineering for a small portion of a ranking test), an individual or company might be able to learn their job better, but all experts exist because they know their industry better than most others. Stock brokers, real estate speculators, marketers, etc. Others can learn brokerage or speculation or traditional marketing, but it takes time and effort to be good at it.
Not entirely with this piece. It starts by saying it wants “…government regulation of the organic and paid search policies of Google”. It pretty much lost my vote from there on in.
Which Government would that be? Haven’t the Chinese Govt. been working ahead of the rest of us in this direction? Perhaps it should be them. Or would the US Govt. police it for the rest of the world? Which government could you trust right now and would every future administration be trust-worthy?
Users are the ultimate regulator. The minute you get central or government regulation involved you end up with a bigger problem than you started with.
Regulation of the web is anathema. Leave it be. It’s a free platform and it needs to run like a free market economy living or dying on whether people choose to use it or not. If Google and other traditional search engines abuse their position, people will eventually go elsewhere.
In fact you should start a campaign to promote the use of Twitter for searches. You get up-to-the-minute results and opinion about relevant content from a range of sources some objective some subjective, much of it expert. And its results are not swayed by budget or influence
It addresses all your apparent concerns over search engine power and if enough people did it, imagine the amount of SEO money you could save. Where’s Bono? I’ve just solved 3rd world hunger.
Another frustrated SEO from a big company?
Come on TechCrunch, we have heard this a million times before.
Government regulation of Google results will only serve blue-chip corporations that are banging their heads against the wall because they cannot get their Flash sites to rank.
Reality check please.
This article adds nothing whatsoever, but I would like to add absolutely nothing to it as well.
I want to second this outstanding opinion.
Except that this isn’t new. It has been the fundamental complaint for years. it is the main reason I use Bing now. Not that they are any better, just that I won’t support a monopoly.
The government? Why are we all looking to the government to take over and regulate everything? This seems so over the top to me… So will we have a committee with they guy who thinks it’s a bunch of tubes? Or some old geezer who has ties to the telecomms chairing it? The amount of danger and destruction the gubment men bring to any endeavor is monumental… and you want them to control Search Engine Marketing?
I had to check the calendar to see if we were somehow at April 1 again. This is so outrageous to me that it seems like a joke.
I don’t understand the problem… are users unhappy with the top 3 results on google? No? Then why would it be benificial for them to change? Even if it is more tranparent and regulated, people will still only go to the top 3 sites…
I think Google should disclose their methodologies in certain amount because Google themselves record information and collecting data by forcing webmasters to disclose their businesses..
Interesting post. Google is dominant and yes, they can make or break a business. That sucks, if you dont know how to play the game. Many cant afford it, dont know how and ultimately give up.
But exactly how would this be regulated, what measures is it possible to take in effect?
Dear Anonymous,
Google is not as stupid and fierce as the Cosa Nostra, to evidently put harm to people, who say something against them.
But I completely understand your fear. It is the part of the problem!
Google has grown to a superpower with perfect dominance not only in the ecommerce field, but
it is one of the most admired and powerful companies in the world concerning INNOVATION power! And with a reason – I do admire Google too in this respect! Their strategy, vision and great R&D.
BUT ….
Google is acting like a state, making peace around its borders. So even the admiration can be seen as the part of the strategic game.
(1) We geeks got more free tools that we can eat.
(2) The masses got a reallly nice brand with a lot of fun and free stuff. Be sure googling stimulates the happiness zones in your brain, because 99.92% of the time there is some success experience looking at the results and clicking through.
(3) The business has “full control” over online marketing with adwords –
…… but with Google somehow in the role of the bookmaker and manager sending the companies big and small in the place to fight against one another.
* Auctioning scare top places is ingenious if you own all the places!
* Game theory can show us: Biding in this battle field is a nightmare.
Any Sound Regulation Strategies?
There are some reasons against transparency, as I have seen in the comments. Regulation could be quite easy anyway. Google could be forced to let the results rotate in a systematic or random way. So the top three listed organic items for the top part of searches are just different for 10 searches with exactly the same query e.g.
In effect this means much less revenue for Google of course, but from an economic point of view a good part of Google’s revenue of today is just not legitimate “sound profits”, it is created on the foundation of the ever growing market dominance, actually a Google Monopoly in most conutries of the world.
Actually I see regulation only as the ultima ratio, I like the “free” in “free market” – but If governments will not act soon, the “free market” of information will be lost.
——-
BTW two years ago I have written a much to long comment concerning the issue. I identified Google as the “Worldbank of Reachability” and the troubleof this position. A German blogger then quoted my comment in full length – and hey I am still living healthy.
Google is just busy with the declared Mission: Organizing the information of the world.
Read this in 21st century as: Organizing the world.
The post of 2007 is only in German, sorry: Media-Blog » Blog Archive » Google – eine Welt-Bank der Erreichbarkeit … http://blog.fir...media.de/?p=799
“Those traffic generators that use rule-based algorithms to determine result sets must publicly disclose their methodologies. That is the means by which all businesses can compete freely in the organic and paid search marketplaces.”
Err, ok if its all closed, then nobody has an edge, and if its all open then no one has an edge either. I see no difference between the two except trust in Google’s intentions. Regulation is not the way to trust Google’s intentions, after the fact litigation is the way. If there is something funky going on with the algorithm to give some sites in the know an advantage, that should be apparent whether the algorithm is public or not.
The only result I see from an open algorithm is 1st it forces google to stabilize and slow changes or improvements to the algorithm due to the cost and time incurred with documenting that algorithm properly in the public eye. 2nd it lets people can game the system quicker. Both of those things in the end makes google infinitely less flexible to react to people exploiting the search results to bring so called “organic” search referrals to their site. In the end that “gaming”, or “marketing” as is called here, is no more organic than a paid ad.
Brilliant. How to get people commenting on your blog. Be secretive. Be controversial. Get people guessing. Get people hopping mad.
Why is everyone assuming this was written by someone from Microsoft? Could it not have been written by someone at Google to create a backlash against “someone at Microsoft”. Or maybe by somoene at Techcrunch wanting to create a bit of focus?
Good clean fun, I reckon.
Just because you don’t understand something or cant game doesn’t mean its voodoo and not science.
Back to pinning up some more voodoo dolls.
I really can’t understand the author’s point of view…
Which government ? Yours (US) ? Mine (FR) ? Depends on the research ? On your internet provider ?
However, I may understand the main problem, and I would quote “Tamir Strauss” : What about an open source search algorithm ? Maybe not delivered by Google, but another company / organization.
Even if it would suffer from loooooots of problems, this 4-words-based (Open Source Search Algorithm) issue is much more relevant than all the stuff you wrote.
My initial attraction to google was the fact that I could market within my budget and get some return. I’ve had some successes with google advertisements that were within budget and were simply a result of being nimbler than my large competitors.
My concern as a small business owner is that “transparency” would simply mean that the system would be gamed by the highest bidder every time.
As it is now short term advantages can be gained, but transparency would remove even those.
While the author of this post has made some good points about the dominance of Google and its control over the search market, introducing government control to a very fast paced, ever changing marketplace is not the answer.
When it comes down to it Google does its job by showing the customer what they are looking for. Companies may get upset if they are not ranking well but that’s because one of two reasons, either your site is just not relevant to the search query, or your site is poorly optimized. The companies that tend to complain most about not ranking well in search engines are those 500 pound gorillas who think “if i build it they will come.” Well that’s only true if you build your site to standards.
Regardless of whose controlling the algorithms that dictate the top 10 results, your site will always need to be digestible by crawlers.
So complaining about search engine dominance and unfair market share is one thing, but expecting Google to send more traffic your way just because your big player is not not a valid argument.
This post is totally flawed, the scant parts of logic that seem to support the overall premise are unbelievably weak and in some cases just ignorant.
I love this “I’ve seen companies choose to not work with Google’s competitors for fear that by building those relationships, they’re damaging the ability to be indexed properly on Google and are anxious that result sets will be compromised.”
wtf! as if google have the time or the will to do this, they dont need to.
and..
“Many likewise believe that by having a monetization relationship through Google, they will somehow achieve higher quality listings through organic search.”
‘Many’ may believe this but that doesnt make it so, we do SEoO for companies spending a fortune on PPC but that doesnt make the SEO task any easier, if anything increasingly it gets harder and harder as we are pitted against other high quality SEO agencies.
The whole post relies upon areguments prefixed with ’someone told me’ I once observed someone do something without basis, without measurement and without seeing any results of their action’.
The paralell between the nation states and Google is so flawed it is an insult to the english language.
If this post wasnt so ridicuous I’d spend more time breaking it down but I have a country that urgently needs saving by my SEO wizard warriors ;-(
Let me make a radical suggestion – stop trying to game SEO. I’ve been marketing brands on the internet now for 10 years, quite successfully, and I’ve never once bothered to hire an SEO company. Focus on the content, functionality, the brand and making your business better and the traffic will come.
Sadly, I agree with most here…transparency would make things probably WORSE, not better. It would make it easier to manipulate search results, resulting in an “EBay” like environment where top results could simply be paid for.
The top position of any search term must be a round-robin of the top 10 results i.e. first page as in google, and the next 10 is round robined in the second page, so on so forth upto say top 10 pages. This is the best technological solution which google can implement easily. This gaurantees no one site has monopoly on a search term by SEO, at best SEO techniques can aim for page position 1st page, 2nd page, etc as opposed to 1st, 2nd, 3rd, etc position.
You are a troll. Who wants to be top of the third page of results? No one!
If I want to buy something I don’t Google for it I go to Amazon. You Google for information, you shop at Amazon.
Simple really.
This is absolute trash. Its not that Google is manually picking on sites which also show on Yahoo or Bing. From the way I see it, Google is more than interested in doing business with me if I have good content on my site regardless of my site showing on Yahoo or Bing.
This seems like such a bad idea for several reason:
* With full disclosure, spammers will be able to own the search results making them useless.
* How much lead time will they be required to give before making changes to maintain their full disclosure?
* Opening the door to government regulations means we are opening the door to every (world) government making their own regulations that need to be followed. What a mess!
It boggles my mind that anyone could actually think that bringing in the government to help decide who gets to the top and bottom of paid (or organic) listings could be better than the market as we have it.
http://domusinc.blogspot.com
While it sounds good to implement some regulations. The internet has actually been the brightest spot and biggest success of the last 20 years. All the things that have been heavily regulated by politicians not so much.
This whole article reeks of someone who has some sour grapes because they run one of “the largest sites on the internet” but aren’t given the preferential treatment they believe they deserve. In particular:
“Accounts can be shut down at any time, without notice to the website owner. In other words, if you haven’t successfully obtained enough traffic to your site from organic listings and you decide to rely on paid search, you still face a situation where regardless of how much you bid per click you may or may not show up at the top of the paid results.”
Correct, and why should it be any different? Why should some mega company with much deeper pockets than mine be entitled to completely dominate the advertising space because of their incumbent position? They only reason the author would be affected by the way the price bid auction with google works is because:
- Their ads stink and nobody wants to click them
- People do click them, but return to google almost immediately because it turns out it’s not at all relevant to the original search
So I stay quit your whining, your time would be better spent providing quality content that people are interested in. The other plus of actually being useful, you’re not entirely dependent on google any more as people share the love via social networking services.
Glenn
“Alexander, is that you…..?!”
This will never happen as long as Obama is President. Which company donated more $$$$ to Obama than any other……….you got it…..Google.
We’ll see.
We now have 100 banks who are too big to fail.
Perhaps there are 100 brands who are too big not to rank?
This administration is very good for incumbents. Not so much for the small and scrappy.
Just my 2¢.
Regulation? Perhaps now we need to also bail-out the companies that are failing in the SEM/SEO ranking.
Google’s algorithm? That’s the billion dollar question. As a SEO consultant for over 12 years. I’ve seen it where back in the days when Alta Vista was king. Where supposedly you can have the font color same as the background, keywords stuffing, and alpha-numeric domain plays a part in the ranking. So now Google like Microsoft is a leader in their industry, someone want to cry ‘foul’ or ‘monopoly’ in order to get the government involve. WoW! Nothing new. Just do the research and play the game. If you can’t play nicely, you can always sit out.
Every empire begins as a cartel.
No, the time has not come. Go away please.
Clearly some clever SEO pissed on your bonfire.
Tough titties!
This post is the equivalent of complaining about women because you can’t get a date…ultimately, the problem isn’t them, it’s you and you’ll have to change in order to get a more favorable result.
Thought provoking article, and some equally interesting comments. The bottom line here is that Google is in a monopoly position, and it has been the long standing response of most governments to regulate in monopoly situations. While it is not clear exactly how regulation might help, or what form it might take, I think that regulation will eventually apply to online search. It is too important a service not to attract the attention of government.
I can’t decide whether it’s pathetic that the author wants to remain anonymous rather than standing up to Google, or scary that, despite their policy “Don’t be evil,” a “well known executive” at “one of the largest sites on the internet” is afraid of Google like a store owner of a mob boss…
The capital of Switzerland is Bern.
Just thought I’d add that so that there’d be one sentence on this page that was actually informative.
Yes it can be challenging to rank well in Google for competitive terms and run an effective Adwords campaign. But Google actually does publish a SEO Starter Guide http://bit.ly/LPdMS which provides enough info for anyone to figure out how to rank #1 for almost any Web search.
Deep Throat’s premise seems no different than having the government regulate how to run a successful ad campaign on American Idol.
I see, so the author is saying the government should interfere with the free market and force a private company to reveal trade secrets? Why stop there, lets force Coca Cola to give up its formula, and KFC to!
The person who wrote this comes off as a socialist who should be punched in the nose. We need LESS, not MORE, government regulation. The government doesn’t know **** about the SEO business, so it should not be regulated anything.
This is why America is falling apart, we have government sticking its nose in industries it has no experience, the author of this article is an idiot, it is Google’s algorithm and they have the right to do what they want for it, private ownership rights are guaranteed under the U.S. Constitution though most people are too stupid to read it or care.
I run a small site. I’m not well-versed in SEO. I was lucky to get my domain name in 1996 and somehow survive, which means I’m often in the Top 3 search results. I look to attract traffic from a variety of sources, including search. My best draw recently? Twitter. The author, to me, sounds a bit out-of-touch. I also think the suggested answer (government regulation) is a cowardly one (as is the anonymous posting). Can’t we be more creative?
When one player dominates—or appears to dominate—a particular field, through hard work, innovation, corruption, or otherwise, the knives will inevitably come out, somewhere, from someone. It’s human nature to resent dominance and the imposition of will by others at one’s expense or perceived expense.
@igniguy and @joshuabaer are essentially correct. If the market delivers a better product—one good enough to change user behavior—then time will deliver that real change in behavior. Regulation is not a solution. Full stop.
As a marketer, I find that SEO/SEM is a necessary evil. Sort of like buying ads or attending industry events put on by media so that they at least know about you enough to cover you. You can claim some level of separation between pure organic search and paid search, but if you’re looking for something, do you care which you click on?
I think what we really need is a better answer to search on the web: a mix between what is intuitive and what is desirable. Maybe it’s not search at all. Maybe it’s the way we query for what we want.
If I say I want a search without ads or I want a search with only ads or a mix, or if I want to really search deep (I need some nuanced bit of data) or just need basic information (like if there’s a wiki entry somewhere), these are factors that would make SEO worthwhile.
Right now everyone is just gaming the system as best as they can to keep up knowing that the road to the internet starts with Google. There is no neutral playing field, and until there is a will to change, there never will be.
It’s not a fascinating read. It’s silly. Already can’t believe the time spent on it.
I agree that too much power is gathered up in Google, however I disagree that we should consider “regulating” them. I really don’t think the federal government would mix well in the ever changing and adaptive business environment that is the Internet.
I believe that major players that spend tremendous amounts of money with Google should form a Group of Concerned Advertisters. But many of the top dogs in e-commerce would need to participate. Imagine having the collective power of turning off 20% of Google’s search revenue. Even 5% would be huge. We, as an industry, need to start applying collective pressure.
Whatever drugs that the “…well known executive at one of the largest sites on the Internet..” is taking should be shared because it seems to make the complications of government intervention seem like a welcoming gesture. Someone once said; “…the government can’t solve your problems, the government is your problem” and now, this unnamed source wants to bring them into yet another free market to destroy it? Get serious. Free markets and activities are the best source of regulation; self regulation. It’s at the will of those who use it and not the politicians who spend their time trying to effect the changes that THEY want and not what people want. Everyone is free to use whatever search engine that they want, or to not use one to find what they need. Let’s keep this as a free will area and stop trying to control all companies under the guise of “regulation”.
Interesting post but, for me at least, it loses all credibility being anonymous.
i honestly don’t think getting the government involved will help… They will just end up trying to take over google