I’ve mostly been a spectator in this whole Rupert Murdoch de-indexing his news sites from Google circus. First because I didn’t really believe he even knew what he was talking about (or how much traffic he’d lose), and more recently because Erick Schonfeld took the story here at TechCrunch.
But suddenly this is a fascinating story to me for a bunch of reasons. This may be less about the self destruction of traditional journalism and more about the search wars.
Mahalo CEO Jason Calacanis, who used to work for Murdoch’s Digital Chief Jonathan Miller when the two were at AOL, posted a video last week (embedded below) with a simple suggestion: Not only should Murdoch de-index from Google, but he should get Bing to pay him for the exclusive right to index it. TechCrunch Europe’s Mike Butcher has been sniffing down a similar trail.
If other media companies joined Murdoch Google could actually find itself in a very difficult position, where Bing had content that Google didn’t. If you knew that Wall Street Journal and, say, New York TImes content was only in Bing search results, mainstream search users would suddenly have a big reason to go to Bing.
This would shift the balance of power away from search engines and to the content sites – if they could pull it off. Bidding wars over rights to index content would conceivably break out between Google and Microsoft, just as bidding wars have broken out in the past over the right to serve search ads into third party publishing sites.
If Murdoch is going to go through with this de-indexing Mexican standoff thing, he might as well do it the right way and drive the fear of God into Google. As a spectator, I’ll enjoy watching the fireworks.
Of course there’s another sideshow going on here as well – the renegotiation of the MySpace search deal with Google that ends next year. That deal brings in $300 million a year to News Corp., and it’s clear Google is done paying that much money.
nice post. it would indeed be an interesting development, if content owners can take some of the power away from almighty google.
this could propel bing to be a powerhouse for news and have bragging rights.
Doesn’t this assume that I need Murdoch’s news? If he can get everybody to do so, hey quite the coup. Ya think that the New York Times is going to facilitate News Corp? I’ll miss the journal in the same way I do now — I am not a subscriber. 90% of the times that I wind up there, it is the result of a blog entry.
Is Murdoch going to prevent his links from being blogged about as well?
I don’t think this is about what you need or what any particular individual needs for that matter. I think Murdoch knows that his content has little value to the individual as there are an endless supply of free alternatives. I believe that he knows charging individuals for news will not work in the long term.
But he wants to be paid for the live streaming of Hannity, Beck, WSJournal, sports and the like.
I think he is going to sell the online content to someone who can enhance the advertising revenue. That would most likely be MSBing.
It could really shake up what SE people are using. It’s happened it the past.
That’s a great idea, except for the fact that news is a cheap commodity and getting cheaper and there are probably about one or two zillion alternative news providers ready, willing and able to fill any void. The problem is not how more news or particular news gets to market, but it’s separating the wheat from the chaff, and Google is the best there is at that still. Unless News Corp wants to keeps it’s news a trade secret, there is never going to be anything to stop people from repeating and commenting upon what News Corp said. Murdoch is going down a rat hole on this. There are much better ways to deal with the problem. Punching Google is not the right one.
News is cheap, opinion is pricey. Opinion from objective, learnt, professional, those-who-matters is priceless. Take your pick. You want bunch of guys who happened to be around a source to take some photos and tweet the world about it. Or you want a professional news reporter to work with his content team who brings out a thorough detailed investigative report with complete objective mind.
You guys have it backwards. Opinion is cheap, news is pricey. Very pricey.
Accurate, timely, fact-checked reporting – and the accountability that goes along with it – is very expensive. Sending a guy out to Kabul or Tikrit to get the straight dope from the field is expensive. A real news room filled with professionals who run around town to get the story, and separate fact from fiction, is expensive.
The collective hive mind of the blogosphere has created a flowing gusher of opinion that antiquates the old punditry-for-hire concept. Yes, there’s still high-profile gasbag personalities like Rush, Keith and Glen – but the rest of us fill in the cracks everyday wherever an “Add Comment” button is to be found.
HuffingtonPost doesn’t have a news room, they don’t have fact-checkers, they don’t have investigative journalism. What they have are links to other content that other people paid real money to create and – opinion. Dirt cheap opinion.
When the “old media” finally suffocates, who exactly is going to foot the bill for the real reporting that we’ve come to regard as a commodity? HuffPost? TruthDig?
I guess we’ll just lean on the BBC to do the heavy lifting.
Totally agree. I think murdoch is panicking. Tradition news outlets will be dead in the next 5-10yrs. The generation that reads news from these media outlets is getting smaller. Just give me the damn news alert on my cellphone. I will check for videos on youtube. Then whenever I can I check twitter or my reader for what the bloggers are saying.
Frigtard, where do you think you’ll get your newsalerts from? Your Politico.com can’t cover a terrorist attack in Karachi. For that, you need large news organizations. A 5 man news blog based in Mountain View cannot cover the happenings in Sudan.
HuffPo et al are still news aggregators. I didn’t see any of their reporters covering the recently concluded state elections here in India, and this will be a newsflash to you, but trust me, they never will cover that either.
All the idiots foretelling the end of traditional news media, tell me one thing: what alternative do you have? Don’t say Twitter, or YouTube…most people around the world don’t even have access to two square meals a day, let alone these things. Do you think you will see a live twitter feed of bomb attacks in Sudan? I don’t think so..
News definitely is cheap, but what about real-time analysis. That’s what WSJ is famous for…
LOL @the Washington Post
There is not anyone left at the Washington post who knows how to make money with news.
A lot of you tend to forget that blogs can’t cover all type of news.
Sure, a TechCrunch can beat NYT hands down when it comes to reporting tech stories. But when you wade through the bulk of news – international affairs, wars, world politics – no blog can hold a candle to the kind of resources (and reach) that major newspapers have. Do you think the Prime Minister of China would invite a Drudge Report reporter for a one on one tete-e-tete in his office? I don’t think so..
But a blog could link to articles of major newspapers and have little excerpts/summaries as well.
And Google could simply index those blogs.
There’s always BBC world news.
This is not about Bing or Google or the shift in power. This is significantly bigger. If News Corp goes through with the de-indexing and then goes and sells the rights, there will finally be a business modek for both small and large content producing entities.
All websites will start participating and content will once again cost the public money. Following that the major search engines become subscription style services with niche search engines being a big new industry and the whole landscape turns into cable television through your computer.
This is just a way for publishers to claw back some of their livelihood from the search engines. No need for search engines to become subscription based as they would still be ad supported and bring in plenty of cash. They just would have to give up some of their profits, and it’s about time.
This is the search engines getting squeezed in what I would think of as the ‘traffic stack’. Publishers have been getting squeezed for years, but now the balance of power is shifting toward publishers.
Would the GOOG millionaire employees have to give back all their fancy sport cars too? If Murdoch is serious there will be some paper millionaires jumping ship to AAPL for their next free ride.
Google accomplishes more to benefit the average person in one year than Microsoft has returned in its entire history. With Microsoft we have a couple of greedy founders one of whom farts away his money on franchises and another who pisses it away on his charitable elections, as if he’s the god of benefactors. What BS. Microsoft sucked so much out of the software economy, it’s a miracle anyone outside of Seattle still has a job. And now the prick BG wants to give it back like that’s supposed to be enlightened somehow. Give me Brin and Page any day of the week. At least they are helping the luddites in our nation to stop reading dead trees.
You sound like a disgruntled MS employee or a Goog stock holder and has no idea what software means. I am not suggesting MS is great and Goog is bad. Both have contributed to common man in their own right and have different means to make money. Both are profit making machines that are in the job to increase share holder value.
Enjoy Windows 7 while you can, genius. I’ve been using Unix solutions for 30 years.
Backslash, backslash, into the pretty “folder” you go.
Using Unix for 30 years. Obviously I am not living in cave. I moved on to something more modern like Ubuntu and Win 7 (looking forward for Chrome OS next week).
Quick clarification, you are not Larry Page (in disguise right). If you are, then you are really pissing me of wasting your time posting silly comments here instead of increasing my Goog stock value.
What does GK stand for, Got Krabs? Only an novice idiot would literally believe I’ve used Unix for 30 years. Unix, Xenix, Apollo, Solaris, HPUX, BSD, Linux, Ubuntu. What the hell is the difference. You wouldn’t know.
STFU Communist.
MS created jobs and wealth. And your socialist ass thinks they are evil because they made an insane profit?
STFU
But just for the record. I hearts Google and it’s cloud.
+1
How is giving money away greedy? Your assertion was petitio principii to begin with, but the general incongruity suggests benightedness.
As for your broader assertions, I don’t have the time, patience or the hope that you will comprehend to expend time debating them.
First Twitter licenses content to search engines..
Second, WSJ and NYtimes
Soon Bloggerati ll want $$ too
Then all Ma-pa shop websites ll charge to index..
And Google/Search Engines ll come up with “ContentSense” program to allow everyone on the Web get paid to provide content.
Back to SQUARE ONE. Zero Sum game
search for specific wsj content common sense users use the wsj homepage search bar.
search for specific nyt content common sense users use the nyt homepage search bar.
inbound search traffic from google is probably from scrappy surfers that dont have a unique
bond with either site.
COP – You make a good point. Content providers need search and search needs content.
If something like this happens I am writing my own firefox news aggregation add-on. I don’t want to go to Bing to search in the WSJ and Google to search in the LATimes.
Murdoch will do anything. However, his argument is against crappy traffic. I think he wants subscription revenues again. That’s what he’s known his entire (long) life.
he wants more money he could possibly gets
Appropriate name. If it’s strategic then Bing isn’t the answer. If it’s tactical then it’s a smart short term leverage play. I think it’s strategic because the Bing play is like playing dotcom companies against each other. Not sustainable. I do applaud the fact that TechCrunch analyzed this so that folks who equate Google to a deity now realize they have vunerabilities. I’d pick Murdoch over Eric in a street fight.
it would be interesting if he manages to create the equivalent of sky on the net, I’d be tempted to pay for a sci-fi channel for example, as I don’t have/want a TV – I’d resent having to pay a license fee to the BBC for a web based TV channel though.
Rupert may have an interesting angle, primarily because he has the WSJ – the only non-porn content money maker on the web.
To try to expand this WSJ model will result in the same results as the NYT has already experienced once and their massive lay-offs indicate it didn’t work out well, as they are withering away as the national “paper” of record.
Any search model that involves paying content providers like the NYT directly will soon be overwhelmed by other “paper” news providers begging ‘bing’ and ‘yahoo’ to pay them too.
There is little content on the WWW worth a nickle.
Only successful pay site I use is ‘Scientific American’ which has tons of content of value and intelligent readers, who appreciate SCIAM Digital services.
I suppose Twitter has its place, but I wouldn’t pay a even a penny to look at it. Given their October numbers yesterday, I would guess that I’m not the only one in that category.
Everyone on FaceBook stops participating in a couple of weeks, month at most. Only audience FB has is Scrabble players — and they’re useless, because they are not out buying crap from FB. Soon most intelligent people will realize that FB is just one giant “Nigerian Phish” site.
Oh please, go check out the WSJ site, there is not one piece of “must-see, can get it only here” business news on that thing.
In fact, there are tons of business blogs out there that produce higher quality and/or bring you key excerpts from other in-depth analysis beyond anything the WSJ offers, and that you would hardly ever come across otherwise.
This post is built on a lot of wrong assumptions. By the way, Wsj.com has the same traffic size and pageviews as techcrunch. Can techcruch go against Google? http://www.alex...iteinfo/wsj.com
“If other media companies joined Murdoch Google could actually find itself in a very difficult position, where Bing had content that Google didn’t. If you knew that Wall Street Journal and, say, New York TImes content was only in Bing search results, mainstream search users would suddenly have a big reason to go to Bing.”
Please see my comment regarding the first article about Murdoch ousting Google… give some love to a comment hours ago
Maybe Michael had that thought in his head hours before you commented on techcrunch.
The original post brings up the point of Murdoch using Bing against Google. The youtube video was posted 4 days ago, so there is no reason to “give some love to a comment hours ago”.
maybe so, but who cares…
You cared enough to comment and to update your Twitter a few times. So I guess you care.
link?
http://www.tech...comment-3095451
honestly, it is a comment that most people will realize when preparing an article. Also, I don’t know every body reading every comments in TechCrunch article.
I read them ALL. I would hate to miss an opportunity to go TROLLING…
Jason’s right – he should be running one of these big search engines. This is a pretty brilliant idea and MS or Yahoo could certainly afford it to do something like that.
Well, he is running a search engine and this is nice strategy on Jasons part… recommending a disruptive tactic publicly to a Google adversary…
I don’t see them killing Google on this because how many Google searches are news related? I could see a number going to bing if this goes through, but all in all, a little competition never hurt anyone…
It’s an interesting idea, but the prediction that google would loose double digit search share if a few big media companies decide to opt out is a pretty ridiculous. People would just go to the actual news site if they want to search and continue using google for everything else.
With your last sentence, you just admitted that Murdoch doesn’t necessarily need Google.
We would only know if there were statistics for how much traffic Google drives to the WSJ for example. It really doesn’t prove anything…
Clayton seriously? How many Google searches are news related? How about 124 million search queries a month? Plus the 1.22 million queries for local news, 2.74 million for daily news, the 52 million queries for news(paper)(s)? And start to count the millions and millions of search queries for specific stations, newspapers, news sources, etc, such as ABC news, NBC news, CNN news, Channel 8/6 News…
Imagine you controlled TIME magazine; 37.2 million queries a month for “Times”, not counting queries for “The Times”, “Time Magazine”, “Times Magazine” (yeah misspellings count too!), etc, and you made this move and switched to Bing only… you’re transferring all that traffic from Google to Bing!
Not only would you take a huge chunk of Google’s revenue over time, but you increase consumer trust on a different brand (Bing in this case).
If I can’t get to Techcrunch, and all the other news sources that come before it in ranking for “tech news” (Engadget, Slashdot, CNet, Wired…), through Google’s search, BUT I can from Bing’s, trust me I would have no choice but to use Microsoft’s search. And honestly who wants to use multiple search engines? I will use the search engine that helps me connect with the content I want, and if that’s not Google so be it…
I see this is a bad thing for search providers since it might cost them a chunk of their revenue, BUT a great thing for publishers because they can now get a piece of the pie, AND a great thing for consumers since news sources will have a more compelling reason to output MORE great content to get MORE of the pie. The more good content you publish, the more traffic you get. The more traffic you get, there’re more reasons for someone to want to be the sole indexer of your content. And pay the premium for the benefit.
How? “Time” or “time magzine” are not news-related queries, they are searches for the URL of time magazine, which is a generic search query.
The thing here is that news sites cannot create artificial scarcity of information to profit from, because information on the internet is overabudant.
Also remember that people switched to google primarily because of the quality of results, not merely the size and completeness of its index.
My next business is going to be an aggregator for all things Wall Street Journal related. My blog will discuss every article of note in the paper, and my staff in India is awaiting the next article. I hope that my lowly blog can drive a small fraction of that traffic!
If Google and Bing can pay Twitter to index content (on time) created by others freely, why cant they pay WSJ and other media companies that pay for the content to be created? Murdoch, might have just saved online media companies by giving them a business model.
oops! Didn’t watch the video before commenting… my bad.
I agree with you. Moreover, according to another TC article linked above, Google brings only about 25% of traffic to WSJ site. Supposing that 1) this traffic is not so much monetizable comparing to their regular visitors, and 2) their online ads revenue is only a fraction of printed ads revenue, Murdoch will actually not lose so much as some people would want to believe.
Then, there is this fact that Google pays Twitter to index them. Some people are terrified by the idea that search engine ought to pay to the source they index, but I think they indeed could.
Many of these content providers are businesses with staff, salaries, operations. Google doesn’t have a problem making money off them, why don’t they get something in return besides “traffic” from search engines and related ad sales. Obviously traffic from search engines and ad sales couldn’t save many of these traditional media companies, so maybe a more reciprocol relationship with content indexers will help.
I really dont think any other media owners or corps would make such a mistake by de-indexing from google. Google makes people recognise companies with their search engine and I dont see any one loosing revenue in terms of ads and popularity.
Social media and bookmarking sites where users share, save, and link to content found on sites such as the Wall Street Journal and New York TImes would still be indexed by Google.
Good point! I wonder how that would work for Google from a legal perspective.
I think it’s a ridiculous idea. Search engines, any search engine, should be a gateway to any website. How the website deals with it’s search visitors is up to them. Once you start on this path, where does it end? Will it stop with media companies or will bloggers want in on it too? If Murdoch wants out of Google, fine. Are they the only source of news? God no, anyone could get any story anywhere else. The big reason why people would want to go to a newspaper is for opinion articles and why go there for restaurant reviews when we have Yelp, Urbanspoon and others? Book reviews? I’ve got Amazon. This is a desperate attempt for newspapers to make a stand and it’s more stupid than desperate. Embrace the internet, don’t hide from it.
not when one has a monopoly
gotta make money Sandra. can’t run news orgs on love and fresh air.
Google built their monopoly simply by having a better search engine. People could easily find information through it and went there instead of using the portals that dominated the late 90’s. How did Microsoft build their monopoly? I believe they went with the “embrace, extend, extinguish” policy. Not exactly something you write on a Hallmark card.
As for the newspapers, they have fought the internet for a long time and still refuse to fully take advantage of it. Even if Microsoft pays the newspapers, how long will that last them, a year, two? This seems more of a last ditch effort to make money rather than a long term strategy.
Unfortunately for most of us, there is a dramatic shift away from everything being free and I think Murdoch and gang are going to lead the charge. While people continue to claim that this can not happen due to so many sources available, I do believe that institutions like Murdoch are tough to reckon with. If old media unites around this idea, it can pose a real threat to how new media functions and operates.
Agreed. I think this is the way to go; everyone benefits: the content providers who are currently bleeding cash, the consumers (since they’ll be able to read material produced by quality journalists), and the fact that Google will no longer be the monopoly that it currently is should push them to be more innovative, as Ask.com, Yahoo.com and Bing.com have been.
Amen to that.
Did you seriously just say that Yahoo, Ask and Bing innovate more than Google?! Google is one of the few companies that could be considered a monopoly that continues to innovate. I think the monopoly that we should looking to topple is News Corp not Google.
Not really. A monopoly company can be innovative. The fact that it is innovative does not cover another fact that it is a monopoly.
Look at how content providers are losing money at the same time how much money Google is making, it is not hard to see that Google is becoming a monopoly power and such trend will hurt productivity eventually.
1. Google, at best, is an accidental monopoly… It is, because we who have made it so. Unlike, in case of an OS, where it is so much more difficult to make a shift, in the case of a search engine it is just a mouse click away. So discontented fellas can move any other search engine.
2. Search is not just about indexing content. It is about understanding search queries and giving out the most relevant results. Which is what Google is very good at.
3. Search engine’s are just a gateway. Ironically, when businesses are paying Google to lead traffic to their sites through targeted advertising, some wan’t to be paid for leading such traffic for free! The disaffected media companies can charge the customers for viewing their page instead. Or even better get them to view an ad before reading the content.
3. Because of the Fair use clause, a search engine cannot be stopped from indexing any publicly available information.
4. Search is just part of the story, most of Google’s success is the result of its world wide partnership for displaying targeted ads on the websites of partner organisations.
5. If Microsoft & Google have agreed to pay for indexing twitter, its because they will have direct access to the users & their content which they can analyse and place targeted advertisements. Newspaper websites do not provide such value.
The only way this works is if the traditional media companies create content that people want, and that isn’t created elsewhere (and indexed by Google). This is part of their hesitation – there are a LOT of new models, and a lot of great, out of work journalists. The NYT – and even WSJ – aren’t what they used to be. THIS is what will stop Murdoch, not anything else.
Is it just me, or does dividing up the internet sound like a horrible idea. As a consumer, I hate monopolies, and exclusivity on the web.
However, I also understand the nature of business and this will undoubtably happen in some shape or form in the future.
Jason is right. This idea is a bit unsettling too.
But the exclusivity happened recently when Microsoft and then Google bought indexing rights for the whole Twitter stream.
Actually, both of these Twitter deals were specifically *non-exclusive.* Any other search engine or web property can purchase the same rights as Microsoft and Google.
Michael – If Microsoft offered you a bunch of cash would you give exclusive rights for Techcrunch to Bing?
define “a bunch of cash.”
Michael, just tell us for how much would you sell yourself to Bing.
That’s the issue. Murdoch would have to charge Bing enough to offset the resultant loss in ad revenue from googles traffic.
And Bing doesn’t necessarily want to be the one to set such a dangerous precedent of paying for exclusive content; and starting a war that has a zero sum gain for the search providers.
If Murdoch wants to hurt Google he should just buy Yahoo!
Clearly, Murdoch isn’t making much from Google to begin with.
They’ve made it quite clear that the bulk of traffic coming from Google is people who read one story and leave — this type of visitor is not considered valuable.
Most Google traffic–paid and organic–is rubbish across industries. Business that are overly dependent on Google as a source of traffic should be doing everything they can to reduce that dependency.
Like andy writes, this shows the economies which won’t make an exclusive pay-for-content deal work. Bing can’t rationally pay the amount needed to compensate for Google traffic on the top 10 news sites.
$10 million a year
$10 million a year good enough?
for $250-500,000 a year–in advance–Mike would do an exclusive with Bing I think.
Or Google could just choose to no longer pay attention to any site’s robots.txt and index everything anyways.
I hate to sound like one of those “information must be free!” zealots, but I don’t see how you are going to block someone or something (aside from putting up a paywall) from getting at content. Even if you put blocks in place, it’s still an uphill battle if the person wants the information enough.
Two ways.
1) Sue Google if they don’t honor robots.txt. A company like Google may need to cough up billions if they trespass, uninvited.
2) Any website can employ some programs to watch for heavy requests from a single or group of IP addresses per second, minute, hour, and thus block traffic from that IP.
1) There is no legal way to offer something to the public, but sue a member of the public for accessing it.
2) Think about how many IPs Google owns or could own.
There is no practical way to block a dedicated agent from indexing publicly available information. It would become an expensive technology war and Google would have the upper hand with both better tech experts and a bigger capital reserve.
Russian? You definitely sound like one.
?? what does that have to do with anything? He makes a good point.
1) I am not a lawyer. But, I can say this: General public are INVITED to the party. Search engines, which are run by business entity can crawl by INVITATION only. Unless any bias is shown in controlling a search engine, the content web site wins the war.
2) Not as many as a simple program can keep track of them and list of pages it accesses at what frequency.
It is very practical to block ANY dedicated agent from indexing. I am a programmer/architect, and can always ready to accept challenge. Let me know if you find someone willing to pay for 3-6 month contract work at handsome price (> $100). By the way there must a billion programmers who can do this, technically.
At the end it is less of a technical challenge, and more of a legal challenge. You can’t break into a house, just because you can break their lock. It is ILLEGAL, however flimsy the lock may be!
This is what I was thinking too.. would it be illegal for google to index Newscorp’s public websites if murdoch didn’t want them to? Or could they just go ahead and do it anyways?
I hope that they would just index them anyways. I would hate to have to search on Bing, Google, Yahoo to find all the information on a topic if they all had different exclusivity deals.
Exactly! These comments about it being a brilliant move are as ridiculous as Calcanis’ comments and self adulation. News Corp and other sites don’t have control over who indexes them. They can ask Google not to index, which Google has been honoring so far, however, there’s no reason why it would continue to do so when it effects their bottom line. Your self proclaimed brilliance is based on a shaky assumption.
The legal battle would be interesting, but Google could end up losing in the end due to the criteria for Fair Use which considers the effect on potential market value. There is obviously market value since Bing would be paying for the right to index and Google ignoring the robots file would not be considered Fair Use since it erodes this value. Some interesting links for further reading:
http://w2.eff.o...evada_order.pdf
http://fairuse....apter9/9-b.html
http://arstechn...006/01/6063.ars
It would indeed be a masterful stroke, if there was any chance of it happening. Microsoft and Yahoo aren’t stupid. Setting a “pay to index” precedent would change the search engine business in a way that could only harm then in the long term.
I’d also question how valuable it to search companies to have newspaper content indexed (outside of current events). When was the last time a NYT article popped up in your search results?
+1
+2
These days everything and everyone important has a website, people who want their news from a specific source go directly to that url, the rest of the news is replicated several times (even the headlines), and on current events blogs and twitter beat those traditionals hands-down, examples? Iran elections, Hudson plane landing, Micheal Jackson’s death.
Even the recent scamville debacle, while techcrunch was uncovering the rip-offs the NYTimes were busy pilling praise on Zynga.
I for one, the only reason I’d go to NYT’s website or any other of the type is confirmation from an ‘authoritative source’ NOT to discover news, anybody who does that is about 2 – 3 days behind!
+3
I can’t remember the last time I searched for something and found what I wanted at NYT or WSJ. So Bing having exclusive rights would have no impact on my search engine of choice.
Oh, it is you. Do not confuse the behavior of an individual with the behavior of a huge market which has many different segments.
Long term? Hell, I’d give it two weeks before websites started blocking Bing in robots.txt as grassroots retaliation.
the “walled internet garden” didn’t work when AOL and CompuServe tried it… why would it work now?
Looking at generation Y and beyond I believe they simply won’t read the news if they have to pay for it… end of story.
Then their won’t be (commercial) news. Someone (the end user) always has to pay in some form.
If ad’s don’t cover the costs (and from what I’ve seen and has been said they don’t) then Murdoch will close news corp. and take his money.
He’s a clever f***er that Calacanis!
It’s a good idea – it’s conceivable the major publishers could band together with bing and outflank google, the issue will be the sheer marketing costs necessary to get the word out that google no longer has access to this content and to get users to change their behavior.
If you knew now that if you searched in Google news for something you wouldn’t get results from the WSJ/London Times etc – would you migrate to bing for the same search? doubtful.
bing would need to pony up a large amount of cash to poach away a sufficient number of publishers to pull this off.
In the short term google is going to spank News Corp over indexing their content and then spank them again when it comes to renegotiating the MySpace deal
This sounds absolutely terrible for consumers. How in the world are we supposed to know which search engine to search for which content? Awful.
I’d need a an app to go search all search engine results…all just to end up back in the same place – yuck.
Hello meta search app!
True
but would the meta-search app have to pay Bing and Google for access?
you see where this slippery slope is going?
As a user, I could care less about whether Bing had an exclusive with the WSJ. I think the drop off in traffic would hurt WSJ far more than Google. As a user, I would just click on another newspaper article from Google.
You’ve never read WSJ, right?
Why read them when you have The Economist
why WSJ? i want to know the time NOT how to build the watch. if you have time to read The Economist from cover to cover, you’re a better man than me.
Which is also a pay-site. No thanks.
But yeah, the Economist is great for a long plane rides.
Yep, the Economist is always one of my top picks for air travel.
It wouldn’t work. 3rd party sites would proxy the content into google’s index and Murdoch would have a hell of a time stomping on all of them.
But Murdoch intent is not to stomp websites for linking to them, that’s not the idea. In my opinion, what he want is that the very big players like Google will pay them for indexing their content.
What makes media content any different from any other product? There are costs associated with producing it and it is up to the producer to decide how to pay for those costs and whether to be ad-supported/free or paid subscription. How many small business owners want to give their product away for free? I love all the free content on the web and rarely pay for any content (I do subscribe to a professional website for their exclusive learning material not indexed as far as I can discover.), but I respect content owners wishes to do what they want with their product.
But practically speaking the content will still end up indexed by google
Good post lending balance to Schoenfeld’s myopia. Murdoch’s idea was given legs when twitter got paid by bing and thence google to access their feeds.
Funny though, this assumes that most of the links in which people find this content is from search engines and not from various other sources like blogs or direct traffic, which are arguably more current with the times than a random internet search.
Still, I think bing could really hit it off by providing an easy way to search through all of the online content of a particular news publication and make it easily accessible to everyone. Right now, the current “click here for sites similar to this” is a pure JOKE.
Yet again old media looks for someway to hamstring the internet. Its openness and journalistic meritocracy pose a threat beyond the media realm. I am not sure that Murdoch fully realizes what a monster Google has become. They have their hand in seemingly everything and generally do an excellent job. This to me is just one more nail in the coffin of old top down, controlled media and good riddance.
Paul
http://healthjo...b.blogspot.com/
I can’t wait to pirate news!
+1
check sidewiki comment on this post
https://www.goo...exicanstandoff/
I have also been a spectator in the whole Murdoch thing, but following the story quite closely.
I really hadnt thought about it the way you have put it forward Michael. I totally agree that if he is to deindex all his sites from Google, why not offer the content to a rival at a price.
Not only is he then protecting his content, but at the same time, he could be earning millions per year to give Bing (or even Google) the rights to index the content.
Fantastic idea, and one i think News Corp should certainly be looking into.
Nice post Arrington, keep it up
If you had to look for the content you wanted on 4-5 search engines it would end up frustrating you and harming consumers as a whole. Also, if Murdoch wanted to charge Google for indexing his content, he would have to charge everyone else as well.
Otherwise, he could have a legal battle on his hands that isn’t worth the cost.
If you are going to give something away for free then give it away. Don’t charge specific people/ companies because they are rich.
What? Google brings 25% of the WSJ readers, do you think most of them would care less if the News would come from WSJ or another decent source?
Just because YOU NEED WSJ doesn’t mean 99% of the others do. And those that need WSJ go directly on it anyways, not through Google.
People with special habits always make the mistake of thinking they’re like the average Joe = where the numbers are.
You are right. The same information lives in a lot of places. But people pay 2 dollars for a bottle of water that they can get for free from the tap. Information, just like water, tastes better with the right logo
I agree. Although I might prefer to get some news from the WSJ perspective, I have little problem getting it from elsewhere. The Twitter agreement might have set an important precedent but there will always be a provider with an ad revenue dependent business model that would quickly fill the WSJ vacuum.
Take it one step further.
Make all the Facebook content exclusive to Bing.
Since Facebook is an MS partner.
That would be all the more reason for users to leave Facebook for Twitter or better yet The Next Big Site that has better policies. (A lot of Facebook users like and use Google.)
Is there any law that would prohibit anyone from indexing a web site? Most websites are public and free to access, so what would restrict a search engine from accessing it?
I know you can use the robot.txt files to not be indexed…but isn’t that more of a courtesy than a legal requirement?
Great point. robots.txt is a courtesy, not a law. This would just discourage search engines from honoring it.
heh? Site can always block an IP
You think google as “an IP”?
true, but that’s not how you can stop a major search engine.
content providers may have to lock their sites behind a login. and the search indexer of choice will have to act as a proxy for a logged-in user and display the content in some kind of viewer or frame.
for most content providers, closed website is death to the website popularity. WSJ is making this work, though.
It’s not in Microsoft, Yahoo, or Googles long term interest to strike a deal like this because it is a slippery slope. Sure, there might be short term benefits for Bing, etc., but they’d be ultimately moving the power into the hands of content providers, which would be a boneheaded move strategically.
You can crap in one hand and have media outlets de-index from Google in the other hand. Let’s see which gets filled first.
search engines paying for content wtf is that?! what happened to “information must be free”? don’t let that ancient cunt Murdoch win! he can take his content and go, hopefully no one will follow.
Too many sources for news; it’s a commodity.
Terrible for consumers. This is not how the free market is supposed to work. This is not open competition based on merit. It’s using money to secure more future money.
If I own a search engine I come out immediately and announce that we will never pay to index.
This is exactly how the free market is supposed to work.
+1 It’s up to the consumer and the competition to strong arm a company. That’s how the free market works.
Awesome post Michael !! Keep up the good work
Michael, asking you about a totally unrelated thing – Why is there a small smiley at the very bottom of the each page on TechCrunch ?
It’s because Techcrunch runs on Wordpress.com and they put that smiley there.
I’m going to need a search engine to tell me which search engine to use to find the content that I want. Can’t wait.
I’m surprised no one has yet mentioned the fundamental basis on which Google took over search. Authenticity. Page Rank eliminated the commercial basis on which most search engines ranked websites. People realized they were being scammed by this ’searchola!’ Google’s searches were based on the valid popularity and semantic authenticity of the results. Of course, this started the wars with SEO, but Google has stayed steps ahead with constant vigilance. Murdoch is thinking with a Web .02 mentality. Wow, I think I just invented a term for this bs. WEB .02.
Haha!
Isn’t there a key issue being overlooked? I thought that there was already established law concerning the copyright legality of creating indexes of website content. After all, Google crawls a large chunk of the entire web including these news sites. What will stop Google from indexing this material whether these sites want it or not? Or do some of the sites such as WSJ permit Google special indexing access that bypasses subscriber logins (which of course could easily be rescinded by WSJ)? If so, that should be reported as an essential part of the story.
I think you might be overestimating the importance of New York Times & Wall Street Journal to a “mainstream” audience. But it would be fun to watch Microsoft throw more money down the toilet.
SO.. if Murdoch does de-index google in favor for another engine, and other media outlets start doing the same what would we have?
Probably what we have with radio and tv. Different stations to view the news/content that fits a persons taste, or lack of.
Soon you might see – engine A carries content from media source A, and engine b carries content for media source b, and so on and so on.
So is Murdoch’s ploy really a gambit to get everyone riled up while ‘networks’ are recreated on the web and then be able to specifically target marketing dollars based on viewer demographics?
Somehow wrangling a large portion of the web into a more mainstream media model?
hmm, I wonder. But I digress as a previous poster stated “i’m an idiot” maybe so, just my thoughts.
In the short-term, I suspect this would be a huge opportunity for a meta-search engine to just merge the results of bing and google, and we’d all act like nothing has changed.
However, with products like http://beta.thoora.com getting really good at aggregating and managing news, I think the long term for news search may be limited.
yeh – agree with you on this.. this is the right time to be trying something like this if you are a news publisher.. I am surprised none of the others have tried it.. more detail here:
http://www.riaz...n-to-something/
Jason’s idea is really smart Michael. With big enough content coalitions, the barganing power would shift to the content owners.
But one company has already played this card to its advantage. Twitter. Pay me, and I’ll let you index my content in a timely way. They got payouts from both Google and Bing.
With Newscorp type of sites, the value proposition would be different. Pay me and I’ll give you exclusivity to my brand. From a purely information content perspective, this would not hurt Google. People could possibly find 98% of the same info contained in Newscorp sites (like breaking news) on other sites indexed by Google. But from a branding perspective, Bing could put the logos of the Newscorp brands on their main page and tell users that if you like these sites, this is the only search engine where you can find their stuff. Bing is already image intensive on its main search page.
The percentage of people who would recognize at least one Newscorp brand is almost 100%. And a significant amount of them would take that as a reason to switch.
We already know that Google doesn’t fare much better than the other search engines in blind tests. So it’s mostly a brand thing. The company is in the top 10 in the Interbrand list. The equation becomes what is stronger? Google? Or Bing + WSJ + Fox +New York Post + any other content site in the Bing coalition.
I like this strategy a lot.
+1
Awesome post. I personally might not care about Wall Street Journal content but what if Wikipedia did the same? I can’t image a search without Wikipedia results and would probably switch.
What sites contain must-have content for you?
Ugh… if the ability to block a search engine is proven legal, content providers will finally have the golden goose they’ve been looking for, and the Internet will never be the same.
It seems inevitable this will be a Supreme Court decision one day.
you can do that in robots.txt
Super, a tiered internet that everyone can be a part of, if they can afford it!
@denniscrow,
The problem with Page Rank and Search is, people must Search in order to receive the most relevant page or query results.
There are signs that people are searching less, opting into life-streams, brand-streams and whatever other relevant information has been passed on or recommended to them. Information is delivered, you remember, like television delivered information in a passive way.
People have fallen in love with not having to search or ask questions.
The general public doesn’t know or care about many things that happen online. They have their favorite sites bookmarked or a comparable app on their desktop or iPhone.
As a knowledge repository, the content of the web is useless once information is not being indexed or put behind paywalls. In effect, we’re simply repeating the cycle that many thought/think the open web ( and open source ) have tried overcome.
This wall creates scarcity, which creates demand, which creates value, which creates paywalls. Information will cost you, if you can afford to log into the web and search for information about your respective query. ie., medical, research, etc. News (or information)–technically, is not a commodity if everyone is contributing to it and then allowing everyone equal access to it.
There is nothing smart or ingenious about Murdoch beating the big bad Google or siding with Bing in an attempt to leverage a *business.
The implications here, should be considered far greater than a school yard fight.
Totally agree, in the best case scenario, I’ll be able to find information for free, using the three or four dominant search engines. In a worst case scenario, I’ll have to pay for the limited info each engine will give me. Crappy
@Cody Issacson
But do we ever want the Supreme Court dictating the rules of the web in this country or anywhere?
They may try, but the beast is to wild for one government to control. It would be a sad day if the media conglomerates and governments controlled the entire shooting match. This is all we have left.
Oh no — that’s what I meant. I can’t imagine 9 academics telling us what can be stuffed into “the tubes” and what can’t be.
If they did, search would go overseas and maybe underground like BitTorrent. Everyone would lose.
Wasn’t this clearly the next logical step? Bing should have done Twitter, facebook, digg, etc way before now as they had the leverage in place: http://www.loca...on-the-horizon/
ronald — 10:47 AM on November 26, 2008:
“To fight Google one has to know what information is and how humans process it, and understand how Google’s work relates to that. Meaning how does Google use links, map reduce and big table in relation to what they call Information. The easiest way to break that model is to eliminate/reduce links.”
http://gigaom.c...-to-win-online/
This article is just about news, but one can actually push it father along and take away product search, or most of it. By integrating a typical search flow into what the search engine returns at which point in time of decision making. Some of it is already in Bing.
Yes I talked about this to Steve B., what seems now a long time ago.
Beyond that, one could integrate Facebook data into search queries to provide better context and therefor better results for search and display ads.
“Stupidscript, personalized search is making a huge difference in search results, I’ve been finding. Logged in, I can see pages that were buried back in the 4th page of search results leapfrogging to page 1.”
Danny Sullivan: http://searchen...e-results-28445
Hurting Google’s business is actually not that hard, one can even make money in the process. It requires just some scale to be effective.
I think Om’s got it backwards. The way to break the Googlebot is with linkspam, not fewer links.
If you comment just on the quote above,that was me. Om is to nice to have some fun with Google.
Can’t Google just spider the sites anyway? If they don’t require a login, Google can just ignore the robots.txt no follow and spider it anyway. They could even change their boy name so Newscorp can’t block it. It would be a game of cat-n-mouse.
I’m sure Newscorp would love taking money from Google through the courts.