Is Google getting ready to serve up display ads to people based on their Web surfing habits (as opposed to their Web searching habits)? Ever since the DoubleClick acquisition closed, industry watchers have been waiting to see how Google would dip its toes into behavioral ad targeting. One rumor going around is that Google is going to target ads to people who use the Google Toolbar, which is now bundled with Dell PCs.

The rumor came to us via an online measurement startup that expects Google to make an announcement about a new service leveraging the Google Toolbar at the upcoming Audience Measurement 3.0 conference later this month, which Google is sponsoring.

The rumor could be an attempt to spread FUD, but it is not just startups that are playing the privacy card. In a discussion with Washington Post editors and reporters on June 4th, Microsoft CEO Steve Ballmer raised a similar privacy issue in relation to the Dell deal (see video below):

Why is that toolbar there? Do you think it is there to help you? No. It is there to report data about everything you do on your PC.

Now that Google’s age of innocence is over, competitors will be bringing up privacy concerns every chance they can. Google already collects so much data on what people do on the Web. With the increasingly widespread toolbar, though, Google gathers data well beyond the search bar.

Since the Google Toolbar can track every site you visit, that data could theoretically be used to target ads served by Google (including DoubleClick display ads anywhere on the Web, or to further refine search ads). For instance, you could browse a Lumix digital camera on Amazon, and then see ads for digital cameras when you land on an unrelated travel site that happens to serve up DoubleClick ads. Or perhaps the next time you do a search on Google, it will push a Lumix ad out to you. Google could also use the data to create a Web measurement service that competes with comScore, Quantcast, Hitwise, or Compete.

These are all hypothetical at this point. But there is nothing stopping Google from doing so. Per Google’s general privacy policy, it reserves the right to process “personal information” for the purpose of:

Providing our products and services to users, including the display of customized content and advertising;

And the separate Google Toolbar privacy policy doesn’t say anything specifically about not using the data it collects for advertising purposes. In fact, it notes that:

Certain optional Toolbar features operate by sending Google the addresses or other information about sites when you visit them.

But it also notes that users can disable the toolbar’s ability to collect personal data if they choose (presuming they can figure out how to do that). At the very least, Google has certainly thought about doing something like this. One patent issued last March describes a way of:

tracking user behavior, determining a user topic interest (e.g., from a plurality of different candidate topics) based on the monitored behavior, and serving ads relevant to the determined user topic interest.

Google did not respond to an email I sent asking whether it intends to use the Google Toolbar to target ads at users.

If Google does start targeting ads based on Web surfing habits, you can be sure that Microsoft and others will add that to its list of concerns it brings to Washington. Ballmer, in that same discussion quoted above, believe it or not, relishes the prospect of competing against Google on privacy. Here is a fuller excerpt from the video above:

One of the things you can reward users on is their privacy. You can literally say, “Hey look, you will cede this data to us if you use our search engine, but we are going to pay you.” And it’s a trade. If your don’t like the trade, it’s ok. Don’t use our search engine or don’t use it in a certain way. And there will be competition between us and Google and whoever else along that line.

. . .the number of people who have any clue what data is being collected or not being collected by them—Any of you own a Dell PC at home, personally? There is not much about you it does not know . . . to Google, because it is their toolbar. We just won the HP deal, but anyway.

Why is that toolbar there? Do you think it is there to help you? No. It is there to report data about everything you do on your PC. I am not trying to say this is nefarious or bad, I am just saying being clear is probably the most important thing. And any user can say, “This is clear and this is OK with me.”

I actually think we are going to have to compete on privacy policy.

When he says he is willing to pay users to give up their privacy, he is being literal. Who would you trust more with your privacy, Google or Microsoft? I’m not sure I trust either one.