All forms of research whether they be polls, surveys, focus groups, or mall intercepts, inherit some form of bias. In part, poor research methodologies have lead to a ‘taint’ in perceptions to the industry. The Internet has been abused and not properly utilized when it comes to research.

Polls, surveys and all other forms are different beasts by nature. Each have it’s strengths and weaknesses and all are valid as long as the methods and results are used appropriately and interpreted with care.

Heavy clickers and professional survey takers exist, and many are trying to minimize their negative impact on polling, surveying, and advertising. We in fact are very careful that we integrate with a site successfully, remain sensitive to the user experience, and take pride in our adherence to sound research fundamentals.

Crowd Science Demographics also uses a zero-footprint approach. It takes up no permanent page real estate and can live along side any existing polls, advertising or other widgets. We simply only leave a small and polite trace on a small number of visitors. After all, it’s part of our research rigor.

Also, you can’t change the questions? I have questions specific to my audience I would like to ask, such as what motorcycle they own, if they plan on buying a new one soon, if they plan on buying new parts soon, etc.

Visitor demographics are valuable to webmasters and bloggers, not only for selling advertising, but also for developing content more suited to the audience.

How data is collected is also very important from both a data validity p.o.v. and a user experience p.o.v.

Data Validity:
Inviting people to participate in surveys via ads as described above is subject to significant bias. Reference Dave Morgan’s article “Outing the Heavy Clicker” on Mediapost which, among other things, states…
“Who are these “heavy clickers”? They are predominantly female, indexing at a rate almost double the male population. They are older. They are predominantly Midwesterners, with some concentrations in Mid-Atlantic States and in New England. What kinds of content do they like to view when they are on the Web? Not surprisingly, they look at sweepstakes far more than any other kind of content. Yes, these are the same people that tend to open direct mail and love to talk to telemarketers.”

Polls that are elegantly integrated into a site, like those offered by Vizu’s Power-Polls service, can “train” visitors to participate over time by featuring interesting, contextually relevant polls. As a result, they can generate much broader and higher participation rates than invitations to participate in surveys. The improved participation, in turn, can lead to higher quality data about site visitors.

All sections of your site are not equal. To really understand your audience, you may need to understand whether or not different demographics frequent different sections of your site. At Vizu, we have seen sites that have drastically different types of visitors on different sections of the site. Consequently, we designed our Power-Polls service to be able to assess demographics on various sections of a site separately and then roll all of the data into a top level view.

User Experience:
People love polls and generally are not big fans of surveys. Having worked with 1000’s of web publishers at Vizu, one of the concerns we heard loud and clear was that of maintaining a high quality visitor experience. Most publishers did not feel that taking their visitors off site to complete a lengthy survey for which there was no reward fit their conception of a positive user experience. On the other hand, many sites get very positive responses from their regular visitors regarding the addition of high quality polling content that offers the opportunity for instantaneous self expression and self assessment. Additionally, given the low click through rates for ads, many sites are not keen on allocating a lot of valuable real estate and impressions to solicitations to take surveys.

Ease of Use:
If you are going to go through the trouble to paste some code on your site to gather visitor demographics, why not paste some code that does more than just that solicit visitors to take a demographic survey? A good polling platform will help you engage your visitors and can improve your site experience. Sites can even opt into professionally produced, contextually relevant polls or rotate in your own polls which you can use as the basis for additional content.

All this is not to say that surveys do not have their place, it just depends what your over all objectives are.

We used these types of surveys while I worked at Nielsen Online in New Zealand. The product is called Market Intelligence. The surveys are surprisingly accurate and do reflect the users on the sites. For example a site aimed at teenage girls does show a massive skew to teenage girls. Sites aimed at business and finance do show a large skew to male, educated, high income earners.

I was sceptical when I first saw the surveys to but the do work.

We used these types of surveys while I worked at Nielsen Online in New Zealand. The product is called Market Intelligence. The surveys are surprisingly accurate and do reflect the users on the sites. For example a site aimed at teenage girls does show a massive skew to teenage girls. Sites aimed at business and finance do show a large skew to male, educated, high income earners.

I was sceptical when I first saw the surveys to but the do work.

As for how that’s done – I don’t really like the incentives idea. After all, incentives are what started the decay of the information being provided by panels and other analysis software of old, so it seems to me like honesty (and therefore a higher value) can only be found in voluntary, un-coaxed responses. Crowd Science seems to have a handle on these things.

But, as the tools to judge sentiment based on actual usage and interaction mature, why wouldn’t you avail of both data sets: looking at what someone tells you they want as well as they’re actual behavior should be at least twice as interesting as one or the other, no?

If you’ve got both available, why not avail of the belt and braces, too?

Traditional marketing folk love demographics, but the new ones love implicit data. We’ll see…

Incentives might help, though they distort metrics in other ways.

Ultimately though I think we must move *way* past the survey stage, and work harder to measure the implications of the clickstream, time spent on pages, etc. Those are objective numbers that reflect what people really do rather than the more dubious what they say they do.

I just wanted to chime in and let you know that we’re floored with the response, figuratively and literally.

I think we’ve got load under control now. (Joyent have been fantastic this morning responding quickly and bumping our capacity — thanks Linda and Blake!)

I also wanted to let you know that we have certain protections built into the system for when we run into load issues. Specifically, the initial request that our publisher websites make is for a tiny static file (called start.js) that gets served from a CDN. The request to our application servers is made after the page completes loading, so if it’s slow, as it was this morning for WatkinsCJ, then it shouldn’t have any adverse affect on visitors. Except the ones watching response times in Firebug.

Anyway, thanks again for the response, and we’ll keep everybody updated.

And thanks Jason for the coverage, much appreciated.

For example: http://www.crow...e.com/overview/ is 404

MOD_PYTHON ERROR

ProcessId: 24006
Interpreter: ‘westley_django’

ServerName: ‘app.crowdscience.com’
DocumentRoot: ‘/opt/csw/apache2/share/htdocs/westley’

URI: ‘/signin/’
Location: None
Directory: None
Filename: ‘/opt/csw/apache2/share/htdocs/westley/signin’
PathInfo: ‘/’

Phase: ‘PythonHandler’
Handler: ‘django.core.handlers.modpython’

Traceback (most recent call last):

File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 1537, in HandlerDispatch
default=default_handler, arg=req, silent=hlist.silent)

File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 1202, in _process_target
module = import_module(module_name, path=path)

File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 304, in import_module
return __import__(module_name, {}, {}, ['*'])

ImportError: No module named django.core.handlers.modpython

Thanks for the invite. I’m in! Let me see if this thing really works!

For example on UnLtdWorld we have a platform called the Research Lab: http://www.unltdworld.com/lab (and featured here on TCUK: http://uk.techc...e-a-suggestion/ ) that provides insights into demographics (and more) dynamically (as a side not this is not for advertising, but to openly provide critical social insights to social entrepreneurs, relevant organisations and research).

[ Logo here ]

Of course, Crowd Science is going to be running into one major problem: people need to honestly take these surveys for the site to have any real value.

I actually think that people are more likely to be honest if there isnt any (monetary) incentive. There are many websites which offer to conduct surveys that pay people for their opinion. In such cases people just join to earn money.
However since people are volunterily opting for the survery, they are doing it just out of interest and are more likely to be honest.
But the question is how many people are actually going to opt for the survey?