Crowd Science, a new demographics service that aims to help online publishers maximize their audience and ad revenue, has launched today in an invite-only beta. The site compares itself to Google Analytics, but explains that it focused on “demographics and attitudes rather than clicks”. The first 100 TechCrunch readers to go here and enter TECHCRUNCH as the promo code will be able to join the beta immediately.
The service tabulates demographic data by sporadically offering users an offer to participate in an embedded survey. To cut back on phony entries, the site only offers the survey to visitors it deems acceptable (it would probably rule out IPs that refreshed a page numerous times). Surveys are also presented in an unobtrusive fashion in the hopes that only people that are genuinely interested in participating will do so.
Crowd Science’s real value emerges after the data has been collected. The site performs detailed analysis of the customizable surveys, presenting site publishers with trends including age, sex, and income, that they can use to help tailor their content and advertising. Finally, the site also allows publishers to create “Media Kits” containing selected portions of their data results which can be sent to advertisers.
Of course, Crowd Science is going to be running into one major problem: people need to honestly take these surveys for the site to have any real value. For the time being, there isn’t any incentive for anyone to fill out the survey – instead, the site is relying on user interest. At first glance this sounds like a fatal flaw, but Crowd Science CEO John Martin counters that most people are willing to fill out a survey on sites they are genuinely interested in. He also explains that these surveys really only need 800-1500 participants to be statistically relevant.
Crowd Science will see competition from a number of audience-analyzing sites, including Vizu and, to some extent, NetRatings. The site raised $2 million in Series A funding from Granite Ventures last December.
This seems like great stuff — we are analytic junkies!
The use of surveys for crowdsourcing demographics doesn’t scale, IMHO. In fact the use of surveys for measuring opinion is going away as monitoring tools can judge sentiment through language analysis across social media, understand demos like age, location and gender by analyzing users’ profiles and understand emerging in trends in real time rather than having to wait for a survey to return enough results to be viable.
Also rewarding people for taking surveys taints results- monitoring and measurement via search/discovery and analysis tools return actual opinions.
There is a big change taking place in market research and it is a move away from tools like surveys.
Agree completely with #2. We should rather look for semantic analysis of user interaction (comments, content posted, exit links, incoming links etc.) – that would scale perfectly, not be biased and would easily transfer between topics…
I am going to try it if I can get in on the beta.
I figure it may be able to add some value.
Vicky H
If 800 – 1000 respondents is the number they’re aiming for, I’m sure they’ll get it by making the offer to a few tens of thousand visitors. However, it doesn’t seem like a particularly innovative way to tease out demographic data. A cooler approach, at least for younger demographics, is to tie the answering of survey questions with the awarding of points and prizes in a social game. Check out http://www.xuqa.com and http://www.iminlikewithyou.com . In the latter implementation, the “cost” of answering questions is almost zero; they pop up, one at a time, in a cute little flash window that’s tightly integrated into the rest of the user experience. “Points” are awarded instantly for answering questions and can be immediately redeemed for the right to interact with other members on the site (cute girls, cute guys) in meaningful ways.
I actually think that people are more likely to be honest if there isnt any (monetary) incentive. There are many websites which offer to conduct surveys that pay people for their opinion. In such cases people just join to earn money.
However since people are volunterily opting for the survery, they are doing it just out of interest and are more likely to be honest.
But the question is how many people are actually going to opt for the survey?
Thanks TC,, lets try,
where would omniture and company would go now
Their logo looks strangely familiar…anybody remember AudioGalaxy?
[ Logo here ]
Totally agree with comment #2.
For example on UnLtdWorld we have a platform called the Research Lab: http://www.unltdworld.com/lab (and featured here on TCUK: http://uk.techc...e-a-suggestion/ ) that provides insights into demographics (and more) dynamically (as a side not this is not for advertising, but to openly provide critical social insights to social entrepreneurs, relevant organisations and research).
I’ve implemented it. We’ll see what kind of metrics I can get, if any at all.
Yahoo!
Thanks for the invite. I’m in! Let me see if this thing really works!
This is what I got. Django seems to be quite popular these days
.
MOD_PYTHON ERROR
ProcessId: 24006
Interpreter: ‘westley_django’
ServerName: ‘app.crowdscience.com’
DocumentRoot: ‘/opt/csw/apache2/share/htdocs/westley’
URI: ‘/signin/’
Location: None
Directory: None
Filename: ‘/opt/csw/apache2/share/htdocs/westley/signin’
PathInfo: ‘/’
Phase: ‘PythonHandler’
Handler: ‘django.core.handlers.modpython’
Traceback (most recent call last):
File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 1537, in HandlerDispatch
default=default_handler, arg=req, silent=hlist.silent)
File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 1202, in _process_target
module = import_module(module_name, path=path)
File “/opt/csw/lib/python/site-packages/mod_python/importer.py”, line 304, in import_module
return __import__(module_name, {}, {}, ['*'])
ImportError: No module named django.core.handlers.modpython
It seems they are having issues over there. I added the JS to my test site and I am noticing 10,000 ms request times. Also, it seems like a lot of their links on the front page are missing.
For example: http://www.crow...e.com/overview/ is 404
Nevermind, link works now but JavaScript loading still slow.
It’s launch morning and response has been fantastic. We’re trying to bring in as many beta participants as possible so please be patient. We’ll keep bringing in more as we can.
Thanks all!
RabbleDabble.com has joined the Beta group and is trying this out in a trial period for evaluation. Seems like a great idea but we’ll have to wait til the end of the evaluation until we can report more!
Hi all,
I just wanted to chime in and let you know that we’re floored with the response, figuratively and literally.
I think we’ve got load under control now. (Joyent have been fantastic this morning responding quickly and bumping our capacity — thanks Linda and Blake!)
I also wanted to let you know that we have certain protections built into the system for when we run into load issues. Specifically, the initial request that our publisher websites make is for a tiny static file (called start.js) that gets served from a CDN. The request to our application servers is made after the page completes loading, so if it’s slow, as it was this morning for WatkinsCJ, then it shouldn’t have any adverse affect on visitors. Except the ones watching response times in Firebug.
Anyway, thanks again for the response, and we’ll keep everybody updated.
And thanks Jason for the coverage, much appreciated.
As we are all barraged with more and more information, contact requests, and “offers” my hypothesis is that this type of voluntary participation survey is at risk of measuring mostly the demographics of people who “don’t have a life” rather than a cross section of users.
Incentives might help, though they distort metrics in other ways.
Ultimately though I think we must move *way* past the survey stage, and work harder to measure the implications of the clickstream, time spent on pages, etc. Those are objective numbers that reflect what people really do rather than the more dubious what they say they do.
@John Martin, Great communication and transparency. Well Done.
Traditional marketing folk love demographics, but the new ones love implicit data. We’ll see…
@Martin Edic and others: agreed that these types of intercept surveys skew results. All forms of measurement and analysis do. Somewhat agree surveys are “going away;” that’ll take years.
But, as the tools to judge sentiment based on actual usage and interaction mature, why wouldn’t you avail of both data sets: looking at what someone tells you they want as well as they’re actual behavior should be at least twice as interesting as one or the other, no?
If you’ve got both available, why not avail of the belt and braces, too?
I can’t see how surveys will dwindle into obsolescence, whatever form they take going forward. Foregoing the development of magic, it’s my opinion that there is no way to realize the identities and attitudes of the traffic except to ASK them.
As for how that’s done – I don’t really like the incentives idea. After all, incentives are what started the decay of the information being provided by panels and other analysis software of old, so it seems to me like honesty (and therefore a higher value) can only be found in voluntary, un-coaxed responses. Crowd Science seems to have a handle on these things.
Of course, Crowd Science is going to be running into one major problem: people need to honestly take these surveys for the site to have any real value.
We used these types of surveys while I worked at Nielsen Online in New Zealand. The product is called Market Intelligence. The surveys are surprisingly accurate and do reflect the users on the sites. For example a site aimed at teenage girls does show a massive skew to teenage girls. Sites aimed at business and finance do show a large skew to male, educated, high income earners.
I was sceptical when I first saw the surveys to but the do work.
Of course, Crowd Science is going to be running into one major problem: people need to honestly take these surveys for the site to have any real value.
We used these types of surveys while I worked at Nielsen Online in New Zealand. The product is called Market Intelligence. The surveys are surprisingly accurate and do reflect the users on the sites. For example a site aimed at teenage girls does show a massive skew to teenage girls. Sites aimed at business and finance do show a large skew to male, educated, high income earners.
I was sceptical when I first saw the surveys to but the do work.
Forgot to mention, web-based semantic analysis seems too much like an overly complicated tech novelty, and “social media” is a really, really narrow platform for that, if you look at it closely. This seems to bridge to dynamic accuracy really well, anyways – there’s no indication that trends can outrun surveys, let alone lightweight ones like CS uses. Why make it more complicated than it has to be? Not only is more complexity an unnecessary hassle, but in this case, it seems like it would lose to simplicity outright.
Disclosure: This post is from Vizu, a web poll based audience characterization service mentioned in the original post.
Visitor demographics are valuable to webmasters and bloggers, not only for selling advertising, but also for developing content more suited to the audience.
How data is collected is also very important from both a data validity p.o.v. and a user experience p.o.v.
Data Validity:
Inviting people to participate in surveys via ads as described above is subject to significant bias. Reference Dave Morgan’s article “Outing the Heavy Clicker” on Mediapost which, among other things, states…
“Who are these “heavy clickers”? They are predominantly female, indexing at a rate almost double the male population. They are older. They are predominantly Midwesterners, with some concentrations in Mid-Atlantic States and in New England. What kinds of content do they like to view when they are on the Web? Not surprisingly, they look at sweepstakes far more than any other kind of content. Yes, these are the same people that tend to open direct mail and love to talk to telemarketers.”
Polls that are elegantly integrated into a site, like those offered by Vizu’s Power-Polls service, can “train” visitors to participate over time by featuring interesting, contextually relevant polls. As a result, they can generate much broader and higher participation rates than invitations to participate in surveys. The improved participation, in turn, can lead to higher quality data about site visitors.
All sections of your site are not equal. To really understand your audience, you may need to understand whether or not different demographics frequent different sections of your site. At Vizu, we have seen sites that have drastically different types of visitors on different sections of the site. Consequently, we designed our Power-Polls service to be able to assess demographics on various sections of a site separately and then roll all of the data into a top level view.
User Experience:
People love polls and generally are not big fans of surveys. Having worked with 1000’s of web publishers at Vizu, one of the concerns we heard loud and clear was that of maintaining a high quality visitor experience. Most publishers did not feel that taking their visitors off site to complete a lengthy survey for which there was no reward fit their conception of a positive user experience. On the other hand, many sites get very positive responses from their regular visitors regarding the addition of high quality polling content that offers the opportunity for instantaneous self expression and self assessment. Additionally, given the low click through rates for ads, many sites are not keen on allocating a lot of valuable real estate and impressions to solicitations to take surveys.
Ease of Use:
If you are going to go through the trouble to paste some code on your site to gather visitor demographics, why not paste some code that does more than just that solicit visitors to take a demographic survey? A good polling platform will help you engage your visitors and can improve your site experience. Sites can even opt into professionally produced, contextually relevant polls or rotate in your own polls which you can use as the basis for additional content.
All this is not to say that surveys do not have their place, it just depends what your over all objectives are.
Nice idea. I’ve plugged it in, though it can’t validate the script on my site, yet I can search for it just fine if I view the source.
Also, you can’t change the questions? I have questions specific to my audience I would like to ask, such as what motorcycle they own, if they plan on buying a new one soon, if they plan on buying new parts soon, etc.
Disclaimer: This post is from Crowd Science, and I promise to try and keep it short and sweet.
All forms of research whether they be polls, surveys, focus groups, or mall intercepts, inherit some form of bias. In part, poor research methodologies have lead to a ‘taint’ in perceptions to the industry. The Internet has been abused and not properly utilized when it comes to research.
Polls, surveys and all other forms are different beasts by nature. Each have it’s strengths and weaknesses and all are valid as long as the methods and results are used appropriately and interpreted with care.
Heavy clickers and professional survey takers exist, and many are trying to minimize their negative impact on polling, surveying, and advertising. We in fact are very careful that we integrate with a site successfully, remain sensitive to the user experience, and take pride in our adherence to sound research fundamentals.
Crowd Science Demographics also uses a zero-footprint approach. It takes up no permanent page real estate and can live along side any existing polls, advertising or other widgets. We simply only leave a small and polite trace on a small number of visitors. After all, it’s part of our research rigor.