uTest, a startup that allows companies to outsource their QA testing to ‘the cloud’ has just concluded its latest quarterly bug battle, during which it put some of the world’s largest social networks to the test. Hundreds of participants (many of which have been involved in product testing for over a year) did their best to uncover flaws across Facebook, MySpace, and LinkedIn, with $3,000 up for grabs for the testers who identified the most crucial bugs.

Below are the final results for the number of bugs found. It’s worth noting that this data is prone to bias and may well overstate the number of ’showstopping bugs’ (testers probably had a strong incentive to rate their bugs as ’showstopper’ so as to have a better chance at the prize). But it also meshes fairly well with anecdotal experience.

• Facebook: 416 uTesters found 243 bugs (.58 bugs/tester), including 14.40% that were deemed
  showstoppers by the tester
• LinkedIn: 399 uTesters found 250 bugs (.63 bugs/tester), including 9.78% that were deemed
  showstoppers by the tester.
• MySpace: 304 uTesters found 225 bugs (.74 bugs/tester), including 10.80% that were deemed
  showstoppers by the tester

Aside from logging each bug that they found, testers were also asked to rate each social network in a handful of categories and to detail some of the most apparent issues. Overall, the contest deemed Facebook to have the most complete feature set, but the social network was criticized for not having a high enough default security level (the options to keep things private are there, but are not always set by default).

LinkedIn took the title of “Best in Overall Quality”, but testers had issues syncing data with Outlook, which they considered “problematic for the ’professional social network’”

MySpace didn’t take the top prize in any of the bug testing categories, but was considered to have “a loyal following”, as it drew consistently high marks from a subset of users. Common complaints included “Overlapping images, backgrounds, titles, etc.” and slow page load times.

Testers’ choice for best overall quality: 
1.  LinkedIn (45%) 
2.  Facebook (37%)   
3.  MySpace (17%) 
 
Testers’ choice for best usability:  
1.  Facebook (39%) 
2.  LinkedIn (38%) 
3.  MySpace (17%) 
 
Testers’ choice for best feature set: 
1.  Facebook (46%) 
2.  LinkedIn (36%) 
3.  MySpace (18%)

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

uTest, the crowdsourcing QA startup that lets companies rely on external developers to help them identify bugs, has closed a $5 million Series B funding round led by Longworth Venture Partners and Egan-Managed Capital, along with existing investors Mesco Ltd. and Massachusetts Technology Development Corp. The new round brings the startup’s total funding to around $7.8 million after a $2.3 million Series A round last year and some early seed funding.

The site offers customers a web based platform and tools for monitoring testing and QA cycles, which are available to them free of charge (they only pay for the testing completed by the crowdsourced community). Community members are paid depending on the number and type of bugs they find, and the marketprice for bug finds fluctuates depending on the number of bugs left to find, the demand for testers, and other criteria. Since launching in Febuary, the uTest community has grown to 11,000 users.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

The startup is trying a crowdsourcing approach to testing software bugs. Anyone can sign up to test software and make some cash. uTest estimates that its testers will be able to rake in anywhere from a few hundred to a few thousand dollars per month, depending on tester-expertise and bug pricing.

It is important to note that bug prices will fluctuate in real-time based on a variety of parameters, including: Bug type (logical, GUI), type of application (Web, desktop), number of testers that fit the required profile for the testing environment, bugs left to find, and more.

Over 2000 testers from around the world have already signed-up, so it seems the company’s pay-per-bug model is resonating well across testing professionals.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

Boston-based uTest has raised $1.7M to bring to market a crowd-sourced quality assurance (QA) marketplace and community. The service has begun recruiting testers in anticipation of its official launch, expected in early 2008.

No fuzzy logic here; uTest presents clear and communicable value propositions: For companies in need of QA, uTest is providing an on-demand environment for the management of full testing cycles. For testers, uTest is providing an oDesk-like marketplace through which they can be hired and paid on a Pay-per-Bug basis.

The company’s founders recognized that QA and usability testing solutions are inefficient. QA departments are either under-utilized when waiting for versions to test, or over-extended when a new version comes out. To most SMBs, QA outsourcing is neither cost effective nor available “on-demand”. Then, of course, there are cost-cutting measures that leave companies unable to sufficiently fund QA efforts, ultimately shipping buggy applications to users like us.

The uTest testing platform itself is entirely Web-based and provides the management of complete QA cycles, from creating/loading test scripts, to selecting the target community (profile and environment) of testers required. The platform also provides test case management tools, real-time information on bugs and defects logged, statistical information on release maturity level, as well as QA coverage and market readiness. Support for bug tracking systems includes Bugzilla, Jira, FogBugs and will be expanded over time.

In a smart business move, uTest will not charge companies to use their platform to manage testing and QA cycles—this will be entirely free of charge. uTest will only charge for the services provided by the community of testers.

The uTest offering makes a lot of sense to me and I expect it will be warmly embraced for several reasons. First, consider for a moment that for many SMBs, QA management solutions such as those offered by Mercury or IBM-Rational are beyond their reach. Speaking from personal experience, the majority of the testing cycles I’ve seen have been “managed” on Word or Excel. The “on-demand” model which has been proven time and time again on the Web from CRM to ERP, should work just as well for QA. The fact that the platform will be offered free of charge, pushes the offering to “no-brainer land”.

Second, beyond the manageability aspects of uTest’s service, companies will obviously enjoy the cost saving aspects of paying testers by the bug. It’s this aspect that in my opinion will make uTest’s offering relevant not only to SMBs but to large organizations as well. Logic would dictate that if they embraced off-shoring and near-shoring, crowd-sourced QA shouldn’t be too jagged of a pill to swallow.

Third, recruiting a userbase of testers should not be difficult. There are droves of potential testers in countries such as India, China, Russia, Bulgaria, Estonia, etc. Also, getting hired through sites like oDesk, Elance and RentACoder is becoming increasingly difficult due to the growing number of service providers. These same individuals can theoretically provide testing services instead of programming.

All-in-all uTest sounds quite promising and has earned a place on my companies to watch in 2008 list.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.