uTest, a startup that allows companies to outsource their QA testing to ‘the cloud’ has just concluded its latest quarterly bug battle, during which it put some of the world’s largest social networks to the test. Hundreds of participants (many of which have been involved in product testing for over a year) did their best to uncover flaws across Facebook, MySpace, and LinkedIn, with $3,000 up for grabs for the testers who identified the most crucial bugs.
Below are the final results for the number of bugs found. It’s worth noting that this data is prone to bias and may well overstate the number of ‘showstopping bugs’ (testers probably had a strong incentive to rate their bugs as ‘showstopper’ so as to have a better chance at the prize). But it also meshes fairly well with anecdotal experience.
- Facebook: 416 uTesters found 243 bugs (.58 bugs/tester), including 14.40% that were deemed
showstoppers by the tester - LinkedIn: 399 uTesters found 250 bugs (.63 bugs/tester), including 9.78% that were deemed
showstoppers by the tester. - MySpace: 304 uTesters found 225 bugs (.74 bugs/tester), including 10.80% that were deemed
showstoppers by the tester
Aside from logging each bug that they found, testers were also asked to rate each social network in a handful of categories and to detail some of the most apparent issues. Overall, the contest deemed Facebook to have the most complete feature set, but the social network was criticized for not having a high enough default security level (the options to keep things private are there, but are not always set by default).
LinkedIn took the title of “Best in Overall Quality”, but testers had issues syncing data with Outlook, which they considered “problematic for the ‘professional social network'”
MySpace didn’t take the top prize in any of the bug testing categories, but was considered to have “a loyal following”, as it drew consistently high marks from a subset of users. Common complaints included “Overlapping images, backgrounds, titles, etc.” and slow page load times.
Testers’ choice for best overall quality:
1. LinkedIn (45%)
2. Facebook (37%)
3. MySpace (17%)
Testers’ choice for best usability:
1. Facebook (39%)
2. LinkedIn (38%)
3. MySpace (17%)
Testers’ choice for best feature set:
1. Facebook (46%)
2. LinkedIn (36%)
3. MySpace (18%)