This is a guest post by Nabeel Hyatt, Founder and CEO of Conduit Labs, which is the creator of Loudcrowd and other social games that help you experience music with your friends. His personal blog can be found at nabeelhyatt.com and he can be followed on Twitter @nabeel.
Yesterday, Facebook announced they are going to drastically alter the way applications can message users once again, likely throwing a wrench into every app developers’ growth rate. Hints of the coming turmoil appeared last week when Facebook changed the way feeds work. This caused enough worry that apparently Mark Pincus, Founder/CEO of Zynga, canceled his appearance at Harvard Business School so he could sit with his team and figure out what the impact would be to the viral rates of their massive hits such as Farmville and Cafe World. That’s not surprising, since getting posts in the feed is critical to continued growth, but the myopic focus on the “viral rate” by some in the industry has created an over-dependence on perhaps the wrong number.
(As an aside, those who complain about Facebook being an unreliable channel to build a business off of should try dealing with a retailer such as Best Buy. They will be happy to take a 50% cut of your revenue and then one week decide to eliminate your entire section of products and ship them back to you. Oh, and they’ll bill you for the shipping.)
In all the talks about virality the general focus is on the new, clever, and sometimes underhanded ways to increase your viral rate. What gets lost is the core message that, as Siqi Chen of Serious Business puts it, viral messaging tactics are just a force multiplier on the inherent viralness of your product. Or, in simpler terms, how good your product is in the eyes of your users actually is the most important thing. Viral messaging is just a way of greasing the skids on that user’s intent. It is an important later step, but not the root.
And how do we measure that intent? It turns out that in Facebook at least, the level of retention is the best public number to predict likelihood of a hit.
Real retention numbers for other people’s products are notoriously hard to come by, but in Facebook there is good 30 day retention data called the DAU/MAU Ratio – which can also be called Stickiness. This is the ratio of Daily Active Users to Monthly Active Users. For example, a DAU/MAU ratio of 50% would mean that the average user of your app is using it 15 out of 30 days that month.
It turns out this simple metric is enough to predict, with a high level of probability, the success of a product. For example, look at the correlation between the following set of Facebook games.
Here we have games that fit a broad set of criteria, in terms of brand association, demographic appeal, play style, and time since launch. See how the second column and the fourth column are almost perfectly in order? Despite this broad cross-section of games it appears there is a very direct correlation between stickiness and success. Let’s take a deeper view of the data.
For those who aren’t stat geeks, this is called the coefficient of determination (R2) and predicts whether two correlated data sets accurately predict future success. If everything lined up perfectly on the linear regression line above you would have an R2 value of 1, and then we could say there is a perfect correlation between Stickiness (x axis) and the Size of the app (y axis). Using these social games we have a rather astonishing “fit” of 0.77.
Stepping back, this data is quite remarkable actually, since you would expect Stickiness to go down as you get huge. It would stand to reason that your 20 millionth user, who might be experiencing their first Facebook game, is going to be harder to retain than your 1,000th. The fact that this is not happening yet, that no one has found the edge of users where suddenly retention metrics collapse, says something very powerful indeed about the potential size of the social gaming market.
But it isn’t the market size implications that are the big takeaway, the relationship between Stickiness and success is. This of course will also cause a messaging arms race around retention, and the prevalence of Free Gifts is a good example of that already happening. But just as with the viral rate it’s important to remember those are all simply tactics to enhance a products inherent retention. The clear inference is that building something users love to come back to is the best predictor of success.
How are you getting DAU data?
There are several services that provide DAU data, both public and private. Appdata and Allfacebook would be two I would recommend.
From the small sample in the graph it looks more like a curved fit line would suit, an exponential correlation. Which matches the expectations of a network effect in these games, which is why the ‘viral effect’ is so critical.
However the sample is far too small to say much.
“this is called the coefficient of determination (R2) and predicts whether two correlated data sets accurately predict future success.”
Absolutely not !! R2 doesn’t tell you if prediction is accurate. And I am not even a stat geek. :-/
R2 is a measure of how well the regression line fits the existing data. It is not a measure of the predictive quality of the regression equation. (Ref: Applied Regression Analysis – Dielman)
Your right, r2 doesn’t tell you if a prediction is accurate. It tells you the probability that those data sets predict a future outcome.
To be more specific (and more stat-wonky than I wanted to be for this article) it is referred to as a “conditional probability distribution.”
Wikipedia actually has good overview articles on these subjects for those interested, I would suggest starting here and going as deep as you like:
http://en.wikip...f_determination
The difference in growth between DAUs and MAUs for Cafe World indicates that it’s stickiness is falling sharply. Unlike Farmville’s which is growing…
The first set of Cafe World numbers have a big caveat, that was their first week of public launch so they are going to be an aberration. Since they point they seem to have stabilized to a very respectable number.
If you are interested in trends of Stickiness data, Eric von Coelln wrote a great post here:
http://www.insi...gaming-success/
In the last 2 weeks CW has grown MAU by 47% and DAU by 21% according to AppData.
While it is too early to tell, I think CW’s higher burn out rate will cause it to have less stickiness than Farmville.
Nabeel:
This article couldn’t be more timely or interesting. I’ve been thinking a lot about the metric that best captures virality and retention, and I think this is the best I’ve seen so far.
Brill.
-Gabe
hmm.. seems this comment system automatically rejects contrarian comments
Interesting data, but the conclusion is backwards.
My original comment isn’t displaying (yet) so I’ve posted it here:
http://insomani...as-it-backwards
yeah right. way to try to draw attention to your lil tool. how’s it some ppl think this is the right target audience for circus tricks?! why not post to the nurses’ crunch (without offense to the nurses, but they really wouldn’t know the intricacies of dau/mau).
oh btw, ur argument about inverse correlation is a bit empty – which just proves it was never about that argument.
Great post! Super informative, fact based and relevant to what I (and probably most people) expect of TC. Kudos!
Those most be pretty popular applications. I know because I have them both blocked.
I agree on the importance of stickiness. But, the analysis in this piece seems to have missed something important. Yes, DAU/MAU is a measure of how often someone comes back to an app.
BUT, it is also a measure of the growth rate of the app. When you have lots of new users DAU/MAU will be high, whether or not those users ever use the app more than once.
Look at the Sep 22 DAU/MAU data for Cafe World in the table. Since the app had just launched, DAU/MAU is near unity. That doesn’t mean the app is sticky, just that most of the users are new (ie, high growth rate).
Eric, you are absolutely correct, if you’re viral rate is phenomenally high, or you are in a state of unprecedented ad spending, it will alter your DAU/MAU ratio. Cafe World shows this, and the first 6-8 weeks of Farmville showed this as well.
It felt like a little too much detail to cover in this brief piece because those are generally not sustainable ratios. Maintaining an incredibly high viral rate for any length of time is generally not possible. And indeed, you are already seeing Cafe World settle into what I suspect is their “natural” ratio.
It is hard to boil any one stat down, but to be clear DAU/MAU ratio is a derivative of retention and customer acquisition. As with many things, there are many ways to skin the cat.
I cant understand why facebook is trying to make the app developers life harder when those application are the main reason that facebook is still alive.
Facebook will have no revenue without those apps
Interesting focus on social media games. It would be interesting to see how other social networks like linkedin posted articles and Wadja.com #labels increase stickiness on their sites. It seems to me these two examples do so by creating real value to the user and therefore value to the Social network which see a very high level of loyal and repeat users.