Earlier this week in a post comparing real estate sites Trulia and Zillow, I suggested that the most important success factor for these sites is how comprehensive they are. The more listings the better because home buyers want to go to one place to find every home on the market. They want a single dashboard from which they can filter down the choices.
But just how comprehensive are these sites, and how accurate are their listings? Trulia offers 3.5 million listings nationwide, and Zillow has 3.1 million. But people look for houses in local markets, not nationwide. What matters is comprehensiveness withing local market
A few hours after I put up that post, I heard from Roost, a competitor to both Trulia and Zillow (with only 1.4 million listings in the markets it covers) that differentiates itself by getting its data directly from the same Multiple Listing Services (MLSs) that real estate brokers use. They just happened to have study in their back pocket (which they commissioned and paid for) that compares the “accuracy” of search results in three cities (Dallas, Miami, and San Diego) across different real-estate search services (Roost, Zillow, Trulia, Yahoo, and Google). The study, which was done by real-estate industry consultants the WAV Group, defines accuracy as the percentage of listing results that match listings the MLS for that city.
The results are in the chart above and, not surprisingly, Roost comes out looking great. For each city, it returns between 95 and 99 percent of the listings in the MLS. Trulia’s accuracy in the study ranges between a pitiful 9 percent for San Diego to 61 percent for Miami. (Zillow generally does worse across the board, with its accuracy ranging between 12 percent and 36 percent across the three cities).
Trulia disputes these results. Heather Fernandez, vice president of marketing, says:
The data looks very questionable, and not in line with our internal coverage data. Our data shows that we have roughly 70% coverage in most major metros.
And indeed, if you do a search for homes for sale in San Diego on Trulia, you get 4,395 results, compared to 6,036 on Roost. That’s 73 percent. (Zillow claims 7,661 listings in the San Diego city limits). Even if half of them are stale listings or not accurate in some other way, it’s hard to get to the 9 percent that the Roost-financed study claims. That’s because for some reason, the WAV study only compared homes in each city with exactly 3 bedrooms and 2 baths, within a $50,000 price range.
That methodology seems random and flawed to me. Would Trulia’s accuracy be greater if the study had looked at homes in San Diego that cost $400,000 to $450,000 instead of $300,000 to $350,000? Comprehensiveness would have been a virtue in the study’s methodology as well as in what it was trying to measure.
Still 70 percent accuracy is not that great, and it doesn’t seem like Zillow is any better. If the MLS in any given city is the benchmark, both have a lot of work to do. And Trulia, for one, is striking deals with different MLSs to incorporate their data. But it only has 14 so far, out of about 900 nationwide. MLS-based sites like Roost and Redfin may have more listings in the markets they serve, but they don’t serve every market yet. For instance, in San Diego, Redfin tracks 6,300 homes for sale, better even than Roost. Yet neither has any listings in New York, and Redfin only has 473,000 listings total.
Yet Redfin CEO Glenn Kelman also balked at my earlier suggestion that either Trulia or Zillow are even close to comprehensive within any given market. In an e-mail to me, he said:
What got me was that almost any real estate site has more homes for sale than Trulia or Zillow.
Frenandez doesn’t think that having the most listings matters. She responds:
Listings are commoditized –there are dozens of sites that offer basic listing information in every city across the country. It’s not a competitive advantage for an Internet company.
What is more important, she says, is the filtering the site allows home buyers to do to help them make an informed decision. I’d say it’s both. Those filtering tools (heat maps, sales comps, local school info) are also becoming commodities. You want to cast your net as wide as possible before you filter down so you don’t miss out on that one house that fits all of your criteria.