Real-Time Twitter Search, Hold The “Real-Time”
by MG Siegler on May 31, 2009

Many believe the greatest potential of Twitter lies in its ability to perform real-time searches of various keywords. So when that functionality is delayed by some 3 hours, as it is right now, and has been throughout much of the night, with no explanation, you can imagine that users are going to get a little annoyed.

Go ahead, search for anything right now — a good example is for the word “the,” as it’s used in a ton of tweets. The most recent results you’ll find are from 3 hours ago. [Update below, it's back with a huge gaping hole.] Not only does this badly impact my vanity searches, but there are companies who now rely on Twitter Search to run services such as brand management. Imagine the horror Comcast must be feeling right now not being able to see my tweets constantly bitching about their crap service in real-time.

Likewise, Trending Topics is not working as it also relies on Twitter Search. So we’re being tricked into thinking people actually care about the MTV Movie Awards.

We’ve gotten tipped this a number of times throughout the day, but I’ve largely been ignoring it, trying to give Twitter the benefit of the doubt to at least update us on what is going on. But this is ridiculous. Fix your damn search functionality Twitter, you’re not much use without it.

aa

Update: And it’s back up — with a nice little 4 hour gaping hole of tweets not indexed. Go ahead and try this query and if you go back far enough, you’ll see that it all of a sudden jumps back 4 hours at one point. All those tweets, apparently, lost.

Advertisement

Comments rss icon

  • I had a feeling this was going to happen when they turned it on for everyone by dropping it in the sidebar. Why release sitewide without proper testing? Kinda confusing, but oh well.

  • Meh, tekkie QQ.

    It’s good enough. You’re spoiled.

  • Zero word from them. I do tons of searches a day on stuff and this blows. LAME

  • I think putting it on the homepage just opened it up to the average Joe and this started to kill their servers. It’s also possible that there are way more tweets than before, which makes indexing choke.

  • @Drew, I think they did the testing (some users, including myself had the sidebar search before it was rolled out.)

    Perhaps it is a hardware malfunction and no one is around on a Sunday evening to fix it? I give the Twitter folks the benefit of the doubt but I agree this is something that should be fixed as soon as they can.

  • Yes, i find the search very important and if that goes down, its difficult to connect. I think it is an integral part of the whole system.

  • Google Wave has the potential to destroy Twitter. Knowing Google they won’t have any of these problems. Twitter should sell before that happens.

    They really should’ve sold during the Oprah hype. Hope it works out for them thoughs since they worked so hard.

  • There’s another problem i’ve just encountered…I am unable to unfollow folks whom i have been following so far….try it yourself…

  • Johnnie Twotten - May 31st, 2009 at 10:56 pm PDT

    You can still search at least one subset of Twitter: http://friendfe...rvice%3Atwitter

  • Finally! We now know how Twitter will monetize. Twitter service with delayed search results free, “real-time” search results will be a small charge:)

  • He Siegler, you seem to have missed one more issue. Check http://twitter.com/TimMoore, the last tweet is shown as 3 hours ago and now open http://search.t...rch?q=tweetknot the same tweet by timmoore is shown as 6 hours old. Is it actual search issue or just a bug in search where they are showing time as time+ 3hours

  • Best line in the post.

    “…..Twitter, you’re not much use without it.”

    It didn’t occur to me that this was a problem because I encountered this earlier. I did a search for Nadal to see which of his fans where crying and I was surprised given the magnitude of what happened, that people had stopped tweeting about it.

  • I noticed this happening while using Twitter Search during the MTV Movie Awards. I kept trying to refresh and nada. I went searching for other tools to see the updated feed, but fell out of the conversation so much that I gave up. So much for broadcasting live Twitter feeds and not being able to keep up with the conversation in one place.

  • Perhaps it’s because the MTV Movie Awards will appear at different times across the country and Twitter couldn’t just delay search functionality for MTV Movie Awards related stuff, so they delayed everything …

  • Oh good, at least I’m not going crazy. I noticed this a couple hours ago when the conversation about Google Wave suddenly stopped dead. I thought maybe that since the conversation had turned in earnest into how Wave is going to snuff Twitter, that they had blocked the keyword. Then I thought maybe it was blocked because of spammers.

    I searched Twitter status and could not find a single word about it.

    Twitter should take a lesson from Flickr. When Flickr has problems they are immediately up front about it and keep people informed, sometimes every five minutes, as to the progress on fixing it.

    And what about the other day when Twitter did a scheduled downtime in the middle of the day in the U.S., where the majority of their users are? How lame is that? I’ve been doing this since 1968 and scheduled computer maintenance is always done in the middle of the night. It’s inconvenient but it’s part of the job.

    Well, I hope they get it together. Twitter is a great concept and extremely useful.

    • They didn’t want to have their developers come in on the weekends. At least that was the explanation given as to why the API wasn’t properly posting where a tweet came from, and won’t until Tuesday morning.

      No matter how popular your site is, you cannot afford black-eyes like this. Spend some of that money – someone is willing to come in on a Sunday to fix problems.

      • you’re a tool. do you have any idea what a bitch it is to index that much data? or do you just make it a habit to talk out your ass all the time?

        • Are you kidding me? The API broke because the database file for source applications exceeded 1MB. Yeah, it’s a total bitch to manage a 1MB file.

          • hey, everybody! i’ve got this FANTASTIC idea! why don’t we just take a bunch of developers and have them work on WEEKENDS! because everybody knows, tired developers produce MUCH better code, right? hey, when i solve complex math problems, i LOVE to come into work on a weekend and do it!

            or better yet, why don’t you just throw MORE developers at the problem??? that’ll show that pesky problem a thing or two!

            ummm, yeah no. try reading the mythical man month. this approach has already been documented for the past 40 years.

        • And as far as the stream goes, they keep 3-4 weeks of tweets stored already. Is it ‘talking out my ass’ to imagine that a company worth $1B might have a redundancy plan to keep 4 hours worth of tweets? What about the developers who rely on that data for their applications?

          My point is, it’s in the companies best interest not to lose 4 hours of data. Unless you want to argue that it’s a good thing to lose data, move on and try to show-up someone else.

          • they’re not LOSING data, they’re INDEXING that data. when you come up with a fantastic architecture for indexing a bunch of data real time, let me know. love to see it.

  • understand, sometime I am just a bitch about twitter, most of the time though, I am the lover!

  • Oh don’t be so dramatic MG…

  • I still find it funny that they’re not indexing what goes on perpetually. That 4 hour block of time should not be gone – sure maybe search servers were behind – but you’re telling me they can just lose 4 hours of data?

    If they aren’t indexing and storing every single tweet that goes through their network, they must have missed the memo about how big real-time search is.

    This boggles my mind. And when little things like changing your avatar don’t work for weeks, it makes me wonder what they’re spending their time doing?

    You have a billion dollar baby. Change it’s damn diapers and teach it some tricks! This sort of stuff will not be tolerated forever. Free passes are running out.

  • Wow, maybe they read my blog (hmm, maybe not!) about how they could generate money by adding a delay to their search!

    http://danzambo...diction-markets

  • The only SLA that customers of a free service can rightly expect on a Sunday night is no SL :-)

  • At last!
    Stop using the ‘real-time’ s..t altogether, there’s NOTHING real-time there.

  • … and CAN NOT be, btw.

  • Twitter should have cashed out already!

  • That’s funny, because from where I’m standing, Twitter Search is up-to-date and this post is more than 12 hours old.

  • twitter = next myspace

  • I noticed this a couple of weeks ago, actually. It was about 1 hour behind for several hours. I asked around, and other people had noticed it, too.

  • Will this Titter search give any information, like Google search? I don’t understand what *is* there to search *in* Twitter?

  • Good catch and humorous, timely post. I had noticed the delay in just watching new posts come up at an event. 3-4 hours lag is unacceptable given that there are a lot of new startup search engines that will outperform Twitter’s Summize. As they say, competition is good so hopefully this reality will light a fire under Twitter to fine-tune its search capabilities so they are not so dated.

  • If real time is what it says to be it will revolutionize search and be a marketer’s dream and could lead to huge advertising revenue for Twitter.

  • “Fix your damn search functionality Twitter, you’re not much use without it.”

    Thank God you were here to let them know what needed to be done. Otherwise search might still be down!

  • I feel their pain. So their index engine might have crashed, an alert went out, the person in charge was away and couldn’t get it restarted for four hours. They probably haven’t lost the tweets (in the DB), just the entries within the index.

    They can probably rebuild the index. Problem is, that might take days – literally (imagine rebuilding a massive index while keeping it up to date at the same time = very slow). So a four hour gap is an acceptable loss for a couple days until the index is rebuilt.

  • Twitter search not being real-time is one thing – its archive only going back a couple of days is another thing – I’ve been watching the backlog getting smaller week by week while working on twitter-friends.com.
    Google wouldn’t even have to rebuild a Twitter-like application. Indexing all Tweets (and dents and Buschfunk etc.) and building a loooong backlog could be enough? But, I’m not so sure this will happen. Google’s blogsearch seems to be experiencing the same problems as Twitter’s search from time to time.

  • Twitter is a nice way to promote business I guess:)

  • Google’s blogsearch seems to be experiencing the same problems as Twitter’s search from time to time.

  • I still think there is a lot great keywords that are highly searched but people don’t put down the whole keyword due the limited tweet box. I know this months old post but seems to me twitter has the slow periods don’t know from volume or that last hacker attack.

Leave Comment

Commenting Options

Enter your personal information to the left, or sign in with your Facebook account by clicking the button below.

Alternatively, you can create an avatar that will appear whenever you leave a comment on a Gravatar-enabled blog.

Trackback URL
Short URL
bugbugbug