By now we’ve all likely seen the new “report spam” links on each Twitter user’s profile. Twitter rolled out that feature on twitter.com yesterday, and plans to roll out an API, so third-party apps can use it, sometime today. But Twitter has also quietly rolled out another new feature aimed at cleaning up the stream: Blocking duplicate tweets.
If you try to enter text for a tweet that is exactly the same as a tweet you’ve recently sent, it will block it, saying “duplicate text” in a white drop down overlay (pictured below). It would appear that there is a time window for this, and it may be 24-hours. I tried copying-and-pasting two tweets that I sent a few hours ago, and those were both blocked. But then I tried one from two days ago, and it went through.
Obviously, it would seem the idea here it to stop Twitter users from spamming their followers with multiple tweets. But not everyone is happy about this. As one user wrote to us, “I often send the same tweet out 3 times in a 24 hour period, to try and cope with different time zones.” Luckily, Twitter is not scanning links to see if they’ve already been tweeted by you, it just seems to be looking for an exact match of the text, so slightly altering your tweet’s text will work.
Duplicates (or “dupes”) have long been a problem on other social sites like Digg. That service does scan links to see if they’ve already been submitted and lets you know before accepting the link. Of course, there are plenty of workarounds, which is why we still see Digg’s comment sections flamed with “Dupe!” For Twitter to start blocking duplicate links would be silly, since that’s one of the most compelling uses of the site. People find a link they like and then retweet it.
[thanks Andrew]
Is this a new feature? It’s been the default behaviour of the API for quite a while, if I’m not mistaken.
Actually, in the past Twitter has only blocked duplicate tweets posted back-to-back. Now you can’t post exactly the same tweet more than once in an (apparent) 24-hour period unless you slightly tweak the text.
Thanks for the clarification! That’s pretty bad and apparently creating a lot of confusion among (my) Twitter client users.
Seems as if another “feature” was killed, too: long tweets with 141 chars or more are silently dumped now. No truncation or URL shortening anymore.
Might be a bug, though.
finally they are doing something about all the spam and dupe tweets. Those are more than annoying.
Not a real solution since i have seen plenty of spammers use random word generators and random text pulled from the internet.
They should really check for dupe URLs from the same user in a given time. And not only ban the user but also that URL.
Careful, you may be suggesting untenable changes to their business model.
Also keep in mind that blocking duplicate url’s in order to stop spammers won’t work, as they can submit the same url to url shortning services many times, and each time they will get a different url, so they will end up sending different urls pointing to the same address and twitter won’t be able to catch them.
I believe twitter would rather trust its users to report someone as spammer. And those with high spam rate can be blocked or their tweets can be throttled
i know someone who owns about 6,000 twitter spam accounts. spam accounts with about 90,000 message every 80 minutes. its pretty funny watching people say twitter is “teh most revoluntionary site ever” which is just a site to look at spam
Why would people do that?
Last time i checked all twitter links had the ref=”nofollow”, telling crawlers to not give the link extra reputation.
Just add a space.
Doesn’t that sort of defeat the purpose of retweeting?
No. It doesn’t count other users’ tweets against you.
banning the user and their links would be nice. I am sick of those bri.tney fuc.ked accts!!!! >_<
At least they’ve made it (ever so slightly) easier to block new followers. I’m still waiting for a setting to allow follower moderation only.
Awesome photo choice.
+1
actually +2, she is just that hot
Is she anyone we know?
> As one user wrote to us, “I often send the same tweet out 3 times in a 24 hour period, to try and cope with different time zones.”
Dear whomever does this: Stop it, stop it right now. There are people who use Twitter a lot, I check my stream every hour or so and if you do this I’ll be seeing your tweet however many times you tweet it out. If someone follows so many people that your tweet gets lost then tough.
You really check your Twitter every hour, 24 hours a day?
I think you’re the one who should be banned, for your own sake of course
I stand by my quote.
I write celebrity gossip, which i know is a minefield of rehashed ‘gossip’, but i write about celebrities from different parts of the world, i have found that 3 tweets in 24 hours is fine with most people (my followers at least), but at the same time, news i wrote at 9am GMT also gets a Tweet at 9am from a timezone which is 12 hours in front of my location.
Im not saying it will work for everyone, but judging by my analytics, it works very well for me.
But if it’s not happening “right now,” that kind of defeats the purpose of Twitter.
At the very least, phrase your tweets for other timezones as recaps. CNN puts “Previously Recorded” on their 3 AM rerun of the evening news, so it makes sense to do the same for a Twitter stream.
I think what Zacqary Adam Green has suggested is actually quite a good idea, and one ill use from now on.
Thanks for the tip sir
I agree with Samuel Ryan on this. It bugs me when people do this. I follow 500 or so people and I read all their tweets. Waking up in the morning I read through the ones made while I was asleep and when I’m AFK I catch up when I get back.
If someone’s following too many people to read it, then they won’t give a shit about what you’re saying anyway.
Great article and bonus points for using Grace Park photos from Battlestar Galactica
So happy to see there are some BSG fans at TechCrunch. Very cool. Oh yeah, the twitter thing is cool too. ha!
yeah. i miss BSG even if it did tail off in the end.
This isn’t a new feature, its been around for awhile.
I see its only been implemented today though.
I have been sending the same tweet out 3 times a in its 24 hour period for a long time, and its only started to get blocked today. (within the last few hours actually.)
Yeah, I think it was just turned on today, before today I had no issues sending out multiple tweets in duplicate.
I think that this solution will not solve the problem.
If you think that someone is spamming just un-follow him.
+1
I don’t understand why people are even talking about spammers and their links. If that’s a concern, you have the twitter-given right to unfollow them.
But what is Guy Kawasaki going to do?
He repeat tweets 3x (8 hours apart)
http://holykaw....he-repeat-tweet
Maybe he’ll stop being an attention whore. WTF, nobody reposts old blog posts because there’s a chance they weren’t read by some dude smoking a peace pipe half-way across the world.
He does love it too much…
These features being made available recently should have been introduced from the offset! Dupes, Lists etc are standard and probably the reason that there is such a considerable dropoff after the initial few tweets. It’s likely that the majority of users just don’t get the functionality, API and interaction through third party clients and are comfortable with Facebook!
cool
It seems to be working, there was this spammer who was sending me messages offering to sell me a Google Wave invite. He found me because I was tweeting that I got my invite finally. (Silly bots)
Happy to see the quick response from Twitter, he was spamming a lot of people today with it.
Thank you for linking me. Much appreciated
I just got a big ol BSG boner! Great analagous pic to twitter redundancy – #8 looks like an infinity sign, and since there are many #8 cylons that were killed when Boomer set off a nuke on that very Baseship, the analogy just works on so many levels. Bravo.
For those who are complaining…just unfollow! If you’re REALLY annoyed, block the person whose tweets you don’t want to see. No one’s forcing you to read anyone’s tweets except those you choose.
i once did a twitter spambot that never tweeted the same things twice, just random words along with the current trends and a link. took days for the account to be banned. And, it was surprisingly effective ; tweeters literally click on anything.
How often were you tweeting with this?
~0.5 tweets per minute with a trending topic in the content. each spam tweet gets ~10 clicks. i was just curious how effective twitter spam is.
Unless they’re doing some fuzzy hashing on tweets, this is completely pointless to stop all but the most inept spammers/reposters. Furthermore, checking for dupe URLs (from the same user) is similarly naive given how easy generating multiple URLs for the same content is. Ideally they’d actually follow the links (and all redirects) and do a fuzzy hash on the content, and block if the content were similar enough to some recently posted link by that user.
I don’t like it. More regulation isn’t a good thing. If you don’t like some tweating something more than once, don’t follow them. But I doubt that most people ever notice cause they are only logged in for so long and miss 90% of the tweets.
This happened to me today (a ‘duplicate post’ blocked). This policy is lame because many Twitter users (myself included) tweet the same thing fairly regularly. The restriction seems to go back beyond 24 hours.
I believe the Twitter Staff believes this is a win win situation, since it would help cut down on the spam from twitter users and would save/slow down the enormous amount of data they have to collect, I mean who wants a database with 1k tweets of the same thing over and over, not too targeted there if they have plans to sell the data.
Wow, only 3 years too late.
I feel it should have started doing this earlier .. but its better to be late trhen never.. finally I feel twitter may become a better place…
Best,
Daina Thomas
Just alter the title text, slightly, and continue to post! Until they figure that out!
Umm why not just have a REPUTATION that gets affected by duplication?
When following someone you can see their stats including DUPLICATION frequency
Yea they didn’t think this threw. I update sports scores and depending where you are time zone wise I need to resend the same tweet. Who is effected by the spam? Don’t follow them then. It’s more a way to cut down on tweets so Twitter doesn’t fail every five minutes.