It turns out the battle for control of Twitter rests almost exclusively in the unique value proposition of XMPP-served track. As Twitter strips away various features of its service to rebuild a scalable fail-whale -proof version, the one remaining hurdle is restoration of a fully-functional Track over IM.

For the last two weeks, a one-way IM service via Gchat inside Gmail or Gtalk standalone has provided a stream of tweets but not the previously enabled ability to post back to Twitter via the IM window. In addition, there is no support for the Track function, which interweaves Tweets from any endpoint on the Twitter network that correspond to the keywords you “track” on. Track was briefly available over SMS several weeks ago, but was then withdrawn.

Continue reading on TechcrunchIT >>

As Bill Gates closed the door for the final time Friday on his ex-office (Ballmer takes over Monday) the rhetoric about continued one day a week doesn’t match the reality. Whether you believe Bill will have an ongoing role in Office and Windows futures, I bet most of Bill’s input is already factored in by the owners of those two dominant sources of Microsoft revenue.

What comes next depends on whether Microsoft can pivot to the open Web paradigm as predicated in the Live Mesh strategy, or meander along while attempting to catch up in search and failing to buy Yahoo. You can find plenty of the latter analysis elsewhere, but here we’ll go for the throat of Microsoft’s disruptive opportunity by using a time-honored approach when faced with few facts but a lot of clues. Namely, building a case out of circumstantial evidence. And a smoking gun.

Continue reading on TechcrunchIT >>

Who controls TinyURL (or Snurl or other URL shortener) controls the high ground in the battle for the Internet platform. Here’s why: Our brains are wired to protect ourselves from threats to our food, oxygen, and water sources. Most attacks on our supply chains come from those closest to us, our families, friends, business associates, and combinations of those groups.

We handle nourishment requests from our loved ones by sharing, which is another way of saying dividing the materials to just above the point where each portion proves insufficient. Children come first since quieting their cries reduces the possibility of alerting outsiders while requiring relatively small amounts compared to peers. Then the spouse or friend is satisfied, setting up a social interaction around the breaking of bread. “Let’s grab a bite and talk about the deal.”

Once business is discussed, the parties fan out to hunt again. These cycles of search, retrieve, partition, and recharge dominate our lives, albeit pushed into the background by social constructs such as school, work, play, etc. We agree to ignore the pedestrian nature of breathing in and out that underlies everything.

So how do we identify the morsels of necessity? If we were computers, the mechanism would be the interrupt, some input device that triggers a disruption that moves resources to process the incoming data, or a timer-like object internally that wakes up and commands processes to predetermined or calculated rules based on aggregate input over time.

Writing those rules, the logic that defines what constitutes adequate input to get our attention, is in itself one of the most resource-intensive costs of the system. How do we decide what to pay attention to? One important way is by gathering the previous signals from those we share with and ranking them according to priority. Children first, etc. Only when I became a father of a teenager did I realize how irrelevant my concerns were in the face of a fourteen year olds’ makeup gathering needs. How she defines her friendships consumes far more bandwidth for the whole family than virtually every other task.

Interestingly, the younger daughter (7) is old enough to understand the value of drafting behind her sister while making a whole lot less noise about it. I learn much more from her processing of her sister’s signals than I do from the original cacophony. In other words, rules come from observation of the way others handle similar problems. From an early age, our younger daughter would sit in the living room with her back to the action, listening and absorbing the turmoil, and refocusing her needs to the lessons learned.

Today’s information systems begin to emulate these cycles in something we can call real time. The town crier, letters, telegraph, phone, email, IM – the stages of evolution to today’s real time swirl of information overload. As RSS allowed us to orchestrate our input streams into a more interactive daily newspaper, it also created a new measure of authority, one derived in a more personalized way to reduce the flow to something manageable in the time allowable while maintaining the fundamental ability to return to the forest and hunt for the next meal.

This is the point where social media constructs have begun to break down – identifying not just the holders of authority but the very rules by which we decide what constitutes authority, integrity, and credibility. Do we friend everybody, nobody; follow, track, hide? Interestingly, the words mirror the dynamics of the hunt, the foraging, the kill, the triage known as sharing.

What social scientists are discovering is that sharing produces a better outcome than hoarding. As Jonathan Schwartz so ably proved in his conversation at Supernova, Sun Micro’s open sharing of software proves much more efficient in creating a partnership with its customers than traditional marketing. The customers establish a connection by downloading the code, and then continually signal their needs, intentions, and even ideas at a granular level based on how they use the code, update it, ask questions, and so on. Sun turns this into an interactive, real time map that produces much greater lead generation and fulfillment than previous systems.

Our nervous systems are designed to use these real time feedback loops to manage the flow of body resources to confront the task of survival. Establishing the relative weight of signals becomes a substantial portion of our investment in survival. Those nodes that produce the highest value of data in the most efficient form reliably over time win. Applications like Twitturly demonstrate the aggregate power of these distilled signals. Whether Twitter is the ultimate instantiation of this intersection can be debated, but the TinyURL in the center of that system is the payload that most directly connects to our core instincts for preservation.

About noon Friday here in California, I happened to click on a Summize tab substituting for Twitter’s Track functionality and monitoring the use of my Twitter screen name. Someone named Scrabo had tweeted “Rumor here at NBC is that Tim Russert passed away”. A minute later another: “@stevegillmor Brokaw getting ready to go on air.”

Turning on NBC, then MSNBC, then CNN, I found nothing: reports on flooding in the Midwest, breaking news about a bomb attack in an Afghan prison, a strange obliviousness on the NBC outlets. Something about the first tweet resonated – “here at NBC” – and I went back to the computer and Summize, finding another tweet directed at me that said Wikipedia was already updated with the news. Jumping to the New York Times, a single line at the top of the home page. Finally, at 12:33 Tom Brokaw broke into programming with the news.

Today Summize has “Tim Russert” at the top of the Trending Topics list, with “Russert” third. The tweets continue to roll in 20 hours after the fact, even now at 9am Pacific at some 200 per hour. Twitter’s international audience lets the story follow the sun, but Russert’s fame is largely U.S. centric. Clearly we have lost what many consider the soul or conscience of our political process at the head of the stretch leading to November.

That same presidential race is the likely culprit in Twitter’s recent collapse and partitioning into minimal services. As the company scrambled to get some coherent strategy in place to keep users from tipping into a stampede away from the service, Twitter’s API was gated, the Web UI was dynamically stripped of pagination, @replies, and sometimes even the array of follow icons as event swarms stressed the servers. Most significantly, IM services over XMPP were the first to disappear and not yet fully restored, and with that the service known as Track that I was emulating with the third party Summize client when Russert collapsed.

We may look back at Monday’s Steve Jobs keynote at the WWDC as the point where Twitter stabilized enough to survive. Because of the intense developer interest in creating applications for the iPhone 3G product, the conference was sold out and, like Twitter services, the media gated to only a certain number from each outlet, whether blogger or mainstream. Missing the cut, I went to Plan B as I’ve often done when trips took me away to New York or CES during Apple rollouts.

As the event began, I followed Qik reports from Mike Arrington, page refreshes of photos and text from EnGadget, Techcrunch, Gizmodo, and Cnet, and a live video aggregation of various Ustreams and commentary from Leo Laporte’s TwiT Live. As Jobs took the stage, a video stream captured a murky view of the stage from too many rows back, but the audio proved unmanageable. Laporte’s chat stream produced a URL to a more stable audio feed that held up throughout the rest of the keynote. Arrington produced two short Qik videos of key sections that surfaced as Qik servers restabilized.

The net effect was exhilarating; a bootstrapped symphony of virtualized Steve Reality Distortion Field funneled through the MacBook AIR that I route every bit of my real time digital life through. Throughout, Twitter remained up except for a ten minute period when Jobs announced the 3G device’s price, and as the event retreated into the past Twitter services unseen for weeks began to reemerge.

Much has been made of the fanaticism spurred by social media events and seminal products such as the iPhone – the swarming of the early adopters, the trivialization of Twitter as a toy, you know the drill and the comments on this post will likely personalize the pushback. But an event such as Russert’s death and the emotional shock wave it produced put the lie to the notion that this stuff is echo chamber or A-List or whatever. 30 minutes before the world knew about this tragedy, someone I don’t know reached out and established a connection based on mutual affinity.

The magic of Twitter, and Ustream, and Qik, and all the social tools just now emerging, is this incredible, subtle, hacked, user-controlled information network, that in a million ways and micro-communities, performs as efficiently and professionally as the greatest media empires on Earth. In fact, the two have merged as we gain access to the tools of the trade while the trade gains access to our hearts and minds. Track will return, and with it a flowering of this new media revolution where the new boss is the same as the old boss: Us. And you’ll see Tim there in the front row, if you look closely.

Our visit the other day to the GooglePlex was extraordinary on a number of levels. At its simplest, the Gmail group opened its kimono to bloggers and what’s left of the mainstream media – full stop. We were asked not to live video the announcements of Gmail Labs, and to clear photographs with the team in case we accidentally revealed some strategic details. But the tour of the Gmail team cubicles, the Google Reader unit, the spam guys, and the user testing facility was fascinating, particularly as it became clear how much was being done with a very small group of code warriors.

By contrast, when Jeff Raikes‘ replacement as president of the Microsoft Business Division, Stephen Elop, took over the unit encompassing Office, Dynamics CRM, and Unified Communications groups, ComputerWorld estimated he controlled at least 26,000 workers responsible for generating fully a third of Microsoft’s 2007 revenue. Certainly Gmail doesn’t represent all of Google’s Apps (Office) investment, but enough of it to make clear how devastating this nimble strike force is and how catastrophically it can undermine Office.

The “features” rolled out in Gmail Labs range from trivial to obvious, but the power is not in what Google engineers have produced on their 20% time initiative. Rather, it’s the feedback loop that results when users can recompile Gmail with a personalized addition of such features. First, they vote with their feet, sending signals to the team not just of what they pick but how long they use it, when they discard it, and what they pick next. It’s a million-plus user testing facility for free, an easy way of extending Google’s original strategy of scaling up as usage grows, and most importantly, a wedge for the viral Greasemonkey development community that the approach aims to stabilize.

Take the first 13 or so apps and throw them out; I like adding photos to chat and bookmarks to email yet probably don’t care if they go away. Now look at the functionality exposed in these sample tools: extending chat for example, which boasts in addition to Photos tools to hide status messages and add keyboard shortcuts to things like Focus chat contact search and the fascinating Focus last chat mole. Moles, by the way, are those little Gchat windows that open up and array intelligently at the bottom of the Gmail window or can be popped out to stand alone.

Let your mind wander a little and you can see how significant this granular control of the Gmail console can become. I asked product manager Keith Coleman whether a Labs feature could remember the position of chat moles once resized and placed outside the Gmail container. Add a focus pull to your favorite mole, say the Twitter XMPP gateway, and then use a version of status removal to filter Track messages. Well, you get the idea, and so did Keith. For free. From highly motivated Plan B users.

Of course, Google is not the only cloud that can take advantage of this iterative feedback loop. On Friday’s Gillmor Gang, I asked Google API lead (and former Hailstorm architect) Mark Lucovsky whether he agreed with many that Microsoft Live Mesh was just about replication of data, or like me, that it was the tip of a Titanic-sized iceberg. Factor in Coleman’s insistence that any third party API was fair game for inclusion in a Google engineer’s Gmail Labs feature, and that would by definition include Microsoft Mesh.

Coleman owns Google Reader and Gchat as part of Gmail Plus, so it seems likely that future services will tie in Apps, aid in constructing enterprise versions filtered around information aggregation, automated push services out along the XMPP bridge, and even developer tools that use the combined services to fashion bug-tracking, code generation, and collaborative project archives. Today, it’s a personalized recompiled Gmail; tomorrow, it will be extended to affinity groups around gesture-mandated dynamic builds that adjust based on behavior and proliferation of open standards.

Part of Google’s impetus to do this was to route around the instability of Greasemonkey scripts which stressed out the Javascript architecture with ad hoc strategies. But Greasemonkey will return with a vengence as soon as developers outside the company realize they can take off from these approved experiments and wire up external API’s. Quick Links, for example, could be extended with a TinyURL-like bridge to encapsulate FriendFeed conversations and export them to the Twitter or Mesh cloud, or be written into Google Reader’s new Shared Item Notes and broadcast to Gmail contacts under user control instead of the current “Friend” contact mining that damages GReader’s privacy integrity.

Inevitably, the combination of user feedback and behavior and external pirate Greasemonkey innovation will reach a boil. Ray Ozzie and Stephen Elop are on notice, and have their work cut out for them. As Lucovsky notes, Redmond is not just sitting idly by: check out Scott Guthrie’s description of Silverlight networking improvements such as Cross Domain Sockets and Background Thread Networking. But while Gmail may not have a third of Google’s revenue, they now have a hell of a lot more developers working for them than last week.

In the past, you could measure Microsoft’s success by others’ weakness. This time it’s different. Google rolls out a five-pronged disruption to the smart phone, the Visual Studio developer base, social media, offline storage, and Webtone pricing – and it bolsters Ray Ozzie’s hand. With a month to go, Bill Gates’ “transition” from 80-20 to 20-80 has Redmond shaking.

The old games just aren’t working. Windows chief Steve Sinofsky spends a ridiculous half hour fending off CNet’s Ina Fried with zero information on Windows 7 and way too much repetition of the purpose of the interview: to discuss the process by which information will be given out in the future. We get it, Steve. You’re not telling the media anything except the ground rules for what you’re not telling the media. Thanks for the heads up.

In classic media terms, this is a Microsoft death spiral story. Vista sucks, Apple market share grows, the PC/Mac ads win awards. iPhone 2 approaches, Android looks cool and competitive, the AT&T WiFi takedown looks like it’s spreading to the Blackberry, the N95, and maybe, just maybe, the other carriers. The Google Web Toolkit harnesses Java developers, the Eclipse dev environment, a growing number of APIs and Javascript libraries hosted on Google servers, and Amazon-slashing AppEngine pricing. Developers, developers, developers.

But each chink in the old Microsoft armor cuts two ways – as a minor glitch in the continuing revenue power of the IT-controlled Windows and Office upgrade path, and a strategic boost for Ozzie and his Mesh strategy. When you hear the open crowd attack the notion of trusting Microsoft to route our data a la Hailstorm and its Passport albatross, you’re also hearing the first stages in acceptance of the new Microsoft mantra.

Mesh abstracts devices and operating systems into objects that can be coordinated and orchestrated to deliver the appearance of a single or composite device. That’s the guiding principle behind virtualization, which permits applications to address these virtual devices as single entities while spreading computational load across machines, domains, and business processes. When you hear people both outside and inside attack Mesh as a synchronization technology, you are hearing political spin about a strategy that has not yet been fully implemented or acknowledged.

From a technical perspective, the largest chunk left to be finished is affinity grouping – taking the atomized identity and social metadata and organizing micro-communities that can act as power brokers in the new information model Mesh creates. From a political perspective, these groups will quickly produce revenue in much higher proportion to the broader less targeted audiences of existing clouds. As that power is increasingly parceled out via Silverlight to the enterprise crowd, Windows and Office become services to be maintained much in the same way that IBM uses open source via its Global Services group.

Key to the transition is the interactive two-way nature of the communication services of the Mesh and Google platforms. Google collects behavioral data from Gmail, Apps, and search, but increasingly the roundtrip between Gchat and Google Reader is producing the high value signals (gestures) that fuel affinity group formation and targeted feedback loops. Mesh atomizes the Google, Facebook, and other social constructs into virtual devices that can be combined from the ground up to attack viral opportunities as they emerge.

If that sounds like Twitter, that’s because Mesh is Twitter’s Plan B. In recent weeks, we’ve seen Twitter quickly come to terms with the underlying problem of their viral success, and particular the unique but transcendent power of swarm characteristics that the (mis)use of the service has created. The very real time XMPP stream that proves irresistible to Twitter power users and its “track” conversation enabler are at the heart of what Ray Ozzie and his team discovered when they first began testing Groove . Mesh leverages that emergent behavior as the central construct of a virtual device router.

The conversation with Twitter and its satellites FriendFeed (listen to the recent Gillmor Gang with the FriendFeed founders), Twhirl, and to some degree Facebook, in recent weeks has really been about Twitter’s sole asset: its people. It’s taken a few punches to the middle to soften things up enough to encourage more transparency on Twitter’s part, but now that the dialogue appears engaged via the media, the real work needs to begin in earnest with Twitter’s owners in the Track cloud. Because Track requires the availability in real time of a dynamic swarm of affinity members, that service makes maximum use of the system, which still can’t deliver it.

Twitter’s problem, then, is to convince its users that they will restore that feature. Those that reject that as important can remain secure knowing virtually every Twitter clone can provide the basic commoditized services they require. In other words, they don’t need Twitter, just Twitter-like functionality. Those who are addicted to Track and real time services know otherwise. In aggregate, as affinity groups, they derive much greater yield from their behavior and micro-cast contract offerings than the rest of the marketplace – better resolution of information streams, increased economic clout, and the Darwinian favoritism of swarm resolution over less dynamic and more static problem solving.

Plan B is not for Twitter, or even Microsoft; it’s for us to do. If Twitter execs respond directly to the marketplace with straight talk, we’ll likely stay with them. But it is incumbent upon us to begin Plan B now, by beginning the process of harvesting our Track/Follow clouds and readying them for the day when other more robust services arrive. Starting now, we use Twitter behavior that reinforces core constructs (140 characters, TinyUrl pointers, Track, asynchronous follow) while discouraging features such as @replies which encourage non-real time use.

Syntax that records conversational techniques needs to be baked out across our affinity groups, for example dropping not just the @ sign but the Twitter name as well after the first citation, only reading upon a context switch. With XMPP and Track down, we have no way to archive via Gmail, but Summize and FriendFeed offer workarounds. If FriendFeed quickly implements XMPP as we discussed on Friday’s Gillmor Gang (you decide whether that will happen) we will have some redundancy on our side sooner than later. Plan B makes sense for more than just tactical Is Twitter Up reasons, but also because Google has similar services available. Adapting and morphing the Gmail/Gchat/Greader tool set to emulate the core Track model is doable, but requires some incentive for the company to move up the stack.

Google’s recent efforts suggest they understand the reality here, that this effort will not be about damaging Microsoft but closing the sale with their users to allow them legitimate trustworthy access to all the data they are collecting. On Microsoft’s side, Twitter needs to end the rivalry between Exchange and SQL Server on the delivery side, and (to be blunt about it) put Office and Windows in their place down the stack. And every time Google announces another disruptive chapter in their creation of the collaborative social network, the champagne flows in Ozzie’s group. Welcome to Plan B.

Robert Scoble. Blame FriendFeed. Steve Rubel. Blame FriendFeed. The Shel puppet. Blame FriendFeed. Dave Winer. Blame FriendFeed. Etc.

FriendFeed is a parasite service built on the back of Twitter. Let’s get this straight. No Twitter, no FriendFeed. Want to kill FriendFeed, as I certainly do? Cut off its oxygen. Take a page from Facebook’s incompetent UnFriend Connect gambit and refuse to pass Twitter posts through non-compliant ex-Google engineering scams.

OK, I’m way off my meds since the company has finally admitted on the Twitter Excuse page that they’ve figured out what the culprit is in the continual service meltdown. It’s the Track command, which as a result of my no-@-sign campaign to evangelize the Twitter XMPP Gtalk gateway, has now reached enough adopters to qualify as an actual threat to Twitter’s massive server farm or whatever access to Fred Wilson’s credit card and an EC2 account buys.

We found an errant API project eating way too much of our Jabber (a flavor of instant messenger) resources. This activity (which we’ve corrected) had an affect of overloading our main database, resulting in the error pages and slowness most people are now encountering.

We’re bringing services back online now. Some will be slower than others for a while, and we’ll be watching IM and IM-based API clients very closely. We’ll also be taking steps to avoid this behavior in the future.
Thanks for your patience!

Update: We’re turning off IM services for the evening (Friday) to allow for the system to recover. We hope to turn things back on Saturday.

In other words, an errant API project sucking Track clouds out of the Twitter core finally reached the critical mass necessary to hip Jack, whoever that is, to the reality that without the XMPP real time gateway, Twitter could just as well be FriendFeed without the siloed conversation spamyards. Further, Twitter engineers are working to minimize slowness and error pages by turning off the only distinguishing, disruptive, essential part of Twitter until the audience goes away at which point the problem will subside and we can turn it back on on “Saturday.”

Remember: I blame FriendFeed for this, and Robert Scoble, Steve Rubell, Dave Winer, and all the rest of the puppets and ex-Techcrunch analysts who, by appearing to rationally debate the pluses and minuses of FriendFeed versus Twitter, suggest FriendFeed even exists in the absence of Twitter. Nik Cubrilovic doesn’t help either with his cogent (except for the Rails part) analysis of Twitter’s scaling problems. Nowhere in this debate (most of it mercifully hidden forever behind the FriendFeed black hole where conversations go to die) was there a word spoken about the fatal Track bug until Jack hit the Off switch.

Now, in the cool clarity of no pulse whatsoever can we begin to rationally approach a solution. Forgetting that Hillary has shown no indication of processing the similar lack of pulse in her White House aspirations, let’s put the blame for all this squarely on the parasite API suckers and their dark master FriendFeed. Good.

What is FriendFeed anyway? It appears to be an aggregator of all things social. For me that means my Twitter feed – which already is pumped indiscriminately and obliviously through my Facebook status updates – and my blog posts – which have completely ceased since I got sucked into Twitter in the first place. As the puppet says: Fascinating. FriendFeed is Twitter, only slower. Here’s my demo of the difference between FriendFeed and Twitter:

Twitter: Hi, I’m having Sugar Pops for breakfast.

Ten minutes later….

FriendFeed: Hi, I’m having Sugar Pops for breakfast.

FriendFeed value add: A conversation cloud forms around the Sugar Pops meme. Louis Gray is having a pre-release alpha bowl of Open Pops, but Dave Winer (who has just noticed there is no Block command in FriendFeed) is busy discussing the politics of breakfast cereal decentralization in the Why We Need Block for FriendFeed room and does not weigh in here because he blocked me some months ago and doesn’t care what I had for breakfast or any other meal thank you very much. Another comment refers to the Winer tangent, several folks debate whether Sugar Pops are still on the market, and Robert Scoble broadcasts the whole mess back to Twitter as a TinyUrl… 20 minutes later.

By the way, errant API suckstreams reamplify all this with even less coherence than @replies provide, since remember: FriendFeed conversations have no way of pointing at each other with the possible exception of a Twitter link… and around the horn we go again. The new Rooms feature has initiated an ICANN-like squatter crisis where we are all encouraged to grab our names before the puppets get to them, which of course spawns another shitstorm of completely hidden conversations – wait, there’s Bob and Shel’s sequel book title. They better hope Loren is reading this in FriendFeed ten minutes later.

Update: Well, it’s “Saturday” morning now and no real time stream. I’ve been using a nifty combination of Summize and its Realtime results page (click refresh to see 2 new posts, or wait until Summize engineers work out the computer doing the refresh for us thing) and Twhirl, whose point and click @reply feature is a joy to use to send irate messages to Jack, whoever that is. Except I don’t blame Jack. I blame FriendFeed. On Twitter.

Bill Gates is sure taking his sweet time retiring. While he is busy hyping yet another Microsoft research project to the CEO Summit, Google has vaulted several huge steps ahead in the cloud infrastructure battle with Friend Connect. Mike Arrington’s audience with WonderWall given all due props, what on earth does this have to do with how to spend the 44 billion left on the table after the Yahoo meltdown?

The product manager of this gizmo is none other than Chris Pratley, the genius (seriously) behind OneNote, the Tablet product that briefly made that platform relatively salivating on my way out the door to Macland. Of course, in the OneNote days, it was impossible to get past Allchin and the Office Palace guards to encourage a free OneNote player for the browser. Now Mesh is in the oven, and Silverlight is that freely redistributable player. The times they have a-changed.

But Bill’s pet projects will just not cut it while Google methodically mows down the marketplace with these silly little social media chunks of code. It’s not that Friend Connect is going to slow Facebook down; to the contrary, it’s going to consolidate Facebook’s equity in social metadata and create a groundswell of OpenID adoption which in turn will drive Open Social app development.

Each new OpenID registration produces warm fuzzy feelings for Web site owners who become part of an expanding network of reuse of the original log-in. The terms of service for accessing social clouds will normalize over the next few months as users gravitate toward sites that leverage their original investment in OpenID registration. It’s a Frequent Flyer strategy, producing affinity based on less work, common interface guidelines, and pressure on Facebook and outside clouds to modify their terms of service to avoid having to reinitialize access to their social data over and over.

The same dynamics are starting to accelerate in real time streams over Jabber and XMPP. Facebook is soon to open access to their Chat service, eventually allowing the kind of piping currently enabled between Gchat/Talk, iChat, AIM, and Twitter, which together produce a common set of streams that all are recorded and archived in Gmail’s Chat repository and made available to a single search. Once users don’t care how or even if this aggregation is going on, they view the composite service as the application, removing the motivation for switching.

Of course, the last time we saw this type of viral spread, it was Adsense carried on the river of the blogosphere. Now, with Twitter’s social graph being formed out of the combination of follow and filtered Track, Friend Connect can provide infrastructure to model the unique characteristics of Twitter’s dynamic graph using Facebook’s avatars. LinkedIn’s business relationships, and, eventually, Open Social widgets across high-value sites. Oh, by the way, MyBlogLog — see ya.

But don’t think that just because Google will prosper that Microsoft won’t. Live Mesh can fit into this like a glove, feeding downstream vertical versions of affinity groups to skinned Silverlight containers. We’re within weeks of offerings already from Twhirl, FriendFeed, Summize, and others we just haven’t been told about yet. All Microsoft needs to do is get Bill his gold watch and get back to work.

Facebook finally has a real problem to deal with – an exceptionally rational and well-thought-out strategy by Google that puts the leading social media cloud in the path of a wave of angry users. The only thing Facebook has going for it is that said users don’t yet know they’re angry.

With its denial of service attack on Google’s Friend Connect, Facebook is serving notice that it feels threatened. By what? Users leveraging their Friend data to form communities outside of the Facebook moat? Forget for a moment that we tell Facebook who our friends are, and those gestures are created and owned by us. If Facebook insists on freezing our data as a condition of using their service, the company is essentially recommending we go elsewhere.

Google is smart enough to realize it doesn’t need to win here to help Facebook lose. Friend Connect does more to incentivize OpenId usage than to sell Google services; OpenId proliferation amortizes the complexity of that solution across multiple cooperating Web sites, particularly those that can make money on harvesting social synergies in conjunction with Adsense. It’s a Pay-Me-Now or Pay-Me-Later offer to Facebook: Play along and leverage your social equity or raise your hand and let your customers know how clueless you are.

Facebook insists it is preserving user privacy by neutering their API for its only stated purpose: “[E]nabling users to share their information with the third party websites and applications they choose.” Instead, in a Casablanca-like statement that gambling is going on (Your winnings, sir) one Charlie Cheever notes Friend Connect “redistributes user information from Facebook to other developers without users’ knowledge, which doesn’t respect the privacy standards our users have come to expect and is a violation of our Terms of Service.”

I love many parts of this, but none more than the part about privacy standards our users have come to expect. The API enables users to share their data with site and apps they choose but somehow Friend Connect does its dirty work without users’ knowledge. If the API enables user control, then what part of its use is without the users’ knowledge? Is there an Alzheimers standard that somehow slipped in here?
Read More

Most of my time these days is spent crossing the blood-brain barrier between Twitter and the rest of the cloud. Twitter stands on one side, a coursing stream of social data emanating from an ad-hoc framework of asynchronous follows and vanity track filtering. On the other side, the legacy blogosphere, RSS items floated via Google Reader shared items and planted in the Twitter stream via TinyURLs.

Managing the transfer of data across the barrier are two applications. One (FriendFeed) is disguised as a social media aggregator, and the other (Twhirl) is disguised as a rich internet application extension of Twitter that allows multiple users, point-and-click UI enhancements of the vanilla Twitter feed, and, common to all third-party apps, a licensing limitation on polling the Twitter API.

On Tuesday, Twitter suffered its first substantial test since the 3-or-so day outage several weeks ago, the Indiana and North Carolina primaries where Barack Obama essentially sealed the nomination of the Democratic Party. As the polls closed and traffic spiked, the Twitter real time gateway through IM and SMS collapsed, leaving those of us who live on that transport high and dry. within minutes, we switched over to Twhirl, which slowly but more quickly came back online than the gateway through, in my case, Gmail’s Gchat.

For the next several hours, I ping-ponged back and forth between the two services, Gchat arrayed on the left of the screen in a vertical browser window, and Twhirl in its AIR container hovering above the right of the screen and notifications rolling up from the bottom of my MacBook AIR as they were received from API requests. The Gchat gateway went up and down, alternating between no service and old tweets paging in as the database of outstanding tweets was flushed, until sometime after 7PM Pacific they synchronized just about the time Obama gave his victory speech.

The outage illustrated one more time (as if it were not obvious already) the need for a scalable and reliable Twitter, or at least one third party service that also provides the gateway functionality: Real time conversations between discoverable endpoints not necessarily aware of each other until the swarming characteristics of an event, an idea, a personality, an affinity group, or any combination of these elements are enabled. Twhirl’s Loic Le Meur announced such features on the May 2nd edition of The Gillmor Gang.

Friendfeed will likely follow suit, but it raises more questions than it answers with its expanded comment infrastructure and extended harvesting of non-Twitter streams such as delicious and blogs. Robert Scoble has used Friendfeed and its Hide function as a refuge from too much noise on Twitter answering his 20k followers, but only when Twitter implements track filtering will mass following cease to be a feature driver.

Less solvable are the tactical feints by startups that masquerade as standards-based solutions to the so-called centralization problem. Gillmor Gangs on Thursday and Friday delved into the mysteries of decentralization, but I remain unconvinced that these strategies do little more than shift the controlling authority for the Twitter namespace to other potential landlords. First, it won’t happen as long as Twitter executives maintain open XMPP access to third parties, and provide timely and responsive solutions to track spam and predictive scalability for event thresholds during the next few months.

Second, a careful reading of tech politics suggests the takeover of Twitter is an unlikely occurrence given the weakness of second tier players like Yahoo and Sun and the strengths of Microsoft and Google. Yahoo looks like Hillary’s shadow campaign as it walks through the motions of building out a social media personalization strategy while Microsoft’s Mesh infrastructure obsoletes the portal logic it’s based on. Sun is courting social media superdelegates while IBM is piling up the popular vote with customers in the midmarket. In both cases, the numbers are brutal in their inevitability. Scott McNealy should engineer a merger of the two weaklings and give Jonathan Schwartz some tools to survive, matching Yahoo users with Sun/Amazon clusters.

But even that unlikely mating would be swift meat for Microsoft, who is all over why Twitter is fundamental to the next phase of the enterprise network. No matter who owns the pipes, the real struggle is to deliver the drugs across the blood brain barrier. Mesh abstracts out the hardware layer at a deeper level than Amazon or Solaris with its virtualization layer — down at the social layer where the users live and control the domain. It’s the users, stupid, as Carville famously put it. Once switching costs are controllable, the user can band together in affinity groups and mandate the price vendors will need to pay to be listened to.

At its simplest (its true power) Twitter is a phone switch for routing information flow. Those who control the flow control the price for the information. In a virtualized platform, the hardware is the razor and the software switch is the blades. The software switch is an affinity-based construct that manages the signal-to-noise ratio of the information flow based on the contouring signals (gestures) of the members of the group. In the language of Twitter, it’s who you follow times what you track divided by how you filter.

The trick is squeezing the firehose down into multiplexed channels across the blood brain barrier and then expanding them as they flood the brain and its synaptic map. The architecture of swarms has unique characteristics that we are seeing modeled in the contortions of Friendfeed, Facebook Connect, Ustream chatrooms, Google Reader Notes, Disqus, and the rest of what Marc Canter calls the open mesh. It goes beyond bootstrapping, harnessing the brain’s ability to add the gut instinct of survivability to the equation of what choices can be made about information triage.

Simply put, you have to have the ability to broadcast an acuity for successful guesses. We’re at the doorway of gesture farming, where individual gesturers go beyond implicit behavior harvesting and aggregation and overtly share not just what they like but what they ignore. We’re seeing this in the political realm, where people are tuning out repetitive and shrill networks built on track spamming (Reverend Wright, Day One, electability) and tuning in to credible authentic sources regardless of media affiliation. They’re going direct via TinyUrl and their social graph (follow/track/filter) ontology.

Those who laugh at Twitter and trivialize it are insulting the very users they want to engage with. In elections, that is a fatal mistake. In technology acquisition and adoption, it is similarly Darwinian. Ballmer’s buh-bye is still being discounted as posturing, but in a real-time conversation, once you’ve met the mettle of the (wo)man, you know what you need to know. I think Ballmer and Gates and Ozzie had already made the calculation before they made the offer, namely that they were looking for a partnership with Yahoo’s users and developers, not with its executives. That is not to say they were not valuable, just that they would have to prove their value in the conversation. They didn’t. The rest is still in play.

Decentralizing Twitter is unnecessary, if not impractical. Dave Winer was right the first time, when he intuitively grasped the power of Twitter was not in what it was designed to be but in what it could be used for. By building on top of it, Winer signaled that instinct that he marshaled into RSS, the gesture of respect, the idea that in Steve Stills’ words, “Somethings happening here, What it is ain’t exactly clear…” Twitter ain’t broke, and we don’t need to fix it.

Week Three of The Gillmor Gang on TechCrunch went as well as could be expected. After the reunion cattle call and last week’s Mesh interrogation of David Treadwell, the apparent news shifted back to the denouement of the Microsoft Yahoo acquisition,
now amusingly being called a merger by the Wall Street Journal and the Times.

When Times Bits editor Saul Hansell kicked off the Gang conversation around the advisability of the deal from Ballmer’s perspective, I quickly vectored back to Live Mesh and why it makes the deal make sense. Of course, Microsoft certainly knew about both projects and their timing when they announced the buyout strategy, and few others did. And just as most analysts handicapped the takeover as being about advertising, so too did most view Mesh as a synchronization play.

The latter is like Bill Clinton comparing the Obama campaign to Jesse Jackson’s campaigns, a casual put down by narrowing the focus to the least disruptive aspects of this year’s political fundamentals. Similarly, calling Mesh just another bigco chaining of the previous monopoly to the next one pigeonholes Redmond as the comfortable old shoe of convicted monopolist. In the political world, that’s called framing.

But look at the Yahoo deal in the context of Mesh and a simple but disruptive model emerges. Microsoft must do several hard things with the deal: Use its brute force to acquire the company, and use its openness and transparency to acquire (retain) Yahoo’s customers (I’ll go watch Twitter for a few seconds while you laugh at the idea of Microsoft openness and transparency).

Look, however, at Mesh not as a synchronization technology but as the political renderer for the architectural absorption of Yahoo. The conventional wisdom is that Mesh is Windows only for starters, that the Mac version will come “soon,” that the Web client doesn’t allow provisioning of the system. But what if Silverlight front ends appear this month for various strategic apps? Indeed, Loic Le Meur is already receiving support from Microsoft France for just such a rendering of his Seesmic video tool. Le Meur’s recent acquisition of Twitter rich client Twhirl is next.

What better way to test the sincerity and credibility of Mesh than to ask Microsoft when such Silverlight tools for managing Mesh will appear. If the answer is “we’re working on it,” then the marketplace’s response is “we’re waiting for it.” If Mesh is merely a synchronization grid, why not empower PC, Mac, and Linux machines at one fell swoop to explore the kinds of collaborative net-aware applications that can use a boost for off-line storage, intelligent caching, and richer display?

Yes, this a political request: We’re saying: If you’re in this to achieve trust, lead with it. And if Mesh is much more than synchronization, like say, an elastic mechanism for moving information flows based on user-contracted behavioral signals (gestures) then establishing those relationships at the browser layer would be akin to the classic Web 2.0 eyeball plays.

Today’s most classic such play is Twitter, which has built a hockey stick ramp up out of the most trivial of offerings. But who has married social graph (follow) with search (track) with swarming (XMPP gateways such as the real-time Twitter-to-Gtalk client) so aggressively that the expanding overlapping circles of affinity groups fueled by Track (a special keyword search function of the Gtalk implementation) are already pretty much invulnerable to cloning. Data portability aside, how do you transfer the subtleties of a viral social map from a system in constant use?

Don’t believe me? Who’s going to make us switch away from a real time feed of such high value that if we stop contributing we destroy its value not just for ourselves but for the rest of the network? Why have we not seen significant defections? There have been notorious resignations (Hugh MacLeod) that only underlined the traction by the speed of the return to the fold. And all manner of attacks on the social order of the community, whether personal or political doesn’t matter, and almost impossible to distinguish at that.

Where it took Facebook some 6 months to land in hot water hell with Beacon, Twitter experiences furious storms of battle testing on an hourly basis. Bot attacks are mounting as publishers and PR and PACs nail up exploits and dial them back to the point where they re-submerge below the radar. The nature of follow versus flow dictates a careful measuring of signal vs. noise calculation with every follow, or surrender to search engines which reduce the service down to an after-the-fact snipe hunt for conversations it might have been good to be in if only we knew about them at the time.

That’s why the Track function is the true driver of Twitter’s ongoing power, just as Technorati vanity searches powered the build out of the blogosphere. At any moment, anyone on the planet can signal to one of these hybrid affinity groups that they have a question, an idea, or an answer, harnessing the power of a community of self-selecting, swarming activists looking for the most efficient way to extract value from the infostream.

Tracking goes well beyond search because of the two-way nature of TwitterNet. It’s keyword driven, yes, a search for information swarming around that topic, but it’s more importantly a gesture to other nodes on the network. Tracking Gillmor and NewsGang triangulates not only gestures toward me but toward the content and issues generated by the shows, producing a social graph of real-time conversations that can be entered into, passed along via @messages to other trackers, and expanded over time by follows of the people discovered through this mechanism.

And when Track spam rears its aggressive head to challenge the effectiveness of this network, we’re faced with important decisions on how to preserve the open virality while drawing a line with bad actors who want to shut down the discussion of important issues as the All Rev. Wright All The Time cable networks have done. From this week’s Gang:

Le Meur: I follow the replies to me in public like most people do. It’s very easy to expand that as Steve pointed out. Because you can just start writing more and everyone tracking that will start showing that. For example I have a bot talking to me, I don’t know what it is, it’s kind of stupid but it says ‘hi Loic how are you today,’ you can tell it’s a bot. This is the first versions of spam using this. And I want it filtered. So that’s something we’re thinking about as well. And adding XMPP to it will allow you to filter that. Another way to think about filtering is I’m adding… Scoble are you still there? No he is gone. So he’s always talking about adding those 20,000 people on twitter. Getting one update every second. I added 6000. And it’s great because I can hear everybody talking because that’s what I want, it’s very exciting. But when Steve says something, I want to be notified in a better way. Out of 6000 people there are many people you want to get notified better. That’s something we’re fixing, getting groups of friends. There are many many ideas we can think of and we’re thinking about those.

Canter: Loic, you’ve got the resources and the ability to create parallel systems so that’s how the web works. You don’t rely just upon Twitter. You can have parallel systems with a DNS backbone to connect these infrastructures together so that not all your balls are in Evan Williams’ vise.

Scoble: that’s exactly what Twhirl’s doing. They’ve already accepted Twhirl and Friendfeed. So if Twitter goes down Friendfeed is still there and vice versa. So there’s already some redundancy coming into the infrastructure through these tools that are starting to hook up the various aggregators and messaging systems that are coming out.

Gillmor Gang 05.02.08

Luckily, and fundamentally I believe by design, Twitter has remained open to a free flow of real-time track-enabled data, and so far has not gated this via API licensing or metering constraints. Like Mesh, the only thing that could stop Twitter would be just such a futile effort at “locking down” the service, giving users an incentive to move and vendors the economics to build an alternative. As long as Mesh walks the walk, and Twitter talks the talk, there’s no way to stop them.

Back in the good old days, big software companies did big things. Little companies tiptoed around in the shadow of the platform makers, gaining enough speed to liftoff and attract enough attention to survive long enough to be acquired. Like the old movie studios of the 30’s and 40’s, the technology studios of the 80’s and 90’s built stars and played them off until the inevitable decline.

So it went with Microsoft, as the seeming invulnerability of Gates’ machine accelerated to the boundaries of global saturation. Though we tend to think of Google as the conqueror, the reality is that Microsoft has struggled most with itself, the victim not of decline but of lack of fuel — the very customers who created the megalith in the first place.

Everything changed with the Net. The platform wars, the browser wars, the widget wars — they’re all really battles in the grappling with the living, breathing, swarm that is the Web. Even the argument over whether Office is dead is bogus, a joke that became a conference that begat a series of endless reiterations of the first O’Reilly Peer-to-Peer conference known as Web 2.0. The stuff that went over the wire now goes over the air; the stuff that used to persist solely on the client now comes from G@d knows where in the Cloud.

Read More