When email was first created in 1965 it was used as a method to communicate between time-shared mainframe computers. Email has rapidly evolved since then, with the evolution of rich desktop clients, corporate email systems and webmail. Despite the evolution in the core messaging system, and despite the explosion in use of email, the default method for accessing and viewing communications has remained the same: chronological order.
The first webmail imitated earlier mail clients by displaying messages in chronological order. The desktop computing paradigm was folders and files, sorted alphabetically. The web paradigm for accessing information has in most cases become chronological order, mostly because of the email and webmail legacy.
A chronological system for indexing information breaks down quickly once the amount of information received reaches a certain critical point. Active users of email constantly moan about the information overload they experience, and the information is only a load because it is difficult to sort through and manage in modern systems. According to the cognitive theory of choice complexity, that feeling of load multiplies with each incremental increase in choices and decisions having to be made. In the email world this leads to a complete breakdown, and the trend of email bankruptcy (deleting all email and starting again).
Chronological order became more common on the web as social networks, such as the Facebook, blogs, feeds, feed readers, FriendFeed and services such as Twitter designed around the same paradigm – leading to most recent being most important. Some call it real-time, others call it information overload.
A default view of chronological order presents a natural barrier to the number of information sources that can be managed effectively (Scoble somehow broke the barrier, he is an exception). With only a few dozen feeds, a hundred or so emails a day and following one hundred or so people on Twitter, I find myself constantly behind and not being able to manage. When I am reading these sources, I find myself simply scanning for what is most relevant and most important – for eg. I will quickly reply to an email from a co-worker, while leaving others to slowly creep into the abyss of my archive.
Chronological order needs to be abandoned in favor of relevance. Without relevance, our ability to manage large sets of information is inefficient. The technology for relevance exist today, for eg. spam filters are able to tell us what we definitely don’t want to read. Real world information retrieval and organization is based on relevance, either what somebody else believes is relevant to us, or what we decide is relevant. Newspaper stories are not laid out in the order that events took place and libraries do not catalog their books in the order they were published.
Web applications that present relevance over chronological have proven to be popular. Techmeme hacked RSS, and instead of reading 50 feeds I can have Techmeme read 20,000 for me. Community-powered sites such as HackerNews are similar, they float up the latest content based on what a like-minded community finds interesting. The TiVo hacked television by taking chronological out of the picture and applying relevance.
Email applications have attempted to hack what is essentially relevance into the traditional chronological order. Old desktop email clients introduced folders and filters. Gmail introduced labels, adding a star to a thread and grouping multiple emails into a thread. Yahoo Mail attempts to highlight emails that it believes are from people close or important to you.
I hand over a lot of information to the applications that I use every day, but I am getting nothing in return (other than ads that creep me out). Every time I click a ‘like’, or I re-tweet, or I bookmark a page, or I spend time reading a post, that information can be stored somewhere and used to figure out what information is most important to me. I would happily exchange that part of my privacy for the ability to save a few hours each day and the pain of having to personally sort through all this information.
The ingredients for a personalized aggregator of all information exist today. A working solution would allow me to funnel far more data into my stream, and to not only discover more, but become more efficient. The second by second and minute by minute chronological order paradigm is broken, and like QWERTY, is a legacy from a world where systems were not smart enough to determine relevancy and real networks did not exist.
Original backwards post here
FYI I was going to publish this post backwards (ie. first para last) as a relevancy/time argument but figured you would have all moaned about it.
You should’ve… Like the reverse Seinfeld episode
Yeah that would have been a little difficult to read. I think the timestamps along the sidebar did their job illustrating the point. Good read!
Nik,
You should! Can you do a mirror post of this backward?
Shervin
Hey Shervin, at your request:
http://www.tech...ance-over-time/
This one is better!
Have you tried My6sense?
Hey Chris,
We never met… but it is a GREAT pleasure for me and our AI folks to be mentioned here by someone who is a Senior Scientist at Yahoo Labs and involved in a Search ranking function project (Linkedin
))
Thanks
Barak
Great article. Very thought provoking. I shall use your suggestions in the next version of the Paly Voice which I’m developing.
I agree and I believe there will be changes in what’s relevant to who. I’ve recently had *more than one* discussion about semantic web technologies, so there definitely is an interest.
Good read!
åŸBlog :Http//www.meng.edublogs.org
agree, obviously (see my email bankruptcy post). we’re getting into AI territory here, and very complicated rule sets. but any software monitoring the emails that i always read when scanning should be able to figure out pretty damn quickly what’s important to me. at least starting with the from address.
You should make the crunchpad do this for you.
Nik, this is one of the few good forward-looking posts on the site anymore so congrats for that. I’d appreciate if more TC writers did something other than try to pull the trigger fastest. However, I have to say “those who can’t, write”, in the sense that this is by far not a new idea, many attempted implementations exist and none of them are all that successful. The reason that no one has yet made a relevance ranking for feed-type information is that it’s really hard.
If you really want this, go work on it. Other people certainly have and probably continue to do so. You are resigning yourself to an “I told you so” position when someone finally implements a workable solution and given the prior art, it’s untenable.
As I said, though, for a writer at TechCrunch to do something other than mock and bitch.
Great article.
åŸBlog :Http//www.meng.edublogs.org
Great post, made me think about how to handle all the information that comes my way.
I like the idea of a repository where all the information is stored wich I ‘re-tweeted, like, or bookmark’.
I think this is why Wave is so relevant today.
I think that Wave is the reason this post was written. TechCrunch seem to be in love with anything Google-y, regardless of it’s objective merit. This post sounds backwards – a justification, of sorts, for Wave.
An important point: Email has not evolved much over the years. The clients have, but the underlying tech, especially the protocol (without getting into a debate on attachment formats) is still text-based and looks a lot like it did decades ago. AND THAT’S A GOOD THING. Keep it simple, people!
Well said. Perhaps Gmail should have a “view by relevance” view that highlights emails from people/companies your past use shows that you will be likely to read/respond to.
and i was sure you’d manage to fit in the soon to be infamous wikipedia quote on why wheels didn’t take off as quickly as you’d think after they were “invented”:
“Wide usage of the wheel was probably delayed because smooth roads were needed for wheels to be effective”
http://en.wikip....org/wiki/Wheel
we need to build the roads. or something. you get what i mean. more importantly, though, that is just one seriously funny quote.
the original quote was great, it seems a few people saw it and edited more craziness into it:
“”The lack of developed roads prevented wide adoption of the wheel for transportation”"
btw, collecting these for a post so send them in. for the funniest effect, lookup the most basic things on wikipedia like ‘window’ (”original windows were nothing more than a hole in a wall”), door, floor, etc. hours of fun.
i can’t decide if that wheel/road quote is the stupidest or smartest thing i’ve ever read. it’s definitely one or the other. maybe both.
this is a great article. I really dislike that companies offering you e-mail don’t make it easier to categorize. gmail is trying with their labels, but I want folders!
yeah this isn’t really what this article is about though.
“Web applications that present relevance over chronological have proven to be popular…”
and will prove to be even more so on mobile, where we have way less time & attention than on our desktops. Mobile device content consumption creates an even stronger need for relevance ranking.
Ah! This is *exactly* what we are attempting to do with Mailbeans: breaking the “flat-chronological-list” paradigm in the email world.
It’s not only about enabling features like filters or labels (which are helpful). It’s also about rethinking the UI, starting from not displaying by default the ‘big list’ of sorted messages.
my6sense is great and google reader is also very useful!
I am wondering why most people view emails in chronological order which doesn’t make much sense. I have been using the following method for years now:
1- I never use the sent folder, I configure my client so that all the emails I send are Bcc to myself. That way all the questions and answers are gathered at the same place (Inbox)
2- I sort the emails by threads (I guess any email client can do that), so that the each conversation aggregates, and they are not mixed over time.
P
Recency is also important. Why not provide both options? Sort by relevance, recency.
There are many use-cases for recency.
Say I get a mail regarding a comment someone left on my facebook post, I’d obv. like to see that on top, when it happens
To add to that, chronology is imp. on sites like FB too.
Say my favorite team won a game, and I want to post a FTW msg on facebook. Wud be pretty stupid if facebook didn’t sort chronologically
keyword is ‘favorite’
Nik — I’m a fellow Aussie with a solution to this problem. Contact me for details… and a demo
Chris.
Nik,
Wonderful post and subject.
Some thoughts:
If we argue that relevance of events is a normal distribution (bell curve), then a chronological sort *guarantees* that the top item in your mailbox or your twitter feed is not the most relevant. When normal distribution applies strictly, there’s a 66% chance the top event in the chronology is not even in the first standard deviation of relevance. This supports the observations of email bankruptcy – after a point you are so far behind, that it’s a better strategy to start from scratch.
However, determining relevance of unseen events is non-trivial and a time-sorted list (heap) is easy for the brain. I think heaps let you do a form of gap encoding by letting you say: “I know I have consumed everything before this point so I’ve already processed all relevant events.” This natural facility with heaps, and the sense of coverage we get from them, is powerful – probably what lulls us into thinking that it can scale forever. Of course, it never does: all of us together can produce more, than one of us can consume.
cheers,
vipul
I´m lost without mail.
http://pitt-new...no-meeting.html
The reason why messaging is in chron order is because that is how most people expect it. It is a comforting and easy to understand analog of physical messaging.
Really what you are looking for is not a reordering, but a filter for unimportant stuff. I suspect with such messages removed, you would view your messages in chron order.
Also, the news examples you cite don’t make sense since they only deal with one time frame: now. Thus, they sort by relevance what has not been removed from view.
If I were to take the article as the spec for a new system, I would assume that the required filtering algorithm must provide a best guess as to what the user is interested in. However, the user is likely to be interested in different things at different times. If the algorithm aggregated these various interests then the result would be like an overcooked stew. No differentiation of form or subtlety of flavours.
Rather, I suggest that the specification ought to be requiring more effective, semi-automatic, subjective classification. To make it much easier to navigate through personal / public data at web standards of scale and heterogeneity. However, it should remain for the user, not the algorithm, to determine what they want to discover at any time.
For us, before trying to apply an algorithm which tries to guess what’s relevant to user, there’s a more obvious and simple step to do with email: grouping and aggregating.
All webmails treat incoming messages like a log file. Doing simple groupings (say: by sender) does not put any AI, but at least allows to differentiate and to scan new emails quicker (look here: http://bit.ly/XCOLT).
On top of relational aggregation and grouping, many other things are possible.
The point is, even if one moves to IM, FB, Twitter and Wave significant shares of his online communication, there’s still need to handle inboxes with hundreds if not thousands of messages, so the flat log-like list is definitely not useful anymore.
It is always good to take the low-hanging fruit first. So mailbeans makes sense from that point of view. Yet, I feel it is at best an incremental solution to a problem that is exponential.
From your FAQ: “With ‘groups’ we refer to a predefined list …”. This gets to the nub of the issue. Wherever it is possible to pre-define a schema for significant classifications then the overload problem is likely to be still marginally manageable for the user. But the coming generation of communication tools and usage patterns are expected to produce vastly greater reams of information and to blow such techniques beyond their specified range of effectiveness.
To some degree the TechCrunch writers probably see themselves as canaries in the mine for the coming deluge. Hence the call-out for pro-active information management solutions.
I agree. Grouping within predefined categories is of course not the perfect solution for solving the information overload issue.
Apart from the fact that is relatively easy to change that predefined set into an user-editable one, the problem is that the user still has to manually flag his contacts as part of a given category (predefined or not).
But I want to focus on what you say about the variety of “realms” of information. It may sound simplistic, but I am sure that if you take 100 people, in the 90% of cases you’ll find similar patterns in their inboxes: everyone gets 90% or more of emails from friends, colleagues, Ebay, travel agents, banks, and a few more ‘classic’ category of senders. Again, is no perfect solution but is simple and lot better than simply scanning a log file entry by entry.
But in a broader sense, of course relatively new patterns like Twitter, FB, Wave etc. need something better than a fixed or semi-manual classification scheme.
I should have mentioned that we are also in agreement over the futility of presenting linear “log-like” UI to a multi-dimensional array of information. The game will be to iteratively present a small but meaningful range of directions against a vast domain of data. Along the lines of Google’s Wonder Wheel (for which the road was already paved).
A simple solution is for people to be more thoughtful and considerate about what they send, to whom and why. I use a filter system so that I un-follow peopleas soon as their Tweets are not relevant and/or I get swamped. I can go to their stream if I want to. Emails like wise, using the subject line as first line filtering.
tragedy of the commons says that will never happen.
Chronology is more important than just being called an artifact of legacy systems. I could sort my Outlook box by subject, sender, time and more, I could even use a combination of those. I stick to chronology because it matches the way the emails were written. If there is a conversation going on, I need to read the first email, first. Doing otherwise only results in embarrassment.
Gmail uses the conversation approach which is nice for a two person exchange but a complex exchange with asides, changing reply all groups and titles gets to be mess quickly.
Chronology remains the best way to handle my email.
how come despite all of my spam filters and marking all of those spam several of those “123greetings.com” are still getting through all of the filters while legit emails are blocked? We have a long way to go towards relevance.
PS & I know a bunch of people who still start at the top of their inbox and work down, often sending out questions or emails that have already been answered had they just started from the bottom up…
Hi Nik,
my6sense team thrilled to discover a person who like us strongly believe in the power of relevant and it’s importance in today’s evolving information streams
This is exactly what we’re doing here at my6sense.
Barak Hachamov
Founder & Visionary geek
my6sense
Here is Roi’s (TC) take on it:
http://www.tech...nse-its-coming/
Using Outlook, sorting by subject and sender is trivial.
I’ve had automatic color-coding of important friends (blue) and professionals (red) since Outlook 2002, and instantly see when someone “Important” has sent me an email.
With a little discipline/convention using the subject you can quickly focus on the most important 80-90%.
Of course, having a great spam filter is also a must !
Some tips:
– http://www.ppcs...blog/e-mail.asp
You’re talking about making a bigger one. You’re talking about making a bigger one.
Nik – strongly aligned to your thoughts but nearly twenty years old..
If your interested in temporal interfaces and their relevance to how we think about things then you should check out the work of David Gelernter. He was talking ( and building ) this stuff in the early 90’s.
Key book: The Aesthetics of Computing
Project: Lifestreams – http://cs-www.c...ifestreams.html
Enjoy!
most regular people don’t get anywhere near to as much email as professional tech bloggers. your problem won’t be solved any time soon–cuz there is no need…unplug your techcrunch brain stem feed for a few hours each day please.
would love it if one of the twitter clients would let me categorize tweets, learn what I think is interesting (through semantic analysis) and then find other tweets / links / users that I should look at based on the AI.
more detail: http://jacksonf...age-the-stream/
What is relevancy?
Define what you want from this information and remove/filter/unfollow those who stop providing that. You are missing the important things because you think you need to follow everything.
There is a new Relevance Engine out there trying to tackle these exact issues and there solution is quite ingenious,, check out http://www.mygeni.org where you only get relevant information
This is a fascinating post, especially since there are so many subjective variables to the idea of relevancy. You would need some pretty creepy algorithms to find the perfect filtering solution to this. D