Seattle based podcast discovery and management service Pluggd is unveiling a major new feature at DEMO this weekend that combines speech recognition and semantic analysis to let users search for and skip to parts of an audio file that are related to topics of interest to them. It’s more than just speech recognition.
This is one of the most compelling examples I’ve seen lately of a growing trend: making multimedia content more granular and letting users take even greater control over the media we consume. We don’t just want to consume what we wish, we want to consume it in the way we wish.
Called “Hear Here”, the new feature is only available for use with a single test file this weekend, but CEO Alex Castro told me that with his team’s background in scaling large distributed computing at places like Amazon and Microsoft, they decided to take on the hardest part first - the relevance determination. Pluggd aims to have hundreds of thousands of podcasts analyzed and searchable by the end of the year, all with nothing required of the original publisher. Castro has been working with speech recognition technologies since he was 17 and at Bell Labs.
TechCrunch first profiled Pluggd when they launched in June. The company’s basic feature set is very cool, but not as cool as this new search function. They now report having more than 100,000 users and say they’ve seen their monthly uniques grow well ahead of schedule. The company has six full time employees and six part-timers; they’ve raised some angel backing and are working to raise further funding. They’re going to need it to crunch the kind of data this new feature seeks to engage with.
Here’s how this new search will work. When users decide they only want to hear a part of a file concerning a given topic, they enter a search term. Pluggd then searches for instances of that term and related terms being used in the file. Relevance is displayed on the file timeline with a heat map, sections of the file most related to your term appear in red, less related in green and unrelated in blue. Hover over any relevant point on the timeline and you’ll see the terms used there that Pluggd determined were related to your search term.
Users can click to listen to the file at that point, or select another option to tag, describe and share a particular section of the file. Castro says the company aims to set that data free, not keep it trapped in Pluggd. All of this is being done first with audio, but the company intends to implement the same technology with video as well. Podzinger already uses speech recognition with podcasts but doesn’t offer the semantic analysis of terms related to your search query. Blinkx does something similar for video, but Pluggd is building on speech recognition and adding even more value to search results.
Castro told me that the company built a language model in house to harden the speech recognition technology they licensed from another vendor. The results of that speech recognition are then passed through a relevance heuristic that Pluggd built by crawling the web to index terms regularly used in connection with each other. Search for Royals in a podcast and Pluggd will show you where else in the file baseball and Kansas City are discussed as well.
From the exploding popularity of short-form video to the basic inclusion of permalink URLs inside of Google Videos to the big interest in startup Viddler months before launch to the Hummer Winblad funded Widgetbox marketplace - putting content in chunks that users can work with is big. That’s with good reason; content chunks, or whatever you want to call them, substantially improve the web’s usefulness for users. Pluggd’s new search feature in particular sounds like just the kind of technology that multi-media search companies will be offering in years to come.
Hot idea, they will go IPO and get very rich. Very envious! Thanks for the great report. Makes a lot of sense. Good thinking. Solid ideas. Nice site!
http://research.sun.com/projec.....php?id=153
http://blog.searchenginewatch......913-122749
I have always liked Plugged. This new feature seems promising.
Such a killer idea. The UI coupled with the semantic associations have the potential to make all search a lot less frustrating.
and podzinger aims at something equivalent too http://www.podzinger.com/
Marshall, did you ask Pluggd in which format they would be openly publishing this metadata? I think it should be OPML. http://www.tablevsjetski.com/?p=7
Paul, I did not ask that, though I considered it. I would be surprised if the company had made any decision on that, though OPML seems like a good way to do it to me too. Liked your post on this btw.
Help us Obi Winer, you’re our only hope.
They’re going after a socially acceptable market with podcasts, but presumably there would be nothing to prevent this software being used to analyse telephone recordings for companies (for compliance purposes, or leak detection - why hello Ms Dunn!) or for security organisations (except perhaps the difficulty of breaking into whatever cosy cartels are already in place).
Farewell ‘phone privacy - “don’t it always seem to go, that you don’t know what you’ve got till it’s gone / They paved paradise, put up a parking lot.”
I have uesd that very cool and speed!
Very nice. I gave you a little write up in my blog.
Rex
Technically Speaking (blog) - http://rexdixon.wordpress.com/
7 Minutes with Rex Dixon (cyberears) - http://www.cyberears.com/index.php/Show/audio/157
7 Minutes with Rex Dixon (clickcaster) - http://clickcaster.com/rex-dixon
If the spooks haven’t already got equivalent or better technology, I would be very surprised. However, selling it to private companies would be a major new intervention into our privacy.
Wasn’t there talk of Google tracking conversations to decide what advertisements to show on TV recently?
It’s a good idea. However, with the sudden rush to release video (iTunes, Unbox etc), wont we be seeing more video casts instead of their audio equivalent as we meander on?
Thanks for the write-up Marshall, it was nice talking to you today and finally putting a face to a name.
Early this morning we pushed out a demo page so that users can play around with HearHere themselves. It’s located at http://www.pluggd.com/demo.
We love feedback so if anyone wants to play with it, send us feedback using the little link in the site’s footer or on our blog at blog.pluggd.com.
Hopefully folks in the world of podcasting will be afforded a more extensive demonstration this week in Ontario. I do believe that Pluggd is the first directory to attempt to integrate this kind of search as a value add for their users.
I noticed no date on when this technology will be pushed live. My feeling is that a demo on a single audio file doesn’t prove the scalability of their architecture. Hence, I don’t think this is worth a blog or a digg until the feature is actually in production on the Pluggd site.
Rob,
We are at DEMO this week and won’t be able to make the PME. We intend to make this available over all podcasts by the end of 2006.
We initially weren’t planning on talking about this until we launched it, but we were invited to announce it at DEMO. Since this was a great opportunity, we took advantage of it even though we still need to process more files.
Alex
Alex,
Sorry to hear that you won’t be there. Given the TechCrunch coverage people will likely be talking about it. If I hear any interesting conversation I’ll let you know over at the Pluggd site.
Rob
Great job Alex!
Wow, looks so cool!
I’m loving the tools that are emerging to describe what’s in all the audio and video on the web. This looks really cool.
Innertoob is a social annotation tool for people who want to use podcasts and other audio/video on the web as the fuel for conversations. Users make “time-posts” along the timeline, say what they want to say about that particular point in the podcast, and then share their comments through links and embeddable players.
A non-technical observation - Keep in mind that 10%-12% of males are colorblind and about 1%-2% of females. The color bar differentiation is not nearly as effective as for those of us afflicted. The drag-over function is helpful, but obviously we are losing a something through the color-based data interface.
Otherwise, it is a very cool product.
Peter,
Yes, we have this very much in the front of our mind. In fact, the plumbing is already in place to convert the heat map to a gray scale where white will indicate “hot” and black will indicate “cool.” We just didn’t have time to finish that feature for the demonstration.
Thanks for the observation and interest.
BFD, podzinger is my more usable, floating over a little picture to show search results is cute, but not really analyzable or interesting. If you wonder if this is a new idea. NO. does the govt have this stuff. YES. (check out http://www.in-q-tel.org and look at CallMiner if you doubt me) We deployed, over a year ago, a system from CallMiner to search/analyze/report on our call center calls. Rocks. This is Pluggd thing is a toy by comparison.