It takes effort to explain the significance of a new product when the immediate benefit to consumers may not be so obvious, and the awkwardly named “pipes” from Yahoo! is no exception. The product name is taken from the world of UNIX where a pipe is a conduit for the transfer of data between applications, while with the Yahoo product it is a conduit for data between web services. In a basic form Yahoo! Pipes allows you to take data from one or more sources and to bring it together, for example – to aggregate a group of feeds.
But Yahoo! Pipes goes beyond what just pipes are and what pipes do though as the application provides functions (or as they are called in the app – modules) that will perform a variety of different actions. There are modules available to prompt the user for input (a variety of input types), different operators to count, loop, cut, count, sort and merge data along with a variety of string and date functions. Because of this already broad base of available functions, Yahoo! Pipes is more akin to a shell scripting environment for the web rather than just a simple conduit between applications. It works like a visual procedural programming language with the output of the process dropping out at the bottom, in the form of text output, RSS, SMS alerts of even JSON. You can use feeds, user input or other pipes as input.
The beauty of the application is with its simplicity – a user can take any sources, user input requests or the above mentioned module and drag+drop them into place and then connect the pipes. Within minutes I had built an application (also known as a pipe, they should probably change the name as not everything can be a pipe) that would search for ‘Techcrunch’ in a variety of feeds, bring that data together, sort it and filter it for unique results. I saved the application and published it, from where I can now execute it at any time and receive the output in a variety of formats. I can take a copy of an existing pipe (application, argggh) and use it as a base template for my own pipe and I can browse an existing library of pipes.
Pipes can take any feed as input, and combined with the already available list of functions proves to be very powerful – my mind is still buzzing thinking about all that can be done with Pipes. I think some of the terminology needs to be cleared up, there needs to be a better introduction on the main page – but besides that this product is fantastic. It was inevitable that such a product would be released, and it is very good for Yahoo! that they managed to be the first of the big web companies to release such a product. The fact that they include Google Base as a default source in Pipes shows that the web is much more about interoperability than the desktop ever was or ever will be.
See Anil Dash, Tim O’Reilly and Jeremey Zawodny for more.
Pipes I likes, but I’m with Nik on the name not being so hot. It sounds medieval.
Nik,
Glad you like it. It’s definitely not a consumer product in the sense that something like Yahoo! Messenger is.
I think the “immediate benefit to consumers” will not be immediate, since Pipes is an enabling technology platform for a whole new class of services.
Finally a great product from Yahoo !
I think the power of Yahoo Pipes will become apparent once you begin combining the Pipes with other widgets and services.
For example, feed your Yahoo Pipes into my RSS2PDF service at http://rss2pdf.com to automatically generate a PDF version of it.
Here’s an example using Nik’s TechCrunch example Pipe:
http://rss2pdf....3F_render%3Drss
Yes, but are they compatible with tubes?
APPARENTLY THIS PRODUCT WAS TOO TECHNICAL FOR ARRINGTON TO INTELLIGENTLY DISCUSS?
PIPES IS NOT UNLIKE THE “VIGNETTE BUILDER” AND “VIGNETTE BUSINESS INTEGRATION STUDIO” PRODUCTS, BUT FOR THE WEBB.
BUILDER:
http://www.vign...s_BuilderLg.gif
INTEGRATION: http://www.vign...01_ss_BISLg.gif
IF YOHOO WAS SMARTS, THEY WOULD ENHANCE THE PRODUCT TO SUPPORT WEB SERVICES: YOU POINT PIPES AT A WSDL, PIPES READS IN THE INPUT PARAMETERS IMPOSED BY THE WSDL, AND PROVIDES AN INTERFACE THAT PERMITS THE USER TO BUILD AN INTERACTIVE EXPERIENCE WIFF IT.
EXTEND THIS INTERFACE TO QUERY OF JDBC AND OTHER SIMILAR DATA SOURCES.
COMPLETE THE USER EXPERIENCE BY LETTING USERS PUBBLISH THE RESULTING INTERFACE TO A WIDGET COMPLETE WIFF YAHOOO “PANAMA” ADVERTISING.
VIGNETTE = PWNED. LOL.
- I wonder if these pipes could be utilized to aggregate pictures from blog posts, rather than showing the whole post or the excerpt.
- Jerome, what are “tubes”?
Jeremy: Good point but many of the readers here are straight-up consumers, so while I think they should know about pipes they probably won’t use it, possibly on the publishing side though
is this what teqlo (jeff nolan, peter rip, et al.) are doing?
Very nice! I aggregated some open source planets into one:
http://pipes.ya...2xGR6s7QZVUMqA/
I think the lines between what people are calling ‘consumers’ and publishers are blurring. Tools like this make it possible for people with limited tech knowledge to use their imaginations and create something.
I think this is cool.
what seems to be missing is a “views builder” in the style of netvibes ecosystem. put some webservice actions in between, pepper it with microformats and then let the poweruser play lego with what has been known as the “web page”. web n+1 is crowdsourcing dbdesign to the masses. they are in fact building a new public sphere inside the walls of google and yahoo.
Every now and again a web app is brought out that makes me sit up and say “holy crap!”. This is one of those times.
i’m not that impressed – it’s like a user-generated, aggregated search engine w/ a weird mouse drag feature
Clever product.
Edit a pipe -> Safari crashes.
Great product.
The Save and Publish Option has Disappeared
And the App is running very slow
Help them if It get a Digg Homepage; this one can’t be DiggMirrored :-p
Sam > works fine for me (Safari 2.0.4). It’s a freshly-launched beta, give it a little slack.
This will be of particular interest to me. I’ve been developing http://www.myownsite.us and trying to make it contain more user-driven items such as feed modules for most popular links (based on clicks), most recently clicked links, and displaying the most popular feeds. I’ve been asked to provide a feed based on keywords found in any other feeds that any user has subscribed to.
I currently have search feeds and have used (but not integrated) sites that bundle feeds together. Is that similar to what Pipes does?
great marketing tool from Yahoo! also lots of services and informational websites now made possible – and for free.
A potentially useful product. I think thout that it will only reach its true potential if it becomes possible for external developers to create new pipes, preferabley in the form of a web service. Or to just be able to use existing web services. That combined with some sort of displaying standard could be very interesting.
Scenario: Take your existing pipe and pass all of the links back to commenters through a reverse DNS lookup (get the ip). Translate the IP’s into latitudes and longitudes, easy enough. And then plot them on a Google Map. Instant map of everyone talking about you. Que evil laugh and thunder clap.
Simon.
Amazing. I don’t know how much I will use :p but neverthless it’s an impressive piece of technology.
I hope sharing pipes was well done, so I can leave to others the work of creating good ones
The team will definitely support external web service pipes so that you can freely extend the system. It’s one of the very next things they’ll do
Very cool
Is it just me, or is Pipes down this morning? I cannot acces the site to check it out. Did the Pipes get clogged?
Yep it’s down for me also.
The Pipes look to be clogged, indeed. It was working for me this AM, so I reckon it got TechCrushed.
I am really excited to try this out when I get out of work, thanks for the heads up!
Yes, down for me too..
Can’t wait to try it!
I like compete. but since when it is allowed to have animated ads on here?
distracting..
That’s because Safari sucks Sam. It hates Javascript.
Pipes, hmmm ….. where have I heard that before? Ah Unix Feature for the last 30 years. Simply tarted up with a visual interface. Gary Perlman then at Ohio State showed men something similar 15 years ago. I guess this is the penalty for not reading the literature.
Cool! You guys are sending too much traffic over there…it’s busted!
I’ll can’t wait to try it out though. For people who aren’t familiar with Linux this is probably a new, cool concept.
http://averageidea.com
Salesforce also release a open architecture call Apex, which TechCrunch covered a few weeks back. I find this similar, yet far more intuitive — when it works.
Looks, neat I will try – but it is still down ?
– I like the idea, but again the terminology is flawed. Pipes is good, but calling (2) things one word is never good, unless its a joke. like –
– Convince the boss, a “Southern tresspass” is when you tricked someone”
- then let everyone know it really means, “a slip during a sexual act, into a un expecting place” –
– Then trick your boss into stating he “Southern’ed trespassed the ugly secretary”- in front of everyone.
– Ok so I went off topic, but it was funny … (atleast to me) , (must have been there)
-RB
It’s not a truck you can just dump things on.
looks good but very very slow… to load page, etc.. (yes, I do have fast boardband connection but still)
/Yuva
is it like to Google Reader.. or anything more??
@ Matt,
@ Yohay Elam #5,
http://www.goog...amp;btnG=Search
It seems to be down right now.
The pipes seemed to be clogged. Maybe if I do a little gambling…
Man, they should’ve named it Tubes.
Looks pretty awesome, that aside. I love seeing quality dev-tools-as-web-app come out of companies, and Yahoo!’s corporate culture seems to support it. Anyone have the odds on whether this idea came out of a Hack Day?
I was very impressed with yahoo offering a service like this that appeals to a more tech audience. Very good move, gives them back that edge they were missing for so long.
Thee pipes are clean!
Yup, as someone said, our pipes are indeed clogged at the moment
We’re working it…
Has anyone had trouble loading? I get “system error cannot load” or an infinte loading symbol. Im running firefox, xp, and on university fiber. Thanks.
Cool.I like it.
Long time ago, in a decade far away (read 80’s and early 90’s), there was a company in the Silicon Valley called Metaphor Cmputer Systems (which was later acquired by IBM). It got lost once IBM took over.
Founded by ex-Xerox PARC researcher David Liddle, Metaphor had a tool very similar to Pipes. Based on Xerox Star, it had some advanced features that made their way later into Windows and Mac. Nostalgia buffs, see http://patrickl...er-systems.html for more info; for the latest incarnation, go here — http://www.meta...uct/Product.asp.
This tool can visually connect a variety of data sources including spreadsheets, word process documents, databases, etc. in addition to API for converting any application to be a data source or a sink.
It is an elegant concept for quickly prototyping data conversion/processing applications and it is nice to see that Yahoo is doing it.
I like the new error message on the sample yahoo pipes posted above. The message says:
Our Pipes are clogged! We’ve called the plumbers!
At least the developers have a sense of humor.
Our Pipes are clogged! We’ve called the plumbers!
Oh Yahoo! You’re so Google-Like.