In this presentation filmed during QCon SF 2008, Jonathan Trevor presents two Yahoo! technologies: Pipes and Y! Query Language (YQL). Both technologies can be used to process data obtained from various sources, but while Pipes is limited to Yahoo web services, YQL can process many types of data.
Watch: Pipes and Y! Query Language (53 min.)
Yahoo Pipes uses a visual tool to specify a series of pipes through which input data flows and is filtered and processed in various ways. There are many data sources like personal data, CSV, feeds, web pages, services like Flick, Yahoo Search, and others. Processing is done through operators like: Filter, Loop, Regex, Sort, Union, and others. The result is shown in a web page, a feed, or given to an application. The service runs in Yahoo’s cloud, is free and does not offer security protection for sensitive data since anyone can copy and run a pipe made by someone else.
YQL is similar to Pipes but uses a textual textual language and can process both Yahoo web services data and any structured data with an URL. It has an SQL like syntax with three statements: SELECT, SHOW, DESC. This approach is more powerful and the applications created are protected.
Trevor explains both technologies, how they have been developed, their current status and shows demo examples.