January 13, 2009

YQL now supports microformats!

Microformats are the revolution that still has to happen. The idea is great: use information that is already published on the web and mark it up to be reusable in other ways. The classic example given in almost any microformats presentation is how you can turn a person's details into a downloadable vCard by adding a few spans with the right classes around the appropriate text content.

Since we publish in HTML anyways, it was deemed a good idea to add this extra layer of semantic goodness to allow software to extract structured data instead of having to write an extra API. Adding microformats is not hard (although there are disputes about the readiness of microformats in terms of internationalization and accessibility) - the consumption however is.

Naturally the best place for this to happen is the browser - the idea was to have the user agent recognize information in microformats and allow the user to re-use this information where it makes sense. Whilst there are several extensions for Firefox that do exactly that, none of the out-of-the-box browser configurations are microformat-aware.

Which means that if we wanted to re-use microformats, we need to write our own parser. Luckily enough this has been done for us in various languages. The pain is still to convert them into other formats once we retrieve them (not to mention the problem of not being able to pull them from a resource on another domain via Ajax).

Yahoo! likes microformats. We've been sneaking them into a lot of our products and we've already started to make their discovery and re-use a lot easier in SearchMonkey.

Now, with the release of YQL, we've gone a step further. We took the content-mixing capabilities of Pipes, and added a SQL-style language and REST interface to access directly without having to paint your data retrieval and filtering flow.

After discussing the microformats capabilities of SearchMonkey with the YQL team over a mug of coffee at my last visit stateside, we decided to see if it would be possible to get YQL to retrieve microformat data from web documents. I am proud to say that my wish has been granted.

If you check the YQL console you'll find that in the "Available Tables" section you can now see "Microformats" (spookily enough with my blog as the demo example). Together with the public REST API you can use this to retrieve information, filter it down, and return it as XML or JSON for re-use.

Say for example you want to get all the location information for a venue in Upcoming without using the API - this is now easy. The Bricklayer's Arms is the notorious location of the London Pub Standards, and you can retrieve the Microformats with the following REST call:

http://query.yahooapis.com/v1/public/yql?q=select * from microformats where url%3D'http%3A%2F%2Fupcoming.yahoo.com%2Fvenue%2F23933%2F'&format=xml

Use filtering to get only what you want:

http://query.yahooapis.com/v1/public/yql?q=select adjunct.item.type from microformats where url%3D%27http%3A%2F%2Fupcoming.yahoo.com%2Fvenue%2F23933%2F%27 &format=xml

You can switch to JSON and provide a callback parameter to use the information directly in JavaScript:

http://query.yahooapis.com/v1/public/yql?q=select * from microformats where url%3D'http%3A%2F%2Fupcoming.yahoo.com%2Fvenue%2F23933%2F'&format=json&callback=beerbeerbeer

This will hopefully make it much easier for people to write software that consumes microformats and maybe bring the "chicken" of adding microformats and the egg of finding good uses for them a bit closer together.

Chris Heilmann
Yahoo Developer Network

Posted at January 13, 2009 6:40 PM | Permalink

Bookmark this on Delicious

Comments

Posted by: janiska at January 14, 2009 4:45 AM

Posted by: Jeremy Keith at January 16, 2009 3:41 AM

Posted by: Daniel Hudson at January 16, 2009 11:35 AM