August 29, 2008

SearchMonkey Rides with the Semantic Web Gang

For those of you who are interested in learning more about how structured data fits in with SearchMonkey and Yahoo! Search strategy, please tune in to the latest Semantic Web Gag podcast to hear our very own Peter Mika discussing topics such as:

• how SearchMonkey is helping to motivate large site owners to use semantic data
• the importance of shared vocabularies for application interoperability
• the emergence of automated tools for extracting semantic data, such as Calais

This podcast is available as a stream or MP3 download. Enjoy!

Evan Goer
Yahoo! SearchMonkey Team

Posted at 4:01 PM | Comments (0)

Show off the videos of your favourite band with the Yahoo Music API

There's a new kid on the API block in Yahoo land: the Yahoo Music API. It comes with programmatic access to the data on Yahoo Music and supports all the goodies you got to know from other Yahoo APIs: XML or JSON output, option to define a callback method for JSON output and filtering via REST parameters. All of this is explained in detail in the extensive API guide and there is also a Music API forum to get help.

The really cool thing about the API is that it allows you to get music videos of your favourite arists and play them immediately in your web site with only a few lines of JavaScript. Want to learn how?

All the code examples here are available live at http://ydntest.com/musicapi/ and you can download them as one zip.

Step One: Get an API key

The first thing you need to do is tell Yahoo Music who you are by getting yourself and API key. This is relatively painless, all you need to do is fill out the form at http://developer.yahoo.com/wsregapp and you get back 65 character API key. Done.

Step Two: Get the list of videos and display them

The API method to use to find a list of videos for a certain artist is "Search for Video" and the REST URL (for the US market) is:

http://us.music.yahooapis.com/video/v1/list/search/video/{artist}?
appid={your application ID}

This returns you an XML document as this is the out-of-the-box standard format. As we want to use the data in JavaScript, we'll need to define JSON as the output format and define a callback function to receive the data.

http://us.music.yahooapis.com/video/v1/list/search/video/{artist}?
appid={your application ID}&format=json&callback=videosreceived

This will return the data as an object and send it to the function videosreceived. In order to use it you can do the following, which (once you added your API key) will give you the data of all Depeche Mode videos:

<script type="text/javascript">
  function videosreceived(data){
    alert(data)
  }
</script>
<script type="text/javascript" 
 src="http://us.music.yahooapis.com/video/v1/list/search/video/
 depeche-mode?format=json&callback=videosreceived&
 appid={your api key}"></script>

Try the plain example out here.

The data is in JSON format and the interesting part is the Video array with information about each video. If you look at the JSON output you'll find that there is a stunning amount of data for each video. What we need to show videos are: the duration, an image of the video, the video ID and the title. The following script will get them. Instead of just alerting them out, let's create an HTML list.

<script type="text/javascript" charset="utf-8">
  function videosreceived(data){
    var videos = data.Videos.Video;
    var list = document.createElement('ul');
    for(var i=0,j=videos.length;i<j;i++){
      var title = videos[i].title;
      var id = videos[i].id;
      var photo = videos[i].Image.url;
      var duration = videos[i].duration;
      var id = videos[i].id;
      var li = document.createElement('li');
      var img = document.createElement('img');
      img.setAttribute('src',photo);
      img.setAttribute('alt','Video screenshot of: ' + title);
      li.appendChild(img);
      li.appendChild(document.createTextNode(title+' ('+duration+')'));
      list.appendChild(li);
    }
    document.body.appendChild(list);
  }
</script>
<script type="text/javascript" 
 src="http://us.music.yahooapis.com/video/v1/list/search/video/
 depeche-mode?format=json&callback=videosreceived&
 appid={your api key}"></script>

Try the list example out here.

This works nicely but looks terrible. For starters the video screenshots are too big and the time information doesn't make much sense in seconds as we are used to seeing them in minutes and seconds. The first problem can be resolved changing the size parameter on the photo URL, the second needs a bit of math:

<script type="text/javascript" charset="utf-8">
  function videosreceived(data){
    var videos = data.Videos.Video;
    var list = document.createElement('ul');
    for(var i=0,j=videos.length;i<j;i++){
      var title = videos[i].title;
      var id = videos[i].id;
      var photo = videos[i].Image.url;
      // resize photo
      photo = photo.replace('=385','=75');
      // turn duration into minutes and seconds
      var duration = videos[i].duration;
      var mins = parseInt(duration/60);
      var seconds = duration % 60;
      if(seconds < 10){
        seconds = '0' + seconds;
      }
      duration = mins + ':' + seconds;
      var id = videos[i].id;
      var li = document.createElement('li');
      var img = document.createElement('img');
      img.setAttribute('src',photo);
      img.setAttribute('alt','Video screenshot of: ' + title);
      li.appendChild(img);
      li.appendChild(document.createTextNode(title+' ('+duration+')'));
      list.appendChild(li);
    }
    document.body.appendChild(list);
  }
</script>
<script type="text/javascript" 
 src="http://us.music.yahooapis.com/video/v1/list/search/video/
 depeche-mode?format=json&callback=videosreceived&
 appid={your api key}"></script>

Try the time and photo size example out here.

Playing the AngelVideos

This displays the list, but it wouldn't play a video yet, now would it? Well, for this the Yahoo Music API has a video player API for us. First, add SWFObject then create a video container and inject the player - ready to play videos.

<script type="text/javascript" src="http://us.js2.yimg.com/us.js.yimg.com/lib/flash/
swfobject/1.0/swfobject.js"></script>
<script type="text/javascript" charset="utf-8">
  function videosreceived(data){
    var player = document.createElement('div');
    player.id = 'videocontainer';
    document.body.appendChild(player);
    var so = new SWFObject('http://d.yimg.com/cosmos.bcst.'+
'yahoo.com/up/fop/embedflv/swf/fop.swf','videoplayer','512',
     '322', '9', '#ffffff');
    so.addVariable('eID','1301797');
    so.addVariable('ympsc','4195351');
    so.addVariable('lang','en');
    so.addVariable('shareEnable','1');
    so.addParam("allowFullScreen","true");
    so.addVariable('enableFullScreen','1');
    so.addVariable('autoStart','0');
    so.addVariable('eh','bandsPlayer.handler');
    so.addParam('allowScriptAccess','always');
    so.write('videocontainer');
    var videos = data.Videos.Video;
    var list = document.createElement('ul');
    for(var i=0,j=videos.length;i<j;i++){
      var title = videos[i].title;
      var id = videos[i].id;
      var photo = videos[i].Image.url;
      // resize photo
      photo = photo.replace('=385','=75');
      // turn duration into minutes and seconds
      var duration = videos[i].duration;
      var mins = parseInt(duration/60);
      var seconds = duration % 60;
      if(seconds < 10){
        seconds = '0' + seconds;
      }
      duration = mins + ':' + seconds;
      var id = videos[i].id;
      var li = document.createElement('li');
      var img = document.createElement('img');
      img.setAttribute('src',photo);
      img.setAttribute('alt','Video screenshot of: ' + title);
      li.appendChild(img);
      li.appendChild(document.createTextNode(title+' ('+duration+')'));
      list.appendChild(li);
    }
    document.body.appendChild(list);
  }
</script>
<script type="text/javascript" 
 src="http://us.music.yahooapis.com/video/v1/list/search/video/
 depeche-mode?format=json&callback=videosreceived&
 appid={your api key}"></script>

Try the player initiation example out here.

In order to play videos, we need to add links to the list calling the player with the right video ID. The player needs the video ID to be preceded by a "v" to work. The player is the element SWFobject injects into the container and you can get a handle to it using getElementById(). The ID is the second parameter in the new SWFObject() call. Doing this, and adding some CSS to make it pretty and we have ourselves a player:

<script type="text/javascript" src="http://us.js2.yimg.com/us.js.yimg.com/lib/flash/
swfobject/1.0/swfobject.js"></script>
<script type="text/javascript" charset="utf-8">
  function videosreceived(data){
    // create a new video player container
    var player = document.createElement('div');
    player.id = 'videocontainer';
    document.body.appendChild(player);
    // inject the video player with the id 'videoplayer'
    var so = new SWFObject('http://d.yimg.com/cosmos.bcst.'+
'yahoo.com/up/fop/embedflv/swf/fop.swf','videoplayer','512',
     '322', '9', '#ffffff');
    so.addVariable('eID','1301797');
    so.addVariable('ympsc','4195351');
    so.addVariable('lang','en');
    so.addVariable('shareEnable','1');
    so.addParam("allowFullScreen","true");
    so.addVariable('enableFullScreen','1');
    so.addVariable('autoStart','0');
    so.addVariable('eh','bandsPlayer.handler');
    so.addParam('allowScriptAccess','always');
    so.write('videocontainer');
    // get a reference to the video player - this needs to be global!
    vidplayer = document.getElementById('videoplayer');
    var videos = data.Videos.Video;
    var list = document.createElement('ul');
    for(var i=0,j=videos.length;i<j;i++){
      var title = videos[i].title;
      var id = videos[i].id;
      var photo = videos[i].Image.url;
      // resize photo
      photo = photo.replace('=385','=75');
      // turn duration into minutes and seconds
      var duration = videos[i].duration;
      var mins = parseInt(duration/60);
      var seconds = duration % 60;
      if(seconds < 10){
        seconds = '0' + seconds;
      }
      duration = mins + ':' + seconds;
      var id = videos[i].id;
      var li = document.createElement('li');
      // creata a link pointing to the playID method of the movie player
      var a = document.createElement('a');
a.setAttribute('href','javascript:vidplayer.playID("v"+'+id+')');
      var img = document.createElement('img');
      img.setAttribute('src',photo);
      img.setAttribute('alt','Video screenshot of: ' + title);
      a.appendChild(img);
      a.appendChild(document.createTextNode(title+' ('+duration+')'));
      li.appendChild(a);
      list.appendChild(li);
    }
    document.body.appendChild(list);
  }
</script>
<script type="text/javascript" 
 src="http://us.music.yahooapis.com/video/v1/list/search/video/
 depeche-mode?format=json&callback=videosreceived&
 appid={your api key}"></script>

Try the full example out here.

Doing things right

This is all nice and good, but it is also rather unclean. When you use "javascript" links, the player needs to be a global variable, and there is not much fallback and security measures (will the API work?) in this example. Please leave some comments and we can do a more generic and cleaner version, this is just meant as a start to get you going.

Chris Heilmann

Yahoo Developer Network

Posted at 8:39 AM | Comments (1)

August 28, 2008

Geeks in Oxford

Oxford was once the intellectual capital of the world and while they might have to share that title now, the place is still buzzing with smart people. Oxford geek night was not what I expected at all. To start with the opening talk wasn't a talk, it was an act!

Ben Walker's geeky music certainly tickled me. It's great to see a musician writing songs about Twitter because that's what he's into. Apparently Nerdcore isn't the only musical genre with musicians that can write code.

There were a couple more talks about music, including Rhodri Marsden's highly amusing rant about selling music you made yourself. There were also some great lightening talks. I particularly liked Rachel Andrew's great summary of your options for taking online payments, and James Turnbull's summary of Django and what we can expect for Django 1.0.

Tom Hughes-Croucher
Yahoo! Developer Network

Posted at 9:22 AM | Comments (2)

August 22, 2008

UK Hadoop User Group 2008

On Tuesday, I went to the first UK Hadoop User Group meetup with Yahoo!'s Doug Cutting, founder of Hadoop, a top-level Apache project developing open-source software for reliable, scalable, distributed computing. About 60 UK and European developers attended. Interestingly, about half of those in the room were already using Hadoop and half hadn't tried it.

I did a few interviews at the event so you can get a flavour of what was covered. Tom White talks about how to get started with Hadoop.

A shout of thanks to our hosts Skills Matter and our co-sponsor Last.fm for making it such a great event. Skills Matter kindly put up their video coverage.

Doug and I particularly enjoyed Klaas Bosteels talking about Dumbo. The Dumbo project is an easy way to use Hadoop Streaming in lovely Python. The talks mentioned in video are also available such as Tom White's talk on Hadoop on EC2, Hadoop Lessons at Last.fm and Smartfrog and Hadoop and, of course, Doug Cutting's introduction to Hadoop.

And now, as I fly away on my magic cloud I wish you happy grid computing.

Tom Hughes-Croucher
Yahoo! Developer Network

Posted at 8:02 AM | Comments (0)

Accessibility 2.0 presentation audio and transcripts

In April the Accessibility2.0 conference in London, England provided a lot of food for thought for the attendees, taking the discussions around removing barriers and inclusive design further than most conferences.

The organizers, AbilityNet have now come around to releasing all the audio and transcripts of the Accessibility 2.0 presentations for download and reading. Yahoo's involvement was my presentation "Fencing in the habitat" and Mike Davies as an expert on the final panel discussion.

This is a great service and a good example of what service you should provide to attendees of accessibility events. We congratulate AbilityNet on this and look forward to next year's conference.

The next event in terms of Accessibility conferences in the UK is Scripting Enabled which still has a few free tickets available.

Posted at 2:53 AM | Comments (0)

August 20, 2008

Greetings from 360 Flex

Editor's note: Greetings from 360 Flex was originally posted on the Yahoo! Flash (R) Blog, which features news and articles on Yahoo! Flash components and libraries.

The Flash Platform team is hanging out at 360 Flex. On Sunday we had a Yahoo! API hands-on workshop showing how to use some of our most popular components such as ActionScript 3 Maps and the Flex AutoCompleteManager. A few lucky folks won a copy of the beginner’s Flex book Learning Flex 3 by yours truly. For those of you who missed the workshop, we’ll be posting a screencast of that session tomorrow, so stay tuned for that.

Along those lines, there’s something new this year for those of you who couldn’t attend (or who’d like to review your favorite sessions). The conference is posting videos of the sessions that you can preview and purchase, and over at Ted Patrick’s blog you can view the sessions for free, and in HD, using the Adobe Media Player.

We’re also hosting an API contest in conjunction with OpenFlux. The winner will receive a game console and other goodies, so show us what you got.

As for me, I just enjoyed a great session on Flex with Rails and RubyAMF by Tony Hillerson, and now I’m attending Josh Tynjala’s Polishing Components for the Masses session. I’d better stop writing and pay attention…I could learn something.

Alaric Cole
Yahoo! Flash Platform

Posted at 8:17 AM | Comments (0)

Rise of the Filipino Web Innovator

It was almost 20 years ago when, as a Computer Science student, I listened to Philippine President, Corazon Aquino, address a packed amphitheater at the University of California at Berkeley. In her speech, President Aquino called for Filipinos to return home to help rebuild the country. I didn't realize then that I would have the potential to heed that call a few years later when setting up the Philippine subsidiary of another global technology firm, and now with Yahoo!, while reiterating that same message to our Pinoy developer community.

Since joining Yahoo! a year ago, I often get questioned about the meaning of being a platform company. I have to reiterate that Yahoo! is not just a platform company, we are working on being an open platform company. This provides great potential for any Pinoy developer who now has an opportunity to leverage the Yahoo! infrastructure and technology and catalyze the Internet economy in the Philippines to help drive the country forward.

Yahoo! is the leading internet brand in the Philippines, and our platform provides the scale to make developers’ creations stand out. Although we have only just started our developer initiatives in the Philippines, I believe that our Hack Day at the University of the Philippines last year and our more recent Yahoo! Developer Network meet-up at TGI Fridays in Makati City last week – will help drive local innovation on the web.

It was good meeting the 80 Pinoy developers who braved the thunderstorm to attend our event. All their feedback will go a long way as we create new initiatives to support the develop community in the Philippines.

We know there are lots of talented Pinoys out there and we want the world to know that there's more to the Philippines than SMS, Boracay and the Love Bug virus. If you are a developer from the Philippines, I would like to invite you to submit your application form by 5p.m. 22 August for a life-changing trip to the Yahoo! HQ in Sunnyvale, California, and represent the Pinoy developer community. Click here for details and good luck!

Lastly, congratulations to Christine Amarra who won our lucky draw and walked away with an iPod.

Onward and upward in the Philippines!

Jojo A. Anonuevo
General Manager, Yahoo! Philippines

Posted at 4:07 AM | Comments (2)

August 19, 2008

Ferrets, Disco, and Community (Barcamping in Leeds)

I spent the weekend at Barcamp in Leeds. Barcamps are getting quite common now, but it is really nice to see communities all around the UK joining in.

With almost 100 attendees, the technology community in the North of England is really buzzing. I was impressed by the range of people that came--everyone from UNIX hackers, web developers, startup founders, and designers.

Some of the strangest parts of this particular event: the very adorable ferret and the YMCA singalong. Still, there was plenty of geekery as well. One of the most interesting sessions for me, and something that would be useful all over the world, was a discussion on how to create and support local tech communities.

There were a few really good ideas on ways to encourage community. Events are an obvious thing, but starting them can be really hard. Two valuable resources: a list of geek-friendly venues and contacts among university professors (with access to lecture halls). Communities also need resources to get going; several people highlighted how visiting larger cities with more mature, established tech communities has been helpful in finding sponsors and identifying investors interested with an interest in engaging more locally.

YDN loves community. If you have a community that you are trying to bootstrap, maybe we can help. Drop me a mail to croucher at yahoo hyphen inc dot com I'd love to hear from you.

Tom Hughes-Croucher
Yahoo! Developer Network

Posted at 4:31 AM | Comments (0) | TrackBack

August 15, 2008

RDF, XSLT, & the Monkey Make 3

If you've been hanging around the YDN recently, you've probably heard a thing or two about SearchMonkey.

And why not? SearchMonkey is pretty darn cool.

It lets you enhance the appearance of search results for your favorite sites. So the next time you need to look up, say, restaurants, a SearchMonkey app can distill all of the important information, like location, price range, and rating all into one place, right there in your search results.

A ton of people have been tinkering with SearchMonkey since it launched in May. One of the main reasons for this (aside from how cool it is...and never mind the $10,000 contest they held recently), is how easy it is to pick up and start developing with.

In this article, I'll go over XSLT and RDF--two of the fundamental concepts that power SearchMonkey. If you're looking to build your first app or you've built a few and want to get more out of it, you'll definitely want to read on.

RDF

RDF stands for the Resource Description Framework. It provides a standard way to organize information into semantic units. Structuring information with RDF allows authors the ability to preserve relational and meta information. This is a very good thing.

For instance, here's a plaintext snippet of a newspaper article in unformatted plaintext:

Link By Link This Is Funny Only if You Know Unix By Noam Cohen Published: May 26, 2008 For a certain subset of Internet users, “Sudo make me a sandwich” may as well be “Take my wife ... please.”

Sure, it's easy for us to identify all the different parts just by context. You could pick out the author's name, the title, and the date it was published without a breaking a sweat. For computers, on the other hand, figuring all this out is at best tedious and in the worst case, pretty much impossible. With a trillion+ webpages out there, being able to programatically index meta-information is more important than ever.

So let's try this again, this time with RDF:


<rdf:Description>
<dc:identifier>http://www.nytimes.com/2008/05/26/business/media/26link.html?_r=1&oref=slogin</dc:identifier>
<dc:relation>Link By Link</dc:relation>
<dc:title>This Is Funny Only if You Know Unix</dc:title>
<dc:creator>Noam Cohen</dc:creator>
<dc:date>Mon May 26 00:00:00 -0700 2008</dc:date>
<dc:rights>Copyright 2008 The New York Times Company</dc:rights>
For a certain subset of Internet users, “Sudo make me a sandwich” may as well be “Take my wife ... please.”
</rdf:Description>


Now that's what I'm talking about! It may look weird with its tags showing, but it looks great when rendered properly by a browser. By adding a little context, programs like SearchMonkey can extract this information and organize it in meaningful ways.

One caveat that I'll get to later in this article: SearchMonkey uses a standard similar to RDF called DataRSS. The same general principles apply, so it's good to have an understanding of RDF first coming into it.

When you go to the SearchMonkey application builder, you have the choice to build either a Custom Data Service, which translates a web page or API call into DataRSS, or a Presentation Application, which builds on data services to build what you see when it's in use on a search result page. Presentation Applications are as simple as filling in the blanks in a template, so most of the work in developing with SearchMonkey is parsing data sources into the right format. That's where XSLT comes in.

XSLT

Now let's say we have this kind of information available to us. How do we actually get at it?

Well, lucky for you, SearchMonkey is built on a standard called XSLT, which was designed for just such a task.

XSLT, or eXtensible Stylesheet Language Transformations, is a W3C specification¹ for how to manipulate XML.

¹ XSLT and RDF are both specifications drafted by the World Wide Web Consortium, or W3C. Once drafted, a specification is handed down to vendors, who are _supposed_ to write software that follow the spec. XSLT is actually a pretty good example of this in practice, as opposed to CSS, which any web developer will tell you, isn't exactly standard across browsers.

XSLT is itself XML, which makes it familiar, if a bit verbose. Using XPath, a querying language inside the XSLT spec², you pick out nodes from the XML tree using selectors, and process them according to different templates that you specify.

² There's a long story behind this. Basically, XPath was motivated by XSLT and something called XPointer. XSLT itself was developed alongside another similarly-featured technology called XQuery. Wikipedia tells me there's something else called XLink, which is summed up eloquently by the title of the 2002 O'Reily article: "XLink: Who Cares?"
It boggles the mind that a topic with so many Xs could be so unbelievably boring.

XSLT in the Sandbox

Let's say you're compiling a list of your favorite cities. Because it seemed like a good idea at the time, you decided to format it in XML, along with additional information, like each city's country, population and area. You want to put this online, but don't want to do any more work as the list grows over time.

What's that about a database?
PHP?
Bah!

Might I suggest XSLT for this heavily rhetorical situation?

Anyway, here's the list of cities:

<?xml version="1.0" encoding="UTF-8"?>
<cities>
<city name="Kyoto">
<country>Japan</country>
<population>1464990</population>
<area units="km²">1779</area>
</city>
<city name="San Francisco">
<country>United States</country>
<population>764976</population>
<area units="km²">600</area>
</city>
<city name="Portland">
<country>United States</country>
<population>568380</population>
<area units="km²">376</area>
</city>
<city name="Bremen">
<country>Germany</country>
<population>548477</population>
<area units="km²">1679</area>
</city>
<city name="Doha">
<country>Qatar</country>
<population>339847</population>
<area units="km²">2574</area>
</city>
</cities>

Looks good. Now how do we turn this into HTML?

Even if you're not too familiar with XML, you'll notice that it bears a striking similarity to what you see when you view a webpage's source code. That's because HTML (or rather XHTML, to be precise) is XML. Since XSLT can transform an XML document into another, that means any XML document can be turned into XHTML and vice-versa.

First Iteration

Let's make a simple XSLT document to list the names of the cities in the list:


<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<html>
<head></head>
<body>
<h1>Cities</h1>
<ul>
<xsl:for-each select="//city">
<li><xsl:value-of select="@name"/></li>
</xsl:for-each>
</ul>
</body>
</html>
</xsl:template>
</xsl:stylesheet>


This should give you a good idea of the basic way XSLT works. First, as with any XML document, we declare our XML header. Once we've done that, we open up the main part of the document, the <xsl:stylesheet> element. The xmlns:xsl="http://www.w3.org/1999/XSL/Transform" specifies the namespace of the document as an XSL Transformation.

Within the stylesheet, we can define a number of <xsl:templates>, which trigger when the specified pattern is matched. This <xsl:template> matches on "/", or the root element, meaning that it triggers at the start of a document. This catch-all will always trigger first, so it's a good place to start constructing our XHTML.

xsl:for-each allows us to iterate through all elements that match a particular query. As I mentioned before, these matcher strings are XPath selectors. The two slashes in //city looks for every instance of a <city element, regardless of hierarchy. We could also have gotten the same result, in this case, if we had used /cities/city. Attributes of an element can be accessed with the prefix @. This is how we get the name of each city for our list.

You can get a more comprehensive guide to XPath at W3 Schools

Second Iteration

An unordered list of cities is alright, but what if we want to display all of our information?

Here's how we could output our information into a table:


<table>
<xsl:for-each select="//city">
<tr>
<td><strong><xsl:value-of select="./@name"/></strong></td>
<td><xsl:value-of select="./population"/></td>
<td><xsl:value-of select="./area"/> km<sup>2</sup></td>
<td><xsl:value-of select="./country"/></td>
</tr>
</xsl:for-each>
</table>


Just as before, we iterate through all of the cities using <xsl:for-each>, this time wrapping everything in a tr tag. Notice that the population, area, and country are not attributes, but rather the actual contents of their respective elements. ./ is just a more explicit convention that I prefer, meaning "this element" (just like a filepath in Unix).

Third Iteration

As a really quick example to throw out there, this final iteration shows off some advanced features that you might find useful.


<h1>My <xsl:value-of select="count(//city)"/> Favorite Cities</h1>
<table>
<thead>
<tr>
<th>Name</th>
<th>Population</th>
<th>Area (sq miles)</th>
<th>Country</th>
</tr>
</thead>
<tbody>
<xsl:for-each select="//city">
<xsl:sort select="./@name"/>
<tr>
<td><strong><xsl:value-of select="./@name"/></strong></td>
<td><xsl:value-of select="format-number(./population, '#,###')"/></td>
<td><xsl:value-of select="format-number(./area * 0.386102159, '#,###')"/></td>
<td><xsl:value-of select="./country"/></td>
</tr>
</xsl:for-each>
</tbody>
</table>


In our <h1&h1;, we count the number of elements from a particular query result by using the function count. There are a lot of useful built-in functions like this. Again, check out the W3 Schools reference for a good listing.

Another thing you can do is sort the results of a <xsl:for-each> by using the <xsl:sort> element. The specifier points the value you want to order on for each element. In our case, we're ordering our list alphabetically by the name of the city.

Finally, you might want to format or operate on a value using XSLT. In this example, we're doing both, by converting the area in km² into square miles by multiplying by the conversion ratio, and then taking that, and formatting it to use comma separators. format-number(), like count(), is a built-in XSLT function.

XSLT in Practice

Now that we're using XSLT with a fair degree of confidence, we're read to go back to our NY Times article. Our example showed how the article might be marked up using RDF, but since we want to use it with SearchMonkey, we'll want to translate that into DataRSS.

Unfortunately, in the real world, markup is wildly inconsistent across different websites. You can never tell how easy it will be to get to the information you want without looking under the hood at the page source code. If you're lucky, your website of interest gives special classes or ids to things you want to get at. If you're not so lucky, there may actually be no consistent solution for how to access information across the site.

In our case, the nytimes.com doesn't just have good markup, they even have some custom tags, like <NYT_BYLINE> that we can take advantage of. Here's what I came up with:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:template match="/">
<adjunctcontainer>
<adjunct id="smid:{$smid}" version="1.0">
<item rel="rel:Posting">
<meta property="dc:identifier"><xsl:value-of select="//meta[@name='articleid']/@content"/></meta>
<meta property="dc:type">News Story</meta>
<meta property="dc:title"><xsl:value-of select="//NYT_HEADLINE"/></meta>
<meta property="dc:creator"><xsl:value-of select="//NYT_BYLINE"/></meta>
<meta property="dc:date"><xsl:value-of select="//meta[@name='pdate']/@content"/></meta>
<meta property="dc:summary"><xsl:value-of select="//NYT_TEXT/p"/></meta>

<item rel="rel:Photo" resource="{//div[@class='image']//img/@src}">
<meta property="media:width"><xsl:value-of select="//div[@class='image']//img/@width"/></meta>
<meta property="media:height"><xsl:value-of select="//div[@class='image']//img/@height"/></meta>
</item>
</item>
</adjunct>
</adjunctcontainer>
</xsl:template>
</xsl:stylesheet>

This should give you a good idea of what to expect with DataRSS. For the most part, you'll be worrying about items and their contents. Items have a rel property that specifies exactly what's represented, whether it's a person, article, photo, etc. Here are the docs for rel properties.

Items have meta elements, with property attributes that work kind of like regular RDF tags. For example, dc:title is a property now, whereas in RDF it was its own tag. Check out the docs to find all of the available properties in DataRSS.

Once you've built a data service in SearchMonkey, you can start writing a presentation layer to pull from it. Since you'll be referencing values pulled from DataRSS, there's a big incentive for diligence in how you name elements. Don't obsess over it, but use what's available with rel's and properties as best as you can.

Now Start Building!

XSLT and RDF are pretty simple on their own. Scraping sites, however, is pretty tedious. Although we didn't build a SearchMonkey Application in this article, we covered all of the basics of how you might go about it.

Once you've gotten the hang of it, it starts to be pretty fun, so get out there and start hacking with SearchMonkey!

Mattt Thompson
YDN Tech Evangelist

Editor's note: Props and fond farewells to Mattt Thompson, who's heading back to college. Mattt, you'll be missed here at the YDN. So long and thanks for all the fishsticks.

Posted at 6:08 PM | Comments (2)

require 'yahoo-music'

include Yahoo::Music

Yahoo::Music.app_id = "..." # Put Your App ID Here


artist = Artist.new("Beirut") # Searches by name and uses first result

album = artist.releases.detect{|r| r.title == "Flying Club Cup"}

puts album.title

puts album.artist

puts "Release Date:" + album.released_on.strftime("%m/%d/%Y")

puts

puts "Tracks"

artist.tracks.each_with_index do |track, i|

  puts "\t%d %s \t%2d:%2d" % [i, track.title, track.duration / 60, track.duration % 60]

end