A monkey standing on the shoulders of giants

First is the easy answer: we get the data from our site owners! Yahoo!'s opinion is that nobody understands the content of a website more than the site owner. Through page markup (both RDFa and microformats), structured data feeds, custom data services created with the SearchMonkey developer tool, and XSLT rules written by third party tools, we'll take whatever data that site owners tell us is important about the content on their site. Even if you're the not the site owner, you can still submit custom data services to SearchMonkey Gallery. If approved, your extraction techniques will be applied by Yahoo Search for others to build applications upon.

What does SearchMonkey feed data look like?

To create a feed, first create an Atom feed of the URLs you want to annotate. Then within each Atom <entry>, add the metadata for the page. For example, assume you have restaurant listings already shown in Yahoo! Search, and you want to show the average review you've collected from your users. One of the entries in your feed would look like:

<y:adjunct version="1.0" name="local" xmlns:y="http://search.yahoo.com/datarss/">
  <y:item rel="dc:subject">
      <y:type typeof="vcard:VCard commerce:Business">
          <y:item rel="vcard:url" resource="http://local.yahoo.com/info-21328305-yahoo-incorporated-sunnyvale"/>
          <y:item rel="review:hasReview">
              <y:type typeof="review:Review">
                  <y:meta property="review:rating" datatype="xsd:decimal">4.5</y:meta>
                  <y:meta property="review:totalRatings" datatype="xsd:integer">32</y:meta>
              </y:type>
          </y:item>
      </y:type>
  </y:item>
</y:adjunct>

Which results in:

It's that easy! Create a couple million of these annotations (hopefully not by hand!), and you are ready to submit it to Site Explorer. When Yahoo processes the feed, your results will be enhanced and appear with 4.5 stars and "32 reviews". Users will see your search result, realize that you actually have reviews available from the page, and will go to your page to read reviews of this restaurant. That's not just theory — our lab monkeys have proved repeatedly that enhanced results actually increase traffic to sites.

How do I do this with XSLT?

Not everybody likes building a feed. If you're XML-savvy, you can use XSLT to instruct Yahoo! how to extract the components of your page that you want to appear in your enhanced results. As a quick example, we'll create a rule for Yahoo! Local. With your favorite XPATH plugin for Firefox (mine is XPather), you can quickly identify xpaths for Yahoo! to follow and extract data from. For example, the following xpath extracts "32 reviews":

//div[@id="yls-dt-tabs"]//li[2]//em

Insert that xpath into Yahoo-provided boilerplate XSLT, and we'll run that to extract the structured data from your site. If this approach seems pretty fragile because it assumes the HTML structure will not change... you're right. Some sites seldom change their HTML, and other sites are really good about naming the important nodes with unique ids. For those sites, XSLT rules are fairly resilient to changes in the page structure. For other sites that have high variation in their page structure or that are constantly making sweeping changes, XSLT extraction usually doesn't end very well. Fortunately, there's yet another approach.

But wait! There's more!

Next is the clever way to do things: it turns out that lot of the web has already been structured by the semantic web. Yahoo! already has billions of documents which have been annotated by either RDF or microformats. This semantic information provides some solid hints about the structured data on the page, which we happily use for SearchMonkey.

Annotating for the semantic web essentially involves tweaking your page markup to provide additional meaning about the elements on that page. The example below uses RDFa, but you can also provide the same information to Yahoo Search using microformats.

<div typeof="vcard:VCard commerce:Business"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
     xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
     xmlns:commerce="http://search.yahoo.com/searchmonkey/commerce/"
     xmlns:review="http://purl.org/stuff/rev#">
      <h1><span property="vcard:fn">Yahoo! Incorporated</span></h1>
      <span>
            <span>User Rating: 
                  <span property="review:rating">4.5</span> out of <span property="review:maxRating">5</span> stars 
                  (<span property="review:totalRatings">32</span> reviews).
            </span>
      </span>
</div>

When Yahoo sees the additional RDFa markup in the HTML, we extract the structured data complete with semantics. Your page is no longer a bag of unstructured words — Yahoo! now has information that helps us understand your page better.

But what about the rest of the web?

Finally, the answer from the big brains: magic. A lot of research has gone into "web data mining" both inside Yahoo, and at academic and corporate research institutions worldwide. Deep inside Yahoo! Research, tribes of monkeys are busy creating new technologies for Yahoo to extract objects out of web pages. I wish I could provide a few examples about this magic, but there is no way that I can summarize decades of named entity recognition research in five lines or less. For a good place to get started, see publications on TREC.

Any other methods that Yahoo! uses to extract structured data from web content?

Pizza — lots of pizza.

Are you a site owner interested in how you can build your site to make it easier for us to extract the structured data? We would love your participation.

Interested in being a monkey? We're also hiring!

Kevin Haas
Senior Engineering Manager, Yahoo! SearchMonkey

What is VoCamp? As the name indicates, VoCamp is influenced by BarCamp, although VoCamp's emphasis is tilted towards hands-on technical work and practical output. VoCamp provides a two day forum for vocabulary creation and discussions on the management of the Semantic Web. Unlike Semantic Web meetups which typically take just a few hours and focus on a single presentation, the VoCamp format is open and provides time to:

• talk about current issues with vocabularies and semantic interoperability
• work in small groups on a wide variety of problems
• get some real ontology hacking done

With both Yahoo! and now Google committed to supporting structured markup, it's never been a better time to come on in and help shape the future of the Semantic Web. And of course, if you have any direct feedback about how we're dealing with structured data, you'll have our ear. If you're in the Bay Area this June, please sign up on the VoCampSunnyvale2009 wiki page. Space is limited, but we will try to expand if necessary. The event is right after SemTech San Jose, so you won't have far to travel. Best of all, it's free!

Evan Goer
Yahoo! SearchMonkey Team

There are 2 things we'd like to announce regarding that:

1. We now support adr, geo and tag microformats. Yay!
2. In the name of progress, we have added an extra namespace to the IDs to make room for other types of extraction. Anyone who is using these for their microformat searches, can now find our structured data with these queries.

• hCard (searchmonkeyid:com.yahoo.page.uf.hcard)
• hCalendar (searchmonkeyid:com.yahoo.page.uf.hcalendar)
• hReview (searchmonkeyid:com.yahoo.page.uf.hreview)
• hAtom (searchmonkeyid:com.yahoo.page.uf.hatom)
• hResume (searchmonkeyid:com.yahoo.page.uf.hresume)
• adr (searchmonkeyid:com.yahoo.page.uf.adr)
• geo (searchmonkeyid:com.yahoo.page.uf.geo)
• tag (searchmonkeyid:com.yahoo.page.uf.tag)
• xfn (searchmonkeyid:com.yahoo.page.uf.xfn)
• RDFa (searchmonkeyid:com.yahoo.page.rdf.rdfa)
• eRDF (searchmonkeyid:com.yahoo.page.rdf.erdf)

And as before, you can look for hResumes with PHP on them, Geocoded pages involving Santa Clara, or RDFa pages related to SearchMonkey

Keep monkeying around.

Paul Tarjan
(|): Chief Technical Monkey :(|)

One of SearchMonkey's niftier features is the ability to create custom data services that uses XSLT to extract structured data from a site's markup. If you don't own the site in question, this is a great way to extract meaningful information and use it to build a SearchMonkey application. Conversely, if you do own the site in question, an custom data extractor can help you quickly prototype your SearchMonkey application before you add structured markup (microformats, eRDF, or RDFa) to your site.

Developers who already know XSLT can write a basic extractor with little trouble — the actual XSLT code for a SearchMonkey extractor usually isn't that complicated. But XSLT is an esoteric language, and the learning curve can be steep.

Which brings us to Intel Mash Maker, a nifty browser extension that enables you to extract structured information from disparate websites and create mashups on-the-fly, without having to write any code. Among other things, Mash Maker is a user-friendly creator of structured information... while SearchMonkey is a large-scale consumer of structured information. Some sort of integration seems natural, and indeed we are proud to announce that the Mash Maker team has just added built-in support for SearchMonkey, enabling you to autogenerate the XSLT code for a SearchMonkey application with a few clicks. How does it work? Check out the tutorials below:

• Creating SearchMonkey Custom Data Services with Intel Mash Maker, by Lucas Massuh of Intel (video)

• SearchMonkey Mash Maker Tutorial, by Chris Lindsey of Yahoo! (SlideShare presentation: English, Spanish, Portuguese; Scribd PDF: English, Spanish, Portuguese).

Currently, this experimental support for SearchMonkey exists in the Firefox version of Mash Maker only. We're hoping to continue to work with the folks down the road at Intel to find new ways to make it easier than ever to create structured information, so try it out and let us know what you think.

Evan Goer
Yahoo! SearchMonkey Team

Evan Goer
Yahoo! SearchMonkey Team

What is structured data, and why is structured data good for search? Traditional search engines crawl the web and extract what metadata they can: the page title, an autogenerated summary, the file size, the MIME-type, the last-updated date, and so on. However, this sort of analysis pales in comparison to what a human being can do simply by glancing at the page. A human can look at the words "Joe's Home Page" and infer, "ah, this page probably belongs to Joe," or look at an image and infer, "ah, that's probably a picture of Joe, the owner of the page." That's easy enough for humans... but what if the search engine could pick out this info and display it directly in the search result?

RDFa relies on using attributes to embed structured data in XHTML. These attributes are not valid in HTML 4, but the W3C has provided an XHTML DTD to validate against. The following example illustrates a simple home page marked up with RDFa data (in bold):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"
          "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:dc="http://purl.org/dc/elements/1.1/"
      xmlns:foaf="http://xmlns.com/foaf/0.1/"
      lang="en" xml:lang="en">
<head>
  <title>The Amazing Home Page of Joe Smith</title>
</head>
<body>
  <h1 property="dc:title">Joe's Home Page</h1>
  <div rel="foaf:maker">
    <h2 property="foaf:name">Joe Smith</h2>
    <div rel="foaf:depiction" resource="http://joesmith.org/images/jsmith.png">
      <img src="/images/jsmith.png" alt="Smiling headshot of Joe" />
      <p property="dc:rights">Creative Commons Attribution 3.0 Unported</p>
    </div>
  </div>
</body>
</html>

In this page, the designer has explicitly stated that the image is a "depiction of the person who made the web page." Adding this information as RDFa can potentially benefit many applications. In the case of Yahoo!, we've designed our search index to extract and store this information.

RDFa support has already enabled some interesting new SearchMonkey applications. For instance, Creative Commons has recently started to deploy RDFa across the web in the form of copyright and licensing information. Every time a Creative Commons user selects a CC license, the generated HTML badge contains RDFa markup indicating the nature of the license. The Creative Commons Infobar uses this data to selectively trigger on pages that declare their license using structured markup:

To get started with RDFa:

• Learn the basics with the W3C RDFa Primer
• Dive into the details with the full RDFa Specification
• Join the community at the RDFa homepage
• Test your structured markup skills with the RDFa Distiller
• Filter on RDFa in Yahoo! searches with the searchmonkeyid:com.yahoo.rdf.rdfa parameter
• Start displaying RDFa to millions of users with the SearchMonkey developer tool

Evan Goer
Yahoo! SearchMonkey Team

The original SearchMonkey post contains examples of both the old and the new SearchMonkey Enhanced Result templates. In this posting, we'll just show examples of the new design. First, an Enhanced Result for HowStuffWorks.com that displays a result with a full abstract:

The first design change you'll notice is that images appear to the right, rather than the left. Our user testing has shown that moving images to the right improves the usability of the overall result. As for links, they now appear in a horizontal row, rather than as a vertical column next to the image. Not only are the deep links are more discoverable when presented separately from the image, but moving the links into a horizontal row provides more space for the summary and key/value pairs.

Now let's take a look at another real-world example from CitySearch, this one displaying a result with key/value pairs:

As in the example above, the image is shifted to the right. The key/value pairs have replaced the abstract, lying just below the deep links.

Careful observers will note that the key and value are now the same color — a minor change from the old template, where the keys were a lighter shade of grey. We found that keys were actually more discoverable when displayed in the same shade as the value.

The SearchMonkey team is confident improves performance, engagement, and discoverability across the board for users and developers. If you have feedback on the new templates, please let us know here, or leave us a comment in the SearchMonkey forum.

Evan Goer
Yahoo! SearchMonkey Team

• how SearchMonkey is helping to motivate large site owners to use semantic data
• the importance of shared vocabularies for application interoperability
• the emergence of automated tools for extracting semantic data, such as Calais

This podcast is available as a stream or MP3 download. Enjoy!

Evan Goer
Yahoo! SearchMonkey Team

And why not? SearchMonkey is pretty darn cool.

It lets you enhance the appearance of search results for your favorite sites. So the next time you need to look up, say, restaurants, a SearchMonkey app can distill all of the important information, like location, price range, and rating all into one place, right there in your search results.

A ton of people have been tinkering with SearchMonkey since it launched in May. One of the main reasons for this (aside from how cool it is...and never mind the $10,000 contest they held recently), is how easy it is to pick up and start developing with.

In this article, I'll go over XSLT and RDF--two of the fundamental concepts that power SearchMonkey. If you're looking to build your first app or you've built a few and want to get more out of it, you'll definitely want to read on.

First, Mark Birbeck of webBackplane has released a set of SearchMonkey tutorials that cover the DataRSS format, creating a DataRSS XSLT extractor, and creating a presentation application. Mark's specific focus is on government websites and expanding the ways these websites can share information with citizens. In fact, one of the reasons Mark is so interested in SearchMonkey is that SearchMonkey apps enable people to see what's possible when you embed RDFa in your pages. Mark also points out that custom data services are an excellent way to prototype these kinds of features before you go to the trouble of changing your markup.

If that weren't enough, Chris Lindsey of the Yahoo! Search Editorial team has posted a two-part SearchMonkey tutorial series on How to Build a Custom Data Service [PDF] and How to Build a SearchMonkey app (Infobar) [PDF]. These step-by-step tutorials are written with the newbie developer in mind, and cover various pitfalls that you might encounter when you're first learning how SearchMonkey works.

Finally, it's worth noting that a number of people have posted SearchMonkey-related presentations on slideshare.net, including Chris Heilmann, Neil Crosby, and David Berkowitz. If you've got SearchMonkey tutorials or interesting apps to share, feel free to let us know.

Evan Goer
Yahoo! SearchMonkey Team

In the past few months, SearchMonkey developers have told us they'd like to use Enhanced Results for site search. Yahoo! and other search engines have long had a site restrict operator (e.g. site:anysite.com) and other site search tools, but we decided to launch a new capability that lets you add a query parameter that automatically turns on the SearchMonkey Enhanced Result for the site you're searching. This is important for site owners because it makes it easier for their communities to get more complete answers when they search on Yahoo! Search.

This new parameter will work with any app that's in the Yahoo! Search Gallery as well as any official app. (To make an app official, a site owner just needs to authenticate their site using Site Explorer and then associate their app with their site when they make it sharable in the Developer Tool)

How it works
To use this functionality, you just need to append a few parameters to a typical Yahoo! Search query string. Here's a quick example:

• Start with a standard query string: http://search.yahoo.com/search?q=u2
• Add the site restrict parameter: &vs=last.fm
• Add the SearchMonkey app ID parameter (case sensitive): &sm=xho
• Final result: http://search.yahoo.com/search?q=u2&vs=last.fm&sm=xho

This query string can be generated using whatever mechanism you choose, including a simple search box on your site or blog. It works with both Infobars and Enhanced Results -- as long as the app is either in the Gallery or is official. If you accidentally try a SearchMonkey app along with a site restriction that doesn't match, the results will look like a typical site-restricted search. Here are a few more examples:

Site: Wikipedia
App: http://gallery.search.yahoo.com/application?smid=knb
SearchMonkey for site search query: http://search.yahoo.com/search?q=george+washington&vs=wikipedia.org&sm=knb
Try a search:

Site: Yahoo! Answers
App: http://gallery.search.yahoo.com/application?smid=ylc
SearchMonkey for site search query: http://search.yahoo.com/search?q=can+pigs+fly?&vs=answers.yahoo.com&sm=ylc
Try a search:

Site: java.sun.com
App: http://gallery.search.yahoo.com/application?smid=Zq0
SearchMonkey for site search query: http://search.yahoo.com/search?p=exception&vs=java.sun.com&sm=Zq0
Try a search:

Before we continue work on this feature, please let us know what you think below or on the Developer group. We'd be particularly interested to hear what else site owners would like to be able to customize in order to make SearchMonkey a more valuable site search tool.

The SearchMonkey team

Last month we opened up the Yahoo! Search Gallery to showcase all of the useful SearchMonkey applications that have been built by developers, site owners and Yahoo!. Today, we’re turning on a few of those applications for all users. Now, the Yelp, Yahoo! Local and LinkedIn Enhanced Results will automatically appear in the search results, eliminating the need for users to go into the Search Gallery to add them.

Why did we start with these applications? Before making an application “default on” we require a few things: access to the site’s structured data through semantic markup or a data feed, a well-designed and broadly useful application, and positive user metrics. To understand how a SearchMonkey app affects user metrics, we generally expose a small percentage of our users to a default-on experience and measure if and how it changes their usage. We started with Yelp, LinkedIn, and Yahoo! Local because they were among our first partners to share structured data. Our tests uncovered that users found these apps useful; in fact, in some cases, we saw a lift in click-through rate of as high as 15 percent.

In addition to testing a "default on" treatment, we also tested giving users the ability to add the LinkedIn Enhanced Result directly from the search results page. We'll continue to use this treatment as another way to promote high quality SearchMonkey apps to users.

While these are the first apps to be automatically included in the search results, they will certainly not be the last. We'll continue to work with our SearchMonkey developers to increase the exposure of other high-quality applications to the search results page in the months to come (so, make sure to submit your applications to the Gallery). Making it easier to find and add SearchMonkey apps is an important step in improving and enriching the search experience for our users.

In addition to turning these Enhanced Results on by default, we've also added "share-with-a-friend" functionality. By clicking the envelope icon on any Enhanced Result, users can now quickly send an email to their friends to share the app.

Let us know how these Enhanced Results are working and others you’d like to see in the comments below!

Amit Kumar
Director, Product Management
Yahoo! Search

Original Enhanced Result Designs (Abstract, Key/Value Pair)

New Enhanced Result Designs (Abstract, Key/Value Pair)

Key differences include:

• Images appear to the right of the abstract, rather than the left. Our latest eye-tracking studies indicate that moving the image helps users selectively discover it — without disrupting their ability to scan the search result page. While we know that users do like relevant images, we also know that images can increase the risk that the result resembles an advertisement. Shifting the image to the right reduces the perception of the result as an advertisement, and caused users to report noticing the images more. The new design also helps users engage the image only when they feel it is relevant.

• Links appear in a horizontal row, rather than as a vertical column next to the image. Research indicates that deep links are more discoverable when presented separately from the image. Additionally, moving the links into a horizontal row provides more real estate for the summary and key/value pairs.

In a subsequent post, one of our researchers will dive into the data behind this redesign in much more detail. Meanwhile, stay tuned — we'll let you know when the new designs are about to go live!

Evan Goer
Yahoo! SearchMonkey Team

Search Monkey arrived in Paris with the sound of flying toy monkeys screeching as they flew through the La Cantine café. The festive, attentive group gathered to hear about the new Yahoo! Search feature and to fling the little guys across the room.

La Cantine is a cross between an Internet Café and a co-op workspace. Its central Paris location provides space to collaborate, learn, share, and explore the future of technology.

Photo credit Sophie Major

The Yahoo! Developer Network and local Yahoos joined forces to explain Search Monkey; an open interface for providing enhanced search result presentations. Participants were able to start building Search Monkey applications in small groups as well as learn about Microformats.

Search Monkey Applications

Search Monkey allows any developer to create enhanced presentations for a web site when it appears in Yahoo! Search. The standard result displays the page title, description, and web site address. A Search Monkey powered result could contain photographs, address, contact information, deep links to particular pages, and additional information from related pages. This makes search result pages more engaging for users.

Search Monkey applications do this by defining small bits of data that should fill the space. Search Monkey automatically grabs microformatted data on the page, as well as providing the standard Yahoo! Search indexed information. The programmer can also define custom data by scraping the desired page or using web services.

Search Monkey opportunities in France

Museums - Provide images, titles, artist, date, and other information for their objects when they appear in search results.
Traffic - Provide traffic conditions, rates, times, and tourist information when a city or train or metro route appears in the search result pages.
Restaurants - Directories could provide ratings, addresses, phone numbers, and summaries.
Theater - Cinema, Theater, Opera results include times, synopsis, rating, and location information.
Sports - Tour de France and other sports include player information, team scores, upcoming events, and standings.

Ted Drake
Web Developer - Global Finance

It turns out that execution speed of a SearchMonkey is key. To make the SearchMonkey Gallery a presentation monkey such as Tweet has to complete within a fraction of a second. Any call to fetch 3rd party takes too long to satisfy this requirement. Certainly calling Twitter's API whose fluctuating response times are all over the map.

Secondly, Twitter's profile API call takes a user ID, which first has to be extracted from Yahoo!'s indexed data. An additional data SearchMonkey can do that and whose output is the input to Tweet's profile feching data monkey. However, this chaining of data monkeys makes Tweet only slower.

Fortunately, Bart hit on a really clever solution: a mashup with Google App Engine, which acts as a simple proxy cache for Twitter data, which SearchMonkey can then consume. The result (after also adding Bart's own FriendNet infobar app):

Not only is the caching a nifty way to smooth out the API response times, but it also helps reduce the number of (rate-limited) API calls required. Read more about it at Bart's place.