Y!OS YQL

Need Help with Remote and Local Filters

Fri, 05 Feb 2010 00:50:23 -0800

Hello. I am trying to figure out how to construct my statement when I am getting data from a JSON feed.

1) I need to select items from the feed that are less than 3 minutes old
2) I need to displayt only certain items from the table

For example:
from the table below, how do I display only and ?

5555
john
1265330712

Syntax error(s) [line 0:-1 expecting statement_ql at ''] error message

Thu, 04 Feb 2010 19:37:04 -0800

Hi all,
I am having this error while running my own datatables.

http://developer.yahoo.com/yql/console/?q=...%27%3B%20%20%0A

USE 'http://www.urbanesia.com/yql/search.xml';

Syntax error(s) [line 0:-1 expecting statement_ql at '']

I have search everywhere and I can't find any solution to this. If you search the word 'statement_ql', there's a very limited result and not helping at all. Did anybody stumble upon this and achieve result? any help is much appreciated.

The urbanesia API:
http://api.urbanesia.com/search?output_typ...1000&row=50

Syntax error(s) [line 0:-1 expecting statement_ql at ''] error message

Thu, 04 Feb 2010 19:37:04 -0800

Got the answer:
http://twitter.com/nomoredistance/status/8663768529

nomoredistance @nataliardianto you need an actual statement in the end. Try: USE "http://devel.urbanesia.com/yql/search.xml" AS foo; SELECT * FROM foo;

This is totally silly. I previously copy paste an example with new line in it, and it didn't work! I then write it down in one single line, and voila! Hope this helps someone since the search result is so scarce when searching this keyword.

QUOTE (Natali @ Feb 4 2010, 07:37 PM) <{POST_SNAPBACK}>

Hi all,
I am having this error while running my own datatables.

http://developer.yahoo.com/yql/console/?q=...%27%3B%20%20%0A

USE 'http://www.urbanesia.com/yql/search.xml';

Syntax error(s) [line 0:-1 expecting statement_ql at '']

I have search everywhere and I can't find any solution to this. If you search the word 'statement_ql', there's a very limited result and not helping at all. Did anybody stumble upon this and achieve result? any help is much appreciated.

The urbanesia API:
http://api.urbanesia.com/search?output_typ...1000&row=50

Syntax error(s) [line 0:-1 expecting statement_ql at ''] error message

Thu, 04 Feb 2010 19:37:04 -0800

We should add support for new line between those statements. I'll take this as a feature enhancement as well.

Sam

Search.suggest lacks field names

Wed, 03 Feb 2010 05:29:37 -0800

The sample query:

select * from search.suggest where query="madonna"

produces a plain array in query.results.Result, not an array of named elements, no field names at all. Since the returned fields are unnamed, there is no way to sort them or do any operation since there is no way to refer to them.

Search.suggest lacks field names

Wed, 03 Feb 2010 05:29:37 -0800

Actually, any reply that has just one field in the reply returns no field names. So, the structure of the reply changes depending on whether you are asking one field or many. This lack of consistency in the reply makes it harder to use as a generic tool. If you provide your users with a series of checkboxes to select which fields to pick, when the number of fields selected goes down to 1, your processing turns out different than when there is more than one? That is not good

Text encoding and YQL

Mon, 01 Feb 2010 07:12:28 -0800

I'm trying to build a pipe and I'm stuck with a YQL issue. I need a query like this:

CODEBOX

select * from html where url="some_uri" and xpath='//div[@class="some_class"]/table/tr/td[p="some_ text"]/h2/a'

The query runs fine when some_text contains just ascii characters. But it returns null results --which is wrong-- when some_text contain non ascii characters, like extended latin characters, or Greek characters which is what I need to use it with. The page where the query runs on at some_url has a utf-8 encoding and shows up just fine.

I've set up a test file at http://mytests.atwebpages.com/test2.html duplicating the structure of the source page I'm working with.

This query returns 2 items as expected:

CODEBOX

select * from html where url="http://mytests.atwebpages.com/test2.html" and xpath='//div[@class="article-list-entry"]/table/tr/td[p="Nikos"]/h2/a'

The following two return null, whereas they should return 1 item each:

CODEBOX

select * from html where url="http://mytests.atwebpages.com/test2.html" and xpath='//div[@class="article-list-entry"]/table/tr/td[p="Γιάννης Αγιάννης"]/h2/a'

CODEBOX

select * from html where url="http://mytests.atwebpages.com/test2.html" and xpath='//div[@class="article-list-entry"]/table/tr/td[p="ÁÉÍÓÚÅÄandÖ"]/h2/a'

The former contains ascii characters in the [p="match"], the latter two do not...

What is going wrong? Is there some limitation with non-ascii text values? Do I need to add a (convert?) function or a parameter somewhere?

I'm stuck with this for quite a while now. Thank you in advance for any help.

Text encoding and YQL

Mon, 01 Feb 2010 07:12:28 -0800

no ideas, anyone?

Returning YQL query in correct publish date order, seems to order on last index currently.

Thu, 28 Jan 2010 08:23:23 -0800

Hi All,

I'm currently trying to get some results to come back to me in date order.

This is almost working perfectly.

select title,abstract,url,date from search.web(0) where query="search term" and sites="www.independent.co.uk" | sort (field='date') | reverse()

The problem is that the dates that are brought back are not the actual dates that the articles were published on, rather they seem to be the dates that they were last indexed on.

I have had more success with change it from search.web to search.news , however now it only brings back results for the last week or two.

Using the yahoo news search brings back results up to 2 months ago, so this is an improvement.

http://uk.news.search.yahoo.com/

is there a way to get the same results as I can here, or ideally just get the first query to work based on the same date system as the news search?

Thanks for any help,

Chris B

Returning YQL query in correct publish date order, seems to order on last index currently.

Thu, 28 Jan 2010 08:23:23 -0800

QUOTE (rhythm.optician @ Jan 28 2010, 08:23 AM) <{POST_SNAPBACK}>

Hi All,

I'm currently trying to get some results to come back to me in date order.

This is almost working perfectly.

select title,abstract,url,date from search.web(0) where query="search term" and sites="www.independent.co.uk" | sort (field='date') | reverse()

The problem is that the dates that are brought back are not the actual dates that the articles were published on, rather they seem to be the dates that they were last indexed on.

I have had more success with change it from search.web to search.news , however now it only brings back results for the last week or two.

Using the yahoo news search brings back results up to 2 months ago, so this is an improvement.

http://uk.news.search.yahoo.com/

is there a way to get the same results as I can here, or ideally just get the first query to work based on the same date system as the news search?

Thanks for any help,

Chris B

In your case, it might be easier to write an open data table around the RSS feed that comes from uk.news.search.yahoo.com since search.news only gives back 2 weeks worth of data.

I'd write an open data around the following URI

http://uk.news.search.yahoo.com/search/new...tart=1&b=11

-- Nagesh

Inconsistency with xml/json

Sun, 24 Jan 2010 19:18:54 -0800

When i run this query with xml i get the desired result but when i run it with json i only get part of the result, why is there a difference?

CODE

select * from html where url="http://www.onemanga.com/Ii_Hito/" and xpath='//*[@id="content-side"]/div/div/div[2]/p'

Inconsistency with xml/json

Sun, 24 Jan 2010 19:18:54 -0800

QUOTE (sgayda2 @ Jan 24 2010, 07:18 PM) <{POST_SNAPBACK}>

When i run this query with xml i get the desired result but when i run it with json i only get part of the result, why is there a difference?
CODE
select * from html where url="http://www.onemanga.com/Ii_Hito/" and xpath='//*[@id="content-side"]/div/div/div[2]/p'

This is because there is no easy way to associate the text elements between XML and JSON. We've discussed this problem before and the best solution today is to use the hybrid version where you ask for JSON in XML format

All you have to do is issue the query with an additional query parameter called callback ensuring that the format parameter is set to 'xml'. THis gets you back the XML data in the JSON envelope which you can then directly add to your webpage by setting innerhtml

CODE

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fwww.onemanga.com%2FIi_Hito%2F%22%20and%20xpath%3D'%2F%2F*%5B%40id%3D%22content-side%22%5D%2Fdiv%2Fdiv%2Fdiv%5B2%5D%2Fp'&format=xml&callback=foo

hope that helps
-- Nagesh

YQL plans with Twitter deprecating search in favor of Streaming?

Sat, 23 Jan 2010 21:46:19 -0800

Twitter is asking developers with high-volume Search applications to migrate to the Streaming API. See http://borasky-research.net/2010/01/18/twi...do-they-mean-2/ and http://groups.google.com/group/twitter-api...8c713bb63fac24c for the details. Are there any plans to "wrap" the Twitter Streaming API so it can be cached / buffered and accessed via YQL?

Google search results...

Thu, 21 Jan 2010 22:15:53 -0800

I've been using the Google search datatable with no issues. However, recently it has started returning only 4 results (despite requesting more) and defaulting the search language to french. Any ideas?

TIA

Google search results...

Thu, 21 Jan 2010 22:15:53 -0800

QUOTE (skwasha @ Jan 21 2010, 10:15 PM) <{POST_SNAPBACK}>

I've been using the Google search datatable with no issues. However, recently it has started returning only 4 results (despite requesting more) and defaulting the search language to french. Any ideas?

TIA

My guess is that you'll gain some additional insight as to why the limitation is 4 records by going to the following:

New Parameter for Sever Side API Calls

I'm not sure why the default is French, but you can "fix" that by adding the gl and hl criteria as in the following statement:

select * from google.search (4,8) where q = "pizza" and gl="en" and hl="en"

Notice that you can pull the next four results using (4,8) and so on. Hope that helps some...

Yahoo Finance Intraday API

Thu, 21 Jan 2010 12:44:19 -0800

Hello,

I'm currently developing a large web application where my client wants to analyze intraday minute data of approx. 500 assets.

I did a basic calculation: 60 minutes x 500 assets = 30.000 queries in just 1 hour.
Which is I believe too much for YQL limits.

Is Yahoo offering a premium API where there is no YQL query limits?

Yahoo Finance Intraday API

Thu, 21 Jan 2010 12:44:19 -0800

Anybody?

Yahoo Finance Intraday API

Thu, 21 Jan 2010 12:44:19 -0800

QUOTE (Black @ Jan 22 2010, 03:59 AM) <{POST_SNAPBACK}>

Anybody?

Please drop me an email at yql-questions at yahoo-inc.com with a few more details on what the application is and what you'd be doing with YQL.

We are very able to support higher limits and we should be able to help you out.

Jonathan

HELP: why xmlHttp.status = 0

Thu, 21 Jan 2010 01:08:48 -0800

Please help me .
This problem abused me for months.

where I copy the URL into Firefox's location bar to visit website ,the website responses correctly,but when Using Ajax with same URL, the xml.status only is 0,just once. i Google lots of staff of which no one can solve it.

below is my code.

CODE

var consumer_Key ="dj0yJmk9WFNzYXluTW5IQW1vJmQ9WVdrOVdEVjRhbU5YTXpZbWNHbzlNVEV4TXpRd09EZ3lNQS0t
JnM9Y29uc3VtZXJzZWNyZXQmeD1kNw--";
var consumer_Secret ="53f395972cef451b2762145f7afe1dab27cfd8db";

var getReqToken =function(){
	var url = "https://api.login.yahoo.com/oauth/v2/get_request_token";  
    url+="?oauth_nonce="+(new Date().getTime());  
	url+="&oauth_timestamp="+(Math.round(new Date().getTime()/1000)); //time zone
	url+="&oauth_consumer_key="+consumer_Key;
	url+="&oauth_signature_method=plaintext"; 
	url+="&oauth_signature="+consumer_Secret+"%26";  
	url+="&oauth_version=1.0";
	url+="&xoauth_lang_pref=en-us";  
	url+="&oauth_callback=http://www.the631.com/mymeme";
	
	var query ={
			q:url,
			asyn:true,
			type:"private",
			data:"liang.wang;",
			proxy:true,
			cbFunction:function(reply){
				alert(reply);
			}
		};
		new Ajax(query); 
}

//
var Ajax = function(options){
 		var xmlHttp;
		if (window.ActiveXObject) {
        	xmlHttp = new ActiveXObject("Microsoft.XMLHTTP");
        }else if (window.XMLHttpRequest) {
       		 xmlHttp = new XMLHttpRequest();
        }
		var url = generateQuery(options);
		var asyn = options.asyn==undefined?true:options.asyn;
		xmlHttp.open("GET", url, asyn);
		if(options.proxy){
			xmlHttp.withCredentials =true;
			xmlHttp.setRequestHeader("Proxy-Authorization","Basic bGlhbmcud2FuZzo4ZCFnV3hMMA==");
		}

	    xmlHttp.onreadystatechange = function(){
			if (xmlHttp.readyState == 4) {//completed
				if(xmlHttp.status==200 || xmlHttp.status == 304){//success 
					
					var args; 
					if(!options.format || options.format.toLowerCase()=="text"){
						args =  xmlHttp.responseText;
					}else{
						if("json"==options.format.toLowerCase()){
							args= JSON.decode( xmlHttp.responseText);	
						}else if("xml"==options.format.toLowerCase()){
							args = xmlHttp.responseXML;
						}
					}
					var callback = options.cbFunction;
					if(typeof(callback)=="function"){
						callback(args);
					}else{
						window[callback](args);
					}
				}
			}
		};
	    xmlHttp.send(options.data==undefined?"":options.data);
		//firefox does't work well when asyn=false;
		if(!asyn){
			if (xmlHttp.readyState == 4) {//completed
				if(xmlHttp.status==200 || xmlHttp.status == 304){//success
					
					var args;
					var type = options.format.toLowerCase();
					if("json"== type){
						args= JSON.decode(xmlHttp.responseText);	
					}else if("xml"==type){
						args =xmlHttp.responseXML;
					}else{
						args= xmlHttp.responseText;
					}
					var callback = options.cbFunction;
					if(typeof(callback)=="function"){
						callback(args);
					}else{
						window[callback](args);
					}
				}
			}
	    }  
 }

HELP: why xmlHttp.status = 0

Thu, 21 Jan 2010 01:08:48 -0800

God damn China.The net slowness kills me so laggy.

I am sorry duplicate posting. i am not mean to

HELP: why xmlHttp.status = 0

Thu, 21 Jan 2010 01:08:48 -0800

God damn China.The net slowness kills me so laggy.

I am sorry duplicate posting. i am not mean to

HELP: where xmlHttp.status = 0

Thu, 21 Jan 2010 01:06:48 -0800

CODE

var consumer_Key ="dj0yJmk9WFNzYXluTW5IQW1vJmQ9WVdrOVdEVjRhbU5YTXpZbWNHbzlNVEV4TXpRd09EZ3lNQS0t
JnM9Y29uc3VtZXJzZWNyZXQmeD1kNw--";
var consumer_Secret ="53f395972cef451b2762145f7afe1dab27cfd8db";

var getReqToken =function(){
	var url = "https://api.login.yahoo.com/oauth/v2/get_request_token";  
    url+="?oauth_nonce="+(new Date().getTime());  
	url+="&oauth_timestamp="+(Math.round(new Date().getTime()/1000)); //time zone
	url+="&oauth_consumer_key="+consumer_Key;
	url+="&oauth_signature_method=plaintext"; 
	url+="&oauth_signature="+consumer_Secret+"%26";  
	url+="&oauth_version=1.0";
	url+="&xoauth_lang_pref=en-us";  
	url+="&oauth_callback=http://www.the631.com/mymeme";
	
	var query ={
			q:url,
			asyn:true,
			type:"private",
			data:"liang.wang;",
			proxy:true,
			cbFunction:function(reply){
				alert(reply);
			}
		};
		new Ajax(query); 
}

//
var Ajax = function(options){
 		var xmlHttp;
		if (window.ActiveXObject) {
        	xmlHttp = new ActiveXObject("Microsoft.XMLHTTP");
        }else if (window.XMLHttpRequest) {
       		 xmlHttp = new XMLHttpRequest();
        }
		var url = generateQuery(options);
		var asyn = options.asyn==undefined?true:options.asyn;
		xmlHttp.open("GET", url, asyn);
		if(options.proxy){
			xmlHttp.withCredentials =true;
			xmlHttp.setRequestHeader("Proxy-Authorization","Basic bGlhbmcud2FuZzo4ZCFnV3hMMA==");
		}

	    xmlHttp.onreadystatechange = function(){
			if (xmlHttp.readyState == 4) {//completed
				if(xmlHttp.status==200 || xmlHttp.status == 304){//success 
					
					var args; 
					if(!options.format || options.format.toLowerCase()=="text"){
						args =  xmlHttp.responseText;
					}else{
						if("json"==options.format.toLowerCase()){
							args= JSON.decode( xmlHttp.responseText);	
						}else if("xml"==options.format.toLowerCase()){
							args = xmlHttp.responseXML;
						}
					}
					var callback = options.cbFunction;
					if(typeof(callback)=="function"){
						callback(args);
					}else{
						window[callback](args);
					}
				}
			}
		};
	    xmlHttp.send(options.data==undefined?"":options.data);
		//firefox does't work well when asyn=false;
		if(!asyn){
			if (xmlHttp.readyState == 4) {//completed
				if(xmlHttp.status==200 || xmlHttp.status == 304){//success
					
					var args;
					var type = options.format.toLowerCase();
					if("json"== type){
						args= JSON.decode(xmlHttp.responseText);	
					}else if("xml"==type){
						args =xmlHttp.responseXML;
					}else{
						args= xmlHttp.responseText;
					}
					var callback = options.cbFunction;
					if(typeof(callback)=="function"){
						callback(args);
					}else{
						window[callback](args);
					}
				}
			}
	    }  
 }

Difference in data returned in XML vs JSON

Thu, 21 Jan 2010 00:14:45 -0800

I've created a YQL query to scrape some text off this page: http://www.conquercancer.ca/site/TR?px=181...amp;pg=personal. Specifically, I'm after the "Achieved: $0.00" string. My YQL statement is:

SELECT * FROM html WHERE url="http://www.conquercancer.ca/site/TR?px=1817895&fr_id=1331&pg=personal" AND xpath="//p[@class='Smaller']"
(permalink)

The HTML for this item is:

CODE

<p class="Smaller">

Goal: $2,500.00<br/>

Achieved: $0.00
</p>

If I select XML as the return format, I get:

CODE

<results>
        <p class="Smaller">Goal: $2,500.00                       <br/>
Achieved: $0.00</p>
    </results>

If I select JSON, I only get the text after the
:

CODE

  "results": {
   "p": {
    "class": "Smaller",
    "br": null,
    "content": "\n                       Achieved: $0.00"
   }

Getting the Goal: $2,500.00 string isn't essential, but I'm curious as to why I'm getting a different result. I've tried pulling the

surrounding the

, but it's returning the same thing:

CODE

  "results": {
   "div": {
    "class": "controlArea",
    "p": [
     {
      "strong": "Mr. Graham Ballantyne"
     },
     {
      "img": {
       "alt": "0 percent of goal achieved.",
       "border": "0",
       "height": "160",
       "src": "../images/thermometer-0.gif",
       "width": "65"
      }
     },
     {
      "class": "Smaller",
      "br": null,
      "content": "\n                       Achieved: $0.00"
     },
     {
      "class": "centered",
      "a": {
       "href": "https://secure2.convio.net/cfrca/site/Donation2?df_id=1303&PROXY_ID=1817895&PROXY_TYPE=20&FR_ID=1331&JServSessionIdr004=bmdzmim1t2.app14a",
       "title": "Make a gift to support Mr. Graham Ballantyne"
      }

Any ideas?

Difference in data returned in XML vs JSON

Thu, 21 Jan 2010 00:14:45 -0800

The XML->JSON conversion can be lossy, especially in specifically this case where you have more than one content node at the same level so you are only seeing the second one. If you are returning HTML I suggest that you use XML instead. You can even do that with JSONP by specifying a callback but setting the format to xml.

Sam

YQL and dapper: a beautiful couple

Thu, 14 Jan 2010 00:07:04 -0800

Hi,

YQL Open Tables and Dapper make a beautiful couple for html scraping. The disadvantage of Dapper for creating uniform API's on existing webpages is that it doesn't allow for fine-grained tweaking of the contents, like e.g. string matching or logical operations on the contents. The disadvantage of YQL Open Tables (with Javascript execute) is that extracting relevant fields from (badly structured) html is not that easy to program, even using the flexible E4X xml object operations.

Combining both services solves both disadvantages! I use Dapper to create a high-level API that extracts the relevant portions of the webpage and this is fed into an Open Table as xml. The Open Table allows for detailed analysis and formatting of the data. The coding is minimal.

The only thing that worries me, is the reliability of Dapper. It's reporting server overload quite often. Is Dapper really the only visual html scraping tool around?

Groeten,
Vic

YQL and websites that use cookie-based authentication

Wed, 13 Jan 2010 09:04:22 -0800

I'm interesting in using YQL to scrape data from a website that uses cookie-based authentication. Is this possible? I've only seen examples of YQL used with OAuth.

YQL and websites that use cookie-based authentication

Wed, 13 Jan 2010 09:04:22 -0800

QUOTE (mightybao @ Jan 13 2010, 09:04 AM) <{POST_SNAPBACK}>

I'm interesting in using YQL to scrape data from a website that uses cookie-based authentication. Is this possible? I've only seen examples of YQL used with OAuth.

Yes, but you have to do the cookie management yourself by reading and writing cookie headers in an open data table.

Sam

YQL and websites that use cookie-based authentication

Wed, 13 Jan 2010 09:04:22 -0800

QUOTE (Sam @ Jan 13 2010, 12:20 PM) <{POST_SNAPBACK}>

Yes, but you have to do the cookie management yourself by reading and writing cookie headers in an open data table.

Sam

Are there any code examples of this? Where would I put the cookie management code, in the execute element I take it?

Also, I would have to pass the cookie back to the user in the YQL request so they could use it in other requests, correct? Or is this something that I could store in the Open Data Table?

Finance Portfolio

Mon, 11 Jan 2010 20:18:00 -0800

Hi,
Exists there a YQL table (or other API) where I can download my personal portfolio data from Yahoo! Finance?
I searched for it, but didn't find anything. Maybe someone knows more?

Thanks

Finance Portfolio

Mon, 11 Jan 2010 20:18:00 -0800

QUOTE (Maas @ Jan 11 2010, 08:18 PM) <{POST_SNAPBACK}>

Hi,
Exists there a YQL table (or other API) where I can download my personal portfolio data from Yahoo! Finance?
I searched for it, but didn't find anything. Maybe someone knows more?

Thanks

Unfortunately, I am not aware of any table that provides user portfolio data from yahoo finance.

YQL and copyright

Mon, 11 Jan 2010 08:24:35 -0800

Hi,

this question is not of technical nature and probably not specific to the use of YQL. Nevertheless, very relevant, I think.

Can anyone tell how the copyright law applies to data that's available on the web and which one would make available through an Open Table?

Concretely, there's a website that has some data I want to make accessible through an Open Table, that has this copyright statement:

All rights reserved. Neither this work nor any part of it may be reproduced, distributed, performed or displayed in any medium, including electronic or digital, without permission in writing from the copyright owner.

Am I allowed to provide this Open Table, or would this be an infringement of the copyright?

And thinking further, if one would use the Open Table to access the data and make it available to users through his own webservice, how is the copyright law applied to that case?

I'd appreciate your comments on this...

Groeten,
Vic

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

Hi,

I've been going bonkers trying to figure out what's wrong with my statement:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

Syntax error(s) [line 1:165 mismatched input 'like' expecting ISNOTNULL]
select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

I don't see why I need this ISNOTNULL.

Any suggestions?

Thanks!

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

QUOTE (anonymousities @ Jan 10 2010, 10:16 PM) <{POST_SNAPBACK}>

Hi,

I've got a problem with my statement:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

Syntax error(s) [line 1:165 mismatched input 'like' expecting ISNOTNULL]
select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

I want to, if the Symbol look up fails, use the query in a wildcard Name search. I guess this means I need two separate SQL statements for the two situations, right?

Thanks!

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

Crud, sorry about that. I don't see how to edit posts, and then I accidentally put my edit in quotes.

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

QUOTE (anonymousities @ Jan 10 2010, 10:16 PM) <{POST_SNAPBACK}>

I've been going bonkers trying to figure out what's wrong with my statement:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

Syntax error(s) [line 1:165 mismatched input 'like' expecting ISNOTNULL]
select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Symbol in ('YHOO') OR Name like ('YHOO%')

Hi,

first of all, you should say "name like 'YHOO%'", without the brackets.

Stripping down your query, this is a statement that's working:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where symbol in ("YHOO")

Now if you replace the where-clause by the second part of your where-clause, you get this:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Name like "YHOO%"

This query gives an error, because it lacks the required key 'symbol'. Now you can see it's quite obvious that if you combine clauses with 'OR', all of them should have the mandatory keys, because each member of the 'OR' is in fact a separate query.

Notice that following query does work, because now the clauses are combined with 'AND', and the 'name'-condition is applied as a local filter to the data that is retrieved by using the 'symbol'-condition as a remote filter:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where symbol in ("YHOO") and Name like "Yahoo%"

Also, there's a lot of case-sensitivity issues going on in this table definition...

Regards,
Vic

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

Thanks for your help, Vic. However, my problem is not solved.

In my application, I want my users to look up stocks by (a) symbol or ( name. The inputs are received in the same box, so I was hoping that the OR statement would let users do either. Any advice on how I can implement this? I suppose I need to go outside my YQL statement, then?

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

QUOTE (anonymousities @ Jan 12 2010, 11:49 AM) <{POST_SNAPBACK}>

Thanks for your help, Vic. However, my problem is not solved.

In my application, I want my users to look up stocks by (a) symbol or ( name. The inputs are received in the same box, so I was hoping that the OR statement would let users do either. Any advice on how I can implement this? I suppose I need to go outside my YQL statement, then?

The table you're using relies on the 'symbol' to be queried, because basically what happens is that this webservice is called:

http://download.finance.yahoo.com/d/quotes...tjoin|,|symbol}

If you dig into the yahoo finance webservice API, you may find a way to retrieve similar data based on stock names and you could create your own open table for processing that data....

I don't know anything about these webservices, so can't help any further.

Groeten,
Vic

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

QUOTE (vic m @ Jan 12 2010, 01:35 PM) <{POST_SNAPBACK}>

The table you're using relies on the 'symbol' to be queried, because basically what happens is that this webservice is called:

http://download.finance.yahoo.com/d/quotes...tjoin|,|symbol}

If you dig into the yahoo finance webservice API, you may find a way to retrieve similar data based on stock names and you could create your own open table for processing that data....

I don't know anything about these webservices, so can't help any further.

Groeten,
Vic

Vic, thanks for your help. Somehow, my query keeps returning NULL.

{

CODEBOX

function yqlGetAll($stockSymbol)	{
require("yahooSDK/Yahoo.inc");
define("API_KEY"<omitted>);  
define("SHARED_SECRET",<omitted>);

$two_legged_app = new YahooApplication(API_KEY,SHARED_SECRET);
if ($two_legged_app == NULL) { 
	return false; 
	}  

$symbolQuery = "select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where symbol in ('$stockSymbol')";
$symbolResponse = $two_legged_app->query($symbolQuery);

var_dump($symbolResponse); 
}

Any suggestions? I followed the code examples (http://developer.yahoo.com/yql/guide/yql-code-examples.html#sdk_yql) pretty closely.

YQL Statement Problem

Sun, 10 Jan 2010 22:16:24 -0800

I updated my YQL statement. I fixed a concatenation problem and added USE, AS statements.

CODEBOX

$symbolQuery = "USE 'http://test.atlambert.com/stuff/yahoo.finance.quotes.xml' AS yahoo.finance.quotes; SELECT Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime FROM yahoo.finance.quotes WHERE Symbol IN ('" . $stockSymbol . "')";

However, I'm getting that goofy key problem I had earlier:

QUOTE (vic m @ Jan 12 2010, 01:21 AM) <{POST_SNAPBACK}>

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where Name like "YHOO%"

This query gives an error, because it lacks the required key 'symbol'. Now you can see it's quite obvious that if you combine clauses with 'OR', all of them should have the mandatory keys, because each member of the 'OR' is in fact a separate query.

Notice that following query does work, because now the clauses are combined with 'AND', and the 'name'-condition is applied as a local filter to the data that is retrieved by using the 'symbol'-condition as a remote filter:

select Name,Symbol,LastTradePriceOnly,Open,DaysHigh,DaysLow,Volume,PercentChange,AfterH
oursChangeRealtime from yahoo.finance.quotes where symbol in ("YHOO") and Name like "Yahoo%"

I don't understand why I need to add an "AND" or "OR" to this. Why can't I just look up all the fields for the given stock symbol?

My goal is to scrape the info for the given stock symbol. Let me know if you have ideas. Thanks!!

No definition found for Table google.news

Sat, 09 Jan 2010 11:11:23 -0800

Hi,

I am getting this error suddenly in my application - yesterday it was working fine

Its also working fine in the YQL console.

Same issue is happening for 'google.search'

Any help would be appreciated.

No definition found for Table google.news

Sat, 09 Jan 2010 11:11:23 -0800

QUOTE (Webshark @ Jan 9 2010, 11:11 AM) <{POST_SNAPBACK}>

Hi,

I am getting this error suddenly in my application - yesterday it was working fine

Its also working fine in the YQL console.

Same issue is happening for 'google.search'

Any help would be appreciated.

Hmmm. Not sure since this isn't much information. Maybe the environment you were using wasn't set on your application or the environment was unavailble?

Sam

No definition found for Table google.news

Sat, 09 Jan 2010 11:11:23 -0800

QUOTE (Webshark @ Jan 9 2010, 11:11 AM) <{POST_SNAPBACK}>

Hi,

I am getting this error suddenly in my application - yesterday it was working fine

Its also working fine in the YQL console.

Same issue is happening for 'google.search'

Any help would be appreciated.

I have just had the same problem. For me it works but in the meantime: You have to make sure that you when you call the YQL API that you include the Community Tables. For example: http://query.yahooapis.com/v1/public/yql?q...ltableswithkeys

Christian

How do you do paging? I'm implementing a table for the Mozilla Labs Jetpack Gallery

Wed, 06 Jan 2010 02:50:08 -0800

I've read the YQL Guide, and I keep reviewing http://developer.yahoo.com/yql/guide/yql-o...entables-paging and I have been looking at a few examples, but I'm still left pretty confused how paging works.

The problem that I am trying to tackle is creating a YQL open data table for the Mozilla labs Jetpack Gallery's jetpacks pages http://jetpackgallery.mozillalabs.com/jetpacks

You flip through the pages of jetpacks with the ?page query variable and there is an order_by query variable. You can only see 10 results per page.

Questions:
- Should I use or or ?
-- How do I use this?
- I am assuming I should use:

http://jetpackgallery.mozillalabs.com/jetpacks

Is this correct?
- In the execute element, I will need to extract the details for each jetpack on the page and organize that somehow, but I am at a loss of how I will access the page's content, and how the response.object should be structured..

Can anyone provide some help? or perhaps point to a data table that I can look at as a reference? or better documentation on how paging works? like a blog post or something.

Thanks in advance,
Erik

Issue with data from YQL

Wed, 06 Jan 2010 02:05:11 -0800

I've written a YQL query which takes a lat/long and produces the weather from it, and on the whole it works fine.

However there are some lat/long's that don't work, but obviously should.

Here's the entire query:

CODE

use 'http://github.com/yql/yql-tables/raw/master/weather/weather.bylocation.xml' as we;select * from we where location in (select place.name from flickr.places where lat=51.526348 and lon=-0.083556) and unit='c'

This works fine, and gives me the weather as I'd expect. However, if I change the lat/long to 51.490961,-0.143913, it changes to somewhere in the USA:

CODE

use 'http://github.com/yql/yql-tables/raw/master/weather/weather.bylocation.xml' as we;select * from we where location in (select place.name from flickr.places where lat=51.490961 and lon=-0.143913) and unit='c'

Breaking it down to try and figure it out, I get this:

CODE

select place.name from flickr.places where lat=51.490961 and lon=-0.143913

Result = Pimlico, London, England, GB, United Kingdom

So this is what gets passed to the weather API, which turns it into some place in America.

Any idea how to resolve this?

Cheers

Issue with data from YQL

Wed, 06 Jan 2010 02:05:11 -0800

QUOTE (Simon @ Jan 6 2010, 02:05 AM) <{POST_SNAPBACK}>

I've written a YQL query which takes a lat/long and produces the weather from it, and on the whole it works fine.

However there are some lat/long's that don't work, but obviously should.

Here's the entire query:
CODE
use 'http://github.com/yql/yql-tables/raw/master/weather/weather.bylocation.xml' as we;select * from we where location in (select place.name from flickr.places where lat=51.526348 and lon=-0.083556) and unit='c'
This works fine, and gives me the weather as I'd expect. However, if I change the lat/long to 51.490961,-0.143913, it changes to somewhere in the USA:
CODE
use 'http://github.com/yql/yql-tables/raw/master/weather/weather.bylocation.xml' as we;select * from we where location in (select place.name from flickr.places where lat=51.490961 and lon=-0.143913) and unit='c'
Breaking it down to try and figure it out, I get this:
CODE
select place.name from flickr.places where lat=51.490961 and lon=-0.143913
Result = Pimlico, London, England, GB, United Kingdom

So this is what gets passed to the weather API, which turns it into some place in America.

Any idea how to resolve this?

Cheers

Strange, that. Going by name is never a clever idea. Use woeid and the the woeid table instead:

CODE

use "http://github.com/yql/yql-tables/raw/master/weather/weather.woeid.xml" as we;select * from we where w in (select place.woeid from flickr.places where lat=51.490961 and lon=-0.143913) and u='c'

That works.

Problem in getting weather forecast

Mon, 04 Jan 2010 10:50:02 -0800

Hy all,

I'm trying to get weather forecasts for some cities through SQL

I would expect the following YQL to work, but it gives to me a "City not found error"

SELECT * FROM weather.forecast WHERE location IN (SELECT woeid FROM geo.places WHERE text="san francisco, ca")

It seems to me that the location filed of the weather.forecast requires some other location ID and not the woeid.
I tried to get the weather information calling the forecastrss wirh the woeid of San Francisco (http://xml.weather.yahoo.com/forecastrss?w=2487956&u=c) and I found in it a different ID fro San Francisco (USCA0987)

Then I tried to use YQL again with that ID and I finally got the forecasts for San Francisco
(SELECT * FROM weather.forecast WHERE location = 'USCA0987')

Is there a way to do that with just one YQL call?

Tnx in advance

Regards

Edoardo

Yahoo Videos Playlist

Wed, 30 Dec 2009 12:20:50 -0800

Hello,

Can you we query a list of video ids from a playlist video id?

Yahoo Videos Playlist

Wed, 30 Dec 2009 12:20:50 -0800

If I understand the question correctly, you have a Yahoo! Video playlist id and you want to get a list of ids for the videos in the list.

Each playlist has an rss feed of the videos in it. You could use the built-in rss table to pull in this data, and then parse through it, using either a yql open table or some other technique, and extract the video ids in the media:player field of each item.

Possible to retrieve redirected destination url?

Tue, 29 Dec 2009 14:30:21 -0800

I'm trying to use the YQL module in Y! Pipes to determine the ultimate destination redirected url.
Example:
http://feedproxy.google.com/~r/Techcrunch/~3/P-%5FqWQXyAPU/
redirects to
http://www.techcrunch.com/2009/12/08/build-it-with-me

Is there a simple way to get this information with YQL?

Possible to retrieve redirected destination url?

Tue, 29 Dec 2009 14:30:21 -0800

QUOTE (GollyJer @ Dec 29 2009, 02:30 PM) <{POST_SNAPBACK}>

I'm trying to use the YQL module in Y! Pipes to determine the ultimate destination redirected url.
Example:
http://feedproxy.google.com/~r/Techcrunch/~3/P-%5FqWQXyAPU/
redirects to
http://www.techcrunch.com/2009/12/08/build-it-with-me

Is there a simple way to get this information with YQL?

If you use y.rest() to make the call you can use followRedirects(false) like this:

use 'http://javarants.com/yql/javascript.xml' as j; select * from j where code='response.object = y.rest("http://feedproxy.google.com/~r/Techcrunch/~3/P-%5FqWQXyAPU/").followRedirects(false).get().headers.location;'

Sam

Possible to retrieve redirected destination url?

Tue, 29 Dec 2009 14:30:21 -0800

QUOTE (Sam @ Jan 4 2010, 09:57 AM) <{POST_SNAPBACK}>

If you use y.rest() to make the call you can use followRedirects(false) like this:

use 'http://javarants.com/yql/javascript.xml' as j; select * from j where code='response.object = y.rest("http://feedproxy.google.com/~r/Techcrunch/~3/P-%5FqWQXyAPU/").followRedirects(false).get().headers.location;'

Sam

Awesome. Thanks Sam.

How to view debug logs

Wed, 23 Dec 2009 01:31:39 -0800

Hi,

I trying to create YQL Open Data Table with using "debug=true" mode.
So I can not afford the id attribute in the diagnostics element of response.
what you've got a limit or note that the log is there to see?

--
okuryu

How to view debug logs

Wed, 23 Dec 2009 01:31:39 -0800

QUOTE (okuryu @ Dec 23 2009, 01:31 AM) <{POST_SNAPBACK}>

Hi,

I trying to create YQL Open Data Table with using "debug=true" mode.
So I can not afford the id attribute in the diagnostics element of response.
what you've got a limit or note that the log is there to see?

--
okuryu

Just, was solved.
I can get the id attribute when cut env parameter. can not use debug and env parameters together?

How to view debug logs

Wed, 23 Dec 2009 01:31:39 -0800

I'm glad you were able to work it out. You can use env and debug together. Here's an example of a query using an open table, specifying an env file, and setting the debug state:

CODE

http://developer.yahoo.com/yql/console/?q=use%20'http://www.datatables.org/geo/geo.placemaker.xml'%20as%20table;%20SELECT%20*%20FROM%20table%20WHERE%20documentContent%20%3D%20"They%20followed%20him%20to%20deepest%20Africa%20and%20found%20him%20there,%20in%20Timbuktu"%20AND%20documentType%3D"text/plain"%20AND%20appid%20%3D%20""&env=store://datatables.org/alltableswithkeys&debug=true

news feed

Tue, 15 Dec 2009 09:29:48 -0800

Hi, I am new to YQL. I am wondering if there is any tutorial for query news from as many sources as possible? Thanks.

news feed

Tue, 15 Dec 2009 09:29:48 -0800

I haven't seen a specific tutorial on this however it should be pretty straightforward to get news from newsfeeds of your choice.

For example getting BBC news would be something like:

CODE

select * from feed where url = "http://newsrss.bbc.co.uk/rss/newsonline_world_edition/front_page/rss.xml";

If you want to select multiple feeds at once you can use the query.multi community tables; e.g.:

CODE

select * from query.multi where query="query1; query2; query3"

After that it would just be a case of including the YQL results in your page, but that depends on your platform, etc.

Can YQL be used to workaround browser restrictions for cross-domain AJAX calls?

Mon, 14 Dec 2009 07:59:18 -0800

In short I have a problem with a cross-domain AJAX request and I am hoping that YQL can be used to workaround this. First some background in case any of this proves relevant:

An online racing game recently advertised three PHP scripts that return lists of detail (in XML format) for :

getPlayers.php = players currently online
getServers.php = currently online players who are also hosting races
getRankings.php = current rankings of all registered players

For unknown reasons that game server (i) will not support JSON/JSONP and (ii) requires authentication using a username/password pair which is freely available upon request. (The idea being that each fan site wanting to make use of this data gets a user/passwd pair to access the PP scripts.) Fan sites that have access to server-side scripting can workaround the cross-domain call issue however I am wanting to display this data at a wiki page hosted at wikia.com. At Wikia I do not have access to server-side scripting options such as PHP, ASP , etc.
The only tool available to me is JavaScript (and notably the jQuery library) which must execute at the browser client.

It may take several weeks (or several months) to persuade the game provider to support jQuery.getJSON() access to this data. That is why have come to this YQL forum.

So my questions for this forum:

Is it possible for YQL to somehow serve as proxy server in the retrieval of this data?
Is it practical to have a data table for each set of data mirror the game server data - refreshing every time an incoming getJSON() request arrives?
Can the user/passwd pair necessary for getting this XML data from the game server be used to construct the table and yet be hidden from public view?
If this is practical then am I correct to assume that I could use jQuery.getJSON() to retrieve that table of data ?
If YQL cannot be used then can you suggest any other interrim solution to what is fundamentally a cross-domain XmlHttpRequest problem?

Can YQL be used to workaround browser restrictions for cross-domain AJAX calls?

Mon, 14 Dec 2009 07:59:18 -0800

Maybe you are interested in this:

http://code.google.com/p/xdrequest/

Cheers,
Francisco.

QUOTE (Najevi @ Dec 14 2009, 07:59 AM) <{POST_SNAPBACK}>

In short I have a problem with a cross-domain AJAX request and I am hoping that YQL can be used to workaround this. First some background in case any of this proves relevant:

An online racing game recently advertised three PHP scripts that return lists of detail (in XML format) for :
getPlayers.php = players currently online
getServers.php = currently online players who are also hosting races
getRankings.php = current rankings of all registered players
For unknown reasons that game server (i) will not support JSON/JSONP and (ii) requires authentication using a username/password pair which is freely available upon request. (The idea being that each fan site wanting to make use of this data gets a user/passwd pair to access the PP scripts.) Fan sites that have access to server-side scripting can workaround the cross-domain call issue however I am wanting to display this data at a wiki page hosted at wikia.com. At Wikia I do not have access to server-side scripting options such as PHP, ASP , etc.
The only tool available to me is JavaScript (and notably the jQuery library) which must execute at the browser client.

It may take several weeks (or several months) to persuade the game provider to support jQuery.getJSON() access to this data. That is why have come to this YQL forum.

So my questions for this forum:
Is it possible for YQL to somehow serve as proxy server in the retrieval of this data?
Is it practical to have a data table for each set of data mirror the game server data - refreshing every time an incoming getJSON() request arrives?
Can the user/passwd pair necessary for getting this XML data from the game server be used to construct the table and yet be hidden from public view?
If this is practical then am I correct to assume that I could use jQuery.getJSON() to retrieve that table of data ?
If YQL cannot be used then can you suggest any other interrim solution to what is fundamentally a cross-domain XmlHttpRequest problem?

Can YQL be used to workaround browser restrictions for cross-domain AJAX calls?

Mon, 14 Dec 2009 07:59:18 -0800

This is easy to do in YQL. You will likely need to make an open data table to do the authentication part if it is Header based rather than query parameter based though. Do you have a very specific example you can post?

Sam

retrieve and select xml-data from yql.storage

Sat, 12 Dec 2009 05:16:30 -0800

Hi,

YQL allows to retrieve and select data from an XML-table, using the 'xml' open table. E.g. like this:

CODE

select edition.biblija-api from xml where url="http://www.dezuidelijkenederlanden.org/Books.xml" and edition.biblija-abbr="WEB"

Is it also possible to perform a similar selection on xml-data stored on yql.storage?

I've stored the same table as above on yql.storage.admin like this:

CODE

insert into yql.storage.admin (url) values ("http://www.dezuidelijkenederlanden.org/Books.xml")

I can retrieve the data using following statement, but it's giving me the xml-encoded version of the xml-data in the results, and if I specify an element name in the select statement, the results are empty.

CODE

select * from yql.storage where name="store://Ng1jFUQURtdsia9QKYnZWU"

returns:

CODE

<result>
<value>  [...]</value>
</result>

Another way of accessing the data would be using a statement like the following, but that gives a parser error, which I don't understand...

CODE

use "store://Ng1jFUQURtdsia9QKYnZWU" as biblebooks;
select * from biblebooks;

returns:

CODE

Table XML Parse Error for biblebooks [line 1:1 Premature end of file.]

The YQL Guide is not mentioning this type of data retrieval from yql.storage explicitly, so maybe it's not supported at all? But then, what's the use of storing xml-tables?

Regards,
Vic

retrieve and select xml-data from yql.storage

Sat, 12 Dec 2009 05:16:30 -0800

Here is very scary way to do what you want:'

use 'http://javarants.com/yql/javascript.xml' as j;
select * from j where code='response.object = eval("var a =" + y.query("select * from yql.storage where name=\"store://Ng1jFUQURtdsia9QKYnZWU\"").results.result.value.text() + ";a")'

I suggest making an open data table for your use case rather than using my javascript table however. The key piece is reevaluating the string you get back as an E4X document:

response.object = eval("var a = " + xmlstring + ";a");

Sam

retrieve and select xml-data from yql.storage

Sat, 12 Dec 2009 05:16:30 -0800

Hi,

thanks for the answer. I was thinking that I missed the obvious straightforward solution, but maybe there isn't.

The XML-interpretion and conversion steps seem like a bit too much overhead. Would it be a better solution to store the data on yql.storage as JSON data? I have no experience using JSON, but from what I read, it seems to be more efficient for importing into javascript.

Another solution I was thinking of, is to create a new Open Table that has a javascript execute section that holds the data hard-coded. Not as nice, but since my data is mostly static, it won't hurt.

Regards,
Vic

CDATA to JSON response format

Wed, 09 Dec 2009 14:50:26 -0800

Hi guys,

First let me start off with, if there is a place for (possible) bug reports to be made other than here in the forums would some kind soul please point me in that direction?

Now onto the problem. I have been trying to get YQL to consume some XML and had issues with the JSON output being "wrong" (i.e. not what I thought it should be). After digging around, the issue quickly boiled down to one of mixing CDATA and non-CDATA (or just multiple CDATA) within the same XML node as outlined below.

Example XML

CODE

<?xml version="1.0" encoding="UTF-8"?>
<a>
  <b><![CDATA[foo]]>bar</b>
</a>

Example YQL query

CODE

select * from xml where url='http://codepad.org/hYfhEnHt/raw.txt'

Try this in the console

The XML output format is fine (it reflects the CDATA, so all the data is available to me) but the JSON output format only gives the "bar" for that "b" node. Whether the bar is wrapped in another CDATA or not appears to make no difference and indeed the number of different pieces (e.g. CDATA, plain text, CDATA, plaintext) result in the same thing: only the last portion being included in the JSON output.

If this is intended behaviour, do let me know. Either way, in the mean time, it looks like will have to come to the rescue.

Cheers

CDATA to JSON response format

Wed, 09 Dec 2009 14:50:26 -0800

I can see why this could be frustrating. I've run into similar issues myself.

We use a library to turn XML into JSON however, I'm going to come up with some scenarios to see how we handle these conversations and if we can improve on it.

Tom
Yahoo! Developer Network

CDATA to JSON response format

Wed, 09 Dec 2009 14:50:26 -0800

Thanks Tom. If you could ping me if any changes to this ever go live, it would be appreciated.

Help with using geo and aiport information

Wed, 09 Dec 2009 12:43:45 -0800

Hello all! I am really new to YQL and have been try to figure it out on my own but to no avail. Here is what I am trying to do. Send a state name such as California to YQL and get back all the airports in that state. Is that possible?

Cheers!

Help with using geo and aiport information

Wed, 09 Dec 2009 12:43:45 -0800

I'm sure that's possible but I don't think we have a table for an API which does that function currently.

You could build one from something like http://www.airportcitycodes.com/statewisecodes.aspx for example.

Tom
Yahoo! Developer Network

Help with using geo and aiport information

Wed, 09 Dec 2009 12:43:45 -0800

There is a community table with some airport data:

http://airports.pidgets.com/v1/

my table has been blocked....

Tue, 08 Dec 2009 02:06:25 -0800

I was in test in the yql console, and i've receive this message :
The current table 'data.html.cssselect' has been blocked. It exceeded the allotted quotas of either time or instructions
I've trying to add an api_key but it dosn't work. How can unlock my table ? thanks.

my table has been blocked....

Tue, 08 Dec 2009 02:06:25 -0800

If you use more than 50 million "units" or 30 seconds of execution time we stop executing your table. This is to make sure the platform is available for everyone.

I've asked one of the team to investigate this table specifically to find out what's happened in this case. However, you should be aware that queries that are too intensive may be blocked. You can always split them up into sub-queries where possible.

Tom
Yahoo! Developer Network

my table has been blocked....

Tue, 08 Dec 2009 02:06:25 -0800

QUOTE (gerard @ Dec 8 2009, 02:06 AM) <{POST_SNAPBACK}>

I was in test in the yql console, and i've receive this message :
The current table 'data.html.cssselect' has been blocked. It exceeded the allotted quotas of either time or instructions
I've trying to add an api_key but it dosn't work. How can unlock my table ? thanks.

Does your table have an execute? If so you might have run into the execution limit (http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html). If your table hits the execution limit or exceeds 30s 4 times in an hour it will be blocked for 1 hour.

input parameter substitution in url - not simply substituting (crosspost :()

Mon, 07 Dec 2009 23:33:32 -0800

Oops,

looks like I've crossposted a YQL-Open Tables question on the general discussion YDN forum...

http://developer.yahoo.net/forum/index.php?showtopic=3725

Sorry for that,
Vic

input parameter substitution in url - not simply substituting

Mon, 07 Dec 2009 12:40:00 -0800

Hi,

I've been looking into YQL for a private project of mine and try to define an open table that is queried like this:

CODE

use "http://github.com/vicmortelmans/yql-tables/raw/master/bible/nl.willibrord.xml" as nl.willibrord;
select * from nl.willibrord where book='John' and chapter=3 and passage=16

The open table should fetch html data from this URL

QUOTE

http://www.bijbel.net/wb/?John+3,16

So I specify the as

CODE

http://www.bijbel.net/wb/?{book}+{chapter},{passage}

The variables in '{}' should be replaced by the 'where'-criteria of the query.

But if I query the table, the console tells me that this URL is called:

CODE

<url execution-time="9" proxy="DEFAULT"><![CDATA[http://www.bijbel.net/wb/?+,&John+3,16]]></url>

Notice the extra occurrence of the '+' and ',' in the GET-section...

Why is YQL doing this ?? I've been thinking this behaviour is triggered by the uritemplate feature, but can't figure out how...

Complete open table definition:

CODE

<?xml version="1.0" encoding="UTF-8"?>
<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">
  <meta>
    <author>Vic Mortelmans</author>
    <description>The table contains the complete bible. Each passage is a record, identified by it's citation reference.</description>
    <sampleQuery>select * from nl.willibrord where book='John' and chapter=3 and passage=16 </sampleQuery>
    <documentationURL>http://docs.google.com/View?id=ddq89pzk_155t6rwmkhp</documentationURL>
  </meta>
  <bindings>
    <select produces="XML" itemPath="*">
      <urls>
        <url>http://www.bijbel.net/wb/?{book}+{chapter},{passage}</url>
      </urls>
      <inputs>
        <key id="book" type="xs:string" paramType="path" required="true"/>
        <key id="chapter" type="xs:integer" paramType="path" required="true"/>
        <key id="passage" type="xs:integer" paramType="path" required="true"/>
      </inputs>
      <execute><![CDATA[ 
        var page = y.query("select * from html where url=@url and xpath='//font[@color=\"red\"]'",{url:request.url});
        response.object = page.results;
      ]]></execute>
      </select>
  </bindings>
</table>

(http://github.com/vicmortelmans/yql-tables/raw/master/bible/nl.willibrord.xml)

Thanks !!
Vic

input parameter substitution in url - not simply substituting

Mon, 07 Dec 2009 12:40:00 -0800

The problem here is that + in a URI is a space character. Instead I think you should use %20 like this:

select * from uritemplate where template ='http://www.bijbel.net/wb/?{book}%20{chapter},{passage}' and book = 'John' and chapter=3 and passage=16

Sam

simpleyql, a free Java library to ease YQL+OAuth development

Mon, 23 Nov 2009 06:33:27 -0800

Hello!

I am one of the developers of simpleyql, a library that simplifies the development of Java applications that need to use OAuth-authenticated YQL APIs.

It may help Java developers that got a bit unmotivated by the complexity of the oAuth core library (simpleyql is built upon it, but hides its complexity by being Yahoo-specific and doing some assumptions).

If you want to see it in action, check MemeThis (a Y! Meme add-on that uset this library), which is free software, just as simpleyql. An interesting aspect is that it's hosted on Google App Engine(*), showcasing how people can combine Google and Yahoo! services to build new functionality and make it available at zero cost!

* simpleyql was designed with GAE as an option in mind, but will work as well on standard web containers.

Regards,
Chester.

simpleyql, a free Java library to ease YQL+OAuth development

Mon, 23 Nov 2009 06:33:27 -0800

QUOTE (Carlos Duarte do Nascimento (Che @ Nov 23 2009, 06:33 AM) <{POST_SNAPBACK}>

Hello!

I am one of the developers of simpleyql, a library that simplifies the development of Java applications that need to use OAuth-authenticated YQL APIs.

It may help Java developers that got a bit unmotivated by the complexity of the oAuth core library (simpleyql is built upon it, but hides its complexity by being Yahoo-specific and doing some assumptions).

If you want to see it in action, check MemeThis (a Y! Meme add-on that uset this library), which is free software, just as simpleyql. An interesting aspect is that it's hosted on Google App Engine(*), showcasing how people can combine Google and Yahoo! services to build new functionality and make it available at zero cost!

* simpleyql was designed with GAE as an option in mind, but will work as well on standard web containers.

Regards,
Chester.

Thanks for the great work!

Create RSS feed with YQL?

Thu, 19 Nov 2009 04:46:18 -0800

Hi

I'm new to YQL, limited in my programming experience, but very interested in what I've seen here so far.

I've been playing around in the YQL console, generating XML from a number of data sources. I'd like to be able to use this data as an RSS feed which can be pulled into iGoogle, Netvibes or WordPress, but can't see how to do this in YQL. Also can't find guidance on this.

Can anyone point me to some guidance, or show examples?

Lee

Create RSS feed with YQL?

Thu, 19 Nov 2009 04:46:18 -0800

Yahoopipes has a YQL module in it where you can use YQL queries. Yahoopipes by nature creates RSS feeds.

QUOTE (lee.stumpymoose @ Nov 19 2009, 04:46 AM) <{POST_SNAPBACK}>

Hi

I'm new to YQL, limited in my programming experience, but very interested in what I've seen here so far.

I've been playing around in the YQL console, generating XML from a number of data sources. I'd like to be able to use this data as an RSS feed which can be pulled into iGoogle, Netvibes or WordPress, but can't see how to do this in YQL. Also can't find guidance on this.

Can anyone point me to some guidance, or show examples?

Lee

Can Open Data Table be created from a HTML table?

Mon, 16 Nov 2009 10:04:10 -0800

I see a lot of info on how to generate open data tables from url querystrings but not much on how to create one from an HTML table on a webpage, if indeed that's possible?

Can Open Data Table be created from a HTML table?

Mon, 16 Nov 2009 10:04:10 -0800

QUOTE (Matthew R @ Nov 16 2009, 10:04 AM) <{POST_SNAPBACK}>

I see a lot of info on how to generate open data tables from url querystrings but not much on how to create one from an HTML table on a webpage, if indeed that's possible?

I made a table for that... http://kid666.com/yql/forms2tables.xml

Can Open Data Table be created from a HTML table?

Mon, 16 Nov 2009 10:04:10 -0800

Thanks! I take it this is javascript for YQL Execute?

Can Open Data Table be created from a HTML table?

Mon, 16 Nov 2009 10:04:10 -0800

I tried the following query and it didn't seem to like it:

QUOTE

use "http://kid666.com/yql/forms2tables.xml" as forms2tables; select * from forms2tables where url="http://www.opendatani.info/datafiles/index.html" and xpath="//table[@class='data']/"

output:

QUOTE

true

131
119
3694

I'm not totally clued up on xpath but I think that should get the table i'm after (it's buried inside quite a few divs though but I don't think that should make a difference with the query I passed)

Right number of open connections

Sun, 15 Nov 2009 19:50:18 -0800

Hi,

I'm doing a bunch of YQL queries every minute (for meme) and I'm wondering what the correct number of open connections I should have at any one time? I don't want to get blocked, but I do want maximum throughput.

All of my requests are 3-legged OAuth authenticated.

get newsheader from yahoo finance with xpath

Sun, 15 Nov 2009 08:35:02 -0800

Hello,
I am trying to get all the links to newheaders on a certain ticker. I was able to get the list from the main page of finance (after inserting a ticker) using this yql:

select * from html where url="http://finance.yahoo.com/q?s=amzn" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a'

But i can't get all the results. So i tried using the same idea on the page that shows all the result (the link to the page: http://finance.yahoo.com/q/h?s=AMZN&t=...T10:00:00-05:00 )

with this yql:

select * from html where url="http://finance.yahoo.com/q/h?s=AMZN&t=2009-11-15T10:00:00-05:00" and xpath='//table[@id="yfncsumtab"]/tbody/tr[2]/td/table[2]/tbody/tr/td[2]/a'

(I used firebug in order to follow the page DOM). But this gives me no results. Anyone can please show me what is the problem? Or suggest a better way to do this?
Thank you

get newsheader from yahoo finance with xpath

Sun, 15 Nov 2009 08:35:02 -0800

QUOTE (yehudasabag @ Nov 15 2009, 08:35 AM) <{POST_SNAPBACK}>

select * from html where url="http://finance.yahoo.com/q/h?s=AMZN&t=2009-11-15T10:00:00-05:00" and xpath='//table[@id="yfncsumtab"]/tbody/tr[2]/td/table[2]/tbody/tr/td[2]/a'

(I used firebug in order to follow the page DOM). But this gives me no results. Anyone can please show me what is the problem? Or suggest a better way to do this?
Thank you

Firefox/Firebug tends to insert tbody elements that don't really exist into the DOM. Try removing the tbody elements and see if it works then.

get newsheader from yahoo finance with xpath

Sun, 15 Nov 2009 08:35:02 -0800

QUOTE (sh1mmer @ Nov 16 2009, 11:48 AM) <{POST_SNAPBACK}>

Firefox/Firebug tends to insert tbody elements that don't really exist into the DOM. Try removing the tbody elements and see if it works then.

Hi, Thanks for the reply. You are right about the tbody, the fire bug indeed added it. But even when i deleted it i am still can't get it to work.
Any other suggestins ? (Sorry about the time it took me to reply, next time it will be faster...)

Thanks

get newsheader from yahoo finance with xpath

Sun, 15 Nov 2009 08:35:02 -0800

how did you delete the tbody element?

get newsheader from yahoo finance with xpath

Sun, 15 Nov 2009 08:35:02 -0800

QUOTE (Matthew R @ Nov 19 2009, 02:05 PM) <{POST_SNAPBACK}>

how did you delete the tbody element?

Well, i figured it up. Sorry about the time it took be to bring the answer.
The yql is:

select * from html where url="http://finance.yahoo.com/q/h?s=AMZN&t=2009-11-15T10:00:00-05:00" and xpath='//table[@id="yfncsumtab"]/tr/td/table/tr/td/table[2]/tr/td[2]/a

I just did a mistake in the DOM path.

About the tbody, i just deleted it from the path.

help with xpath

Mon, 09 Nov 2009 10:48:38 -0800

Im trying to take the score rating for each game. (which can be found down the right hand of the page in a red box)
http://uk.pc.ign.com/articles/104/1042996p1.html

Im using a query

select * from html where url='http://uk.pc.ign.com/articles/104/1042996p1.html' and xpath='/html/body/div[6]/div/div[3]/div[5]/div[2]/div/div[2]/div/div/a'

The return is

true

3694

Im guessing there is something wrong with my xpath.
Help please

help with xpath

Mon, 09 Nov 2009 10:48:38 -0800

Actually the problem is that they don't have the information in the page and use a webservice to fill it in after page load. I created an open data table for their web service and here is the query that grabs the object id from the page and queries it:

CODEBOX

select * from ign.object where id in (select match1 from regex where text in (select href from html where url='http://uk.pc.ign.com/articles/104/1042996p1.html' and xpath='//a[@id="gameProfileLink"]') and expression="objects/[0-9]+/([0-9]+).html")

Here is a link to execute the query: http://y.ahoo.it/Xy9rtxPs

help with xpath

Mon, 09 Nov 2009 10:48:38 -0800

Thanks that worked a charm.
Do you mine making one for the pcgamer website please.

http://www.computerandvideogames.com/artic...CVG-Reviews-RSS

issue in posting to meme

Mon, 09 Nov 2009 01:57:52 -0800

Hi,
array
'method' => string 'GET' (length=3)
'url' => string 'http://query.yahooapis.com/v1/yql?q=INSERT%20INTO%20meme.user.posts%20%28type%2C%20content%29%20VALUES%20%28%22text%22%2C%20%22This%20is%20a%20text%20post%20from%20YQL%20from%20meme%20application%21%22%29&format=json&env=http%3A%2F%2Fdatatables.org%2Falltables.env' (length=264)
'code' => int 400
'requestHeaders' =>
array
0 => string 'Accept: application/json' (length=24)
1 => string 'Authorization: OAuth realm="yahooapis.com",oauth_version="1.0",oauth_nonce="2be567434160e0871459da0d1bee833c",oauth_timestamp="1257760118",oauth_consumer_key="*************",oauth_token="A%3DMaQyYX3KkBvOUN_etEGzKWYDwOVBBVkSgoIuyGJnbnXp3h.TYcEJkXiAIdggzP7fPmqszv_GC194lo
Y44SX2shO3nYVBRC5t_Pk.hOOlsBywS.BeL6qewrJqRTqFJhzBFvr545EnEab4Ydh679tLwZNBYCQVtoC
lQvPkJeHgxT4j7HeG3cIaWmuZK85g3zVHkhEvqen2_mQpzbgD_YGY6VPMmRBnaw_5fmcDwtZy0s'... (length=1135)
'requestBody' => null
'responseHeaders' =>
array
'Date' => string 'Mon, 09 Nov 2009 09:48:38 GMT' (length=29)
'P3P' => string 'policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"' (length=209)
'Access-Control-Allow-Origin' => string '*' (length=1)
'Vary' => string 'Accept-Encoding' (length=15)
'Content-Type' => string 'application/json;charset=utf-8' (length=30)
'Age' => string '0' (length=1)
'Server' => string 'YTS/1.17.21' (length=11)
'Cache-Control' => string 'private, proxy-revalidate' (length=25)
'Transfer-Encoding' => string 'chunked' (length=7)
'Proxy-Connection' => string 'Keep-Alive' (length=10)
'Connection' => string 'Keep-Alive' (length=10)
'responseBody' => string '{"error":{"lang":"en-US","diagnostics":{"publiclyCallable":"true","url":{"execution-time":"139","proxy":"DEFAULT","content":"http://datatables.org/alltables.env"}},"description":"Unable to execute YQL statement, wrong HTTP verb. Allowed HTTP methods for INSERT are PUT, POST and DELETE or GET with JSONP"}}' (length=306)

Please let me know how to go ahead with the issue.I am able to execute select statements but not insert,update and delete.please let me know how to proceed with same
awaiting your kind help

issue in posting to meme

Mon, 09 Nov 2009 01:57:52 -0800

Hi Usha,

You can either use the POST method for the INSERT query where the parameters would be q=your_query and format to get this to work OR append another query parameter to your GET request something like &callback=foo.

We prevent non-idempotent methods on a GET unless you specify a callback parameter OR use the other http methods such as POST, PUT

hope that helps

-- Nagesh

QUOTE (Usha @ Nov 9 2009, 01:57 AM) <{POST_SNAPBACK}>

Hi,
array
'method' => string 'GET' (length=3)
'url' => string 'http://query.yahooapis.com/v1/yql?q=INSERT%20INTO%20meme.user.posts%20%28type%2C%20content%29%20VALUES%20%28%22text%22%2C%20%22This%20is%20a%20text%20post%20from%20YQL%20from%20meme%20application%21%22%29&format=json&env=http%3A%2F%2Fdatatables.org%2Falltables.env' (length=264)
'code' => int 400
'requestHeaders' =>
array
0 => string 'Accept: application/json' (length=24)
1 => string 'Authorization: OAuth realm="yahooapis.com",oauth_version="1.0",oauth_nonce="2be567434160e0871459da0d1bee833c",oauth_timestamp="1257760118",oauth_consumer_key="*************",oauth_token="A%3DMaQyYX3KkBvOUN_etEGzKWYDwOVBBVkSgoIuyGJnbnXp3h.TYcEJkXiAIdggzP7fPmqszv_GC194lo
Y44SX2shO3nYVBRC5t_Pk.hOOlsBywS.BeL6qewrJqRTqFJhzBFvr545EnEab4Ydh679tLwZNBYCQVto
C
lQvPkJeHgxT4j7HeG3cIaWmuZK85g3zVHkhEvqen2_mQpzbgD_YGY6VPMmRBnaw_5fmcDwtZy0s'... (length=1135)
'requestBody' => null
'responseHeaders' =>
array
'Date' => string 'Mon, 09 Nov 2009 09:48:38 GMT' (length=29)
'P3P' => string 'policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"' (length=209)
'Access-Control-Allow-Origin' => string '*' (length=1)
'Vary' => string 'Accept-Encoding' (length=15)
'Content-Type' => string 'application/json;charset=utf-8' (length=30)
'Age' => string '0' (length=1)
'Server' => string 'YTS/1.17.21' (length=11)
'Cache-Control' => string 'private, proxy-revalidate' (length=25)
'Transfer-Encoding' => string 'chunked' (length=7)
'Proxy-Connection' => string 'Keep-Alive' (length=10)
'Connection' => string 'Keep-Alive' (length=10)
'responseBody' => string '{"error":{"lang":"en-US","diagnostics":{"publiclyCallable":"true","url":{"execution-time":"139","proxy":"DEFAULT","content":"http://datatables.org/alltables.env"}},"description":"Unable to execute YQL statement, wrong HTTP verb. Allowed HTTP methods for INSERT are PUT, POST and DELETE or GET with JSONP"}}' (length=306)

Please let me know how to go ahead with the issue.I am able to execute select statements but not insert,update and delete.please let me know how to proceed with same
awaiting your kind help

issue in posting to meme

Mon, 09 Nov 2009 01:57:52 -0800

QUOTE (Nagesh Susarla @ Nov 9 2009, 10:48 AM) <{POST_SNAPBACK}>

Hi Usha,

You can either use the POST method for the INSERT query where the parameters would be q=your_query and format to get this to work OR append another query parameter to your GET request something like &callback=foo.

We prevent non-idempotent methods on a GET unless you specify a callback parameter OR use the other http methods such as POST, PUT

hope that helps

-- Nagesh

Thanks Nagesh it finally worked with GET when i added Callback parameter

Posting to meme issue

Sun, 08 Nov 2009 21:09:28 -0800

Hi

i tried posting a text to meme could it doesnot post the url is
http://query.yahooapis.com/v1/yql?q=INSERT...2Falltables.env

i am a beginer to meme.please help me.awaiting your reply

Thanking you,
Usha

Posting to meme issue

Sun, 08 Nov 2009 21:09:28 -0800

QUOTE (Usha @ Nov 8 2009, 09:09 PM) <{POST_SNAPBACK}>

Hi

i tried posting a text to meme could it doesnot post the url is
http://query.yahooapis.com/v1/yql?q=INSERT...2Falltables.env

i am a beginer to meme.please help me.awaiting your reply

Thanking you,
Usha

Hi please find the code i have used for posting tweet which doesnot work please help me in fixing the issue
$session = YahooSession::requireSession(CONSUMER_KEY,CONSUMER_SECRET,APPID);
$user = $session->getSessionedUser();
$query = 'INSERT INTO meme.user.posts (type, content) VALUES ("text", "This is a text post from YQL from meme application!")';
$response = $session->query($query);

New Elance YDN Section

Fri, 06 Nov 2009 14:57:50 -0800

Elance is an online marketplace that lets anyone hire, manage, and pay independent professionals and contractors online. Whether you are looking for a developer for a specific project or you are an independant developer who works with YDN APIs and webservices, you'll want to know that we've added a group on Elance for developers who know our APIs like the back of their hands.

You can read more about this on the YDN blog or you can just go ahead and check out the Elance YDN group now. Look foward to seeing you there!

Robyn Tippins
Community Manager, YDN

Terms of Use

Fri, 06 Nov 2009 13:34:44 -0800

Does YQL Terms of use paragraph 1.f.iv (listed below) forbid the use of YQL for private stock and/or options trading?

iv. You shall not Sell, lease, share, transfer, or sublicense YQL or derive income from the use of YQL in conjunction with Yahoo! APIs or other web services, whether for direct commercial or monetary gain or otherwise, without Yahoo!'s prior, express, written permission;

Information DataTable and Drag and Drop

Fri, 06 Nov 2009 08:30:36 -0800

Dear all,
I have a little question to ask about use of the DataTable and Drag and Drop. When my datattable load file from a database by a Json, before the complier finish to display the page and the load data on the datatable, so in each point of the page if I try to read the number of record during the load of the page I read 0.
How I can load the Datatable data from JSON before to finish the display of the page.
Or more simply, how I can read the real number of record from datatable during the load of the page?
Thansk for your help

regards