[twitter-dev] Re: track syntax
Joel - thx for this bit of code... very helpful. John http://twapperkeeper.com @jobrieniii http://www.linkedin.com/in/jobrieniii On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp);} ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm, Joel Strellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line //echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp); } On Mon, Jul 27, 2009 at 11:18 AM, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1,
[twitter-dev] Re: track syntax
You can set both the track and follow parameters when using the /1/ statuses/filter URL. Best practices around changing your predicate: http://apiwiki.twitter.com/Streaming-API-Documentation#UpdatingFilterPredicates I can't answer PHP questions, sorry. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Aug 31, 12:32 pm, Polymatheus world.mo...@gmail.com wrote: I've used the code above to start streaming and then dumping the output to a text file every hour to process later. There are a few things I want to clarify, 1) How can the above script be amended to show both follow and track? Is this possible? 2) If I opened a stream to follow 10 users and then a further 5 users joined my site, would I have to close the first stream then open a new stream for the 15 follows? Or just a second stream with the additional 5 users? The second approach reduce the chance of tweets being lost between closing and opening a new stream. 3) I can't seem to close a stream that I opened using the sample code above, I have tried changing the file to simply say: $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); fclose($fp); But it doesn't appear to work :/ Thanks in advance On 10 Aug, 08:09, Joel Strellner j...@twitturly.com wrote: Tom, Yes, that code works perfectly for me exactly as is. You might want to change the connect timeout from 10 to 30 seconds. How long are you waiting before calling it quits? It does take a few seconds for /track to start sending your results. -Joel On Sun, Aug 9, 2009 at 8:17 PM, Tom Fitzgerald ccexpe...@gmail.com wrote: I'm sorry Joel I keep getting a PHP timeout on the code you sent. I'll troubleshoot more and see if I can give you any more details (increase the maximum time, etc). Who knows, maybe its Twitter. Any other thoughts? I'll get back to you more with some detailed info. Are you able to get that exact code working on your server? On Aug 3, 2:12 pm, Joel Strellner j...@twitturly.com wrote: Hi Tom, I am not sure about XML, since I use JSON - it has a much lower over-the-wire data size, and its easier to parse. Let me know if the code works for you. -Joel On Mon, Aug 3, 2009 at 10:55 AM, Tom Fitzgerald ccexpe...@gmail.com wrote: I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curlhttp://stream.twitter.com/spritzer.xml-uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true;
[twitter-dev] Re: track syntax
are you really just opening stream.twitter.org? Normally you would want to open http://stream.twitter.org/path/to/url.xml... Joseph Cheek jos...@cheek.com, www.cheek.com twitter: http://twitter.com/cheekdotcom John Kalucki wrote: You can set both the track and follow parameters when using the /1/ statuses/filter URL. Best practices around changing your predicate: http://apiwiki.twitter.com/Streaming-API-Documentation#UpdatingFilterPredicates I can't answer PHP questions, sorry. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Aug 31, 12:32 pm, Polymatheus world.mo...@gmail.com wrote: I've used the code above to start streaming and then dumping the output to a text file every hour to process later. There are a few things I want to clarify, 1) How can the above script be amended to show both follow and track? Is this possible? 2) If I opened a stream to follow 10 users and then a further 5 users joined my site, would I have to close the first stream then open a new stream for the 15 follows? Or just a second stream with the additional 5 users? The second approach reduce the chance of tweets being lost between closing and opening a new stream. 3) I can't seem to close a stream that I opened using the sample code above, I have tried changing the file to simply say: $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); fclose($fp); But it doesn't appear to work :/ Thanks in advance
[twitter-dev] Re: track syntax
Hi Polymatheus, 1) How can the above script be amended to show both follow and track? Is this possible? John's suggestion is the only way, you must use the new paths. 2) If I opened a stream to follow 10 users and then a further 5 users joined my site, would I have to close the first stream then open a new stream for the 15 follows? Or just a second stream with the additional 5 users? The second approach reduce the chance of tweets being lost between closing and opening a new stream. You must restart the stream, unfortunately, there is no way around this at this point. 3) I can't seem to close a stream that I opened using the sample code Abraham gives a potential solution to this, and I believe its along the right path. I normally only see this if we are lagging behind for whatever reason. Let me know if you have any further questions, since I am the person that wrote the code you're using. -Joel On Mon, Aug 31, 2009 at 12:32 PM, Polymatheus world.mo...@gmail.com wrote: I've used the code above to start streaming and then dumping the output to a text file every hour to process later. There are a few things I want to clarify, 1) How can the above script be amended to show both follow and track? Is this possible? 2) If I opened a stream to follow 10 users and then a further 5 users joined my site, would I have to close the first stream then open a new stream for the 15 follows? Or just a second stream with the additional 5 users? The second approach reduce the chance of tweets being lost between closing and opening a new stream. 3) I can't seem to close a stream that I opened using the sample code above, I have tried changing the file to simply say: $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); fclose($fp); But it doesn't appear to work :/ Thanks in advance On 10 Aug, 08:09, Joel Strellner j...@twitturly.com wrote: Tom, Yes, that code works perfectly for me exactly as is. You might want to change the connect timeout from 10 to 30 seconds. How long are you waiting before calling it quits? It does take a few seconds for /track to start sending your results. -Joel On Sun, Aug 9, 2009 at 8:17 PM, Tom Fitzgerald ccexpe...@gmail.com wrote: I'm sorry Joel I keep getting a PHP timeout on the code you sent. I'll troubleshoot more and see if I can give you any more details (increase the maximum time, etc). Who knows, maybe its Twitter. Any other thoughts? I'll get back to you more with some detailed info. Are you able to get that exact code working on your server? On Aug 3, 2:12 pm, Joel Strellner j...@twitturly.com wrote: Hi Tom, I am not sure about XML, since I use JSON - it has a much lower over-the-wire data size, and its easier to parse. Let me know if the code works for you. -Joel On Mon, Aug 3, 2009 at 10:55 AM, Tom Fitzgerald ccexpe...@gmail.com wrote: I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curlhttp://stream.twitter.com/spritzer.xml-uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while
[twitter-dev] Re: track syntax
If he really is using only those two lines, then yes, it explains some of his error. The header lines are what tells the server what page you're looking for on stream.twitter.com. -Joel On Mon, Aug 31, 2009 at 1:33 PM, Joseph Cheek jos...@cheek.com wrote: are you really just opening stream.twitter.org? Normally you would want to open http://stream.twitter.org/path/to/url.xml... Joseph Cheek jos...@cheek.com, www.cheek.com twitter: http://twitter.com/cheekdotcom John Kalucki wrote: You can set both the track and follow parameters when using the /1/ statuses/filter URL. Best practices around changing your predicate: http://apiwiki.twitter.com/Streaming-API-Documentation#UpdatingFilterPredicates I can't answer PHP questions, sorry. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Aug 31, 12:32 pm, Polymatheus world.mo...@gmail.com wrote: I've used the code above to start streaming and then dumping the output to a text file every hour to process later. There are a few things I want to clarify, 1) How can the above script be amended to show both follow and track? Is this possible? 2) If I opened a stream to follow 10 users and then a further 5 users joined my site, would I have to close the first stream then open a new stream for the 15 follows? Or just a second stream with the additional 5 users? The second approach reduce the chance of tweets being lost between closing and opening a new stream. 3) I can't seem to close a stream that I opened using the sample code above, I have tried changing the file to simply say: $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); fclose($fp); But it doesn't appear to work :/ Thanks in advance
[twitter-dev] Re: track syntax
Tom, Yes, that code works perfectly for me exactly as is. You might want to change the connect timeout from 10 to 30 seconds. How long are you waiting before calling it quits? It does take a few seconds for /track to start sending your results. -Joel On Sun, Aug 9, 2009 at 8:17 PM, Tom Fitzgerald ccexpe...@gmail.com wrote: I'm sorry Joel I keep getting a PHP timeout on the code you sent. I'll troubleshoot more and see if I can give you any more details (increase the maximum time, etc). Who knows, maybe its Twitter. Any other thoughts? I'll get back to you more with some detailed info. Are you able to get that exact code working on your server? On Aug 3, 2:12 pm, Joel Strellner j...@twitturly.com wrote: Hi Tom, I am not sure about XML, since I use JSON - it has a much lower over-the-wire data size, and its easier to parse. Let me know if the code works for you. -Joel On Mon, Aug 3, 2009 at 10:55 AM, Tom Fitzgerald ccexpe...@gmail.com wrote: I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curlhttp://stream.twitter.com/spritzer.xml-uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp);} ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm,JoelStrellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',',
[twitter-dev] Re: track syntax
I'm sorry Joel I keep getting a PHP timeout on the code you sent. I'll troubleshoot more and see if I can give you any more details (increase the maximum time, etc). Who knows, maybe its Twitter. Any other thoughts? I'll get back to you more with some detailed info. Are you able to get that exact code working on your server? On Aug 3, 2:12 pm, Joel Strellner j...@twitturly.com wrote: Hi Tom, I am not sure about XML, since I use JSON - it has a much lower over-the-wire data size, and its easier to parse. Let me know if the code works for you. -Joel On Mon, Aug 3, 2009 at 10:55 AM, Tom Fitzgerald ccexpe...@gmail.com wrote: I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curlhttp://stream.twitter.com/spritzer.xml-uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp);} ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm,JoelStrellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic .
[twitter-dev] Re: track syntax
I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curl http://stream.twitter.com/spritzer.xml -uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp);} ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm,JoelStrellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line //echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1;
[twitter-dev] Re: track syntax
Hi Tom, I am not sure about XML, since I use JSON - it has a much lower over-the-wire data size, and its easier to parse. Let me know if the code works for you. -Joel On Mon, Aug 3, 2009 at 10:55 AM, Tom Fitzgerald ccexpe...@gmail.com wrote: I appreciate the reply Joel. I'll give it a try. I also tried just downloading from the stream api with the curl command line. However I kept getting 'malformed xml' errors. It was weird, each tweet would have the ?xml ... tag before it. That ring any bells with you? Same thing with JSON format but it was a different error, still malformed. All I'm doing is curl http://stream.twitter.com/spritzer.xml -uuser:pass ?xml version=1.0 encoding=UTF-8? is the exact line I get before every status. If I manually clean up the XML (or JSON) it works great. On Aug 2, 7:10 pm, Joel Strellner j...@twitturly.com wrote: Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp);} ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm,JoelStrellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your
[twitter-dev] Re: track syntax
Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm, Joel Strellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line //echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp); } On Mon, Jul 27, 2009 at 11:18 AM, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the following error code: HTTP/1.1 400 Bad Request The actual relevant test PHP code is: $fp = fsockopen($url, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n; } else { echo file handle: .$fp.br/; $header = POST /track.json track = $trackTerms HTTP/1.1\r\n; $header .= Host: stream.twitter.com\r\n; $header .= User-Agent: UserAgent\r\n; $header .= Authorization: Basic . base64_encode($twitUsername . ':' . $twitPwd).\r\n; $header .= Connection: Close\r\n\r\n; echo $header.br/; fwrite($fp, $header); $line = fgets($fp, 4096); echo $line; fclose($fp); }
[twitter-dev] Re: track syntax
Other than my username and password, this is an example that I know is working: ?php $count = 1; $startparsing = false; $keyword_needles[] = 'twitter'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 10); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer: http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode (username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp); } ? The only changes I made was to echo things out and I added a keyword that I was sure would have volume - twitter. It did take a little bit of time to connect. I am assuming that that is because of their current load though, and not the script. On Aug 2, 2:25 am, Tom Fitzgerald ccexpe...@gmail.com wrote: Joel, For some reason when I try your code I get a timeout error. Any suggestions? What you have is exactly what I'm looking for. It could really help me out a jam, thanks! On Jul 27, 4:02 pm, Joel Strellner j...@twitturly.com wrote: Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer:http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line //echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp); } On Mon, Jul 27, 2009 at 11:18 AM, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the
[twitter-dev] Re: track syntax
a) the POST body (i.e. track=Palin) must come after ALL headers. b) change your password, you just sent it to the whole dev community. On Mon, Jul 27, 2009 at 12:18, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the following error code: HTTP/1.1 400 Bad Request The actual relevant test PHP code is: $fp = fsockopen($url, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n; } else { echo file handle: .$fp.br/; $header = POST /track.json track = $trackTerms HTTP/1.1\r\n; $header .= Host: stream.twitter.com\r\n; $header .= User-Agent: UserAgent\r\n; $header .= Authorization: Basic . base64_encode($twitUsername . ':' . $twitPwd).\r\n; $header .= Connection: Close\r\n\r\n; echo $header.br/; fwrite($fp, $header); $line = fgets($fp, 4096); echo $line; fclose($fp); } -- Internets. Serious business.
[twitter-dev] Re: track syntax
Thanks. I should have mentioned that the password is not real (changed the data, but kept the format to illustrate what exactly is being used). On Jul 27, 11:30 am, JDG ghil...@gmail.com wrote: a) the POST body (i.e. track=Palin) must come after ALL headers. b) change your password, you just sent it to the whole dev community. On Mon, Jul 27, 2009 at 12:18, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the following error code: HTTP/1.1 400 Bad Request The actual relevant test PHP code is: $fp = fsockopen($url, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n; } else { echo file handle: .$fp.br/; $header = POST /track.json track = $trackTerms HTTP/1.1\r\n; $header .= Host: stream.twitter.com\r\n; $header .= User-Agent: UserAgent\r\n; $header .= Authorization: Basic . base64_encode($twitUsername . ':' . $twitPwd).\r\n; $header .= Connection: Close\r\n\r\n; echo $header.br/; fwrite($fp, $header); $line = fgets($fp, 4096); echo $line; fclose($fp); } -- Internets. Serious business.
[twitter-dev] Re: track syntax
At first glance, your question is really about HTTP, not Track, as your request appears to be malformed. I'd suggest using an existing HTTP client if at all possible. You could also contrast your implementation with: http://tools.ietf.org/html/rfc2616#section-4.1. Or, get your query working with curl(1) and capture the interaction with a protocol decoder, perhaps Wireshark. Then analyze your client, and fix the deltas. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Jul 27, 11:18 am, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the following error code: HTTP/1.1 400 Bad Request The actual relevant test PHP code is: $fp = fsockopen($url, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n;} else { echo file handle: .$fp.br/; $header = POST /track.json track = $trackTerms HTTP/1.1\r\n; $header .= Host: stream.twitter.com\r\n; $header .= User-Agent: UserAgent\r\n; $header .= Authorization: Basic . base64_encode($twitUsername . ':' . $twitPwd).\r\n; $header .= Connection: Close\r\n\r\n; echo $header.br/; fwrite($fp, $header); $line = fgets($fp, 4096); echo $line; fclose($fp); }
[twitter-dev] Re: track syntax
Here is a working example of how to do /track: $count = 1; $startparsing = false; $keyword_needles[] = 'keyword1'; $keyword_needles[] = 'keyword2'; $keyword_needles[] = 'keyword3'; $keyword_needles[] = 'keyword4'; // if your keywords have spaces, they must be urlencoded (twitter does not support phrases, only the first keyword will be used, the space character and after will be ignored) foreach ($keyword_needles AS $i=$needle) { $keyword_needles[$i] = urlencode($needle); } $poststr = 'track=' . implode(',', $keyword_needles); $fp = fsockopen(stream.twitter.com, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n; } else { $out = POST /track.json HTTP/1.1\r\n; $out .= Host: stream.twitter.com\r\n; $out .= User-Agent: YourUserAgent\r\n; $out .= Referer: http://yourdomain.com\r\n;; $out .= Content-Type: application/x-www-form-urlencoded\r\n; $out .= Authorization: Basic . base64_encode(username:password).\r\n; $out .= Content-length: . strlen($poststr) . \r\n; $out .= Connection: Close\r\n\r\n; $out .= $poststr . \r\n\r\n; fwrite($fp, $out); while (!feof($fp)) { $line = fgets($fp, 4096); if ($startparsing) { if (trim($line) != '') { //echo trim($line) . \n; $tweet_obj = json_decode(trim($line)); // do your stuff here } } else { // view the header lines: uncomment the below line //echo trim($line) . \n; $header_arr[] = $line; $headercount = count($header_arr)-1; if (trim($header_arr[$headercount]) == '') { $startparsing = true; $count = 1; unset($header_arr, $headercount); } } if (trim($line) != '') $count++; } fclose($fp); } On Mon, Jul 27, 2009 at 11:18 AM, Joseph northwest...@gmail.com wrote: I am trying to use the track streaming API, but I'm having trouble with the syntax. The example shows how to use a tracking list stored in a file in the format: track = word1, word2, etc.. I tried the following (following a successful fsockopen call to stream.twitter.api: POST /track.json track = Palin, #fubar HTTP/1.1 Host: stream.twitter.com User-Agent: UserAgent Authorization: Basic cGXybmFzc3TzZGV2OlBhcm5hMzT1Mzl4MDMz Connection: Close and I am getting the following error code: HTTP/1.1 400 Bad Request The actual relevant test PHP code is: $fp = fsockopen($url, 80, $errno, $errstr, 30); if (!$fp) { echo $errstr ($errno)\n; } else { echo file handle: .$fp.br/; $header = POST /track.json track = $trackTerms HTTP/1.1\r\n; $header .= Host: stream.twitter.com\r\n; $header .= User-Agent: UserAgent\r\n; $header .= Authorization: Basic . base64_encode($twitUsername . ':' . $twitPwd).\r\n; $header .= Connection: Close\r\n\r\n; echo $header.br/; fwrite($fp, $header); $line = fgets($fp, 4096); echo $line; fclose($fp); }