Re: /statuses/user_timeline.format missing in action?
I'm tracking this issue here, for the record: http://code.google.com/p/twitter-api/issues/detail?id=135 On Mon, Oct 27, 2008 at 9:17 PM, Alex Payne [EMAIL PROTECTED] wrote: You entirely right Chris. The onus is on us. I'll get this fixed up tomorrow. Sorry to anyone who lost time on this bug! On Mon, Oct 27, 2008 at 7:10 PM, Chris Thompson [EMAIL PROTECTED] wrote: I am the developer of Net::Twitter. Or, at least, I was before I handed it off because I grew tired of trying to keep up with the foibles of the API. But, since the new guy hasn't released anything, my name is still on the most recent version. So I get emails from people, and questions on irc.perl.org about this. The problem in this case lies squarely on Twitter's side. Alex says: Are you quite sure that you're making the request authenticated? It will return a 404 if it can't authenticate you, because that URL doesn't specify a user ID to retrieve a timeline for and thus assumes that you want the timeline for the requesting user. This is not how HTTP Auth works. The correct handshake for a URL that needs Auth is: 1) I request, with no WWW-Authenticate: header 2) Server responds with a 401: Unauthorized and a WWW-Authenticate header containing the realm 3) I re-request with the WWW-Authenticate header containing user/pass 4) Server decides that auth header is good, responds with a 200, or decides it's bad and goes back to #2 Net::Twitter uses perl's libwww (LWP) which, in turn, implements the HTTP protocol to spec. It doesn't send the WWW-Authenticate header until it sees a 401. This is a specific part of HTTP as defined in RFC2617. If you think about it in terms of a browser like firefox, the browser CAN'T send an auth header until it is told it needs one, and it puts up an auth popup with the Realm listed that it got from the 401. LWP is doing the right thing, Twitter simply isn't asking for the auth. If you use curl or wget from the command line to hit the user_timeline url, it works. The reason for this is, you specify user and pass on the command line and both curl or wget just jam the WWW-Authenticate header in there whether it ever gets asked for it or not, violating RFC. Same with Matt Sanford's perl using authorization_basic. This is not part of LWP::UserAgent, but part of HTTP::Headers and what it does is force the WWW-Authorize header into the request, always-on, just like curl and wget, and yet again violating the RFC. LWP is only being finicky if by finicky you mean Implementing RFC2617 as written. I hate to be a pest on this, but the credentials code in Net::Twitter hasn't changed at all since Net::Twitter 1.0.0 way back in March of 2007. You guys are doing the right thing everywhere except user_timeline. If you had it throw the 401 first, you'd get the auth. 404's just flat wrong here. -- Chris Thompson -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: /statuses/user_timeline.format missing in action?
Thanks Matt
I think when you said The LWP library is finicky about
authentication, you nailed it. There's more than one way to tell LWP
how to authenticate, and calling 'basic_authorization' on the request
object seems to make Twitter happy, whereas calling 'credentials' on
the user agent object apparently doesn't. Net::Twitter uses
'credentials' rather than 'basic_authorization'. I'm not sure I want
to do the necessary LWP spelunking to figure out what the difference
is between the two approaches as far as the consumer is concerned.
Curiously, the 'credentials' technique appears to have worked in the
past, still works for 'friends_timeline' on Twitter, and works for
both 'friends_timeline' and 'user_timeline' on the Laconica
implementation of the Twitter API. Go figure.
One thing I notice: if you send the wrong credentials to
'user_timeline', Twitter gives back a 404 Not Found code (rather than
401 Not Authorized). If you send the wrong credentials to
'friends_timeline', Twitter gives back a 401 Not Authorized. This
apparent inconsistency may be a deliberate choice on your part,
though.
I'm going to switch our application code over to using raw LWP rather
than Net::Twitter for the small number of calls we need, and that will
probably take care of it.
Thanks to you and Alex for your help,
Angus
On Oct 22, 5:33 pm, Matt Sanford [EMAIL PROTECTED] wrote:
Hi Angus,
Back in the day I was known to write some Perl so I took a crack at
user_timeline. The following works fine for me:
use LWP::UserAgent;
use HTTP::Request;
my $ua = LWP::UserAgent-new;
my $req=HTTP::Request-new(GET = http://twitter.com/statuses/
user_timeline.json);
$req-authorization_basic('username here', 'password here');
print $ua-request($req)-as_string
Or, the exact command I used:
perl -MLWP::UserAgent -e '$ua = LWP::UserAgent-new;
$req=HTTP::Request-new(GET = http://twitter.com/statuses/
user_timeline.json); $req-authorization_basic($ARGV[0],$ARGV[1]);
print $ua-request($req)-as_string' USERNAME PASSWORD
Where USERNAME and PASSWORD are your user information. The LWP
library is finicky about authentication.
— Matt
On Oct 22, 1:34 pm, angusmci [EMAIL PROTECTED] wrote:
Thanks for your answer.
I decided to bypass the Net::Twitter library altogether, and just talk
to Twitter directly using LWP (Perl's web programming library).
I sent two requests to Twitter, one for http://twitter.com/statuses/
user_timeline.json, and one for http://twitter.com/statuses/
friends_timeline.json. In each case, I sent Twitter valid credentials
(i.e. the username and password that I use to log in to Twitter). I
set the LWP user agent up in exactly the same way each case.
The request for 'user_timeline' returned 404 Not Found; the request
for 'friends_timeline' returned a list of updates from people I'm
following.
If the authentication credentials I'm sending weren't valid, then the
request for 'friends_timeline' should return a 401 error. The fact
that it didn't suggests that the test script is authenticating it
successfully at least in some cases.
I'm sorry to keep bothering you with this, but it really looks as if
some parts - but only some parts - of the Twitter API have a
mysterious prejudice against Perl ('curl' works fine). I'll try with
Ruby and Python test scripts and see if that makes any difference.
Angus
On Oct 21, 2:20 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are there any headers or arguments that are mandatory for
'user_timeline' that are not required for other, similar API
functions?
No, none. If you want the user timeline for the authenticating user,
you need to provide authentication. If you want the user timeline for
another user, you must specify it as part of the URL. That's all
there is to it.
Thanks, Angus
On Oct 20, 9:10 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are you quite sure that you're making the request authenticated? It
will return a 404 if it can't authenticate you, because that URL
doesn't specify a user ID to retrieve a timeline for and thus assumes
that you want the timeline for the requesting user.
On Mon, Oct 20, 2008 at 11:48 AM, angusmci [EMAIL PROTECTED] wrote:
I don't have a way currently to monitor what's actually going out
over
the wire, but I've scattered Data::Dumper calls through the
Net::Twitter code to inspect the data structures used by the libwww
user agent. According to that, Net::Twitter is requesting:
http://twitter.com/statuses/user_timeline.json?count=1;
and it's sending the headers:
'user-agent' = 'Net::Twitter/1.17 (PERL)',
'x-twitter-client-version:' = '1.17',
'x-twitter-client:' = 'Perl Net::Twitter',
'x-twitter-client-url:' = 'http://x4.net/twitter/meta.xml'
The authentication information that it should be sending corresponds
to a valid Twitter account. It's supposedly using
Re: /statuses/user_timeline.format missing in action?
I am the developer of Net::Twitter. Or, at least, I was before I handed it off because I grew tired of trying to keep up with the foibles of the API. But, since the new guy hasn't released anything, my name is still on the most recent version. So I get emails from people, and questions on irc.perl.org about this. The problem in this case lies squarely on Twitter's side. Alex says: Are you quite sure that you're making the request authenticated? It will return a 404 if it can't authenticate you, because that URL doesn't specify a user ID to retrieve a timeline for and thus assumes that you want the timeline for the requesting user. This is not how HTTP Auth works. The correct handshake for a URL that needs Auth is: 1) I request, with no WWW-Authenticate: header 2) Server responds with a 401: Unauthorized and a WWW-Authenticate header containing the realm 3) I re-request with the WWW-Authenticate header containing user/pass 4) Server decides that auth header is good, responds with a 200, or decides it's bad and goes back to #2 Net::Twitter uses perl's libwww (LWP) which, in turn, implements the HTTP protocol to spec. It doesn't send the WWW-Authenticate header until it sees a 401. This is a specific part of HTTP as defined in RFC2617. If you think about it in terms of a browser like firefox, the browser CAN'T send an auth header until it is told it needs one, and it puts up an auth popup with the Realm listed that it got from the 401. LWP is doing the right thing, Twitter simply isn't asking for the auth. If you use curl or wget from the command line to hit the user_timeline url, it works. The reason for this is, you specify user and pass on the command line and both curl or wget just jam the WWW-Authenticate header in there whether it ever gets asked for it or not, violating RFC. Same with Matt Sanford's perl using authorization_basic. This is not part of LWP::UserAgent, but part of HTTP::Headers and what it does is force the WWW-Authorize header into the request, always-on, just like curl and wget, and yet again violating the RFC. LWP is only being finicky if by finicky you mean Implementing RFC2617 as written. I hate to be a pest on this, but the credentials code in Net::Twitter hasn't changed at all since Net::Twitter 1.0.0 way back in March of 2007. You guys are doing the right thing everywhere except user_timeline. If you had it throw the 401 first, you'd get the auth. 404's just flat wrong here. -- Chris Thompson
Re: /statuses/user_timeline.format missing in action?
I'm experiencing a similar issue with my Dashboard Twitter client using XMLHttpRequest. I'm using JSON as my format and receive a Not Found response when requesting the authenticating user's timeline. I pass the user credentials as part of the URL. ie: http://username:[EMAIL PROTECTED]/statuses/user_timeline.json?count=1 Any ideas why this would be happening? I see I'm not the only one. Thanks, Frank On Oct 27, 10:10 pm, Chris Thompson [EMAIL PROTECTED] wrote: I am the developer of Net::Twitter. Or, at least, I was before I handed it off because I grew tired of trying to keep up with the foibles of the API. But, since the new guy hasn't released anything, my name is still on the most recent version. So I get emails from people, and questions on irc.perl.org about this. The problem in this case lies squarely on Twitter's side. Alex says: Are you quite sure that you're making the request authenticated? It will return a 404 if it can't authenticate you, because that URL doesn't specify a user ID to retrieve a timeline for and thus assumes that you want the timeline for the requesting user. This is not how HTTP Auth works. The correct handshake for a URL that needs Auth is: 1) I request, with no WWW-Authenticate: header 2) Server responds with a 401: Unauthorized and a WWW-Authenticate header containing the realm 3) I re-request with the WWW-Authenticate header containing user/pass 4) Server decides that auth header is good, responds with a 200, or decides it's bad and goes back to #2 Net::Twitter uses perl's libwww (LWP) which, in turn, implements the HTTP protocol to spec. It doesn't send the WWW-Authenticate header until it sees a 401. This is a specific part of HTTP as defined in RFC2617. If you think about it in terms of a browser like firefox, the browser CAN'T send an auth header until it is told it needs one, and it puts up an auth popup with the Realm listed that it got from the 401. LWP is doing the right thing, Twitter simply isn't asking for the auth. If you use curl or wget from the command line to hit the user_timeline url, it works. The reason for this is, you specify user and pass on the command line and both curl or wget just jam the WWW-Authenticate header in there whether it ever gets asked for it or not, violating RFC. Same with Matt Sanford's perl using authorization_basic. This is not part of LWP::UserAgent, but part of HTTP::Headers and what it does is force the WWW-Authorize header into the request, always-on, just like curl and wget, and yet again violating the RFC. LWP is only being finicky if by finicky you mean Implementing RFC2617 as written. I hate to be a pest on this, but the credentials code in Net::Twitter hasn't changed at all since Net::Twitter 1.0.0 way back in March of 2007. You guys are doing the right thing everywhere except user_timeline. If you had it throw the 401 first, you'd get the auth. 404's just flat wrong here. -- Chris Thompson
Re: /statuses/user_timeline.format missing in action?
You entirely right Chris. The onus is on us. I'll get this fixed up tomorrow. Sorry to anyone who lost time on this bug! On Mon, Oct 27, 2008 at 7:10 PM, Chris Thompson [EMAIL PROTECTED] wrote: I am the developer of Net::Twitter. Or, at least, I was before I handed it off because I grew tired of trying to keep up with the foibles of the API. But, since the new guy hasn't released anything, my name is still on the most recent version. So I get emails from people, and questions on irc.perl.org about this. The problem in this case lies squarely on Twitter's side. Alex says: Are you quite sure that you're making the request authenticated? It will return a 404 if it can't authenticate you, because that URL doesn't specify a user ID to retrieve a timeline for and thus assumes that you want the timeline for the requesting user. This is not how HTTP Auth works. The correct handshake for a URL that needs Auth is: 1) I request, with no WWW-Authenticate: header 2) Server responds with a 401: Unauthorized and a WWW-Authenticate header containing the realm 3) I re-request with the WWW-Authenticate header containing user/pass 4) Server decides that auth header is good, responds with a 200, or decides it's bad and goes back to #2 Net::Twitter uses perl's libwww (LWP) which, in turn, implements the HTTP protocol to spec. It doesn't send the WWW-Authenticate header until it sees a 401. This is a specific part of HTTP as defined in RFC2617. If you think about it in terms of a browser like firefox, the browser CAN'T send an auth header until it is told it needs one, and it puts up an auth popup with the Realm listed that it got from the 401. LWP is doing the right thing, Twitter simply isn't asking for the auth. If you use curl or wget from the command line to hit the user_timeline url, it works. The reason for this is, you specify user and pass on the command line and both curl or wget just jam the WWW-Authenticate header in there whether it ever gets asked for it or not, violating RFC. Same with Matt Sanford's perl using authorization_basic. This is not part of LWP::UserAgent, but part of HTTP::Headers and what it does is force the WWW-Authorize header into the request, always-on, just like curl and wget, and yet again violating the RFC. LWP is only being finicky if by finicky you mean Implementing RFC2617 as written. I hate to be a pest on this, but the credentials code in Net::Twitter hasn't changed at all since Net::Twitter 1.0.0 way back in March of 2007. You guys are doing the right thing everywhere except user_timeline. If you had it throw the 401 first, you'd get the auth. 404's just flat wrong here. -- Chris Thompson -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: /statuses/user_timeline.format missing in action?
Thanks for your answer.
I decided to bypass the Net::Twitter library altogether, and just talk
to Twitter directly using LWP (Perl's web programming library).
I sent two requests to Twitter, one for http://twitter.com/statuses/
user_timeline.json, and one for http://twitter.com/statuses/
friends_timeline.json. In each case, I sent Twitter valid credentials
(i.e. the username and password that I use to log in to Twitter). I
set the LWP user agent up in exactly the same way each case.
The request for 'user_timeline' returned 404 Not Found; the request
for 'friends_timeline' returned a list of updates from people I'm
following.
If the authentication credentials I'm sending weren't valid, then the
request for 'friends_timeline' should return a 401 error. The fact
that it didn't suggests that the test script is authenticating it
successfully at least in some cases.
I'm sorry to keep bothering you with this, but it really looks as if
some parts - but only some parts - of the Twitter API have a
mysterious prejudice against Perl ('curl' works fine). I'll try with
Ruby and Python test scripts and see if that makes any difference.
Angus
On Oct 21, 2:20 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are there any headers or arguments that are mandatory for
'user_timeline' that are not required for other, similar API
functions?
No, none. If you want the user timeline for the authenticating user,
you need to provide authentication. If you want the user timeline for
another user, you must specify it as part of the URL. That's all
there is to it.
Thanks, Angus
On Oct 20, 9:10 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are you quite sure that you're making the request authenticated? It
will return a 404 if it can't authenticate you, because that URL
doesn't specify a user ID to retrieve a timeline for and thus assumes
that you want the timeline for the requesting user.
On Mon, Oct 20, 2008 at 11:48 AM, angusmci [EMAIL PROTECTED] wrote:
I don't have a way currently to monitor what's actually going out over
the wire, but I've scattered Data::Dumper calls through the
Net::Twitter code to inspect the data structures used by the libwww
user agent. According to that, Net::Twitter is requesting:
http://twitter.com/statuses/user_timeline.json?count=1;
and it's sending the headers:
'user-agent' = 'Net::Twitter/1.17 (PERL)',
'x-twitter-client-version:' = '1.17',
'x-twitter-client:' = 'Perl Net::Twitter',
'x-twitter-client-url:' = 'http://x4.net/twitter/meta.xml'
The authentication information that it should be sending corresponds
to a valid Twitter account. It's supposedly using
'basic_authentication' against 'twitter.com:80', with realm set to
'Twitter API'.
The response headers sent back by Twitter are:
'connection' = 'close',
'client-response-num' = 1,
'set-cookie' = '_twitter_sess=...; domain=.twitter.com; path=/',
'cache-control' = 'no-cache, no-store, must-revalidate, pre-check=0,
post-check=0',
'last-modified' = 'Mon, 20 Oct 2008 18:17:52 GMT',
'status' = '404 Not Found',
'date' = 'Mon, 20 Oct 2008 18:17:52 GMT',
'client-peer' = '128.121.146.100:80',
'content-length' = '73',
'client-date' = 'Mon, 20 Oct 2008 18:17:49 GMT',
'pragma' = 'no-cache',
'content-type' = 'application/json; charset=utf-8',
'server' = 'hi',
'expires' = 'Tue, 31 Mar 1981 05:00:00 GMT'
(I've elided the Twitter session cookie just in case it contains
anything sensitive).
Some more data points:
- the same request submitted to the corresponding API endpoint on
identi.ca works correctly.
- if I add an id parameter to my call to Net::Twitter's
user_timeline() method, Twitter returns a valid result.
- if I substitute 'friends' for 'user_timeline', I get the same
behavior: 404 with no 'id' parameter, 200 if I supply an 'id'
parameter.
- if I substitute 'friends_timeline' for 'user_timeline', it works
without problems: I get a 200 response, and appropriate JSON data
- the code implementing 'friends_timeline' and 'user_timeline' in
Net::Twitter seems to be effectively identical.
- if I send friends_timeline the wrong password, it replies '401 Not
authorized'.
- if I send user_timeline the wrong password, it replies '404 Not
Found'.
- testing from a different host/network produces the same results
- requesting 'http://twitter.com/statuses/user_timeline.json?count=1;'
using curl and the proper authentication credentials works
- this worked successfully at least as recently as September 22nd
The fact that sending an 'id' parameter (which asks Twitter to return
information for the user identified by 'id', and which may not depend
on authentication) made me think that it was an authentication issue
and that Net::Twitter was somehow fumbling authentication (although
since Net::Twitter is using Perl LWP, I don't know how that could
Re: /statuses/user_timeline.format missing in action?
Hi Angus,
Back in the day I was known to write some Perl so I took a crack at
user_timeline. The following works fine for me:
use LWP::UserAgent;
use HTTP::Request;
my $ua = LWP::UserAgent-new;
my $req=HTTP::Request-new(GET = http://twitter.com/statuses/
user_timeline.json);
$req-authorization_basic('username here', 'password here');
print $ua-request($req)-as_string
Or, the exact command I used:
perl -MLWP::UserAgent -e '$ua = LWP::UserAgent-new;
$req=HTTP::Request-new(GET = http://twitter.com/statuses/
user_timeline.json); $req-authorization_basic($ARGV[0],$ARGV[1]);
print $ua-request($req)-as_string' USERNAME PASSWORD
Where USERNAME and PASSWORD are your user information. The LWP
library is finicky about authentication.
— Matt
On Oct 22, 1:34 pm, angusmci [EMAIL PROTECTED] wrote:
Thanks for your answer.
I decided to bypass the Net::Twitter library altogether, and just talk
to Twitter directly using LWP (Perl's web programming library).
I sent two requests to Twitter, one for http://twitter.com/statuses/
user_timeline.json, and one for http://twitter.com/statuses/
friends_timeline.json. In each case, I sent Twitter valid credentials
(i.e. the username and password that I use to log in to Twitter). I
set the LWP user agent up in exactly the same way each case.
The request for 'user_timeline' returned 404 Not Found; the request
for 'friends_timeline' returned a list of updates from people I'm
following.
If the authentication credentials I'm sending weren't valid, then the
request for 'friends_timeline' should return a 401 error. The fact
that it didn't suggests that the test script is authenticating it
successfully at least in some cases.
I'm sorry to keep bothering you with this, but it really looks as if
some parts - but only some parts - of the Twitter API have a
mysterious prejudice against Perl ('curl' works fine). I'll try with
Ruby and Python test scripts and see if that makes any difference.
Angus
On Oct 21, 2:20 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are there any headers or arguments that are mandatory for
'user_timeline' that are not required for other, similar API
functions?
No, none. If you want the user timeline for the authenticating user,
you need to provide authentication. If you want the user timeline for
another user, you must specify it as part of the URL. That's all
there is to it.
Thanks, Angus
On Oct 20, 9:10 pm, Alex Payne [EMAIL PROTECTED] wrote:
Are you quite sure that you're making the request authenticated? It
will return a 404 if it can't authenticate you, because that URL
doesn't specify a user ID to retrieve a timeline for and thus assumes
that you want the timeline for the requesting user.
On Mon, Oct 20, 2008 at 11:48 AM, angusmci [EMAIL PROTECTED] wrote:
I don't have a way currently to monitor what's actually going out over
the wire, but I've scattered Data::Dumper calls through the
Net::Twitter code to inspect the data structures used by the libwww
user agent. According to that, Net::Twitter is requesting:
http://twitter.com/statuses/user_timeline.json?count=1;
and it's sending the headers:
'user-agent' = 'Net::Twitter/1.17 (PERL)',
'x-twitter-client-version:' = '1.17',
'x-twitter-client:' = 'Perl Net::Twitter',
'x-twitter-client-url:' = 'http://x4.net/twitter/meta.xml'
The authentication information that it should be sending corresponds
to a valid Twitter account. It's supposedly using
'basic_authentication' against 'twitter.com:80', with realm set to
'Twitter API'.
The response headers sent back by Twitter are:
'connection' = 'close',
'client-response-num' = 1,
'set-cookie' = '_twitter_sess=...; domain=.twitter.com; path=/',
'cache-control' = 'no-cache, no-store, must-revalidate, pre-check=0,
post-check=0',
'last-modified' = 'Mon, 20 Oct 2008 18:17:52 GMT',
'status' = '404 Not Found',
'date' = 'Mon, 20 Oct 2008 18:17:52 GMT',
'client-peer' = '128.121.146.100:80',
'content-length' = '73',
'client-date' = 'Mon, 20 Oct 2008 18:17:49 GMT',
'pragma' = 'no-cache',
'content-type' = 'application/json; charset=utf-8',
'server' = 'hi',
'expires' = 'Tue, 31 Mar 1981 05:00:00 GMT'
(I've elided the Twitter session cookie just in case it contains
anything sensitive).
Some more data points:
- the same request submitted to the corresponding API endpoint on
identi.ca works correctly.
- if I add an id parameter to my call to Net::Twitter's
user_timeline() method, Twitter returns a valid result.
- if I substitute 'friends' for 'user_timeline', I get the same
behavior: 404 with no 'id' parameter, 200 if I supply an 'id'
parameter.
- if I substitute 'friends_timeline' for 'user_timeline', it works
without problems: I get a 200 response, and appropriate JSON data
- the code implementing
