Thanks Matt I think when you said "The LWP library is finicky about authentication", you nailed it. There's more than one way to tell LWP how to authenticate, and calling 'basic_authorization' on the request object seems to make Twitter happy, whereas calling 'credentials' on the user agent object apparently doesn't. Net::Twitter uses 'credentials' rather than 'basic_authorization'. I'm not sure I want to do the necessary LWP spelunking to figure out what the difference is between the two approaches as far as the consumer is concerned.
Curiously, the 'credentials' technique appears to have worked in the past, still works for 'friends_timeline' on Twitter, and works for both 'friends_timeline' and 'user_timeline' on the Laconica implementation of the Twitter API. Go figure. One thing I notice: if you send the wrong credentials to 'user_timeline', Twitter gives back a 404 Not Found code (rather than 401 Not Authorized). If you send the wrong credentials to 'friends_timeline', Twitter gives back a 401 Not Authorized. This apparent inconsistency may be a deliberate choice on your part, though. I'm going to switch our application code over to using raw LWP rather than Net::Twitter for the small number of calls we need, and that will probably take care of it. Thanks to you and Alex for your help, Angus On Oct 22, 5:33 pm, Matt Sanford <[EMAIL PROTECTED]> wrote: > Hi Angus, > > Back in the day I was known to write some Perl so I took a crack at > user_timeline. The following works fine for me: > > use LWP::UserAgent; > use HTTP::Request; > > my $ua = LWP::UserAgent->new; > my $req=HTTP::Request->new(GET => "http://twitter.com/statuses/ > user_timeline.json"); > $req->authorization_basic('username here', 'password here'); > print $ua->request($req)->as_string > > Or, the exact command I used: > > perl -MLWP::UserAgent -e '$ua = LWP::UserAgent->new; > $req=HTTP::Request->new(GET => "http://twitter.com/statuses/ > user_timeline.json"); $req->authorization_basic($ARGV[0],$ARGV[1]); > print $ua->request($req)->as_string' USERNAME PASSWORD > > Where USERNAME and PASSWORD are your user information. The LWP > library is finicky about authentication. > > — Matt > > On Oct 22, 1:34 pm, angusmci <[EMAIL PROTECTED]> wrote: > > > Thanks for your answer. > > > I decided to bypass the Net::Twitter library altogether, and just talk > > to Twitter directly using LWP (Perl's web programming library). > > > I sent two requests to Twitter, one for "http://twitter.com/statuses/ > > user_timeline.json", and one for "http://twitter.com/statuses/ > > friends_timeline.json". In each case, I sent Twitter valid credentials > > (i.e. the username and password that I use to log in to Twitter). I > > set the LWP user agent up in exactly the same way each case. > > > The request for 'user_timeline' returned 404 Not Found; the request > > for 'friends_timeline' returned a list of updates from people I'm > > following. > > > If the authentication credentials I'm sending weren't valid, then the > > request for 'friends_timeline' should return a 401 error. The fact > > that it didn't suggests that the test script is authenticating it > > successfully at least in some cases. > > > I'm sorry to keep bothering you with this, but it really looks as if > > some parts - but only some parts - of the Twitter API have a > > mysterious prejudice against Perl ('curl' works fine). I'll try with > > Ruby and Python test scripts and see if that makes any difference. > > > Angus > > > On Oct 21, 2:20 pm, "Alex Payne" <[EMAIL PROTECTED]> wrote: > > > > > Are there any headers or arguments that are mandatory for > > > > 'user_timeline' that are not required for other, similar API > > > > functions? > > > > No, none. If you want the user timeline for the authenticating user, > > > you need to provide authentication. If you want the user timeline for > > > another user, you must specify it as part of the URL. That's all > > > there is to it. > > > > > Thanks, Angus > > > > > On Oct 20, 9:10 pm, "Alex Payne" <[EMAIL PROTECTED]> wrote: > > > >> Are you quite sure that you're making the request authenticated? It > > > >> will return a 404 if it can't authenticate you, because that URL > > > >> doesn't specify a user ID to retrieve a timeline for and thus assumes > > > >> that you want the timeline for the requesting user. > > > > >> On Mon, Oct 20, 2008 at 11:48 AM, angusmci <[EMAIL PROTECTED]> wrote: > > > > >> > I don't have a way currently to monitor what's actually going out > > > >> > over > > > >> > the wire, but I've scattered Data::Dumper calls through the > > > >> > Net::Twitter code to inspect the data structures used by the libwww > > > >> > user agent. According to that, Net::Twitter is requesting: > > > > >> > http://twitter.com/statuses/user_timeline.json?count=1&; > > > > >> > and it's sending the headers: > > > > >> > 'user-agent' => 'Net::Twitter/1.17 (PERL)', > > > >> > 'x-twitter-client-version:' => '1.17', > > > >> > 'x-twitter-client:' => 'Perl Net::Twitter', > > > >> > 'x-twitter-client-url:' => 'http://x4.net/twitter/meta.xml' > > > > >> > The authentication information that it should be sending corresponds > > > >> > to a valid Twitter account. It's supposedly using > > > >> > 'basic_authentication' against 'twitter.com:80', with realm set to > > > >> > 'Twitter API'. > > > > >> > The response headers sent back by Twitter are: > > > > >> > 'connection' => 'close', > > > >> > 'client-response-num' => 1, > > > >> > 'set-cookie' => '_twitter_sess=...; domain=.twitter.com; path=/', > > > >> > 'cache-control' => 'no-cache, no-store, must-revalidate, > > > >> > pre-check=0, > > > >> > post-check=0', > > > >> > 'last-modified' => 'Mon, 20 Oct 2008 18:17:52 GMT', > > > >> > 'status' => '404 Not Found', > > > >> > 'date' => 'Mon, 20 Oct 2008 18:17:52 GMT', > > > >> > 'client-peer' => '128.121.146.100:80', > > > >> > 'content-length' => '73', > > > >> > 'client-date' => 'Mon, 20 Oct 2008 18:17:49 GMT', > > > >> > 'pragma' => 'no-cache', > > > >> > 'content-type' => 'application/json; charset=utf-8', > > > >> > 'server' => 'hi', > > > >> > 'expires' => 'Tue, 31 Mar 1981 05:00:00 GMT' > > > > >> > (I've elided the Twitter session cookie just in case it contains > > > >> > anything sensitive). > > > > >> > Some more data points: > > > > >> > - the same request submitted to the corresponding API endpoint on > > > >> > identi.ca works correctly. > > > >> > - if I add an id parameter to my call to Net::Twitter's > > > >> > user_timeline() method, Twitter returns a valid result. > > > >> > - if I substitute 'friends' for 'user_timeline', I get the same > > > >> > behavior: 404 with no 'id' parameter, 200 if I supply an 'id' > > > >> > parameter. > > > >> > - if I substitute 'friends_timeline' for 'user_timeline', it works > > > >> > without problems: I get a 200 response, and appropriate JSON data > > > >> > - the code implementing 'friends_timeline' and 'user_timeline' in > > > >> > Net::Twitter seems to be effectively identical. > > > >> > - if I send friends_timeline the wrong password, it replies '401 Not > > > >> > authorized'. > > > >> > - if I send user_timeline the wrong password, it replies '404 Not > > > >> > Found'. > > > >> > - testing from a different host/network produces the same results > > > >> > - requesting > > > >> > 'http://twitter.com/statuses/user_timeline.json?count=1&;' > > > >> > using curl and the proper authentication credentials works > > > >> > - this worked successfully at least as recently as September 22nd > > > > >> > The fact that sending an 'id' parameter (which asks Twitter to return > > > >> > information for the user identified by 'id', and which may not depend > > > >> > on authentication) made me think that it was an authentication issue > > > >> > and that Net::Twitter was somehow fumbling authentication (although > > > >> > since Net::Twitter is using Perl LWP, I don't know how that could > > > >> > happen). However, 'friends_timeline' works as expected. > > > > >> > Thanks again for your help, > > > > >> > Angus > > > > >> > On Oct 20, 1:51 pm, "Alex Payne" <[EMAIL PROTECTED]> wrote: > > > >> >> Certain headers like If-Modified-Since will effect the responses we > > > >> >> return. Can you please provide full header output? > > > > >> >> On Sat, Oct 18, 2008 at 11:44 AM, angusmci <[EMAIL PROTECTED]> > > > >> >> wrote: > > > > >> >> > The problem was reported by one of our users, but when I tested > > > >> >> > our > > > >> >> > software with my own Twitter account - which is alive and in good > > > >> >> > standing - I got the same 404 message. > > > > >> >> > If I use 'curl', i.e. > > > > >> >> > curl -u user:passhttp://twitter.com/statuses/user_timeline.json > > > > >> >> > It works fine. However, if I use the Perl Net::Twitter equivalent, > > > >> >> > i.e. > > > > >> >> > use Net::Twitter; > > > >> >> > my $twitter = Net::Twitter->new(username => "user", password => > > > >> >> > "pass"); > > > >> >> > my $timeline = $twitter->user_timeline({ count => 1 }); > > > >> >> > print $twitter->http_code, " ", $twitter->http_message(); > > > > >> >> > I get the 404 Not Found error. > > > > >> >> > Both my simple Net::Twitter test case and our actual application > > > >> >> > have > > > >> >> > worked well in the past, and our Net::Twitter module is > > > >> >> > up-to-date. > > > > >> >> > Is it possible that Twitter's behavior depends in some way on > > > >> >> > other > > > >> >> > headers sent by the client? > > > > >> >> > Thanks for your help, > > > > >> >> > Angus > > > > >> >> > On Oct 17, 9:44 pm, "Alex Payne" <[EMAIL PROTECTED]> wrote: > > > >> >> >> Nope, hasn't changed. Is it possible that the user you're > > > >> >> >> requesting > > > >> >> >> was deleted or marked as a spammer? > > > > >> >> >> On Fri, Oct 17, 2008 at 3:48 PM, angusmci <[EMAIL PROTECTED]> > > > >> >> >> wrote: > > > > >> >> >> > A part of our app that made use of the '/statuses/ > > > >> >> >> > user_timeline.format' endpoint has broken, and tests show that > > > >> >> >> > we're > > > >> >> >> > now getting a 404 result from that (using 'curl', or the Perl > > > >> >> >> > Net::Twitter module). > > > > >> >> >> > Has the API changed recently? > > > > >> >> >> > Angus > > > > >> >> >> -- > > > >> >> >> Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x > > > > >> >> -- > > > >> >> Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x > > > > >> -- > > > >> Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x > > > > -- > > > Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
