OK so I'm not complaining about the 3200 limit here, but I would like to know the most robust way to fetch all 3200 of a user's available tweets.
At the moment I'm calling user_timeline recursively, 100 tweets at a time (the maximum of 200 seems to timeout too often), passing in the oldest status id from one call onto the next as the max_id parameter. This usually works pretty well, but I'm relying on an assumption that I now realise doesn't always seem to hold: I'm expecting the response to only be an empty list when I reach the ~3200 limit. I've noticed with some timelines that an empty list will sporadically be returned well before this limit. If I keep trying the call I eventually get results, and for some accounts it happens quite regularly, often taking several attempts to get a response, but the empty list state triggers my script's "importing complete" end state prematurely. Obviously I can't rely on this assumption anymore. I'm handling errors fine, but this isn't an error, it's an HTTP 200 response with an empty result set. I've seen other apps first check the user's tweet count, then divide Max(3200, total_count) by the number of tweets per page to get a page count, and simply page++ their way through user_timeline. This seems even less robust though, as the tweet count may sometimes change by the time you get to the end of the list, and also, I've noticed the 3200 limit isn't exactly a hard cutoff. Has anyone had any experience of this and can offer advice on how best to deal with it. Seems like the 200 status is a bug in the API too. Cheers, - James -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
