[twitter-dev] Re: Archive
Just got the static IP set up. It's more money, but should be worth it for 200x the API requests! Is there a way to authenticate with an account even for pages that don't require it? If not, there definitely should be. On Mar 13, 2:44 pm, Alex Payne a...@twitter.com wrote: You could always get a static IP and have us whitelist that :) On Fri, Mar 13, 2009 at 11:35, Kyle Tolle kyle.to...@gmail.com wrote: I am still encountering these issues. Wondering if I could get any help here. I feel bad bumping, but it's better than spawning another thread. Thanks, Kyle On Mar 10, 10:08 am, Kyle Tolle kyle.to...@gmail.com wrote: I first make a call tohttp://twitter.com/users/show/username.xmlto get the number of tweets. I then set the count to something like 500, and determine the number of pages. With these, I callhttp://twitter.com/statuses/user_timeline/username.xml?page=##count=## (Note: The API says you can have a count up to 3200, but any time I try that, I get an error.) So I believe I'm paging through the statuses/user_timeline corrects. I have the very first and lastest tweets I've made, but somehow there are 6 missing in between. Also, is there a way to callhttp://twitter.com/statuses/user_timeline/username.xml with authentication? I have my twitter account whitelisted with 20k api calls per hour, but my machines are on DHCP so I can't have an IP whitelisted. I got rate limited earlier with the aforementioned calls, so I suspect they aren't being authed. I appreciate the help! On Mar 10, 9:29 am, Doug Williams d...@twitter.com wrote: Kyle, Can you give an example of the calls you are using to get user tweets? It sounds like you need to page through the statuses/user_timeline method [1]. Where are you having problems? [1] -http://apiwiki.twitter.com/REST+API+Documentation#usertimeline Doug Williams Twitter API Supporthttp://twitter.com/dougw On Tue, Mar 10, 2009 at 7:20 AM, Kyle Tolle kyle.to...@gmail.com wrote: I've had trouble getting all the tweets for users while using an app to pull down what should be the entire list of tweets for that user. Is this the sort of restriction you mention? I'm not even really sure how to go about seeing which ones are missing to see if there is a pattern. I'd really like to make an archive/statistical utility, but it will not do well at either if it can't get all the tweets! I see these sorts of threads have been around for years... just wondering how much progress has been made on it. On Mar 9, 6:07 pm, Alex Payne a...@twitter.com wrote: We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
[twitter-dev] Re: Archive
On Sun, Mar 15, 2009 at 3:34 PM, Kyle Tolle kyle.to...@gmail.com wrote: Is there a way to authenticate with an account even for pages that don't require it? If not, there definitely should be. Sure, just always use your auth creds when you send a request. TJL
[twitter-dev] Re: Archive
I tried doing this with C#. The same code auths me when auth is required, but when auth is not required, it doesn't auth me. Here's the code: HttpWebRequest client = (HttpWebRequest)WebRequest.Create(xmlURL); client.UseDefaultCredentials = false; client.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequired; client.Credentials = credentials; client.Timeout = 2; client.PreAuthenticate = true; Anyone know why this wouldn't auth me on non-auth-required API calls? On Mar 15, 3:49 pm, TjL luo...@gmail.com wrote: On Sun, Mar 15, 2009 at 3:34 PM, Kyle Tolle kyle.to...@gmail.com wrote: Is there a way to authenticate with an account even for pages that don't require it? If not, there definitely should be. Sure, just always use your auth creds when you send a request. TJL
[twitter-dev] Re: Archive
The problem is that PreAuthenticate doesn't actually preauthenticate.
After one successful 401 Authentication Required it will then send the
credentials with every subsequent request.
The is the code I use to hit API's where auth is optional:
XDocument GetXDocumentFromUri(Uri uri)
{
Debug.Assert(uri != null);
Debug.Assert(uri.Scheme == Uri.UriSchemeHttp || uri.Scheme
== Uri.UriSchemeHttps);
HttpWebRequest req = HttpWebRequest.Create(uri) as
HttpWebRequest;
req.CookieContainer = _cookies;
NetworkCredential creds = Credentials.GetCredentials();
req.Headers.Add(Authorization, Basic +
Convert.ToBase64String(Encoding.ASCII.GetBytes(creds.UserName + : +
creds.Password)));
WebResponse resp = req.GetResponse();
using (StreamReader rdr = new
StreamReader(resp.GetResponseStream()))
{
return XDocument.Load(rdr);
}
}
Please only use basic auth over SSL!
Kyle Tolle wrote:
I tried doing this with C#. The same code auths me when auth is
required, but when auth is not required, it doesn't auth me.
Here's the code:
HttpWebRequest client = (HttpWebRequest)WebRequest.Create(xmlURL);
client.UseDefaultCredentials = false;
client.AuthenticationLevel =
System.Net.Security.AuthenticationLevel.MutualAuthRequired;
client.Credentials = credentials;
client.Timeout = 2;
client.PreAuthenticate = true;
Anyone know why this wouldn't auth me on non-auth-required API calls?
On Mar 15, 3:49 pm, TjL luo...@gmail.com wrote:
On Sun, Mar 15, 2009 at 3:34 PM, Kyle Tolle kyle.to...@gmail.com wrote:
Is there a way to authenticate with an account even for pages that
don't require it? If not, there definitely should be.
Sure, just always use your auth creds when you send a request.
TJL
[twitter-dev] Re: Archive
I am still encountering these issues. Wondering if I could get any help here. I feel bad bumping, but it's better than spawning another thread. Thanks, Kyle On Mar 10, 10:08 am, Kyle Tolle kyle.to...@gmail.com wrote: I first make a call tohttp://twitter.com/users/show/username.xmlto get the number of tweets. I then set the count to something like 500, and determine the number of pages. With these, I callhttp://twitter.com/statuses/user_timeline/username.xml?page=##count=## (Note: The API says you can have a count up to 3200, but any time I try that, I get an error.) So I believe I'm paging through the statuses/user_timeline corrects. I have the very first and lastest tweets I've made, but somehow there are 6 missing in between. Also, is there a way to callhttp://twitter.com/statuses/user_timeline/username.xml with authentication? I have my twitter account whitelisted with 20k api calls per hour, but my machines are on DHCP so I can't have an IP whitelisted. I got rate limited earlier with the aforementioned calls, so I suspect they aren't being authed. I appreciate the help! On Mar 10, 9:29 am, Doug Williams d...@twitter.com wrote: Kyle, Can you give an example of the calls you are using to get user tweets? It sounds like you need to page through the statuses/user_timeline method [1]. Where are you having problems? [1] -http://apiwiki.twitter.com/REST+API+Documentation#usertimeline Doug Williams Twitter API Supporthttp://twitter.com/dougw On Tue, Mar 10, 2009 at 7:20 AM, Kyle Tolle kyle.to...@gmail.com wrote: I've had trouble getting all the tweets for users while using an app to pull down what should be the entire list of tweets for that user. Is this the sort of restriction you mention? I'm not even really sure how to go about seeing which ones are missing to see if there is a pattern. I'd really like to make an archive/statistical utility, but it will not do well at either if it can't get all the tweets! I see these sorts of threads have been around for years... just wondering how much progress has been made on it. On Mar 9, 6:07 pm, Alex Payne a...@twitter.com wrote: We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
[twitter-dev] Re: Archive
You could always get a static IP and have us whitelist that :) On Fri, Mar 13, 2009 at 11:35, Kyle Tolle kyle.to...@gmail.com wrote: I am still encountering these issues. Wondering if I could get any help here. I feel bad bumping, but it's better than spawning another thread. Thanks, Kyle On Mar 10, 10:08 am, Kyle Tolle kyle.to...@gmail.com wrote: I first make a call tohttp://twitter.com/users/show/username.xmlto get the number of tweets. I then set the count to something like 500, and determine the number of pages. With these, I callhttp://twitter.com/statuses/user_timeline/username.xml?page=##count=## (Note: The API says you can have a count up to 3200, but any time I try that, I get an error.) So I believe I'm paging through the statuses/user_timeline corrects. I have the very first and lastest tweets I've made, but somehow there are 6 missing in between. Also, is there a way to callhttp://twitter.com/statuses/user_timeline/username.xml with authentication? I have my twitter account whitelisted with 20k api calls per hour, but my machines are on DHCP so I can't have an IP whitelisted. I got rate limited earlier with the aforementioned calls, so I suspect they aren't being authed. I appreciate the help! On Mar 10, 9:29 am, Doug Williams d...@twitter.com wrote: Kyle, Can you give an example of the calls you are using to get user tweets? It sounds like you need to page through the statuses/user_timeline method [1]. Where are you having problems? [1] -http://apiwiki.twitter.com/REST+API+Documentation#usertimeline Doug Williams Twitter API Supporthttp://twitter.com/dougw On Tue, Mar 10, 2009 at 7:20 AM, Kyle Tolle kyle.to...@gmail.com wrote: I've had trouble getting all the tweets for users while using an app to pull down what should be the entire list of tweets for that user. Is this the sort of restriction you mention? I'm not even really sure how to go about seeing which ones are missing to see if there is a pattern. I'd really like to make an archive/statistical utility, but it will not do well at either if it can't get all the tweets! I see these sorts of threads have been around for years... just wondering how much progress has been made on it. On Mar 9, 6:07 pm, Alex Payne a...@twitter.com wrote: We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
[twitter-dev] Re: Archive
Kyle, Can you give an example of the calls you are using to get user tweets? It sounds like you need to page through the statuses/user_timeline method [1]. Where are you having problems? [1] - http://apiwiki.twitter.com/REST+API+Documentation#usertimeline Doug Williams Twitter API Support http://twitter.com/dougw On Tue, Mar 10, 2009 at 7:20 AM, Kyle Tolle kyle.to...@gmail.com wrote: I've had trouble getting all the tweets for users while using an app to pull down what should be the entire list of tweets for that user. Is this the sort of restriction you mention? I'm not even really sure how to go about seeing which ones are missing to see if there is a pattern. I'd really like to make an archive/statistical utility, but it will not do well at either if it can't get all the tweets! I see these sorts of threads have been around for years... just wondering how much progress has been made on it. On Mar 9, 6:07 pm, Alex Payne a...@twitter.com wrote: We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
[twitter-dev] Re: Archive
I first make a call to http://twitter.com/users/show/username.xml to get the number of tweets. I then set the count to something like 500, and determine the number of pages. With these, I call http://twitter.com/statuses/user_timeline/username.xml?page=##count=## (Note: The API says you can have a count up to 3200, but any time I try that, I get an error.) So I believe I'm paging through the statuses/user_timeline corrects. I have the very first and lastest tweets I've made, but somehow there are 6 missing in between. Also, is there a way to call http://twitter.com/statuses/user_timeline/username.xml with authentication? I have my twitter account whitelisted with 20k api calls per hour, but my machines are on DHCP so I can't have an IP whitelisted. I got rate limited earlier with the aforementioned calls, so I suspect they aren't being authed. I appreciate the help! On Mar 10, 9:29 am, Doug Williams d...@twitter.com wrote: Kyle, Can you give an example of the calls you are using to get user tweets? It sounds like you need to page through the statuses/user_timeline method [1]. Where are you having problems? [1] -http://apiwiki.twitter.com/REST+API+Documentation#usertimeline Doug Williams Twitter API Supporthttp://twitter.com/dougw On Tue, Mar 10, 2009 at 7:20 AM, Kyle Tolle kyle.to...@gmail.com wrote: I've had trouble getting all the tweets for users while using an app to pull down what should be the entire list of tweets for that user. Is this the sort of restriction you mention? I'm not even really sure how to go about seeing which ones are missing to see if there is a pattern. I'd really like to make an archive/statistical utility, but it will not do well at either if it can't get all the tweets! I see these sorts of threads have been around for years... just wondering how much progress has been made on it. On Mar 9, 6:07 pm, Alex Payne a...@twitter.com wrote: We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
[twitter-dev] Re: Archive
We retain any and all tweets that aren't deleted by users. We have all the tweets ever :) We have some artificial limits in place restricting how far back in time/pages you retrieve tweets. As our data stores increase in reliability for large working sets, we'll be able to lift these restrictions and provide a reasonable quality of service to API clients. On Mon, Mar 9, 2009 at 13:07, shapper mdmo...@gmail.com wrote: Hello, When are tweets archived? It depends of how long they were created or twitter keeps always N tweets visible? Can I access older tweets? And can followers access my older tweets? Thanks, Miguel -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
