[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
On Sat, Sep 5, 2009 at 2:54 AM, Michael Steuer wrote: > Do you have any code examples for this? http://twitreport.tntluoma.com/id-to-name.html.sh It's a work in progress > So you basically maintain a local db of id-to- screen_names? yes. each file in a given directory is named with the ID. Each file has two lines: the first line is the users' "full name" and the second is their "screenname" If I need to lookup an ID, I just have to look to see if a file exists with a matching filename. If it doesn't, I call Twitter. Due to the nature of the way Twitter works, most of the people who started using it are people who follow me (which is how they found out about it) so there's a lot of overlap of "friendship circles" or whatever you want to call them. I haven't done much advertising of the service, which means that it hasn't grown beyond its means. > Do you get all of a user's friend/follower id's and then look up their screen > names with > individual API calls? Yes, if not cached. > What do you do when a user has more relationships than the API > limit will allow you to query? I have placed an artificial limit on the number of relationships the script will check. I think it's 2,000. So if @sween decides to follow @oprah and @oprah uses TwitReports, she's not going to get as good of as report as others. However, for the majority of the cases, I usually see people with fewer than 2,000 followers following people with fewer than 2,000 followers. > And what about users that change their > screen name? In short, can you provide a bit more background to your > methods? Changing screen names is the biggest drawback, of course. The new version of the script has built-in expiration which can be set to any number of days. I've set mine to 7. I wish that follower/friends information came with names as well as IDs, but everything I've heard from Twitter about this on this list tells me not to hold my breath… TjL ps — perhaps a better plan than just expiring the data would be validating it, which is to say: every 7/14/30 days, check to see if the information is still accurate. This could be done at off-peak hours (say 3-6am eastern time USA) when my need for API hits is lower.
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
On Sat, Sep 5, 2009 at 11:09 AM, owkaye wrote: > The ideal solution is for Twitter to "change the system" and allow each > account to have only one screen name, all the time, forever, with no > changes. Then a separate "id" value is not required because all account > identification will be done by the original screen name. This is overkill. First of all, there are hundreds, if not thousands, of abandoned usernames. I was able to take one over, which didn't interfere with anything: all my stats remained the same, I didn't lose any of my followers or friends. I know at least two people who have changed usernames to protect their privacy (switching from their "real names" to something more anonymous). > Email doesn't just let you change your address whenever you feel like it, > and I see no reason why Twitter should allow screen name changes either ... Twitter isn't email. Signed: @tj formerly @luomat forever 7344012 :-)
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
I've done this this way: Every time I get a user's data- I store it in my database. Doesn't matter if its from gardenhose or a REST method. I track various versions of it and keep all changes (good data warehousing practice). Then, when I crawl a user's list of friends I ask the database if we already have the user (by id). If we've already got them, then I don't waste an API call in asking for it again (as I only have so many). If we don't have them, then I ask Twitter for it. If the next person I go through has all the same friends, then I never make an additional REST API requests. I also do this when accessing the Search API, to make sure I have all user profiles (at least in some stage). Unfortunately I have to do this one by screen_name/from_user because the Search API returns different user id numbers (Which I also store so I can match them if I must). I've got nearly 8M twitter user profiles this way so far and its always growing. -dave On Sep 8, 12:41 pm, dizid wrote: > Owkaye, > > Well, Twitter would not need to change the system (with id's an > screenname's) because there is a user advantage to being able to > change your screenname. > > But, maybe the Twitter API could return both ID and screen_name with > only 1 call to the API. > Programmers can then choose to work with ID and/or screen_name and the > API calls would still be limited. Only the data returned would be (a > little) more... > > But, as it stands, there seems to be no other way to get screen_names > (for friends and or followers) without doing 1 API call for each > person (ID) on the returndata, which amounts to alot of API calls. > > On Sep 5, 5:09 pm, owkaye wrote: > > > The ideal solution is for Twitter to "change the system" and > > allow each account to have only one screen name, all the > > time, forever, with no changes. Then a separate "id" value > > is not required because all account identification will be > > done by the original screen name. > > > REST and SEARCH would finally be consistent. No extra calls > > to figure out who the user really is. Users would complain > > until they got used to the fact that they cannot change > > their screen names on a whim anymore, but they will learn to > > deal with it soon enough. > > > Email doesn't just let you change your address whenever you > > feel like it, and I see no reason why Twitter should allow > > screen name changes either ... except that it takes more > > work to standardize the system in this way than to continue > > with what already exists. > > > But with only the screen name as each unique account > > identifier things would certainly be much simpler. Many > > fewer requests to the server. Less data storage. And being > > that Twitter is supposed to be simple this seems like a goal > > worth pursuing, at least from my point of view. > > > Owkaye > > > > >> When i request friends (or followers) from the Twitter > > > >> API i want to get the screen_name's based on the id's. > > > > >> I use users/show for this, inputting the id and > > > >> getting back de screen_name. > > > >> This costs ALOT of API calls and i run into the API > > > >> rate limit fast, especially with many friends. > > > > >> Is there a better way of getting screen_names for > > > >> friends / followers? > > > >> ( Better, meaning in fewer API calls.)
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
Owkaye, Well, Twitter would not need to change the system (with id's an screenname's) because there is a user advantage to being able to change your screenname. But, maybe the Twitter API could return both ID and screen_name with only 1 call to the API. Programmers can then choose to work with ID and/or screen_name and the API calls would still be limited. Only the data returned would be (a little) more... But, as it stands, there seems to be no other way to get screen_names (for friends and or followers) without doing 1 API call for each person (ID) on the returndata, which amounts to alot of API calls. On Sep 5, 5:09 pm, owkaye wrote: > The ideal solution is for Twitter to "change the system" and > allow each account to have only one screen name, all the > time, forever, with no changes. Then a separate "id" value > is not required because all account identification will be > done by the original screen name. > > REST and SEARCH would finally be consistent. No extra calls > to figure out who the user really is. Users would complain > until they got used to the fact that they cannot change > their screen names on a whim anymore, but they will learn to > deal with it soon enough. > > Email doesn't just let you change your address whenever you > feel like it, and I see no reason why Twitter should allow > screen name changes either ... except that it takes more > work to standardize the system in this way than to continue > with what already exists. > > But with only the screen name as each unique account > identifier things would certainly be much simpler. Many > fewer requests to the server. Less data storage. And being > that Twitter is supposed to be simple this seems like a goal > worth pursuing, at least from my point of view. > > Owkaye > > > >> When i request friends (or followers) from the Twitter > > >> API i want to get the screen_name's based on the id's. > > > >> I use users/show for this, inputting the id and > > >> getting back de screen_name. > > >> This costs ALOT of API calls and i run into the API > > >> rate limit fast, especially with many friends. > > > >> Is there a better way of getting screen_names for > > >> friends / followers? > > >> ( Better, meaning in fewer API calls.)
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
The ideal solution is for Twitter to "change the system" and allow each account to have only one screen name, all the time, forever, with no changes. Then a separate "id" value is not required because all account identification will be done by the original screen name. REST and SEARCH would finally be consistent. No extra calls to figure out who the user really is. Users would complain until they got used to the fact that they cannot change their screen names on a whim anymore, but they will learn to deal with it soon enough. Email doesn't just let you change your address whenever you feel like it, and I see no reason why Twitter should allow screen name changes either ... except that it takes more work to standardize the system in this way than to continue with what already exists. But with only the screen name as each unique account identifier things would certainly be much simpler. Many fewer requests to the server. Less data storage. And being that Twitter is supposed to be simple this seems like a goal worth pursuing, at least from my point of view. Owkaye > >> When i request friends (or followers) from the Twitter > >> API i want to get the screen_name's based on the id's. > >> > >> I use users/show for this, inputting the id and > >> getting back de screen_name. > >> This costs ALOT of API calls and i run into the API > >> rate limit fast, especially with many friends. > >> > >> Is there a better way of getting screen_names for > >> friends / followers? > >> ( Better, meaning in fewer API calls.)
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
Thanks for your replies, I will try to explain my question better: My goal is to backup my friends (e.g. the people i follow) in a .cvs file. I first do an API call to get all friends, like this http://twitter.com/friends/ids.json?screen_name=#tu#"; method="get" result="output" /> This works great and returns (after using 'deserializejson') an array of all friends ID's which i append to a Java stringbuffer and write to a .csv file. But, what i really need are screen_name's (and not ID's), So, i am looping over this friends ID array to get screen_name for each ID with API calls to http://twitter.com/users/show.json But when a someone has, for example, 3000 friends he's is following, that means i have to do 3000 API calls, and that is the problem. So, my question: is there a way to get a users friends screen_name's in, say,1 API call, just like you can get 3000 ID's with 1 call? I have not yet tried to get my IP whitelisted (just localhost at the moment), so i have only 150 API calls/hour. But even with a whitelisted IP i would run into ratelimit problems because EACH id-to-screen_name conversion needs 1 API call. Hoping that someone found a much better way for this :) On Sep 5, 8:54 am, Michael Steuer wrote: > Do you have any code examples for this? So you basically maintain a > local db of id-to- screen_names? Do you get all of a user's friend/ > follower id's and then look up their screen names with individual API > calls? What do you do when a user has more relationships than the API > limit will allow you to query? And what about users that change their > screen name? In short, can you provide a bit more background to your > methods? > > Thanks! > > Michael > > On Sep 4, 2009, at 11:23 PM, TjL wrote: > > > > > caching is the best answer i have found > > > On Fri, Sep 4, 2009 at 9:01 PM, dizid wrote: > > >> Hi, > > >> When i request friends (or followers) from the Twitter API i want to > >> get the screen_name's based on the id's. > > >> I use users/show for this, inputting the id and getting back de > >> screen_name. > >> This costs ALOT of API calls and i run into the API rate limit fast, > >> especially with many friends. > > >> Is there a better way of getting screen_names for friends / > >> followers? > >> ( Better, meaning in fewer API calls.) > > >> Thank you.
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
Do you have any code examples for this? So you basically maintain a local db of id-to- screen_names? Do you get all of a user's friend/ follower id's and then look up their screen names with individual API calls? What do you do when a user has more relationships than the API limit will allow you to query? And what about users that change their screen name? In short, can you provide a bit more background to your methods? Thanks! Michael On Sep 4, 2009, at 11:23 PM, TjL wrote: > > caching is the best answer i have found > > On Fri, Sep 4, 2009 at 9:01 PM, dizid wrote: >> >> Hi, >> >> When i request friends (or followers) from the Twitter API i want to >> get the screen_name's based on the id's. >> >> I use users/show for this, inputting the id and getting back de >> screen_name. >> This costs ALOT of API calls and i run into the API rate limit fast, >> especially with many friends. >> >> Is there a better way of getting screen_names for friends / >> followers? >> ( Better, meaning in fewer API calls.) >> >> Thank you. >>
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
caching is the best answer i have found On Fri, Sep 4, 2009 at 9:01 PM, dizid wrote: > > Hi, > > When i request friends (or followers) from the Twitter API i want to > get the screen_name's based on the id's. > > I use users/show for this, inputting the id and getting back de > screen_name. > This costs ALOT of API calls and i run into the API rate limit fast, > especially with many friends. > > Is there a better way of getting screen_names for friends / followers? > ( Better, meaning in fewer API calls.) > > Thank you. >
[twitter-dev] Re: Getting screen_name from id without gazillion API calls?
statuts/friends and status/followers i guess On Sep 5, 3:01 am, dizid wrote: > Hi, > > When i request friends (or followers) from the Twitter API i want to > get the screen_name's based on the id's. > > I use users/show for this, inputting the id and getting back de > screen_name. > This costs ALOT of API calls and i run into the API rate limit fast, > especially with many friends. > > Is there a better way of getting screen_names for friends / followers? > ( Better, meaning in fewer API calls.) > > Thank you.
