I vote for that, too! Same scenario, same issues... bulk status request is the right solution, also for users you get from the Search API...
= Oren On Oct 20, 8:02 pm, Michael Steuer <mste...@gmail.com> wrote: > Hi, > > The reason why I¹m using followers/ids and then users/show is efficiency: > > I¹m maintaining a local cache of my users social graph. I¹m also maintaining > local user objects for my users and for their followers. Since both the > social graph and user info are subject to change, both need periodic > updating... They way I¹m doing that now is as follows: > > 1. I request followers/ids for each of my users > 2. If I detect new followers I add them to my users social graph / If I > detect followers removed, I remove them from my users social graph > > Subsequently I parse my user object table for users whose: > 1. info hasn¹t been updated in X days > 2. have no info because they were added as numeric IDs only via the > followers/ids method described above > > I then request users/show for each user matching condition 1 or 2 above. > > This way, I only get an updated user object for each unique user once, when > they¹re first added, or when I expire a previous update to their info. When > I get the followers of another new user, chances are I already know the > majority of his followers user information. > > I¹m not using statuses/followers because I would be getting the same > information over and over and over and over again... Especially when you¹re > talking about users with a lot of followers, it¹s really inefficient > considering you probably already store user info on most of the user¹s > followers... It would be an equally efficient method if overlap in followers > didn¹t exist... Since it does, I believe my approach is more efficient, and > faster over time, as your user database grows and your basically just > querying the social graph... > > ALL THAT SAID I would LOVE to have a method that allows me to get user > objects in batch... If I could request 100 user objects by numeric id in one > API call, the above would be exponentially efficient and result in far fewer > calls to Twitter. > > I am definitely interested in your feedback on my logic above and if you > think it holds... > > Thanks! > > Michael. > > On 10/15/09 7:02 PM, "Tim Haines" <tmhai...@gmail.com> wrote: > > > > > > > FYI, My backend cares. > > > On Fri, Oct 16, 2009 at 2:07 PM, jmathai <jmat...@gmail.com> wrote: > > >> I'm curious why you're using followers/ids and then users/show for > >> each id? I tried using that and using statuses/followers and found > >> that the total times were in the same ballpark. statuses/followers > >> requires far fewer api calls if you're interested in user objects. > > >> FYI, I do want to add and say I agree that either method is EXTREMELY > >> inefficient. Regardless what the argument against pages and for > >> cursors are...the current implementation is painful from an end user > >> perspective. Our backend doesn't really care, but our users don't > >> like to wait 10-30 minutes for a web page to gather a social graph. > > >> I wish instead of a cursor I could get a snapshot id, # of pages and a > >> page parameter. I don't know how it's implemented, but the ability to > >> deterministically parallelize the calls - is such a benefit to the end > >> user. Pages let me do that. > > >> On Oct 15, 9:17 am, Michael Steuer <mste...@gmail.com> wrote: > >>> > That's great!! I'm currently using the suggested method (get IDs, then > >>> > do > >>> > users/show for each of them) and it's horrendously slow and cumbersome. > >>> It'd > >>> > be great if you could get a 100 user objects at the time, based on 100 > >>> > ids > >>> > you provide.. > > >>> > On 10/14/09 7:30 PM, "Chad Etzel" <c...@twitter.com> wrote: > > >>>> > > I agree. I'm lobbying the team for something like this. > >>>> > > -Chad > > >>>> > > On Wed, Oct 14, 2009 at 10:21 PM, Josh Roesslein > >>>> > > <jroessl...@gmail.com> > >>>> wrote: > > >>>>> > >> Yeah we really need a way to bulk request user payloads by giving a > >>>>> list of > >>>>> > >> IDs. > > >>>>> > >> On Wed, Oct 14, 2009 at 9:19 PM, Tim Haines <tmhai...@gmail.com> > >>>>> wrote: > > >>>>>> > >>> Are you suggesting I should retrieve the 2k users 1 at a time > >>>>>> > >>> from > >>>>>> > >>> users/show once I have the ids? I'd essentially like to do this, > but > >>>>>> > >>> 100 at a time. > > >>>>>> > >>> I know I can get the 7000 ids in 2 calls (1 even without the > >>>>>> cursors) > >>>>>> > >>> - but I actually want the whole user objects.. > > >>>>>> > >>> Tim. > > >>>>>> > >>> On Oct 15, 2:56 pm, Chad Etzel <c...@twitter.com> wrote: > >>>>>>> > >>>> If you are pulling down the entire social graph, why not use > >>>>>>> > >>>> the > >>>>>>> > >>>> social graph calls which would deliver all 7000 ids in 2 calls? > > >>>>>>> > >>>> You can also parallelize this process by looping through > >>>>>>> different > >>>>>>> > >>>> users on each thread instead of using each thread to grab a > >>>>>>> different > >>>>>>> > >>>> page/cursor of the same user. > > >>>>>>> > >>>> Regarding the code issue you submitted, if you have the users > cached > >>>>>>> > >>>> locally, you could use the social graph methods to determine > >>>>>>> > >>>> the > >>>>>>> > >>>> missing/new 2k users pretty quickly using the social graph > >>>>>>> methods and > >>>>>>> > >>>> comparing ids. > > >>>>>>> > >>>> -Chad > > >>>>>>> > >>>> On Wed, Oct 14, 2009 at 9:50 PM, Tim Haines > >>>>>>> > >>>> <tmhai...@gmail.com> > wrote: > > >>>>>>>> > >>>>> Hi Chad, > > >>>>>>>> > >>>>> Statuses/followers. > > >>>>>>>> > >>>>> I've just timed another attempt - it took 25 minutes to > >>>>>>>> retrieve 17957 > >>>>>>>> > >>>>> followers with statuses/followers. > > >>>>>>>> > >>>>> Is there anything I can elaborate on in the filed issue to > >>>>>>>> > >>>>> make > it > >>>>>>>> > >>>>> clearer? > > >>>>>>>> > >>>>> Tim. > > >>>>>>>> > >>>>> On Oct 15, 2:42 pm, Chad Etzel <c...@twitter.com> wrote: > >>>>>>>>> > >>>>>> Hi Tim, > > >>>>>>>>> > >>>>>> You said "Retrieving 7000 followers just took > 20 minutes > for me." > >>>>>>>>> > >>>>>> Can you explain what you meant by that? > > >>>>>>>>> > >>>>>> Are you using the friends/ids, followers/ids methods or the > >>>>>>>>> > >>>>>> statuses/friends, statuses/followers methods? > > >>>>>>>>> > >>>>>> -Chad > > >>>>>>>>> > >>>>>> On Wed, Oct 14, 2009 at 8:12 PM, Tim Haines > >>>>>>>>> <tmhai...@gmail.com> wrote: > > >>>>>>>>>> > >>>>>>> Hi'ya, > > >>>>>>>>>> > >>>>>>> I'm migrating my code to use cursors at the moment. It's > >>>>>>>>>> frustrating > >>>>>>>>>> > >>>>>>> that calls need to be synchronous rather than how paged > >>>>>>>>>> calls could be > >>>>>>>>>> > >>>>>>> asynchronous. Retrieving 7000 followers just took > 20 > >>>>>>>>>> minutes for > >>>>>>>>>> > >>>>>>> me. > > >>>>>>>>>> > >>>>>>> I filed an issue that proposes a solution here: > > >>>>>>>>>> >>>>>>>http://code.google.com/p/twitter-api/issues/detail?id=1078 If > you > >>>>>>>>>> > >>>>>>> retrieve friends or followers, please take a look and > >>>>>>>>>> > >>>>>>> give > it a star > >>>>>>>>>> > >>>>>>> if it's important to you. > > >>>>>>>>>> > >>>>>>> If anyone can suggest a work around for this, I'd be > >>>>>>>>>> > >>>>>>> happy > >>>>>>>>>> to hear it. > > >>>>>>>>>> > >>>>>>> Cheers, > > >>>>>>>>>> > >>>>>>> Tim. > > >>>>> > >> -- > >>>>> > >> Josh
