[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-07 Thread John Kalucki
First you have to assume no changes to the set. Users with any significant following will see constant churn. Factoring out natural churn then: Ideally, the results are the same. Practically, the results are the same. In a very few corner cases they are not. For the next several weeks, for edges

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-07 Thread Jeffrey Greenberg
John,Please clarify this scenario. If one makes a complete set of calls starting from cursor -1 unto the end at one moment, and then another set of the same calls later is there any invariance? If so what? >From the statements above I understand: - always 5000 followers are returned (if the user

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread Brian Smith
John Kalucki wrote: > No. If we are to offer real-time social graph changes, they'll be via > the Streaming API. In the mean time, there is no low-latency high- > throughput way to determine changes to the social graph. Attempts to > simulate this at large scale via repeated polling are likely to

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread John Kalucki
I described, in some detail, the reasons for cursors here: http://groups.google.com/group/twitter-development-talk/msg/badfb7b6074aab10 If the details are uninteresting, the high-level summary is this: The paged API was designed in a previous era. Paging is simply too expensive and totally imprac

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread John Kalucki
; -Original Message- > > From: twitter-development-talk@googlegroups.com [mailto:twitter- > > development-t...@googlegroups.com] On Behalf Of John Kalucki > > Sent: Sunday, October 04, 2009 11:17 PM > > To: Twitter Development Talk > > Subject: [twitter-dev] Re: T

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread John Kalucki
There is no snapshotting. 5,000 edges are returned on each call. Few users have more than 5,000 followers or more than 5,000 followings. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Oct 6, 11:58 am, jmathai wrote: > On Oct 6, 11:06 am, Jesse Stay wrote: > > > I said the

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread Tim Haines
On Wed, Oct 7, 2009 at 7:58 AM, jmathai wrote: > > I'd be willing to sacrifice some accuracy for speed since I'm not > doing anything like auto-unfollow. From a sample set of 150k calls to > the api the average latency I have (from the west coast) is .85 > seconds. Grabbing a follower list seri

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread jmathai
On Oct 6, 11:06 am, Jesse Stay wrote: > I said the same thing in the last thread about this - still no clue what > Twitter is doing with cursors and how it is any different than the previous > paging methods. > Jesse Is the main advantage that the new method takes a snapshot of the followers lis

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread Brian Smith
, October 04, 2009 11:17 PM > To: Twitter Development Talk > Subject: [twitter-dev] Re: Twitter, Please Explain How Cursors Work > > > I haven't looked at all the parts of the system, so there's some > chance that I'm missing something. > > The method retu

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread Jesse Stay
I said the same thing in the last thread about this - still no clue what Twitter is doing with cursors and how it is any different than the previous paging methods. Jesse On Tue, Oct 6, 2009 at 10:22 AM, Dewald Pretorius wrote: > > Thanks John. However, I will be the first to put up my hand and

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-06 Thread Dewald Pretorius
Thanks John. However, I will be the first to put up my hand and say that I have no clue what you said. Can someone please translate John's answer into easy to understand language, with specific relation to the questions I asked? Dewald On Oct 5, 1:17 am, John Kalucki wrote: > I haven't looked

[twitter-dev] Re: Twitter, Please Explain How Cursors Work

2009-10-04 Thread John Kalucki
I haven't looked at all the parts of the system, so there's some chance that I'm missing something. The method returns the followers in the reverse chronological order of edge creation. Cursor A will have the most recent 5,000 edges, by creation time, B the next most recent 5,000, etc. The last c