I'm just now noticing this (I agree - why was this being announced over the holidays???) - this will make it near impossible to process large users. This is a *huge* change that just about kills any of the larger services processing very large amounts of social graph data. Please reconsider allowing the all-in-one calls. I don't want to have to explain to our users with hundreds of thousands of followers why Twitter isn't letting us read their Social Graph. (nor do I think Twitter wants us to) I had a lot of high hopes with Ryan Sarver's announcements last year of lifting limits, but this is really discouraging.
Jesse On Sun, Dec 27, 2009 at 7:29 PM, Dewald Pretorius <dpr...@gmail.com> wrote: > What is being deprecated here is the old pagination method with the > &page parameter. > > As noted earlier, it is going to cause great pain if the API is going > to assume a cursor of -1 if no cursor is specified, and hence enforce > the use of cursors regardless of the size of the social graph. > > The API is currently comfortably returning social graphs smaller than > 200,000 members in one call. I very rarely get a 502 on social graphs > of that size. It makes no sense to force us to make 40 API where 1 API > call currently suffices and works. Those 40 API calls take between 40 > and 80 seconds to complete, as opposed to 1 to 2 seconds for the > single API call. Multiply that by a few thousand Twitter accounts, and > it adds hours of additional processing time, which is completely > unnecessary, and will make getting through a large number of accounts > virtually impossible. > > > On Dec 27, 7:45 pm, Zac Bowling <zbowl...@gmail.com> wrote: > > I agree with the others to some extent. Although its a good signal to > stop > > using something ASAP when something is depreciated, saying depreciated > and > > not giving definite time-line on it's removal isn't good either. (Source > > params are deprecated but still work and don't have solid deprecation > date, > > and I'm still going on using them because OAuth sucks for desktop/mobile > > situations still and would die with a 15 day heads up on removal). > > > > Also iPhone app devs using this API will would probably have a hard time > > squeezing a 15 day return on Apple right now. > > > > Zac Bowling > > > > On Sun, Dec 27, 2009 at 3:28 PM, Dewald Pretorius <dpr...@gmail.com> > wrote: > > > I agree 100%. > > > > > Calls without the starting cursor of -1 must still return all > > > followers as is currently the case. > > > > > As a test I've set my system to use cursors on all calls. It inflates > > > the processing time so much that things become completely unworkable. > > > > > We can programmatically use cursors if showuser says that the person > > > has more than a certain number of friends/followers. That's what I'm > > > currently doing, and it works beautifully. So, please do not force us > > > to use cursors on all calls. > > > > > On Dec 24, 7:20 am, Aki <yoru.fuku...@gmail.com> wrote: > > > > I agree with PJB. The previous announcements only said that the > > > > pagination will be deprecated. > > > > > > 1. > http://groups.google.com/group/twitter-api-announce/browse_thread/thr. > > > .. > > > > 2. > http://groups.google.com/group/twitter-api-announce/browse_thread/thr. > > > .. > > > > > > However, both of the announcements did not say that the API call > > > > "without" page parameter to get > > > > all IDs will be removed or replaced with cursor pagination. > > > > The deprecation of this method is not being documented as PJB said. > > > > > > On Dec 24, 5:00 pm, PJB <pjbmancun...@gmail.com> wrote: > > > > > > > Why hasn't this been announced before? Why does the API suggest > > > > > something totally different? At the very least, can you please > hold > > > > > off on deprecation of this until 2/11/2010? This is a new API > change. > > > > > > > On Dec 23, 7:45 pm, Raffi Krikorian <ra...@twitter.com> wrote: > > > > > > > > yes - if you do not pass in cursors, then the API will behave as > > > though you > > > > > > requested the first cursor. > > > > > > > > > Willhelm: > > > > > > > > > Your announcement is apparently expanding the changeover from > page > > > to > > > > > > > cursor in new, unannounced ways?? > > > > > > > > > The API documentation page says: "If the cursor parameter is > not > > > > > > > provided, all IDs are attempted to be returned, but large sets > of > > > IDs > > > > > > > will likely fail with timeout errors." > > > > > > > > > Yesterday you wrote: "Starting soon, if you fail to pass a > cursor, > > > the > > > > > > > data returned will be that of the first cursor (-1) and the > > > > > > > next_cursor and previous_cursor elements will be included." > > > > > > > > > I can understand the need to swap from page to cursor, but was > > > pleased > > > > > > > that a single call was still available to return (or attempt to > > > > > > > return) all friend/follower ids. Now you are saying that, in > > > addition > > > > > > > to the changeover from page to cursor, you are also getting rid > of > > > > > > > this? > > > > > > > > > Can you please confirm/deny? > > > > > > > > > On Dec 22, 4:13 pm, Wilhelm Bierbaum <wilh...@twitter.com> > wrote: > > > > > > > > We noticed that some clients are still calling social graph > > > methods > > > > > > > > without cursor parameters. We wanted to take time to make > sure > > > that > > > > > > > > people were calling the updated methods which return data > with > > > cursors > > > > > > > > instead of the old formats that do not. > > > > > > > > > > As previously announced in September (http://bit.ly/46x1iL) > and > > > > > > > > November (http://bit.ly/3UQ0LU), the legacy data formats > > > returned > > > > > > > > as a result of calling social graph endpoints without a > cursor > > > > > > > > parameter are deprecated and will be removed. > > > > > > > > > > These formats have been removed from the API wiki since > > > September. > > > > > > > > > > You should always pass a cursor parameter. Starting soon, if > you > > > fail > > > > > > > > to pass a cursor, the data returned will be that of the first > > > cursor > > > > > > > > (-1) and the next_cursor and previous_cursor elements will be > > > included. > > > > > > > > > > If you aren't seeing next_cursor and previous_cursor in your > > > results, > > > > > > > > you are getting data back in the old format. You will need to > > > adjust > > > > > > > > your parser to handle the new format. > > > > > > > > > > We're going to start assuming you want data in the new format > > > > > > > > (users_list / users / user or id_list / ids / id) instead of > the > > > old > > > > > > > > format (users / user or ids / id) regardless of your passing > a > > > cursor > > > > > > > > parameter as of 1/11/2010. > > > > > > > > > > * The old formats will no longer be returned after 1/11/2010. > > > > > > > > * Start using the new formats now by passing the 'cursor' > > > parameter. > > > > > > > > > > To recap, the old endpoints at > > > > > > > > > > /statuses/friends.xml > > > > > > > > /statuses/followers.xml > > > > > > > > > > returned > > > > > > > > > > <users type="array"> > > > > > > > > <user> > > > > > > > > <!-- ... omitted ... --> > > > > > > > > </user> > > > > > > > > </users> > > > > > > > > > > or JSON like [{/*user record*/ /*, .../] > > > > > > > > > > whereas > > > > > > > > > > /statuses/friends.xml?cursor=n > > > > > > > > /statuses/followers.xml?cursor=n > > > > > > > > > > return data that looks like > > > > > > > > > > <users_list> > > > > > > > > <users type="array"> > > > > > > > > <user> > > > > > > > > <!-- ... omitted ... --> > > > > > > > > </user> > > > > > > > > </users> > > > > > > > > <next_cursor>7128872798413429387</next_cursor> > > > > > > > > <previous_cursor>0</previous_cursor> > > > > > > > > </users_list> > > > > > > > > > > or, the JSON equivalent: > > > > > > > > > > {"users":[{/*user record*/} /*, ...*/], "next_cursor":0, > > > > > > > > "previous_cursor":0} > > > > > > > > > > and the old endpoints at > > > > > > > > > > /friends/ids.xml > > > > > > > > /followers/ids.xml > > > > > > > > > > returned data that looks like > > > > > > > > > > <ids> > > > > > > > > <id>1</id> > > > > > > > > <id>2</id> > > > > > > > > <id>3</id> > > > > > > > > </ids> > > > > > > > > > > whereas > > > > > > > > > > /friends/ids.xml?cursor=n > > > > > > > > /followers/ids.xml?cursor=n > > > > > > > > > > return data that looks like > > > > > > > > > > <id_list> > > > > > > > > <ids> > > > > > > > > <id>1</id> > > > > > > > > <id>2</id> > > > > > > > > <id>3</id> > > > > > > > > </ids> > > > > > > > > <next_cursor>1288724293877798413</next_cursor> > > > > > > > > <previous_cursor>-1300794057949944903</previous_cursor> > > > > > > > > </id_list> > > > > > > > > > > or, the JSON equivalent: > > > > > > > > > > {"ids":[1, 2, 3], "next_cursor":0, "previous_cursor":0} > > > > > > > > > > If you have any questions or comments, please feel free to > post > > > them > > > > > > > > to twitter-development-talk. > > > > > > > > > > Thanks! > > > > > > > > > > -- > > > > > > > > Wilhelm Bierbaum > > > > > > > > Twitter Platform Team > > > > > > > > -- > > > > > > Raffi Krikorian > > > > > > Twitter Platform Teamhttp://twitter.com/raffi >
