Yes ...I agree for your opinion
On Sep 17, 1:36 am, "Jim Renkel" <james.ren...@gmail.com> wrote: > I agree with John that to achieve higher user visible reliability, API > requests should be wrapped in a retry loop. > > However, PLEASE, PLEASE, PLEASE, do NOT use linear backoff, i.e., > subsequent retries are delayed by an amount of time chosen uniformly at > random up to the same maximum amount for each retry. This will lead to > disasters for all API users as failed API requests, when retried, will > tend to bunch up in ever increasing bunches, leading to a higher, not > lower, failure rate. > > You should instead use exponential backoff, where the maximum amount of > delay time increases for each retry. Queuing theory and experience > indicate that the optimum factor used to increase the maximum delay for > each retry should be 2.0. > > The earliest implementations of both Ethernet and TCP, and I'm sure > other protocols, tried linear backoff and experienced this problem in > spades. When the backoff was changed to exponential, the problems > miraculously went away. > > Jim Renkel > > > > -----Original Message----- > From: twitter-development-talk@googlegroups.com > > [mailto:twitter-development-t...@googlegroups.com] On Behalf Of John > Kalucki > Sent: Wednesday, September 16, 2009 12:56 > To: Twitter Development Talk > Subject: [twitter-dev] Re: non json response > > I've been following the internal dialog on tracking this issue down. > Given what we know, I don't think there's anything that you can change > to the request parameters to reduce the chances of this happening. > From a given client, the chances of this happening to a request are > pretty close to random. Different clients, however, seem from the > outside to operate differently, as they tend to routed through > different infrastructure. There also may be differences in the quality > of the code, especially around how errors are handled. > > If you want higher reliability, I'd suggest wrapping nearly all > network API calls in a retry loop. If you get any sort of error: tcp, > http, parser, etc. retry with linear backoff. > > -John Kaluckihttp://twitter.com/jkalucki > Services, Twitter Inc. > > On Sep 16, 10:23 am, Naveen A <knig...@gmail.com> wrote: > > Is there a specific way we can construct our request to mitigate the > > non-json response? I have used a few different twitter clients on the > > same mobile device and some of them do not seem to be plagued with the > > bad data like we are? Does including something in the header help get > > us through whatever filter is returning the bad data? > > > Maybe the Twitter cookies that are returned back on each request? > > Currently, we don't pass them back on subsequent requests because they > > shouldn't be necessary, but if it will make some of the bad JSON > > responses go away, I'll spend the time to implement it.. > > > These bad json responses have been a problem for over a month now and > > while I realize it is a difficult problem to track down, the fact > > remains that the API is not functioning correctly. > > > A response from the twitter team would be greatly appreciated. > > > On Sep 13, 6:01 am, Rudifa <rudi.far...@gmail.com> wrote: > > > > I just had one non-json response, in the middle of about 10 requests > > > made with curl -vvv (other responses were correct) > > > > Below are 3 requests and the non-json response bracketted by 2 good > > > responses which contain the response time and other logging data. > > > > HTH > > > Rudi > > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > -vvvhttp://twitter.com/users/show/rudifa.json > > > * About to connect() to twitter.com port 80 (#0) > > > * Trying 168.143.161.20... connected > > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > /users/show/rudifa.json HTTP/1.1 > > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > OpenSSL/0.9.7l zlib/1.2.3 > > > > Host: twitter.com > > > > Accept: */* > > > > < HTTP/1.1 200 OK > > > < Date: Sun, 13 Sep 2009 09:45:23 GMT > > > < Server: hi > > > < X-RateLimit-Limit: 150 > > > < X-Transaction: 1252835123-2408-31139 > > > < Status: 200 OK > > > < ETag: "df090f6c8147e20ba7fe81315a66b9af" > > > < Last-Modified: Sun, 13 Sep 2009 09:45:23 GMT > > > < X-RateLimit-Remaining: 124 > > > < Content-Type: application/json; charset=utf-8 > > > < Pragma: no-cache > > > < Content-Length: 1176 > > > < Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, > > > post-check=0 > > > < Expires: Tue, 31 Mar 1981 05:00:00 GMT > > > < X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254 > > > < X-RateLimit-Reset: 1252836853 > > > < Set-Cookie: lang=en; path=/ > > > < Set-Cookie: > > _twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWU5OGQyZmU3NWVkY2RhZjhkYT > k5% 250ANTBlNTA4OTk0MzhhIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz > > %250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--66931156c75554797fc576876bdec52dc > 705 736e; > > > domain=.twitter.com; path=/ > > > < Vary: Accept-Encoding > > > < Connection: close > > > < > > > * Closing connection #0 > > > {"profile_sidebar_border_color":"BDDCAD","description":"Wrote > firmware > > > for world-class osciloscopes for many years. Now learning iPhone > > > programming tricks. Loves > > > skiing.","url":null,"screen_name":"rudifa","status": > > {"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":nu > ll, "source":"<a > > > href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</ > > > a>","created_at":"Thu Sep 10 16:49:49 +0000 > > > 2009","in_reply_to_screen_name":null,"id": > > > 3890997267,"truncated":false,"text":"De retour de la T\u00eate de > > Parmelan"},"following":null,"verified":false,"profile_text_color":"33333 > 3", "followers_count": > > > 9,"profile_background_image_url":"http://a1.twimg.com/ > > > profile_background_images/17762518/ > > > DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000 > > > 2009","notifications":null,"friends_count": > > 29,"profile_link_color":"0084B4","profile_background_tile":false,"favour > ite s_count": > > 0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Ber > n", "location":"Geneva","name":"Rudi > > > Farkas","profile_sidebar_fill_color":"DDFFCC","id": > > > 36797542,"statuses_count":52,"utc_offset": > > 3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/ > > > me-bsp-6a_normal.png"} > > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > -vvvhttp://twitter.com/users/show/rudifa.json > > > * About to connect() to twitter.com port 80 (#0) > > > * Trying 168.143.161.20... connected > > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > /users/show/rudifa.json HTTP/1.1 > > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > OpenSSL/0.9.7l zlib/1.2.3 > > > > Host: twitter.com > > > > Accept: */* > > > > < HTTP/1.0 200 OK > > > < Connection: Close > > > < Pragma: no-cache > > > < cache-control: no-cache > > > < Refresh: 0.1 > > > < Content-Type: text/html; charset=iso-8859-1 > > > < > > > <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" > "http://www.w3.org/ > > > TR/1999/REC-html401-19991224/strict.dtd"> > > > <!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" > > > "http://www.w3.org/TR/html4/strict.dtd";> --> > > > <HTML> > > > <HEAD> > > > <META HTTP-EQUIV="Refresh" CONTENT="0.1"> > > > <META HTTP-EQUIV="Pragma" CONTENT="no-cache"> > > > <META HTTP-EQUIV="Expires" CONTENT="-1"> > > > <TITLE></TITLE> > > > </HEAD> > > > <BODY><P></BODY> > > > </HTML> > > > * Closing connection #0 > > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > -vvvhttp://twitter.com/users/show/rudifa.json > > > * About to connect() to twitter.com port 80 (#0) > > > * Trying 168.143.161.20... connected > > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > /users/show/rudifa.json HTTP/1.1 > > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > OpenSSL/0.9.7l zlib/1.2.3 > > > > Host: twitter.com > > > > Accept: */* > > > > < HTTP/1.1 200 OK > > > < Date: Sun, 13 Sep 2009 09:45:35 GMT > > > < Server: hi > > > < X-RateLimit-Limit: 150 > > > < X-Transaction: 1252835135-57410-29384 > > > < Status: 200 OK > > > < ETag: "df090f6c8147e20ba7fe81315a66b9af" > > > < Last-Modified: Sun, 13 Sep 2009 09:45:35 GMT > > > < X-RateLimit-Remaining: 123 > > > < Content-Type: application/json; charset=utf-8 > > > < Pragma: no-cache > > > < Content-Length: 1176 > > > < Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, > > > post-check=0 > > > < Expires: Tue, 31 Mar 1981 05:00:00 GMT > > > < X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254 > > > < X-RateLimit-Reset: 1252836853 > > > < Set-Cookie: lang=en; path=/ > > > < Set-Cookie: > > _twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWY0NDM0ODQ3YWI2NDk5OTg4OT > Y3% 250AYWQ1ZjU5Njg3OWIwIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz > > > %250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA-- > > > d454f0c630e2e82ace19b7056947734184aea7ed; domain=.twitter.com; > path=/ > > > < Vary: Accept-Encoding > > > < Connection: close > > > < > > > * Closing connection #0 > > > {"profile_sidebar_border_color":"BDDCAD","description":"Wrote > firmware > > > for world-class osciloscopes for many years. Now learning iPhone > > > programming tricks. Loves > > > skiing.","url":null,"screen_name":"rudifa","status": > > {"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":nu > ll, "source":"<a > > > href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</ > > > a>","created_at":"Thu Sep 10 16:49:49 +0000 > > > 2009","in_reply_to_screen_name":null,"id": > > > 3890997267,"truncated":false,"text":"De retour de la T\u00eate de > > Parmelan"},"following":null,"verified":false,"profile_text_color":"33333 > 3", "followers_count": > > > 9,"profile_background_image_url":"http://a1.twimg.com/ > > > profile_background_images/17762518/ > > > DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000 > > > 2009","notifications":null,"friends_count": > > 29,"profile_link_color":"0084B4","profile_background_tile":false,"favour > ite s_count": > > 0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Ber > n", "location":"Geneva","name":"Rudi > > > Farkas","profile_sidebar_fill_color":"DDFFCC","id": > > > 36797542,"statuses_count":52,"utc_offset": > > 3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/ > > > me-bsp-6a_normal.png"} > > > > On Sep 12, 6:57 pm, Raffi Krikorian <ra...@twitter.com> wrote: > > ... > > read more »- Hide quoted text - > > - Show quoted text -
