I've been following the internal dialog on tracking this issue down. Given what we know, I don't think there's anything that you can change to the request parameters to reduce the chances of this happening. >From a given client, the chances of this happening to a request are pretty close to random. Different clients, however, seem from the outside to operate differently, as they tend to routed through different infrastructure. There also may be differences in the quality of the code, especially around how errors are handled.
If you want higher reliability, I'd suggest wrapping nearly all network API calls in a retry loop. If you get any sort of error: tcp, http, parser, etc. retry with linear backoff. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Sep 16, 10:23 am, Naveen A <knig...@gmail.com> wrote: > Is there a specific way we can construct our request to mitigate the > non-json response? I have used a few different twitter clients on the > same mobile device and some of them do not seem to be plagued with the > bad data like we are? Does including something in the header help get > us through whatever filter is returning the bad data? > > Maybe the Twitter cookies that are returned back on each request? > Currently, we don't pass them back on subsequent requests because they > shouldn't be necessary, but if it will make some of the bad JSON > responses go away, I'll spend the time to implement it.. > > These bad json responses have been a problem for over a month now and > while I realize it is a difficult problem to track down, the fact > remains that the API is not functioning correctly. > > A response from the twitter team would be greatly appreciated. > > On Sep 13, 6:01 am, Rudifa <rudi.far...@gmail.com> wrote: > > > I just had one non-json response, in the middle of about 10 requests > > made with curl -vvv (other responses were correct) > > > Below are 3 requests and the non-json response bracketted by 2 good > > responses which contain the response time and other logging data. > > > HTH > > Rudi > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > > -vvvhttp://twitter.com/users/show/rudifa.json > > * About to connect() to twitter.com port 80 (#0) > > * Trying 168.143.161.20... connected > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > > /users/show/rudifa.json HTTP/1.1 > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > > > OpenSSL/0.9.7l zlib/1.2.3 > > > Host: twitter.com > > > Accept: */* > > > < HTTP/1.1 200 OK > > < Date: Sun, 13 Sep 2009 09:45:23 GMT > > < Server: hi > > < X-RateLimit-Limit: 150 > > < X-Transaction: 1252835123-2408-31139 > > < Status: 200 OK > > < ETag: "df090f6c8147e20ba7fe81315a66b9af" > > < Last-Modified: Sun, 13 Sep 2009 09:45:23 GMT > > < X-RateLimit-Remaining: 124 > > < Content-Type: application/json; charset=utf-8 > > < Pragma: no-cache > > < Content-Length: 1176 > > < Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, > > post-check=0 > > < Expires: Tue, 31 Mar 1981 05:00:00 GMT > > < X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254 > > < X-RateLimit-Reset: 1252836853 > > < Set-Cookie: lang=en; path=/ > > < Set-Cookie: > > _twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWU5OGQyZmU3NWVkY2RhZjhkYTk5% > > 250ANTBlNTA4OTk0MzhhIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz > > %250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--66931156c75554797fc576876bdec52dc705 > > 736e; > > domain=.twitter.com; path=/ > > < Vary: Accept-Encoding > > < Connection: close > > < > > * Closing connection #0 > > {"profile_sidebar_border_color":"BDDCAD","description":"Wrote firmware > > for world-class osciloscopes for many years. Now learning iPhone > > programming tricks. Loves > > skiing.","url":null,"screen_name":"rudifa","status": > > {"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":null, > > "source":"<a > > href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</ > > a>","created_at":"Thu Sep 10 16:49:49 +0000 > > 2009","in_reply_to_screen_name":null,"id": > > 3890997267,"truncated":false,"text":"De retour de la T\u00eate de > > Parmelan"},"following":null,"verified":false,"profile_text_color":"333333", > > "followers_count": > > 9,"profile_background_image_url":"http://a1.twimg.com/ > > profile_background_images/17762518/ > > DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000 > > 2009","notifications":null,"friends_count": > > 29,"profile_link_color":"0084B4","profile_background_tile":false,"favourite > > s_count": > > 0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Bern", > > "location":"Geneva","name":"Rudi > > Farkas","profile_sidebar_fill_color":"DDFFCC","id": > > 36797542,"statuses_count":52,"utc_offset": > > 3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/ > > me-bsp-6a_normal.png"} > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > > -vvvhttp://twitter.com/users/show/rudifa.json > > * About to connect() to twitter.com port 80 (#0) > > * Trying 168.143.161.20... connected > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > > /users/show/rudifa.json HTTP/1.1 > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > > > OpenSSL/0.9.7l zlib/1.2.3 > > > Host: twitter.com > > > Accept: */* > > > < HTTP/1.0 200 OK > > < Connection: Close > > < Pragma: no-cache > > < cache-control: no-cache > > < Refresh: 0.1 > > < Content-Type: text/html; charset=iso-8859-1 > > < > > <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/ > > TR/1999/REC-html401-19991224/strict.dtd"> > > <!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" > > "http://www.w3.org/TR/html4/strict.dtd";> --> > > <HTML> > > <HEAD> > > <META HTTP-EQUIV="Refresh" CONTENT="0.1"> > > <META HTTP-EQUIV="Pragma" CONTENT="no-cache"> > > <META HTTP-EQUIV="Expires" CONTENT="-1"> > > <TITLE></TITLE> > > </HEAD> > > <BODY><P></BODY> > > </HTML> > > * Closing connection #0 > > > rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl > > -vvvhttp://twitter.com/users/show/rudifa.json > > * About to connect() to twitter.com port 80 (#0) > > * Trying 168.143.161.20... connected > > * Connected to twitter.com (168.143.161.20) port 80 (#0)> GET > > /users/show/rudifa.json HTTP/1.1 > > > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 > > > OpenSSL/0.9.7l zlib/1.2.3 > > > Host: twitter.com > > > Accept: */* > > > < HTTP/1.1 200 OK > > < Date: Sun, 13 Sep 2009 09:45:35 GMT > > < Server: hi > > < X-RateLimit-Limit: 150 > > < X-Transaction: 1252835135-57410-29384 > > < Status: 200 OK > > < ETag: "df090f6c8147e20ba7fe81315a66b9af" > > < Last-Modified: Sun, 13 Sep 2009 09:45:35 GMT > > < X-RateLimit-Remaining: 123 > > < Content-Type: application/json; charset=utf-8 > > < Pragma: no-cache > > < Content-Length: 1176 > > < Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, > > post-check=0 > > < Expires: Tue, 31 Mar 1981 05:00:00 GMT > > < X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254 > > < X-RateLimit-Reset: 1252836853 > > < Set-Cookie: lang=en; path=/ > > < Set-Cookie: > > _twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWY0NDM0ODQ3YWI2NDk5OTg4OTY3% > > 250AYWQ1ZjU5Njg3OWIwIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz > > %250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA-- > > d454f0c630e2e82ace19b7056947734184aea7ed; domain=.twitter.com; path=/ > > < Vary: Accept-Encoding > > < Connection: close > > < > > * Closing connection #0 > > {"profile_sidebar_border_color":"BDDCAD","description":"Wrote firmware > > for world-class osciloscopes for many years. Now learning iPhone > > programming tricks. Loves > > skiing.","url":null,"screen_name":"rudifa","status": > > {"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":null, > > "source":"<a > > href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</ > > a>","created_at":"Thu Sep 10 16:49:49 +0000 > > 2009","in_reply_to_screen_name":null,"id": > > 3890997267,"truncated":false,"text":"De retour de la T\u00eate de > > Parmelan"},"following":null,"verified":false,"profile_text_color":"333333", > > "followers_count": > > 9,"profile_background_image_url":"http://a1.twimg.com/ > > profile_background_images/17762518/ > > DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000 > > 2009","notifications":null,"friends_count": > > 29,"profile_link_color":"0084B4","profile_background_tile":false,"favourite > > s_count": > > 0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Bern", > > "location":"Geneva","name":"Rudi > > Farkas","profile_sidebar_fill_color":"DDFFCC","id": > > 36797542,"statuses_count":52,"utc_offset": > > 3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/ > > me-bsp-6a_normal.png"} > > > On Sep 12, 6:57 pm, Raffi Krikorian <ra...@twitter.com> wrote: > > > > Hi all. > > > > This is an extremely high priority problem for us, and for me > > > personally, to fix. > > > > If you're having this problem, please free to reach out to me, but > > > please try to include: > > > > * the IP address your request is coming from > > > * the request you're making > > > * the response you got back > > > * the time that the request/response was made/received > > > > (or, a tcpdump or curl output when doing -vvv would be sufficient also) > > > > On Sep 10, 2009, at 10:05 PM, Matthew Ranney <m...@ranney.com> wrote: > > > > > I'm seeing tons of these as well. > > > > > However, I've found that if you follow the suggestion of the META > > > > tag to simply "refresh" in 0.1 seconds if you get this bogus > > > > response, you can hide most of this from users, especially if they > > > > are on a fast network. > > > > > On Thu, Sep 10, 2009 at 2:09 PM, Monica Keller <monica.kel...@gmail.com > > > > > wrote: > > > > > We see this error 75% of the time. Have you guys made an progress on > > > > resolving the issue ? > > > > > On Sep 6, 8:14 pm, archF6 <tylerjpeter...@gmail.com> wrote: > > > > > I am able to consistently reproduce this error. I am making GET > > > > > requests via PHP from IP: 96.30.16.192. I receive the error without > > > > > fail after periods of inactivity lasting 2 hours or more. The > > > > header > > > > > response code is 200. Please let me know if I can provide any > > > > > additional info that might help you diagnose the problem, or if you > > > > > have suggestions about how best to handle. > > > > > > Thanks. > > > > > > On Sep 6, 3:35 pm, Rudifa <rudi.far...@gmail.com> wrote: > > > > > > > I have seen this same http page with empty body > > > > > > <!DOCTYPEhtmlPUBLIC "-//W3C//DTDHTML4.01//EN" "http://www.w3.org/ > > > > > > TR/1999/REC-html401-19991224/strict.dtd"> > > > > > > <!-- <!DOCTYPEHTMLPUBLIC "-//W3C//DTDHTML4.01//EN" > > > > > > "http://www.w3.org/TR/html4/strict.dtd";> --> > > > > > > <HTML> > > > > > > <HEAD> > > > > > > <META HTTP-EQUIV="Refresh" CONTENT="0.1"> > > > > > > <META HTTP-EQUIV="Pragma" CONTENT="no-cache"> > > > > > > <META HTTP-EQUIV="Expires" CONTENT="-1"> > > > > > > <TITLE></TITLE> > > > > > > </HEAD> > > ... > > read more »
