The Adwords API appears to be behind some load balancer. Using netstat, I
could trace back the "hanging" to a specific IP. This is the output during
a "successful" request:
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 0 172.31.42.112:49312 74.125.24.101:443
ESTABLISHED 3939/python
tcp6 0 0 ::1:37156 ::1:5432
ESTABLISHED 3939/python
And this is the "hanging" request:
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 1543 172.31.42.112:57350 74.125.24.113:443
ESTABLISHED 3939/python
tcp6 0 0 ::1:37156 ::1:5432
ESTABLISHED 3939/python
So it appears that the server at 74.125.24.113 is having problems.
Could anyone confirm this?
Best regards,
Danilo
On Wednesday, October 1, 2014 2:16:08 PM UTC+2, Danilo B. wrote:
>
> I work on a project where we have a lot of outgoing HTTP requests to the
> Adwords API. The system worked good until last week, where a lot of the
> outgoing calls to Adwords started getting timeouts without any response
> from the Server.
>
> I managed to find a request to the reporting URL that seemed to trigger
> the problem every 5-10 requests (using the python-requests library):
>
> requests.post('
> https://adwords.google.com/api/adwords/reportdownload/v201406',
> data='__rdxml=XXX',
> headers={
> 'clientCustomerID': 1234,
> 'developerToken': 'XXX',
> 'Accept-Encoding': 'gzip',
> 'User-Agent': 'XXX',
> 'Content-Type': 'application/x-www-form-urlencoded',
> 'Authorization': u'Bearer XXX'
> },
> stream=False)
>
> With this, every few requests the following exception occured:
>
> > ConnectionError: HTTPSConnectionPool(host='adwords.google.com',
> port=443): Max retries exceeded with url:
> /api/adwords/reportdownload/v201406 (Caused by <class
> 'httplib.BadStatusLine'>: '')
>
> This only happens on our Amazon EC2 instance though. If I try to do the
> request from my local host, everything runs fine for hundreds of requests.
>
> The problem not only occurs when querying the reports, but also when using
> the SOAP API via the googleads library / suds.
>
> Unauthenticated GET requests on the same resource are not affected:
>
> >>> for i in range(500):
> ... url = '
> https://adwords.google.com/api/adwords/reportdownload/v201406'
> ... requests.get(url)
> ...
> <Response [400]>
> <Response [400]>
> (...)
>
> I tried to debug the call using strace. The file descriptor is created,
> then a connection is established:
>
> fcntl(8, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(8, F_SETFL, O_RDWR) = 0
> connect(8, {sa_family=AF_INET, sin_port=htons(443),
> sin_addr=inet_addr("74.125.24.113")}, 16) = 0
> setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
> getpeername(8, {sa_family=AF_INET, sin_port=htons(443),
> sin_addr=inet_addr("74.125.24.113")}, [16]) = 0
>
>
> open("/var/venv/radar/local/lib/python2.7/site-packages/requests/cacert.pem",
> O_RDONLY) = 9
> read(9, "XXX"..., 4096) = 4096
> (...)
> read(9, "XXX"..., 4096) = 1234
> read(9, "", 4096) = 0
> close(9) = 0
> munmap(0x7f63311fe000, 4096) = 0
>
> Then the TLS handshake takes place and the request is sent. But a response
> is never returned.
>
> write(8,
> "\26\3\1\1\37\1\0\1\33\3\3T+\343\2175\21\361\221X\224P\371uf\362\313O\272\353;\21"...,
>
> 292) = 292
> read(8, "\26\3\3\0=\2\0", 7) = 7
> read(8,
> "\0009\3\3T+\343\217\277\377`\370\220\177\350\nh\214\10\305W\247\37\267e\252\200\235\32\331g\276"...,
>
> 59) = 59
> read(8, "\26\3\3\16\366", 5) = 5
> read(8,
> "\v\0\16\362\0\16\357\0\7q0\202\7m0\202\6U\240\3\2\1\2\2\10%\323H%\200`\276"...,
>
> 3830) = 3830
> read(8, "\26\3\3\1M", 5) = 5
> read(8,
> "\f\0\1I\3\0\27A\4]\255\356\356\35F\340UW/w\361\323\334\244\212\242k\223\244\377\2167"...,
>
> 333) = 333
> read(8, "\26\3\3\0\4", 5) = 5
> read(8, "\16\0\0\0", 4) = 4
> write(8,
> "\26\3\3\0F\20\0\0BA\4\325\306k,\300\244*\246\34\32\232\322/,\21F\337\230\254\270\332"...,
>
> 126) = 126
> read(8, "\26\3\3\0\256", 5) = 5
> read(8,
> "\4\0\0\252\0\1\211\300\0\244\03003\177/St\230G\250ov\210&P\376
> uI?k\306"..., 174) = 174
> read(8, "\24\3\3\0\1", 5) = 5
> read(8, "\1", 1) = 1
> read(8, "\26\3\3\0(", 5) = 5
> read(8,
> "\0\0\0\0\0\0\0\0\204\251\275\247/L\205v\273R\253\207\255\215v\227\263P\r%\200r\357\7"...,
>
> 40) = 40
> write(8,
> "\27\3\3\6\2\234\35\255\207D\357\357\324\222\nZ\267\377LgY\7Zm\270\344q\20d\3406\305"...,
>
> 1543) = 1543
> fcntl(8, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(8, F_SETFL, O_RDWR) = 0
> read(8, <unfinished ...>
>
> After a few minutes of no response data, a timeout aborts the read call.
> As there is no response data, there is also no valid HTTP header and status
> code, which causes the httplib.BadStatusLine error.
>
> The connection to the server appears to be established successfully, the
> server responds with data to the TLS handshake. Also, in about 90% of the
> cases the request seems to work successfully. Therefore I suspect that the
> problem is on the server side.
>
> Any pointers on solving this problem?
>
--
--
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
Also find us on our blog and Google+:
https://googleadsdeveloper.blogspot.com/
https://plus.google.com/+GoogleAdsDevelopers/posts
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
You received this message because you are subscribed to the Google
Groups "AdWords API Forum" group.
To post to this group, send email to adwords-api@googlegroups.com
To unsubscribe from this group, send email to
adwords-api+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en
---
You received this message because you are subscribed to the Google Groups
"AdWords API Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to adwords-api+unsubscr...@googlegroups.com.
Visit this group at http://groups.google.com/group/adwords-api.
To view this discussion on the web visit
https://groups.google.com/d/msgid/adwords-api/bdbbda06-e024-4c50-bb33-aec7282550f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
