Hi Liam,

Liam Middlebrook wrote on 25.07.2017:

> Hi Aleksandar,

> Responses inline.

> On 07/24/2017 11:57 PM, Aleksandar Lazic wrote:
>> Hi Liam,
>> 
>> Liam Middlebrook wrote on 24.07.2017:
>> 
>>> Hi,
>> 
>>> I'm currently running HAProxy within an Openshift Origin cluster. Until
>>> a recent update of Openshift I did not experience issues with connection
>>> timeouts, the connections would last up until the specified timeout as
>>> defined by the application.
>>>
>>> After an update to Openshift I changed HAProxy settings around to give a
>>> global 600s timeout for client and server. However when I make a form
>>> upload request the connection is killed after 30 seconds. When I signal
>>> an XHR Replay in my network inspector the connection lasts longer than
>>> the 30 seconds and is able to successfully upload the file.
>> 
>> This smells like this timeout.
>> 
>> ###
>> ROUTER_DEFAULT_SERVER_TIMEOUT 30s
>> Length of time within which a server has to acknowledge or send data. 
>> (TimeUnits)
>> ###
>> 
>> https://docs.openshift.org/latest/architecture/core_concepts/routes.html#env-variables
>> 
>> You can change it via.
>> 
>> I assume here that you have the openshift router in the default 
>> namespace and the router is deployed as "router".
>> 
>> Too much routers here ;-)
>> 
>> oc env -n default dc/router ROUTER_DEFAULT_SERVER_TIMEOUT=1h

> I wish this were the case. I've changed the router's deployment config
> quite a bit. At first extending it to 5m, then I grew suspicious that
> HAProxy settings were even changing and set it to a lower 15s.

uff that's low, imho.

> I found that when adjusting this environment variable the timeout limit
> does not change. This is especially odd since the generated
> haproxy.config file appears to have the proper changed values.
>
> sh-4.2$ grep -i timeout haproxy.config  -n |head -n 10
> 11:  stats timeout 2m
> 42:  timeout connect 5s
> 45:  timeout client 15s
> 48:  timeout server 15s
> 51:  timeout http-request 10s
> 54:  # Long timeout for WebSocket connections.
> 56:  timeout tunnel 1h
> 218:  timeout check 5000ms
> 259:  timeout check 5000ms
> 300:  timeout check 5000ms
>
> All the timeout values after line 300 are the same as lines 218, 259,
> and 300.

That means the 'oc env ...' works, good.

> With this config I am still receiving connection timeouts at 30s.

> I'm not sure if this is important to note, but the requests that are
> timing out are HTTP file uploads.

Yep I have seen this line in one of your previous mail.

>Jul 24 23:51:08 proton.csh.rit.edu haproxy[127]: 67.188.94.238:43996
>[24/Jul/2017:23:50:38.543] fe_sni~
>be_edge_http_gallery_gallery/5792c687271726c3c4b5d54ae219aaa2
>85/0/1/-1/29913 -1 0 - - CHVN 1/0/0/0/0 0/0 "POST /upload HTTP/1.1"

###
CH   The client aborted while waiting for the server to start responding.
          It might be the server taking too long to respond or the client
          clicking the 'Stop' button too fast.

VN   A cookie was provided by the client, none was inserted in the
          response. This happens for most responses for which the client has
          already got a cookie.
###

The timing looks also odd.
http://cbonte.github.io/haproxy-dconv/1.5/configuration.html#8.4

What's in your gallery log?
Does ANY Upload works?

>>> I asked in irc with no luck. Any ideas why this may be happening>
>> Do you mean the #openshift-dev channel on Freenode?
>> 
> I had asked in the #haproxy channel a few weeks ago and have just
> recently found time to re-explore this issue.

Okay I don't know that this channel exists 8-O, every day is a learning 
day ;-)

>>> Thanks,
>>>
>>> Liam Middlebrook (loothelion)
>> 
> Thanks,
> Liam Middlebrook (loothelion)

-- 
Best Regards
Aleks


Reply via email to