Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
See the tool here https://github.com/summerwind/h2spec Generic tests for HTTP/2 server 1. Starting HTTP/2 × 1: Sends a client connection preface -> The endpoint MUST accept client connection preface. Expected: SETTINGS Frame (flags:0x00) Actual: Error: unexpected EOF 2. Streams and Multiplexing × 1: Sends a PRIORITY frame on idle stream Error: unexpected EOF × 2: Sends a WINDOW_UPDATE frame on half-closed (remote) stream Error: unexpected EOF × 3: Sends a PRIORITY frame on half-closed (remote) stream Error: unexpected EOF × 4: Sends a RST_STREAM frame on half-closed (remote) stream Error: unexpected EOF × 5: Sends a PRIORITY frame on closed stream Error: unexpected EOF 3. Frame Definitions 3.1. DATA × 1: Sends a DATA frame Error: unexpected EOF × 2: Sends multiple DATA frames Error: unexpected EOF × 3: Sends a DATA frame with padding Error: unexpected EOF 3.2. HEADERS × 1: Sends a HEADERS frame Error: unexpected EOF × 2: Sends a HEADERS frame with padding Error: unexpected EOF × 3: Sends a HEADERS frame with priority Error: unexpected EOF 3.3. PRIORITY × 1: Sends a PRIORITY frame with priority 1 Error: unexpected EOF × 2: Sends a PRIORITY frame with priority 256 Error: unexpected EOF × 3: Sends a PRIORITY frame with stream dependency Error: unexpected EOF × 4: Sends a PRIORITY frame with exclusive Error: unexpected EOF × 5: Sends a PRIORITY frame for an idle stream, then send a HEADER frame for a lower stream ID Error: unexpected EOF 3.4. RST_STREAM × 1: Sends a RST_STREAM frame Error: unexpected EOF 3.5. SETTINGS × 1: Sends a SETTINGS frame Error: unexpected EOF 3.7. PING × 1: Sends a PING frame Error: unexpected EOF 3.8. GOAWAY × 1: Sends a GOAWAY frame Error: unexpected EOF 3.9. WINDOW_UPDATE × 1: Sends a WINDOW_UPDATE frame with stream ID 0 Error: unexpected EOF × 2: Sends a WINDOW_UPDATE frame with stream ID 1 Error: unexpected EOF 3.10. CONTINUATION × 1: Sends a CONTINUATION frame Error: unexpected EOF × 2: Sends multiple CONTINUATION frames Error: unexpected EOF 4. HTTP Message Exchanges × 1: Sends a GET request Error: unexpected EOF × 2: Sends a HEAD request Error: unexpected EOF × 3: Sends a POST request Error: unexpected EOF × 4: Sends a POST request with trailers Error: unexpected EOF 5. HPACK × 1: Sends a indexed header field representation Error: unexpected EOF × 2: Sends a literal header field with incremental indexing - indexed name Error: unexpected EOF × 3: Sends a literal header field with incremental indexing - indexed name (with Huffman coding) Error: unexpected EOF × 4: Sends a literal header field with incremental indexing - new name Error: unexpected EOF × 5: Sends a literal header field with incremental indexing - new name (with Huffman coding) Error: unexpected EOF × 6: Sends a literal header field without indexing - indexed name Error: unexpected EOF × 7: Sends a literal header field without indexing - indexed name (with Huffman coding) Error: unexpected EOF × 8: Sends a literal header field without indexing - new name Error: unexpected EOF × 9: Sends a literal header field without indexing - new name (huffman encoded) Error: unexpected EOF × 10: Sends a literal header field never indexed - indexed name Error: unexpected EOF × 11: Sends a literal header field never indexed - indexed name (huffman encoded) Error: unexpected EOF × 12: Sends a literal header field never indexed - new name Error: unexpected EOF × 13: Sends a literal header field never indexed - new name (huffman encoded) Error: unexpected EOF × 14: Sends a dynamic table size update Error: unexpected EOF × 15: Sends multiple dynamic table size update Error: unexpected EOF Hypertext Transfer Protocol Version 2 (HTTP/2) 3. Starting HTTP/2 3.5. HTTP/2 Connection Preface × 1: Sends client connection preface -> The server connection preface MUST be the first frame the server sends in the HTTP/2 connection. Expected: SETTINGS Frame (flags:0x00) Actual: Error: unexpected EOF ✔ 2: Sends invalid connection preface 4. HTTP Frames 4.1. Frame Format × 1: Sends a frame with unknown type Error: unexpected EOF × 2: Sends a frame with undefined flag Error: unexpected EOF × 3: Sends a frame with reserved field bit Error: unexpected EOF 4.2. Frame Size × 1: Sends a DATA frame with 2^14 octets in length Error: unexpected EOF × 2: Sends a large size DATA frame that exceeds the SETTINGS_MAX_FRAME_SIZE Error: rea
Re: Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
A very stripped down version of config
global
log 127.0.0.1local0
log 127.0.0.1local1 notice
maxconn 4096
pidfile /var/run/haproxy.pid
stats socket /var/run/haproxy.stat mode 600 level admin expose-fd
listeners
daemon
tune.ssl.default-dh-param 1024
tune.ssl.cachesize 10
ssl-default-bind-ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
tune.bufsize 16384
tune.maxrewrite 1024
ssl-engine rdrand
ssl-mode-async
nbthread 4
defaults
log global
mode http
compression algo gzip
compression type text/html text/plain text/css application/javascript
application/octet-stream application/json
option httplog
option dontlognull
option redispatch
option tcp-smart-accept
option tcp-smart-connect
option forwardfor
timeout check 5s
timeout client 50s
timeout tunnel 6s
timeout connect 20s
timeout http-keep-alive 15s
timeout http-request 30s
timeout queue 20s
timeout server 50s
hash-balance-factor 125
balance hdr(Cookie)
hash-type consistent djb2
stats enable
stats hide-version
stats auth :
stats uri /statz
default-server inter 5s fall 3 rise 1
frontend http
bind :::80 v4v6
redirect scheme https if !{ ssl_fc }
default_backend assets_backend
frontend https
bind :::443 v4v6 ssl crt no-tls-tickets alpn h2,http/1.1
maxconn 4096
default_backend assets_backend
On Sun, Nov 26, 2017 at 11:04 PM Willy Tarreau wrote:
On Mon, Nov 27, 2017 at 03:08:30AM +, Robin Anil wrote:
> > See the tool here https://github.com/summerwind/h2spec
>
> Thanks for the link, but could you also post your config ? Given that 100%
> of your tests ended up in "Error: unexpected EOF", I guess there's
> something
> completely wrong somewhere! If the config is not working for whatever
> reason,
> I would at least like to be able to emit a warning or an error when this
> condition is detected.
>
> Willy
>
Re: Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
I am running HaProxy 1.8 within docker container using a fork of the official build and pointed them to the latest 1.8.0 instead of rc-4. So it is built using openssl1.1 Docker hub link: https://hub.docker.com/_/haproxy/ See the Dockerfile https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/alpine/Dockerfile Just replaced these two lines ENV HAPROXY_VERSION 1.8.0 ENV HAPROXY_MD5 6ccea4619b7183fbcc8c98bae1f9823d On Sun, Nov 26, 2017 at 11:29 PM Willy Tarreau wrote: > On Mon, Nov 27, 2017 at 06:15:48AM +0100, Willy Tarreau wrote: > > On Mon, Nov 27, 2017 at 05:10:16AM +0000, Robin Anil wrote: > > > A very stripped down version of config > > > > Thank you, I'll check if anything there can explain this. > > So with your config I'm getting this : > > Finished in 52.3802 seconds > 145 tests, 110 passed, 4 skipped, 31 failed > > I had to disable ssl-mode-async as my openssl version doesn't support it. > > Of the 31, I'm seeing real bugs compared to what is expected to work, such > as : > > 3.8. GOAWAY > Ã 1: Sends a GOAWAY frame > -> The endpoint MUST accept GOAWAY frame. >Expected: Connection closed > Actual: Timeout > > Others are expected for now : > > 3.10. CONTINUATION > Ã 1: Sends a CONTINUATION frame > -> The endpoint MUST accept CONTINUATION frame. >Expected: HEADERS Frame (stream_id:1) > Actual: Connection closed > > 4. HTTP Message Exchanges > Ã 4: Sends a POST request with trailers > -> The endpoint MUST respond to the request. > Expected: HEADERS Frame (stream_id:1) >Actual: Connection closed > > And the other ones are very useful as they likely indicate missing checks. > So I'll take a look. Thanks! > > Willy >
Re: Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
Sorry, this one, not the alpine build https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/Dockerfile On Sun, Nov 26, 2017 at 11:36 PM Robin Anil wrote: > I am running HaProxy 1.8 within docker container using a fork of the > official build and pointed them to the latest 1.8.0 instead of rc-4. So it > is built using openssl1.1 > > Docker hub link: https://hub.docker.com/_/haproxy/ > See the Dockerfile > https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/alpine/Dockerfile > > Just replaced these two lines > ENV HAPROXY_VERSION 1.8.0 > ENV HAPROXY_MD5 6ccea4619b7183fbcc8c98bae1f9823d > > On Sun, Nov 26, 2017 at 11:29 PM Willy Tarreau wrote: > >> On Mon, Nov 27, 2017 at 06:15:48AM +0100, Willy Tarreau wrote: >> > On Mon, Nov 27, 2017 at 05:10:16AM +, Robin Anil wrote: >> > > A very stripped down version of config >> > >> > Thank you, I'll check if anything there can explain this. >> >> So with your config I'm getting this : >> >> Finished in 52.3802 seconds >> 145 tests, 110 passed, 4 skipped, 31 failed >> >> I had to disable ssl-mode-async as my openssl version doesn't support it. >> >> Of the 31, I'm seeing real bugs compared to what is expected to work, such >> as : >> >> 3.8. GOAWAY >> Ã 1: Sends a GOAWAY frame >> -> The endpoint MUST accept GOAWAY frame. >>Expected: Connection closed >> Actual: Timeout >> >> Others are expected for now : >> >> 3.10. CONTINUATION >> Ã 1: Sends a CONTINUATION frame >> -> The endpoint MUST accept CONTINUATION frame. >>Expected: HEADERS Frame (stream_id:1) >> Actual: Connection closed >> >> 4. HTTP Message Exchanges >> Ã 4: Sends a POST request with trailers >> -> The endpoint MUST respond to the request. >> Expected: HEADERS Frame (stream_id:1) >>Actual: Connection closed >> >> And the other ones are very useful as they likely indicate missing checks. >> So I'll take a look. Thanks! >> >> Willy >> >
Re: Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
I can get the tests to match yours now. Looks like it was a connection issue. Finished in 70.5089 seconds 145 tests, 116 passed, 0 skipped, 29 failed Also, If you haven't noticed the --strict option, in that mode I can get the test to hang on 4.2. Frame Size ✔ 1: Sends a DATA frame with 2^14 octets in length 2: Sends a large size DATA frame that exceeds the SETTINGS_MAX_FRAME_SIZE On Sun, Nov 26, 2017 at 11:39 PM Robin Anil wrote: > Sorry, this one, not the alpine build > > > https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/Dockerfile > > On Sun, Nov 26, 2017 at 11:36 PM Robin Anil wrote: > >> I am running HaProxy 1.8 within docker container using a fork of the >> official build and pointed them to the latest 1.8.0 instead of rc-4. So it >> is built using openssl1.1 >> >> Docker hub link: https://hub.docker.com/_/haproxy/ >> See the Dockerfile >> https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/alpine/Dockerfile >> >> Just replaced these two lines >> ENV HAPROXY_VERSION 1.8.0 >> ENV HAPROXY_MD5 6ccea4619b7183fbcc8c98bae1f9823d >> >> On Sun, Nov 26, 2017 at 11:29 PM Willy Tarreau wrote: >> >>> On Mon, Nov 27, 2017 at 06:15:48AM +0100, Willy Tarreau wrote: >>> > On Mon, Nov 27, 2017 at 05:10:16AM +, Robin Anil wrote: >>> > > A very stripped down version of config >>> > >>> > Thank you, I'll check if anything there can explain this. >>> >>> So with your config I'm getting this : >>> >>> Finished in 52.3802 seconds >>> 145 tests, 110 passed, 4 skipped, 31 failed >>> >>> I had to disable ssl-mode-async as my openssl version doesn't support it. >>> >>> Of the 31, I'm seeing real bugs compared to what is expected to work, >>> such >>> as : >>> >>> 3.8. GOAWAY >>> Ã 1: Sends a GOAWAY frame >>> -> The endpoint MUST accept GOAWAY frame. >>>Expected: Connection closed >>> Actual: Timeout >>> >>> Others are expected for now : >>> >>> 3.10. CONTINUATION >>> Ã 1: Sends a CONTINUATION frame >>> -> The endpoint MUST accept CONTINUATION frame. >>>Expected: HEADERS Frame (stream_id:1) >>> Actual: Connection closed >>> >>> 4. HTTP Message Exchanges >>> Ã 4: Sends a POST request with trailers >>> -> The endpoint MUST respond to the request. >>> Expected: HEADERS Frame (stream_id:1) >>>Actual: Connection closed >>> >>> And the other ones are very useful as they likely indicate missing >>> checks. >>> So I'll take a look. Thanks! >>> >>> Willy >>> >>
Re: Haproxy 1.8 http/2 mode does not pass the h2spec conformance test
Hi WIlly, Most of the tests pass now in 1.8.2. However, the following test still hangs in the --strict mode. 2: Sends a large size DATA frame that exceeds the SETTINGS_MAX_FRAME_SIZE On Sun, Nov 26, 2017 at 11:54 PM Robin Anil wrote: > I can get the tests to match yours now. Looks like it was a connection > issue. > Finished in 70.5089 seconds > > 145 tests, 116 passed, 0 skipped, 29 failed > > > > Also, If you haven't noticed the --strict option, in that mode I can get > the test to hang on > >4.2. Frame Size > > ✔ 1: Sends a DATA frame with 2^14 octets in length > > 2: Sends a large size DATA frame that exceeds the > SETTINGS_MAX_FRAME_SIZE > > On Sun, Nov 26, 2017 at 11:39 PM Robin Anil wrote: > >> Sorry, this one, not the alpine build >> >> >> https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/Dockerfile >> >> On Sun, Nov 26, 2017 at 11:36 PM Robin Anil wrote: >> >>> I am running HaProxy 1.8 within docker container using a fork of the >>> official build and pointed them to the latest 1.8.0 instead of rc-4. So it >>> is built using openssl1.1 >>> >>> Docker hub link: https://hub.docker.com/_/haproxy/ >>> See the Dockerfile >>> https://github.com/docker-library/haproxy/blob/0c9c27713bfca8505331e0da2664a9454755c7b9/1.8-rc/alpine/Dockerfile >>> >>> Just replaced these two lines >>> ENV HAPROXY_VERSION 1.8.0 >>> ENV HAPROXY_MD5 6ccea4619b7183fbcc8c98bae1f9823d >>> >>> On Sun, Nov 26, 2017 at 11:29 PM Willy Tarreau wrote: >>> >>>> On Mon, Nov 27, 2017 at 06:15:48AM +0100, Willy Tarreau wrote: >>>> > On Mon, Nov 27, 2017 at 05:10:16AM +, Robin Anil wrote: >>>> > > A very stripped down version of config >>>> > >>>> > Thank you, I'll check if anything there can explain this. >>>> >>>> So with your config I'm getting this : >>>> >>>> Finished in 52.3802 seconds >>>> 145 tests, 110 passed, 4 skipped, 31 failed >>>> >>>> I had to disable ssl-mode-async as my openssl version doesn't support >>>> it. >>>> >>>> Of the 31, I'm seeing real bugs compared to what is expected to work, >>>> such >>>> as : >>>> >>>> 3.8. GOAWAY >>>> Ã 1: Sends a GOAWAY frame >>>> -> The endpoint MUST accept GOAWAY frame. >>>>Expected: Connection closed >>>> Actual: Timeout >>>> >>>> Others are expected for now : >>>> >>>> 3.10. CONTINUATION >>>> Ã 1: Sends a CONTINUATION frame >>>> -> The endpoint MUST accept CONTINUATION frame. >>>>Expected: HEADERS Frame (stream_id:1) >>>> Actual: Connection closed >>>> >>>> 4. HTTP Message Exchanges >>>> Ã 4: Sends a POST request with trailers >>>> -> The endpoint MUST respond to the request. >>>> Expected: HEADERS Frame (stream_id:1) >>>>Actual: Connection closed >>>> >>>> And the other ones are very useful as they likely indicate missing >>>> checks. >>>> So I'll take a look. Thanks! >>>> >>>> Willy >>>> >>>
HAProxy 1.8 takes too long to update new config
In Http/2 mode with threads enabled, updating the config on a live serving haproxy server takes several minutes. If I turn this off, the update of config is near instantaneous. The config change itself is just adding a newline in the file. So it feels like haproxy is waiting for connections to close down or something... My question is, is this a known behavior? Definitely not desired, not unless the stats page has some indicator that update is happening and connections are being migrated over or something. Robin
Re: HAProxy 1.8 takes too long to update new config
I can isolate that to http2, not threads at v1.8.2 Separately, I am seeing a lot of intermittent failures with POST/PUT requests, I see the following returned from haproxy. 400 Bad request Your browser sent an invalid request. I can verify these requests never hit the backend. On Thu, Dec 28, 2017 at 1:47 AM Robin Anil wrote: > In Http/2 mode with threads enabled, updating the config on a live serving > haproxy server takes several minutes. > > If I turn this off, the update of config is near instantaneous. > > The config change itself is just adding a newline in the file. So it feels > like haproxy is waiting for connections to close down or something... > > My question is, is this a known behavior? Definitely not desired, not > unless the stats page has some indicator that update is happening and > connections are being migrated over or something. > > Robin >
Re: HAProxy 1.8 takes too long to update new config
I just saw. since I don't know how to add to a thread which I am not part of (I still haven't subscribed to the haproxy mailing list). I will have to indicate here that the issue I am having is not in firefox but on the latest chrome stable. I can consistently make it happen in our discourse instance behind haproxy 1.8.2 when http2 is enabled. The config update issue is the main purpose of this thread though. On Thu, Dec 28, 2017 at 2:55 AM Lucas Rolff wrote: > Robin, there’s also an ongoing thread with Firefox which has the same > issues, especially with post/put requests in 1.8.2, you might wanna keep an > eye on that one as well > > Get Outlook for iOS <https://aka.ms/o0ukef> > ------ > *From:* Robin Anil > *Sent:* Thursday, December 28, 2017 9:36:56 AM > *To:* haproxy@formilux.org > *Subject:* Re: HAProxy 1.8 takes too long to update new config > > I can isolate that to http2, not threads at v1.8.2 > > Separately, I am seeing a lot of intermittent failures with POST/PUT > requests, I see the following returned from haproxy. > 400 Bad request > Your browser sent an invalid request. > > > I can verify these requests never hit the backend. > > > > > > > On Thu, Dec 28, 2017 at 1:47 AM Robin Anil wrote: > >> In Http/2 mode with threads enabled, updating the config on a live >> serving haproxy server takes several minutes. >> >> If I turn this off, the update of config is near instantaneous. >> >> The config change itself is just adding a newline in the file. So it >> feels like haproxy is waiting for connections to close down or something... >> >> My question is, is this a known behavior? Definitely not desired, not >> unless the stats page has some indicator that update is happening and >> connections are being migrated over or something. >> >> Robin >> >
Re: HAProxy 1.8 takes too long to update new config
- from a phone, xquz typoz On Dec 28, 2017 3:31 AM, "Willy Tarreau" wrote: Hi Robert, On Thu, Dec 28, 2017 at 07:47:03AM +0000, Robin Anil wrote: > In Http/2 mode with threads enabled, updating the config on a live serving > haproxy server takes several minutes. > > If I turn this off, the update of config is near instantaneous. I'm sorry but I don't understand what you call "this" above nor what you mean by "updating the config". If the server is running in http2 mode, and servicing connections, updating the config as shown below is no longer instantaneous. Takes over 5 minutes. Even when the update is a no-op like adding a new line at the end of the existing config. If the server is running in non http2 mode then any config updates are near instantaneous haproxy -f new_haproxy.cfg -p haproxy.pid -sf $(cat haproxy.pid) > The config change itself is just adding a newline in the file. So it feels > like haproxy is waiting for connections to close down or something... How do you proceed for the update and how do you measure this "time" ? For me "updating" means reloading, where a new process parses the new file. Since I hardly see how it could take time to parse a file in which you only added a few lines, it's unclear to me what exact problem you're facing. Could you maybe explain the steps you're taking and what you're observing ? Thanks, Willy
Re: HAProxy 1.8 takes too long to update new config
I see, not only is the command stuck, the stats page does not load beyond top header part and that state continues to persist for about 5-6 minutes. Here is my configuration. defaults log global mode http compression algo gzip compression type text/html text/plain text/css application/javascript application/octet-stream application/json option httplog option dontlognull option redispatch option tcp-smart-accept option tcp-smart-connect option forwardfor timeout check 5s timeout client 50s timeout tunnel 6s timeout connect 20s timeout http-keep-alive 15s timeout http-request 30s timeout queue 20s timeout server 50s hash-balance-factor 125 balance hdr(Cookie) hash-type consistent djb2 stats enable stats hide-version stats uri /statz default-server inter 5s fall 3 rise 1 On Thu, Dec 28, 2017 at 4:27 AM Lukas Tribus wrote: > Hello, > > > > I'm sorry but I don't understand what you call "this" above nor what you > > mean by "updating the config". > > > > If the server is running in http2 mode, and servicing connections, > updating > > the config as shown below is no longer instantaneous. Takes over 5 > minutes. > > So what you are saying is that *the command* to reload haproxy takes 5 > minutes. I assume that within that timeframe, both old and new > requests continue to be served. > > H2 timeouts are different than H1. Please share you configuration, > especially everything timeout related. I assume "timeout client" is > more than a few seconds in your configuration? > > > You may want to lower "timeout client" and configure hard-stop-after > according to your expectation: > > http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#hard-stop-after > > The different timeout behavior in H2 is already documented: > > http://git.haproxy.org/?p=haproxy-1.8.git;a=commitdiff;h=75df9d7a7acb741e8413a6c2f3d6b6fe07b44bb8 > > > > Regarding email threads, you can look for the post on mail-archive.org > and hit that Reply-to button at the bottom (it will use the msg-id as > in the in-reply-to header, at least in thunderbird) - or you use do it > on your own, the msg-id of the email is at in the bottom line: > https://www.mail-archive.com/haproxy@formilux.org/ > > > lukas >
Re: HAProxy 1.8 takes too long to update new config
hard-stop-after is indeed helping, now it waits for exactly 30s (as configured in the hard-stop-after). Thanks for clarifying. On Thu, Dec 28, 2017 at 11:01 AM Robin Anil wrote: > I see, not only is the command stuck, the stats page does not load beyond > top header part and that state continues to persist for about 5-6 minutes. > > Here is my configuration. > > > defaults > log global > mode http > compression algo gzip > compression type text/html text/plain text/css application/javascript > application/octet-stream application/json > option httplog > option dontlognull > option redispatch > option tcp-smart-accept > option tcp-smart-connect > option forwardfor > timeout check 5s > timeout client 50s > timeout tunnel 6s > timeout connect 20s > timeout http-keep-alive 15s > timeout http-request 30s > timeout queue 20s > timeout server 50s > hash-balance-factor 125 > balance hdr(Cookie) > hash-type consistent djb2 > stats enable > stats hide-version > stats uri /statz > > default-server inter 5s fall 3 rise 1 > > On Thu, Dec 28, 2017 at 4:27 AM Lukas Tribus wrote: > >> Hello, >> >> >> > I'm sorry but I don't understand what you call "this" above nor what you >> > mean by "updating the config". >> > >> > If the server is running in http2 mode, and servicing connections, >> updating >> > the config as shown below is no longer instantaneous. Takes over 5 >> minutes. >> >> So what you are saying is that *the command* to reload haproxy takes 5 >> minutes. I assume that within that timeframe, both old and new >> requests continue to be served. >> >> H2 timeouts are different than H1. Please share you configuration, >> especially everything timeout related. I assume "timeout client" is >> more than a few seconds in your configuration? >> >> >> You may want to lower "timeout client" and configure hard-stop-after >> according to your expectation: >> >> http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#hard-stop-after >> >> The different timeout behavior in H2 is already documented: >> >> http://git.haproxy.org/?p=haproxy-1.8.git;a=commitdiff;h=75df9d7a7acb741e8413a6c2f3d6b6fe07b44bb8 >> >> >> >> Regarding email threads, you can look for the post on mail-archive.org >> and hit that Reply-to button at the bottom (it will use the msg-id as >> in the in-reply-to header, at least in thunderbird) - or you use do it >> on your own, the msg-id of the email is at in the bottom line: >> https://www.mail-archive.com/haproxy@formilux.org/ >> >> >> lukas >> >
Intermittent 500 errors with http/2 and 1.8.4
After seeing the 1.8.4 release, I decided to upgrade and try to run http/2
again
Unfortunately as soon I enable it, I see intermittent 500 errors from our
fleet. I know a lot of work went in between 1.8.2 and 1.8.3 to reduce these
errors, yes its very few but a customer did notice, so we had to roll it
back. Let me know how I can help diagnose this. [image: Screen Shot
2018-02-10 at 12.45.14 AM.png]
All I need is to enable * alpn h2,http/1.1 *to see 500 errors happening.
global
log 127.0.0.1local0
log 127.0.0.1local1 notice
maxconn 4096
pidfile /var/run/haproxy.pid
stats socket /var/run/haproxy.stat mode 600 level admin
daemon
hard-stop-after 30s
tune.ssl.default-dh-param 1024
tune.ssl.cachesize 10
ssl-default-bind-ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
tune.bufsize 16384
tune.maxrewrite 1024
ssl-engine rdrand
ssl-mode-async
nbthread 4
defaults
log global
mode http
compression algo gzip
compression type text/html text/plain text/css application/javascript
application/octet-stream application/json
option httplog
option dontlognull
option redispatch
option tcp-smart-accept
option tcp-smart-connect
option forwardfor
timeout check 5s
timeout client 50s
timeout tunnel 6s
timeout connect 20s
timeout http-keep-alive 15s
timeout http-request 30s
timeout queue 20s
timeout server 50s
hash-balance-factor 125
balance hdr(Cookie)
hash-type consistent djb2
stats enable
stats hide-version
stats auth redacted:redacted
stats uri /redacted
default-server inter 5s fall 3 rise 1
frontend health_check
bind *:8081
timeout client 5s
monitor-uri /
frontend http
bind :::80 v4v6
maxconn 4096
redirect scheme https if !{ ssl_fc }
default_backend assets_backend
frontend https
bind :::443 v4v6 ssl crt /etc/ssl/www.ourdomain.com.crt no-sslv3
no-tls-tickets *# alpn h2,http/1.1*
maxconn 4096
-
