Hi Tim, On Thu, Apr 26, 2018 at 05:33:09PM +0200, Tim Düsterhus wrote: > Hi > > I have got a frontend in mode http that sets various headers > unconditionally: > > > http-response set-header Expect-CT > > "max-age=3600; report-uri=\"https://xxx.report-uri.com/r/d/ct/reportOnly\""; > > http-response set-header Expect-Staple > > "max-age=3600; > > report-uri=\"https://xxx.report-uri.com/r/d/staple/reportOnly\";; > > includeSubDomains" > > http-response set-header Public-Key-Pins-Report-Only > > "pin-sha256=\"Vjs8r4z+xxx+eWys=\"; pin-sha256=\"xxx/ltjyo=\"; > > pin-sha256=\"xxx/uEtLMkBgFF2Fuihg=\"; > > report-uri=\"https://xxx.report-uri.io/r/default/hpkp/reportOnly\";; > > max-age=86400" > > http-response set-header Referrer-Policy "same-origin" > > http-response set-header Strict-Transport-Security > > "max-age=31536000; includeSubDomains" > > http-response set-header X-Content-Type-Options nosniff > > http-response set-header X-Frame-Options SAMEORIGIN > > http-response set-header X-XSS-Protection "1; > > mode=block" > > This frontend talks (among others) to a backend that also sets a header > unconditionally: > > > http-response set-header Content-Security-Policy "xxx report-uri > > https://xxx.report-uri.com/r/d/csp/enforce";; > > Sometimes haproxy does not set all the headers in a response (namely: > X-Frame-Options and X-XSS-Protection are sometimes missing): > > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:24:24 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF41041A > > X-Content-Type-Options: nosniff > > X-Frame-Options: SAMEORIGIN > > X-XSS-Protection: 1; mode=block > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:24:49 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF46041D > > X-Content-Type-Options: nosniff > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:24:55 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF49041F > > X-Content-Type-Options: nosniff > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:24:57 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF4A0421 > > X-Content-Type-Options: nosniff > > [timwolla@~]curl -I https://example.com/ |grep 'X-' 17:24:59 > > % Total % Received % Xferd Average Speed Time Time Time > > Current > > Dload Upload Total Spent Left > > Speed > > 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- > > 0 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF4F0477 > > X-Content-Type-Options: nosniff > > X-Frame-Options: SAMEORIGIN > > X-XSS-Protection: 1; mode=block > > [timwolla@~]curl -I https://example.com/ |grep 'X-' 17:25:05 > > % Total % Received % Xferd Average Speed Time Time Time > > Current > > Dload Upload Total Spent Left > > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > > 0 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF530491 > > X-Content-Type-Options: nosniff > > X-Frame-Options: SAMEORIGIN > > X-XSS-Protection: 1; mode=block > > [timwolla@~]curl -I https://example.com/ |grep 'X-' 17:25:07 > > % Total % Received % Xferd Average Speed Time Time Time > > Current > > Dload Upload Total Spent Left > > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > > 0 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF5404B3 > > X-Content-Type-Options: nosniff > > X-Frame-Options: SAMEORIGIN > > X-XSS-Protection: 1; mode=block > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:25:09 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF580598 > > X-Content-Type-Options: nosniff > > X-Frame-Options: SAMEORIGIN > > X-XSS-Protection: 1; mode=block > > [timwolla@~]http -v https://example.com/ |grep 'X-' 17:25:12 > > X-UA-Compatible: IE=edge > > X-Req-ID: EXAMPLE-5AE1EF66067F > > X-Content-Type-Options: nosniff > > The logs for the first two requests: > > Apr 26 15:24:49 xxx haproxy[7565]: 2003:xxx:53728 > > [26/Apr/2018:15:24:49.681] fe_https~ bk_xxx/nginx 0/0/1/252/253 200 16912 - > > - ---- 11/8/0/1/0 0/0 {xxx|HTTPie/0.9.2} "GET / HTTP/1.1" > > EXAMPLE-5AE1EF41041A > > Apr 26 15:24:55 xxx haproxy[7565]: 2003:xxx:53730 > > [26/Apr/2018:15:24:55.034] fe_https~ bk_xxx/nginx 0/0/0/203/203 200 16911 - > > - ---- 10/7/0/1/0 0/0 {xxx|HTTPie/0.9.2} "GET / HTTP/1.1" > > EXAMPLE-5AE1EF46041D > > The hex value in the request IDs is: %Ts%rt (thus there have only been > two requests in between those two). > > I'm running haproxy 1.8.8 on Debian Stretch, installed from Debian > Backports. I've enabled http2. I don't run with threads: > > > [root@~]haproxy -vv > > HA-Proxy version 1.8.8-1~bpo9+1 2018/04/19 > > Copyright 2000-2018 Willy Tarreau <[email protected]> > > > > Build options : > > TARGET = linux2628 > > CPU = generic > > CC = gcc > > CFLAGS = -g -O2 -fdebug-prefix-map=/build/haproxy-1.8.8=. > > -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time > > -D_FORTIFY_SOURCE=2 > > OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 > > USE_LUA=1 USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_NS=1 > > > > Default settings : > > maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 > > > > Built with OpenSSL version : OpenSSL 1.1.0f 25 May 2017 > > Running on OpenSSL version : OpenSSL 1.1.0f 25 May 2017 > > OpenSSL library supports TLS extensions : yes > > OpenSSL library supports SNI : yes > > OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 > > Built with Lua version : Lua 5.3.3 > > Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT > > IP_FREEBIND > > Encrypted password support via crypt(3): yes > > Built with multi-threading support. > > Built with PCRE version : 8.39 2016-06-14 > > Running on PCRE version : 8.39 2016-06-14 > > PCRE library supports JIT : yes > > Built with zlib version : 1.2.8 > > Running on zlib version : 1.2.8 > > Compression algorithms supported : identity("identity"), > > deflate("deflate"), raw-deflate("deflate"), gzip("gzip") > > Built with network namespace support. > > > > Available polling systems : > > epoll : pref=300, test result OK > > poll : pref=200, test result OK > > select : pref=150, test result OK > > Total: 3 (3 usable), will use epoll. > > > > Available filters : > > [SPOE] spoe > > [COMP] compression > > [TRACE] trace > > Any ideas?
Not that many ideas. Could you retry by setting "tune.maxrewrite" to a larger value ? It defaults to 1024, and maybe you're already adding 1kB of response and there's no more room in the response buffer. It's just a guess, I could be completely wrong. Willy
