Re: Few problems seen in haproxy? (threads, connections).
Hi Krishna, On Thu, Oct 11, 2018 at 12:04:31PM +0530, Krishna Kumar (Engineering) wrote: > I must say the improvements are pretty impressive! > > Earlier number reported with 24 processes: 519K > Earlier number reported with 24 threads: 79K > New RPS with system irq tuning, today's git, >configuration changes, 24 threads:353K > Old code with same tuning gave: 290K OK that's much better but I'm still horrified by the time taken in the load balancing algorithm. I thought it could be fwlc_reposition(), which contains an eb32_delete()+eb32_insert(), so I decided to replace this with a new eb32_move() which moves the node within the tree, and it didn't change anything here. Also I figured that I cannot manage to reach that high time spent in this lock (300ms here, 58s for you). There is one possible difference that might explain it, do you have a maxconn setting on your servers ? If so, is it possible that it's reached ? You can take a look at your stats page and see if the "Queue/Max" entry for any backend is non-null. Indeed, I'm seeing that once a server is saturated, we skip it for the next one. This part can be expensive. Ideally we should remove such servers from the tree until they're unblocked, but there is one special case making this difficult, which is the dynamic limitation (minconn+maxconn+fullconn). However I think we could improve this so that only this use case would be affected and not the other ones. I'm also seeing that this lock could be replaced by an RW lock. But before taking a deeper look, I'm interested in verifying that it's indeed the situation you're facing. Thanks, Willy
Re: Setting response headers conditionally
Yep, that's what I ended up doing following the advice from serverfault. Now it looks logical and I must admit I did not check the log for warnings (I'm running it with systemd but I expected any warnings still to be shown when I emit `systemctl reload`, I was wrong) Thank you! On Mon, 15 Oct 2018 at 11:26, Cyril Bonté wrote: > Hi, > > Le 14/10/2018 à 22:39, Ivan Kurnosov a écrit : > > I have the following config, it's under the `frontend` section for tls > > connection and haproxy terminates https connections: > > > > acl domain-acl-host hdr(host) -i domain.tld > > rspadd X-Foo:\ bar if domain-acl-host > > rspadd X-Baz:\ baz > > http-response set-header X-Bar bar if domain-acl-host > > use_backend backend_name if domain-acl-host > > > > The `use_backend` directive works conditionally as expected (there are > > multiple different domain names served, and they are chosen correctly) > > > > But headers are not added/set to the response conditionally. > > > > I expect 3 extra headers to be added there: `X-Foo`, `X-Baz`, and > > `X-Bar`, but only `X-Baz` is added: > > > > < HTTP/1.1 302 Found > > < Server: nginx > > < Content-Type: text/html; charset=UTF-8 > > < Transfer-Encoding: chunked > > < Cache-Control: max-age=0, must-revalidate, private > > < Date: Sun, 14 Oct 2018 20:25:59 GMT > > < Location: https://domain.tld/somewhere/else > > < X-Baz: baz > > > > I'm sure I'm missing something trivial, but reading documentation or > > google did not help. > > Well, did you have a look at the warnings emitted by haproxy on startup > saying your acl will never match for "rspadd X-Foo" and "http-response > set-header" ? You can't manipulate response headers based on request > headers acl (they're not in the memory buffer anymore). > > You can capture the request header in a variable and modify your acl to > use this variable instead. > > Example: > http-request set-var(txn.host) hdr(host) > acl domain-acl-host var(txn.host) -i domain.tld > > See the documentation for details: > - > > http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-http-request > - http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#var > > > PS: it's `haproxy 1.8.8` > > > > PPS: I originally asked it at https://serverfault.com/q/935492/45086 as > well > > > > -- > > With best regards, Ivan Kurnosov > > > -- > Cyril Bonté > -- With best regards, Ivan Kurnosov
Re: Setting response headers conditionally
Hi, Le 14/10/2018 à 22:39, Ivan Kurnosov a écrit : I have the following config, it's under the `frontend` section for tls connection and haproxy terminates https connections: acl domain-acl-host hdr(host) -i domain.tld rspadd X-Foo:\ bar if domain-acl-host rspadd X-Baz:\ baz http-response set-header X-Bar bar if domain-acl-host use_backend backend_name if domain-acl-host The `use_backend` directive works conditionally as expected (there are multiple different domain names served, and they are chosen correctly) But headers are not added/set to the response conditionally. I expect 3 extra headers to be added there: `X-Foo`, `X-Baz`, and `X-Bar`, but only `X-Baz` is added: < HTTP/1.1 302 Found < Server: nginx < Content-Type: text/html; charset=UTF-8 < Transfer-Encoding: chunked < Cache-Control: max-age=0, must-revalidate, private < Date: Sun, 14 Oct 2018 20:25:59 GMT < Location: https://domain.tld/somewhere/else < X-Baz: baz I'm sure I'm missing something trivial, but reading documentation or google did not help. Well, did you have a look at the warnings emitted by haproxy on startup saying your acl will never match for "rspadd X-Foo" and "http-response set-header" ? You can't manipulate response headers based on request headers acl (they're not in the memory buffer anymore). You can capture the request header in a variable and modify your acl to use this variable instead. Example: http-request set-var(txn.host) hdr(host) acl domain-acl-host var(txn.host) -i domain.tld See the documentation for details: - http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-http-request - http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#var PS: it's `haproxy 1.8.8` PPS: I originally asked it at https://serverfault.com/q/935492/45086 as well -- With best regards, Ivan Kurnosov -- Cyril Bonté
