Sure, the biggest problem is to delete header by matching prefix:
load_blacklist = function(service)
local prefix = '/etc/haproxy/configs/maps/header_blacklist'
local blacklist = {}
blacklist.req = {}
blacklist.res = {}
blacklist.req.str = Map.new(string.format('%s_%s_req.map', prefix,
service), Map._str)
blacklist.req.beg = Map.new(string.format('%s_%s_req_beg.map', prefix,
service), Map._beg)
return blacklist
end
blacklist = {}
blacklist.testsite = load_blacklist('testsite')
is_denied = function(bl, name)
return bl ~= nil and (bl.str:lookup(name) ~= nil or bl.beg:lookup(name)
~= nil)
end
req_header_filter = function(txn, service)
local req_headers = txn.http:req_get_headers()
for name, _ in pairs(req_headers) do
if is_denied(blacklist[service].req, name) then
txn.http:req_del_header(name)
end
end
end
core.register_action('req_header_filter', { 'http-req' },
req_header_filter, 1)
śr., 18 lis 2020 o 12:46 Julien Pivotto <roidelapl...@inuits.eu> napisał(a):
> On 18 Nov 12:33, Maciej Zdeb wrote:
> > Hi again,
> >
> > So "# some headers manipulation, nothing different then on other
> clusters"
> > was the important factor in config. Under this comment I've hidden from
> you
> > one of our LUA scripts that is doing header manipulation like deleting
> all
> > headers from request when its name begins with "abc*". We're doing it on
> > all HAProxy servers, but only here it has such a big impact on the CPU,
> > because of huge RPS.
> >
> > If I understand correctly:
> > with nbproc = 20, lua interpreter worked on every process
> > with nbproc=1, nbthread=20, lua interpreter works on single
> process/thread
> >
> > I suspect that running lua on multiple threads is not a trivial task...
>
> If you can share your lua script maybe we can see if this is doable
> more natively in haproxy
>
> >
> >
> >
> >
> > wt., 17 lis 2020 o 15:50 Maciej Zdeb <mac...@zdeb.pl> napisał(a):
> >
> > > Hi,
> > >
> > > We're in a process of migration from HAProxy[2.2.5] working on multiple
> > > processes to multiple threads. Additional motivation came from the
> > > announcement that the "nbproc" directive was marked as deprecated and
> will
> > > be killed in 2.5.
> > >
> > > Mostly the migration went smoothly but on one of our clusters the CPU
> > > usage went so high that we were forced to rollback to nbproc. There is
> > > nothing unusual in the config, but the traffic on this particular
> cluster
> > > is quite unusual.
> > >
> > > With nbproc set to 20 CPU idle drops at most to 70%, with nbthread = 20
> > > after a couple of minutes at idle 50% it drops to 0%. HAProxy
> > > processes/threads are working on dedicated/isolated CPU cores.
> > >
> > > [image: image.png]
> > >
> > > I mentioned that traffic is quite unusual, because most of it are http
> > > requests with some payload in headers and very very small responses
> (like
> > > 200 OK). On multi-proc setup HAProxy handles about 20 to 30k of
> connections
> > > (on frontend and backend) and about 10-20k of http requests. Incoming
> > > traffic is just about 100-200Mbit/s and outgoing 40-100Mbit/s from
> frontend
> > > perspective.
> > >
> > > Did someone experience similar behavior of HAProxy? I'll try to collect
> > > more data and generate similar traffic with sample config to show a
> > > difference in performance between nbproc and nbthread.
> > >
> > > I'll greatly appreciate any hints on what I should focus. :)
> > >
> > > Current config is close to:
> > > frontend front
> > > mode http
> > > option http-keep-alive
> > > http-request add-header X-Forwarded-For %[src]
> > >
> > > # some headers manipulation, nothing different then on other
> clusters
> > >
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 1
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 2
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 3
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 4
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 5
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 6
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 7
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 8
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process 9
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 10
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 11
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 12
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 13
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 14
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 15
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 16
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 17
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 18
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 19
> > > bind x.x.x.x:443 ssl crt /etc/cert/a.pem.pem alpn h2,http/1.1
> process
> > > 20
> > > default_backend back
> > >
> > > backend back
> > > option http-keep-alive
> > > mode http
> > > http-reuse always
> > > option httpchk GET /health HTTP/1.0\r\nHost:\ example.com
> > > http-check expect string OK
> > >
> > > server slot_0_checker 10.x.x.x:31180 check weight 54
> > > server slot_1_checker 10.x.x.x:31146 check weight 33
> > > server slot_2_checker 10.x.x.x:31313 check weight 55
> > > server slot_3_checker 10.x.x.x:31281 check weight 33 disabled
> > > server slot_4_checker 10.x.x.x:31717 check weight 55
> > > server slot_5_checker 10.x.x.x:31031 check weight 76
> > > server slot_6_checker 10.x.x.x:31124 check weight 50
> > > server slot_7_checker 10.x.x.x:31353 check weight 48
> > > server slot_8_checker 10.x.x.x:31839 check weight 33
> > > server slot_9_checker 10.x.x.x:31854 check weight 44
> > > server slot_10_checker 10.x.x.x:31794 check weight 60 disabled
> > > server slot_11_checker 10.x.x.x:31561 check weight 56
> > > server slot_12_checker 10.x.x.x:31814 check weight 57
> > > server slot_13_checker 10.x.x.x:31535 check weight 44 disabled
> > > server slot_14_checker 10.x.x.x:31829 check weight 43 disabled
> > > server slot_15_checker 10.x.x.x:31655 check weight 40 disabled
> > >
>
>
>
> --
> (o- Julien Pivotto
> //\ Open-Source Consultant
> V_/_ Inuits - https://www.inuits.eu
>
