Re: parsing reload cmd output from master cli
> Em 5 de fev. de 2021, à(s) 04:33, Willy Tarreau escreveu:
>
> On Wed, Feb 03, 2021 at 06:32:33PM -0300, Joao Morais wrote:
>>
>> Hello William, here[1] is some context. I implemented a parsing of the
>> reload command sent to the master cli, leave it running for a while and got
>> an index out of bounds (in my code) this week.
>>
>> I'm using this lay out to parse the reload output:
>>
>> // 1 3 4 6
>> 8 8
>> //
>> 0...|...6...|...2...|...8...|...4...|...0...|...8
>> //
>> // #
>>
>> // 1 master 0 2
>> 0d00h01m28s 2.2.3-0e58a34
>> // # workers
>> // 3 worker 1 0
>> 0d00h00m00s 2.2.3-0e58a34
>> // # old workers
>> // 2 worker [was: 1]1
>> 0d00h00m28s 2.2.3-0e58a34
>> // # programs
>> //
>>
>> Apparently I found a line that: starting char isn't '#', have 32 chars or
>> more, have less than 48 chars. Is that even possible?
>
> I had a quick look at the code producing this output and it writes
> full lines at once, but the second one is a string which is mostly a
> constant ("worker", "master"), or taken from a structure for old
> programs. So I'm seeing two possibilities:
> - either your parser received an incomplete line due to network buffering
>and decided to parse it before getting the trailing "\n"; this is a
>pretty common issue in line-oriented parsers ported to consume data
>from other processes ;
>
> - or the process memory was corrupted so that an old program's "child->id"
>contained some crap and a "\n", which would have resulted in printing
>the PID, this crappy string, and the rest of the line on a second separate
>line.
>
> The last point seems pretty possible given that your outdated version is
> affected by 123 known bugs, 2 of which could cause memory corruption.
So we probably ended with the first one - currently using 2.2.8, this output
came from my drafts when I was designing the remote restart. =)
>> How would you expand the lay out above, so I can improve my parser and my
>> tests? Thanks!
>
> Please read the chunk_appendf() statements in cli_io_handler_show_proc(),
> they're pretty explicit and you'll get the exact output format for various
> types of dumps. But that's basically what's documented above, it's just
> that it may help you to figure some constants there (or that some fields
> are unsigned for example).
Thank you so much for the insight! This will help us to prepare the code for
such corner cases.
~jm
Re: A faster way to lookup 5k hosts+paths
> Em 5 de fev. de 2021, à(s) 04:17, Willy Tarreau escreveu:
>
> Hi Joao,
>
> On Tue, Feb 02, 2021 at 09:03:06PM -0300, Joao Morais wrote:
>>
>> Hello list, I've about 5000 hostnames + path that should be mapped to 3000
>> backends or so. I'm using map converters and the lay out is something like
>> this:
>>
>> /dir/file.map
>>d1.tld/path/sub back1
>>d1.tld/path back2
>>d2.tld/path/sub/other back3
>>d2.tld/path/sub back4
>>d2.tld/path back5
>>d3.tld/path/sub back6
>>d3.tld/path back7
>>
>> And frontend looks like this:
>>
>>http-request set-var(req.backend) base,map_beg(/dir/file.map)
>>use_backend %[var(req.backend)] if { var(req.backend) -m found }
>>
>> The problem here is that the map has about 5k lines and the search isn't
>> indexed. I could eg split this map into smaller ones and create an index of
>> maps stored in another map, but cannot use the result because the file name
>> param in the converter cannot be dynamic. Any other idea to help haproxy find
>> the backend in a faster way?
>
> Not sure why you're saying it's not indexed. It uses map_beg so it will
> use a prefix tree for the lookup, it should be pretty fast.
Hi Willy, thanks for clarifying this. My question and my concern were based on
the doc which says "Other keys are stored in lists, so the first matching
occurrence will be used.”. My understanding were that it wasn’t indexed, but
instead comparing one item at a time from the top, specially when map_reg
belongs to other keys as well. So it’s good to know that map_beg (and also
map_dir?) already does a pretty good job.
> By the way, you could simplify your rules this way:
>
> use_backend base,map_beg(/dir/file.map)
>
> Indeed, a use_backend rule which doesn't find the backend will do nothing
> and continue to next rules.
Ah sure, I’m aware of this construction - except the fact that I can safely
remove the condition, tks - but I’ve actually some other configs between the
converter and the use_backend. Should’ve used a “…” between both in the
example. =)
~jm
[ANNOUNCE] haproxy-2.4-dev7
Hi, HAProxy 2.4-dev7 was released on 2021/02/05. It added 153 new commits after version 2.4-dev6. This version is quite large for two reasons, first one being that I wanted to issue it last week but got stuck on an issue that I didn't imagine would take my week-end and week, and second one because some changes were upheld by other ones that had to be reworked several times. But eventually all of this got sorted out and could be merged. No less than 29 bugs were addressed this time, showing that some significant activity remains on stabilization, and comforts me in the decision of closing the merge window at the end of next week so that all those which remain have the time to be sorted out for the release. The main updates this time are in the following areas: - TCP to HTTP upgrades: these ones were reportedly broken on the last few -dev versions (since dev3 or dev4, I don't remember) and have now been addressed. - HTX: end of the migration to remove the special EOM (end of message) block that was causing some difficulties, particularly when CONNECT was used in H2, but was also maintaining a certain level of complexity in various parts of the muxes. There is a non-null risk of corner case issues here, though it has run through extensive testing and even been deployed in production and it looks like everything's fine by now. If you observe anything strange such as hanging requests or responses, or breakage with tunnels (you shouldn't), please report them. - Websocket: the WebSocket translation from H2 to H1 is now fully supported (RFC8441) as well as H1 to H2. The Extended CONNECT support is also advertised so that compatible clients will not need to set up a second H1 connection to use websocket anymore. - SSL: server-side certificates can now be updated at run time from the CLI. - Some significant lifting was done to the Prometheus exporter, including new fields, better descriptions and some filtering. I've seen quite a bunch pass in front of me but do not well understand what it does, all that interests me is that some users are happy with these changes so I guess they were long awaited :-) - health-check/agent-check: some old code that used to be there to force the address/port from the server's address while parsing the config had some long-tail effects on various parts of the code, making it a nightmare to update them dynamically or even from state files. And of course, this was just an antique relic not needed anymore some cleaning was done there (mostly unused code removal), and will help further consolidation in this area in the near future. - HTTP: make sure to reject non-compliant status-101 upgrades. We used not to check for the upgrade header for example but better stay safe and closer to the standard here as we're playing with tunnels. The 101 status code is now also rejected internally and externally on any H2 response. - The place L7 retries are dealt with changed slightly from the connection error handler to an analyser. This must not cause any difference in practice except make the code more maintainable and robust. Similarly if you're using them and notice a change of behavior, please raise your hand. - now we make sure never to emit any payload for bodyless responses (204, 304, HEAD). In the past such contents could have been produced from inside haproxy (Lua for example, maybe http-request return), now the contents will be discarded if ever present. - debug: "show fd" now reports a bit more information such as the number of calls to the registered tasks (useful to detect never-ending loops), local and report ports (useful to match against netstat), a few more info about the internal handlers and protocol, and an indicator of suspicious state (e.g. too many calls, or inconsistent state). This will help bug reporters to quickly isolate a few candidates when something looks odd. - debug/monitoring: "show profiling" will now report the CPU calls, %usage, and latency of each running task since the last time profiling was turned on. "show tasks" will enumerate the currently running tasks and their counts. Do not abuse by bots, this is expensive. Finally, crashes provoked by BUG_ON() statements in DEBUG_STRICT mode will now dump a call trace in addition to the error message, hopefully helping developers spot certains issues easier. - HTTP: chunk size used to be limited to 2 GB because we used to rely on 31-bit integers at a time where nobody else would support larger values either. This was extended to 4 PB to satisfy some extremely rare but existing use cases. And a usual, new regtests (123 running on my machine before any push here), significant cleanups all over the code, doc and CI improvements. I certainly have not done justice to those having worked hard on certain pa
[PATCH] DOC/MINOR: ROADMAP: adopt the Roadmap to the current state
Hi.
attached a patch for the Roadmap.
There is also the bandwidth limitation as open entry due
to this fact I assume it's not easy to handle bandwith
limitation within haproxy.
Regards
Aleks
>From 8a77687ca480feb286fd394d533570b079d4be27 Mon Sep 17 00:00:00 2001
From: Aleksandar Lazic
Date: Fri, 5 Feb 2021 11:09:35 +0100
Subject: [PATCH] DOC/MINOR: ROADMAP: adopt the Roadmap to the current state
---
ROADMAP | 15 +--
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/ROADMAP b/ROADMAP
index a797b84eb..67cd7edaa 100644
--- a/ROADMAP
+++ b/ROADMAP
@@ -3,9 +3,10 @@ Medium-long term wish list - updated 2019/06/15
Legend: '+' = done, '-' = todo, '*' = done except doc
2.1 or later :
- - return-html code xxx [ file "xxx" | text "xxx" ] if
-
- - return-raw [ file "xxx" | text "xxx" ] if
+ + return-html code xxx [ file "xxx" | text "xxx" ] if
+ + return-raw [ file "xxx" | text "xxx" ] if
+Both are available since 2.2
+https://www.haproxy.com/blog/announcing-haproxy-2-2/#native-response-generator
- have multi-criteria analysers which subscribe to req flags, rsp flags, and
stream interface changes. This would result in a single analyser to wait
@@ -34,8 +35,10 @@ Legend: '+' = done, '-' = todo, '*' = done except doc
- add support for event-triggered epoll, and maybe change all events handling
to pass through an event cache to handle temporarily disabled events.
- - evaluate the changes required for multi-process+shared mem or multi-thread
+ + evaluate the changes required for multi-process+shared mem or multi-thread
+thread-local+fast locking.
+HAProxy uses Threads since 1.8. In 2.0 is the theading enhanced.
+https://www.haproxy.com/blog/haproxy-2-0-and-beyond/#cloud-native-threading-logging
Old, maybe obsolete points :
- clarify licence by adding a 'MODULE_LICENCE("GPL")' or something equivalent.
@@ -61,7 +64,7 @@ Unsorted :
- random cookie generator
- - fastcgi to servers
+ + fastcgi to servers. Available since 2.1 https://www.haproxy.com/blog/haproxy-2-1/#fastcgi
- hot config reload
@@ -73,4 +76,4 @@ Unsorted :
- dynamic weights based on check response headers and traffic response time
- - various kernel-level acceleration (multi-accept, ssplice, epoll2...)
+ + various kernel-level acceleration (multi-accept, ssplice, epoll2...)
--
2.25.1
Re: Makefile, environment variables and REGTESTS_TYPES
On Fri, Feb 05, 2021 at 11:06:01AM +0100, William Lallemand wrote:
> On Fri, Feb 05, 2021 at 10:31:53AM +0100, William Lallemand wrote:
>
> > Ok, I'm going to do the change in the help command then.
> >
>
> In fact I just take a look again at this, and I think we've done the
> patch the wrong way.
>
> In 'run-regtests.sh' there is already a default setting:
>
> REGTESTS_TYPES="${REGTESTS_TYPES:-any}"
>
> So I suggest we change this one and remove the one from the Makefile, so we
> could have the previous behavior.
>
> The Makefile is not suppose to change these variables, even
> HAPROXY_PROGRAM and VTEST_PROGRAM are not set in the Makefile.
>
> It makes more sense this way in my opinion.
I think you're right, indeed.
Willy
Re: Makefile, environment variables and REGTESTS_TYPES
On Fri, Feb 05, 2021 at 10:31:53AM +0100, William Lallemand wrote:
> Ok, I'm going to do the change in the help command then.
>
In fact I just take a look again at this, and I think we've done the
patch the wrong way.
In 'run-regtests.sh' there is already a default setting:
REGTESTS_TYPES="${REGTESTS_TYPES:-any}"
So I suggest we change this one and remove the one from the Makefile, so we
could have the previous behavior.
The Makefile is not suppose to change these variables, even
HAPROXY_PROGRAM and VTEST_PROGRAM are not set in the Makefile.
It makes more sense this way in my opinion.
--
William Lallemand
Re: Makefile, environment variables and REGTESTS_TYPES
On Fri, Feb 05, 2021 at 08:41:47AM +0100, Willy Tarreau wrote:
> Hi William,
>
> On Fri, Jan 29, 2021 at 02:44:27PM +0100, William Lallemand wrote:
> > Hello List,
> >
> > According to `make reg-tests-help` the REGTESTS_TYPES parameter must be
> > configured as an environment variable and not a make argument.
> >
> > However since patch 3bad3d5 ("BUILD: Makefile: exclude broken tests by
> > default"), it does not work anymore with an environment variable.
> >
> > Looking at the several CI files we have, it is used as a make
> > argument everywhere.
>
> I must confess I didn't even know it was expected to be used as an
> environment variable and have always used it as a make argument, given
> that these *are* exported as argument variables anyway.
I was surprised it was documented this way too, because we never used
the Makefile with environment variable before.
> Here's what I'm using for the make-reg-test-all script for example:
>
> make reg-tests VTEST_PROGRAM=/usr/local/bin/vtest
> REGTESTS_TYPES=default,bug,devel,slow -- --j 8 "$@" || exit 1
>
> > I'm going to update the `make reg-tests-help` command with the correct
> > syntax if there isn't any complain.
>
> OK.
>
> > We could fix the issue by using "?=" when setting the default value, but
> > it would make it the only variable that use "?=" in the Makefile and I'm
> > not sure we want to proceed this way.
>
> No please avoid this, I find the mixing of arguments and environment
> really confusing to use. The only valid use of "?=" is when you want
> to support pre-setting a variable that will be completed later,
> but this usually remains as it doesn't allow you to remove options,
> and in general it's really bad as you get different behaviors for
>
>$ VAR=xxx make
>
> and:
>
>$ make VAR=xxx
>
> I think the reg-test users are a small enough group that any required
> change can be quickly adopted if needed anyway, without having to take
> too much care about backwards compatibility, as long as it remains
> convenient to use.
>
Ok, I'm going to do the change in the help command then.
--
William Lallemand
Re: HAProxy ratelimit based on bandwidth
On 26.01.21 20:27, Aleksandar Lazic wrote: Hi. On 26.01.21 05:54, Sangameshwar Babu wrote: > Hello Team, > > I would like to get some suggestions on setting up ratelimit on HAProxy 1.8 version, > my current setup is as below. > > 1000+ rsyslog clients(TCP) -> HAProxy (TCP mode) -> backend centralized rsyslog server. > > I have the below stick table and acl's through which I am able to mark a source as > "abuse" if the client crosses the limit post which all new connections from the > same client are rejected until stick table timer expires. > > haproxy.cfg > - > stick-table type ip size 200k expire 2m store gpc0,conn_rate(2s),bytes_in_rate(1s),bytes_in_cnt > > acl data_rate_abuse sc1_bytes_in_rate ge 100 > acl data_size_abuse sc1_kbytes_in ge 1 > > tcp-request connection silent-drop if data_rate_abuse > tcp-request connection reject if data_size_abuse > > However I would like to configure in such a way that once a client sends about > "x bytes" of data the connection should be closed instantly instead of marking it > abuse and simultaneous connections being rejected. +1 I have a similar issue and hope that we get suggestions to get a answer here. > Kindly let me know if the above can be configured with HAProxy version 1.8. I will need it for 2.2+ Looks like this feature is not yet available when I look into the roadmap. There is a "bandwidth limits" entry. http://git.haproxy.org/?p=haproxy.git;a=blob;f=ROADMAP;h=a797b84eb95298807cefa03edaa69583d8007c5b;hb=HEAD#l22 I have seen there also some points which are already implemented therefore I will send a patch to update the roadmap. > BR > Sangam Regards Aleks
