Re: [PATCH] MINOR: sample: Add a regmatch converter

2021-04-13 Thread Willy Tarreau
On Tue, Apr 13, 2021 at 10:52:15PM -0600, Thayne McCombs wrote: > > On 4/13/21 3:44 AM, Willy Tarreau wrote: > > I'm failing to see how it differs from regsub() which already does the > > same with the reference (you'd write \1 instead of 1) and also allows to > > compose something out of multiple

Re: [PATCH] MINOR: sample: Add a regmatch converter

2021-04-13 Thread Thayne McCombs
On 4/13/21 3:44 AM, Willy Tarreau wrote: I'm failing to see how it differs from regsub() which already does the same with the reference (you'd write \1 instead of 1) and also allows to compose something out of multiple matches. Am I missing something, or a case where regsub() is really not conv

Re: [PATCH] MINOR: sample: add json_string

2021-04-13 Thread Willy Tarreau
On Wed, Apr 14, 2021 at 03:02:20AM +0200, Aleksandar Lazic wrote: > > But then, could it make sense to also support "strict integers": values > > that can accurately be represented as integers and which are within the > > JSON valid range for integers (-2^52 to 2^52 with no decimal part) ? > > This

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread Willy Tarreau
On Wed, Apr 14, 2021 at 02:03:52AM +, Rmrf99 wrote: > Thanks Chris, Willy! this make me clear now. > Glad to see dynamically add/remove servers feature under development. If you're interested by with feature, please try the latest 2.4-dev, you already have the "add server" feature on the comma

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread Rmrf99
Thanks Chris, Willy! this make me clear now. Glad to see dynamically add/remove servers feature under development. ‐‐‐ Original Message ‐‐‐ On Wednesday, April 14, 2021 1:19 AM, Christopher Faulet wrote: > Le 13/04/2021 à 18:15, John Lauro a écrit : > > > Sounds like the biggest part o

Re: [PATCH] MINOR: sample: add json_string

2021-04-13 Thread Aleksandar Lazic
On 13.04.21 11:26, Willy Tarreau wrote: Hi Aleks, On Mon, Apr 12, 2021 at 10:09:08PM +0200, Aleksandar Lazic wrote: Hi. another patch which honer the feedback. Thank you. FWIW I agree with all the points reported by Tim. I'll add a few comments and/or suggestions below. On a general note, pl

stable-bot: Bugfixes waiting for a release 2.3 (5), 2.1 (14)

2021-04-13 Thread stable-bot
Hi, This is a friendly bot that watches fixes pending for the next haproxy-stable release! One such e-mail is sent periodically once patches are waiting in the last maintenance branch, and an ideal release date is computed based on the severity of these fixes and their merge date. Responses t

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread Christopher Faulet
Le 13/04/2021 à 18:15, John Lauro a écrit : Sounds like the biggest part of hot restarts is the cost of leaving the old process running as they have a lot of long running TCP connections, and if you do a lot of restarts the memory requirements build up.  Not much of an issue for short lived htt

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread Willy Tarreau
On Tue, Apr 13, 2021 at 12:15:33PM -0400, John Lauro wrote: > Sounds like the biggest part of hot restarts is the cost of leaving the old > process running as they have a lot of long running TCP connections, and if > you do a lot of restarts the memory requirements build up. Not much of an > issue

Re: [RFC PATCH 4/8] MINOR: uri_normalizer: Add a `sort-query` normalizer

2021-04-13 Thread Christopher Faulet
Le 13/04/2021 à 18:05, Tim Düsterhus a écrit : Christopher, On 4/13/21 4:59 PM, Christopher Faulet wrote: +/* Sorts the parameters within the given query string. Returns an ist containing + * the new path and backed by `trash` or IST_NULL if the `len` not sufficiently + * large to store the res

Re: [RFC PATCH 3/8] MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri

2021-04-13 Thread Christopher Faulet
Le 13/04/2021 à 18:03, Tim Düsterhus a écrit : Christopher, On 4/13/21 4:38 PM, Christopher Faulet wrote: At the end it remains your choice. The function is quite good. I just wonder if it could be valuable to also handle single dot-segment here in addition to double dot-segment. Thus, the norm

Re: [RFC PATCH 0/8] URI normalization / Issue #714

2021-04-13 Thread Christopher Faulet
Le 13/04/2021 à 17:45, Tim Düsterhus a écrit : Christopher, On 4/13/21 2:41 PM, Christopher Faulet wrote: Sorry for the delay. I'll comment your patches by replying inline when No delay experienced. You said that you'd try this week and it's still this week. So this is fine :-) appropriate.

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread John Lauro
Sounds like the biggest part of hot restarts is the cost of leaving the old process running as they have a lot of long running TCP connections, and if you do a lot of restarts the memory requirements build up. Not much of an issue for short lived http requests (although it would be nice if keep al

Re: [RFC PATCH 8/8] MINOR: uri_normalizer: Add a `percent-upper` normalizer

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, and this final one adds a normalizer to turn the hex digits of percent encoding into uppercase. Uppercase is the variant preferred by the URI RFC, so this is what we do. This one looks good. -- Christopher Faulet

Re: [RFC PATCH 4/8] MINOR: uri_normalizer: Add a `sort-query` normalizer

2021-04-13 Thread Tim Düsterhus
Christopher, On 4/13/21 4:59 PM, Christopher Faulet wrote: +/* Sorts the parameters within the given query string. Returns an ist containing + * the new path and backed by `trash` or IST_NULL if the `len` not sufficiently + * large to store the resulting path. + */ +struct ist uri_normalizer_q

Re: [RFC PATCH 3/8] MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri

2021-04-13 Thread Tim Düsterhus
Christopher, On 4/13/21 4:38 PM, Christopher Faulet wrote: At the end it remains your choice. The function is quite good. I just wonder if it could be valuable to also handle single dot-segment here in addition to double dot-segment. Thus, the normalizer should be renamed "dot-segments" or som

Re: [RFC PATCH 7/8] MINOR: uri_normalizer: Support returning detailed errors from uri normalization

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, this is in prepatation for the next normalizer which normalizes character case of the percent encoding. The resources I found are not clear on whether a percent that is not followed by two hex digits is valid or not. Most brows

Re: [RFC PATCH 0/8] URI normalization / Issue #714

2021-04-13 Thread Tim Düsterhus
Christopher, On 4/13/21 2:41 PM, Christopher Faulet wrote: Sorry for the delay. I'll comment your patches by replying inline when No delay experienced. You said that you'd try this week and it's still this week. So this is fine :-) appropriate. The quality of the whole series is pretty good

Re: [PATCH] JWT payloads break b64dec convertor

2021-04-13 Thread Willy Tarreau
On Tue, Apr 13, 2021 at 04:44:38PM +0200, Moemen MHEDHBI wrote: > > But then how about having just *your* functions > > without relying on the other ones ? Now that you've extended the existing > > function, you can declare it yours, remove all the configurable stuff and > > keep the simplified ver

Re: [RFC PATCH 6/8] MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, most of the patch is moving around the config parser to support ingesting the new argument. This one looks good. -- Christopher Faulet

Re: [RFC PATCH 5/8] OPTIMIZE: uri_normalizer: Optimize allocations in uri_normalizer_query_sort

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, I did not perform any measurements at all. But not reallocating for every parameter should be better :-) This one may be useless if you use the trash buffer to store the query parameters. -- Christopher Faulet

Re: [RFC PATCH 4/8] MINOR: uri_normalizer: Add a `sort-query` normalizer

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, This one comes with dynamic allocation. The next patch will add an optimization for a small number of arguments. However dynamic allocation within the main processing logic is pretty ugly, so this should be looked at further.

Re: [PATCH] JWT payloads break b64dec convertor

2021-04-13 Thread Moemen MHEDHBI
On 13/04/2021 11:39, Willy Tarreau wrote: >> You can find attached the patches 0001-bis and 0002-bis modifying the >> existing functions (introducing an url flag) to see how it looks like. >> This solution may be cleaner (no chunk allocation and we don't loop >> twice over input string) but has t

Re: [RFC PATCH 3/8] MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, I'm not very happy with the normalization logic, because am processing the URI in reverse. This requires me to directly access offsets instead of using the `ist` API. However this way I don't need to backtrack once I encounter a

Re: [RFC PATCH 2/8] MINOR: uri_normalizer: Add `http-request normalize-uri`

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, something simple for a start. This one adds the http-request action and a very simple normalizer to test whether it works. Turns out it does :-) You can see the new `ist` helpers in action already. I'm pretty happy that I was a

Re: [RFC PATCH 1/8] MINOR: uri_normalizer: Add uri_normalizer module

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, I used uri_auth.[ch] as the basis for the source file structure (comments and stuff). Thanks, nothing to say about this one :) -- Christopher Faulet

Re: [RFC PATCH 0/8] URI normalization / Issue #714

2021-04-13 Thread Christopher Faulet
Le 08/04/2021 à 20:59, Tim Duesterhus a écrit : Willy, Christopher, Not sure who of you is better suited to review this series, so I'm adding both of you :-) I'm tagging this as RFC, because it's large and quite a bit outside of my comfort zone. However the patches are as clean as possible. The

httpd ProxyPass / ProxyPassReverse to haproxy 2.3.9 ?

2021-04-13 Thread Thomas Elsgaard
Hi list Moving from httpd (reverse proxy) to haproxy 2.3.9 In httpd i have following: ServerName test.server.dk ProxyPreserveHost On ProxyPass / http://localhost:8080/aaa/bbb/ ProxyPassReverse / http://localhost:8080/aaa/bbb/ What is the correct way to do this in haproxy 2 ? Something with http

Re: About the 'Hot Restarts' of haproxy

2021-04-13 Thread Willy Tarreau
On Tue, Apr 13, 2021 at 01:31:12AM +, Rmrf99 wrote: > In this Slack engineering blog post: > https://slack.engineering/migrating-millions-of-concurrent-websockets-to-envoy/ > > they replace HAProxy with Envoy for **Hot Restart**, just curious does > HAProxy new version will have similar appro

Re: 2Mrps : kudos to the team

2021-04-13 Thread Willy Tarreau
On Sat, Apr 10, 2021 at 01:34:16PM +0200, Ionel GARDAIS wrote: > Hi team, list, > > If you haven't already read it, go read this blog article : > [ > https://www.haproxy.com/blog/haproxy-forwards-over-2-million-http-requests-per-second-on-a-single-aws-arm-instance/ > | > https://www.haproxy.c

Re: [PATCH] BUG/MINOR: sample: Fix adjusting size in field converter

2021-04-13 Thread Willy Tarreau
Hi Thayne, On Sun, Apr 11, 2021 at 11:26:59PM -0600, Thayne McCombs wrote: > Adjust the size of the sample buffer before we change the "area" > pointer. The change in size is calculated as the difference between the > original pointer and the new start pointer. But since the > `smp->data.u.str.are

Re: [PATCH] MINOR: sample: Add a regmatch converter

2021-04-13 Thread Willy Tarreau
Hi Thayne, On Tue, Apr 13, 2021 at 02:11:25AM -0600, Thayne McCombs wrote: > Add a new sample converter that finds the first regex match and returns > the substring for that match, or a capture group, if an index is > provided. > --- > doc/configuration.txt| 22 +++ > reg-test

Re: [PATCH] JWT payloads break b64dec convertor

2021-04-13 Thread Willy Tarreau
Hi Jarno, On Tue, Apr 13, 2021 at 06:19:47AM +, Jarno Huuskonen wrote: > Hello, > > On Tue, 2021-04-06 at 01:58 +0200, Moemen MHEDHBI wrote: > > Thanks Willy and Tim for your feedback. > > > > You can find attached the updated patches with fixed coding style (now > > set correctly in my edit

Re: [PATCH] JWT payloads break b64dec convertor

2021-04-13 Thread Willy Tarreau
Hi Moemen, On Tue, Apr 13, 2021 at 12:41:39AM +0200, Moemen MHEDHBI wrote: > >> in such case should we rather use dynamic allocation ? > > > > No, there are two possible approaches. One of them is to use a trash > > buffer using get_trash_chunk(). The trash buffers are "large enough" > > for anyth

Re: [PATCH] MINOR: sample: add json_string

2021-04-13 Thread Willy Tarreau
Hi Aleks, On Mon, Apr 12, 2021 at 10:09:08PM +0200, Aleksandar Lazic wrote: > Hi. > > another patch which honer the feedback. Thank you. FWIW I agree with all the points reported by Tim. I'll add a few comments and/or suggestions below. On a general note, please be careful about your indenting,

Bid Writing, Fundraising and Volunteering Workshops

2021-04-13 Thread NFP Workshops
NFP WORKSHOPS Affordable Training Courses Bid Writing: The Basics Do you know the most common reasons for rejection? Are you gathering the right evidence? Are you making the right arguments? Are you using the right terminology? Are your numbers right? Are you learning from rejections?

[PATCH] MINOR: sample: Add a regmatch converter

2021-04-13 Thread Thayne McCombs
Add a new sample converter that finds the first regex match and returns the substring for that match, or a capture group, if an index is provided. --- doc/configuration.txt| 22 +++ reg-tests/converter/regmatch.vtc | 39 +++ src/sample.c | 66