Re: Clients occasionally see truncated responses

2021-04-08 Thread Nathan Konopinski
One other behavior I've observed, nginx has an ssl_buffer_size ( http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_buffer_size) option that defaults to 16kb. As I decrease its value, I start seeing the same body size != content length errors from clients. I tried experimenting with tune.id

Re: [PATCH] MINOR: sample: add json_string

2021-04-08 Thread Aleksandar Lazic
Hi. Sorry I have now seen the copy paste error. please use this patch Regards Alex On 08.04.21 21:55, Aleksandar Lazic wrote: Hi. Attached the patch to add the json_string sample. In combination with the JWT patch is a pre-validation of a bearer token part possible. I have something like t

[PATCH] MINOR: sample: add json_string

2021-04-08 Thread Aleksandar Lazic
Hi. Attached the patch to add the json_string sample. In combination with the JWT patch is a pre-validation of a bearer token part possible. I have something like this in mind. http-request set-var(sess.json) req.hdr(Authorization),word(2,.),ub64dec,json_string('$.iss') http-request deny unl

[RFC PATCH 8/8] MINOR: uri_normalizer: Add a `percent-upper` normalizer

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, and this final one adds a normalizer to turn the hex digits of percent encoding into uppercase. Uppercase is the variant preferred by the URI RFC, so this is what we do. Best regards Tim Düsterhus Apply with `git am --scissors` to automatically cut the commit message. -- >8

[RFC PATCH 7/8] MINOR: uri_normalizer: Support returning detailed errors from uri normalization

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, this is in prepatation for the next normalizer which normalizes character case of the percent encoding. The resources I found are not clear on whether a percent that is not followed by two hex digits is valid or not. Most browsers and servers appear to support it, so I opted t

[RFC PATCH 2/8] MINOR: uri_normalizer: Add `http-request normalize-uri`

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, something simple for a start. This one adds the http-request action and a very simple normalizer to test whether it works. Turns out it does :-) You can see the new `ist` helpers in action already. I'm pretty happy that I was able to implement this completely with the new `ist

[RFC PATCH 5/8] OPTIMIZE: uri_normalizer: Optimize allocations in uri_normalizer_query_sort

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, I did not perform any measurements at all. But not reallocating for every parameter should be better :-) Best regards Tim Düsterhus Apply with `git am --scissors` to automatically cut the commit message. -- >8 -- Do not reallocate for each parameter. --- src/uri_normalizer.

[RFC PATCH 3/8] MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, I'm not very happy with the normalization logic, because am processing the URI in reverse. This requires me to directly access offsets instead of using the `ist` API. However this way I don't need to backtrack once I encounter a `../` which I consider to be a win. Best regards

[RFC PATCH 1/8] MINOR: uri_normalizer: Add uri_normalizer module

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, I used uri_auth.[ch] as the basis for the source file structure (comments and stuff). Best regards Tim Düsterhus Apply with `git am --scissors` to automatically cut the commit message. -- >8 -- This is in preparation for future patches. --- Makefile

[RFC PATCH 6/8] MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, most of the patch is moving around the config parser to support ingesting the new argument. Best regards Tim Düsterhus Apply with `git am --scissors` to automatically cut the commit message. -- >8 -- This adds an option to supress `../` at the start of the resulting path. --

[RFC PATCH 4/8] MINOR: uri_normalizer: Add a `sort-query` normalizer

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, This one comes with dynamic allocation. The next patch will add an optimization for a small number of arguments. However dynamic allocation within the main processing logic is pretty ugly, so this should be looked at further. Best regards Tim Düsterhus Apply with `git am --sc

[RFC PATCH 0/8] URI normalization / Issue #714

2021-04-08 Thread Tim Duesterhus
Willy, Christopher, Not sure who of you is better suited to review this series, so I'm adding both of you :-) I'm tagging this as RFC, because it's large and quite a bit outside of my comfort zone. However the patches are as clean as possible. They include full documentation and each normalizer c

Re: [PATCH] CLEANUP: Remove useless malloc() casts

2021-04-08 Thread Willy Tarreau
On Thu, Apr 08, 2021 at 08:05:23PM +0200, Tim Duesterhus wrote: > Willy, > > apparently I still had this one lying around. Found it while cleaning up local > branches :-) Ah yes that's indeed cleaner this way. Applied, thank you! Willy

[PATCH] CLEANUP: Remove useless malloc() casts

2021-04-08 Thread Tim Duesterhus
Willy, apparently I still had this one lying around. Found it while cleaning up local branches :-) Best regards Tim Düsterhus Apply with `git am --scissors` to automatically cut the commit message. -- >8 -- This is not C++. --- include/haproxy/chunk.h| 2 +- include/haproxy/listener.h | 2

Re: [PATCH 1/2] CLEANUP: htx: Make http_get_stline take a `const struct`

2021-04-08 Thread Willy Tarreau
On Thu, Apr 08, 2021 at 07:40:03PM +0200, Tim Düsterhus wrote: > Willy, > > On 4/3/21 8:39 PM, Tim Duesterhus wrote: > > Nothing is being modified there, so this can be `const`. > > As you've taken my other series and not this one, consider this to be a push > :-) Oh, sorry for this, it got drow

Re: [PATCH 1/2] CLEANUP: htx: Make http_get_stline take a `const struct`

2021-04-08 Thread Tim Düsterhus
Willy, On 4/3/21 8:39 PM, Tim Duesterhus wrote: Nothing is being modified there, so this can be `const`. As you've taken my other series and not this one, consider this to be a push :-) Best regards Tim Düsterhus

Re: [PATCH 0/3] Additional ist functions

2021-04-08 Thread Willy Tarreau
Hi Tim, On Thu, Apr 08, 2021 at 07:32:10PM +0200, Tim Düsterhus wrote: > > No emergency but since I guess you're using them in your code, it would > > be nice that your first caller uses either a secured or explicit version. > > I'll opt for the explicit version, because for a secured version I'd

Re: help for implementation of first fetch function "sample_fetch_json_string"

2021-04-08 Thread Aleksandar Lazic
Tim, you are great ;-) On 08.04.21 18:14, Tim Düsterhus wrote: Aleks, On 4/8/21 5:07 PM, Aleksandar Lazic wrote: http-request set-var(sess.json) %[req.hdr(Authorization),b64dec,json_string("\$.kubernetes\\.io/serviceaccount/namespace")] http-request set-var() does not expect the %[] syntax,

Re: [PATCH 0/3] Additional ist functions

2021-04-08 Thread Tim Düsterhus
Willy, On 4/7/21 7:56 PM, Willy Tarreau wrote: Overall it all looks good so I've merged it. I'd just have one small request regarding istappend(), it's the first really unsafe function we have in this collection that could be used inside a loop and cause buffer overflows, especially since ist st

Re: help for implementation of first fetch function "sample_fetch_json_string"

2021-04-08 Thread Tim Düsterhus
Aleks, On 4/8/21 5:07 PM, Aleksandar Lazic wrote: http-request set-var(sess.json) %[req.hdr(Authorization),b64dec,json_string("\$.kubernetes\\.io/serviceaccount/namespace")] http-request set-var() does not expect the %[] syntax, because it always takes a sample. Even the following returns th

help for implementation of first fetch function "sample_fetch_json_string"

2021-04-08 Thread Aleksandar Lazic
Hi. I try to implement "sample_fetch_json_string" based on https://github.com/cesanta/mjson. Because I haven't implemented a fetch function until now it would be nice when somebody helps me and point me into the right direction. Maybe I have overseen a documentation in the doc directory. Let

Re: [HAP 2.4-dev] Quotes in str fetch sample

2021-04-08 Thread Aleksandar Lazic
Hi. Never mind. I have send the header in base64 and decode it. ```shell curl -vH 'Authorization: '$(< /datadisk/git-repos/haproxy/dev/json/test-data.json.base64 ) http://127.0.0.1:8080 * Trying 127.0.0.1:8080... * TCP_NODELAY set * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > GET / H

haproxy 2.4 opentracing using x-b3-traceid header

2021-04-08 Thread Andrea Bonini
Hi everyone, i have a question about opentracing addon. Is there a way for using headers info about a trace created from an outside service? what i want is using x-b3-traceid header from an incoming request as traceid and use spanid as parentid Thanks -- Bonini Andrea

Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-08 Thread Илья Шипицин
чт, 8 апр. 2021 г. в 14:25, Willy Tarreau : > On Wed, Apr 07, 2021 at 05:26:24PM +0500, ??? wrote: > > we run "all features anebled" gcc and clang builds, for example > > BUG/MINOR: tools: fix parsing "us" unit for timers · > > haproxy/haproxy@a683805 (github.com) > > < > https://github.c

Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-08 Thread Willy Tarreau
On Wed, Apr 07, 2021 at 05:26:24PM +0500, ??? wrote: > we run "all features anebled" gcc and clang builds, for example > BUG/MINOR: tools: fix parsing "us" unit for timers · > haproxy/haproxy@a683805 (github.com) >

Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-08 Thread Willy Tarreau
Hi Miroslav! On Wed, Apr 07, 2021 at 11:32:16AM +0200, Miroslav Zagorac wrote: > Hello, > > due to the modified function declaration, the opentracing filter can no > longer be compiled. > > In commit 9533a7038 new parameters have been added to the declaration > of function parse_logsrv(). Oops,