Re: Inconsistent reading of txn vars from Lua script
On Tue, May 11, 2021 at 05:41:28PM -0300, Joao Morais wrote:
>
>
> > Em 10 de mai. de 2021, à(s) 18:04, Willy Tarreau escreveu:
> >
> > On Mon, May 10, 2021 at 10:41:36PM +0200, Willy Tarreau wrote:
> >>> core.register_action("auth", { "http-req" }, function(txn)
> >>> txn:set_var("txn.code", 401, true)
> >
> > So the problem is exactly here and it works as designed. This
> > argument "ifexist" was added a year ago to avoid Lua allocating
> > random variable names:
> >
> > 4e172c93f ("MEDIUM: lua: Add `ifexist` parameter to `set_var`")
> >
> > What the "true" argument does here is to refrain from creating
> > the variable if it does not exist. After you look it up from the
> > service, the variable gets created and it exists, hence why it
> > then works next times.
> >
> > If you want it to always be created (which I assume you want
> > to), just drop this argument or explicitly set it to false.
>
> Thanks Willy for the explanation and sorry about the false alarm, I didn't
> see the whole picture here.
No problem, it made me search and (re)discover this area :-)
> Just to confirm how it works, I created the snippet below:
>
> http-request lua.auth ## assigning txn.core
> http-request return lf-string %[var(txn.code)] content-type text/plain
>
> It worked since the first run and this is the only place I declared txn.code.
> Does this mean that a var is created in the following conditions?
> Any change in the sentences below?
>
> - after the first read from a Lua script
> - after the first write from a Lua script provided that ifexists parameter is
> set to false
> - always exists, if used anywhere in the configuration file
It's not a matter of first or second access. It's that the function
you used initially resulted in always allocating an entry for the
variable's name, causing some huge memory usage for those who were
using them like maps and performing random lookups there. In order
to avoid this, Tim added an extra argument saying that we're just
performing an opportunistic lookup and that the variable must not
be created if it does not exist.
Other parts of the code (the native config parts I mean) which use
variables always result in a creation because these names are static.
So my understanding is that it can be simplified to this:
- a variable declared in the config always exists
- a variable accessed from Lua with ifexists set to true will not
be created but will be found if it exists
- a vraiable accessed from Lua with ifexists set to false or not
present will always be created during the lookup.
Interestingly, the code for variables was initially made for the config,
so it doesn't seem to destroy variable names when they're released since
that was pointless with the config. I think that code should be revisited
in 2.5 to improve the situation (e.g. by marking that the variable was
dynamically allocated maybe), but I don't know this part well so I'll
probably stop before starting to suggest stupidities :-)
Willy
Re: [PATCH] BUILD/MINOR: opentracing: fixed compilation with filter, enabled
Hi Ilya, On Tue, May 11, 2021 at 10:59:46PM +0500, ??? wrote: > I tried opentracing ci, I estimate 1-2 weeks to stabilize. Can we delay 2.4 > after that? If you mean delaying 2.4 after some extra CI is ready, not at all. What would delay it is a big last minute regression, but postponing a release just to wait for something never works as stuff continues to accumulate. Believe me, that's what turned the planed 6-months cycle of 1.5 into 4.5 years. In addition, the feedback from users just after a release is always faster and more valuable than anything we can try to imagine at the last minute. Do not worry, we do have time to continue to improve the CI (which is already great), there's no emergency and whatever you provide can be merged at any moment since it focuses on the development tree, so do not feel pressured by an upcoming version nor any timing in general. Cheers, Willy
Re: Inconsistent reading of txn vars from Lua script
> Em 10 de mai. de 2021, à(s) 18:04, Willy Tarreau escreveu:
>
> On Mon, May 10, 2021 at 10:41:36PM +0200, Willy Tarreau wrote:
>>> core.register_action("auth", { "http-req" }, function(txn)
>>> txn:set_var("txn.code", 401, true)
>
> So the problem is exactly here and it works as designed. This
> argument "ifexist" was added a year ago to avoid Lua allocating
> random variable names:
>
> 4e172c93f ("MEDIUM: lua: Add `ifexist` parameter to `set_var`")
>
> What the "true" argument does here is to refrain from creating
> the variable if it does not exist. After you look it up from the
> service, the variable gets created and it exists, hence why it
> then works next times.
>
> If you want it to always be created (which I assume you want
> to), just drop this argument or explicitly set it to false.
Thanks Willy for the explanation and sorry about the false alarm, I didn’t see
the whole picture here.
Just to confirm how it works, I created the snippet below:
http-request lua.auth ## assigning txn.core
http-request return lf-string %[var(txn.code)] content-type text/plain
It worked since the first run and this is the only place I declared txn.code.
Does this mean that a var is created in the following conditions? Any change in
the sentences below?
- after the first read from a Lua script
- after the first write from a Lua script provided that ifexists parameter is
set to false
- always exists, if used anywhere in the configuration file
Thanks,
Re: [PATCH] BUILD/MINOR: opentracing: fixed compilation with filter, enabled
I tried opentracing ci, I estimate 1-2 weeks to stabilize. Can we delay 2.4 after that? On Tue, May 11, 2021, 10:37 PM Miroslav Zagorac wrote: > Hello, > > The inclusion of header files proxy.h and tools.h was added to the > addons/ot/include/include.h file. Without this HAProxy cannot be > compiled if the OpenTracing filter is to be used. > > Best regards, > > -- > Zaga > > What can change the nature of a man? >
[PATCH] BUILD/MINOR: opentracing: fixed compilation with filter, enabled
Hello, The inclusion of header files proxy.h and tools.h was added to the addons/ot/include/include.h file. Without this HAProxy cannot be compiled if the OpenTracing filter is to be used. Best regards, -- Zaga What can change the nature of a man? >From 6fe243c20b938afdfa934d6c075ae154bac9a976 Mon Sep 17 00:00:00 2001 From: Miroslav Zagorac Date: Tue, 11 May 2021 19:21:54 +0200 Subject: [PATCH] BUILD/MINOR: opentracing: fixed compilation with filter enabled The inclusion of header files proxy.h and tools.h was added to the addons/ot/include/include.h file. Without this HAProxy cannot be compiled if the OpenTracing filter is to be used. --- addons/ot/include/include.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/addons/ot/include/include.h b/addons/ot/include/include.h index 3c0d11094..f185a53b2 100644 --- a/addons/ot/include/include.h +++ b/addons/ot/include/include.h @@ -31,8 +31,10 @@ #include #include #include +#include #include #include +#include #include #include "config.h" -- 2.30.1
Re: Error "line too long" after adding some conditions
Tim, Adis Thanks a lot for your hints. I'll give a try. Kind regards, Tom On 11.05.21 17:35, Tim Düsterhus wrote: Willy, On 5/11/21 5:21 PM, Willy Tarreau wrote: What do you guys think a more acceptable limit would be for each of them ? Both can be increased if needed, they will only use a little bit more memory during the parsing. In Tom's case two alternative solutions that are much more readable were available. I consider these lowish limits a good thing, because it forces users to rethink what they are doing and possibly ask on the list to find a better solution. Best regards Tim Düsterhus
Re: Error "line too long" after adding some conditions
On Tue, May 11, 2021 at 05:35:17PM +0200, Tim Düsterhus wrote: > Willy, > > On 5/11/21 5:21 PM, Willy Tarreau wrote: > > What do you guys think a more acceptable limit would be for each of them ? > > Both can be increased if needed, they will only use a little bit more > > memory during the parsing. > > > > In Tom's case two alternative solutions that are much more readable were > available. I consider these lowish limits a good thing, because it forces > users to rethink what they are doing and possibly ask on the list to find a > better solution. This is a good point :-) Willy
Re: Error "line too long" after adding some conditions
Willy, On 5/11/21 5:21 PM, Willy Tarreau wrote: What do you guys think a more acceptable limit would be for each of them ? Both can be increased if needed, they will only use a little bit more memory during the parsing. In Tom's case two alternative solutions that are much more readable were available. I consider these lowish limits a good thing, because it forces users to rethink what they are doing and possibly ask on the list to find a better solution. Best regards Tim Düsterhus
Re: Error "line too long" after adding some conditions
Tom, On 5/11/21 4:15 PM, Tom wrote: Using haproxy 2.1.3: I've configured multiple conditions to a "use_backend" directive like this: use_backend example_192.168.1.30 if host_test.example.com || host_a01.test.example.com || host_a02.test.example.com || host_a03.test.example.com || host_a04.test.example.com || ... || ... When I add some more conditions, then I got an error after reloading the daemon: "line too long, truncating at word 65, position 783:..." How can I configure more conditions in the "use_backend"-directive above without having the error? In this specific case (using ||) you can just repeat the 'use_backend' directive in multiple lines: use_backend example_192.168.1.30 if host_test.example.com use_backend example_192.168.1.30 if host_a01.test.example.com use_backend example_192.168.1.30 if host_a02.test.example.com use_backend example_192.168.1.30 if host_a03.test.example.com As Adis said you can also define the same ACL multiple times: acl test_dot_example req.hdr(host) test.example.com acl test_dot_example req.hdr(host) a01.test.example.com acl test_dot_example req.hdr(host) a02.test.example.com acl test_dot_example req.hdr(host) a03.test.example.com use_backend example_192.168.1.30 if test_dot_example Best regards Tim Düsterhus
Re: [PATCH] BUG/MINOR: http_act: Fix normalizer names in error messages
Hi Tim, On Mon, May 10, 2021 at 11:21:20PM +0200, Tim Duesterhus wrote: > These places were forgotten when the normalizers were renamed. (...) All of your 3 patches applied, thank you! Willy
Re: Error "line too long" after adding some conditions
Hello, On Tue, May 11, 2021 at 04:23:09PM +0200, Adis Nezirovic wrote: > On 5/11/21 4:15 PM, Tom wrote: > > When I add some more conditions, then I got an error after reloading the > > daemon: > > > > "line too long, truncating at word 65, position 783:..." > > I'm also interested in technical reason for this limitation :), waiting for > smarter people to speak ;) There's a default limit of 2kB per line and 64 words. They can be changed respectively by defining LINESIZE and MAX_LINE_ARGS. This last one needs to be changed in defaults.h though. The reason for the 2kB limit (which started at 256 long ago) was that it's what fits in a terminal, and quite frankly when you're editing a single line that is that long, it's a horrible pain. And also please bear with those at the end of the chain who help you with bug reports and who have to face such monstrosities (such as a long horizontal scroll wheel in github issues that prevents from reading any context). Regarding the max args of 64. It used to be considered insanely huge when it was defined. What do you guys think a more acceptable limit would be for each of them ? Both can be increased if needed, they will only use a little bit more memory during the parsing. Willy
Re: Error "line too long" after adding some conditions
On 5/11/21 4:15 PM, Tom wrote: Using haproxy 2.1.3: I've configured multiple conditions to a "use_backend" directive like this: use_backend example_192.168.1.30 if host_test.example.com || host_a01.test.example.com || host_a02.test.example.com || host_a03.test.example.com || host_a04.test.example.com || ... || ... When I add some more conditions, then I got an error after reloading the daemon: "line too long, truncating at word 65, position 783:..." I'm also interested in technical reason for this limitation :), waiting for smarter people to speak ;) How can I configure more conditions in the "use_backend"-directive above without having the error? Maybe you can use named acl, repeat it multiple times: "With named ACLs, specifying the same ACL name multiple times will cause a logical OR of the conditions, so the last block can also be expressed as: acl evil path_beg /evil/ acl evil path_end /evil http-request deny if evil " Example from our blog post: https://www.haproxy.com/blog/introduction-to-haproxy-acls/ Best regards, -- Adis Nezirovic Software Engineer HAProxy Technologies - Powering your uptime! 375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US +1 (844) 222-4340 | https://www.haproxy.com
Error "line too long" after adding some conditions
Hi Using haproxy 2.1.3: I've configured multiple conditions to a "use_backend" directive like this: use_backend example_192.168.1.30 if host_test.example.com || host_a01.test.example.com || host_a02.test.example.com || host_a03.test.example.com || host_a04.test.example.com || ... || ... When I add some more conditions, then I got an error after reloading the daemon: "line too long, truncating at word 65, position 783:..." How can I configure more conditions in the "use_backend"-directive above without having the error? Any hints for this? Many thanks. Kind regards, Tom
Faster paid guest posting and link building service available
*I am a blogger outreach. I can provide you advertisement through guest posts on general and casino sites and tell me pls which niche sites you need and i have another best * *websites pls contact me i will post on your article very soon* SiteName DA PA https://thelonelyat.home.blog 89 43 https://voicesfromtheblogs.com/ 50 36 https://humanengineers.com/ 37 36 https://www.newgenapps.com/ 47 40 https://tricksbystg.org 26 37 https://updatedideas.com/ 42 36 https://pittythings.com/ 44 33 https://whatisfullformof.com/ 58 40 waiting your positive reply thanks best regards
