Hi Zach,
(Replying on the ML since your reply was directly to me only)
On 08/02/2024 21:10, Zach Pearson wrote:
Thanks for the reply, Tristan, and sorry for the late reply!
To add a little more context for our haproxy setup, we currently deploy haproxy
along with a sidecar control-plane that programs dynamic data such as backends.
We also have a sidecar application that we use to offload more complex logic
via SPOE. Inside of haproxy, we also use a few lua converters and fetchers as
well.
Seems like you already have as good a setup as it gets for that side of
things then! I'm jealous :-)
I think breaking up our features into their own jinja files will help a lot! As
we expand the environments we are deployed in this will help us better control
the features in each environment.
Nice if that helps you out at least a bit!
Beyond managing our configs, one thing we are trying to handle nicely is
multiple flows such as guest traffic vs authenticated traffic; for guest
traffic we do stricter anti-abuse/bot checking, for example.
This leads to a case where we have large blocks of configs that are all
applying some acls, which we’ve found can be difficult to manage and easy to
make mistakes.
It almost feels like we need some sort of control flow or logical blocks that exist between the frontend and backend. > We are considering doing this via multiple frontends and having our
public frontend choose which flow the traffic should go through and
loopback to the appropriate frontend. Of course, this will incur some
latency overhead, though.
If you have any advice or experience in this area that would be greatly
appreciated!
I can certainly relate to this issue, and for very similar reasons.
Alas, I have not found any better solution so far. And the looping
approach always felt wrong to me, so I have kept on abusing txvars and
so on to avoid it... which is not very fun indeed.
For what it's worth, we discussed tangential topics, in October, mainly
in the 2.9-dev7 announce thread if you missed it
(https://www.mail-archive.com/haproxy@formilux.org/msg44121.html)
Ended up a mix of both sides of large-config issues:
- dpapi, socket api, XDS, ... for dynamic management of runtime config
- extensibility issues for complex configs, notably with regards to
SPOE, filters, alternatives to LUA, ... for "this"
It's a bit of a messy thread and there's a lot to read, but I think the
portion of the thread that starts here is most relevant:
https://www.mail-archive.com/haproxy@formilux.org/msg44122.html
Aleksandar made a follow-up thread about filters here
https://www.mail-archive.com/haproxy@formilux.org/msg44164.html, as they
are one of the best extension points currently, but it was just him and
me in the end.
That said, I'm sure us 3 are not the only ones trying to -- as Willy
summarizes it well -- "put too much of the application into the load
balancer".
So if we want to see improvements for that use-case (essentially HAProxy
as a programmable edge, rather than "just" a load balancer / reverse
proxy) it'd be quite helpful to hear details/opinions/suggestion from
you and others on the matter. (and thus I end up asking rather than
answering, sorry :)
Thanks again!
Zach
Thanks,
Tristan
