Le 28/03/2024 à 18:14, Christopher Faulet a écrit :
Thanks Lokesh, Abhijeet and Aleksandar for your feedback. This truly help us.
Thanks too to Pierre and Mattia for their feedback on the request mirroring.
Rest assured that we take this into account in our reflections.
After some internal discussions and also regarding to feedback we had
internally, we've decided to invest some time to rewrite the engine for the
3.1. A feature request was created to keep the conversation going
(https://github.com/haproxy/haproxy/issues/2502).
But to sum up, the idea is to rethink the engine to make it simpler. Of course,
the engine will be based on recent core features. But features will also be
reduced to remain maintainable. Among other things, the asynchronous mode will
be removed because it is far too complex and most probably unused. It is clearly
an over-designed feature. Then we will not invest time on the payload streaming
support. This feature is pretty complex to implement and this is mainly why we
never made the current SPOE evolved. Let's be reasonable this time. This will
probably influence the design.
For people who have invested time in SPOAs development, the protocol will be
kept compatible. The purpose here is to rewrite the engine itself. So it is
above all internal to HAProxy. For the 3.0, the warning about the SPOE will be
changed to notify users some configuration changes should be expected in future
versions. Some other small changes should be expected for the 3.0. But the heavy
lifting will be performed on the 3.1.
Of course, it is still a subject under discussion. The above issue is here to
collect ideas for the next steps but also for more long term features. Feel free
to feed it.
Hi,
As announced, the SPOE was finally refactored. This new SPOE will be shipped
with the 3.1-dev4. It is a full rewrite of the engine, based on a dedicated SPOP
multiplexer. It means a "spop" proxy mode, used for SPOE backends, was added. It
is now the default mode for backends used by SPOE engines. "mode spop" can be
explicitly specified, but otherwise, the mode is detected during post-parsing.
So, these backends cannot be mixed for a raw TCP usage.
Thanks to this refactoring, SPOP connections are now properly reused, similarly
to HTTP connections. So, it is now possible to properly balance connections when
several servers are configured. The management of IDLE connections used on HTTP
backends is used for SPOP connections. It is a huge improvement compared to what
was performed by hand with the pool of SPOE applets.
In parallel, the notion of child and parent streams was added. In this context,
the SPOE stream, used to exchange messages with SPOA is the child while the main
stream is the parent. The child stream is created by the parent stream to
perform some processing for its parent. In addition, it is possible from a child
stream to retrieve variables of the parent stream. This feature can be used to
perform some processing in the child stream based on properties of the parent
stream. The first and probably the main usage is to apply stickiness rules in a
child stream based on info from the parent stream. dedicated scopes was added to
retrieve parent variables (psess, ptxn, preq and pres).
Here is a very simple example:
frontend front-http
bind *:8889
tcp-request content set-var(txn.client_src) src
filter spoe engine ip-reputation config /spoe-ip-reputation.conf
http-request send-spoe-group ip-reputation check-client-ip
http-response set-header X-Ip-Score %[var(sess.iprep.ip_score)]
default_backend back-http
backend back-http
server www 127.0.0.1:8000
backend iprep-backend
mode spop
timeout server 10s
stick-table type ip size 200k expire 30m
stick on var(ptxn.client_src)
server iprep-srv1 127.0.0.1:12345
server iprep-srv2 127.0.0.1:12345
server iprep-srv3 127.0.0.1:12345
server iprep-srv4 127.0.0.1:12345
With this configuration, SPOP sessions are sticky on servers based on the client
source address. The configuration manual was updated accordingly. Check it for
details.
About other changes, the fragmentation support and the asynchronous mode were
removed. Without the streaming feature, the fragmentation is just useless. And
the asynchronous mode is overkill and just incompatible with the new
architecture. Following SPOE parameters were also removed and are silently
ignored when found in SPOE configuration: maxconnrate, maxerrrate,
max-waiting-frames, timeout hello and timeout idle. SPOE log format was adapted
because several info no longer exist.
The first and main step is finished. But there are some missing parts. For
instance, connection closures are a bit dirty for now because the internal API
must evolve a bit first. This will be achieved before the 3.1 release. Stats
regarding the SPOE must be enriched. Some parameters must be added (in
configuration and in the protocol). For instance the maximum number of streams
per connection. A graceful disconnect, to allow SPOA reloads for instance, is
also on the todo list. It could be good to be able to configure the format of
SPOE log messages. I also want to add a way to not necessarily request ACKs for
NOTIFY frames. Of course, all ideas are welcome ;)
I'm really glad to have (almost) achieved this refactoring. But it must be
tested now. So I encourage everyone with a SPOA to test it before the 3.1
release to unearth all bugs.
Thanks,
--
Christopher Faulet
