Hi Willy.
Am 25-07-2014 19:28, schrieb Willy Tarreau:
Hi all,
Here I'm putting down my analysis of what was right and what was wrong
in the development model for 1.4 and 1.5, hoping to improve it for 1.6.
[long e-mail, even for one from me].
[long and detailed Informations snipped]
Concerning the new features, no promises, but we know that we need to
progress in the following areas :
- multi-process : better synchronization of stats and health checks,
and find a way to support peers in this mode. I'm still thinking a
lot that due to the arrival of latency monsters that are SSL and
compression, we could benefit from having a thread-based
architecture
so that we could migrate tasks to another CPU when they're going to
take a lot of time. The issue I'm seeing with threads is that
currently the code is highly dependent on being alone to modify any
data. Eg: a server state is consistent between entering and leaving
a health check function. We don't want to start adding huge mutexes
everywhere.
How about to be able to inject health check status over sockets (unix,
ip, ...).
The Idea is to have the possibility to inform haproxy about the status
from a service over the health-check-socket.
This makes possible to use a distributed check infrastructure and inform
haproxy about the state of a service.
This socket can also be used for internal use e.g. haproxy forked
process.
Similar way could be used with peers storage.
When haproxy is able to send $Session-Information to redis, memcache,
... then you can "better" use current environment resources.
- hot reconfiguration : some users are abusing the reload mechanism
to
extreme levels, but that does not void their requirements. And many
other users occasionally need to reload for various reasons such as
adding a new server or backend for a specific customer. While in
the
past it was not possible to change a server address on the fly, we
could now do it easily, so we could think about provisionning a few
extra servers that could be configured at run time to avoid a
number
of reloads. Concerning the difficulty to bind the reloaded
processes,
Simon had done some work in this area 3 years ago with the master-
worker model. Unfortunately we never managed to stabilize it
because
of the internal architecture that was hard to adapt and taking a
lot
of time. It could be one of the options to reconsider though, along
with FD passing across processes. Similarly, persistent server
states
across reloads is often requested and should be explored.
Similar as USR2 in nginx?
http://nginx.org/en/docs/control.html
With this signal nginx forks new processes "send" the FD to the new
process and deliver the old requests with the old processes and new with
the new process. This have also zero downtime for the client.
I don't know if the FOSS version is able to persistent server states
across reloads, maybe the commercial one.
[snipp]
- HTTP/2 : this is the most troublesome part that we absolutely need
to work on, because it may redefine the whole product's
architecture
(and I'm currently working on identifying the shortest path to
having
something acceptable). It's critically important because when
HTTP/2
starts to be deployed, there will be the products which support it,
and the other ones... A naive approach could consist in having a
protocol converter to receive HTTP/2 and convert the frames to
HTTP/1 but that is highly counter productive since it will actually
significantly slow down communications. The reason is that browsers
will try hard not to emit multiple connections, so all objects
would
be serialized, making things much worse than with HTTP/1. So we
really need to instantiate multiple streams from a single
connection
and currently haproxy is not at all architectured this way
(initially
it's a pipe with one socket at each end). Actually though many
entities were removed from the struct session and it could be an
opportunity for going even further and definitely get rid of it.
I know haproxy not need to much external library but in this case could
be a good solution to use a external lib like http://nghttp2.org/
[snipp]
If anyone has any comment / question / suggestion, as usual feel free
to
keep the discussion going on.
Done ;-).
Thank you willy and your team for this very awesome peace of SW.
Cheers Aleks
For now enough talking, I'm going back to real work :-)
Willy
---
[1] http://yarchive.net/comp/linux/development_speed.html
