QUIC draft 23 docs are out

[Aleksandar Lazic]

Hi.

Lucas Pardue have shared the info via twitter, jfyi.

Transport - https://tools.ietf.org/html/draft-ietf-quic-transport-23
TLS   - https://tools.ietf.org/html/draft-ietf-quic-tls-23
Recovery  - https://tools.ietf.org/html/draft-ietf-quic-recovery-23
Version-Independent - https://tools.ietf.org/html/draft-ietf-quic-invariants-07

The working group main page is this.

https://tools.ietf.org/wg/quic/

Regards
Aleks

Re: 2.0 regression? Control socket and ACL patterns with spaces

[Conrad Hoffmann]

Hi, 

On Thu, 2019-09-12 at 08:12 +0200, Willy Tarreau wrote:
> Hi Conrad,
> 
> On Wed, Sep 11, 2019 at 04:53:18PM +0200, Conrad Hoffmann wrote:
> > Hey again,
> > 
> > sorry for reviving an old(ish) thread, but I would love to get some
> > kind of feedback, even if it's just that this will stay this way
> > and I
> > need to deal with it.
> > 
> > I also tried with the payload syntax in the meantime, but to no
> > avail.
> > Is there any supported way of adding patterns with spaces to an ACL
> > over the command socket at all in 2.0?
> 
> I didn't notice this thread last week, thanks for reviving it. Well
> your report and your attempt at fixing it perfectly illustrate why
> I absolutely never use such functions like strcspn(). The reason is
> that such functions tend to make you write context-unaware parsers.
> It's like looking for a closing parethesis without counting the
> number of opening ones.
> 
> Instead we should use the same tokenizer as the config parser uses,
> it's quite simple and not prone to such issues. Plus we'd even get
> support for quotes and hex characters as a bonus (though we could
> imagine not supporting some of them in the backported version, but
> I'm not sure of the benefit of *not* supporting them).

That's great to hear! I trust your judgement there, and it would
probably be the cleaner solution (as it re-enables the previous
behavior), but I'll just mention that I'd be perfectly fine with using
the payload syntax if that worked for "acl add" in case that's easier
to implement.

Anyways, thanks for picking that up, looking forward to whatever
solution :)

Thanks a lot,
Conrad


> 
> Maybe we should do that before issuing 2.0.6 so that we don't keep
> the CLI broken for too long.
> 
> Willy
-- 
Conrad Hoffmann
Systems and Traffic Engineering

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

Managing Director: Artem Fishman | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG
Charlottenburg  | HRB 110657B  

Re: Upgrading from 1.7 to 2.0 causes connection spam

[Elias Abacioglu]

Alright, thanks Willy and Lukas!

But I wonder why the build-up of connections is between our haproxy-no-tls
and haproxy-tls and not the "real" backends. The communication between
haproxy-no-tls and haproxy-tls is "mode tcp" and "http-reuse never" and
since it's mode tcp, it shouldn't really use the http-reuse?
Both haproxy-no-tls and haproxy-tls uses our backends. So wouldn't I see
this problem between the haproxy-no-tls and our backends as well if I was
affected?
I believe I could use "pool-max-conn 0" between haproxy-no-tls and
haproxy-tls since it's "mode tcp" and I don't want to reuse connections as
it breaks the "option forwardfor", sounds logical?

I've never actually compiled haproxy before, I just install Vincent Bernats
PPA packages.
I guess I could take his packages and try to build my own deb with the
latest snapshot.

So a bit off-topic, a pipeline that does nightly builds would make it
easier to test changes and bugfixes.
If anyone on this list have the time and competence, I could provide the
hardware and probably bandwidth (well depending on nr downloads).
(I should probably lift this part of the conversation to a separate thread)

/Elias

On Thu, Sep 12, 2019 at 8:04 AM Willy Tarreau  wrote:

> Hi guys,
>
> On Wed, Sep 11, 2019 at 09:28:23PM +0200, Lukas Tribus wrote:
> > Hello Elias,
> >
> > On Wed, Sep 11, 2019 at 11:52 AM Elias Abacioglu
> >  wrote:
> > > So we do zero config changes, upgrade haproxy to 2.0.x + restart
> haproxy and like a minute or so then it runs out of resources.
> > > Each haproxy (v2.0.5, no-TLS) have an request rate of 55-90K/s.
> > > Each haproxy (v1.7.11, TLS) have an request rate of 15-20/s.
> > > Each haproxy (v2.0.5, no-TLS) have a connection rate of 7-12K/s.
> > > Each haproxy (v1.7.11, TLS) has a connection rate of 6-7K/s.
> > >
> > > I have no clue why a zero config change upgrade would break this
> setup. Anyone that can help me go forward with troubleshooting this or
> explain what might cause it to mass up established connections?
> >
> > It's a bug. There are lots of fixes in the 2.0 tree after 2.0.5. You
> > could either wait for 2.0.6, or try the current stable git tree (2.0
> > git [1] or the snapshot from last night [2]).
> >
> > Beginning to troubleshoot 2.0.5 at this point does not make a lot of
> > sense inmho considering the amount and importance of the fixes
> > committed after 2.0.5.
>
> Given the connection rate, I suspect it could be related to the excess
> of idle conns kept on the backend side. I faced the exact same problem
> recently during a test with ulimit -n 100k. The CPU usage was 100% on
> a single core at 700 conn/s! Strace showed nothing abnormal, until I
> figured that connect() would take 1.4 millisecond, confirmed by perf()
> showing that the system was trying hard to find a spare source port. I
> addressed it with the commit below that was backported to latest 2.0:
>
>   4cae3bf631 ("BUG/MEDIUM: connection: don't keep more idle connections
> than ever needed")
>
> Elias, one work-around for the issue is to add "pool-max-conn 0" on your
> server lines, or "http-reuse never", or "http-reuse always", depending
> on whether the next stage supports keep-alive well.
>
> And to second what Lukas said, please give the latest snapshot a try, it
> would save you from having to adjust your configuration.
>
> Cheers,
> Willy
>

Re: Upgrading from 1.7 to 2.0 causes connection spam

[Willy Tarreau]

On Thu, Sep 12, 2019 at 11:45:00AM +0200, Elias Abacioglu wrote:
> Alright, thanks Willy and Lukas!
> 
> But I wonder why the build-up of connections is between our haproxy-no-tls
> and haproxy-tls and not the "real" backends. The communication between
> haproxy-no-tls and haproxy-tls is "mode tcp" and "http-reuse never" and
> since it's mode tcp, it shouldn't really use the http-reuse?

Ah indeed, I missed this point. Thus it could be another bug, but it
looked so similar to the one I recently addressed that it appeared
almost obvious to me it was the same :-/

> Both haproxy-no-tls and haproxy-tls uses our backends. So wouldn't I see
> this problem between the haproxy-no-tls and our backends as well if I was
> affected?

Likely yes, depending on how the connections are closed.

> I believe I could use "pool-max-conn 0" between haproxy-no-tls and
> haproxy-tls since it's "mode tcp" and I don't want to reuse connections as
> it breaks the "option forwardfor", sounds logical?

For "mode tcp" there's no pool nor reuse.

> I've never actually compiled haproxy before, I just install Vincent Bernats
> PPA packages.
> I guess I could take his packages and try to build my own deb with the
> latest snapshot.
> 
> So a bit off-topic, a pipeline that does nightly builds would make it
> easier to test changes and bugfixes.
> If anyone on this list have the time and competence, I could provide the
> hardware and probably bandwidth (well depending on nr downloads).
> (I should probably lift this part of the conversation to a separate thread)

We do have nightly snapshots but we don't provide builds anymore. I
stopped doing that around 1.5 when deployments started to diverge due
to the various libraries found in field with different distros. Now
we rely on distros to ship an up-to-date well-known version and we
provide sources for those who want to tune anything for their use
case. I'm not much interested in having to go back to the extra
entropy in bug reports caused by random build combinations, nor
having to arbitrate what option should be enabled in the default
build even if that forces someone else to install extra packages
that they don't need.

Regards,
Willy

Re: [PATCH] MINOR: connection: add "set-authority" and "authority" sample fetch

[Emmanuel Hocdet]

patch update with bug fix

> Le 10 sept. 2019 à 14:19, Emmanuel Hocdet  a écrit :
> 
> 
> Hi,
> 
> Included, my first proposal for « set-authority » action, to set 
> custom "authority" sample  fetch.
> 
> Use case could be to use « sni authority » in server line.
> For "proxy-v2-options authority », authority is pick from custom
> authority (« set-authority »), ppv2 authority or ssl_fc_sni.
> Sample fetch « authority » could be do the same, but i don’t
> know if it’s a good idea.
> 
> ++
> Manu
> 
> 



0001-MINOR-connection-add-set-authority-and-authority-sam.patch
Description: Binary data

Re: 2.0 regression? Control socket and ACL patterns with spaces

[Willy Tarreau]

On Thu, Sep 12, 2019 at 10:14:40AM +0200, Conrad Hoffmann wrote:
> That's great to hear! I trust your judgement there, and it would
> probably be the cleaner solution (as it re-enables the previous
> behavior), but I'll just mention that I'd be perfectly fine with using
> the payload syntax if that worked for "acl add" in case that's easier
> to implement.
> 
> Anyways, thanks for picking that up, looking forward to whatever
> solution :)

Sadly this change was made as part of this patch :

  abbf607 ("MEDIUM: cli: Add payload support")

and it touches multiple areas at once, so it will be more complicated
than I tought to unroll it because this requires extra care to be sure
we don't break the payload support.

Thus I'd rather not keep 2.0.6 on hold for this fix, and would rather
fix it independently.

If you have time to create an entry in the github issue tracker for
this so that we don't forget it, that would be great (feel free to
send a link to this thread). Otherwise I'll do it later, but I'm sure
you're less likely to forget it than I am :-)

Thanks,
Willy

Re: RFC uuid for log-format

[Willy Tarreau]

Hi Luca,

On Wed, Sep 11, 2019 at 10:31:27AM +, Schimweg, Luca wrote:
> Hey Tim,
> 
> Thanks, I missed the empty line. Fixed it
> Yeah, the email is correct. There is no github.com account associated with
> this email, so I'll use my private email to commit, which has a github.com
> account.

I've now merged your patch but I have applied a few minor adaptations:
  - there were space warnings in "git am" due to trailing spaces at the
end of a few lines (these ones appear in red when doing "git diff"
when you enable color mode, so that you can easily spot them)

  - I noticed an indentation issue in smp_fetch_uuid() which used 4
spaces everywhere while smp_check_uuid() was fine.

  - I got a warning related to %llx being used while the argument was
uint64_t (I know, both are the same but gcc likely assumes that there
are some platforms where it's not the case). Thus I added a long long
explicit cast and took this opportunity to add "ll" at the end of the
mask, for 32-bit platforms.

  - I replaced the check (args[0].data.sint == 0) in smp_fetch_uuid()
with !args[0].type to validate there is no arg which is more
accurate.

I've seen your comment about smp_check_uuid() not being called when
there is no argument. I tested and that's indeed the case. I don't
remember if this was the intended behavior or not, that's something
we should recheck to make sure either no sample fetch relies on this
or if it's a bug (and probably we should ensure it's always called,
and document it as such).

I was also thinking that the random generation could be centralized
as a generic function to produce random bytes. I'll possibly have a
look at this some time later.

Thanks,
Willy

Re: Linux Builds broken on Travis CI

[Willy Tarreau]

Hi Tim,

On Fri, Sep 06, 2019 at 04:30:24PM +0200, Tim Düsterhus wrote:
> Dear List
> 
> something between 02bac85bee664976f6dcecc424864e9fb99975be and
> f909c91e8a739b9ef7409b399259201fe883771c broke all the Linux builds on
> Travis CI:
> 
> - 41 reg tests fail with a timeout
> - 3 reg tests pass
> 
> FreeBSD works fine.
> 
> Somebody really ought to take a look. I might try to bisect if I find a
> bit of spare time. If someone beats me to it: Go ahead.

I've been quite annoyed with this a number of times and ended up not
looking at build reports anymore due to this. I've spent some time
looking at the cause as well and bisecting, coming to the conclusion
that apparently the travis VMs are regularly overloaded. Most of the
times we see TCP connection timeouts on the loop back preventing the
vtest client from reaching haproxy! I've even seen a number of "out
of memory" messages hitting the client. It's possible that their
hypervisor is sometimes running out of memory. Maybe their service
is abused by other projects which induce a huge load. At some point I
used to click "build again", which managed to randomly work, but I
gave up, being used to seeing this constantly red :-(

I think that we should simply disable reg tests and stick to build
tests only. There's nothing worse than getting used to seeing errors,
as by not seeing a difference between a build error and a test error
we get trained to ignore results.

Maybe we can keep the reg tests for cron jobs, but given that they
similarly fail I don't see the benefit either.

I too would like to see them turn green again :-/

Cheers,
Willy

Re: [PATCH]: MEDIUM: enabling threads on osx

[Willy Tarreau]

Hi David,

On Sun, Sep 08, 2019 at 04:48:54PM +0100, David CARLIER wrote:
> Hi,
> 
> Another little patch for improving osx support.

Well, you should split your patch as it does at least 3 different things:
  - rename thread_info to ha_thread_info (it's not even clear if it's the
type or the variable that causes conflict, thus please mention it in
the commit message). If it's only the type, as I expect, it would be
nice if we can avoid renaming the variable everywhere.

  - enable USE_CPU_AFFINITY for OSX. Please try to improve your change
using "#if defined(__APPLE__)" instead of "#if !defined(__APPLE__)"
as the construct looks quite confusing with nested #ifs.

  - enable USE_THREAD for OSX

Otherwise I think the changes are OK, thanks for doing this!
Willy

Re: [PATCH]: MEDIUM: enabling threads on osx

[Willy Tarreau]

On Fri, Sep 13, 2019 at 05:03:34AM +0200, Willy Tarreau wrote:
>   - enable USE_CPU_AFFINITY for OSX. Please try to improve your change
> using "#if defined(__APPLE__)" instead of "#if !defined(__APPLE__)"
> as the construct looks quite confusing with nested #ifs.

On this last point, if the set_affinity code becomes really too messy,
probably that we should have a wrapper in compat.h to do this per-os.

Willy

[RFC] MINOR: attempt to insert into priority queue when full

[Patrick Hemmer]

This is a draft patch which essentially allows a high priority request 
to replace a low priority request when the queue is full.


Currently if a high priority request comes along, and the destination 
queue is full, the request gets rejected (or redispatched if possible). 
This change allows inserting into the queue if there are lower priority 
requests present. When doing so, the lowest priority request is evicted 
to make room.


The intent is for that evicted request to be handled as if it just came 
in. Meaning if there were a brand new request coming in, and the 
configuration were such that the request should be rejected, then the 
evicted request should be rejected. If redispatch is enabled a new 
request would be sent to a different server, and so the evicted request 
should be as well.



Now as for why this is tagged RFC: The implementation for the handling 
of the evicted request is bad. If you notice inside pendconn_displace(), 
the section of code that handles this (lines 498-505) has no business 
being there. Those operations should be handled higher up. It also 
doesn't work as intended, as instead of responding to the client with a 
503, it just kills the connection.


However I'm having difficulty finding the appropriate way to do this, 
and could use some guidance. I've tried several different things, and 
none of the behaviors are quite right.
The simplest solution of just performing __pendconn_unlink() and then 
waking the task results in the request being redispatched. The higher 
level code assumes that if the request was in a queue, and is now no 
longer in a queue, then redispatch is the appropriate action.


Thanks

-Patrick
From a3c8ba92a05ec877662359f963ece0cfa82051f8 Mon Sep 17 00:00:00 2001
From: Patrick Hemmer 
Date: Thu, 12 Sep 2019 22:56:51 -0400
Subject: [PATCH] MINOR: attempt to insert into priority queue when full

This makes it so that when a queue (server or proxy) is full, that we try to
insert into the queue and evict a request of lower priority.

The evicted request will either be redispatched if `option redispatch` is
enabled, or rejected if not.
---
 include/proto/queue.h |   1 +
 src/backend.c |   3 +-
 src/queue.c   | 119 ++
 3 files changed, 122 insertions(+), 1 deletion(-)

diff --git a/include/proto/queue.h b/include/proto/queue.h
index a7ab63b35..156bf3964 100644
--- a/include/proto/queue.h
+++ b/include/proto/queue.h
@@ -37,6 +37,7 @@
 extern struct pool_head *pool_head_pendconn;
 
 struct pendconn *pendconn_add(struct stream *strm);
+int pendconn_displace(struct stream *strm);
 int pendconn_dequeue(struct stream *strm);
 void process_srv_queue(struct server *s);
 unsigned int srv_dynamic_maxconn(const struct server *s);
diff --git a/src/backend.c b/src/backend.c
index 1b01536c1..56340d2de 100644
--- a/src/backend.c
+++ b/src/backend.c
@@ -976,7 +976,8 @@ int assign_server_and_queue(struct stream *s)
(srv->nbpend || srv->served >= srv_dynamic_maxconn(srv))) {
 
if (srv->maxqueue > 0 && srv->nbpend >= srv->maxqueue)
-   return SRV_STATUS_FULL;
+   // queue is full. see if priority allows us to 
insert
+   return pendconn_displace(s);
 
p = pendconn_add(s);
if (p)
diff --git a/src/queue.c b/src/queue.c
index 30b7ef056..ea015272a 100644
--- a/src/queue.c
+++ b/src/queue.c
@@ -185,6 +185,33 @@ void pendconn_unlink(struct pendconn *p)
pendconn_queue_unlock(p);
 }
 
+/* Comapres 2 pendconn queue priority keys.
+ *
+ * Returns :
+ *   -1 if k1 < k2
+ *   0  if k1 == k2
+ *   1  if k1 > k2
+ */
+static int key_cmp(u32 k1, u32 k2)
+{
+   if (KEY_CLASS(k1) < KEY_CLASS(k2))
+   return -1;
+   if (KEY_CLASS(k1) > KEY_CLASS(k2))
+   return 1;
+
+   if (k1 < NOW_OFFSET_BOUNDARY())
+   k1 += 0x10; // key in the future
+   if (k2 < NOW_OFFSET_BOUNDARY())
+   k2 += 0x10; // key in the future
+
+   if (k1 < k2)
+   return -1;
+   if (k1 > k2)
+   return 1;
+
+   return 0;
+}
+
 /* Retrieve the first pendconn from tree . Classes are always
  * considered first, then the time offset. The time does wrap, so the
  * lookup is performed twice, one to retrieve the first class and a second
@@ -212,6 +239,31 @@ static struct pendconn *pendconn_first(struct eb_root 
*pendconns)
return eb32_entry(node2, struct pendconn, node);
 }
 
+/* Retrieve the last pendconn from tree .
+ * Follows the same semantics as pendconn_first.
+ */
+static struct pendconn *pendconn_last(struct eb_root *pendconns)
+{
+   struct eb32_node *node, *node2 = NULL;
+   u32 key;
+
+   node = eb32_last(pendconns);
+   if (!node)
+   return NULL;
+
+   key = KEY_CLASS_OFFSET_BOUNDARY(node->key);
+   // We don't have eb32_lookup_l

Re: [PATCH]: MEDIUM: enabling threads on osx

[David CARLIER]

Hi ok fair points I forgot the haproxy's policy for more targeted changes :-).
For the the thread_info it is both, as it is a pointer type in macOS.
Here 3 patchset.

Regards.

On Fri, 13 Sep 2019 at 04:03, Willy Tarreau  wrote:
>
> Hi David,
>
> On Sun, Sep 08, 2019 at 04:48:54PM +0100, David CARLIER wrote:
> > Hi,
> >
> > Another little patch for improving osx support.
>
> Well, you should split your patch as it does at least 3 different things:
>   - rename thread_info to ha_thread_info (it's not even clear if it's the
> type or the variable that causes conflict, thus please mention it in
> the commit message). If it's only the type, as I expect, it would be
> nice if we can avoid renaming the variable everywhere.
>
>   - enable USE_CPU_AFFINITY for OSX. Please try to improve your change
> using "#if defined(__APPLE__)" instead of "#if !defined(__APPLE__)"
> as the construct looks quite confusing with nested #ifs.
>
>   - enable USE_THREAD for OSX
>
> Otherwise I think the changes are OK, thanks for doing this!
> Willy


0003-BUILD-MEDIUM-threads-enable-cpu_affinity-on-osx.patch
Description: Binary data


0002-BUILD-MEDIUM-threads-refactoring-thread_info-struct.patch
Description: Binary data


0001-BUILD-SMALL-threads-enable-threads-on-osx.patch
Description: Binary data

HAproxy Installation in AIX 6.1 and above

[Jaiswal, Vivek]

Hello HAProxy,

We at our company wants to install HAProxy latest stable 2.0 .
We are using AIX 6.1 (in dev) and above(AIX 7.2 in prod) Operating system.

We are facing issue and no documents to install HAProxy in AIX.
Can someone help us with the installation procedure and if possible the 
configuration procedure as well.

Thanks
Vivek

[https://www.aciworldwide.com/-/media/aci-footer]
This email message and any attachments may contain confidential, proprietary or 
non-public information. The information is intended solely for the designated 
recipient(s). If an addressing or transmission error has misdirected this 
email, please notify the sender immediately and destroy this email. Any review, 
dissemination, use or reliance upon this information by unintended recipients 
is prohibited. Any opinions expressed in this email are those of the author 
personally.

Re: RFC uuid for log-format

[Schimweg, Luca]

Hey Willy,

thanks so much!

Best regards
Luca

On 13.09.19, 04:48, "Willy Tarreau"  wrote:

Hi Luca,

On Wed, Sep 11, 2019 at 10:31:27AM +, Schimweg, Luca wrote:
> Hey Tim,
> 
> Thanks, I missed the empty line. Fixed it
> Yeah, the email is correct. There is no github.com account associated with
> this email, so I'll use my private email to commit, which has a github.com
> account.

I've now merged your patch but I have applied a few minor adaptations:
  - there were space warnings in "git am" due to trailing spaces at the
end of a few lines (these ones appear in red when doing "git diff"
when you enable color mode, so that you can easily spot them)

  - I noticed an indentation issue in smp_fetch_uuid() which used 4
spaces everywhere while smp_check_uuid() was fine.

  - I got a warning related to %llx being used while the argument was
uint64_t (I know, both are the same but gcc likely assumes that there
are some platforms where it's not the case). Thus I added a long long
explicit cast and took this opportunity to add "ll" at the end of the
mask, for 32-bit platforms.

  - I replaced the check (args[0].data.sint == 0) in smp_fetch_uuid()
with !args[0].type to validate there is no arg which is more
accurate.

I've seen your comment about smp_check_uuid() not being called when
there is no argument. I tested and that's indeed the case. I don't
remember if this was the intended behavior or not, that's something
we should recheck to make sure either no sample fetch relies on this
or if it's a bug (and probably we should ensure it's always called,
and document it as such).

I was also thinking that the random generation could be centralized
as a generic function to produce random bytes. I'll possibly have a
look at this some time later.

Thanks,
Willy