[PATCH] BUG/MEDIUM: map: Fix memory leak in the map converter

2019-04-12 Thread Nenad Merdanovic
The allocated trash chunk is not freed properly and causes a memory leak
exhibited as the growth in the trash pool allocations. Bug was introduced
in commit 271022 (BUG/MINOR: map: fix map_regm with backref).

This should be backported to all branches where the above commit was
backported.
---
 src/map.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/map.c b/src/map.c
index 903c5b26..cf148930 100644
--- a/src/map.c
+++ b/src/map.c
@@ -207,6 +207,8 @@ static int sample_conv_map(const struct arg *arg_p, struct 
sample *smp, void *pr
  tmptrash->area,
  pat->data->u.str.area,
  (regmatch_t *)smp->ctx.a[0]);
+   free_trash_chunk(tmptrash);
+
if (len == -1)
return 0;
 
-- 
2.19.1




Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Richard Russo
Thank you; I had missed the context from 1.9.6.  I've updated my test machine 
and will report back on Monday (or earlier, if it runs into trouble)

-- 
  Richard Russo
  to...@enslaves.us

On Fri, Apr 12, 2019, at 4:17 AM, Olivier Houchard wrote:
> Hi,
> 
> On Fri, Apr 12, 2019 at 08:37:10AM +0200, Maciej Zdeb wrote:
> > Hi Richard,
> > 
> > Those patches from Olivier (in streams) are related to my report from
> > thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
> > out it wasn't a single bug and issue is not entirely fixed yet.
> > 
> > Currently I'm testing some additional patches from Olivier which hopefully
> > fix the issue definitely.
> > 
> 
> Indeed, the rmoeval of SI_FL_ERR in si_update_both() was bogus, and covered
> misuses of it.
> With the great help of Maciej, we investigated this, and I just pushed what
> we fixed so far. Richard, I'd be really interested in knowing if you still
> have issues with the latest master.
> 
> Thanks !
> 
> Olivier
> 
> > pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):
> > 
> > > It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> > > flag, process_stream can keep going back to redo around stream.c:line 
> > > 2503:
> > >
> > > if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> > > si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> > > ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> > > (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> > > CF_MASK_ANALYSER))
> > >  goto redo;
> > >
> > > Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> > > else does, the goto will get called forever. I don't understand this
> > > section enough to try to reproduce this, but I found several processes
> > > stuck here on a machine testing from yesterday's HEAD.
> > >
> > > Richard
> > >
> > > --
> > >   Richard Russo
> > >   to...@enslaves.us
> > >
> > >
>



Re: QAT intermittent healthcheck errors

2019-04-12 Thread Marcin Deranek

Hi Emeric,

On 4/12/19 5:26 PM, Emeric Brun wrote:


Do you have ssl enabled on the server side?


Yes, ssl is on frontend and backend with ssl checks enabled.


If it is the case could replace health check with a simple tcp check (without 
ssl)?


What I noticed before that if I (re)start HAProxy and reload immediately 
no stuck processes are present. If I wait before reloading stuck 
processes show up.
After disabling checks (I still keep ssl enabled for normal traffic) 
reloads work just fine (tried many time). Do you know how to enable TCP 
healthchecks while keeping SSL for non-healthcheck requests ?



Regarding the show info/lsoff  it seems there is no more sessions on client 
side but remaining ssl jobs (CurrSslConns) and I supsect the health checks to 
miss a cleanup of their ssl sessions using the QAT. (this is just an assumption)


In general instance where I test QAT does not have any "real" client 
traffic except small amount of healtcheck requests per frontend which 
are internally handled by HAProxy itself. Still TLS handshake still 
needs to take place. There are many more backend healthchecks. Looks 
like your assumption was correct..

Regards,

Marcin Deranek


On 4/12/19 4:43 PM, Marcin Deranek wrote:

Hi Emeric,

On 4/10/19 2:20 PM, Emeric Brun wrote:


On 4/10/19 1:02 PM, Marcin Deranek wrote:

Hi Emeric,

Our process limit in QAT configuration is quite high (128) and I was able to 
run 100+ openssl processes without a problem. According to Joel from Intel 
problem is in cleanup code - presumably when HAProxy exits and frees up QAT 
resources. Will try to see if I can get more debug information.


I've just take a look.

Engines deinit ar called:

haproxy/src/ssl_sock.c
#ifndef OPENSSL_NO_ENGINE
void ssl_free_engines(void) {
  struct ssl_engine_list *wl, *wlb;
  /* free up engine list */
  list_for_each_entry_safe(wl, wlb, _engines, list) {
  ENGINE_finish(wl->e);
  ENGINE_free(wl->e);
  LIST_DEL(>list);
  free(wl);
  }
}
#endif
...
#ifndef OPENSSL_NO_ENGINE
  hap_register_post_deinit(ssl_free_engines);
#endif

I don't know how many haproxy processes you are running but if I describe the 
complete scenario of processes you may note that we reach a limit:


It's very unlikely it's the limit as I lowered number of HAProxy processes 
(from 10 to 4) while keeping QAT NumProcesses equal 32. HAProxy would have 
problem with this limit while spawning new instances and not tearing down old 
ones. In such a case QAT would not be initialized for some HAProxy instances 
(you would see 1 thread vs 2 thread). About threads read below.


- the master sends a signal to older processes, those process will unbind and 
stop to accept new conns but continue to serve remaining sessions until the end.
- new processes are started and immediately and init the engine and accept 
newconns.
- When no more sessions remains on an old process, it calls the deinit function 
of the engine before exiting


What I noticed is that each HAProxy with QAT enabled has 2 threads (LWP) - 
looks like QAT adds extra thread to the process itself. Would adding extra 
thread possibly mess up HAProxy termination sequence ?
Our setup is to run HAProxy in multi process mode - no threads (or 1 thread per 
process if you wish).


I'm also supposed that old processes are stucked because there is some sessions 
which never ended, perhaps I'm wrong but a strace on an old process
could be interesting to know why those processes are stucked.


strace only shows these:

[pid 11392] 23:24:43.164619 epoll_wait(4,  
[pid 11392] 23:24:43.164687 <... epoll_wait resumed> [], 200, 0) = 0
[pid 11392] 23:24:43.164761 epoll_wait(4,  
[pid 11392] 23:24:43.953203 <... epoll_wait resumed> [], 200, 788) = 0
[pid 11392] 23:24:43.953286 epoll_wait(4,  
[pid 11392] 23:24:43.953355 <... epoll_wait resumed> [], 200, 0) = 0
[pid 11392] 23:24:43.953419 epoll_wait(4,  
[pid 11392] 23:24:44.010508 <... epoll_wait resumed> [], 200, 57) = 0
[pid 11392] 23:24:44.010589 epoll_wait(4,  

There are no connections: stucked process only has UDP socket on random port:

[root@externallb-124 ~]# lsof -p 6307|fgrep IPv4
hapee-lb 6307 lbengine   83u IPv4 3598779351  0t0 UDP *:19573



You can also use the 'master CLI' using '-S' and you could check if it remains 
sessions on those older processes (doc is available in management.txt)


Before reload
* systemd
  Main PID: 33515 (hapee-lb)
    Memory: 1.6G
    CGroup: /system.slice/hapee-1.8-lb.service
    ├─33515 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
    ├─34858 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
    ├─34859 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
    ├─34860 

Re: [External] Re: QAT intermittent healthcheck errors

2019-04-12 Thread Emeric Brun
Hi Marcin,

Do you have ssl enabled on the server side? If it is the case could replace 
health check with a simple tcp check (without ssl)?

Regarding the show info/lsoff  it seems there is no more sessions on client 
side but remaining ssl jobs (CurrSslConns) and I supsect the health checks to 
miss a cleanup of their ssl sessions using the QAT. (this is just an 
assumption) 

R,
Emeric

On 4/12/19 4:43 PM, Marcin Deranek wrote:
> Hi Emeric,
> 
> On 4/10/19 2:20 PM, Emeric Brun wrote:
> 
>> On 4/10/19 1:02 PM, Marcin Deranek wrote:
>>> Hi Emeric,
>>>
>>> Our process limit in QAT configuration is quite high (128) and I was able 
>>> to run 100+ openssl processes without a problem. According to Joel from 
>>> Intel problem is in cleanup code - presumably when HAProxy exits and frees 
>>> up QAT resources. Will try to see if I can get more debug information.
>>
>> I've just take a look.
>>
>> Engines deinit ar called:
>>
>> haproxy/src/ssl_sock.c
>> #ifndef OPENSSL_NO_ENGINE
>> void ssl_free_engines(void) {
>>  struct ssl_engine_list *wl, *wlb;
>>  /* free up engine list */
>>  list_for_each_entry_safe(wl, wlb, _engines, list) {
>>  ENGINE_finish(wl->e);
>>  ENGINE_free(wl->e);
>>  LIST_DEL(>list);
>>  free(wl);
>>  }
>> }
>> #endif
>> ...
>> #ifndef OPENSSL_NO_ENGINE
>>  hap_register_post_deinit(ssl_free_engines);
>> #endif
>>
>> I don't know how many haproxy processes you are running but if I describe 
>> the complete scenario of processes you may note that we reach a limit:
> 
> It's very unlikely it's the limit as I lowered number of HAProxy processes 
> (from 10 to 4) while keeping QAT NumProcesses equal 32. HAProxy would have 
> problem with this limit while spawning new instances and not tearing down old 
> ones. In such a case QAT would not be initialized for some HAProxy instances 
> (you would see 1 thread vs 2 thread). About threads read below.
> 
>> - the master sends a signal to older processes, those process will unbind 
>> and stop to accept new conns but continue to serve remaining sessions until 
>> the end.
>> - new processes are started and immediately and init the engine and accept 
>> newconns.
>> - When no more sessions remains on an old process, it calls the deinit 
>> function of the engine before exiting
> 
> What I noticed is that each HAProxy with QAT enabled has 2 threads (LWP) - 
> looks like QAT adds extra thread to the process itself. Would adding extra 
> thread possibly mess up HAProxy termination sequence ?
> Our setup is to run HAProxy in multi process mode - no threads (or 1 thread 
> per process if you wish).
> 
>> I'm also supposed that old processes are stucked because there is some 
>> sessions which never ended, perhaps I'm wrong but a strace on an old process
>> could be interesting to know why those processes are stucked.
> 
> strace only shows these:
> 
> [pid 11392] 23:24:43.164619 epoll_wait(4,  
> [pid 11392] 23:24:43.164687 <... epoll_wait resumed> [], 200, 0) = 0
> [pid 11392] 23:24:43.164761 epoll_wait(4,  
> [pid 11392] 23:24:43.953203 <... epoll_wait resumed> [], 200, 788) = 0
> [pid 11392] 23:24:43.953286 epoll_wait(4,  
> [pid 11392] 23:24:43.953355 <... epoll_wait resumed> [], 200, 0) = 0
> [pid 11392] 23:24:43.953419 epoll_wait(4,  
> [pid 11392] 23:24:44.010508 <... epoll_wait resumed> [], 200, 57) = 0
> [pid 11392] 23:24:44.010589 epoll_wait(4,  
> 
> There are no connections: stucked process only has UDP socket on random port:
> 
> [root@externallb-124 ~]# lsof -p 6307|fgrep IPv4
> hapee-lb 6307 lbengine   83u IPv4 3598779351  0t0 UDP *:19573
> 
> 
>> You can also use the 'master CLI' using '-S' and you could check if it 
>> remains sessions on those older processes (doc is available in 
>> management.txt)
> 
> Before reload
> * systemd
>  Main PID: 33515 (hapee-lb)
>    Memory: 1.6G
>    CGroup: /system.slice/hapee-1.8-lb.service
>    ├─33515 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
> /etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
>    ├─34858 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
> /etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
>    ├─34859 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
> /etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
>    ├─34860 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
> /etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
>    └─34861 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
> /etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
> * master CLI
> show proc
> #         
> 33515   master  0   0   0d 00h00m31s
> # workers
> 34858   worker  1   0   0d 00h00m31s
> 34859   worker  2   0   0d 00h00m31s
> 34860   worker  3   0   0d 00h00m31s
> 

Re: [External] Re: QAT intermittent healthcheck errors

2019-04-12 Thread Marcin Deranek

Hi Emeric,

On 4/10/19 2:20 PM, Emeric Brun wrote:


On 4/10/19 1:02 PM, Marcin Deranek wrote:

Hi Emeric,

Our process limit in QAT configuration is quite high (128) and I was able to 
run 100+ openssl processes without a problem. According to Joel from Intel 
problem is in cleanup code - presumably when HAProxy exits and frees up QAT 
resources. Will try to see if I can get more debug information.


I've just take a look.

Engines deinit ar called:

haproxy/src/ssl_sock.c
#ifndef OPENSSL_NO_ENGINE
void ssl_free_engines(void) {
 struct ssl_engine_list *wl, *wlb;
 /* free up engine list */
 list_for_each_entry_safe(wl, wlb, _engines, list) {
 ENGINE_finish(wl->e);
 ENGINE_free(wl->e);
 LIST_DEL(>list);
 free(wl);
 }
}
#endif
...
#ifndef OPENSSL_NO_ENGINE
 hap_register_post_deinit(ssl_free_engines);
#endif

I don't know how many haproxy processes you are running but if I describe the 
complete scenario of processes you may note that we reach a limit:


It's very unlikely it's the limit as I lowered number of HAProxy 
processes (from 10 to 4) while keeping QAT NumProcesses equal 32. 
HAProxy would have problem with this limit while spawning new instances 
and not tearing down old ones. In such a case QAT would not be 
initialized for some HAProxy instances (you would see 1 thread vs 2 
thread). About threads read below.



- the master sends a signal to older processes, those process will unbind and 
stop to accept new conns but continue to serve remaining sessions until the end.
- new processes are started and immediately and init the engine and accept 
newconns.
- When no more sessions remains on an old process, it calls the deinit function 
of the engine before exiting


What I noticed is that each HAProxy with QAT enabled has 2 threads (LWP) 
- looks like QAT adds extra thread to the process itself. Would adding 
extra thread possibly mess up HAProxy termination sequence ?
Our setup is to run HAProxy in multi process mode - no threads (or 1 
thread per process if you wish).



I'm also supposed that old processes are stucked because there is some sessions 
which never ended, perhaps I'm wrong but a strace on an old process
could be interesting to know why those processes are stucked.


strace only shows these:

[pid 11392] 23:24:43.164619 epoll_wait(4,  
[pid 11392] 23:24:43.164687 <... epoll_wait resumed> [], 200, 0) = 0
[pid 11392] 23:24:43.164761 epoll_wait(4,  
[pid 11392] 23:24:43.953203 <... epoll_wait resumed> [], 200, 788) = 0
[pid 11392] 23:24:43.953286 epoll_wait(4,  
[pid 11392] 23:24:43.953355 <... epoll_wait resumed> [], 200, 0) = 0
[pid 11392] 23:24:43.953419 epoll_wait(4,  
[pid 11392] 23:24:44.010508 <... epoll_wait resumed> [], 200, 57) = 0
[pid 11392] 23:24:44.010589 epoll_wait(4,  

There are no connections: stucked process only has UDP socket on random 
port:


[root@externallb-124 ~]# lsof -p 6307|fgrep IPv4
hapee-lb 6307 lbengine   83u IPv4 3598779351  0t0 
UDP *:19573




You can also use the 'master CLI' using '-S' and you could check if it remains 
sessions on those older processes (doc is available in management.txt)


Before reload
* systemd
 Main PID: 33515 (hapee-lb)
   Memory: 1.6G
   CGroup: /system.slice/hapee-1.8-lb.service
   ├─33515 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   ├─34858 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   ├─34859 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   ├─34860 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   └─34861 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234

* master CLI
show proc
# 
33515   master  0   0   0d 00h00m31s
# workers
34858   worker  1   0   0d 00h00m31s
34859   worker  2   0   0d 00h00m31s
34860   worker  3   0   0d 00h00m31s
34861   worker  4   0   0d 00h00m31s

After reload:
* systemd
 Main PID: 33515 (hapee-lb)
   Memory: 3.1G
   CGroup: /system.slice/hapee-1.8-lb.service
   ├─33515 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234 -sf 
34858 34859 34860 34861 -x /run/lb_engine/process-1.sock
   ├─34858 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   ├─34859 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 
/etc/lb_engine/haproxy.cfg -p /run/hapee-lb.pid -S 127.0.0.1:1234
   ├─34860 /opt/hapee-1.8/sbin/hapee-lb -Ws -f 

Re: `stats bind-process` broken

2019-04-12 Thread William Lallemand
Patrick, Christian,

Could you try the attached patch? It should fix the issue.

Thanks


-- 
William Lallemand
>From 71dae971bf9b59ec8ef2f94cab2c0d762744c2a9 Mon Sep 17 00:00:00 2001
From: William Lallemand 
Date: Fri, 12 Apr 2019 14:40:36 +0200
Subject: [PATCH] BUG/MEDIUM: mworker/cli: don't override the stats_fe
 configuration

Patrick Hemmer reported that the stats bind-process directive does not
work anymore.

The worker's internal stats socket was using the global stats_fe
frontend, which is a problem since its configuration can be changed. The
bind-process configuration was overriden by the creation of the worker's
internal stats socket.

With this patch, the worker uses its own frontend which does not collide
anymore with the stats_fe.

This must be backported to 1.9
---
 src/cli.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/src/cli.c b/src/cli.c
index ef6280645..d3e4b6140 100644
--- a/src/cli.c
+++ b/src/cli.c
@@ -96,6 +96,7 @@ static struct cli_kw_list cli_keywords = {
 extern const char *stat_status_codes[];
 
 static struct proxy *mworker_proxy; /* CLI proxy of the master */
+static struct proxy *mworker_fe;/* CLI frontend used in the workers */
 
 static char *cli_gen_usage_msg(struct appctx *appctx)
 {
@@ -2554,15 +2555,14 @@ int mworker_cli_sockpair_new(struct mworker_proc *mworker_proc, int proc)
 		return -1;
 	}
 
-	/* XXX: we might want to use a separate frontend at some point */
-	if (!global.stats_fe) {
-		if ((global.stats_fe = alloc_stats_fe("GLOBAL", "master-socket", 0)) == NULL) {
-			ha_alert("out of memory trying to allocate the stats frontend");
+	if (!mworker_fe) {
+		if ((mworker_fe = alloc_stats_fe("GLOBAL", "master-socket", 0)) == NULL) {
+			ha_alert("out of memory trying to allocate the mworker frontend");
 			goto error;
 		}
 	}
 
-	bind_conf = bind_conf_alloc(global.stats_fe, "master-socket", 0, "", xprt_get(XPRT_RAW));
+	bind_conf = bind_conf_alloc(mworker_fe, "master-socket", 0, "", xprt_get(XPRT_RAW));
 	if (!bind_conf)
 		goto error;
 
@@ -2570,14 +2570,14 @@ int mworker_cli_sockpair_new(struct mworker_proc *mworker_proc, int proc)
 	bind_conf->level |= ACCESS_LVL_ADMIN; /* TODO: need to lower the rights with a CLI keyword*/
 
 	bind_conf->bind_proc = 1UL << proc;
-	global.stats_fe->bind_proc = 0; /* XXX: we should be careful with that, it can be removed by configuration */
+	mworker_fe->bind_proc = 0;
 
 	if (!memprintf(, "sockpair@%d", mworker_proc->ipc_fd[1])) {
 		ha_alert("Cannot allocate listener.\n");
 		goto error;
 	}
 
-	if (!str2listener(path, global.stats_fe, bind_conf, "master-socket", 0, )) {
+	if (!str2listener(path, mworker_fe, bind_conf, "master-socket", 0, )) {
 		free(path);
 		ha_alert("Cannot create a CLI sockpair listener for process #%d\n", proc);
 		goto error;
@@ -2587,7 +2587,7 @@ int mworker_cli_sockpair_new(struct mworker_proc *mworker_proc, int proc)
 
 	list_for_each_entry(l, _conf->listeners, by_bind) {
 		l->accept = session_accept_fd;
-		l->default_target = global.stats_fe->default_target;
+		l->default_target = mworker_fe->default_target;
 		l->options |= (LI_O_UNLIMITED | LI_O_NOSTOP);
 		/* it's a sockpair but we don't want to keep the fd in the master */
 		l->options &= ~LI_O_INHERITED;
-- 
2.21.0



Re: `stats bind-process` broken

2019-04-12 Thread Christian Ruppert

Hey Guys,

I can confirm those issues as well as the proposed fix/workaround to 
solve the issue.
I upgraded our "nbproc" setup from 1.7.x to 1.9.6 today and noticed some 
missing entries from the stats socket, e.g.:

# echo 'show stat' | socat stdio /run/haproxy.stat|wc -l
1442

Which is correct, while after the upgrade it was indeed showing stats 
from a random proc:

# echo 'show stat' | socat stdio /run/haproxy.stat|wc -l
341

etc.

Adding the "process 1" to the "stats socket" line seems to help.

On 2019-04-11 18:24, Willy Tarreau wrote:

Hi Patrick,

On Thu, Apr 11, 2019 at 12:18:14PM -0400, Patrick Hemmer wrote:
With haproxy 1.9.6 the `stats bind-process` directive is not working. 
Every

connection to the socket is going to a random process:

Here's a simple reproduction:
Config:
   global
       nbproc 3
       stats socket /tmp/haproxy.sock level admin
       stats bind-process 1


Testing:
   # for i in {1..5}; do socat - unix:/tmp/haproxy.sock <<< "show 
info" |

grep Pid: ; done
   Pid: 33371
   Pid: 33373
   Pid: 33372
   Pid: 33373
   Pid: 33373


This must be pretty annoying. I don't have memories of anything changed
regarding the bind-process stuff between 1.8 and 1.9 (the threads have
moved a lot however). It could be a side effect of some of these 
changes

though.

However I'm seeing that adding "process 1" on the "stats socket" line
itself fixes the problem. I suspect the issue is located in the 
propagation

of the frontend's mask to the listener, I'll look at this.

Thanks!
Willy



-Patrick


--
Regards,
Christian Ruppert



Re: who controls https://github.com/haproxy/haproxy ?

2019-04-12 Thread Илья Шипицин
yep, I'm going to play in my own fork as well.
I will show something soon

пт, 12 апр. 2019 г. в 16:50, Tim Düsterhus :

> Ilya,
>
> Am 12.04.19 um 10:54 schrieb Илья Шипицин:
> > I wish to enable travis-ci, cirrus-ci builds for github repo
> > I do not see any public member there.
> >
> > who does control it ?
> >
>
> Willy does. And FWIW I've played around with Travis CI in a GitHub fork
> of mine: https://github.com/TimWolla/haproxy/pull/1
>
> Best regards
> Tim Düsterhus
>


Re: who controls https://github.com/haproxy/haproxy ?

2019-04-12 Thread Tim Düsterhus
Ilya,

Am 12.04.19 um 10:54 schrieb Илья Шипицин:
> I wish to enable travis-ci, cirrus-ci builds for github repo
> I do not see any public member there.
> 
> who does control it ?
> 

Willy does. And FWIW I've played around with Travis CI in a GitHub fork
of mine: https://github.com/TimWolla/haproxy/pull/1

Best regards
Tim Düsterhus



Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Olivier Houchard
Hi,

On Fri, Apr 12, 2019 at 08:37:10AM +0200, Maciej Zdeb wrote:
> Hi Richard,
> 
> Those patches from Olivier (in streams) are related to my report from
> thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
> out it wasn't a single bug and issue is not entirely fixed yet.
> 
> Currently I'm testing some additional patches from Olivier which hopefully
> fix the issue definitely.
> 

Indeed, the rmoeval of SI_FL_ERR in si_update_both() was bogus, and covered
misuses of it.
With the great help of Maciej, we investigated this, and I just pushed what
we fixed so far. Richard, I'd be really interested in knowing if you still
have issues with the latest master.

Thanks !

Olivier

> pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):
> 
> > It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> > flag, process_stream can keep going back to redo around stream.c:line 2503:
> >
> > if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> > si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> > ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> > (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> > CF_MASK_ANALYSER))
> >  goto redo;
> >
> > Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> > else does, the goto will get called forever. I don't understand this
> > section enough to try to reproduce this, but I found several processes
> > stuck here on a machine testing from yesterday's HEAD.
> >
> > Richard
> >
> > --
> >   Richard Russo
> >   to...@enslaves.us
> >
> >



who controls https://github.com/haproxy/haproxy ?

2019-04-12 Thread Илья Шипицин
hello,

I wish to enable travis-ci, cirrus-ci builds for github repo
I do not see any public member there.

who does control it ?

thanks!


Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Maciej Zdeb
Hi Richard,

Those patches from Olivier (in streams) are related to my report from
thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
out it wasn't a single bug and issue is not entirely fixed yet.

Currently I'm testing some additional patches from Olivier which hopefully
fix the issue definitely.

pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):

> It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> flag, process_stream can keep going back to redo around stream.c:line 2503:
>
> if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> CF_MASK_ANALYSER))
>  goto redo;
>
> Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> else does, the goto will get called forever. I don't understand this
> section enough to try to reproduce this, but I found several processes
> stuck here on a machine testing from yesterday's HEAD.
>
> Richard
>
> --
>   Richard Russo
>   to...@enslaves.us
>
>