Re: Bug when loading multiple configuration files

[Ben Cabot]

Hi Willy, Bryan,
Thanks for looking at this and getting it fixed quickly.

Thanks,
Ben

On 26 May 2016 at 17:01, Willy Tarreau <w...@1wt.eu> wrote:
> Hi Ben,
>
> On Wed, May 25, 2016 at 08:41:53AM +0100, Ben Cabot wrote:
>> Sorry I forgot include the build details. The configuration its self
>> does not seem to matter, you get the error if you if you load 2 empty
>> files or 2 with any listen or frontend / backend configurations. Its
>> just the fact you are loading 2 configuration files that causes the
>> problem.
>
> Thanks for reporting this. In fact it's interesting because this cleanup
> patch has uncovered a real bug. Look at readcfgfile() in cfgparse.c, the
> parsers are registered for each file. It just had the effect of wasting
> memory and slightly slowing down the config parser as the number of files
> increased, but now it fails. One more reason to keep it, and maybe even
> to backport it in the end.
>
> I've merged the attached patch to fix it.
>
> Thanks,
> Willy



-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org

Re: Bug when loading multiple configuration files

[Ben Cabot]

Sorry I forgot include the build details. The configuration its self
does not seem to matter, you get the error if you if you load 2 empty
files or 2 with any listen or frontend / backend configurations. Its
just the fact you are loading 2 configuration files that causes the
problem.

HA-Proxy version 1.7-dev3-1416746-24 2016/05/20
Copyright 2000-2016 Willy Tarreau <wi...@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -m64 -march=x86-64 -O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement
  OPTIONS = USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_STATIC_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.1t  3 May 2016
Running on OpenSSL version : OpenSSL 1.0.1t  3 May 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[TRACE] trace
[COMP] compression


Ben

On 24 May 2016 at 23:59, Bryan Talbot <bryan.tal...@playnext.com> wrote:
> The OP didn’t provide many details, but I am able to reproduce this too using 
> 1.7-dev and the config files shown below. Git bisect shows the break at the 
> commit mentioned.
>
>
> $> cat haproxy.cfg haproxy2.cfg
> global
>
> defaults
> timeout client 5s
> timeout server 5s
> timeout connect 5s
> mode http
>
> listen www
> bind :8000
>
>
> listen www2
> bind :8001
>
>
> $> cat git-bisect-run.sh
> #!/bin/bash -e
> make clean
> make TARGET=generic USE_OPENSSL=1 ADDLIB=-lcrypto 
> SSL_INC=/usr/local/opt/openssl/include SSL_LIB=/usr/local/opt/openssl/lib 
> USE_ZLIB=1 USE_PCRE=1 -j4
> ./haproxy -c -f ./haproxy.cfg -f ./haproxy2.cfg || exit 1
> ./haproxy -vv
>
>
>
>
>
>> On May 24, 2016, at May 24, 4:50 AM, Ben Cabot <b...@loadbalancer.org> wrote:
>>
>> Hi all,
>> I think we have found an issue when using multiple configuration
>> files. The config parser tries to register the listen section twice
>> causing the error below.
>>
>> [root@lbmaster haproxy]# /usr/local/sbin/haproxy -f
>> /etc/haproxy/haproxy.cfg -f /etc/haproxy/haproxy_manual.cfg
>> [ALERT] 144/113841 (10937) : register section 'listen': already registered.
>> [ALERT] 144/113841 (10937) : Could not open configuration file
>> /etc/haproxy/haproxy_manual.cfg : Success
>>
>>
>> It looks to be introduced in 5e4261b0 but I'm unsure how to fix it.
>> Please can someone take a look.
>>
>> Thanks,
>>
>> Ben
>>
>



-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org

Re: Howto masquerade real server in a two armed transparent setup

[Ben Cabot]

Hi Steffen,
If I understand the question right to give your real servers internet
access you should be able to enable ip forwarding then use a masquerade
rule.

You can use  "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE "  then
"echo “1” > /proc/sys/net/ipv4/ip_forward" .


Ben


On 12 November 2015 at 06:54, HAproxy  wrote:

> Trying to make a two armed transparent setup like mentioned here
>  to do both
> transparently L4 [SSL] DR + L7 SSL termination load balanced services.
>
> I’m having the load balanced services working, but I wont the real servers
> to be able to access the public internet and to have other non-balanced
> services like management ssh access through HAproxy VMs to my real servers.
>
> Question is now how do I possible masquerade the real servers to access
> public internet through my active/passiveHAproxies, currently setup like
> mentioned here . Any hints/URLs are
> welcomed?
>
> TIA
>
> /Steffen
>



-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org

Re: [PATCH] BUG: config: external-check command validation is checking for incorrect arguments.

[Ben Cabot]

Sorry I'd not noticed that. Everything looks good now thank you Willy.

Ben


On 4 October 2015 at 09:41, Willy Tarreau  wrote:

> Resending after some mails were lost du to low space on device...
>
> On Fri, Oct 02, 2015 at 11:15:35PM +0200, Willy Tarreau wrote:
> > On Fri, Oct 02, 2015 at 03:56:58PM +0200, Igor Wiedler wrote:
> > > Hello,
> > >
> > > I wanted to test the external-check option in 1.6 (master) and it
> seems like
> > > the validation logic is broken. I was wondering what the status of
> this patch
> > > is: http://marc.info/?l=haproxy=144240175729490=2
> > > . Can we get it
> merged?
> >
> > Just done, thanks for bringing it up, I indeed missed it!
> >
> > Ben, your mailer mangled the patch, I had to redo it by hand, it looks
> > OK but I would appreciate it if you could double-check.
> >
> > Thanks!
> > Willy
> >
>



-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org

[PATCH] BUG: config: external-check command validation is checking for incorrect arguments.

[Ben Cabot]

When using the external-check command option HAProxy was failing to
start with a fatal error "'external-check' cannot handle unexpected
argument". When looking at the code it was looking for an incorrect
argument. Also correcting an Alert message text as spotted by by
PiBa-NL.


diff --git a/src/cfgparse.c b/src/cfgparse.c
index 71d3fea..5d72240 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -3350,9 +3350,9 @@ int cfg_parse_listen(const char *file, int
linenum, char **args, int kwm)
 }

if (!strcmp(args[1], "command")) {
-   if (alertif_too_many_args(1, file, linenum,
args, _code))
+   if (alertif_too_many_args(2, file, linenum,
args, _code))
goto out;
-   if (*(args[1]) == 0) {
+   if (*(args[2]) == 0) {
Alert("parsing [%s:%d] : missing
argument after '%s'.\n",
  file, linenum, args[1]);
err_code |= ERR_ALERT | ERR_FATAL;
@@ -3362,9 +3362,9 @@ int cfg_parse_listen(const char *file, int
linenum, char **args, int kwm)
curproxy->check_command = strdup(args[2]);
}
else if (!strcmp(args[1], "path")) {
-   if (alertif_too_many_args(1, file, linenum,
args, _code))
+   if (alertif_too_many_args(2, file, linenum,
args, _code))
goto out;
-   if (*(args[1]) == 0) {
+   if (*(args[2]) == 0) {
Alert("parsing [%s:%d] : missing
argument after '%s'.\n",
  file, linenum, args[1]);
err_code |= ERR_ALERT | ERR_FATAL;
@@ -7330,7 +7330,7 @@ int check_config_validity()
clear = 1;
}
if (curproxy->check_command[0] != '/' &&
!curproxy->check_path) {
-   Alert("Proxy '%s': '%s' does not have
a leading '/' and 'external-command path' is not set.\n",
+   Alert("Proxy '%s': '%s' does not have
a leading '/' and 'external-check path' is not set.\n",
  curproxy->id, "external-check command");
cfgerr++;
}

Thanks,

Ben


-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org

Re: External-check command problem

[Ben Cabot]

Hi All,
Has anyone been able to confirm this bug and the patch? I wanted to
confirm I had not missed any potential issues elsewhere?

Thank you,

Ben

On 4 September 2015 at 13:50, Ben Cabot <b...@loadbalancer.org> wrote:
> We have been seeing the following error when trying to use the
> external-check command option.
>
> [ALERT] 246/120705 (25012) : parsing [/etc/haproxy/haproxy.cfg:31] :
> 'external-check' cannot handle unexpected argument
> '/var/lib/loadbalancer.org/check/http_ntlm_proxy_check.sh'.
> [ALERT] 246/120705 (25012) : Error(s) found in configuration file :
> /etc/haproxy/haproxy.cfg
> [ALERT] 246/120705 (25012) : Proxy 'VIP_Name' : 'option
> external-check' unable to find required 'external-check command'.
> [ALERT] 246/120705 (25012) : Fatal errors found in configuration.
>
> Also the same "cannot handle unexpected argument" error occurs for
> "external-check path".
>
> I have tried with the latest build haproxy-ss-20150904 and can
> reproduce the problem. I believe it is because it is checking for an
> invalid number of arguments but may be completely wrong.
>
> Here is my suggested fix.
>
> --- src/cfgparse.org 2015-09-04 13:26:36.070579303 +0100
> +++ src/cfgparse.c 2015-09-04 13:36:31.194587549 +0100
> @@ -3349,9 +3349,9 @@
>  }
>
>   if (!strcmp(args[1], "command")) {
> - if (alertif_too_many_args(1, file, linenum, args, _code))
> + if (alertif_too_many_args(2, file, linenum, args, _code))
>   goto out;
> - if (*(args[1]) == 0) {
> + if (*(args[2]) == 0) {
>   Alert("parsing [%s:%d] : missing argument after '%s'.\n",
>file, linenum, args[1]);
>   err_code |= ERR_ALERT | ERR_FATAL;
> @@ -3361,9 +3361,9 @@
>   curproxy->check_command = strdup(args[2]);
>   }
>   else if (!strcmp(args[1], "path")) {
> - if (alertif_too_many_args(1, file, linenum, args, _code))
> + if (alertif_too_many_args(2, file, linenum, args, _code))
>   goto out;
> - if (*(args[1]) == 0) {
> + if (*(args[2]) == 0) {
>   Alert("parsing [%s:%d] : missing argument after '%s'.\n",
>file, linenum, args[1]);
>   err_code |= ERR_ALERT | ERR_FATAL;
>
>
>
> Test config:
> global
> daemon
> stats socket /var/run/haproxy.stat mode 600 level admin
> pidfile /var/run/haproxy.pid
> log /dev/log local4
> maxconn 4
> tune.bufsize 16384
> tune.maxrewrite 1024
> external-check
>
> defaults
> mode http
> balance roundrobin
> timeout connect 4000
> timeout client 42000
> timeout server 43000
> log global
>
> peers loadbalancer_replication
> peer lbmaster localhost:7778
> peer lbslave localhost:7778
>
> listen VIP_Name
> bind 192.168.63.122:80 transparent
> mode http
> balance leastconn
> cookie SERVERID insert nocache indirect
> server backup 127.0.0.1:9081 backup  non-stick
> option external-check
> external-check command
> /var/lib/loadbalancer.org/check/http_ntlm_proxy_check.sh
> option accept-invalid-http-request
> option http-keep-alive
> option forwardfor
> option redispatch
> option abortonclose
> maxconn 4
> option httplog
> server RIP_Name 192.168.63.100  weight 100  cookie RIP_Name
> check  inter 4000  rise 2  fall 2  minconn 0  maxconn 0
> on-marked-down shutdown-sessions
>
> thanks,
>
> Ben
>
> --
> LOADBALANCER.ORG LTD.
> www.loadbalancer.org
> supp...@loadbalancer.org



-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org

Re: email alerts

[Ben Cabot]

Hello Andrey,

As Maik has previously mentioned that patch will only work on the older
version. We have someone working on an updated patch but cannot supply an
ETA at the moment as there are a couple of other things in the queue before
it.



On 24 October 2014 12:59, Andrey Zakabluk a.zakab...@velcom.by wrote:

 Hi!
 I use HA-Proxy version 1.5.6 2014/10/18/ . Red hat 6.

 I want use alert via mail. I find informations about this:
 ---
 Patch HAProxy
 So this brings us on to option 3 patch HAProxy to send the alerts, after
 all how hard can it be?
 As I don’t really want to write my own SMTP client or use any other
 library’s lets go with the easy option of using mailx from the mailutils
 package as we know it works. The following was written for HAProxy dev18.
 Now I’m no developer so take the code more of a proof of concept instead of
 something to add to your production environment.

 Most of the work is already done for us, as HAProxy has functions for
 setting a server up or down and also has an array containing the server
 name, server’s status etc. So all we need to do is add our own function to
 send the email and parse the email address from the configuration file.

 This done in the following patch files:
 cfgparse.c
 checks.c
 log.c
 global.h

 So in the configuration file I have added the option “email_alert” to the
 global section with to and from address.

 http://blog.loadbalancer.org/3-ways-to-send-haproxy-health-check-email-alerts/
 -

 I try add text from patch file in the end original file and make haproxy
 file.
 /opt/haproxy/global.h.diff  /opt/haproxy/include/types/global.h

 cat /opt/haproxy/cfgparse.diff /opt/haproxy/haproxy-1.5.6/src/cfgparse.c

 cat /opt/haproxy/log.diff /opt/haproxy/haproxy-1.5.6/src/log.c

 cat /opt/haproxy/checks.diff /opt/haproxy/haproxy-1.5.6/src/checks.c


 And while compiling I have error:

 [webadmin@srv-lb-qa haproxy-1.5.6_with_diff]$ make TARGET=linux2628
 CPU=generic
 gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing
  -DCONFIG_HAP_LINUX_SPLICE -DTPROXY -DCONFIG_HAP_LINUX_TPROXY
 -DCONFIG_HAP_CRYPT -DENABLE_POLL -DENABLE_EPOLL -DUSE_CPU_AFFINITY
 -DASSUME_SPLICE_WORKS -DUSE_ACCEPT4 -DNETFILTER -DUSE_GETSOCKNAME
 -DCONFIG_HAPROXY_VERSION=\1.5.6\ -DCONFIG_HAPROXY_DATE=\2014/10/18\ \
   -DBUILD_TARGET='linux2628' \
   -DBUILD_ARCH='' \
   -DBUILD_CPU='generic' \
   -DBUILD_CC='gcc' \
   -DBUILD_CFLAGS='-O2 -g -fno-strict-aliasing' \
   -DBUILD_OPTIONS='' \
-c -o src/haproxy.o src/haproxy.c
 In file included from include/proto/proxy.h:28,
  from include/common/cfgparse.h:30,
  from src/haproxy.c:61:
 include/types/global.h:207: error: expected identifier or ‘(’ before ‘--’
 token
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:213: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:215: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:216: error: expected identifier or ‘(’ before ‘+’
 token
 In file included from src/haproxy.c:76:
 include/types/global.h:207: error: expected identifier or ‘(’ before ‘--’
 token
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:213: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:215: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:216: error: expected identifier or ‘(’ before ‘+’
 token
 In file included from include/proto/channel.h:35,
  from src/haproxy.c:85:
 include/types/global.h:207: error: expected identifier or ‘(’ before ‘--’
 token
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:209: error: stray ‘@’ in program
 include/types/global.h:213: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:215: error: expected identifier or ‘(’ before ‘+’
 token
 include/types/global.h:216: error: expected identifier or ‘(’ before ‘+’
 token
 make: *** [src/haproxy.o] Error 1



 Pls help me! I don't know how use patch(




-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org

Email alerts

[Ben Cabot]

Hi All,

I've been looking into sending email alerts from HAProxy and have written a
short blog on what I found with a few a few simple examples :
http://blog.loadbalancer.org/3-ways-to-send-haproxy-health-check-email-alerts/

If anyone has any thoughts ,suggestions or improvements please let me know.

Thanks,
-- 
LOADBALANCER.ORG LTD.
www.loadbalancer.org
supp...@loadbalancer.org