Re: HAProxy doesn't respect the `hold valid 1s` setting
Can any help to fix this bug? I reported here long time ago. On 14 August 2016 at 15:06, Tao Wang <twang2...@gmail.com> wrote: > Hi, > > As the issue reported on Github haproxy repo are disabled, I transfer > the issue here. > > I created a Github repo for this issue, > https://github.com/twang2218/haproxy-dns-issue > > All the information are there. Here is the issue description. > > # The issue > > The HAProxy doesn't respect the `hold valid 1s` setting, so after the > `docker-compose scale app=10`, the reverse proxy will not redirect the > traffic to the other `app` service, only to the original resolved IP > address. [haproxy/haproxy#74](https://github.com/haproxy/haproxy/issues/74) > > # Setup > > Here is a Docker Compose project to reproduce the issue. > > 3 services are defined in the `docker-compose.yml`. > > ```yaml > version: '2' > services: > haproxy: > image: haproxy:latest > volumes: > - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg > ports: > - "80:80" > depends_on: > - app > - syslog > app: > image: node:latest > volumes: > - ./index.js:/app/index.js > command: node /app/index.js > syslog: > image: bobrik/syslog-ng:latest > volumes: > - ./log:/var/log/syslog-ng > ``` > > `haproxy` is an HAProxy server as a reverse proxy server. The > `haproxy.cfg` is the following: > > ```python > global > log syslog daemon > > defaults > timeout connect 5000ms > timeout client 5ms > timeout server 5ms > > resolvers docker_dns > nameserver dns "127.0.0.11:53" > timeout retry 1s > hold valid 1s > > listen tcp_proxy > mode tcp > bind :80 > option tcplog > log global > server app app:8000 check resolvers docker_dns resolve-prefer ipv4 > ``` > > `app` is just a simple node.js web service which return the client IP > and the IP and hostname of current, so it will be easy to know which > server served the request. > > # Reproduce the problem > > Start up the stack. > > ```bash > $ docker-compose up -d > Creating network "haproxyissue74_default" with the default driver > Creating haproxyissue74_app_1 > Creating haproxyissue74_syslog_1 > Creating haproxyissue74_haproxy_1 > ``` > > Scale the `app` to `10`. > > ```bash > $ docker-compose scale app=10 > Creating and starting haproxyissue74_app_2 ... done > Creating and starting haproxyissue74_app_3 ... done > Creating and starting haproxyissue74_app_4 ... done > Creating and starting haproxyissue74_app_5 ... done > Creating and starting haproxyissue74_app_6 ... done > Creating and starting haproxyissue74_app_7 ... done > Creating and starting haproxyissue74_app_8 ... done > Creating and starting haproxyissue74_app_9 ... done > Creating and starting haproxyissue74_app_10 ... done > ``` > > And check load balance result. > > ```bash > $ curl http://localhost/ > :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% > $ curl http://localhost/ > :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% > $ curl http://localhost/ > :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% > $ curl http://localhost/ > :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% > $ > ``` > > If the `hold valid 1s` works, HAProxy should resolve the `app` ip > address in different, however, it looks like doesn't work. > > To verify the DNS resolution's randomness, we can query the `app` DNS > within `haproxy` container. > > Enter the container > > ```bash > $ docker-compose exec haproxy bash > root@a2243d2c5e7c:/# > ``` > > Install `dnsutils` for `nslookup` and `dig` command. > > ```bash > root@a2243d2c5e7c:/# apt-get update && apt-get install -y dnsutils > Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB] > Get:2 http://security.debian.org jessie/updates/main amd64 Packages [366 kB] > ... > Setting up rename (0.20-3) ... > update-alternatives: using /usr/bin/file-rename to provide > /usr/bin/rename (rename) in auto mode > Setting up xml-core (0.13+nmu2) ... > Processing triggers for libc-bin (2.19-18+deb8u4) ... > Processing triggers for sgml-base (1.26+nmu4) ... > root@a2243d2c5e7c:/# > ``` > > And then, we can use `dig` command to check the DNS query result. > > ```bash > root@a2243d2c5e7c:/# dig app | grep app > ; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> app > ;app. IN A > app. 600 IN A 172.30.0.5 > app. 600 IN A 172.30.0.9 > app. 600 IN A 172.30.0.7 > app
HAProxy doesn't respect the `hold valid 1s` setting
Hi, As the issue reported on Github haproxy repo are disabled, I transfer the issue here. I created a Github repo for this issue, https://github.com/twang2218/haproxy-dns-issue All the information are there. Here is the issue description. # The issue The HAProxy doesn't respect the `hold valid 1s` setting, so after the `docker-compose scale app=10`, the reverse proxy will not redirect the traffic to the other `app` service, only to the original resolved IP address. [haproxy/haproxy#74](https://github.com/haproxy/haproxy/issues/74) # Setup Here is a Docker Compose project to reproduce the issue. 3 services are defined in the `docker-compose.yml`. ```yaml version: '2' services: haproxy: image: haproxy:latest volumes: - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg ports: - "80:80" depends_on: - app - syslog app: image: node:latest volumes: - ./index.js:/app/index.js command: node /app/index.js syslog: image: bobrik/syslog-ng:latest volumes: - ./log:/var/log/syslog-ng ``` `haproxy` is an HAProxy server as a reverse proxy server. The `haproxy.cfg` is the following: ```python global log syslog daemon defaults timeout connect 5000ms timeout client 5ms timeout server 5ms resolvers docker_dns nameserver dns "127.0.0.11:53" timeout retry 1s hold valid 1s listen tcp_proxy mode tcp bind :80 option tcplog log global server app app:8000 check resolvers docker_dns resolve-prefer ipv4 ``` `app` is just a simple node.js web service which return the client IP and the IP and hostname of current, so it will be easy to know which server served the request. # Reproduce the problem Start up the stack. ```bash $ docker-compose up -d Creating network "haproxyissue74_default" with the default driver Creating haproxyissue74_app_1 Creating haproxyissue74_syslog_1 Creating haproxyissue74_haproxy_1 ``` Scale the `app` to `10`. ```bash $ docker-compose scale app=10 Creating and starting haproxyissue74_app_2 ... done Creating and starting haproxyissue74_app_3 ... done Creating and starting haproxyissue74_app_4 ... done Creating and starting haproxyissue74_app_5 ... done Creating and starting haproxyissue74_app_6 ... done Creating and starting haproxyissue74_app_7 ... done Creating and starting haproxyissue74_app_8 ... done Creating and starting haproxyissue74_app_9 ... done Creating and starting haproxyissue74_app_10 ... done ``` And check load balance result. ```bash $ curl http://localhost/ :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% $ curl http://localhost/ :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% $ curl http://localhost/ :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% $ curl http://localhost/ :::172.30.0.4 → 7d10f99e3d60 @ [172.30.0.2]% $ ``` If the `hold valid 1s` works, HAProxy should resolve the `app` ip address in different, however, it looks like doesn't work. To verify the DNS resolution's randomness, we can query the `app` DNS within `haproxy` container. Enter the container ```bash $ docker-compose exec haproxy bash root@a2243d2c5e7c:/# ``` Install `dnsutils` for `nslookup` and `dig` command. ```bash root@a2243d2c5e7c:/# apt-get update && apt-get install -y dnsutils Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB] Get:2 http://security.debian.org jessie/updates/main amd64 Packages [366 kB] ... Setting up rename (0.20-3) ... update-alternatives: using /usr/bin/file-rename to provide /usr/bin/rename (rename) in auto mode Setting up xml-core (0.13+nmu2) ... Processing triggers for libc-bin (2.19-18+deb8u4) ... Processing triggers for sgml-base (1.26+nmu4) ... root@a2243d2c5e7c:/# ``` And then, we can use `dig` command to check the DNS query result. ```bash root@a2243d2c5e7c:/# dig app | grep app ; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> app ;app. IN A app. 600 IN A 172.30.0.5 app. 600 IN A 172.30.0.9 app. 600 IN A 172.30.0.7 app. 600 IN A 172.30.0.12 app. 600 IN A 172.30.0.2 app. 600 IN A 172.30.0.11 app. 600 IN A 172.30.0.8 app. 600 IN A 172.30.0.10 app. 600 IN A 172.30.0.13 app. 600 IN A 172.30.0.6 root@a2243d2c5e7c:/# dig app | grep app ; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> app ;app. IN A app. 600 IN A 172.30.0.6 app. 600 IN A 172.30.0.10 app. 600 IN A 172.30.0.13 app. 600 IN A 172.30.0.11 app. 600 IN A 172.30.0.7 app. 600 IN A 172.30.0.2 app. 600 IN A 172.30.0.5 app. 600 IN A 172.30.0.9 app. 600 IN A 172.30.0.12 app. 600 IN A 172.30.0.8 root@a2243d2c5e7c:/# dig app | grep app ; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> app ;app. IN A app. 600 IN A 172.30.0.13 app. 600 IN A 172.30.0.11 app. 600 IN A 172.30.0.8 app. 600 IN A 172.30.0.12 app. 600 IN A 172.30.0.5 app. 600 IN A 172.30.0.6 app. 600 IN A 172.30.0.7 app. 600 IN A 172.30.0.9 app. 600 IN A 172.30.0.2 app. 600 IN A 172.30.0.10 root@a2243d2c5e7c:/# ``` As shown above, DNS will return the records in
