Re: PROXY protocol and check port
Hello, Le mar. 17 déc. 2019 à 11:11, Willy Tarreau a écrit : > Hi Olivier, > > On Tue, Dec 17, 2019 at 09:20:21AM +0100, Olivier D wrote: > > That's not what I was saying. I'm already using "show server state", and > > that's exactly what leads me to hours of debugging : between two versions > > of my haproxy config file, I changed backend server port from 80 to 443. > > When HAProxy reloaded, it loaded server state file and loaded both the > > previous state(up/down) of the server, but also the server port. That's > why > > HAProxy never used port 443 as backend port and was still using old port > 80. > > This is not clearly stated in the configuration (or maybe I missed it ?), > > that's why I was asking whether it is an expected behaviour. In my mind, > > the server state was only used to get server status between two reload, > not > > more informations. > > Bah, state-file strikes again :-( > > We predicted exactly this when it was designed. Some people complain that > their changes are lost and others that their changes are kept. We discussed > this already with Baptiste and we figured that the only solution to this is > to change the format to include, for each and every field, *both* the > config > config state and the current state, so that upon reload, the state file > parser can detect what changed from previous running instance: if the > config > changed, you want the config change to be taken into account. If the config > didn't change, then very likely you want to keep the previous state. > > It's a really tricky thing because most of the time a reload is made in > order to apply a change, but this change may conflict with what is in the > state file in a way that's not that determinist :-/ Personally I don't like > the idea of having two authoritative sources for a single configuration, > which is why we must make this state file processing stricter and its > contents more detailed. > > It's worth noting that some people are already abusing it by writing > their changes directly to the state file and avoid touching the config > (typically just have a server-template directive in the config and all > the details in the state file). This shows how tricky the situation can > become every time we try to address one's needs and break someonee else's > :-/ > > Willy > Ah! thank you for the explanation ! Just explaining this in doc should be enough. I always thought that only the state (UP/DOWN/MAINT) was loaded from that file. What would you think of the following modification ? server-state-file Specifies the path to the file containing state of servers. [...] Please note that when we say "state of servers", it means not only server status (up/down/maint) but also IP/port. [...] if "state-only" is present, then only server state (up/down/maint) will be reloaded. IP/port will be filled with what is provided in config file. Well, like this it's quite ugly but you see my point :) It's backward-compatible, and you can enforce the behaviour you expect. Olivier
Re: PROXY protocol and check port
Hi Olivier, On Tue, Dec 17, 2019 at 09:20:21AM +0100, Olivier D wrote: > That's not what I was saying. I'm already using "show server state", and > that's exactly what leads me to hours of debugging : between two versions > of my haproxy config file, I changed backend server port from 80 to 443. > When HAProxy reloaded, it loaded server state file and loaded both the > previous state(up/down) of the server, but also the server port. That's why > HAProxy never used port 443 as backend port and was still using old port 80. > This is not clearly stated in the configuration (or maybe I missed it ?), > that's why I was asking whether it is an expected behaviour. In my mind, > the server state was only used to get server status between two reload, not > more informations. Bah, state-file strikes again :-( We predicted exactly this when it was designed. Some people complain that their changes are lost and others that their changes are kept. We discussed this already with Baptiste and we figured that the only solution to this is to change the format to include, for each and every field, *both* the config config state and the current state, so that upon reload, the state file parser can detect what changed from previous running instance: if the config changed, you want the config change to be taken into account. If the config didn't change, then very likely you want to keep the previous state. It's a really tricky thing because most of the time a reload is made in order to apply a change, but this change may conflict with what is in the state file in a way that's not that determinist :-/ Personally I don't like the idea of having two authoritative sources for a single configuration, which is why we must make this state file processing stricter and its contents more detailed. It's worth noting that some people are already abusing it by writing their changes directly to the state file and avoid touching the config (typically just have a server-template directive in the config and all the details in the state file). This shows how tricky the situation can become every time we try to address one's needs and break someonee else's :-/ Willy
Re: PROXY protocol and check port
Hi Olivier, On Tue, Dec 17, 2019 at 7:20 PM Olivier D wrote: > Hello Igor, > > > Le lun. 16 déc. 2019 à 23:41, Igor Cicimov > a écrit : > >> Hi, >> >> On Tue, Dec 17, 2019 at 2:55 AM Olivier D wrote: >> >>> Hello, >>> >>> I found what was wrong : I was using "load-server-state-from-file" and >>> previous config file was using port 80 as server port. >>> It seems using this instruction loads previous server state but also >>> previous srv_port. >>> Is this an expected behaviour ? >>> >> >> Yes, basically it is your responsibility to dump the current state into >> the file otherwise you'll gate outdated data as you noticed. For example I >> add: >> >> ExecReload=/bin/echo "show servers state" | /usr/bin/socat stdio >> /run/haproxy/admin.sock > /var/lib/haproxy/state >> >> in the systemd service file. >> > > That's not what I was saying. I'm already using "show server state", and > that's exactly what leads me to hours of debugging : between two versions > of my haproxy config file, I changed backend server port from 80 to 443. > When HAProxy reloaded, it loaded server state file and loaded both the > previous state(up/down) of the server, but also the server port. That's why > HAProxy never used port 443 as backend port and was still using old port 80. > Ah I get ya, yeah have not had a case like that yet but it is strange. Probably should wait for someone with knowledge of the code to answer your question :-/ > This is not clearly stated in the configuration (or maybe I missed it ?), > that's why I was asking whether it is an expected behaviour. In my mind, > the server state was only used to get server status between two reload, not > more informations. > > Olivier > >
Re: PROXY protocol and check port
Hello Igor, Le lun. 16 déc. 2019 à 23:41, Igor Cicimov a écrit : > Hi, > > On Tue, Dec 17, 2019 at 2:55 AM Olivier D wrote: > >> Hello, >> >> I found what was wrong : I was using "load-server-state-from-file" and >> previous config file was using port 80 as server port. >> It seems using this instruction loads previous server state but also >> previous srv_port. >> Is this an expected behaviour ? >> > > Yes, basically it is your responsibility to dump the current state into > the file otherwise you'll gate outdated data as you noticed. For example I > add: > > ExecReload=/bin/echo "show servers state" | /usr/bin/socat stdio > /run/haproxy/admin.sock > /var/lib/haproxy/state > > in the systemd service file. > That's not what I was saying. I'm already using "show server state", and that's exactly what leads me to hours of debugging : between two versions of my haproxy config file, I changed backend server port from 80 to 443. When HAProxy reloaded, it loaded server state file and loaded both the previous state(up/down) of the server, but also the server port. That's why HAProxy never used port 443 as backend port and was still using old port 80. This is not clearly stated in the configuration (or maybe I missed it ?), that's why I was asking whether it is an expected behaviour. In my mind, the server state was only used to get server status between two reload, not more informations. Olivier
Re: PROXY protocol and check port
Hi, On Tue, Dec 17, 2019 at 2:55 AM Olivier D wrote: > Hello, > > I found what was wrong : I was using "load-server-state-from-file" and > previous config file was using port 80 as server port. > It seems using this instruction loads previous server state but also > previous srv_port. > Is this an expected behaviour ? > Yes, basically it is your responsibility to dump the current state into the file otherwise you'll gate outdated data as you noticed. For example I add: ExecReload=/bin/echo "show servers state" | /usr/bin/socat stdio /run/haproxy/admin.sock > /var/lib/haproxy/state in the systemd service file.
Re: PROXY protocol and check port
Hello, I found what was wrong : I was using "load-server-state-from-file" and previous config file was using port 80 as server port. It seems using this instruction loads previous server state but also previous srv_port. Is this an expected behaviour ? Olivier Le ven. 13 déc. 2019 à 18:32, Olivier D a écrit : > Hello all, > I struggle with what seemed a very easy config : > > listen test:443 > id 20609 > bind-process 16 > balance source > hash-type consistent > mode tcp > bind x.x.x.x:443 > server s1 192.168.x.x:443 id 2158 check weight 5 send-proxy port 80 > server s2 192.168.x.x:443 id 2168 check weight 5 send-proxy port 80 > > I expect traffic to be routed to port 443 of backend server, and checks to > be performed without proxy-protocol on port 80. tests seem ok, but > traffic seems also routed to port 80 (I checked with tcpdump) instead of > 443. > > Can you at least confirm that my expectations are correct ? Or am I just > very tired on a friday evening ? > > I'm using HAProxy 1.9.12 > > Thank you ! >
PROXY protocol and check port
Hello all, I struggle with what seemed a very easy config : listen test:443 id 20609 bind-process 16 balance source hash-type consistent mode tcp bind x.x.x.x:443 server s1 192.168.x.x:443 id 2158 check weight 5 send-proxy port 80 server s2 192.168.x.x:443 id 2168 check weight 5 send-proxy port 80 I expect traffic to be routed to port 443 of backend server, and checks to be performed without proxy-protocol on port 80. tests seem ok, but traffic seems also routed to port 80 (I checked with tcpdump) instead of 443. Can you at least confirm that my expectations are correct ? Or am I just very tired on a friday evening ? I'm using HAProxy 1.9.12 Thank you !
