Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
Lukasz Michalski lm@... writes: I setup two haproxy instances - one in tcp mode for protocol detection and the second one for routing http requests application servers. Works like a charm on my development machine. Thanks again! Łukasz Dear Lukasz, Could you maybe post the config files of your two haproxy instances? Thanks!
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
I have got Flash talking to CouchDB. I think anyone with such setup will hit the issue with policy-file-request/ being sent by Flash 9 to port 843, and then retrying on the port it's trying to connect to (for heavily fire-walled environments the only options are ports 80 and 443 really). I had some luck with following haproxy config (replace myip): global log 127.0.0.1 local0 log 127.0.0.1 local1 notice #log loghostlocal0 info maxconn 4096 #chroot /usr/share/haproxy user haproxy group haproxy daemon #debug #quiet defaults log global modetcp option tcplog option dontlognull retries 3 option redispatch maxconn 2000 contimeout 5000 clitimeout 5 srvtimeout 5 frontend couchdb-in bind myip:80 tcp-request inspect-delay 5s acl traffic_is_http req_proto_http tcp-request content accept if traffic_is_http use_backend flash-socket-policy if !traffic_is_http default_backend couchdb backend couchdb server couchdb5984 127.0.0.1:5984 maxconn 32 frontend couchdb-ssl-in bind myip:443 tcp-request inspect-delay 5s acl traffic_is_ssl req_ssl_ver gt 0 tcp-request content accept if traffic_is_ssl use_backend flash-socket-policy if !traffic_is_ssl default_backend couchdb-ssl backend couchdb-ssl mode tcp server couchdb6984 127.0.0.1:6984 maxconn 32 backend flash-socket-policy mode tcp server policy843 127.0.0.1:843 maxconn 32
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/13/2011 07:59 AM, Willy Tarreau wrote: I think that have to setup some tcp content analyzer that will be able to read first 15 bytes from traffic and then route connection to stunnel or haproxy only if this is not a policy file request. Yes, then you can proceed that way : - have haproxy in TCP mode in front of stunnel so that it detects either a valid and complete SSL message, or at least 22 bytes of non-SSL traffic (the XML request) ; - make it route to the appropriate server when the request is not SSL. I setup two haproxy instances - one in tcp mode for protocol detection and the second one for routing http requests application servers. Works like a charm on my development machine. Thanks again! Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/13/2011 07:59 AM, Willy Tarreau wrote: Many thanks for your help! It is great that it is possible to use haproxy instead of adding another software for frontend. I did not know about this patch. If this way of requesting flash policy files is well defined and standard, then we should merge this patch as it may ease some setups. Yes, this is described here: http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html (see: Socket policy request process) In theory it should. However with what is above, you don't need it, which saves you from maintaining a patch. I am trying it now. Thanks again! Regards, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On Tue, Dec 13, 2011 at 10:00:50AM +0100, Lukasz Michalski wrote: On 12/13/2011 07:59 AM, Willy Tarreau wrote: Many thanks for your help! It is great that it is possible to use haproxy instead of adding another software for frontend. I did not know about this patch. If this way of requesting flash policy files is well defined and standard, then we should merge this patch as it may ease some setups. Yes, this is described here: http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html (see: Socket policy request process) OK thanks for the pointer. Indeed, it looks well specified enough. Cheers, Willy
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
Hi Lukasz, You must use stunnel in front of haproxy in order to be able to inspect pure clear HTTP traffic. Concerning your HAProxy configuration, it's not fully accurate. Please give a try to the configuration below: frontend mode tcp tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if HTTP default_backend nodejs_flashpolicy backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check cheers On Mon, Dec 12, 2011 at 8:10 AM, Lukasz Michalski l...@zork.pl wrote: On 12/10/2011 04:29 PM, Baptiste wrote: Hi, There is an ACL for that: req_proto_http (and an alias exists: HTTP) you can choose you backend based on it's return: tcp-request content inspect-delay 1s use_backend bk_http if HTTP use_backend bk_xml if !HTTP I am not sure how this should be configured. I added to my conf file: tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check haproxy does not let me use_backend nodejs_flashpolicy in my frontend: [ALERT] 345/074250 (26349) : Unable to use proxy 'nodejs_flashpolicy' with wrong mode, required: http, has: tcp. But now I realized that the connection is dropped by stunnel and does not even get to haproxy. It seems that the best way would be to inspect the data on ports 80 and 443 and if I get xml string (which is always the same) then connect to appropriate service, otherwise use stunnel/haproxy. The problem is that I cannot find any software for linux that is capable of doing so. If you know any please share. Many thanks, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/12/2011 09:52 AM, Baptiste wrote: Hi Lukasz, You must use stunnel in front of haproxy in order to be able to inspect pure clear HTTP traffic. Concerning your HAProxy configuration, it's not fully accurate. Please give a try to the configuration below: frontend mode tcp tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if HTTP default_backend nodejs_flashpolicy backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check cheers Thanks for quick reply. I tried mode tcp on frontend but then I cannot route HTTP requests to my backends. This my current config: # Flash plugin will not send send policy-file-request/ # on client connection if policy file can be requested # from the same host on port 843. # We cannot use different port here. frontend flash_policy bind 0.0.0.0:843 mode tcp default_backend nodejs_flashpolicy # Main frontend frontend proxy bind 0.0.0.0:80 bind 0.0.0.0:8443 mode http option httplog maxconn 20 timeout client 8640 default_backend webapp tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP #use_backend nodejs_flashpolicy if !HTTP #websockets acl is_websocket path_beg /socket.io acl is_websocket hdr(Upgrade) -i WebSocket acl is_websocket hdr_beg(Host) -i ws use_backend websocket if is_websocket backend webapp mode http option httplog option httpclose server cherrypy1 localhost:7000 check backend websocket mode http option httplog timeout server 8640 timeout queue 5000 timeout connect 8640 option forwardfor no option httpclose option http-server-close option forceclose server node1 localhost:12000 maxconn 2000 check backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check I have to route http traffic to this two backends based on request path, host and HTTP headers. Additionally if !HTTP traffic is detected on proxy frontend I would like to route it to nodejs_flashpolicy. I tried tcp mode on frontend but it caused a random backend to be selected when valid HTTP request came in. Is it true that my acl's are ignored in tcp mode? Anyway, my websocket traffic is over SSL and I have to make this decision before SSL connection is terminated by stunnel, because flash plugin sends unencrypted xml data to port 443 if port 843 is not available. Thanks for your help, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
well, if your websocket traffic is encrypted, how do you want haproxy to read the headers Your last chance is to try to use dst_port acl to match websocket port or to use two separated frontend.. cheers On Mon, Dec 12, 2011 at 10:10 AM, Lukasz Michalski l...@zork.pl wrote: On 12/12/2011 09:52 AM, Baptiste wrote: Hi Lukasz, You must use stunnel in front of haproxy in order to be able to inspect pure clear HTTP traffic. Concerning your HAProxy configuration, it's not fully accurate. Please give a try to the configuration below: frontend mode tcp tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if HTTP default_backend nodejs_flashpolicy backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check cheers Thanks for quick reply. I tried mode tcp on frontend but then I cannot route HTTP requests to my backends. This my current config: # Flash plugin will not send send policy-file-request/ # on client connection if policy file can be requested # from the same host on port 843. # We cannot use different port here. frontend flash_policy bind 0.0.0.0:843 mode tcp default_backend nodejs_flashpolicy # Main frontend frontend proxy bind 0.0.0.0:80 bind 0.0.0.0:8443 mode http option httplog maxconn 20 timeout client 8640 default_backend webapp tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP #use_backend nodejs_flashpolicy if !HTTP #websockets acl is_websocket path_beg /socket.io acl is_websocket hdr(Upgrade) -i WebSocket acl is_websocket hdr_beg(Host) -i ws use_backend websocket if is_websocket backend webapp mode http option httplog option httpclose server cherrypy1 localhost:7000 check backend websocket mode http option httplog timeout server 8640 timeout queue 5000 timeout connect 8640 option forwardfor no option httpclose option http-server-close option forceclose server node1 localhost:12000 maxconn 2000 check backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check I have to route http traffic to this two backends based on request path, host and HTTP headers. Additionally if !HTTP traffic is detected on proxy frontend I would like to route it to nodejs_flashpolicy. I tried tcp mode on frontend but it caused a random backend to be selected when valid HTTP request came in. Is it true that my acl's are ignored in tcp mode? Anyway, my websocket traffic is over SSL and I have to make this decision before SSL connection is terminated by stunnel, because flash plugin sends unencrypted xml data to port 443 if port 843 is not available. Thanks for your help, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
Hi,
On Mon, Dec 12, 2011 at 09:20:28PM +0100, ??ukasz Michalski wrote:
W dniu 2011-12-12 19:17, Baptiste pisze:
well, if your websocket traffic is encrypted, how do you want haproxy
to read the headers
It works this way:
1. From flash plugin I open socket connection to port 443 to make HTTP
requests.
2. Flash connects to port 843 and sends string policy-file-request/
and expects xml reply with info where it is allowed to connect.
3. If port 843 is not available then flash plugin sends string
policy-file-request using connection opened at (1).
4. If flash plugin receives valid reply from(3) then it opens new
connection to port 443 where I am allowed to make my encrypted HTTP
requests.
Points 2 and 3 are outside my control - it is done by flash plugin
behind the scenes. On server side I have to detect string
policy-file-request/ and route that connection to the backend that
will send valid xml reply - note that this string is sent to port meant
to be used for HTTP data exchange - it could be encrypted (443) or not (80).
SSL adds more complexity because in this case all this is happening on
stunnel and stunnel at point (3) resets my connection as invalid SSL
handshake.
I think that have to setup some tcp content analyzer that will be able
to read first 15 bytes from traffic and then route connection to stunnel
or haproxy only if this is not a policy file request.
Yes, then you can proceed that way :
- have haproxy in TCP mode in front of stunnel so that it detects
either a valid and complete SSL message, or at least 22 bytes
of non-SSL traffic (the XML request) ;
- make it route to the appropriate server when the request is not SSL.
It would look like this :
frontend port443
bind :443
tcp-request inspect-delay 5s
acl traffic_is_ssl req_ssl_ver -gt 0
acl enough_non_ssl_bytes req_len -ge 22
tcp-request content accept if traffic_is_ssl # accept SSL
tcp-request content accept if enough_non_ssl_bytes # accept non-SSL
# at this point we have something valid in the buffer
use_backend ssl_backend if traffic_is_ssl
default_backend xml_backend
backend ssl_backend
server stunnel 127.0.0.1:4443
backend xml_backend
server xml-server 127.0.0.1:8883
You can even proceed that way for HTTP too and even have them all on the
same port if you want :
frontend port443
bind :443
tcp-request inspect-delay 5s
acl traffic_is_ssl req_ssl_ver-gt 0
acl traffic_is_http req_proto_http
acl enough_non_ssl_bytes req_len-ge 22
tcp-request content accept if traffic_is_ssl # accept SSL
tcp-request content accept if traffic_is_http # accept HTTP
tcp-request content accept if enough_non_ssl_bytes # accept
non-{SSL|HTTP}
# at this point we have something valid in the buffer
use_backend ssl_backend if traffic_is_ssl
use_backend http_backend if traffic_is_http
default_backend xml_backend
Or you can have ssl+xml checks on port 443 and http+xml checks on port 80.
The HTTP backend should have mode http in it so that it completely analyses
the HTTP request.
Maybe I can setup second haproxy in tcp mode with this patch:
https://github.com/dvv/farm/blob/master/flash-policy.diff
I did not know about this patch. If this way of requesting flash policy files
is well defined and standard, then we should merge this patch as it may ease
some setups.
as my tcp content analyzer? Will this patch work in tcp mode?
In theory it should. However with what is above, you don't need it,
which saves you from maintaining a patch.
Regards,
Willy
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/10/2011 04:29 PM, Baptiste wrote: Hi, There is an ACL for that: req_proto_http (and an alias exists: HTTP) you can choose you backend based on it's return: tcp-request content inspect-delay 1s use_backend bk_http if HTTP use_backend bk_xml if !HTTP I am not sure how this should be configured. I added to my conf file: tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check haproxy does not let me use_backend nodejs_flashpolicy in my frontend: [ALERT] 345/074250 (26349) : Unable to use proxy 'nodejs_flashpolicy' with wrong mode, required: http, has: tcp. But now I realized that the connection is dropped by stunnel and does not even get to haproxy. It seems that the best way would be to inspect the data on ports 80 and 443 and if I get xml string (which is always the same) then connect to appropriate service, otherwise use stunnel/haproxy. The problem is that I cannot find any software for linux that is capable of doing so. If you know any please share. Many thanks, Łukasz
route !HTTP connections to tcp backend instead of dropping in HTTP mode
Hi, Is it possible to route non-HTTP connections to specific backend instead of returning bad request in http mode? Some of my clients sends a small XML request (~30 bytes) to HTTP port and I need to route that connection to specific backend in tcp mode. If not, do you know any software that I can put in front of haproxy that can read that first 30 bytes and make a decision if this connection should be routed to haproxy or that specific bakcend? Something like stunnel for SSL connections. Many thanks for any information, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
Hi, There is an ACL for that: req_proto_http (and an alias exists: HTTP) you can choose you backend based on it's return: tcp-request content inspect-delay 1s use_backend bk_http if HTTP use_backend bk_xml if !HTTP cheers 2011/12/10 Łukasz Michalski l...@zork.pl: Hi, Is it possible to route non-HTTP connections to specific backend instead of returning bad request in http mode? Some of my clients sends a small XML request (~30 bytes) to HTTP port and I need to route that connection to specific backend in tcp mode. If not, do you know any software that I can put in front of haproxy that can read that first 30 bytes and make a decision if this connection should be routed to haproxy or that specific bakcend? Something like stunnel for SSL connections. Many thanks for any information, Łukasz
