HAProxy and flash sockets
Hi, I am trying to use HAProxy as proxy for socket.io transports. Standard XMLHttpRequest and WebSockets work ok but I have trouble with flash transport. The flashtransport uses flash plugin to establish socket connection and after that data is send and received using WebSockets protocol. To make flash socket connect I need to provide "flash policy file" to the plugin. According to the documentation[1] flash attempts to download policy file from server available on port 843 at the same domain and if it fails it tries to use destination host and port. My current config looks like this: defaults log 127.0.0.1:514 local1 debug # Flash player will not send send policy-file-request # on client connection if policy file can be requested # from the same host on port 843. # We cannot use different port here. frontend flashpolicy_server bind 0.0.0.0:843 mode tcp default_backend nodejs_flashpolicy # Main frontend frontend proxy bind 0.0.0.0:80 bind 0.0.0.0:8443 mode http option httplog maxconn 20 timeout client 8640 default_backend webapp #websockets acl is_websocket path_beg /socket.io acl is_websocket hdr(Upgrade) -i WebSocket acl is_websocket hdr_beg(Host) -i ws use_backend websocket if is_websocket backend webapp mode http option httplog option httpclose server cherrypy1 localhost:7000 check backend websocket mode http option httplog timeout server 8640 timeout queue 5000 timeout connect 8640 option forwardfor no option httpclose option http-server-close option forceclose server node1 localhost:12000 maxconn 2000 check backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check Everything works but one scenario: if port 843 is blocked on client side then flash plugin is sending '\0' to the proxy frontend and expects xml file. In this case I should route this connection to nodejs_flashpolicy instead of websocket backend. Do you have any solution to this? the best solution would be to check first X bytes for ''. Worse but still working solution would be to route all invalid http requests to the nodejs_flashpolicy. I cannot find any way to configure HAProxy to make it work. If you have any suggestions please share. I am using haproxy 1.4.17 [1] http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html Thanks! Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/10/2011 04:29 PM, Baptiste wrote: > Hi, > > There is an ACL for that: req_proto_http (and an alias exists: HTTP) > you can choose you backend based on it's return: > > tcp-request content inspect-delay 1s > use_backend bk_http if HTTP > use_backend bk_xml if !HTTP > I am not sure how this should be configured. I added to my conf file: tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check haproxy does not let me use_backend nodejs_flashpolicy in my frontend: [ALERT] 345/074250 (26349) : Unable to use proxy 'nodejs_flashpolicy' with wrong mode, required: http, has: tcp. But now I realized that the connection is dropped by stunnel and does not even get to haproxy. It seems that the best way would be to inspect the data on ports 80 and 443 and if I get xml string (which is always the same) then connect to appropriate service, otherwise use stunnel/haproxy. The problem is that I cannot find any software for linux that is capable of doing so. If you know any please share. Many thanks, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/12/2011 09:52 AM, Baptiste wrote: > Hi Lukasz, > > You must use stunnel in front of haproxy in order to be able to > inspect pure clear HTTP traffic. > > Concerning your HAProxy configuration, it's not fully accurate. > Please give a try to the configuration below: > > frontend > mode tcp > tcp-request inspect-delay 1s > use_backend nodejs_flashpolicy_http if HTTP > default_backend nodejs_flashpolicy > > backend nodejs_flashpolicy > mode tcp > server node1 localhost:10843 maxconn 2000 check > > backend nodejs_flashpolicy_http > mode http > option httplog > server node1 localhost:10843 maxconn 2000 check > > cheers > Thanks for quick reply. I tried mode tcp on frontend but then I cannot route HTTP requests to my backends. This my current config: # Flash plugin will not send send # on client connection if policy file can be requested # from the same host on port 843. # We cannot use different port here. frontend flash_policy bind 0.0.0.0:843 mode tcp default_backend nodejs_flashpolicy # Main frontend frontend proxy bind 0.0.0.0:80 bind 0.0.0.0:8443 mode http option httplog maxconn 20 timeout client 8640 default_backend webapp tcp-request content accept if !HTTP tcp-request inspect-delay 1s use_backend nodejs_flashpolicy_http if !HTTP #use_backend nodejs_flashpolicy if !HTTP #websockets acl is_websocket path_beg /socket.io acl is_websocket hdr(Upgrade) -i WebSocket acl is_websocket hdr_beg(Host) -i ws use_backend websocket if is_websocket backend webapp mode http option httplog option httpclose server cherrypy1 localhost:7000 check backend websocket mode http option httplog timeout server 8640 timeout queue 5000 timeout connect 8640 option forwardfor no option httpclose option http-server-close option forceclose server node1 localhost:12000 maxconn 2000 check backend nodejs_flashpolicy mode tcp server node1 localhost:10843 maxconn 2000 check backend nodejs_flashpolicy_http mode http option httplog server node1 localhost:10843 maxconn 2000 check I have to route http traffic to this two backends based on request path, host and HTTP headers. Additionally if !HTTP traffic is detected on proxy frontend I would like to route it to nodejs_flashpolicy. I tried tcp mode on frontend but it caused a random backend to be selected when valid HTTP request came in. Is it true that my acl's are ignored in tcp mode? Anyway, my websocket traffic is over SSL and I have to make this decision before SSL connection is terminated by stunnel, because flash plugin sends unencrypted xml data to port 443 if port 843 is not available. Thanks for your help, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/13/2011 07:59 AM, Willy Tarreau wrote: Many thanks for your help! It is great that it is possible to use haproxy instead of adding another software for frontend. > > I did not know about this patch. If this way of requesting flash policy files > is well defined and standard, then we should merge this patch as it may ease > some setups. Yes, this is described here: http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html (see: "Socket policy request process") > > In theory it should. However with what is above, you don't need it, > which saves you from maintaining a patch. > I am trying it now. Thanks again! Regards, Łukasz
Re: route !HTTP connections to tcp backend instead of dropping in HTTP mode
On 12/13/2011 07:59 AM, Willy Tarreau wrote: >> >> I think that have to setup some tcp content analyzer that will be able >> to read first 15 bytes from traffic and then route connection to stunnel >> or haproxy only if this is not a "policy file request". > > Yes, then you can proceed that way : > > - have haproxy in TCP mode in front of stunnel so that it detects > either a valid and complete SSL message, or at least 22 bytes > of non-SSL traffic (the XML request) ; > - make it route to the appropriate server when the request is not SSL. > I setup two haproxy instances - one in tcp mode for protocol detection and the second one for routing http requests application servers. Works like a charm on my development machine. Thanks again! Łukasz