Re: reqrep syntax
On 9/5/17 8:26am, Holger Just wrote: > With a modern HAProxy (i.e. anything >= 1.6), you can just use > > http-request set-path %[path,regsub(^/static,/)] Brilliant, thanks. I spent 20 minutes trying to figure how to submit a patch for the documentation, but failed completely. Maintainers of this project: please consider making it easier to submit documentation fixes. I know it can involve more work engaging with a community, but the rewards can be useful. With a task tracker, pull requests, etc its a little hard for a casual user to help. The things that need improvement in the docs are: 1. In the documentation for reqrep make a big note that the examples provided are not the current recommended approach and to instead use http-request if possible. Otherwise the official documentation reads like it is best practice. 2. In the docs for reqrep, put in an example of what the regex is matching. For example "GET /some/path?param=true HTTP/1.1" 3. In the docs for http-request [1] separate out the possible parameters on new lines. I read that doc 10 times and never noticed I needed to read across because someone wanted to save paper when the docs were printed by wrapping the parameters. So I never saw set-path. 4. In the cbonte html version, it would be lovely if set-path was a searchable keyword. "path" is and that's just an acl parameter. Thanks for your help and to everyone working on this project. Cheers Ari [1] https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#http-request -- --> Aristedes Maniatis CEO, ish https://www.ish.com.au GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A signature.asc Description: OpenPGP digital signature
reqrep syntax
In the manual [1] there is an example for using reqrep with syntax like this: # replace "/static/" with "/" at the beginning of any request path. reqrep ^([^\ :]*)\ /static/(.*) \1\ /\2 But a typical http request looks like this: GET /haproxy-dconv/1.7/configuration.html HTTP/1.1 Host: cbonte.github.io User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-GB,en;q=0.5 Accept-Encoding: gzip, deflate, br DNT: 1 Connection: keep-alive Upgrade-Insecure-Requests: 1 Cache-Control: max-age=0 Firstly, is there no better/cleaner way to rewrite the path than this messy regex which seems to process all the http headers rather than just the request line that contains the path? And not only that, but we need to consume all three parts of that line including the HTTP version. The ACL processing engine already can get at the path, so it is available somewhere inside haproxy, just not in a way we can easily rewrite. Secondly, why was this particular regex used in the example? If I'm reading this correctly, quoting the string would simplify it to: reqrep '^([^ :]*) /static/(.*)' '\1 /\2' But is this even clearer and safer: reqrep '^([A-Z]+) /static(/.*)' '\1 \2' or narrow this down (where appropriate to the type of request) to reqrep '^(GET|POST) /static(/.*)' '\1 \2' Perhaps the example would be clearer like this (even though this is slower and not needed here, it is clearly demonstrating the point): reqrep '^(GET|POST) /static(/.*) (HTTP/.+)' '\1 \2 \3' Have I got all the above right? I'm finding examples on the internet like this [2] where the author believes the first part is the url scheme which I think is wrong. Thanks Ari [1] https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4.2-reqrep [2] https://fromanegg.com/post/2014/12/05/how-to-rewrite-and-redirect-with-haproxy/ -- --> Aristedes Maniatis CEO, ish https://www.ish.com.au GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A signature.asc Description: OpenPGP digital signature
Re: redirect with variables
On 27/09/2016 6:10pm, Aristedes Maniatis wrote:
> I'd like to redirect subdomains in a generic way like this:
>
> acme.example.com -> acme.example2.com
>
>
> I tried this:
>
> redirect prefix http://%[hdr(host),field(1,'.')].example2.com code 301 unless
> { hdr_end(host) -i .example2.com }
>
> but haproxy 1.6 doesn't like the %[hdr()] notation in this location. Is there
> a way to achieve this?
I think I've just discovered the trick which makes it work or break. When I
have this line as just "redirect prefix..." then I get a redirect response to
"http://%[hdr(x-original-site),field(1,.)].example2.com/"
When I add "http-request" to the start of this line, then it works fine.
I don't understand from the documentation the difference between these two
directives, however from my initial tests it does seem they are different and
http-request runs earlier in the processing of a response than just a bare
"redirect".
It would be useful if the documentation better explained the %[] operator and
when it is allowed to be used.
Thanks
Ari
--
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
signature.asc
Description: OpenPGP digital signature
redirect with variables
I'd like to redirect subdomains in a generic way like this:
acme.example.com -> acme.example2.com
I tried this:
redirect prefix http://%[hdr(host),field(1,'.')].example2.com code 301 unless {
hdr_end(host) -i .example2.com }
but haproxy 1.6 doesn't like the %[hdr()] notation in this location. Is there a
way to achieve this?
Cheers
Ari
--
------>
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
signature.asc
Description: OpenPGP digital signature
Re: Allowing 500 errors to pass through
On 11/11/2015 12:00pm, Bryan Talbot wrote: > 1. Will haproxy remove a server from its backend pool if it returns a 500 > response to a request (I'm not talking about the health check, but just a > regular request) > > > > Not by default but haproxy can monitor normal traffic and take action if > desired. See the 'observe' and 'on-error' options. Ah, then by default it does what I want. Perfect. For others following along, the manual is here: https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#observe That observe feature is really cool and I hadn't seen it before. > 2. Can haproxy be instructed to ignore 500 errors for its health check (I > still want to detect that the server has gone away and doesn't respond, but > the 500 error might be transient or it might just be on one page due to > misconfiguration and doesn't warrant removing the whole server). > > > Probably, but if you don't care about the HTTP request or response, why not > just use TCP health checks? Ah, of course! When looking for complex solutions I missed the simple thing in front of me. Thanks Ari -- --> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A signature.asc Description: OpenPGP digital signature
Allowing 500 errors to pass through
I've got a situation with haproxy 1.5.x I'm trying to understand better. In my situation, several Apache httpd servers sit behind haproxy and behind that are the actual application servers. httpd is using mod-jk to load balance all the applications to all the web servers. So if the application server returns a 500 error, apache httpd will pass that through to haproxy and then to the end user. So far, all good. But I don't want haproxy to then remove that server from the pool. The 500 error could have been from a single application server, and we don't want to it lock out since it is already load balanced behind all the httpd servers. We will end up locking out one of the httpd server which really wasn't at fault. So my question: 1. Will haproxy remove a server from its backend pool if it returns a 500 response to a request (I'm not talking about the health check, but just a regular request) 2. Can haproxy be instructed to ignore 500 errors for its health check (I still want to detect that the server has gone away and doesn't respond, but the 500 error might be transient or it might just be on one page due to misconfiguration and doesn't warrant removing the whole server). I think in the long term I'd like to remove the httpd layer to my setup, but for now it performs important mod_rewrite work that I can't easily replace. Once removed, haproxy will be exposed directly to the application servers and can be cleverer about removing them from the pool if they die. Am I right in guessing that question (2) might be solved by the new features in haproxy 1.6? Thanks Ari -- --> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A signature.asc Description: OpenPGP digital signature
Re: SSL hardware acceleration
On 27/05/2014 6:59pm, Lukas Tribus wrote: > Hi, > > >> Without purchasing specific expensive add-on cards [1], is there >> something specific to some modern CPUs which will accelerate SSL >> handling in haproxy 1.5? >> >> That is, should I be looking for something in a CPU which will >> improve performance considerably? There is an Intel instruction >> set called AES-NI but I don't know if that applies to HTTPS# >> traffic. As I understand, the initial negotiation in SSL is rsa/dsa >> but then the payload is transported using symmetric key encryption >> (like AES?). >> >> I'm only looking to handle about 50Mb/s of SSL traffic, so I'm not >> aiming very high. But it would be nice to know the headroom is there. > > Bandwidth is not really the limiting factor, handshakes per second is. > AES-NI gives you a nice performance boost but doesn't help with handshakes > afaik. > > Whats important, among other points, is having enough entropy, and the RDRAND > feature of modern CPUs can help you there (if you trust your CPU vendor). > > Otherwise, there some software projects like haveged or audio entropy daemon > that can feed random data in the kernel. > > > Keep-alive and session id resumption are very important features to scale > a SSL enabled site, so double check that those things are working properly. Right, so then it isn't about AES at all, but the public key negotiation and key generation. We are running on Freebsd 10 which feeds /dev/random from yarrow and that in turn grabs entropy from the CPU and other places. So I think we should be good since we are unlikely to run out of entropy there. aesni_load="YES" in loader.conf should take care of the AES side of things If the NSA wanted credit card numbers they could just go get them from Mastercard directly, and there isn't really much else of great espionage interest in the transactional data. So I'm not overly concerned about the backdoors in the Intel CPUs. Thanks for the useful information. Ari -- --> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
SSL hardware acceleration
Without purchasing specific expensive add-on cards [1], is there something specific to some modern CPUs which will accelerate SSL handling in haproxy 1.5? That is, should I be looking for something in a CPU which will improve performance considerably? There is an Intel instruction set called AES-NI but I don't know if that applies to HTTPS traffic. As I understand, the initial negotiation in SSL is rsa/dsa but then the payload is transported using symmetric key encryption (like AES?). I'm only looking to handle about 50Mb/s of SSL traffic, so I'm not aiming very high. But it would be nice to know the headroom is there. Cheers Ari [1] http://www.cavium.com/processor_security_nitrox-III.html -- ------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
CVE-2011-3192 and Range requests
What is the vulnerability [1] of an Apache httpd server with haproxy in front of it? 1. haproxy is fine, httpd will still suffer from DoS attacks 2. haproxy may itself suffer DoS 3. haproxy is fine and will protect an httpd server from DoS Thanks for an excellent product. Ari [1] http://article.gmane.org/gmane.comp.apache.announce/59 -- --> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: listen -> unique IP:port
On 8/01/10 1:31 AM, Willy Tarreau wrote: Hi, On Fri, Jan 08, 2010 at 12:51:46AM +1100, Aristedes Maniatis wrote: I'm working on a web UI for administering the haproxy config and I have a question I can't find the answer to in the documentation. Are the listener blocks unique with respect to IP:port? That is, is this valid: listen bind 1.2.3.4:80 acl someAcl... server 1.2.5.1 ifsomeAcl listen bind 1.2.3.4:80 acl someOtherAcl... server 1.2.5.1 if someOtherAcl Are these listeners like a Unix listener, only one unique IP:port pair is allowed. Or are duplicates OK and they are executed in order from the top to the bottom under a backend server is matched? This depends on the OS in fact. Haproxy accepts such a configuration, but on most OSes it will not be able to bind the port for the second instance because the OS will refuse to bind it a second time. Thanks for that clarification. So to implement virtual hosts (which need to do in order to keep the health checks separated per virtual host, not per backend server) we'll end up with one great big listen block with lots of ACLs each pointing requests to the correct backend. I think that's clear: you might want to add to the documentation that the 'bind' directive creates an OS level listener which corresponds to what you see in netstat. That might help clarify for others. Ari -- ------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
listen -> unique IP:port
I'm working on a web UI for administering the haproxy config and I have a question I can't find the answer to in the documentation. Are the listener blocks unique with respect to IP:port? That is, is this valid: listen bind 1.2.3.4:80 acl someAcl... server 1.2.5.1 ifsomeAcl listen bind 1.2.3.4:80 acl someOtherAcl... server 1.2.5.1 if someOtherAcl Are these listeners like a Unix listener, only one unique IP:port pair is allowed. Or are duplicates OK and they are executed in order from the top to the bottom under a backend server is matched? Thanks Ari Maniatis -- --> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
