haproxy administration web interface
Hi, A simple question, is there any web interface to administer haproxy via web? -- Regards; Israel Garcia
Re: Session stickiness over HTTP and HTTPS
Is this a common use case? Yes. I see that section 3.1 in the configuration guide discusses using stunnel for this, but it's not clear whether haproxy will choose the sticky server based on stunnel's X-Forwarded-For header or it will choose the destination by the stunnel machine's address? You can balance on X-Forwarded-For or sourceip (you want x-forwarded-for). You could also inject cookies to archieve stickyness. Just read the documentation. ;) Best regards, Craig
Re: haproxy administration web interface
Hi, On 07.12.09 20:49, Israel Garcia wrote: Hi, A simple question, is there any web interface to administer haproxy via web? A simple answer: Nope, at least no free one I have heard of. Maybe you could find something from loadbalancer.org However, I am currently looking into developing a simple twisted and/or django based REST-webservice to manage some aspects of Haproxy. Currently, I am planning the following features: * Create and edit a complete configuration by using something like haproxy-config (http://github.com/finnlabs/haproxy) * Add, edit and remove complete sections * Allow member servers of backends and listeners to be added and removed * Use the stats-socket to interface directly with Haproxy * Set the weight of individual backend servers (for Haproxy 1.4) * Provide a (readonly) webservice API to the various Haproxy stats Optionally: Provide a callback interface to perform certain used defined actions based on state changes of ressources by providing callbacks to which user code can register itself. This interface could be called from something like syslog-ng in nearly realtime. I plan on hacking on it during the evenings / nights of the upcomming 26c3. So if you have any ideas, feel free to provide them here. --Holger
Re: Session stickiness over HTTP and HTTPS
On 07.12.09 23:19, Anthony Urso wrote: Hi: I am looking for advice on the best way to load-balance HTTP and HTTPS traffic such that once a session is established with either protocol, haproxy continues to send new requests from that session to the same web server. Is this a common use case? This indeed pretty common (although, I tend to avoid this for the sake of simplicity using cookie-based sessions et al.) However, as HTTP is a stateless protocol by definition, which does not inherently have the concept of a session, you have to decide for yourself (or your app) what exactly a session makes. Using this info you can then tell Haproxy how to match a specific stateless request from a client and send it to the correct server which then holds its session data. For some well-documented examples see the architecture guide. [1] Additionally, it is always a good idea to put the configuration manual [2] under your pillow at night ;) I see that section 3.1 in the configuration guide discusses using stunnel for this, but it's not clear whether haproxy will choose the sticky server based on stunnel's X-Forwarded-For header or it will choose the destination by the stunnel machine's address? As stated above, this is up to you. In this case I think, it makes only sense to have it use the X-Forwarded-For header of stunnel. You can configure both. --Holger [1] http://haproxy.1wt.eu/download/1.3/doc/architecture.txt [2] http://haproxy.1wt.eu/download/1.3/doc/configuration.txt
haproxy stats truncation
Hi,
Looking over the bug matrix, I see two bugs referencing stats truncation,
http://haproxy.1wt.eu/knownbugs-1.3.html
stats output sometimes truncated fixed in 1.3.14.5
and
stats output limited to 16 kB fixed in 1.3.14.4
I'm running 1.3.22, and i appear to be hitting this bug (i've also tested with
1.3.15.9 and 1.3.20)
Worth noting is that none of the pages being served THROUGH haproxy have this
issue, they are all work fine.
Also, haproxy is fantastic, I absolutely love it! But I really want to get some
cool stats to impress with, help!
-Adam
curl -v 'http://x.x.x.x:8080/'
* About to connect() to x.x.x.x port 8080 (#0)
* Trying x.x.x.x... connected
* Connected to x.x.x.x (x.x.x.x) port 8080 (#0)
GET / HTTP/1.1
User-Agent: curl/7.19.6 (i386-apple-darwin10.0.0) libcurl/7.19.6 zlib/1.2.3
Host: x.x.x.x:8080
Accept: */*
* HTTP 1.0, assume close after body
HTTP/1.0 200 OK
Cache-Control: no-cache
Connection: close
Content-Type: text/html
htmlheadtitleStatistics Report for HAProxy/title
meta http-equiv=content-type content=text/html; charset=iso-8859-1
style type=text/css!--
body { font-family: arial, helvetica, sans-serif; font-size: 12px; font-weight:
normal; color: black; background: white;}
th,td { font-size: 10px; align: center;}
h1 { font-size: x-large; margin-bottom: 0.5em;}
h2 { font-family: helvetica, arial; font-size: x-large; font-weight: bold;
font-style: italic; color: #6020a0; margin-top: 0em; margin-bottom: 0em;}
h3 { font-family: helvetica, arial; font-size: 16px; font-weight: bold; color:
#b00040; background: #e8e8d0; margin-top: 0em; margin-bottom: 0em;}
li { margin-top: 0.25em; margin-right: 2em;}
.hr {margin-top: 0.25em; border-color: black; border-bottom-style: solid;}
.titre {background: #20D0D0;color: #00; font-weight: bold;}
.total {background: #20D0D0;color: #80;}
.frontend {background: #e8e8d0;}
.backend{background: #e8e8d0;}
.active0{background: #ff9090;}
.active1{background: #ffd020;}
.active2{background: #a0;}
.active3{background: #c0ffc0;}
.active4{background: #a0;}
.active5{background: #a0e0a0;}
.active6{background: #e0e0e0;}
.backup0{background: #ff9090;}
.backup1{background: #ff80ff;}
.backup2{background: #c060ff;}
.backup3{background: #b0d0ff;}
.backup4{background: #c060ff;}
.backup5{background: #90b0e0;}
.backup6{background: #e0e0e0;}
.rls {letter-spacing: 0.2em; margin-right: 1px;}
table.tbl { border-collapse: collapse; border-style: none;}
table.tbl td { border-width: 1px 1px 1px 1px; border-style: solid solid solid
solid; padding: 2px 3px; border-color: gray;}
table.tbl th { border-width: 1px; border-style: solid solid solid solid;
border-color: gray;}
table.tbl th.pxname {background: #b00040; color: #40; font-weight: bold;
border-style: solid solid none solid; padding: 2px 3px; white-space: nowrap;}
table.tbl th.empty { border-style: none; empty-cells: hide; background: white;}
table.tbl th.desc { background: white; border-style: solid solid none solid;
text-align: left; padding: 2px 3px;}
table.lgd { border-collapse: collapse; border-width: 1px; border-style: none
none none solid; border-color: black;}
table.lgd td { border-width: 1px; border-style: solid solid solid solid;
border-color: gray; padding: 2px;}
table.lgd td.noborder { border-style: none; padding: 2px; white-space: nowrap;}
--
/style/head
bodyh1a href=http://haproxy.1wt.eu/; style=text-decoration:
none;HAProxy version 1.3.22, released 2009/10/14/a/h1
h2Statistics Report for pid .../h2
hr width=100% class=hr
h3gt; General process information/h3
table border=0 cols=4trtd align=left nowrap width=1%
pbpid = /b (process #1, nbproc = 1)br
buptime = /b 0d 0h31m01sbr
* Closing connection #0
* Failure when receiving data from the peer
curl: (56) Failure when receiving data from the peer
bsystem
Re: haproxy administration web interface
On 12/7/09, Duncan Hall dun...@viator.com wrote: Israel Garcia wrote: Hi, A simple question, is there any web interface to administer haproxy via web? Hi Duncan, HAProxy has a dashboard that shows the status of the load balancing. It doesn't include an interface for making changes. Yes, a very good stats page in deed..:-) PFsense (a FreeBSD based firewall http://pfsense.org) has a php interface for administering the haproxy package, but it does not include all of the configuration options (yet). I'll take a look... thanks in advance regards, Israel. Regards, Duncan -- Regards; Israel Garcia
Re: haproxy administration web interface
On 12/7/09, Holger Just w...@meine-er.de wrote: Hi, On 07.12.09 20:49, Israel Garcia wrote: Hi, A simple question, is there any web interface to administer haproxy via web? Hi Holder, A simple answer: Nope, at least no free one I have heard of. Maybe you could find something from loadbalancer.org However, I am currently looking into developing a simple twisted and/or django based REST-webservice to manage some aspects of Haproxy. That's very interesting.. I'll take a look at this... I'll keep you posted.. Currently, I am planning the following features: * Create and edit a complete configuration by using something like haproxy-config (http://github.com/finnlabs/haproxy) * Add, edit and remove complete sections * Allow member servers of backends and listeners to be added and removed * Use the stats-socket to interface directly with Haproxy * Set the weight of individual backend servers (for Haproxy 1.4) * Provide a (readonly) webservice API to the various Haproxy stats Optionally: Provide a callback interface to perform certain used defined actions based on state changes of ressources by providing callbacks to which user code can register itself. This interface could be called from something like syslog-ng in nearly realtime. I plan on hacking on it during the evenings / nights of the upcomming 26c3. So if you have any ideas, feel free to provide them here. thanks in advance regards, Israel. --Holger -- Regards; Israel Garcia
Re: haproxy stats truncation
Hi Adam, On Mon, Dec 07, 2009 at 08:05:58PM -0500, Adam Jacob Muller wrote: Hi, Looking over the bug matrix, I see two bugs referencing stats truncation, http://haproxy.1wt.eu/knownbugs-1.3.html stats output sometimes truncated fixed in 1.3.14.5 and stats output limited to 16 kB fixed in 1.3.14.4 I'm running 1.3.22, and i appear to be hitting this bug (i've also tested with 1.3.15.9 and 1.3.20) Worth noting is that none of the pages being served THROUGH haproxy have this issue, they are all work fine. Also, haproxy is fantastic, I absolutely love it! But I really want to get some cool stats to impress with, help! OK. Can you please post your config (at least the relevant part for the section handling the stats) ? You can remove any IP/password if you want. But we need to see timeouts, options, defaults, etc... I suspect there is too small a timeout on the client side and that haproxy is closing the connection while curl is displauing the output on your screen. Regards, Willy
Re: Haproxy server timeouts?
Hi, On Sat, Dec 05, 2009 at 12:11:54AM +0100, XANi wrote: Dnia 2009-12-04, pi?? o godzinie 17:57 -0500, Naveen Ayyagari pisze: The issue we have is that our scripts are dependent on external resources, so php execution time can vary wildly. (...) Yes i meant processor cores, basically if you have extreme cases like 80 processes on 8 cores then imo its better to use less processes and queue reqests in proxy (too much content switching is bad thing for performance), but if in your case its just because php waits for something and not because server is overloaded it wont change much. You might want to consider checking if other http servers liek lighttpd also have that bug If you are fetching data from external resources, you may want to split the access between 2 distinct haproxy backends (which might very well point to the same servers). That implies you know what URLs remain local and which ones fetch remote data. Then you can proceed like this : frontend www acl remote_content path_beg /x/y/z use_backend bk_remote if remote_content default_backend bk_local backend bk_local timeout server 5s server www1 1.1.1.1 maxconn 100 check backend bk_remote timeout server 50s server www1 1.1.1.1 maxconn 5 track bk_local/www1 That way, you allow more time for remote resources, but you don't permit them to fill your queues, as they have a dedicated queue and maxconn. It's a very basic QoS principle but it works very well because you prevent expensive processing from saturating your servers. Regards, Willy
Re: haproxy stats truncation
Hi Willy, I have right now... defaults timeout client 5s timeout connect 5s timeout server 5s option nolinger listen stats x.x.x.x:8080 mode http stats uri / Actually, I just tracked down the issue (partially anyway). option nolinger is/was causing this. Moving nolinger into the frontend block seems to keep the fin_wait1 down and makes the stats page work! odd that I never get this issue when haproxy is proxying though, only with the stats page? How nice though that haproxy lets you set such low-level options on a per-request basis! I guess its not really a bug then :) -Adam On Dec 8, 2009, at 12:17 AM, Willy Tarreau wrote: Hi Adam, On Mon, Dec 07, 2009 at 08:05:58PM -0500, Adam Jacob Muller wrote: Hi, Looking over the bug matrix, I see two bugs referencing stats truncation, http://haproxy.1wt.eu/knownbugs-1.3.html stats output sometimes truncated fixed in 1.3.14.5 and stats output limited to 16 kB fixed in 1.3.14.4 I'm running 1.3.22, and i appear to be hitting this bug (i've also tested with 1.3.15.9 and 1.3.20) Worth noting is that none of the pages being served THROUGH haproxy have this issue, they are all work fine. Also, haproxy is fantastic, I absolutely love it! But I really want to get some cool stats to impress with, help! OK. Can you please post your config (at least the relevant part for the section handling the stats) ? You can remove any IP/password if you want. But we need to see timeouts, options, defaults, etc... I suspect there is too small a timeout on the client side and that haproxy is closing the connection while curl is displauing the output on your screen. Regards, Willy
Re: haproxy administration web interface
Hi, On Mon, Dec 07, 2009 at 02:49:30PM -0500, Israel Garcia wrote: Hi, A simple question, is there any web interface to administer haproxy via web? The only web interfaces I'm aware of are the ones in commercial products using haproxy (Exceliance ALOHA, Loadbalancer.org, maybe others ?). But quite frankly, it's very hard not to be limited by a web interface for a load balancer. A load balancer is not a firewall, it does only dirty things. You always have to use a lot of tricks. Passing special parameters, placing proper ACLs at the right place, rewrite headers, etc... And believe me, every time I heard hey no I just need a basic setup, the guys finally had to use funny tricks for a special case. So if you're looking for a web interface to make config writing easier, I'd suggest that you use scripts and templates instead to generate your configuration. Then you'll find that you have the power of the flat file with the ability to add services and servers using very few parameters. In the end it might be a better solution than a web interface. Regards, Willy
Re: haproxy stats truncation
On Tue, Dec 08, 2009 at 12:29:24AM -0500, Adam Jacob Muller wrote: Hi Willy, I have right now... defaults timeout client 5s timeout connect 5s timeout server 5s option nolinger It's this one (nolinger). listen stats x.x.x.x:8080 mode http stats uri / Actually, I just tracked down the issue (partially anyway). option nolinger is/was causing this. Ah yes :-) Moving nolinger into the frontend block seems to keep the fin_wait1 down and makes the stats page work! odd that I never get this issue when haproxy is proxying though, only with the stats page? Because you're lucky! nolinger tells the system that the last data queued in buffers have to be discarded as soon as the connection is closed. I think that you're not seeing it in proxied traffic because your browser closes the connection last, which means it has received those data. But if your browser had closed its output channel first, you would have encountered the same issue. You can have the same problem when testing haproxy's monitor uri from another component. Do you really have that many fin_wait1 sockets to require the option ? The only case where it was needed (and implemented) was because of a bug in a remote TCP stack used by all the clients. So I find it a bit strange that you need it. How nice though that haproxy lets you set such low-level options on a per-request basis! To be precise it's not on a per-request basis but per-service :-) I guess its not really a bug then :) Indeed. It's just doing what it's supposed to do : drop pending data at the end of the connection. Regards, Willy
Re: Session stickiness over HTTP and HTTPS
On Tue, Dec 08, 2009 at 12:56:03AM +0100, Holger Just wrote: On 07.12.09 23:19, Anthony Urso wrote: Hi: I am looking for advice on the best way to load-balance HTTP and HTTPS traffic such that once a session is established with either protocol, haproxy continues to send new requests from that session to the same web server. Is this a common use case? This indeed pretty common (although, I tend to avoid this for the sake of simplicity using cookie-based sessions et al.) However, as HTTP is a stateless protocol by definition, which does not inherently have the concept of a session, you have to decide for yourself (or your app) what exactly a session makes. Exactly ! When I have to do this, I use stunnel to transform HTTPS into HTTP, and just use the same cookie for both services (most often both protocols point to the same frontend/backend anyway). Using a source address is generally fine on LANs because PCs don't change their IP often. But it's not practical on the net where you can generally find approximately 5% of your clients who regularly come with a different IP address because of the proxy farms they have to go through. Regards, Willy
