Cercas Eléctricas- Si Detienen Ladrones Infórmese Aquí
Cercas Eléctricas- Si Detienen Ladrones Infórmese Aquí Si no puede ver la imagen HAGA CLIC AQUÍ
Problem with health checking
Hi, I've been running haproxy 1.4.24 on centos 6.5 for over 2 years, load balancing a pair of Oracle 11g app servers on Windows 2008r2. It's worked perfectly the entire time. We have recently built two new app servers, same version of Oracle 11g on Windows 2012R2. For health checking we check for http status 200 on the homepage of the application. For the new servers the health checking is failing. We are getting a layer 7 timeout. However when I try and access the page using either curl or lynx it downloads instantly with status 200. Any tips as to where I go from here in troubleshooting would be great. I've checked and double checked the config and the only difference for this load balancer over the existing ones is the IP and port numbers. Right now we are load balancing without health checking, but obviously that's not a solution. Any suggestions would be great, thanks in advance John
Re: Haproxy 1.6.7 segmentation fault under load
Hi Lukas, On Thu, Aug 04, 2016 at 07:02:50PM +0200, Lukas Tribus wrote: > > I'm wondering if anything changed recently in the way zlib does its > > buffer management (ie: maybe they slightly inflated the buffers for > > various maintenance and the current allocator now returns too small > > ones). > > But zlib 1.2.3 was released in 2005, and zlib 1.2.8 (the most recent > release) is from 2013 [2]. In fact CentOS 6.7 (from your report) is based on > old software releases, so it seems unlikely that this is a recent regression > (if those 2 reports really are the same bug). But we all know that enterprise distros tend to backport lots of stuff, so if there's a fix for specific bug that introduces this behaviour it could be in RH's zlib. > > It may be very useful to build with libslz instead of building without > > zlib. It would stress the exact same code paths in haproxy, you would > > still get compression and we'd see if the issue can be reproduced. > > While googling around I found another report [3], a similar/same crash in > memcpy() while using zlib. Apparently switching to libslz fixed the issue > for them. Yes, it seems so. The article is very instructive by the way. Unfortunately we don't know the root cause there. I trust zlib which is used everywhere, but its use in haproxy is quite tricky, maybe sometimes we manage to upset it, I don't know :-/ James, do you know if sometimes you may be running low on memory ? We could imagine a failed memory allocation. Also, did you configure maxzlibmem in your global section ? Just trying to spot what might be happening... Regards, Willy
Re: Haproxy 1.6.7 segmentation fault under load
Hi Willy, Am 04.08.2016 um 17:57 schrieb Willy Tarreau: Thanks for all the useful info. We already had another one reported, with zlib-1.2.3 + haproxy-1.6.4 on CentOS 6.7, all packages up to date. It appeared it was dying in memcpy() called from zlib. Which is what James' stack trace shows as well [1]. I'm wondering if anything changed recently in the way zlib does its buffer management (ie: maybe they slightly inflated the buffers for various maintenance and the current allocator now returns too small ones). But zlib 1.2.3 was released in 2005, and zlib 1.2.8 (the most recent release) is from 2013 [2]. In fact CentOS 6.7 (from your report) is based on old software releases, so it seems unlikely that this is a recent regression (if those 2 reports really are the same bug). It may be very useful to build with libslz instead of building without zlib. It would stress the exact same code paths in haproxy, you would still get compression and we'd see if the issue can be reproduced. While googling around I found another report [3], a similar/same crash in memcpy() while using zlib. Apparently switching to libslz fixed the issue for them. Regards, Lukas [1] https://www.mail-archive.com/haproxy@formilux.org/msg23404.html [2] http://www.zlib.net/ChangeLog.txt [3] https://engineering.udacity.com/haproxy-segfault-adventure-18cc1a7e1171#.7sh9myvj0
Re: SEGFAULT with HAProxy 1.6.5 and server-state-file
On Mon, Aug 01, 2016 at 11:28:48PM +0200, Olivier Doucet wrote: > If I remember correctly, it happens when I removed a server and add another > one. But I'm 100% sure that they had different names. I'm sure because I > always get warnings from reload, because ID did not match (BTW, it would be > great to find a way to set id on server to avoid this). > > Will get back to you tomorrow with a working example I hope. But you > already found two ways to get segfault, not so bad ;) Olivier, please use 1.6.7, there was a bug in the dump in 1.6.5 causing some incorrect data to be dumped sometimes (don't remember exactly what), but while I doubt it is related, at least it can needlessly complicate your troubleshooting. Regards, willy
Re: Haproxy 1.6.7 segmentation fault under load
Hi James, On Tue, Aug 02, 2016 at 06:25:22PM +, James Hartshorn wrote: > Thanks for the info, I had neglected to check if ubuntu provided zlib. > > ldd /usr/sbin/haproxy > linux-vdso.so.1 => (0x7ffe4ed8) > libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x7fc9e975f000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7fc9e955b000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7fc9e9196000) > /lib64/ld-linux-x86-64.so.2 (0x7fc9e9998000) > > Here's how I've been building zlib: > > cd /opt > export ZLIB_VERSION=1.2.8 > > tar -xzvf zlib-$ZLIB_VERSION.tar.gz > > cd /opt/zlib-$ZLIB_VERSION > make clean > ./configure --static > make > > And the relevant section of the haproxy make: > > USE_ZLIB=1 ZLIB_INC=/opt/zlib-$ZLIB_VERSION/ ZLIB_LIB=/opt/zlib-$ZLIB_VERSION/ > > > Today we will try building with os provided zlib, and also running without > compression and possibly building without zlib. Thanks for all the useful info. We already had another one reported, with zlib-1.2.3 + haproxy-1.6.4 on CentOS 6.7, all packages up to date. It appeared it was dying in memcpy() called from zlib. I'm wondering if anything changed recently in the way zlib does its buffer management (ie: maybe they slightly inflated the buffers for various maintenance and the current allocator now returns too small ones). It may be very useful to build with libslz instead of building without zlib. It would stress the exact same code paths in haproxy, you would still get compression and we'd see if the issue can be reproduced. For now we only have two reports of crashes with compression and they are fairly recent, so we can suspect a change in zlib that triggered the issue. thanks! Willy
Re: [PATCH] Re: make install wants to install haproxy-systemd-wrapper
On Fri, Jul 29, 2016 at 08:48:38PM +0200, Vincent Bernat wrote: > ??? 29 juillet 2016 20:40 CEST, Shawn Heisey : > > > fi; \ > > done > > install -d "$(DESTDIR)$(SBINDIR)" > > - install haproxy $(EXTRA) "$(DESTDIR)$(SBINDIR)" > > + install haproxy $(wildcard haproxy-systemd-wrapper) \ > > + $(EXTRA) "$(DESTDIR)$(SBINDIR)" > > > > You can just use the shell for that. That would be less mysterious than > the use of wildcard make function without a wildcard. > > install haproxy $(EXTRA) "$(DESTDIR)$(SBINDIR)" > [ ! -x haproxy-systemd-wrapper ] || install haproxy-systemd-wrapper > "$(DESTDIR)$(SBINDIR)" It's not even needed, what is needed in fact is to specify the same TARGET at install time so that it keeps the same EXTRA value : make install TARGET=linux2628 Note that for people doing : make TARGET=linux2628 all install It should work without any issues. Willy
Re: 51Degrees Trie Update Patch
Hi Ben,
sorry I missed this one.
On Wed, Jul 06, 2016 at 11:25:17AM +, Ben Shillito wrote:
> Attached is a patch submission which makes changes to the 51Degrees Trie
> implementation to work with recent changes to
> github.com/51Degrees/Device-Detection/src/trie/51Degrees.c.
Thanks, but that makes me realize something really bad :
--- a/src/51d.c
+++ b/src/51d.c
@@ -235,7 +235,7 @@ static void _51d_set_device_offsets(struct sample *smp)
(global._51degrees.header_names + index)->len,
msg->chn->buf->p, idx, &ctx) == 1) {
(offsets->firstOffset +
offsets->size)->httpHeaderOffset = *(global._51degrees.header_offsets + index);
- (offsets->firstOffset + offsets->size)->deviceOffset =
fiftyoneDegreesGetDeviceOffset(ctx.line + ctx.val);
+ (offsets->firstOffset + offsets->size)->deviceOffset =
fiftyoneDegreesGetDeviceOffset(&global._51degrees.data_set, ctx.line + ctx.val);
offsets->size++;
}
As you can see the API was changed in the master branch which is also the
one which was documented in the stable version as being the one to pull
from. This is really not cool because stable users who apply what is written
in the readme will get a build failure. Could you please at least provide an
update for the build procedure in the readme that we can backport to 1.6 to
reference a branch that continues to work for these users ? While it will not
fix the problem for existing users, at least they can find a hint in newer
releases about how to fix their issues.
Thanks,
Willy
