Bus error for proxy_hcheck on Solaris
Probably an alignment problem, but I don't immediately see how:
at modules/proxy/mod_proxy_hcheck.c:410
wctx = 0x70706461
So this is not an address usable for a pointer.
But wctx comes from hc->s->context:
{name = "f9328\000/www.kippdata.de/", '\000' , scheme
= "http", '\000' , hostname = "www.kippdata.de",
'\000' ,
route = '\000' , redirect = '\000' times>, flusher = '\000' , uds_path = '\000' 255 times>,
hcuri = '\000' , hcexpr = '\000' times>, "\001", lbset = 0, retries = 0, lbstatus = 0, lbfactor = 1,
min = 0, smax = 1, hmax = 0,
flush_wait = 1, index = 0, passes = 1815278865, pcount = -1571419579,
fails = 3, fcount = 0, hash = {def = 2239854630, fnv = 2173387474},
status = 10,
flush_packets = flush_off, method = GET, updated = 0, error_time =
6000, ttl = 0, retry = 0, timeout = 0, acquire = 0, ping_timeout =
3000, conn_timeout = 0,
interval = 0, recv_buffer_size = 5242880, io_buffer_size = 0, elected
= 0, busy = 0, port = 80, transferred = 536870912, read =
3400067598672358249, context = 0x70706461,
keepalive = 0, disablereuse = 0, is_address_reusable = 0, retry_set =
1, timeout_set = 0, acquire_set = 1, ping_timeout_set = 0,
conn_timeout_set = 0,
recv_buffer_size_set = 0, io_buffer_size_set = 1, keepalive_set = 1,
disablereuse_set = 0, was_malloced = 0, is_name_matchable = 0}
and the context member gets set in line 403:
hc->context = wctx;
If I add a debug statement there the address looks different and fine. I
don't immediately see where the address changes. Sparc is known to be
picky about alignment. One might not observe this on x86.
Regards,
Rainer
Re: Bus error for proxy_hcheck on Solaris
I should add I'm using prefork (other problems with event and worker)
and I didn't access the web server. Simply starting it and waiting a bit
results in those cores.
A complete stack:
#0 hc_get_hcworker (ctx=ctx@entry=0xe3818, worker=worker@entry=0xf9328,
p=p@entry=0x11ff38)
at modules/proxy/mod_proxy_hcheck.c:410
wctx = 0x70706461
hc = 0xe59f8
wptr = 0xe6058 "f9328"
#1 0xfeb432f4 in hc_check_http (worker=0xf9328, p=0x11ff38,
ctx=0xe3818) at modules/proxy/mod_proxy_hcheck.c:650
status =
c = {pool = 0xfeeba558, base_server = 0xfe8fbe60,
vhost_lookup_data = 0xff08a398, local_addr = 0x0, client_addr = 0x1c00,
client_ip = 0x0, remote_host = 0x0,
remote_logname = 0x0, local_ip = 0x5f5e100 out of bounds>,
local_host = 0xfeb49788 "modules/proxy/mod_proxy_hcheck.c",
id = -21649064, conn_config = 0x529dd,
notes = 0x6ed21e3c, input_filters = 0x529dd, output_filters =
0xe3818, sbh = 0xcc290, bucket_alloc = 0xfeb5a0fc, cs = 0x0,
data_in_input_filters = 10,
data_in_output_filters = 0, clogging_input_filters = 0,
double_reverse = -1, aborted = 0, keepalive = 188, keepalives =
-24133952, log = 0xfecc2b7c,
log_id = 0xfe8fbec0
"▒▒y`▒▒\223\060▒▒\207\220▒▒\222▒▒\217▒<", current_thread =
0xfecc2aec, slaves = 0x0, master = 0x0, ctx = 0xfecc8b18,
suspended_baton = 0xfe8fbf20, requests = 0x0, empty = 0x0,
filters = 0x56a136d3, async_filter = 335104}
wctx =
cond = 0x4
backend = 0x0
hc =
r = 0x6ed21e3c
method =
#2 hc_check (worker=0xf9328, now=1453405907335111, p=0x11ff38,
ctx=0xe3818) at modules/proxy/mod_proxy_hcheck.c:757
s = 0xcc290
rv =
#3 hc_watchdog_callback (state=, data=0xe3818,
pool=) at modules/proxy/mod_proxy_hcheck.c:839
n = 0
workers = 0xf8bb0
worker = 0xf9328
i =
rv =
now =
balancer =
ctx = 0xe3818
s =
conf =
p = 0x11ff38
#4 0xfecc2d08 in wd_worker (thread=, data=0xd3710) at
modules/core/mod_watchdog.c:203
ctx = 0x11df30
curr = 1453405907235104
wl = 0xd37f8
w = 0xd3710
rv =
locked = 1
probed =
inited = 0
mpmq_s = 1
#5 0xff095468 in dummy_worker (opaque=0x107f88) at
apr-1.5.2/threadproc/unix/thread.c:142
thread = 0x107f88
Config in global server:
ProxyHCExpr ok234 {%{REQUEST_STATUS} =~ /^[234]/}
ProxyPass "/" "balancer://mycluster/"
BalancerMember "http://www.kippdata.de/"; \
hcexpr=ok234 hcmethod=GET hcinterval=10
Regards,
Rainer
Am 21.01.2016 um 20:59 schrieb Rainer Jung:
Probably an alignment problem, but I don't immediately see how:
at modules/proxy/mod_proxy_hcheck.c:410
wctx = 0x70706461
So this is not an address usable for a pointer.
But wctx comes from hc->s->context:
{name = "f9328\000/www.kippdata.de/", '\000' , scheme
= "http", '\000' , hostname = "www.kippdata.de",
'\000' ,
route = '\000' , redirect = '\000' , flusher = '\000' , uds_path = '\000' ,
hcuri = '\000' , hcexpr = '\000' , "\001", lbset = 0, retries = 0, lbstatus = 0, lbfactor = 1,
min = 0, smax = 1, hmax = 0,
flush_wait = 1, index = 0, passes = 1815278865, pcount = -1571419579,
fails = 3, fcount = 0, hash = {def = 2239854630, fnv = 2173387474},
status = 10,
flush_packets = flush_off, method = GET, updated = 0, error_time =
6000, ttl = 0, retry = 0, timeout = 0, acquire = 0, ping_timeout =
3000, conn_timeout = 0,
interval = 0, recv_buffer_size = 5242880, io_buffer_size = 0, elected
= 0, busy = 0, port = 80, transferred = 536870912, read =
3400067598672358249, context = 0x70706461,
keepalive = 0, disablereuse = 0, is_address_reusable = 0, retry_set =
1, timeout_set = 0, acquire_set = 1, ping_timeout_set = 0,
conn_timeout_set = 0,
recv_buffer_size_set = 0, io_buffer_size_set = 1, keepalive_set = 1,
disablereuse_set = 0, was_malloced = 0, is_name_matchable = 0}
and the context member gets set in line 403:
hc->context = wctx;
If I add a debug statement there the address looks different and fine. I
don't immediately see where the address changes. Sparc is known to be
picky about alignment. One might not observe this on x86.
Regards,
Rainer
Re: Bus error for proxy_hcheck on Solaris
That is weird... does this help?
hc->context = (void *)wctx;
??
> On Jan 21, 2016, at 2:59 PM, Rainer Jung wrote:
>
> Probably an alignment problem, but I don't immediately see how:
>
> at modules/proxy/mod_proxy_hcheck.c:410
>wctx = 0x70706461
> So this is not an address usable for a pointer.
>
> But wctx comes from hc->s->context:
>
> {name = "f9328\000/www.kippdata.de/", '\000' , scheme =
> "http", '\000' , hostname = "www.kippdata.de", '\000'
> ,
> route = '\000' , redirect = '\000' ,
> flusher = '\000' , uds_path = '\000' ,
> hcuri = '\000' , hcexpr = '\000' ,
> "\001", lbset = 0, retries = 0, lbstatus = 0, lbfactor = 1, min = 0, smax
> = 1, hmax = 0,
> flush_wait = 1, index = 0, passes = 1815278865, pcount = -1571419579, fails
> = 3, fcount = 0, hash = {def = 2239854630, fnv = 2173387474}, status = 10,
> flush_packets = flush_off, method = GET, updated = 0, error_time = 6000,
> ttl = 0, retry = 0, timeout = 0, acquire = 0, ping_timeout = 3000,
> conn_timeout = 0,
> interval = 0, recv_buffer_size = 5242880, io_buffer_size = 0, elected = 0,
> busy = 0, port = 80, transferred = 536870912, read = 3400067598672358249,
> context = 0x70706461,
> keepalive = 0, disablereuse = 0, is_address_reusable = 0, retry_set = 1,
> timeout_set = 0, acquire_set = 1, ping_timeout_set = 0, conn_timeout_set = 0,
> recv_buffer_size_set = 0, io_buffer_size_set = 1, keepalive_set = 1,
> disablereuse_set = 0, was_malloced = 0, is_name_matchable = 0}
>
> and the context member gets set in line 403:
>
> hc->context = wctx;
>
> If I add a debug statement there the address looks different and fine. I
> don't immediately see where the address changes. Sparc is known to be picky
> about alignment. One might not observe this on x86.
>
> Regards,
>
> Rainer
Re: Bus error for proxy_hcheck on Solaris
Hold in a tic... I don't use the context field in ->s.
Re: Bus error for proxy_hcheck on Solaris
I found one place, which we should never hit, that does adjust hc->context, but that's only if the bal-mgr changes settings, which so far it can't. Anyone, removed that.
Re: Bus error for proxy_hcheck on Solaris
Based on your stack, then that was the section you hit. But I have no idea how you hit it. The test is: if (hc->s->method != worker->s->method) but neither is ever changed :/ > On Jan 21, 2016, at 3:22 PM, Jim Jagielski wrote: > > I found one place, which we should never hit, that > does adjust hc->context, but that's only if the > bal-mgr changes settings, which so far it can't. > > Anyone, removed that.
Re: Bus error for proxy_hcheck on Solaris
At least it works now, does probing and also hc() works. Cool. Rainer Am 21.01.2016 um 21:28 schrieb Jim Jagielski: Based on your stack, then that was the section you hit. But I have no idea how you hit it. The test is: if (hc->s->method != worker->s->method) I'll have a look. but neither is ever changed :/ On Jan 21, 2016, at 3:22 PM, Jim Jagielski wrote: I found one place, which we should never hit, that does adjust hc->context, but that's only if the bal-mgr changes settings, which so far it can't. Anyone, removed that.
Re: Bus error for proxy_hcheck on Solaris
I added logging to that section... let me know if you see 'Updating hc worker ...' (it's @ DEBUG) because, if you do, something is wonky. > On Jan 21, 2016, at 3:53 PM, Rainer Jung wrote: > > At least it works now, does probing and also hc() works. Cool. > > Rainer > > Am 21.01.2016 um 21:28 schrieb Jim Jagielski: >> Based on your stack, then that was the section you hit. But I >> have no idea how you hit it. The test is: >> >> if (hc->s->method != worker->s->method) > > I'll have a look. > >> but neither is ever changed :/ >> >>> On Jan 21, 2016, at 3:22 PM, Jim Jagielski wrote: >>> >>> I found one place, which we should never hit, that >>> does adjust hc->context, but that's only if the >>> bal-mgr changes settings, which so far it can't. >>> >>> Anyone, removed that.
Re: Bus error for proxy_hcheck on Solaris
On Thu, Jan 21, 2016 at 9:53 PM, Rainer Jung wrote: > > Am 21.01.2016 um 21:28 schrieb Jim Jagielski: >> >> Based on your stack, then that was the section you hit. But I >> have no idea how you hit it. The test is: >> >> if (hc->s->method != worker->s->method) > > > I'll have a look. Could it be that sizeof(flush_packets) == 1 in struct proxy_worker_shared (still 'apr_time_t updated' would be aligned but possibly not 'method')? What's APR_OFFSETOF(proxy_worker_shared, method)? (Btw, shouldn't new 'method' field go to the end of the struct?)
