Re: core dumps in axis2_raw_xml_in_out_msg_recv_invoke_business_logic_sync()
Hi Eric,
What is the Axis2/C version are you using? And is your service storing
and accessing any data between requests?
Thanks,
-Manjula.
On Tue, 2009-06-09 at 13:32 -0500, Haszlakiewicz, Eric wrote:
> I'm running axis as a module in Apache httpd 2.2.9, running in pre-fork
> mode. Occasionally, when I start up the server, the first request sent
> causes the httpd process to crash. Has anyone seen anything like this
> before? I don't have much time to debug this right now, but here's some
> possibly useful info:
>
> httpd crashes with a SIGSEGV. Running gdb on the core dump:
> (gdb) where
> #0 0x in ?? ()
> #1 0x00625ced in
> axis2_raw_xml_in_out_msg_recv_invoke_business_logic_sync (
> msg_recv=0x8aa9d78, env=0x8ab5490, msg_ctx=0x8ab5560,
> new_msg_ctx=0x8ab6a40) at raw_xml_in_out_msg_recv.c:209
> #2 0x0062580a in axis2_msg_recv_invoke_business_logic
> (msg_recv=0x1671a4,
> env=0x8ab5490, in_msg_ctx=0x8ab5560, out_msg_ctx=0x8ab6a40)
> at msg_recv.c:481
> etc...
>
> line 209 of raw_xml_in_out_msg_recv.c is:
>result_node = AXIS2_SVC_SKELETON_INVOKE(svc_obj, env, om_node,
> new_msg_ctx);
>
> That macro does:
> #define AXIS2_SVC_SKELETON_INVOKE(svc_skeleton, env, node, msg_ctx) \
> ((svc_skeleton)->ops->invoke (svc_skeleton, env, node, msg_ctx))
>
> (gdb) print svc_obj
> $1 = (axis2_svc_skeleton_t *) 0x8aaa9b8
> (gdb) print svc_obj[0].ops[0]
> $10 = {init = 0, invoke = 0, on_fault = 0, free = 0, init_with_conf = 0}
>
> So, something didn't get initialized properly.
> If I restart apache, even without changing anything, things work fine.
>
> eric
core dumps in axis2_raw_xml_in_out_msg_recv_invoke_business_logic_sync()
I'm running axis as a module in Apache httpd 2.2.9, running in pre-fork
mode. Occasionally, when I start up the server, the first request sent
causes the httpd process to crash. Has anyone seen anything like this
before? I don't have much time to debug this right now, but here's some
possibly useful info:
httpd crashes with a SIGSEGV. Running gdb on the core dump:
(gdb) where
#0 0x in ?? ()
#1 0x00625ced in
axis2_raw_xml_in_out_msg_recv_invoke_business_logic_sync (
msg_recv=0x8aa9d78, env=0x8ab5490, msg_ctx=0x8ab5560,
new_msg_ctx=0x8ab6a40) at raw_xml_in_out_msg_recv.c:209
#2 0x0062580a in axis2_msg_recv_invoke_business_logic
(msg_recv=0x1671a4,
env=0x8ab5490, in_msg_ctx=0x8ab5560, out_msg_ctx=0x8ab6a40)
at msg_recv.c:481
etc...
line 209 of raw_xml_in_out_msg_recv.c is:
result_node = AXIS2_SVC_SKELETON_INVOKE(svc_obj, env, om_node,
new_msg_ctx);
That macro does:
#define AXIS2_SVC_SKELETON_INVOKE(svc_skeleton, env, node, msg_ctx) \
((svc_skeleton)->ops->invoke (svc_skeleton, env, node, msg_ctx))
(gdb) print svc_obj
$1 = (axis2_svc_skeleton_t *) 0x8aaa9b8
(gdb) print svc_obj[0].ops[0]
$10 = {init = 0, invoke = 0, on_fault = 0, free = 0, init_with_conf = 0}
So, something didn't get initialized properly.
If I restart apache, even without changing anything, things work fine.
eric
RE: nightly build failing? (trying to solve some memory leaks)
>-Original Message- >From: uthaiyashan...@gmail.com > >On Wed, Jun 3, 2009 at 9:09 PM, Haszlakiewicz, Eric > wrote: >> >> I'm trying to track down some memory leaks I'm seeing (just a >> axis2_stub_create_foo() immediately followed by >axis2_stub_free() leaks >> memory), > > >Can you give some more details about the leak, so that we can have a >look? We haven't come across such problem upto this time. Possibly it >might have been introduced recently. I updated to Axis2c-1.6.0, and AXIS2-NIGHTLY-367, and the memory leaks went away. Now I'm just getting a core dump in axis2_raw_xml_in_out_msg_recv_invoke_business_logic_sync() on the first request sent through, but only sometimes. I'll start a separate thread for that issue. >We have forwarded your mail to Axis2/Java list. They will help us to >fix the problem. ok, thanks. eric
RE: nightly build failing? (trying to solve some memory leaks)
>-Original Message- >From: Supun Kamburugamuva [mailto:supu...@gmail.com] >Sent: Wednesday, June 03, 2009 11:04 PM >To: Apache AXIS C User List >Subject: Re: nightly build failing? (trying to solve some memory leaks) > >Hi Eric, > >You need to go inside modules/tool/axis2-aar-maven-plugin, >modules/tool/axis2-mar-maven-plugin directories and build them >first, before building Axis2. > >Supun Don't tell me, I'm not trying to build it. I was just looking for the snapshot build for that day, and the standard build appears to be broken. (it's still broken as of right now, but the 3 week old snapshot seems to work for me) eric
Re: soap in client call contains gabage character -- A critical bug in guththila writer
Hi Shankar, Thanks much for paying attention to this. My colleague Frank Zhou has already filed a jila report with the input data and testing code. It also contains the suggested fix. The jila bug is AXIS2C-1375. We believe this is a blocker, as the soap message in pure English and it is small (~16K). This affects everyone using guththila (now the default). Thanks! Gordon From: Uthaiyashankar To: Apache AXIS C User List Cc: axis-c-...@ws.apache.org Sent: Tuesday, June 9, 2009 7:34:29 AM Subject: Re: soap in client call contains gabage character -- A critical bug in guththila writer Hi Gordon, I'll have a look. Can you create a jira and attach the patch and the test code? Regards, Shankar On Mon, Jun 8, 2009 at 11:40 PM, Gordon Brown wrote: > Can anyone in the development team please take a look at this one bug in > Guththila component? > At least the potential fix I provided in this message thread? > > == > The potential fix is to define GUTHTHILA_BUF_POS as the following: > > if ((_buffer)->pre_tot_data > _pos) > return ((_buffer)->buff[(_buffer)->cur_buff-1] + _pos); > else > return ((_buffer)->buff[(_buffer)->cur_buff] + _pos - > (_buffer)->pre_tot_data); > == > It is a problem in the buffer management, so without fixing this bug, users > should not use guththila at this point. > > Thanks! > Gordon > > From: Gordon Brown > To: axis-c-...@ws.apache.org; shan...@wso2.com; sam...@wso2.com > Cc: axis-c-user@ws.apache.org > Sent: Friday, June 5, 2009 2:15:42 PM > Subject: Re: soap in client call contains gabage character -- A critical bug > in guththila writer > > OK, since no one reply to my question, I have to debug the code and found > out that guththila has a bug in managing buffer when seriazlize thea axiom > tree (the soap structure) before actually send out the request, and I have a > potential fix. This is really a critical bug I think, so I hope some > developers can take a look at this problem. I am attaching the test > input data and code snappet to reproduce the problem. > > Basically, the bug occurs in guththila_xml_writer.c. > The guththila_xml_writer (I call it the soap serializer) maintains an array > of buffers dynamically when it writes the soap structure into the buffers. > The bug will occur in the following situation: > > Let's say I have an element 12345 > somewhere in the soap structure. Now before this element, there are lots of > other elements, and when the guththila_xml_writer trys to process this > element, the first buffer is ALMOST full, it does not have enough space > to write the whole element name (the start tag) into the > buffer, it has to create a new buffer, so it writes first buffer (still a few more bytes left empty), and writes "doDeleteFirst" > at the very beginning of the second buffer. > > The first buffer (Buffer length 16384): > -- > |** > The second buffer (Buffer length 32768): > --- > |doDeleteFirst-| > > As the second buffer becomes the current buffer, when the writer trys to > process the end tag (), it uses an elem stack to track > the namespace prefix and localname as in the following code: (starting from > line 1396) > > > elem->name = guththila_tok_list_get_token(&wr->tok_list, env); > > elem->prefix = guththila_tok_list_get_token(&wr->tok_list, env); > > elem->name->start = GUTHTHILA_BUF_POS(wr->buffer, elem_start); > > elem->name->size = elem_len; > > elem->prefix->start = GUTHTHILA_BUF_POS(wr->buffer, > elem_pref_start); > > elem->prefix->size = pref_len; > > > The macro GUTHTHILA_BUF_POS is defined as this: > > #ifndef GUTHTHILA_BUF_POS > #define GUTHTHILA_BUF_POS(_buffer, _pos) > ((_buffer).buff[(_buffer).cur_buff] + _pos - (_buffer).pre_tot_data) > #endif > The bug occurs when it calcuate elem->prefix->start = > GUTHTHILA_BUF_POS(wr->buffer, elem_pref_start): > > The elem_pref_start has a value of 16375, the pre_tot_data has a value of > 16379 (the first buffer length is 16384), they are calculated based on the > first buffer data, but the current buffer is the second one, so > elem->prefix->start points to gabage! > > I hope this makes sense to you. Use my test case you will see this quickly. > When you run the same XML data I attached, first set a break point at line > 392 in the file guththila_xml_writer_wrapper, and set the hit count as 514 > in the break properties (the 514th element in ), then > debug step by step. > > The potential fix is to define GUTHTHILA_BUF_POS as the following: >
RE : soap in client call contains gabage character -- A critical bug in guththila writer
Frank Zhou create JIRA AXIS2C-1375 for this already HTH -Message d'origine- De : uthaiyashan...@gmail.com [mailto:uthaiyashan...@gmail.com] De la part de Uthaiyashankar Envoyé : 9 juin 2009 10:34 À : Apache AXIS C User List Cc : axis-c-...@ws.apache.org Objet : Re: soap in client call contains gabage character -- A critical bug in guththila writer Hi Gordon, I'll have a look. Can you create a jira and attach the patch and the test code? Regards, Shankar On Mon, Jun 8, 2009 at 11:40 PM, Gordon Brown wrote: > Can anyone in the development team please take a look at this one bug > in Guththila component? At least the potential fix I provided in this > message thread? > > == > The potential fix is to define GUTHTHILA_BUF_POS as the following: > > if ((_buffer)->pre_tot_data > _pos) > return ((_buffer)->buff[(_buffer)->cur_buff-1] + _pos); > else > return ((_buffer)->buff[(_buffer)->cur_buff] + _pos - > (_buffer)->pre_tot_data); == > It is a problem in the buffer management, so without fixing this bug, users > should not use guththila at this point. > > Thanks! > Gordon > > From: Gordon Brown > To: axis-c-...@ws.apache.org; shan...@wso2.com; sam...@wso2.com > Cc: axis-c-user@ws.apache.org > Sent: Friday, June 5, 2009 2:15:42 PM > Subject: Re: soap in client call contains gabage character -- A > critical bug in guththila writer > > OK, since no one reply to my question, I have to debug the code and > found out that guththila has a bug in managing buffer when seriazlize > thea axiom tree (the soap structure) before actually send out the > request, and I have a potential fix. This is really a critical bug I > think, so I hope some developers can take a look at this problem. I am > attaching the test input data and code snappet to reproduce the > problem. > > Basically, the bug occurs in guththila_xml_writer.c. > The guththila_xml_writer (I call it the soap serializer) maintains an > array of buffers dynamically when it writes the soap structure into > the buffers. The bug will occur in the following situation: > > Let's say I have an element > 12345 > somewhere in the soap structure. Now before this element, there are lots of > other elements, and when the guththila_xml_writer trys to process this > element, the first buffer is ALMOST full, it does not have enough space > to write the whole element name (the start tag) into the > buffer, it has to create a new buffer, so it writes first buffer (still a few more bytes left empty), and writes "doDeleteFirst" > at the very beginning of the second buffer. > > The first buffer (Buffer length 16384): > -- > > |** > The second buffer (Buffer length 32768): > -- > - > |doDeleteFirst-- ---| > > As the second buffer becomes the current buffer, when the writer trys > to process the end tag (), it uses an elem stack > to track the namespace prefix and localname as in the following code: > (starting from line 1396) > > > elem->name = guththila_tok_list_get_token(&wr->tok_list, > env); > > elem->prefix = guththila_tok_list_get_token(&wr->tok_list, > env); > > elem->name->start = GUTHTHILA_BUF_POS(wr->buffer, > elem_start); > > elem->name->size = elem_len; > > elem->prefix->start = GUTHTHILA_BUF_POS(wr->buffer, > elem_pref_start); > > elem->prefix->size = pref_len; > > > The macro GUTHTHILA_BUF_POS is defined as this: > > #ifndef GUTHTHILA_BUF_POS > #define GUTHTHILA_BUF_POS(_buffer, _pos) > ((_buffer).buff[(_buffer).cur_buff] + _pos - (_buffer).pre_tot_data) > #endif The bug occurs when it calcuate elem->prefix->start = > GUTHTHILA_BUF_POS(wr->buffer, elem_pref_start): > > The elem_pref_start has a value of 16375, the pre_tot_data has a value > of 16379 (the first buffer length is 16384), they are calculated based > on the first buffer data, but the current buffer is the second one, so > elem->prefix->start points to gabage! > > I hope this makes sense to you. Use my test case you will see this > quickly. When you run the same XML data I attached, first set a break > point at line 392 in the file guththila_xml_writer_wrapper, and set > the hit count as 514 in the break properties (the 514th element in > ), then debug step by step. > > The potential fix is to define GUTHTHILA_BUF_POS as the following: > > if ((_buffer)->pre_tot_data > _pos) > return ((_buffer)->buff[(_buffer)->cur_buff-1] + _pos); > else > return ((_buffer)->buff[(_buffer)->cur_buff] + _pos - > (_buffer)->pre_tot_data); GUTHTHILA_BUF_POS
Re: soap in client call contains gabage character -- A critical bug in guththila writer
Hi Gordon, I'll have a look. Can you create a jira and attach the patch and the test code? Regards, Shankar On Mon, Jun 8, 2009 at 11:40 PM, Gordon Brown wrote: > Can anyone in the development team please take a look at this one bug in > Guththila component? > At least the potential fix I provided in this message thread? > > == > The potential fix is to define GUTHTHILA_BUF_POS as the following: > > if ((_buffer)->pre_tot_data > _pos) > return ((_buffer)->buff[(_buffer)->cur_buff-1] + _pos); > else > return ((_buffer)->buff[(_buffer)->cur_buff] + _pos - > (_buffer)->pre_tot_data); > == > It is a problem in the buffer management, so without fixing this bug, users > should not use guththila at this point. > > Thanks! > Gordon > > From: Gordon Brown > To: axis-c-...@ws.apache.org; shan...@wso2.com; sam...@wso2.com > Cc: axis-c-user@ws.apache.org > Sent: Friday, June 5, 2009 2:15:42 PM > Subject: Re: soap in client call contains gabage character -- A critical bug > in guththila writer > > OK, since no one reply to my question, I have to debug the code and found > out that guththila has a bug in managing buffer when seriazlize thea axiom > tree (the soap structure) before actually send out the request, and I have a > potential fix. This is really a critical bug I think, so I hope some > developers can take a look at this problem. I am attaching the test > input data and code snappet to reproduce the problem. > > Basically, the bug occurs in guththila_xml_writer.c. > The guththila_xml_writer (I call it the soap serializer) maintains an array > of buffers dynamically when it writes the soap structure into the buffers. > The bug will occur in the following situation: > > Let's say I have an element 12345 > somewhere in the soap structure. Now before this element, there are lots of > other elements, and when the guththila_xml_writer trys to process this > element, the first buffer is ALMOST full, it does not have enough space > to write the whole element name (the start tag) into the > buffer, it has to create a new buffer, so it writes first buffer (still a few more bytes left empty), and writes "doDeleteFirst" > at the very beginning of the second buffer. > > The first buffer (Buffer length 16384): > -- > |** > The second buffer (Buffer length 32768): > --- > |doDeleteFirst-| > > As the second buffer becomes the current buffer, when the writer trys to > process the end tag (), it uses an elem stack to track > the namespace prefix and localname as in the following code: (starting from > line 1396) > > > elem->name = guththila_tok_list_get_token(&wr->tok_list, env); > > elem->prefix = guththila_tok_list_get_token(&wr->tok_list, env); > > elem->name->start = GUTHTHILA_BUF_POS(wr->buffer, elem_start); > > elem->name->size = elem_len; > > elem->prefix->start = GUTHTHILA_BUF_POS(wr->buffer, > elem_pref_start); > > elem->prefix->size = pref_len; > > > The macro GUTHTHILA_BUF_POS is defined as this: > > #ifndef GUTHTHILA_BUF_POS > #define GUTHTHILA_BUF_POS(_buffer, _pos) > ((_buffer).buff[(_buffer).cur_buff] + _pos - (_buffer).pre_tot_data) > #endif > The bug occurs when it calcuate elem->prefix->start = > GUTHTHILA_BUF_POS(wr->buffer, elem_pref_start): > > The elem_pref_start has a value of 16375, the pre_tot_data has a value of > 16379 (the first buffer length is 16384), they are calculated based on the > first buffer data, but the current buffer is the second one, so > elem->prefix->start points to gabage! > > I hope this makes sense to you. Use my test case you will see this quickly. > When you run the same XML data I attached, first set a break point at line > 392 in the file guththila_xml_writer_wrapper, and set the hit count as 514 > in the break properties (the 514th element in ), then > debug step by step. > > The potential fix is to define GUTHTHILA_BUF_POS as the following: > > if ((_buffer)->pre_tot_data > _pos) > return ((_buffer)->buff[(_buffer)->cur_buff-1] + _pos); > else > return ((_buffer)->buff[(_buffer)->cur_buff] + _pos - > (_buffer)->pre_tot_data); > GUTHTHILA_BUF_POS is used everywhere, so I really hope some developer can > take over this case and fix it! > > Thanks! > Gordon > > > From: Gordon Brown > To: axis-c-user@ws.apache.org > Cc: axis-c-...@ws.apache.org > Sent: Wednesday, June 3, 2009 12:49:21 AM > Subject: soap in client call contains gabage character -- Very very puzzling > > Hi All, > > I need urgent help with a
