Hi again Samisa, And, also, I don't have any similar database accessing logic etc. Therefore, it is hard to conclude without seeing the actual code. Also, the user seems to be using a modified version of "http_server_main.c", it is actually a cpp file if you go through that portion which I highlighted in his log. May be some issues due to not handling concurrency properly could also lead him into trouble.
However, what I can say is that there was a bug if accept(2) was to fail, according to the tests I did. And, according to literature available online that can fail. Plus, we do have a mechanism of retrying, if it does fail which did not work due to the segfault. Regards, Senaka >> Were you able to re-create the problem without passing -1 to accept? > > No I was not able to make accept fail without passing -1. Added a printf > there to print any failure in accept and it didn't. However, ran into the > segfault issue when accept actually failed, which I had to do manually. > > I did test with 1000 back-2-back mtom requests in 5 terminals, and also > 1000 repeated calls to an echo client that sends 1000 back-2-back > requests, and in 4 terminals. > > None of these cases failed. There were occasional hangs but still it > didn't result in any failure on either client or server side. > > However, I didn't have any way to test multiple requests made to a single > port from multiple machines. That may be a different scenario. > > Regards, > Senaka > >> >> Samisa... >> >> Senaka Fernando wrote: >>>> Senaka Fernando wrote: >>>> >>>>> Hi Samisa, >>>>> >>>>> I fixed two bugs regarding stream handling inside >>>>> simple_http_svr_conn.c. >>>>> This fixes an issue of this nature. >>>>> >>>>> Say, we have two server threads that try to serve requests offered to >>>>> the >>>>> axis2 server. Now, according to what is intended, one must serve and >>>>> the >>>>> other must wait until it can serve. The previous implementation >>>>> caused >>>>> this to fail with a segfault. But, with the fix, it will rather not >>>>> crash >>>>> but, loop until it gets a chance to serve the request. >>>>> >>>>> >>>> Did you test before the fix and did it seg fault for you? Can you >>>> please >>>> explain the scenario under which it seg faulted? If it is a problem, I >>>> too should be able to re-create it. >>>> >>>> And how did you solve the problem? By adding a sleep? >>>> >>> >>> I made the accept(2) call fail. By passing a -1 to it. Once that >>> failed, >>> there were two places that segfaulted which I already fixed, few >>> minutes >>> ago. No, I didn't add a sleep(). I simply prevented the segfault by >>> adding >>> necessary is_null checks before calling methods. Then, the server just >>> kept on spawning threads and freeing them as it can't serve the >>> request. >>> This is the expected behaviour in http_svr_thread.c I believe. >>> >>> In a real world scenario, a situation similar to a -1 passed to >>> accept(2) >>> will be temporal. So a respawned thread down the line should server the >>> request without any problem. >>> >>> Passing -1 to accept was done by, >>> >>> Index: util/src/network_handler.c >>> =================================================================== >>> --- util/src/network_handler.c (revision 634578) >>> +++ util/src/network_handler.c (working copy) >>> @@ -221,7 +221,7 @@ >>> AXIS2_ENV_CHECK(env, AXIS2_CRITICAL_FAILURE); >>> >>> cli_len = sizeof(cli_addr); >>> - cli_socket = accept(svr_socket, (struct sockaddr *) &cli_addr, >>> &cli_len); >>> + cli_socket = accept(-1, (struct sockaddr *) &cli_addr, &cli_len); >>> if (cli_socket < 0) >>> AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, >>> "[Axis2][network_handler] Socket accept \ >>> --- >>> >>> I'm not sure whether this is what really lead to the core dump, or was >>> it >>> something else. >>> >>> This was all that I could try on based on the less informative >>> valgrind.log that was sent. :(... >>> >>> Also i did test 1000 back-to-back requests and they all were served >>> without any issues. >>> >>> Regards, >>> Senaka >>> >>> >>>> Samisa... >>>> >>>> >>>>> Could this solve the core dump issue? >>>>> >>>>> Regards, >>>>> Senaka >>>>> >>>>> >>>>> >>>>>> Raghavendra SM wrote: >>>>>> >>>>>> >>>>>>> As expected, when I put a delay of about 50 milliseconds between >>>>>>> the >>>>>>> requests there is no dump. >>>>>>> >>>>>>> But we would still like to know the following, >>>>>>> + Isn't Axis capable of handling such huge bombardment of requests >>>>>>> (of >>>>>>> the order of 1000's) one after another without any delay between >>>>>>> the >>>>>>> requests? Has anyone tried such client before? >>>>>>> >>>>>>> >>>>>>> >>>>>> Yes it is supposed to. We have done performance testing that proved >>>>>> that >>>>>> Axis2/C can serve more than 300 rps. >>>>>> >>>>>> >>>>>> >>>>>>> + Can we make out anything out of the valgrind report or the axis >>>>>>> logs >>>>>>> that were attached? >>>>>>> >>>>>>> >>>>>>> >>>>>> Unfortunately not. When valgrind runs, it makes the system slow, >>>>>> that >>>>>> prevents the system crashing in situations like that you are >>>>>> experiencing. To solve this kind of a problem, it is a must that you >>>>>> provide us with some sample code that can be used to reproduce the >>>>>> problem. Without having a way to re-produce the situation, there is >>>>>> no >>>>>> way to fix this problem - it could well be a problem in Axis2/C as >>>>>> well >>>>>> as it could also be in the code that you use. >>>>>> >>>>>> Samisa... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Regards, >>>>>>> ~raghav >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Senaka Fernando [mailto:[EMAIL PROTECTED] >>>>>>> Sent: Thursday, March 06, 2008 4:22 PM >>>>>>> To: Apache AXIS C User List >>>>>>> Subject: RE: Core dumps in Axis libraries >>>>>>> >>>>>>> Hi Raghav, >>>>>>> >>>>>>> Can you please try this with --enable-guthilla=yes, and send the >>>>>>> dump.log, >>>>>>> and also the appropriate log_files generated by the server/client. >>>>>>> >>>>>>> There also can be issues with your "condition variable" being used >>>>>>> in >>>>>>> multiple threads. >>>>>>> >>>>>>> It also seems that if you pause between your requests you might get >>>>>>> this >>>>>>> solved. According to what you say, when you increase the log_level, >>>>>>> obviously you waste some time there which might be the solution >>>>>>> rather. >>>>>>> You can use the AXIS2_SLEEP() and AXIS2_USLEEP() inside Axis2/C to >>>>>>> pause >>>>>>> between requests. You can get an idea on how to use these functions >>>>>>> by >>>>>>> taking a look at the Axis2/C user_guide/client samples. >>>>>>> >>>>>>> Regards, >>>>>>> Senaka >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> As suggested, I tried using axis2c-1.3.0 both with and without >>>>>>>> --enable-guthilla=yes. But I still see different dumps >>>>>>>> consistently. >>>>>>>> Please find the attached file with back-traces. >>>>>>>> >>>>>>>> Are you using the same service client instance from multiple >>>>>>>> threads? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>>> We have a axis thread pool with default number of threads and a >>>>>>>>>>> set >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> of our own threads(about 10, lets call them "database threads"). >>>>>>>> + On receiving a HTTP soap request, axis thread invokes the >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> appropriate >>>>>>> >>>>>>> >>>>>>> >>>>>>>> function that is bound to our logic. >>>>>>>> + Now, the axis thread requests a "database thread" to perform the >>>>>>>> appropriate query and waits on a condition variable. >>>>>>>> + our "db thread" performs the necessary query and signals on that >>>>>>>> condition variable so that axis thread shall take the control >>>>>>>> over. >>>>>>>> + axis thread will create the om output and dispatch as usual. >>>>>>>> >>>>>>>> My doubts: >>>>>>>> + does this design has any apparent drawback when there is a >>>>>>>> flurry >>>>>>>> of >>>>>>>> back-to-back soap requests? >>>>>>>> + Is there a need to increase the axis thread pool size? If yes, >>>>>>>> how >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> to >>>>>>> >>>>>>> >>>>>>> >>>>>>>> do it? >>>>>>>> + As told before, the dump is seen during a particular scenario >>>>>>>> only. >>>>>>>> i.e: while trying to add a list of already existing users to my >>>>>>>> database. For each user there is a SOAP request generated and the >>>>>>>> requests are sequential, i.e the second request is sent only after >>>>>>>> the >>>>>>>> response for previous request is received. >>>>>>>> + if I increase the log level of my own module that uses axis2c, >>>>>>>> there >>>>>>>> is no dump. On increasing the log level, my module writes quite a >>>>>>>> bit >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> of >>>>>>> >>>>>>> >>>>>>> >>>>>>>> information to my log file. >>>>>>>> >>>>>>>> Your help is appreciated. >>>>>>>> >>>>>>>> Thanks & Regards, >>>>>>>> ~raghav >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Samisa Abeysinghe [mailto:[EMAIL PROTECTED] >>>>>>>> Sent: Wednesday, March 05, 2008 7:25 PM >>>>>>>> To: Apache AXIS C User List >>>>>>>> Subject: Re: Core dumps in Axis libraries >>>>>>>> >>>>>>>> Raghavendra SM wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Thanks Manjula, >>>>>>>>> >>>>>>>>> But thats not a viable option for us. We would still want to >>>>>>>>> continue >>>>>>>>> using axis2C-1.0 for some more time. >>>>>>>>> >>>>>>>>> Are these dumps well known? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> No. >>>>>>>> >>>>>>>> Are you using the same service client instance from multiple >>>>>>>> threads? >>>>>>>> >>>>>>>> Samisa... >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> Samisa Abeysinghe >>>>>> Software Architect; WSO2 Inc. >>>>>> >>>>>> http://www.wso2.com/ - "Oxygenating the Web Service Platform." >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>> >>>>>> >>>>>> >>>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>> >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> Samisa Abeysinghe >>>> Software Architect; WSO2 Inc. >>>> >>>> http://www.wso2.com/ - "Oxygenating the Web Service Platform." >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>> >>>> >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >>> >>> >> >> >> -- >> Samisa Abeysinghe >> Software Architect; WSO2 Inc. >> >> http://www.wso2.com/ - "Oxygenating the Web Service Platform." >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
