Thanks for the suggestions. I have been playing with the timeouts to trigger the problem. Typically I would like to have them around 15minutes....and yes that is apparently not sufficient for the load test I am doing.
The setup is three machines. One with tomcat running that connects to a 2nd one running MySQL. THe third is running jMeter (300 'users'). That said, I am baffled by the fact that some connections take >15minutes. >From the start, I included a dbcp pool to try and optimize the db resource usage, but I am starting to wonder whether that is not causing more problems than it is solving. I'll try to isolate the dbcp pool code and see how that behaves. In the end I guess it can also be the db server. For now I am going to try to not have timeouts during doGet processing, rather than try to deal with them. Something I did not understand from the suggestions is > 1. You make DB requests not only from tomcat which is suboptimal > anyway, but also directly from the servlet, which would mean that you > utterly need to refactor your application anyway. I am not sure I understand the difference "from tomcat" versus "from the servlet" Again, all suggestions warmly welcomed! Peter Leon Rosenberg-3 wrote: > > On 12/30/06, Peter Coppens <[EMAIL PROTECTED]> wrote: >> >> Actually it just seems to be related to the fact that under heavy load >> the db >> connection starts to take longer than the timeout. > > Thats a lot. What is your timeout configured to? > >> >> Apparently, a call to request.getSession() somewhere in the middel of the >> doGet processing will also trigger invalidating the session, which is >> kind >> of a nuisance as one should then apparently constantly check whether the >> session has not expired during request processing. >> >> I assume this is Servlet spec compliant, but it does seem to make my life >> rather complex. >> >> Would there be anyone having any suggestions to deal with this. > > We might need some more details on your application to give you > meaningful suggestions, but just from the hip I'd say following: > > 1. You make DB requests not only from tomcat which is suboptimal > anyway, but also directly from the servlet, which would mean that you > utterly need to refactor your application anyway. > > 2. Apparently your timeouts are way to low. On the other hand, it > can't be healthy if your requests last longer than hours (default > timeout for the session are 2 hours) > > 3. In case you can't change anything other, you could consider > starting a thread on the start of the request which just calls > session.get/put with dummy attributes periodically to prevent the > session from being timeouted. However this feature has to be > implemented carefully, since it can cause much harm with regard to > garbage collection if your thread instances sticks and keep references > to the session or similar. > > By the way, what do you mean by "heavy load"? > > Regards > Leon > >> >> Thanks, >> >> Peter >> >> >> Leon Rosenberg-3 wrote: >> > >> > On 12/30/06, Peter Coppens <[EMAIL PROTECTED]> wrote: >> >> >> >> I am gathering more evidence that this is related to a session >> expiring >> >> on >> >> one hand and a request being processed for that same session. >> >> >> >> I have been debugging the tomcat code a bit, and I have the >> *impression* >> >> that the expiration handling is not thread safe. It seems possible at >> >> first >> >> sight that the background processor decides a session is expired while >> at >> >> the same time another thread starts a request for that same session. >> >> >> >> I will try to debug a bit more and if I find more I will let you know. >> > >> > The question is whether the next request get the "right" session again >> > or not. I had the impression from your first post, that this is the >> > case: >> > Request A - Session 1 >> > Request B - Session 2 <-- which is wrong >> > Request C - Session 1 again. >> > >> > I observed this behaviour 3 years ago on a resin 2.1.x, but had not >> > enough time to debug it. >> > >> > regards >> > Leon >> >> >> >> Thanks, >> >> >> >> Peter >> >> >> >> PS What does "O/T" mean ? >> > >> > Off Topic >> > >> >> >> >> >> >> Martin Gainty wrote: >> >> > >> >> > Agreed >> >> > Once you have your use cases and test cases identified >> >> > >> >> > If you want the server to maintain its own side of the relationship >> >> > independent of client activity then you should consider container >> >> managed >> >> > persistence >> >> > More info avaialable at >> >> > >> >> >> http://www.javaworld.com/javaworld/jw-08-2006/jw-0828-persistence.html?page=6 >> >> > >> >> > Feel free to contact me offline as this is definitely O/T >> >> > Martin-- >> >> > >> >> >> --------------------------------------------------------------------------- >> >> > This e-mail message (including attachments, if any) is intended for >> the >> >> > use of the individual or entity to which it is addressed and may >> >> contain >> >> > information that is privileged, proprietary , confidential and >> exempt >> >> from >> >> > disclosure. If you are not the intended recipient, you are notified >> >> that >> >> > any dissemination, distribution or copying of this communication is >> >> > strictly prohibited. >> >> > >> >> >> --------------------------------------------------------------------------- >> >> > Le présent message électronique (y compris les pièces qui y sont >> >> annexées, >> >> > le cas échéant) s'adresse au destinataire indiqué et peut contenir >> des >> >> > renseignements de caractère privé ou confidentiel. Si vous n'êtes >> pas >> >> le >> >> > destinataire de ce document, nous vous signalons qu'il est >> strictement >> >> > interdit de le diffuser, de le distribuer ou de le reproduire. >> >> > ----- Original Message ----- >> >> > From: "Leon Rosenberg" <[EMAIL PROTECTED]> >> >> > To: "Tomcat Users List" <users@tomcat.apache.org> >> >> > Sent: Friday, December 29, 2006 6:31 AM >> >> > Subject: Re: session#getId changes during doGet invocation under >> heavy >> >> > load >> >> > >> >> > >> >> >> Do I understand it right, that you made it a reproduceable >> testcase? >> >> >> If so, can we have a look on it? >> >> >> >> >> >> thank you >> >> >> Leon >> >> >> >> >> >> On 12/29/06, Peter Coppens <[EMAIL PROTECTED]> wrote: >> >> >>> >> >> >>> Thanks Chuck. >> >> >>> >> >> >>> I have done some further research and I have the impression that >> >> there >> >> >>> is >> >> >>> some kind of race condition where a session that is being removed >> >> >>> because of >> >> >>> a timeout is also handling requests. >> >> >>> >> >> >>> The loggings indicate that a session is destroyed but then >> >> nevertheless >> >> >>> a >> >> >>> doGet is invoked with a request parameter that refers to that >> timed >> >> out >> >> >>> session. >> >> >>> >> >> >>> If I crank up the timeout, seriously reducing the chances a >> session >> >> >>> times >> >> >>> out before it has completed all the client requests it is supposed >> to >> >> >>> handle >> >> >>> during the test, the problem does no longer occur. >> >> >>> >> >> >>> I am not sure where that leaves me. I am still uncertain as to >> what >> >> the >> >> >>> servlet is doing wrong. >> >> >>> >> >> >>> Would you (or anyone else) have any other comments on this? >> >> >>> >> >> >>> Thanks, >> >> >>> >> >> >>> Peter >> >> >>> >> >> >>> >> >> >>> >> >> >>> Caldarale, Charles R wrote: >> >> >>> > >> >> >>> >> From: Peter Coppens [mailto:[EMAIL PROTECTED] >> >> >>> >> Subject: session#getId changes during doGet invocation under >> >> >>> >> heavy load >> >> >>> >> >> >> >>> >> THe problem I run into is that, under heavy load, during the >> >> >>> >> doGet invocation for the login request the session attached >> >> >>> >> to the request is changed by some other thread (the value >> >> >>> >> returned from getId changes and attributes that are originally >> >> >>> >> set are removed). >> >> >>> > >> >> >>> > This is most likely an application problem - storing session- or >> >> >>> > request-specific data in an inappropriately scoped structure >> (e.g., >> >> a >> >> >>> > servlet field). Under light load, this won't hurt, since the >> >> >>> insertion >> >> >>> > and retrieval for a given request don't overlap any other >> requests. >> >> >>> As >> >> >>> > the load increases, so does the probability of an overlap and >> >> >>> erroneous >> >> >>> > retrieval of data for another request. >> >> >>> > >> >> >>> > - Chuck >> >> >>> > >> >> >>> > >> >> >>> > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE >> >> >>> PROPRIETARY >> >> >>> > MATERIAL and is thus for use only by the intended recipient. If >> you >> >> >>> > received this in error, please contact the sender and delete the >> >> >>> e-mail >> >> >>> > and its attachments from all computers. >> >> >>> > >> >> >>> > >> >> --------------------------------------------------------------------- >> >> >>> > To start a new topic, e-mail: users@tomcat.apache.org >> >> >>> > To unsubscribe, e-mail: [EMAIL PROTECTED] >> >> >>> > For additional commands, e-mail: [EMAIL PROTECTED] >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> >> >> >>> -- >> >> >>> View this message in context: >> >> >>> >> >> >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-under-heavy-load-tf2892448.html#a8085181 >> >> >>> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >>> >> >> >>> >> >> >>> >> --------------------------------------------------------------------- >> >> >>> To start a new topic, e-mail: users@tomcat.apache.org >> >> >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >> >> >>> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >>> >> >> >>> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To start a new topic, e-mail: users@tomcat.apache.org >> >> >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> >> >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> >> >> >> >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-under-heavy-load-tf2892448.html#a8097754 >> >> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To start a new topic, e-mail: users@tomcat.apache.org >> >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> >> >> > >> > --------------------------------------------------------------------- >> > To start a new topic, e-mail: users@tomcat.apache.org >> > To unsubscribe, e-mail: [EMAIL PROTECTED] >> > For additional commands, e-mail: [EMAIL PROTECTED] >> > >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-under-heavy-load-tf2892448.html#a8099384 >> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To start a new topic, e-mail: users@tomcat.apache.org >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > --------------------------------------------------------------------- > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/session-getId-changes-during-doGet-invocation-under-heavy-load-tf2892448.html#a8100719 Sent from the Tomcat - User mailing list archive at Nabble.com. --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
