Re: Uploading large files and session timeout
Thanks, Chris! I took the threaddump and found that Tomcat's http service thread is still blocked on the read from the client after we called the forward method. At least, that's how I interpreted this, but below is the particular thread's dump: http-443-1 daemon prio=6 tid=0x4c20b000 nid=0x1720 runnable [0x4f85f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293) at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789) - locked 0x0b925a00 (a java.lang.Object) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746) at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75) - locked 0x0b925a10 (a com.sun.net.ssl.internal.ssl.AppInputStream) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:735) at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:814) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) As suggested I took the dump a few times while the browser is waiting, and in all cases the http thread is blocked on the read. I took another dump after the browser has received the response, and the above mentioned thread is in wait state. I then ran my test with a couple of other browsers and found the following: Firefox is the only browser where this delay happens. IE and Chrome show the reply right away after forward is complete. That makes me wonder if FF is not setting the content-length header correctly (it is not using chunked as far as I can tell). The call to forward is the last thing the upload Servlet does, so there are no more cleanups we do in the code. We are using the commons-fileupload and making use of their streaming feature, so we/commons-upload do not create any temp files. I wonder if some one else out there can try this scenario and see if they can reproduce this. If you would like to see the full thread dumps, please let me know (they are not huge, only a few threads) if I can post it to the forum. Thanks. Sai Pullabhotla On Sun, Jul 10, 2011 at 9:27 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sai, On 7/9/2011 8:55 AM, Sai Pullabhotla wrote: I added the system property org.apache.catalina.session.StandardSession.ACTIVITY_CHECK and set it to true, and it appears to be preventing the session timeout. Glad to see it's working out for you. That's a good news. Some one told me that there might be some performance issues, but I'm not sure how significant they are. It was I who mentioned potential performance degradation. If you aren't in a super-high-concurrency situation, I don't think you'll notice. The call to forward from the Servlet finishes in no time. However, it takes about minute by the time the browser renders this JSP. While the browser is waiting for the reply, the session is getting destroyed. My question is, what is taking so long after the request is forwarded. Take a thread dump. Or, multiple thread dumps during this one-minute interval. They should tell the story. The code snippet we use to forward the request is - request.getRequestDispatcher(/MyJSP.jsp).forward(request, response); I've a debug line before this call and after this call, they both get printed right one after the other with no delay, but the browser does not get anything back for about a minute. This might be a keepalive timeout, or the JSP might not be flushing the buffer or something like that. If I upload fairly small files, the browser gets the response back almost right away. The delay in receiving the reply on large files is not new (or not caused by the new additional system property). Does your code do anything after the forward() call above? If you have clean-up code that, say, deletes the temporary files, etc. it's possible that it is delaying the return from your servlet's service() method and so Tomcat isn't indicating to the client that the transaction is complete. You should put a log message just before your return (or just before the end of the method if you have no returns in there) and see when that gets printed. That will at least tell you if the delay is in your code or elsewhere. Any ideas on what could be going on? Does Tomcat some how feeds (or tries to) the whole multipart
Re: Uploading large files and session timeout
A little more info on the org.apache.catalina.session.StandardSession.ACTIVITY_CHECK system property: The last time I said that the above property is keeping the session alive, I was only partially correct. The way it is working is - session is kept alive for the duration of the upload. The response is sent to the client indicating upload successful. However, the next request made immediately (I click on a button soon after the large upload) fails because the session is destroyed as soon as tomcat receives the request. What I'm rather looking for is - to keep the session alive for the duration of the timeout after a large upload. So if my session timeout is 1 minute, it would be nice if I can make a second request within a minute after a large upload which might have taken 5 minutes. I also tried the STRICT_COMPLIANCE system property and set it to true to see if that makes any difference, but noticed the same behavior. I then tried the programmatic approach as below: Upload Servlet retrieves the current session timeout and caches it in a local variable (of the service/dopost). Changes the session timeout to 5 minutes from 30 seconds (original, just for testing, not that we will ever use 30 seconds in production) After upload finishes, and before forwarding to the JSP, the servlet resets the timeout to cached timeout of 30 seconds. This did not work either, the subsequent request (right after a large upload which took a minute or two) fails with session timed out. Any ideas on how to make this work? Thanks. Sai Pullabhotla On Mon, Jul 11, 2011 at 8:29 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Thanks, Chris! I took the threaddump and found that Tomcat's http service thread is still blocked on the read from the client after we called the forward method. At least, that's how I interpreted this, but below is the particular thread's dump: http-443-1 daemon prio=6 tid=0x4c20b000 nid=0x1720 runnable [0x4f85f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293) at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789) - locked 0x0b925a00 (a java.lang.Object) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746) at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75) - locked 0x0b925a10 (a com.sun.net.ssl.internal.ssl.AppInputStream) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:735) at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:814) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) As suggested I took the dump a few times while the browser is waiting, and in all cases the http thread is blocked on the read. I took another dump after the browser has received the response, and the above mentioned thread is in wait state. I then ran my test with a couple of other browsers and found the following: Firefox is the only browser where this delay happens. IE and Chrome show the reply right away after forward is complete. That makes me wonder if FF is not setting the content-length header correctly (it is not using chunked as far as I can tell). The call to forward is the last thing the upload Servlet does, so there are no more cleanups we do in the code. We are using the commons-fileupload and making use of their streaming feature, so we/commons-upload do not create any temp files. I wonder if some one else out there can try this scenario and see if they can reproduce this. If you would like to see the full thread dumps, please let me know (they are not huge, only a few threads) if I can post it to the forum. Thanks. Sai Pullabhotla On Sun, Jul 10, 2011 at 9:27 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sai, On 7/9/2011 8:55 AM, Sai Pullabhotla wrote: I added the system property org.apache.catalina.session.StandardSession.ACTIVITY_CHECK and set it to true, and it appears to be preventing the session timeout. Glad to see it's working out for you. That's a good news. Some one told me that there might be some performance issues, but I'm not sure how significant they are. It was I who mentioned potential performance degradation. If you aren't in a super-high-concurrency
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sai, On 7/11/2011 9:29 AM, Sai Pullabhotla wrote: I took the threaddump and found that Tomcat's http service thread is still blocked on the read from the client after we called the forward method. At least, that's how I interpreted this, but below is the particular thread's dump: http-443-1 daemon prio=6 tid=0x4c20b000 nid=0x1720 runnable [0x4f85f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293) at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789) - - locked 0x0b925a00 (a java.lang.Object) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746) at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75) - locked 0x0b925a10 (a com.sun.net.ssl.internal.ssl.AppInputStream) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:735) at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:814) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) If it's waiting on a request line, it looks like a dangling keep-alive request to me. On the other hand, a web browser that got a complete response should be showing the page, even if it's got a keep-alive connection still open with the server. Firefox is the only browser where this delay happens. IE and Chrome show the reply right away after forward is complete. That makes me wonder if FF is not setting the content-length header correctly (it is not using chunked as far as I can tell). Try using LiveHttpHeaders, FireBug, or any number of extensions that can show you the HTTP request and response headers. If you need mroe details, you could use a packet-sniffer like Wireshark to see the actual bytes including chunked details, etc. The call to forward is the last thing the upload Servlet does, so there are no more cleanups we do in the code. We are using the commons-fileupload and making use of their streaming feature, so we/commons-upload do not create any temp files. Good to know. Is there some other component in the mix, like a web server or other proxy? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4bVU0ACgkQ9CaO5/Lv0PAo+QCeJYi21CzvZ9wjMTmjqTkAhWG1 h8QAoJB8UQjlWOFy5YutX6QopkxOP3Bq =8b5R -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
It seems like there are two quite different issues/discussions going on in this same thread, with the same subject line. It is a bit confusing, even if originally they relate to the same problem. Would it not be better to split this ? - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 André, On 7/11/2011 3:59 PM, André Warnier wrote: It seems like there are two quite different issues/discussions going on in this same thread, with the same subject line. It is a bit confusing, even if originally they relate to the same problem. Would it not be better to split this ? +1 - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4bWhwACgkQ9CaO5/Lv0PACbwCgrhZki+oyL26z3d0mQ/D9FdrZ XX8AoJvXzSwMumYuiGTjHdlM5F2sFIKx =8b2P -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
I agree. At this point, I'm not so concerned about the Firefox issue. I will start a separate thread on it later. I still would like to get some help on keeping the session alive for the duration of the configured timeout, after a response is sent for a large request. Any ideas will be greatly appreciated. Thanks. Sai Pullabhotla On Mon, Jul 11, 2011 at 3:16 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 André, On 7/11/2011 3:59 PM, André Warnier wrote: It seems like there are two quite different issues/discussions going on in this same thread, with the same subject line. It is a bit confusing, even if originally they relate to the same problem. Would it not be better to split this ? +1 - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4bWhwACgkQ9CaO5/Lv0PACbwCgrhZki+oyL26z3d0mQ/D9FdrZ XX8AoJvXzSwMumYuiGTjHdlM5F2sFIKx =8b2P -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
Sai Pullabhotla wrote: A little more info on the org.apache.catalina.session.StandardSession.ACTIVITY_CHECK system property: The last time I said that the above property is keeping the session alive, I was only partially correct. The way it is working is - session is kept alive for the duration of the upload. The response is sent to the client indicating upload successful. However, the next request made immediately (I click on a button soon after the large upload) fails because the session is destroyed as soon as tomcat receives the request. What I'm rather looking for is - to keep the session alive for the duration of the timeout after a large upload. So if my session timeout is 1 minute, it would be nice if I can make a second request within a minute after a large upload which might have taken 5 minutes. I also tried the STRICT_COMPLIANCE system property and set it to true to see if that makes any difference, but noticed the same behavior. I then tried the programmatic approach as below: Upload Servlet retrieves the current session timeout and caches it in a local variable (of the service/dopost). Changes the session timeout to 5 minutes from 30 seconds (original, just for testing, not that we will ever use 30 seconds in production) After upload finishes, and before forwarding to the JSP, the servlet resets the timeout to cached timeout of 30 seconds. This did not work either, the subsequent request (right after a large upload which took a minute or two) fails with session timed out. Any ideas on how to make this work? I think that you need to scroll back in this thread (to July 8), and re-read an answer which Charles provided to a previous question of mine. A partial answer resides in this property, which can be adjusted in Tomcat 7 (but apparently not in previous releases) : quote from : http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html org.apache.catalina.session.StandardSession.LAST_ACCESS_AT_START If this is true, the last accessed time for sessions will be calculated from the beginning of the previous request. If false, the last accessed time for sessions will be calculated from the end of the previous request. This also affects how the idle time is calculated. If org.apache.catalina.STRICT_SERVLET_COMPLIANCE is set to true, the default of this setting will be true, else the default value will be false. unquote To take your example above : a) initial conditions : - the standard session timeout is 30 seconds - the last access time in the session is set at the beginning of a request (as per the Servlet Spec, according to Charles) b) file upload request : - at time T, a file upload request starts. The session timeout is 30 seconds, and the last access time is set at T. - your servlet sets the session timeout to 5 minutes. - the file upload takes 3 minutes, so at the end of the file upload, the time is T+180 s. minutes - the servlet resets the session timeout to 30 s. - the file upload request is finished, the servlet stops c) a new request comes in at time T+181 s. : - the session timeout is (again) 30 seconds - the last access time stored in the session is T (see above) -- the timeout is exceeded, and the session is expired In other words, changing the session timeout during the file upload request processing achieved nothing. What is needed, is either : - while the file upload is ongoing, regularly update the session last access time, so that it never gets older than the current time + the timeout. But there seems to be no API for that. OR - arrange for the last access time to be updated /at the end/ of the file upload request, instead of the beginning. (which seems to be possible using the system property mentioned above, in Tomcat 7). BUT - I think that this is not enough, because imagine the following sequence : timeline Thread 1 Threads 2..n T : a file upload request A starts(nothing) T+29 s. file upload request A ongoing a new request B comes in T+60 s. file upload request A ongoing a new request C comes in T+180 s. file upload request A finishes session last access is updated to T+180 s. T+200 s. (nothing) a new request D comes in (in this, I assume that several requests can come in, in parallel, for the same session. Is that possible ?) When request B comes in, the last access time in the session is still T. That's fine, because it is not more than T+30 s., so the session is still valid. When request C comes in however, the time is T+60 s., which means that the session timeout (T+30 s.) is exceeded. So I guess that request C will get a new session. (And in addition, it will cancel the first session). But for request D, it should work again, since now the last access is T+180 s., and the request comes in at T+200 s., which is less than 30 seconds later than T+180 s. Now the
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 André, On 7/11/2011 4:54 PM, André Warnier wrote: I think that you need to scroll back in this thread (to July 8), and re-read an answer which Charles provided to a previous question of mine. A partial answer resides in this property, which can be adjusted in Tomcat 7 (but apparently not in previous releases) : quote from : http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html org.apache.catalina.session.StandardSession.LAST_ACCESS_AT_START Unfortunately, OP is using TC 6, where this configuration property is not available. I'm not sure what the behavior is in TC6... I'd have to read the code. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4bcboACgkQ9CaO5/Lv0PCLDwCgqlKDnuWS5Qhj47U8fWdb+w0B BrUAniE1/tpykyqimP62s9k95FreOyMg =t/Gc -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sai, On 7/9/2011 8:55 AM, Sai Pullabhotla wrote: I added the system property org.apache.catalina.session.StandardSession.ACTIVITY_CHECK and set it to true, and it appears to be preventing the session timeout. Glad to see it's working out for you. That's a good news. Some one told me that there might be some performance issues, but I'm not sure how significant they are. It was I who mentioned potential performance degradation. If you aren't in a super-high-concurrency situation, I don't think you'll notice. The call to forward from the Servlet finishes in no time. However, it takes about minute by the time the browser renders this JSP. While the browser is waiting for the reply, the session is getting destroyed. My question is, what is taking so long after the request is forwarded. Take a thread dump. Or, multiple thread dumps during this one-minute interval. They should tell the story. The code snippet we use to forward the request is - request.getRequestDispatcher(/MyJSP.jsp).forward(request, response); I've a debug line before this call and after this call, they both get printed right one after the other with no delay, but the browser does not get anything back for about a minute. This might be a keepalive timeout, or the JSP might not be flushing the buffer or something like that. If I upload fairly small files, the browser gets the response back almost right away. The delay in receiving the reply on large files is not new (or not caused by the new additional system property). Does your code do anything after the forward() call above? If you have clean-up code that, say, deletes the temporary files, etc. it's possible that it is delaying the return from your servlet's service() method and so Tomcat isn't indicating to the client that the transaction is complete. You should put a log message just before your return (or just before the end of the method if you have no returns in there) and see when that gets printed. That will at least tell you if the delay is in your code or elsewhere. Any ideas on what could be going on? Does Tomcat some how feeds (or tries to) the whole multipart request (after it is consumed by the Servlet) to the forwarded JSP? The multipart request is still available to the JSP (because you used a forward call) put it's already been parsed by the servlet so there shouldn't be anything else to read. I doubt it because, I did not notice any temp files being created where the request might have been cached. The memory usage was pretty low during the whole process so the request is definitely not cached in memory. Tomcat does not cache anything at all in these cases... everything will be done by your file-upload component. Time for more debugging. :) - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEUEARECAAYFAk4ZtukACgkQ9CaO5/Lv0PA6SwCXVK1r4bnZ/+fOxb4udH8Av6xo /gCgoKf6Z0QDe1q/Ps8zSI3ERX/VIR8= =mU2x -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
Thank you all for the input. I added the system property org.apache.catalina.session.StandardSession.ACTIVITY_CHECK and set it to true, and it appears to be preventing the session timeout. That's a good news. Some one told me that there might be some performance issues, but I'm not sure how significant they are. Now, one thing I noticed in the same test 800 MB file (client and server on the same system) with a session timeout of 30 seconds... After the Servlet received/stored all the files (which could be up to 5 files in a single upload request), it creates a Bean/List with status of each file (e.g. file 1 successful, file2 failed because a file with the same name already exists etc..). Sets this status bean in request scope, and forwards the request to a JSP. The call to forward from the Servlet finishes in no time. However, it takes about minute by the time the browser renders this JSP. While the browser is waiting for the reply, the session is getting destroyed. My question is, what is taking so long after the request is forwarded. The code snippet we use to forward the request is - request.getRequestDispatcher(/MyJSP.jsp).forward(request, response); I've a debug line before this call and after this call, they both get printed right one after the other with no delay, but the browser does not get anything back for about a minute. If I upload fairly small files, the browser gets the response back almost right away. The delay in receiving the reply on large files is not new (or not caused by the new additional system property). Any ideas on what could be going on? Does Tomcat some how feeds (or tries to) the whole multipart request (after it is consumed by the Servlet) to the forwarded JSP? I doubt it because, I did not notice any temp files being created where the request might have been cached. The memory usage was pretty low during the whole process so the request is definitely not cached in memory. Thanks. Sai Pullabhotla On Fri, Jul 8, 2011 at 4:09 PM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: André Warnier [mailto:a...@ice-sa.com] Subject: Re: Uploading large files and session timeout The do this cleanly, the servlet would need to call HttpSession.getMaxInactiveInterval() at the beginning, to save the existing value, then call HttpSession.setMaxInactiveInterval() to set a new one for the duration of the transfer, then process the upload, then at the end call HttpSession.setMaxInactiveInterval() to reset the old value, right ? Correct, but slightly more complicated due to the previously undisclosed possibility of concurrent uploads associated with a single session. Since the session expiration is determined in function of the MaxInactiveInterval and the lastAccessTime, and since Tomcat has no way to know at the beginning of a request, how long this request will take to process, I would think that the lastAccessTime would be updated *at the end* of a request, not at the beginning. Tomcat 7 (which is not what the OP is using) has these system properties to control session timeout behavior: StandardHostValve.ACCESS_SESSION StandardSession.ACTIVITY_CHECK StandardSession.LAST_ACCESS_AT_START (Determining the actions taken for each property setting is left as an exercise for the Tomcat documentation reader.) Tomcat 6 uses only the ACTIVITY_CHECK property, and session expiration is based only on when the most recent request arrived - as required by the spec. The properties in Tomcat 7 appear to have been added as workarounds for the shortcomings of the spec that you have pointed out. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
Sai Pullabhotla wrote: We have an application that uploads files using a Servlet deployed in Tomcat 6. While this works most of the times, occasionally we run into issues uploading large files. If the upload takes longer then the session timeout, the session gets invalidated right after the upload. Tis means no further requests are accepted unless the user logs back in. Is this the expected behavior? Is there any way to work around this and keep the session active? I guess one way to fix this is to have a large session timeout like an hour or two, but we prefer not to do that for obvious reasons. Responding in the absolute, if there is a session timeout functionality, then it must be based on a) a timeout value (a number of seconds e.g.) b) somewhere, an indicator of when the session was last active, which gets updated at some point So the problem would boil down to knowing where this last active time is stored, and update it regularly during the file upload. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
How large are the files in question, and how long until the timeout? My app does *a lot* of file uploading (and downloading), and I have not run across this in the years I've used Tomcat. That's been since v3, but maybe I've just never hit that limit. Also, are you using a library like the Apache Common's FileUpload ( http://commons.apache.org/fileupload/) or have you written your own? On Fri, Jul 8, 2011 at 10:25 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: We have an application that uploads files using a Servlet deployed in Tomcat 6. While this works most of the times, occasionally we run into issues uploading large files. If the upload takes longer then the session timeout, the session gets invalidated right after the upload. Tis means no further requests are accepted unless the user logs back in. Is this the expected behavior? Is there any way to work around this and keep the session active? I guess one way to fix this is to have a large session timeout like an hour or two, but we prefer not to do that for obvious reasons. Thanks in advance for your help. Sai Pullabhotla - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- Hell hath no limits, nor is circumscrib'd In one self-place; but where we are is hell, And where hell is, there must we ever be --Christopher Marlowe, *Doctor Faustus* (v, 121-24)
Re: Uploading large files and session timeout
Just to give more details... The session timeout setting is stored in our application's database. Admins can change the session timeout from the UI we provide. We did this to make it easy for our customers to set the desired timeout rather than telling them going into web.xml and updating the timeout. After a successful login we create the session, and set the timeout on each session that is created using a HTTPSessionListener that is attached to the context. As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Perhaps can be done if we find the Tomcat's internal session object, but prefer not to do it. Am I correct or am I missing something? Sai Pullabhotla On Fri, Jul 8, 2011 at 9:31 AM, André Warnier a...@ice-sa.com wrote: Sai Pullabhotla wrote: We have an application that uploads files using a Servlet deployed in Tomcat 6. While this works most of the times, occasionally we run into issues uploading large files. If the upload takes longer then the session timeout, the session gets invalidated right after the upload. Tis means no further requests are accepted unless the user logs back in. Is this the expected behavior? Is there any way to work around this and keep the session active? I guess one way to fix this is to have a large session timeout like an hour or two, but we prefer not to do that for obvious reasons. Responding in the absolute, if there is a session timeout functionality, then it must be based on a) a timeout value (a number of seconds e.g.) b) somewhere, an indicator of when the session was last active, which gets updated at some point So the problem would boil down to knowing where this last active time is stored, and update it regularly during the file upload. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
So your images are being stored to a database. As blobs? That's a difference between our apps: I store the images to a repository and keep a short record of the in a database. I can't advise you on Tomcat, but if the database is the bottleneck, a workaround might be to write your images to temporary files, and use a separate process to move those images into the database. On Fri, Jul 8, 2011 at 3:03 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Just to give more details... The session timeout setting is stored in our application's database. Admins can change the session timeout from the UI we provide. We did this to make it easy for our customers to set the desired timeout rather than telling them going into web.xml and updating the timeout. After a successful login we create the session, and set the timeout on each session that is created using a HTTPSessionListener that is attached to the context. As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Perhaps can be done if we find the Tomcat's internal session object, but prefer not to do it. Am I correct or am I missing something? Sai Pullabhotla On Fri, Jul 8, 2011 at 9:31 AM, André Warnier a...@ice-sa.com wrote: Sai Pullabhotla wrote: We have an application that uploads files using a Servlet deployed in Tomcat 6. While this works most of the times, occasionally we run into issues uploading large files. If the upload takes longer then the session timeout, the session gets invalidated right after the upload. Tis means no further requests are accepted unless the user logs back in. Is this the expected behavior? Is there any way to work around this and keep the session active? I guess one way to fix this is to have a large session timeout like an hour or two, but we prefer not to do that for obvious reasons. Responding in the absolute, if there is a session timeout functionality, then it must be based on a) a timeout value (a number of seconds e.g.) b) somewhere, an indicator of when the session was last active, which gets updated at some point So the problem would boil down to knowing where this last active time is stored, and update it regularly during the file upload. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- Hell hath no limits, nor is circumscrib'd In one self-place; but where we are is hell, And where hell is, there must we ever be --Christopher Marlowe, *Doctor Faustus* (v, 121-24)
RE: Uploading large files and session timeout
From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Subject: Re: Uploading large files and session timeout As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Your servlet could call HttpSession.setMaxInactiveInterval() as needed to prevent a timeout. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chuck, On 7/8/2011 3:20 PM, Caldarale, Charles R wrote: From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Subject: Re: Uploading large files and session timeout As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Your servlet could call HttpSession.setMaxInactiveInterval() as needed to prevent a timeout. +1 Our webapp uses this technique when we hand-off the user from our primary webapp to a secondary one, and then get them back after some interval. Sometimes, the interaction with that second webapp can last longer than the session timeout in the primary (30 minutes) and we don't want users to have to log back in. So, just before we hand-off the user to the secondary webapp, we change the session timeout to a higher value, and then re-set it to the lower value when they come back. The OP could adopt this technique when uploading files. One could even look at the Content-Length and make a judgement call about whether the timeout needs to be adjusted at all (maybe above 5MiB requires a session-timeout adjustment, but others do not). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4XY18ACgkQ9CaO5/Lv0PBrswCfZXb1Ym5s8S/+otMtDwhHe8O+ 1C4AoK+WAh4YQ/RNEr0Bk2a9m72XtRxs =Du3g -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sai, On 7/8/2011 10:25 AM, Sai Pullabhotla wrote: If the upload takes longer then the session timeout, the session gets invalidated right after the upload. Tis means no further requests are accepted unless the user logs back in. Is this the expected behavior? Yes, under certain configurations. Is there any way to work around this and keep the session active? Yes. See the org.apache.catalina.session.StandardSession.ACTIVITY_CHECK system property in http://tomcat.apache.org/tomcat-6.0-doc/config/systemprops.html#Sessions I think that's exactly what you're looking for. Note that there is some overhead associated with tracking these active requests and if you want super-high performance, you might want to go with another solution. I guess one way to fix this is to have a large session timeout like an hour or two, but we prefer not to do that for obvious reasons. Or use Chuck's suggestion to change the timeout for certain requests, then change them back. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4XY/MACgkQ9CaO5/Lv0PBR8ACfWrOf+3q1k1jqgokp4og7dzPn VJYAoI1BP6QvnfZub5TmxM2TweobIeLn =I6BG -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
Thanks to all the replies. I will answer/comment on the last few emails in this one. Our application has nothing to do with storing images in the database. All the upload servlet does is saves the data to a file on the file system. The size of the files vary and are not in our control. They could be as big as a 1gb. As far as changing the session timeout from the servlet, I do not think it works well when multiple uploads are going simultaneously under one session, if we reset (we must, we cannot leave the timeout at infinity) the timeout to normal value after each upload. I tried this some time ago and it gets crazy with the concurrency and I'm not sure if that's the way to do it. I just ran a test with a session timeout set to 30 seconds, and uploading a 800MB file that takes longer than 30 seconds. Below are some debug lines: Fri Jul 08 15:03:18 CDT 2011 Session created: E213791CFB9AD89B0C3DCB63AFBDAAF3 Fri Jul 08 15:03:18 CDT 2011 Timeout on session E213791CFB9AD89B0C3DCB63AFBDAAF3 is set to 30 Fri Jul 08 15:03:33 CDT 2011 Upload Servlet received the request Fri Jul 08 15:04:11 CDT 2011 Session destoryed: E213791CFB9AD89B0C3DCB63AFBDAAF3 Fri Jul 08 15:05:15 CDT 2011 Upload Servlet finished processing the request After the upload servlet was hit, the session destroyed 38 seconds later. Not sure why it is not exactly 30. All 800MB still made it to the server. But the next requests require re-login as the session was destroyed. The session created, the timeout setting and the session destroyed messages are logged from the session listener. And finally, what is the use of disableUploadTimeout on tomcat's connector? Is it for session timeout or the actual socket timeouts? Thanks in advance for any other input/ideas you may have to offer. Sai Pullabhotla On Fri, Jul 8, 2011 at 2:20 PM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Subject: Re: Uploading large files and session timeout As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Your servlet could call HttpSession.setMaxInactiveInterval() as needed to prevent a timeout. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Uploading large files and session timeout
From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Subject: Re: Uploading large files and session timeout As far as changing the session timeout from the servlet, I do not think it works well when multiple uploads are going simultaneously under one session, if we reset (we must, we cannot leave the timeout at infinity) the timeout to normal value after each upload. Don't reset it after each upload, reset it after all uploads for a session are done. Just keep an atomic counter of the number of active uploads and only reset the session timeout when it's zero. And finally, what is the use of disableUploadTimeout on tomcat's connector? Is it for session timeout or the actual socket timeouts? Socket - and the default value is already true. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Uploading large files and session timeout
Caldarale, Charles R wrote: From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Subject: Re: Uploading large files and session timeout As far as I know, the session's lastAccessTime gets updated on each request from the client (by the container), and there is no public API to update the last access time. Your servlet could call HttpSession.setMaxInactiveInterval() as needed to prevent a timeout. The do this cleanly, the servlet would need to call HttpSession.getMaxInactiveInterval() at the beginning, to save the existing value, then call HttpSession.setMaxInactiveInterval() to set a new one for the duration of the transfer, then process the upload, then at the end call HttpSession.setMaxInactiveInterval() to reset the old value, right ? But there is still something that to me seems illogical : Since the session expiration is determined in function of the MaxInactiveInterval and the lastAccessTime, and since Tomcat has no way to know at the beginning of a request, how long this request will take to process, I would think that the lastAccessTime would be updated *at the end* of a request, not at the beginning. And if so, how can it happen that a session is expired after even a long file upload ? In other words, are there circumstances which may cause the lastAccessTime /not/ to be updated ? (such as, if the user interrupts the upload, or an error happens in the middle etc..) If I go by the name, HttpSession.setMaxInactiveInterval() changes the timeout for this session. Which means that after doing that, and finishing the file upload, then as long at this same session remains valid, further requests in this same session will be handled with the new timeout. And as long as that user keeps accessing the application, the timeout will remain at - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Uploading large files and session timeout
From: André Warnier [mailto:a...@ice-sa.com] Subject: Re: Uploading large files and session timeout The do this cleanly, the servlet would need to call HttpSession.getMaxInactiveInterval() at the beginning, to save the existing value, then call HttpSession.setMaxInactiveInterval() to set a new one for the duration of the transfer, then process the upload, then at the end call HttpSession.setMaxInactiveInterval() to reset the old value, right ? Correct, but slightly more complicated due to the previously undisclosed possibility of concurrent uploads associated with a single session. Since the session expiration is determined in function of the MaxInactiveInterval and the lastAccessTime, and since Tomcat has no way to know at the beginning of a request, how long this request will take to process, I would think that the lastAccessTime would be updated *at the end* of a request, not at the beginning. Tomcat 7 (which is not what the OP is using) has these system properties to control session timeout behavior: StandardHostValve.ACCESS_SESSION StandardSession.ACTIVITY_CHECK StandardSession.LAST_ACCESS_AT_START (Determining the actions taken for each property setting is left as an exercise for the Tomcat documentation reader.) Tomcat 6 uses only the ACTIVITY_CHECK property, and session expiration is based only on when the most recent request arrived - as required by the spec. The properties in Tomcat 7 appear to have been added as workarounds for the shortcomings of the spec that you have pointed out. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
