Re: isHexDigit error problems and upgrading Tomcat and jdk
On 07/06/2012 00:54, Konstantin Kolinko wrote: 2012/6/7 Miguel González Castañosmiguel_3_gonza...@yahoo.es: Hi, We are getting isHexDigit errors again, although there is no malformed URL requests It is parameter parsing code. It cannot claim the request as malformed (API does not allow it). It can only skip malformed parameters. Does that mean that it won't show up in the access log? It is a problem with a parameter used by the parsing code? How can I log any info that could show where is the problem in the code or when is this happening? It's making Tomcat to pause and then to stop. It sets some error flag that can be examined. FailedRequestFilter is example of a filter that checks that flag. I understand that function has to be added in the parsing code, isn't it? We are using Tomcat 5.5 and jdk 1.5 (from Sun). As some of you have suggested me in the past, I'm considering to upgrade to a more up-to-date Tomcat and/or jdk so I can get more support and help from the community since probably the stability issues are solved in newer versions. I have set up a test environment and Tomcat 7.0.27 and jdk 1.7 seems to work fine. Do you suggest me to upgrade to Tomcat 6 or 7? What about jdk? 1.6 or 1.7? 1.6 is more widely tested (many years), but for a new system I would go with 1.7. It's not a new system, it's been running for 3 years already. I don't want to risk that any library has changed its behavior with tomcat 7 or something similar. I heard that Oracle plans to stop provide free updates for 1.6 this summer. So they will be available only as Java for Business. Does it mean that it is probably better to go for 1.7 instead? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: isHexDigit error problems and upgrading Tomcat and jdk
Am 07.06.2012 10:13, schrieb Miguel González Castaños: ... Do you suggest me to upgrade to Tomcat 6 or 7? What about jdk? 1.6 or 1.7? 1.6 is more widely tested (many years), but for a new system I would go with 1.7. It's not a new system, it's been running for 3 years already. I don't want to risk that any library has changed its behavior with tomcat 7 or something similar. I heard that Oracle plans to stop provide free updates for 1.6 this summer. So they will be available only as Java for Business. Does it mean that it is probably better to go for 1.7 instead? Maybe you should have a look at oracles lifecycle policy: http://www.oracle.com/technetwork/java/eol-135779.html If you need to provide a secure system with current patches you should take the effort to upgrade to java 7. This provides you with 3 more years of Oracle updates. Stefan - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Unable to build tomcat-connectors
Hi There... One guy helped me to fix this issue.. The probelm was headers from the default apr instalation (in /usr/include), with the apache (that is installed from source) headers. i could successfully compile mod_jk whe i configur with CPPFLAGS=-I/usr/local/apache/include ./configure --with-apxs=/usr/local/apache/bin/apxs Thanks Regards SharonSreedh thad.humphr...@gmail.com wrote: On Sun, Jun 3, 2012 at 10:10 AM, Sharon S Sreedh sharonkai...@gmail.com wrote: I already run the configure script --with-apxs2=/path/to/apxs --with-apache=/path/to/apache and header file are in the directory apache/include when I compile with ./configure -with-apxs2=/path/to/apxs alone I got, configure: error: Cannot find the WebServer then I compiled with -with-apxs2=/path/to/apxs --with-apache=/path/to/apache this time compilation completed succefully but build faild with above error. I'm not familiar with a switch --with-apxs2=. When I run ./configure --help, I see --with-apxs[=FILE], but not an apxs2. On Sun, Jun 3, 2012 at 6:01 PM, Thad Humphries thad.humphr...@gmail.com wrote: On Sun, Jun 3, 2012 at 2:44 AM, Sharon S Sreedh sharonkai...@gmail.com wrote: Hi, I was tring to build tomcat-connectors but it doesnt come in success. I tried a number of times.. But all the time it ends up error. Please sugest me a way to make this win. - Linux Kernel version: 2.6.18-308.el5.028stab099.3 GCC version : gcc-4.1.2-52.el5_8.1 Server version: Apache/2.4.2 (Unix) Server built: Jun 3 2012 16:46:17 Server's Module Magic Number: 20120211:3 Server loaded: APR 1.4.6, APR-UTIL 1.4.1 Compiled using: APR 1.4.6, APR-UTIL 1.4.1 Architecture: 32-bit Server MPM: event threaded: yes (fixed thread count) forked: yes (variable process count) Server compiled with -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=256 -D HTTPD_ROOT=/usr/local/apache -D SUEXEC_BIN=/usr/local/apache/bin/suexec -D DEFAULT_PIDLOG=logs/httpd.pid -D DEFAULT_SCOREBOARD=logs/apache_runtime_status -D DEFAULT_ERRORLOG=logs/error_log -D AP_TYPES_CONFIG_FILE=conf/mime.types -D SERVER_CONFIG_FILE=conf/httpd.conf == tomcat-connectors - tomcat-connectors-1.2.37-src Build error --- . . . . /usr/local/src/httpd-2.4.2/include/ap_hooks.h:39:17: error: apr.h: No such file or directory /usr/local/src/httpd-2.4.2/include/ap_hooks.h:40:23: error: apr_hooks.h: No such file or directory /usr/local/src/httpd-2.4.2/include/ap_hooks.h:41:32: error: apr_optional_hooks.h: No such file or directory In file included from /usr/local/src/httpd-2.4.2/include/ap_config.h:136, from mod_jk.c:29: /usr/local/src/httpd-2.4.2/os/unix/os.h:31:17: error: apr.h: No such file or directory mod_jk.c:30:21: error: apr_lib.h: No such file or directory mod_jk.c:31:22: error: apr_date.h: No such file or directory mod_jk.c:32:27: error: apr_file_info.h: No such file or directory mod_jk.c:33:25: error: apr_file_io.h: No such file or directory In file included from /usr/local/src/httpd-2.4.2/include/httpd.h:47, from mod_jk.c:34: /usr/local/src/httpd-2.4.2/include/ap_release.h:25:41: error: apr_general.h: No such file or directory In file included from mod_jk.c:34: /usr/local/src/httpd-2.4.2/include/httpd.h:51:24: error: apr_tables.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:52:23: error: apr_pools.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:53:22: error: apr_time.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:54:28: error: apr_network_io.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:55:25: error: apr_buckets.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:56:22: error: apr_poll.h: No such file or directory /usr/local/src/httpd-2.4.2/include/httpd.h:57:29: error: apr_thread_proc.h: No such file or directory In file included from /usr/local/src/httpd-2.4.2/include/httpd.h:61, from mod_jk.c:34: . . . Attaching Full Build Output with this. Please help me to resolve this. Thanks Regards Sharon Sreedh Just my guess: In running the configure
Re: isHexDigit error problems and upgrading Tomcat and jdk
2012/6/7 Miguel González Castaños miguel_3_gonza...@yahoo.es: On 07/06/2012 00:54, Konstantin Kolinko wrote: 2012/6/7 Miguel González Castañosmiguel_3_gonza...@yahoo.es: Hi, We are getting isHexDigit errors again, although there is no malformed URL requests It is parameter parsing code. It cannot claim the request as malformed (API does not allow it). It can only skip malformed parameters. Does that mean that it won't show up in the access log? Yes, it would not show in access log. It is a problem with a parameter used by the parsing code? It is problem with parameters submitted by client. Broken parameters are ignored (and this warning is printed). Parameters parsing is performed lazily on your first call to getParameter() or similar methods. It is not allowed to fail. In recent versions of Tomcat this and similar failures set a flag in the request object to signal that an error happened. This flag can be tested by your code (using FailedRequestFilter as an example), or you can just configure and use FailedRequestFilter as is, if it is available in that version of Tomcat. What ever you do upon detecting the failure - is your choice. The FailedRequestFilter rejects the request. How can I log any info that could show where is the problem in the code or when is this happening? It's making Tomcat to pause and then to stop. This failure cannot make Tomcat pause and stop. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: / getting added to action path
I added AccessLogValve in server.xml and was able to find the exact problem. The problem is not with /, the problem is following: 1. When I load the screen initially, it loads fine, its able to find all the components. 2. When I click on save, it calls following servlet 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] POST /uom/servlet/uom.update HTTP/1.1 302 - 3. After save loads again (it displays the saved data after clicking on save in the same JSP), it calls components like this: 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/hsnlogo.gif HTTP/1.1 400 1109 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/logout.gif HTTP/1.1 400 1106 10.16.10.90 - - [06/Jun/2012:13:41:56 -0400] GET /uom/*action*/images/pixel_trans.gif HTTP/1.1 400 1121 It is adding 'action' in the URL and I have no clue where it is coming from. On Wed, Jun 6, 2012 at 7:38 AM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2012/6/6 Dharamshila Khandelwal dharmshil...@gmail.com: Tim, It worked fine with IE 6. Tomcat logs clearly say that the path is invalid. Wierd this is that when the screen is loaded for the first time, it works fine. If I click on save (which makes a call to servlet) , it starts giving invalid path error. I know that its not a tomcat issue, but is there any configuration I need to change to make sure that the paths are always taken relative to base context? 1. What is that save that you click on? calls a servlet using relative path. 2. What exactly Tomcat version you are using? 6.0.32 3. Configure an AccessLogValve inside the Host element in server.xml and use a custom pattern in it to log Referer request header. That is to log what requests are actually coming to Tomcat and from what pages. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Can someone suggest mail some opensource mail server which works well with Tomcat.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kiran, On 6/6/12 8:59 PM, Kiran Badi wrote: Is google mail a part of google apps or its normal gmail service which we use? It's all the same thing. The only difference is the usernames. I feel there might be some limits on the number of requests like google api's That may be true. Can you elaborate more please. Not really. If you have questions about Google Mail, go read their documentation and service agreement(s). Any feedback on apache james ? I've never used it, so I can't comment. What I would ask is: why are you are specifically looking for a Java-based mail server? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q1pkACgkQ9CaO5/Lv0PCLdgCgkEv7fA4aHOo0XB96KOTCEB7s RZ8AoIxaupdxPDEz+U8vjHMAvb0iJPUH =bVPD -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: / getting added to action path
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dharamshila, On 6/7/12 9:58 AM, Dharamshila Khandelwal wrote: I added AccessLogValve in server.xml and was able to find the exact problem. The problem is not with /, the problem is following: 1. When I load the screen initially, it loads fine, its able to find all the components. 2. When I click on save, it calls following servlet 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] POST /uom/servlet/uom.update HTTP/1.1 302 - 3. After save loads again (it displays the saved data after clicking on save in the same JSP), it calls components like this: 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/hsnlogo.gif HTTP/1.1 400 1109 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/logout.gif HTTP/1.1 400 1106 10.16.10.90 - - [06/Jun/2012:13:41:56 -0400] GET /uom/*action*/images/pixel_trans.gif HTTP/1.1 400 1121 It is adding 'action' in the URL and I have no clue where it is coming from. It seems clear to me that you are not fully-qualifying your URLs in your web pages. How are you generating your URLs? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q3JsACgkQ9CaO5/Lv0PAYEACgm38nfQhCvZ000eFRJmWfYYS+ LeQAoMNvt1V7m5WFDceo0HZD6qxrKDja =vbq8 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: getting frustrated with web sockets
On 06/06/2012 15:54, Ravi wrote: Mark / Chris, Essentially, if I want to use websockets, you are asking me to rewrite my app + rewrite grails + rewrite hibernate + rewrite apache http utilities + rewrite several other web libraries I use. I don't they're asking you to do that at all. I do not think this is a reasonable option. I think you may misunderstand the place WebSocket has it's intended purpose. Have you read the WebSocket Spec? It's quite readable. p On 6/5/2012 9:04 PM, Christopher Schultz wrote: Mark, On 6/5/12 5:42 PM, Mark Thomas wrote: WebSocket != HTTP and you simply can't replace one with the other. +1 You can't just shout websocket and make a webapp work better with no other changes. It's not just like adding transport encryption by adding an s to the protocol name, it's a completely different protocol that just happens to use HTTP to bootstrap itself. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- [key:62590808] signature.asc Description: OpenPGP digital signature
Re: isHexDigit error problems and upgrading Tomcat and jdk
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 6/7/12 4:13 AM, Miguel González Castaños wrote: It's not a new system, it's been running for 3 years already. I don't want to risk that any library has changed its behavior with tomcat 7 or something similar. I don't believe it's less risky to move up to Tomcat 6 than it is to move to Tomcat 7. I heard that Oracle plans to stop provide free updates for 1.6 this summer. So they will be available only as Java for Business. Does it mean that it is probably better to go for 1.7 instead? Only you can guess what is better for you. As with the Tomcat upgrade, I don't believe it's any less risky to upgrade to Java 1.6 than it is to upgrade to Java 1.6. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q59QACgkQ9CaO5/Lv0PD+MQCcCLvc3rlplLhyFyI8k5+f0GKH mYEAoJzmI5JWAYYXPX1ZilSblGbNMTW2 =dg6q -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: / getting added to action path
I use the URL like this: link rel=stylesheet href=style/core/style.css type=text/css Once I click save, it add 'action' in front of it. Please note that this problem is only with IE7 8. This works absolutely fine in IE6. On Thu, Jun 7, 2012 at 12:53 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dharamshila, On 6/7/12 9:58 AM, Dharamshila Khandelwal wrote: I added AccessLogValve in server.xml and was able to find the exact problem. The problem is not with /, the problem is following: 1. When I load the screen initially, it loads fine, its able to find all the components. 2. When I click on save, it calls following servlet 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] POST /uom/servlet/uom.update HTTP/1.1 302 - 3. After save loads again (it displays the saved data after clicking on save in the same JSP), it calls components like this: 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/hsnlogo.gif HTTP/1.1 400 1109 10.16.10.90 - - [06/Jun/2012:13:41:55 -0400] GET /uom/*action*/images/logout.gif HTTP/1.1 400 1106 10.16.10.90 - - [06/Jun/2012:13:41:56 -0400] GET /uom/*action*/images/pixel_trans.gif HTTP/1.1 400 1121 It is adding 'action' in the URL and I have no clue where it is coming from. It seems clear to me that you are not fully-qualifying your URLs in your web pages. How are you generating your URLs? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q3JsACgkQ9CaO5/Lv0PAYEACgm38nfQhCvZ000eFRJmWfYYS+ LeQAoMNvt1V7m5WFDceo0HZD6qxrKDja =vbq8 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: isHexDigit error problems and upgrading Tomcat and jdk
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 6/7/12 4:13 AM, Miguel González Castaños wrote: On 07/06/2012 00:54, Konstantin Kolinko wrote: It is parameter parsing code. It cannot claim the request as malformed (API does not allow it). It can only skip malformed parameters. Does that mean that it won't show up in the access log? That depends upon how you have configured your access log: you can configure an access log to log only requests that failed and the details of that request like this: Valve className=org.apache.catalina.valves.AccessLogValve conditionIf=org.apache.catalina.parameter_parse_failed / You will probably want to put %t and %r in your pattern. Unfortunately, this is a POST that is being processed and it's hard to use the RequestDumperFilter to dump the requests of only certain requests that might be the only way to get the offending parameter name or value that is causing this problem. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q6dEACgkQ9CaO5/Lv0PA6bQCfcHsaExvYzrgBFl9WM+3Lg+DS cssAnR1rCga318tOzveCdGDho+5/FCE+ =lPTF -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: / getting added to action path
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dharamshila, On 6/7/12 1:44 PM, Dharamshila Khandelwal wrote: I use the URL like this: link rel=stylesheet href=style/core/style.css type=text/css You need to make sure that your URL starts with a / in order to make it work when the containing document is re-locatable. Once I click save, it add 'action' in front of it. Please note that this problem is only with IE7 8. This works absolutely fine in IE6. It sounds very unlikely that MSIE 6 specifically is randomly adding the word action to your URL. Maybe you have some MSIE6-specific Javascript that is altering its behavior. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q6l8ACgkQ9CaO5/Lv0PD54wCfWO8RKs4ClKvB1GyH0RT/qsjO ZfIAoKeE11gwxt+vxEvyrN0okTVEK7Hm =1M8x -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
[POLL] Finer-grained manager user-access privileges?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 All, I was just answering a question on StackOverflow[1] about limiting the operations a particular user could perform when using the manager app (e.g. deploy, undeploy, start, stop, etc.). It seems to me that this has come up on the users' list once or twice in the past, and it wouldn't be a big deal to support this kind of thing right out of the box by just defining a number of additional roles such as: manager-gui-deploy manager-gui-undeploy manager-gui-start etc. Is there any interest in doing something like this? My general feeling is that manager access should either be allowed read-only (which is covered by the manager-status role) or full read/write (which is covered by the manager-gui and manager-sript roles) because hey, you should trust your managers or fire them ;) On the other hand, if there is significant interest in this kind of thing, we should support it out of the box. Thanks, - -chris [1] http://stackoverflow.com/questions/10909471/tomcat-web-application-manager-is-it-possible-to-limit-what-each-user-role-can/10937606#10937606 -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9MwACgkQ9CaO5/Lv0PCqtQCgugP0SxtqNjV9UVOQLlFOnaOj AcoAnjsiqPqvACrATqj8jlfBR4i4W4UK =q9cy -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/6/12 5:33 PM, Jorge Medina wrote: The web application uses Spring/Postgres/Mongo. Are you using MongoDB in-process or anything weird like that? Or are you connecting through some socket-based (or other) API? It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. If what you are observing is accurate (non-heap memory grows, heap stays reasonable) then it will definitely be more difficult to track-down. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? What do your Connectors look like? How many JDBC connections do you have in your connection pool (which you are hopefully using!)? How about the same equivalent for MongoDB? Does your webapp keep lots of files open? Do you have an unusually-large number of JAR files in your webapp? Do you have any native libraries in use within your webapp? What are all the non-default system properties that you are setting at JVM launch time (you can easily see this from a 'ps' list)? Two things that can eat-up native memory fast in a JVM are file descriptors and threads, so let's start there. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9ooACgkQ9CaO5/Lv0PDPyQCfVtddxMDOgQbjmMGC3gvnK+Qq aZMAnjVu67+9Sm2bdYzAd91ZOrYo3DFI =r+vl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
It seems my system is running with an relatively old version of the Tomcat Native Library and old versions of APR and OpenSSL. Tomcat Native 1.1.19 APR 1.3.8 OpenSSL 0.9.8k These are the latest versions available: Tomcat Native 1.1.23 APR 1.4.6 OpenSSL 1.0.1c I will try disabling the Tomcat Native Library to see if this solves the problem. Has anyone experience problems of Tomcat 7 running with these versions? -Jorge On Wed, Jun 6, 2012 at 2:33 PM, Jorge Medina cerebrotecnolog...@gmail.com wrote: Hello, I have an application that runs under Tomcat 7.0.23 that periodically crashes. The java process running tomcat keeps growing in memory until the Linux oom-killer kills the process. I do not get an OutOfMemoryError because the memory leak is not in the Java heap. In fact, it seems the heap is only using 4GB of the max 6GB specified in the -Xmx parameter. Nevertheless the total memory held by the java process keeps growing up to 16GB when the OS kills the process. I haven't been able to find the conditions to reproduce this problem, so I am not able to replicate it. Nevertheless it keeps occurring; sometimes at midnight with no user activity sometimes in the middle of a busy day. The web application uses Spring/Postgres/Mongo. I know this is not a Tomcat related problem, but some of you may have experience a similar problem and may have some suggestions on how to troubleshoot it. I already have read many of the links that come after searching the web for java invoked oom-killer but I still don't have any clue on what causes the problem and how to solve it. It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? -Jorge Jun 4 16:02:49 ip-10-83-35-78 kernel: [1468800.179218] 3795110 pages non-shared Jun 5 06:50:07 ip-10-83-35-78 rsyslogd: [origin software=rsyslogd swVersion=4.2.0 x-pid=599 x-info=http://www.rsyslog.com;] rsyslogd was HUPed, type 'lightweight'. Jun 5 22:06:40 ip-10-83-35-78 kernel: [1576977.209487] java invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209492] java cpuset=/ mems_allowed=0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209496] Pid: 15618, comm: java Not tainted 2.6.32-317-ec2 #36-Ubuntu Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209498] Call Trace: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209507] [8107cbbc] ? cpuset_print_task_mems_allowed+0x8c/0xc0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209513] [810b1723] oom_kill_process+0xe3/0x210 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209516] [810b18a0] __out_of_memory+0x50/0xb0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209519] [810b195f] out_of_memory+0x5f/0xc0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209523] [810b4641] __alloc_pages_slowpath+0x561/0x580 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209526] [810b47d1] __alloc_pages_nodemask+0x171/0x180 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209530] [810b76f7] __do_page_cache_readahead+0xd7/0x220 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209534] [810b785c] ra_submit+0x1c/0x20 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209536] [810b01fe] filemap_fault+0x3fe/0x450 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209541] [810cbef0] __do_fault+0x50/0x680 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209547] [8102afdb] ? __dequeue_entity+0x2b/0x50 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209550] [810cde30] handle_mm_fault+0x260/0x4f0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209555] [814b3ab7] do_page_fault+0x147/0x390 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209558] [814b18e8] page_fault+0x28/0x30 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209560] Mem-Info: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209561] DMA per-cpu: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209563] CPU 0: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209565] CPU 1: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209567] CPU 2: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209569] CPU 3: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209570] DMA32 per-cpu: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209572] CPU 0: hi: 155, btch: 38 usd: 44 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209574] CPU 1: hi: 155, btch: 38 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209575] CPU 2: hi: 155, btch: 38 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209577] CPU 3: hi: 155, btch: 38 usd: 0 Jun 5 22:06:41
Re: Java process killed by oom-killer in Ubuntu
Thanks Martin, but the memory error is not in the Java heap. Basically my java process keeps growing far beyond the maximum java heap size allowed. -Jorge On Wed, Jun 6, 2012 at 5:57 PM, Martin Gainty mgai...@hotmail.com wrote: neceista empezar tomcat con JMX por ejemplo: $CATALINA_HOME/bin/java -Dcom.sun.management.jmxremote -jar bootstrap.jar entonces.. necesita monitorear heap y stack para proceso de TC durante la vida del proceso de tomcat con Jconsole por ejemplo $JAVA_HOME/bin/jconsole localhost:8080 http://docs.oracle.com/javase/1.5.0/docs/guide/management/jconsole.html Saludos Cordiales desde EEUU Martin Gainty __ Porfavor..no altere o disrupta esta communicacion..Gracias Date: Wed, 6 Jun 2012 14:33:22 -0700 Subject: Java process killed by oom-killer in Ubuntu From: cerebrotecnolog...@gmail.com To: users@tomcat.apache.org Hello, I have an application that runs under Tomcat 7.0.23 that periodically crashes. The java process running tomcat keeps growing in memory until the Linux oom-killer kills the process. I do not get an OutOfMemoryError because the memory leak is not in the Java heap. In fact, it seems the heap is only using 4GB of the max 6GB specified in the -Xmx parameter. Nevertheless the total memory held by the java process keeps growing up to 16GB when the OS kills the process. I haven't been able to find the conditions to reproduce this problem, so I am not able to replicate it. Nevertheless it keeps occurring; sometimes at midnight with no user activity sometimes in the middle of a busy day. The web application uses Spring/Postgres/Mongo. I know this is not a Tomcat related problem, but some of you may have experience a similar problem and may have some suggestions on how to troubleshoot it. I already have read many of the links that come after searching the web for java invoked oom-killer but I still don't have any clue on what causes the problem and how to solve it. It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? -Jorge Jun 4 16:02:49 ip-10-83-35-78 kernel: [1468800.179218] 3795110 pages non-shared Jun 5 06:50:07 ip-10-83-35-78 rsyslogd: [origin software=rsyslogd swVersion=4.2.0 x-pid=599 x-info=http://www.rsyslog.com;] rsyslogd was HUPed, type 'lightweight'. Jun 5 22:06:40 ip-10-83-35-78 kernel: [1576977.209487] java invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209492] java cpuset=/ mems_allowed=0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209496] Pid: 15618, comm: java Not tainted 2.6.32-317-ec2 #36-Ubuntu Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209498] Call Trace: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209507] [8107cbbc] ? cpuset_print_task_mems_allowed+0x8c/0xc0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209513] [810b1723] oom_kill_process+0xe3/0x210 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209516] [810b18a0] __out_of_memory+0x50/0xb0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209519] [810b195f] out_of_memory+0x5f/0xc0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209523] [810b4641] __alloc_pages_slowpath+0x561/0x580 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209526] [810b47d1] __alloc_pages_nodemask+0x171/0x180 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209530] [810b76f7] __do_page_cache_readahead+0xd7/0x220 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209534] [810b785c] ra_submit+0x1c/0x20 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209536] [810b01fe] filemap_fault+0x3fe/0x450 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209541] [810cbef0] __do_fault+0x50/0x680 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209547] [8102afdb] ? __dequeue_entity+0x2b/0x50 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209550] [810cde30] handle_mm_fault+0x260/0x4f0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209555] [814b3ab7] do_page_fault+0x147/0x390 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209558] [814b18e8] page_fault+0x28/0x30 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209560] Mem-Info: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209561] DMA per-cpu: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209563] CPU 0: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209565] CPU 1: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209567] CPU 2: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209569] CPU 3: hi: 0, btch: 1 usd: 0 Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209570] DMA32 per-cpu: Jun 5 22:06:41 ip-10-83-35-78 kernel: [1576977.209572] CPU 0: hi: 155, btch: 38
Re: Java process killed by oom-killer in Ubuntu
I am using MongoDB through the Java driver allowing up to 100 connections to the MongoDB server. I also use DBCP with a max size of 50 JDBC connections. My webapp uses about 150 JAR files. There is no native libraries loaded from my webapp as far as I know. All the app is pure Java code. (Nevertheless, Tomcat is using the Tomcat Native Library) Is there a way I can monitor the number of file descriptors in use by the app? I have monitored the number of threads, but I haven't seen anything unusual. (but it could be that the burst is too fast to get catch by the monitoring tool) -Jorge On Thu, Jun 7, 2012 at 11:44 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/6/12 5:33 PM, Jorge Medina wrote: The web application uses Spring/Postgres/Mongo. Are you using MongoDB in-process or anything weird like that? Or are you connecting through some socket-based (or other) API? It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. If what you are observing is accurate (non-heap memory grows, heap stays reasonable) then it will definitely be more difficult to track-down. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? What do your Connectors look like? How many JDBC connections do you have in your connection pool (which you are hopefully using!)? How about the same equivalent for MongoDB? Does your webapp keep lots of files open? Do you have an unusually-large number of JAR files in your webapp? Do you have any native libraries in use within your webapp? What are all the non-default system properties that you are setting at JVM launch time (you can easily see this from a 'ps' list)? Two things that can eat-up native memory fast in a JVM are file descriptors and threads, so let's start there. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9ooACgkQ9CaO5/Lv0PDPyQCfVtddxMDOgQbjmMGC3gvnK+Qq aZMAnjVu67+9Sm2bdYzAd91ZOrYo3DFI =r+vl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
- Original Message - I am using MongoDB through the Java driver allowing up to 100 connections to the MongoDB server. I also use DBCP with a max size of 50 JDBC connections. My webapp uses about 150 JAR files. There is no native libraries loaded from my webapp as far as I know. All the app is pure Java code. (Nevertheless, Tomcat is using the Tomcat Native Library) Is there a way I can monitor the number of file descriptors in use by the app? I have monitored the number of threads, but I haven't seen anything unusual. How many threads have you observed? Total threads, not just threads for the connector. Also, what is the value you are using for thread stack size? -Xss Dan (but it could be that the burst is too fast to get catch by the monitoring tool) -Jorge On Thu, Jun 7, 2012 at 11:44 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/6/12 5:33 PM, Jorge Medina wrote: The web application uses Spring/Postgres/Mongo. Are you using MongoDB in-process or anything weird like that? Or are you connecting through some socket-based (or other) API? It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. If what you are observing is accurate (non-heap memory grows, heap stays reasonable) then it will definitely be more difficult to track-down. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? What do your Connectors look like? How many JDBC connections do you have in your connection pool (which you are hopefully using!)? How about the same equivalent for MongoDB? Does your webapp keep lots of files open? Do you have an unusually-large number of JAR files in your webapp? Do you have any native libraries in use within your webapp? What are all the non-default system properties that you are setting at JVM launch time (you can easily see this from a 'ps' list)? Two things that can eat-up native memory fast in a JVM are file descriptors and threads, so let's start there. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9ooACgkQ9CaO5/Lv0PDPyQCfVtddxMDOgQbjmMGC3gvnK+Qq aZMAnjVu67+9Sm2bdYzAd91ZOrYo3DFI =r+vl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Java process killed by oom-killer in Ubuntu
From: Jorge Medina [mailto:cerebrotecnolog...@gmail.com] Subject: Re: Java process killed by oom-killer in Ubuntu Is there a way I can monitor the number of file descriptors in use by the app? You can see the open files for any process with lsof: http://linux.die.net/man/8/lsof You can also just do ls -l /proc/pid/fd to see what a specific process has open. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/7/12 3:03 PM, Jorge Medina wrote: I am using MongoDB through the Java driver allowing up to 100 connections to the MongoDB server. 100 connections sounds a bit high to me, but I don't know your requirements nor do I know anything about the MongoDB driver. I suspect it's not a JDBC driver since Mongo is document-oriented and not SQL-oriented. I also use DBCP with a max size of 50 JDBC connections. My webapp uses about 150 JAR files. Neither of those seem unusually high. There is no native libraries loaded from my webapp as far as I know. All the app is pure Java code. (Nevertheless, Tomcat is using the Tomcat Native Library) Have you tried disabling the native library to see if it has any effect? tcnative is fairly stable, but it's something that is easy to test. Is there a way I can monitor the number of file descriptors in use by the app? I dunno about realtime, but you can use something like 'lsof' to check on the open file descriptors of a process. Remember that a socket connection requires two file descriptors: one to read and one to write. If you have maxed-out your db connections, you'll have 300 file descriptors open at once. That's not terribly high, but you'll be using fds for other things as well (incoming HTTP connections, etc.) and could use-up a lot of native memory that way. I have monitored the number of threads, but I haven't seen anything unusual. (but it could be that the burst is too fast to get catch by the monitoring tool) What about JVM startup parameters? Also Connector configuration? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/RDnMACgkQ9CaO5/Lv0PDyqACgpBGlvYUBBMMiz5M6fdaYBrPF KVsAn1hz9AH4n2vzIh92EvUEBY9GHz8v =0rwW -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
Only 52 java threads. It used to fluctuate more (we made some changes to the app to perform a task in a single thread rather than spawning multiple threads, but the crash still occurs) . The number of threads is always below 100. jstack -F 21370 | grep ^Thread | wc -l ps -T -p 21370 (This gives me 63) I don't seem to specify the -Xss option: Xms6g -Xmx6g -XX:NewSize=4G -XX:MaxNewSize=4G -XX:SurvivorRatio=6 -XX:MaxPermSize=512M -XX:-UseConcMarkSweepGC -XX:+UseStringCache -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/example/logs -Jorge On Thu, Jun 7, 2012 at 12:07 PM, Daniel Mikusa dmik...@vmware.com wrote: - Original Message - I am using MongoDB through the Java driver allowing up to 100 connections to the MongoDB server. I also use DBCP with a max size of 50 JDBC connections. My webapp uses about 150 JAR files. There is no native libraries loaded from my webapp as far as I know. All the app is pure Java code. (Nevertheless, Tomcat is using the Tomcat Native Library) Is there a way I can monitor the number of file descriptors in use by the app? I have monitored the number of threads, but I haven't seen anything unusual. How many threads have you observed? Total threads, not just threads for the connector. Also, what is the value you are using for thread stack size? -Xss Dan (but it could be that the burst is too fast to get catch by the monitoring tool) -Jorge On Thu, Jun 7, 2012 at 11:44 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/6/12 5:33 PM, Jorge Medina wrote: The web application uses Spring/Postgres/Mongo. Are you using MongoDB in-process or anything weird like that? Or are you connecting through some socket-based (or other) API? It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. If what you are observing is accurate (non-heap memory grows, heap stays reasonable) then it will definitely be more difficult to track-down. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? What do your Connectors look like? How many JDBC connections do you have in your connection pool (which you are hopefully using!)? How about the same equivalent for MongoDB? Does your webapp keep lots of files open? Do you have an unusually-large number of JAR files in your webapp? Do you have any native libraries in use within your webapp? What are all the non-default system properties that you are setting at JVM launch time (you can easily see this from a 'ps' list)? Two things that can eat-up native memory fast in a JVM are file descriptors and threads, so let's start there. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9ooACgkQ9CaO5/Lv0PDPyQCfVtddxMDOgQbjmMGC3gvnK+Qq aZMAnjVu67+9Sm2bdYzAd91ZOrYo3DFI =r+vl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/7/12 2:51 PM, Jorge Medina wrote: It seems my system is running with an relatively old version of the Tomcat Native Library and old versions of APR and OpenSSL. Tomcat Native 1.1.19 APR 1.3.8 OpenSSL 0.9.8k Those aren't terribly ancient: APR 1.3.8 was the standard for quite a while, especially on Linux distros that are notorious for lagging-behind the state of the art (I'm looking at /you/, Debian). OpenSSL is one of those things that has been at 0.9.8something for like 5 years. tcnative has gotten a few updates recently, but it's never a bad idea to have the latest. Generally speaking, you should probably be using the tcnative version that came with your Tomcat distribution. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/RENEACgkQ9CaO5/Lv0PBvyQCgxDHOImKMkhD3U/3Za/Sg4gc1 JGsAni2vwaNDFaSKk8MU20Yktf+2Kwa0 =GMCL -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Java process killed by oom-killer in Ubuntu
- Original Message - Only 52 java threads. It used to fluctuate more (we made some changes to the app to perform a task in a single thread rather than spawning multiple threads, but the crash still occurs) . The number of threads is always below 100. jstack -F 21370 | grep ^Thread | wc -l ps -T -p 21370 (This gives me 63) I don't seem to specify the -Xss option: In some applications with a large number of threads (particularly when running on 64-bit hardware) this setting can cause a problems. The default value is pretty large (I think it's 1M on 64-bit systems). Since most apps don't need that large of a value, an easy performance tuning step is to lower the value of -Xss. Since you don't have very many threads, it seems unlikely that this is causing your problem though. That being said, you could try explicitly setting a value for the thread stack size. Finding the right values takes some testing though. I usually start with something like 192k and run a few application tests. If I see any stack overflow exceptions then I increase the value and rerun the tests. Repeat until there are no stack overflow exceptions. On a different note, what is the specific version of the JVM that you are running? If it's not the latest, you could always try upgrading to the latest version. Dan Xms6g -Xmx6g -XX:NewSize=4G -XX:MaxNewSize=4G -XX:SurvivorRatio=6 -XX:MaxPermSize=512M -XX:-UseConcMarkSweepGC -XX:+UseStringCache -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/example/logs -Jorge On Thu, Jun 7, 2012 at 12:07 PM, Daniel Mikusa dmik...@vmware.com wrote: - Original Message - I am using MongoDB through the Java driver allowing up to 100 connections to the MongoDB server. I also use DBCP with a max size of 50 JDBC connections. My webapp uses about 150 JAR files. There is no native libraries loaded from my webapp as far as I know. All the app is pure Java code. (Nevertheless, Tomcat is using the Tomcat Native Library) Is there a way I can monitor the number of file descriptors in use by the app? I have monitored the number of threads, but I haven't seen anything unusual. How many threads have you observed? Total threads, not just threads for the connector. Also, what is the value you are using for thread stack size? -Xss Dan (but it could be that the burst is too fast to get catch by the monitoring tool) -Jorge On Thu, Jun 7, 2012 at 11:44 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge, On 6/6/12 5:33 PM, Jorge Medina wrote: The web application uses Spring/Postgres/Mongo. Are you using MongoDB in-process or anything weird like that? Or are you connecting through some socket-based (or other) API? It looks like a memory leak in native code, not java code; so my usual java toolset is not useful. If what you are observing is accurate (non-heap memory grows, heap stays reasonable) then it will definitely be more difficult to track-down. Tomcat runs behind nginx in a EC2 instance. The application uses Sun (now Oracle) JDK 1.6. Any suggestions on what should I look at? What do your Connectors look like? How many JDBC connections do you have in your connection pool (which you are hopefully using!)? How about the same equivalent for MongoDB? Does your webapp keep lots of files open? Do you have an unusually-large number of JAR files in your webapp? Do you have any native libraries in use within your webapp? What are all the non-default system properties that you are setting at JVM launch time (you can easily see this from a 'ps' list)? Two things that can eat-up native memory fast in a JVM are file descriptors and threads, so let's start there. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q9ooACgkQ9CaO5/Lv0PDPyQCfVtddxMDOgQbjmMGC3gvnK+Qq aZMAnjVu67+9Sm2bdYzAd91ZOrYo3DFI =r+vl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -
puzzling stacktrace
I have just reorganized my tomcat deployment to use catalina base and home. I'm getting an obscure, to me at least, error on startup. During the processing of my webapps I think. I'm on tomcat 5.5.35, and, yes, I'm going to upgrade after I get the base and home thing working. Anybody have some insight into this: 2012-06-07 15:28:47,616 ERROR [main]-digester.Digester: Begin event threw exception java.lang.ClassNotFoundException: org.apache.catalina.deploy.FilterDef at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1438) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1284) at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:205) at org.apache.tomcat.util.digester.Rule.begin(Rule.java:153) at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1276) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.startElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1562) at org.apache.catalina.startup.ContextConfig.applicationWebConfig(ContextConfig.java:348) at org.apache.catalina.startup.ContextConfig.start(ContextConfig.java:1053) at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:261) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:120) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4184) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014) at org.apache.catalina.core.StandardHost.start(StandardHost.java:736) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) at org.apache.catalina.core.StandardService.start(StandardService.java:448) at org.apache.catalina.core.StandardServer.start(StandardServer.java:700) at org.apache.catalina.startup.Catalina.start(Catalina.java:552) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
Re: isHexDigit error problems and upgrading Tomcat and jdk
That depends upon how you have configured your access log: you can configure an access log to log only requests that failed and the details of that request like this: Valve className=org.apache.catalina.valves.AccessLogValve conditionIf=org.apache.catalina.parameter_parse_failed / You will probably want to put %t and %r in your pattern. This looks interesting. In my test environment (with Tomcat 7 and java 1.7) it shows by default Valve className=org.apache.catalina.valves.AccessLogValve directory=logs prefix=localhost_access_log. suffix=.txt pattern=%h %l %u %t quot;%rquot; %s %b / this should keep track of the errors just adding just the entry: conditionIf=org.apache.catalina.parameter_parse_failed ? Unfortunately, this is a POST that is being processed and it's hard to use the RequestDumperFilter to dump the requests of only certain requests that might be the only way to get the offending parameter name or value that is causing this problem. Many thanks for your hints :) Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: puzzling stacktrace
2012/6/8 chad.da...@emc.com: I have just reorganized my tomcat deployment to use catalina base and home. I'm getting an obscure, to me at least, error on startup. During the processing of my webapps I think. I'm on tomcat 5.5.35, and, yes, I'm going to upgrade after I get the base and home thing working. What did you put and where? Please follow instructions in RUNNING.txt closely. The libraries in Tomcat 5 are split over more directories than in Tomcat 6 and later. The paths to them are configured in conf/catalina.properties. I think you put server or common libraries into wrong place, or they are not readable. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
