[Resin-interest] Slow Alarm bug... It's baaaaack!
I just installed 4.0.19 and I'm seeing lots of slow alarm's just like are
mentioned in bug
http://bugs.caucho.com/bug_view_advanced_page.php?bug_id=4291 is there a way
to work around this but, or should I back off and try 4.0.18? If there's
anything you need to help solve this problem, just let me know. Thanks.
(*Chris*)
[11-06-19 22:03:01.512] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[
WebAppExpandDeployGenerator[null]]] 239429ms
[11-06-19 22:03:01.513] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[WebAppExpandDeployGenerator[null]]] 238655ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[WebAppExpandDeployGenerator[null]]] 238290ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[com.caucho.network.listen.TcpSocketLinkListener$SuspendReaper@17991de
1]] 215443ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[com.caucho.http.log.AccessLog@595780d9]] 215181ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[SessionManager[]]] 215026ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 215316ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 215287ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[SessionManager[/resin-admin]]] 215287ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[ConnectionPool[jdbc/resin]]] 215210ms
[11-06-19 22:03:01.964] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 214899ms
[11-06-19 22:03:01.964] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 214899ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 214900ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
larm[alarm[DeployControllerAlarm[null]]] 214900ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[] slow
alarm A
and so on... for hours.
___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Eclipse plugin: How to create a resin 3.1 server?
Hi, I just installed the most recent release of the Resin Eclipse plugin. However, I cannot create resin 3.1 servers. (Only resin 4.0 is shown in the New server dialog) This was possible in previous releases of that plugin. Is this an intentional behavior? Thank you and best regards, Patric ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Possible Resin Classloader leakage for JPA applications
Hi all, I think that resin (at least version 3.1, but most probably 4.0 too) suffers from a critical classloader leak which may cause that the memory occupied by an application will not get released when destroying its servlet context (stopping/redeploying the application). As an result, further restarts/redeployments will sooner or later result in an OutOfMemoryError. Please have a look at resin-3.1.10\modules\jpa\src\javax\persistence\Persistence.java On line 51 there's a WeakHashMap defined, which maps the classloader to the corresponding PersistenceProvider instances: private static WeakHashMapClassLoader,PersistenceProvider[] _providerMap = new WeakHashMapClassLoader,PersistenceProvider[](); The problem is that a PersistenceProvider instance may reference the key indirectly (which is at least true when not using Amber, e.g. EclipseLink) The loaded class will of course reference its classloader, which is the web application's classloader. However according to the javadocs this will prevent the removal of the entry: Thus care should be taken to ensure that value objects do not strongly refer to their own keys, either directly or indirectly, since that will prevent the keys from being discarded. With the consequence that the EnvironmentClassLoader cannot be garbage collected. When looking at the GC roots (for a classloader from a stopped application) this can be seen easily: Class Name | Shallow Heap | Retained Heap -- com.caucho.loader.EnvironmentClassLoader @ 0xe68362d0 | 376 | 131.077.664 '- classloader class org.eclipse.persistence.jpa.PersistenceProvider @ 0xd3bda6d8|0 | 0 '- class org.eclipse.persistence.jpa.PersistenceProvider @ 0xe7bb2010 | 24 |40 '- [3] javax.persistence.spi.PersistenceProvider[4] @ 0xe7bb2020 | 56 | 144 '- value java.util.WeakHashMap$Entry @ 0xe7c72ca8 | 72 | 432 '- [11] java.util.WeakHashMap$Entry[16] @ 0xe118fc18 | 152 | 800 '- table java.util.WeakHashMap @ 0xe118fbe8 | 72 | 928 '- _providerMap class javax.persistence.Persistence @ 0xd1fc55a8 | 32 | 960 '- [1152] java.lang.Object[1280] @ 0xe0b40378 | 10.264 | 1.774.424 -- As you can see, the second last line (_providerMap class javax.persistence.Persistence @ 0xd1fc55a8) which is actually a caucho class, references the old EnvironmentClassLoader. So in my case, 131 MB will not get released. If I am right - is there a workaround available? (Is it possible to configure the EnvironmentClassLoader that it will not load the PersistenceProvider/Persistence class from Amber but from the JPA implementation which my application provides?) Thank you and best regards, Patric ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Slow Alarm bug... It's baaaaack!
On 06/19/2011 11:55 PM, Chris Pratt wrote:
I just installed 4.0.19 and I'm seeing lots of slow alarm's just like
are mentioned in bug
http://bugs.caucho.com/bug_view_advanced_page.php?bug_id=4291 is there
a way to work around this but, or should I back off and try 4.0.18?
If there's anything you need to help solve this problem, just let me
know. Thanks.
What does the postmortem show for the threads and memory before the
problem starts?
-- Scott
(*Chris*)
[11-06-19 22:03:01.512] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[
WebAppExpandDeployGenerator[null]]] 239429ms
[11-06-19 22:03:01.513] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[WebAppExpandDeployGenerator[null]]] 238655ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[WebAppExpandDeployGenerator[null]]] 238290ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[com.caucho.network.listen.TcpSocketLinkListener$SuspendReaper@17991de
1]] 215443ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[com.caucho.http.log.AccessLog@595780d9]] 215181ms
[11-06-19 22:03:01.514] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[SessionManager[]]] 215026ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 215316ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 215287ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[SessionManager[/resin-admin]]] 215287ms
[11-06-19 22:03:01.963] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[ConnectionPool[jdbc/resin]]] 215210ms
[11-06-19 22:03:01.964] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 214899ms
[11-06-19 22:03:01.964] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 214899ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 214900ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
larm[alarm[DeployControllerAlarm[null]]] 214900ms
[11-06-19 22:03:01.965] {CoordinatorThread[]-2} CoordinatorThread[]
slow alarm A
and so on... for hours.
___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Possible Resin Classloader leakage for JPA applications
On 06/20/2011 09:09 AM, Patric Rufflar wrote: Hi all, I think that resin (at least version 3.1, but most probably 4.0 too) suffers from a critical classloader leak which may cause that the memory occupied by an application will not get released when destroying its servlet context (stopping/redeploying the application). As an result, further restarts/redeployments will sooner or later result in an OutOfMemoryError. Thanks. I've filed this as http://bugs.caucho.com/view.php?id=4628 -- Scott Please have a look at resin-3.1.10\modules\jpa\src\javax\persistence\Persistence.java On line 51 there's a WeakHashMap defined, which maps the classloader to the corresponding PersistenceProvider instances: private static WeakHashMapClassLoader,PersistenceProvider[] _providerMap = new WeakHashMapClassLoader,PersistenceProvider[](); The problem is that a PersistenceProvider instance may reference the key indirectly (which is at least true when not using Amber, e.g. EclipseLink) The loaded class will of course reference its classloader, which is the web application's classloader. However according to the javadocs this will prevent the removal of the entry: Thus care should be taken to ensure that value objects do not strongly refer to their own keys, either directly or indirectly, since that will prevent the keys from being discarded. With the consequence that the EnvironmentClassLoader cannot be garbage collected. When looking at the GC roots (for a classloader from a stopped application) this can be seen easily: Class Name | Shallow Heap | Retained Heap -- com.caucho.loader.EnvironmentClassLoader @ 0xe68362d0 | 376 | 131.077.664 '-classloader class org.eclipse.persistence.jpa.PersistenceProvider @ 0xd3bda6d8|0 | 0 '-class org.eclipse.persistence.jpa.PersistenceProvider @ 0xe7bb2010 | 24 |40 '- [3] javax.persistence.spi.PersistenceProvider[4] @ 0xe7bb2020 | 56 | 144 '- value java.util.WeakHashMap$Entry @ 0xe7c72ca8 | 72 | 432 '- [11] java.util.WeakHashMap$Entry[16] @ 0xe118fc18 | 152 | 800 '- table java.util.WeakHashMap @ 0xe118fbe8 | 72 | 928 '- _providerMap class javax.persistence.Persistence @ 0xd1fc55a8 | 32 | 960 '- [1152] java.lang.Object[1280] @ 0xe0b40378 | 10.264 | 1.774.424 -- As you can see, the second last line (_providerMap class javax.persistence.Persistence @ 0xd1fc55a8) which is actually a caucho class, references the old EnvironmentClassLoader. So in my case, 131 MB will not get released. If I am right - is there a workaround available? (Is it possible to configure the EnvironmentClassLoader that it will not load the PersistenceProvider/Persistence class from Amber but from the JPA implementation which my application provides?) Thank you and best regards, Patric ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Possible Resin Classloader leakage for JPA applications
Thanks. I've filed this as http://bugs.caucho.com/view.php?id=4628 Thank you for fixing the bug so quickly. Is it possible to backport it to 3.1 as well? (if not: is a workaround available?) Thanks. Patric Please have a look at resin-3.1.10\modules\jpa\src\javax\persistence\Persistence.java On line 51 there's a WeakHashMap defined, which maps the classloader to the corresponding PersistenceProvider instances: private static WeakHashMapClassLoader,PersistenceProvider[] _providerMap = new WeakHashMapClassLoader,PersistenceProvider[](); The problem is that a PersistenceProvider instance may reference the key indirectly (which is at least true when not using Amber, e.g. EclipseLink) The loaded class will of course reference its classloader, which is the web application's classloader. However according to the javadocs this will prevent the removal of the entry: Thus care should be taken to ensure that value objects do not strongly refer to their own keys, either directly or indirectly, since that will prevent the keys from being discarded. With the consequence that the EnvironmentClassLoader cannot be garbage collected. When looking at the GC roots (for a classloader from a stopped application) this can be seen easily: Class Name | Shallow Heap | Retained Heap -- com.caucho.loader.EnvironmentClassLoader @ 0xe68362d0 | 376 | 131.077.664 '-classloader class org.eclipse.persistence.jpa.PersistenceProvider @ 0xd3bda6d8|0 | 0 '-class org.eclipse.persistence.jpa.PersistenceProvider @ 0xe7bb2010 | 24 |40 '- [3] javax.persistence.spi.PersistenceProvider[4] @ 0xe7bb2020 | 56 | 144 '- value java.util.WeakHashMap$Entry @ 0xe7c72ca8 | 72 | 432 '- [11] java.util.WeakHashMap$Entry[16] @ 0xe118fc18 | 152 | 800 '- table java.util.WeakHashMap @ 0xe118fbe8 | 72 | 928 '- _providerMap class javax.persistence.Persistence @ 0xd1fc55a8 | 32 | 960 '- [1152] java.lang.Object[1280] @ 0xe0b40378 | 10.264 | 1.774.424 -- As you can see, the second last line (_providerMap class javax.persistence.Persistence @ 0xd1fc55a8) which is actually a caucho class, references the old EnvironmentClassLoader. So in my case, 131 MB will not get released. If I am right - is there a workaround available? (Is it possible to configure the EnvironmentClassLoader that it will not load the PersistenceProvider/Persistence class from Amber but from the JPA implementation which my application provides?) Thank you and best regards, Patric ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
