Further update - I think my changes work with respect to this issue, but I
do have a bunch of unit test failures which I need to look into. For info,
here's the diff:
diff --git
a/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/Assembler.java
b/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/Assembler.java
index a6e2bdc4f5..c0d1a8c98f 100644
---
a/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/Assembler.java
+++
b/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/Assembler.java
@@ -69,10 +69,7 @@ import org.apache.openejb.cdi.ThreadSingletonService;
import org.apache.openejb.classloader.ClassLoaderConfigurer;
import org.apache.openejb.classloader.CompositeClassLoaderConfigurer;
import org.apache.openejb.component.ClassLoaderEnricher;
-import org.apache.openejb.config.ConfigurationFactory;
-import org.apache.openejb.config.NewLoaderLogic;
-import org.apache.openejb.config.QuickJarsTxtParser;
-import org.apache.openejb.config.TldScanner;
+import org.apache.openejb.config.*;
import org.apache.openejb.core.ConnectorReference;
import org.apache.openejb.core.CoreContainerSystem;
import org.apache.openejb.core.CoreUserTransaction;
@@ -829,6 +826,14 @@ public class Assembler extends AssemblerTool
implements org.apache.openejb.spi.A
final AppContext appContext = new
AppContext(appInfo.appId, SystemInstance.get(), classLoader,
globalJndiContext, appJndiContext, appInfo.standaloneModule);
appContext.getProperties().putAll(appInfo.properties);
+
+ final Set<Object> keys =
appContext.getProperties().keySet();
+ for (final Object key : keys) {
+ if (! Module.class.isInstance(key)) {
+ appContext.getProperties().remove(key);
+ }
+ }
+
appContext.getInjections().addAll(injections);
appContext.getBindings().putAll(globalBindings);
appContext.getBindings().putAll(appBindings);
diff --git
a/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/JndiEncBuilder.java
b/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/JndiEncBuilder.java
index feff954d40..aed0fda941 100644
---
a/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/JndiEncBuilder.java
+++
b/container/openejb-core/src/main/java/org/apache/openejb/assembler/classic/JndiEncBuilder.java
@@ -384,12 +384,7 @@ public class JndiEncBuilder {
reference = new LinkRef(jndiName);
} else if (BeanManager.class.equals(type)) {
- reference = new LazyObjectReference<BeanManager>(new
Callable<BeanManager>() {
- @Override
- public BeanManager call() throws Exception {
- return new
InjectableBeanManager(WebBeansContext.currentInstance().getBeanManagerImpl());
- }
- });
+ reference = new LazyObjectReference<>(new
BeanManagerLazyReference());
} else if (UserTransaction.class.equals(type)) {
reference = new
IntraVmJndiReference("comp/UserTransaction");
@@ -684,4 +679,11 @@ public class JndiEncBuilder {
}
}
}
+
+ public static class BeanManagerLazyReference implements
Callable<BeanManager> {
+ @Override
+ public BeanManager call() throws Exception {
+ return new
InjectableBeanManager(WebBeansContext.currentInstance().getBeanManagerImpl());
+ }
+ }
}
diff --git
a/container/openejb-core/src/main/java/org/apache/openejb/core/TempClassLoader.java
b/container/openejb-core/src/main/java/org/apache/openejb/core/TempClassLoader.java
index 2cf387e112..285007925f 100644
---
a/container/openejb-core/src/main/java/org/apache/openejb/core/TempClassLoader.java
+++
b/container/openejb-core/src/main/java/org/apache/openejb/core/TempClassLoader.java
@@ -59,6 +59,7 @@ public class TempClassLoader extends URLClassLoader {
private final ClassLoader system;
private final boolean embedded;
private final boolean parentURLClassLoader;
+ private final StackTraceElement[] created;
// 80% of class files are smaller then 6k
private final ByteArrayOutputStream bout = new ByteArrayOutputStream(6
* 1024);
@@ -69,6 +70,7 @@ public class TempClassLoader extends URLClassLoader {
this.system = ClassLoader.getSystemClassLoader();
this.embedded = this.getClass().getClassLoader() == this.system;
this.parentURLClassLoader =
URLClassLoader.class.isInstance(parent);
+ this.created = new Throwable().getStackTrace();
}
/*
In essence, this does 2 things:
1. Defines the Callable for the LazyObjectReference for BeanManager as a
static class. This stops that object from having a reference to
JndiEncBuilder and everything that has attached to it.
2. Strips off the EjbModule / WebModule from the SuperProperties on
AppContext. This is set when deploying through the apps folder and lives
forever. It has the TempClassLoader attached, and therefore all the
AnnotationFinder stuff.
I suspect (2) is too aggressive and is causing the test failures, but will
confirm.
Jon
On Mon, Dec 16, 2019 at 11:27 AM Jonathan Gallimore <
[email protected]> wrote:
> Thanks for sending those over. I'm digging through this - I *think* I have
> pinned down a specific reference that stops that classloader from being
> released. The behaviour for a war/ear deployed in apps/ seems different to
> deploying in webapps/ - I'm assuming that's what you're doing. If you can
> confirm, that would help. I'll test out a patch with some bigger .ear files
> today.
>
> Thanks
>
> Jon
>
> On Thu, Dec 12, 2019 at 9:08 PM Jonathan Gallimore <
> [email protected]> wrote:
>
>> The mailing list seems to be stripping off your images - can you resend
>> and include my email address as well? I'd be keen to dig into this a little
>> more.
>>
>> Thanks
>>
>> Jon
>>
>> On Thu, Dec 12, 2019 at 9:00 PM Paul Carter-Brown
>> <[email protected]> wrote:
>>
>>> Hi,
>>>
>>> The code does seem to use org.apache.openejb.core.TempClassLoader. I can
>>> see one instance in the heap for every war in my ear. + 1.
>>>
>>> The TempClassLoader however still has 100's of strong references to it.
>>> E.g:
>>>
>>> [image: image.png]
>>>
>>> [image: image.png]
>>>
>>>
>>> Paul Carter-Brown
>>> Director
>>> Jini Guru
>>> m: +27 (0) 83 442 7179 <+27834427179>
>>> a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and Leslie
>>> Johannesburg, South Africa
>>> w: jini.guru e: [email protected]
>>>
>>> Disclaimer: This message and/or attachment(s) may contain
>>> privileged, confidential and/or personal information. If you are not the
>>> intended recipient you may not disclose or distribute any of
>>> the information contained within this message. In such case you must
>>> destroy this message and inform the sender of the error. Jini Guru may not
>>> accept liability for any errors, omissions, information and viruses
>>> contained in the transmission of this message. Any opinions, conclusions
>>> and other information contained within this message not related to Jini
>>> Guru official business is deemed to be that of the individual only and is
>>> not endorsed by Jini Guru.
>>>
>>>
>>>
>>> On Thu, Dec 12, 2019 at 9:29 AM Mark Struberg <[email protected]>
>>> wrote:
>>>
>>>> Hi!
>>>>
>>>> All this is just a workaround imo.
>>>>
>>>> Afair we (Romain) introduced a temporary throw-away ClassLoader for
>>>> scanning. That means after doing all the class scans we only keep the
>>>> extracted information but do not keep the Classes in memory. Doesn't this
>>>> work anymore?
>>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>> > Am 12.12.2019 um 00:45 schrieb Paul Carter-Brown
>>>> <[email protected]>:
>>>> >
>>>> > FYI. Graph of the change in heap size on a prod instance after this
>>>> change:
>>>> >
>>>> >
>>>> > Paul Carter-Brown
>>>> > Director
>>>> > Jini Guru
>>>> > m: +27 (0) 83 442 7179
>>>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and
>>>> Leslie
>>>> > Johannesburg, South Africa
>>>> > w: jini.guru e: [email protected]
>>>> >
>>>> > Disclaimer: This message and/or attachment(s) may contain privileged,
>>>> confidential and/or personal information. If you are not the intended
>>>> recipient you may not disclose or distribute any of the information
>>>> contained within this message. In such case you must destroy this message
>>>> and inform the sender of the error. Jini Guru may not accept liability for
>>>> any errors, omissions, information and viruses contained in the
>>>> transmission of this message. Any opinions, conclusions and other
>>>> information contained within this message not related to Jini Guru official
>>>> business is deemed to be that of the individual only and is not endorsed by
>>>> Jini Guru.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Thu, Dec 12, 2019 at 1:17 AM Paul Carter-Brown
>>>> <[email protected]> wrote:
>>>> > Hi Jon,
>>>> >
>>>> > I took a look at the source and worked out that I could add my own
>>>> filters using openejb.additional.exclude.
>>>> >
>>>> > I set it to openejb.additional.exclude=com,net,io,org and the JVM
>>>> heap now sits at around 150MB whereas it would never go below 450MB
>>>> previously!!!
>>>> >
>>>> > Our EJB's and code is all in jars prefixed with guru.jini and hence
>>>> they are not filtered out and the system works 100%.
>>>> >
>>>> > My sense is that there should be an easier way to specify what jars
>>>> and/or packages to process for annotations. I have come across the
>>>> following settings but can't really work out what fits where:
>>>> >
>>>> > openejb.additional.exclude
>>>> > openejb.additional.include
>>>> > openejb.deployments.classpath.filter.descriptors
>>>> > openejb.deployments.classpath.filter.systemapps
>>>> > openejb.deployments.classpath.include
>>>> > openejb.deployments.classpath.exclude
>>>> > openejb.deployments.package.include
>>>> > openejb.deployments.package.exclude
>>>> >
>>>> >
>>>> > P.S. Perhaps also add these to the standard exclusion list as they
>>>> are common and yet won't need to be processed for annotations I guess?
>>>> > com.amazonaws
>>>> > com.fasterxml
>>>> > com.google
>>>> > com.hazelcast
>>>> > io.grpc
>>>> > io.netty
>>>> > org.docx4j
>>>> > org.mongodb
>>>> > org.rocksdb
>>>> > org.asynchttpclient
>>>> > org.apache
>>>> > org.aspectj
>>>> >
>>>> > But then maybe some are too broad and I don't really understand what
>>>> the annotation index/cache is really needed for at runtime? Can it not be
>>>> lazy loaded or discarded if unused? Just thinking out loud here :-( Seems
>>>> like the cache uses a significant amount of heap when considering the drive
>>>> towards tiny micro services.
>>>> >
>>>> > Paul Carter-Brown
>>>> > Director
>>>> > Jini Guru
>>>> > m: +27 (0) 83 442 7179
>>>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and
>>>> Leslie
>>>> > Johannesburg, South Africa
>>>> > w: jini.guru e: [email protected]
>>>> >
>>>> > Disclaimer: This message and/or attachment(s) may contain privileged,
>>>> confidential and/or personal information. If you are not the intended
>>>> recipient you may not disclose or distribute any of the information
>>>> contained within this message. In such case you must destroy this message
>>>> and inform the sender of the error. Jini Guru may not accept liability for
>>>> any errors, omissions, information and viruses contained in the
>>>> transmission of this message. Any opinions, conclusions and other
>>>> information contained within this message not related to Jini Guru official
>>>> business is deemed to be that of the individual only and is not endorsed by
>>>> Jini Guru.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Dec 11, 2019 at 7:18 PM Paul Carter-Brown
>>>> <[email protected]> wrote:
>>>> > Hi Jon,
>>>> >
>>>> > Unfortunately the snapshot behaves exactly the same way
>>>> >
>>>> > Paul Carter-Brown
>>>> > Director
>>>> > Jini Guru
>>>> > m: +27 (0) 83 442 7179
>>>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and
>>>> Leslie
>>>> > Johannesburg, South Africa
>>>> > w: jini.guru e: [email protected]
>>>> >
>>>> > Disclaimer: This message and/or attachment(s) may contain privileged,
>>>> confidential and/or personal information. If you are not the intended
>>>> recipient you may not disclose or distribute any of the information
>>>> contained within this message. In such case you must destroy this message
>>>> and inform the sender of the error. Jini Guru may not accept liability for
>>>> any errors, omissions, information and viruses contained in the
>>>> transmission of this message. Any opinions, conclusions and other
>>>> information contained within this message not related to Jini Guru official
>>>> business is deemed to be that of the individual only and is not endorsed by
>>>> Jini Guru.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Dec 11, 2019 at 3:39 PM Jonathan Gallimore <
>>>> [email protected]> wrote:
>>>> > Hi Paul
>>>> >
>>>> > Would you mind trying your application with a recent snapshot?
>>>> >
>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/tomee/apache-tomee/8.0.1-SNAPSHOT/
>>>> .
>>>> > I recently updated the exclude list.
>>>> >
>>>> > Many thanks
>>>> >
>>>> > Jon
>>>> >
>>>> > On Wed, Dec 11, 2019 at 12:31 PM Paul Carter-Brown
>>>> > <[email protected]> wrote:
>>>> >
>>>> > > Hi,
>>>> > >
>>>> > > I've been trying to lower the memory footprint of an EAR deployed
>>>> to TomEE
>>>> > > 8.0.0.
>>>> > >
>>>> > > Here is a screenshot of the heap histogram. The total Old Gen is
>>>> about
>>>> > > 450MB (after forcing multiple GC's). If I boot TomEE without my EAR
>>>> then
>>>> > > the old gen is about 6MB.
>>>> > >
>>>> > > [image: Screenshot from 2019-12-11 12-53-12.png]
>>>> > >
>>>> > > The entire ear is 140MB zip, most of which is in the ears /lib
>>>> directory
>>>> > > which contains libs such as Kafka, hazelcast, apache POI, Google
>>>> cloud
>>>> > > APIs, AWS client APIs etc etc. In total our code has about 100
>>>> actual
>>>> > > EJB's. If i remove files from the lib folder in the ear then I can
>>>> see that
>>>> > > the memory used by the annotation finder is lowered.
>>>> > >
>>>> > > Is there any way I can tell TomEE that it need not bother scanning
>>>> > > everything in the /lib folder of my EAR for annotations and fulling
>>>> up the
>>>> > > heap. I already
>>>> > > set openejb.deployments.classpath.include=.*jg-arch-core-impl.* to
>>>> scan
>>>> > > only the one jar in /lib which does have EJB's in it and it seems
>>>> to obey
>>>> > > this property but it doesn't seem to mean that annotation
>>>> processing is
>>>> > > skipped for all these other jars in /lib
>>>> > >
>>>> > > Thanks!
>>>> > >
>>>> > > Paul Carter-Brown
>>>> > > Director
>>>> > > Jini Guru
>>>> > > m: +27 (0) 83 442 7179 <+27834427179>
>>>> > > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and
>>>> Leslie
>>>> > > Johannesburg, South Africa
>>>> > > w: jini.guru e: [email protected]
>>>> > >
>>>> > > Disclaimer: This message and/or attachment(s) may contain
>>>> > > privileged, confidential and/or personal information. If you are
>>>> not the
>>>> > > intended recipient you may not disclose or distribute any of
>>>> > > the information contained within this message. In such case you must
>>>> > > destroy this message and inform the sender of the error. Jini Guru
>>>> may not
>>>> > > accept liability for any errors, omissions, information and viruses
>>>> > > contained in the transmission of this message. Any opinions,
>>>> conclusions
>>>> > > and other information contained within this message not related to
>>>> Jini
>>>> > > Guru official business is deemed to be that of the individual only
>>>> and is
>>>> > > not endorsed by Jini Guru.
>>>> > >
>>>> > >
>>>>
>>>>
