Re: Memory leak in EncodingDetector?
Very cool Chris, thanks for the quick reaction! Simon out On Fri, Mar 22, 2024 at 1:41 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > Simon, > > On 3/21/24 12:39, Simon Niederberger wrote: > > Hi Chris > > > > Personally I'd go with > > > > XML_INPUT_FACTORY = > > XMLInputFactory.newFactory(XMLInputFactory.class.getName(), > > EncodingDetector.class.getClassLoader()); > > > > allowing me to place my own JAR in common/lib if I really want to (the > > only scenario I can think of is an edge-case where there's a bug in > > the JVM XMLInputFactory and no upgrade path is available, so I'd place > > my own JAR in common/lib). > > > >> I wonder if there really are any use-cases for applications wanting > Tomcat to specify the XMLInputFactory *to be used for JSP* > > > > That sounds pretty far-fetched. Our use case is simply that we deploy > > frequently (several times a day) on a Tomcat 10.1 installation, and > > have various reasons for not going down the Docker/container/FAT JAR > > path. So, making sure we don't have memory leaks is relevant. > > Done and done. > > Feel free to put the fix in yourself, or grab 10.1.x from GitHub, or wit > for Tomcat 10.1.21 to be released (10.1.20 has already been rolled and > the voting will end shortly, so you'll have to wait for the next release). > > -chris > > > On Thu, Mar 21, 2024 at 1:44 PM Christopher Schultz > > wrote: > >> > >> Simon, > >> > >> On 3/20/24 15:36, Simon Niederberger wrote: > What if you create an empty jaxp.properties file and make it > available to the common ClassLoader (e.g. in > lib/empty-jaxp.jar:/jaxp.properties) -- does that prevent the problem? > >> > > >>> I think that boils down to what I'm already doing with the system > >>> property, getting to the FactoryFinder before it uses > >>> findServiceProvider(). Haven't tried it, but I'm sure it would work. > >> > >> The difference is that Tomcat can ship it as a part of the distribution, > >> while we shouldn't really be setting system properties like that. It > >> might prevent, for example, applications using their own preferred > >> implementation. > >> > >>> imho, Tomcat's EncodingDetector should init with either > >>> > >>> XML_INPUT_FACTORY = XMLInputFactory.newDefaultFactory(); > >>> > >>> to just always use the JVM XMLInputFactory, or then > >>> > >>> XML_INPUT_FACTORY = > >>> XMLInputFactory.newFactory(XMLInputFactory.class.getName(), > >>> EncodingDetector.class.getClassLoader()); > >>> > >>> to honor it's own classloader (maybe > >>> EncodingDetector.class.getClassLoader() is the wrong approach, > >>> basically something getTomcatCommonClassloader()) > >> > >> Yeah, that's probably better than an empty properties file wrapped in a > >> JAR file wrapped in an enigma. > >> > >> We also have the option of using the JreMemoryLeakPreventionListener for > >> this. There are already some XML-related protections in there, though > >> this one is not specifically there. > >> > >> Do you have a preferred technique for fixing this? All of those > >> suggestions seem equally reasonable. I think I have a slight preference > >> for passing the JSP compiler's ClassLoader in to the factory method: > >> > >> XML_INPUT_FACTORY = > >> XMLInputFactory.newFactory(XMLInputFactory.class.getName(), > >> EncodingDetector.class.getClassLoader()); > >> > >> If you absolutey need to get your JSP compiler to have a custom > >> XMLInputFactory, you can copy the JSP compiler into your application and > >> it will use that ClassLoader instead. > >> > >>> What I just don't get is why there's so little online about others > >>> havingEncodingDetector similar issues. Spring + libs like CXF or > >>> jackson-dataformat-xml are common, both those libs have transitive > >>> dependencies on woodstox-core. Throw in ehcache, also common, and your > >>> webapp won't undeploy if it's the first webapp to load a JSP and thus > >>> clinit EncodingDetector. Maybe the public has just given up on clean > >>> undeploying. > >> > >> My guess is that most people don't really care too much about clean > >> shutdowns of their applications. They either bring their applications up > >> and down along with the whole JVM (either standalone or via Docker, > >> etc.) or they use an embedded Tomcat where their own application hosts > >> Tomcat and they start/stop that. > >> > >> Or they disable/ignore memory-leak detection log messages. *shrug* > >> > >> I wonder if there really are any use-cases for applications wanting > >> Tomcat to specify the XMLInputFactory *to be used for JSP*. I suspect > not. > >> > >> -chris > >> > >>> On Wed, Mar 20, 2024 at 7:01 PM Christopher Schultz > >>> wrote: > > Simon, > > On 3/20/24 09:59, Simon Niederberger wrote: > > The whole thing is caused by Maven dependencies which pull in > > com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a > > > > @ServiceProvider(XMLInputFactory.class) > > > > annotation, where
Re: Memory leak in EncodingDetector?
Simon, On 3/21/24 12:39, Simon Niederberger wrote: Hi Chris Personally I'd go with XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); allowing me to place my own JAR in common/lib if I really want to (the only scenario I can think of is an edge-case where there's a bug in the JVM XMLInputFactory and no upgrade path is available, so I'd place my own JAR in common/lib). I wonder if there really are any use-cases for applications wanting Tomcat to specify the XMLInputFactory *to be used for JSP* That sounds pretty far-fetched. Our use case is simply that we deploy frequently (several times a day) on a Tomcat 10.1 installation, and have various reasons for not going down the Docker/container/FAT JAR path. So, making sure we don't have memory leaks is relevant. Done and done. Feel free to put the fix in yourself, or grab 10.1.x from GitHub, or wit for Tomcat 10.1.21 to be released (10.1.20 has already been rolled and the voting will end shortly, so you'll have to wait for the next release). -chris On Thu, Mar 21, 2024 at 1:44 PM Christopher Schultz wrote: Simon, On 3/20/24 15:36, Simon Niederberger wrote: What if you create an empty jaxp.properties file and make it available to the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- does that prevent the problem? > I think that boils down to what I'm already doing with the system property, getting to the FactoryFinder before it uses findServiceProvider(). Haven't tried it, but I'm sure it would work. The difference is that Tomcat can ship it as a part of the distribution, while we shouldn't really be setting system properties like that. It might prevent, for example, applications using their own preferred implementation. imho, Tomcat's EncodingDetector should init with either XML_INPUT_FACTORY = XMLInputFactory.newDefaultFactory(); to just always use the JVM XMLInputFactory, or then XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); to honor it's own classloader (maybe EncodingDetector.class.getClassLoader() is the wrong approach, basically something getTomcatCommonClassloader()) Yeah, that's probably better than an empty properties file wrapped in a JAR file wrapped in an enigma. We also have the option of using the JreMemoryLeakPreventionListener for this. There are already some XML-related protections in there, though this one is not specifically there. Do you have a preferred technique for fixing this? All of those suggestions seem equally reasonable. I think I have a slight preference for passing the JSP compiler's ClassLoader in to the factory method: XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); If you absolutey need to get your JSP compiler to have a custom XMLInputFactory, you can copy the JSP compiler into your application and it will use that ClassLoader instead. What I just don't get is why there's so little online about others havingEncodingDetector similar issues. Spring + libs like CXF or jackson-dataformat-xml are common, both those libs have transitive dependencies on woodstox-core. Throw in ehcache, also common, and your webapp won't undeploy if it's the first webapp to load a JSP and thus clinit EncodingDetector. Maybe the public has just given up on clean undeploying. My guess is that most people don't really care too much about clean shutdowns of their applications. They either bring their applications up and down along with the whole JVM (either standalone or via Docker, etc.) or they use an embedded Tomcat where their own application hosts Tomcat and they start/stop that. Or they disable/ignore memory-leak detection log messages. *shrug* I wonder if there really are any use-cases for applications wanting Tomcat to specify the XMLInputFactory *to be used for JSP*. I suspect not. -chris On Wed, Mar 20, 2024 at 7:01 PM Christopher Schultz wrote: Simon, On 3/20/24 09:59, Simon Niederberger wrote: The whole thing is caused by Maven dependencies which pull in com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a @ServiceProvider(XMLInputFactory.class) annotation, where ServiceProvider is org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the key code section, but I do find that the javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory as registered service provider. As WstxInputFactory is not on the common classpath (it's in WEB-INF/lib), I assume it's the webapp classloader which is used. Below is the stacktrace where EncodingDetector clinit happens (I defined a ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get stacktraces): Currently I'm setting System.setProperty("javax.xml.stream.XMLInputFactory", "com.sun.xml.internal.stream.XMLInputFactoryImpl"); to get a XMLInputFactory
Re: Memory leak in EncodingDetector?
Hi Chris Personally I'd go with XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); allowing me to place my own JAR in common/lib if I really want to (the only scenario I can think of is an edge-case where there's a bug in the JVM XMLInputFactory and no upgrade path is available, so I'd place my own JAR in common/lib). > I wonder if there really are any use-cases for applications wanting Tomcat to > specify the XMLInputFactory *to be used for JSP* That sounds pretty far-fetched. Our use case is simply that we deploy frequently (several times a day) on a Tomcat 10.1 installation, and have various reasons for not going down the Docker/container/FAT JAR path. So, making sure we don't have memory leaks is relevant. Simon On Thu, Mar 21, 2024 at 1:44 PM Christopher Schultz wrote: > > Simon, > > On 3/20/24 15:36, Simon Niederberger wrote: > >> What if you create an empty jaxp.properties file and make it available to > >> the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- > >> does that prevent the problem? > > > > I think that boils down to what I'm already doing with the system > > property, getting to the FactoryFinder before it uses > > findServiceProvider(). Haven't tried it, but I'm sure it would work. > > The difference is that Tomcat can ship it as a part of the distribution, > while we shouldn't really be setting system properties like that. It > might prevent, for example, applications using their own preferred > implementation. > > > imho, Tomcat's EncodingDetector should init with either > > > > XML_INPUT_FACTORY = XMLInputFactory.newDefaultFactory(); > > > > to just always use the JVM XMLInputFactory, or then > > > > XML_INPUT_FACTORY = > > XMLInputFactory.newFactory(XMLInputFactory.class.getName(), > > EncodingDetector.class.getClassLoader()); > > > > to honor it's own classloader (maybe > > EncodingDetector.class.getClassLoader() is the wrong approach, > > basically something getTomcatCommonClassloader()) > > Yeah, that's probably better than an empty properties file wrapped in a > JAR file wrapped in an enigma. > > We also have the option of using the JreMemoryLeakPreventionListener for > this. There are already some XML-related protections in there, though > this one is not specifically there. > > Do you have a preferred technique for fixing this? All of those > suggestions seem equally reasonable. I think I have a slight preference > for passing the JSP compiler's ClassLoader in to the factory method: > >XML_INPUT_FACTORY = > XMLInputFactory.newFactory(XMLInputFactory.class.getName(), > EncodingDetector.class.getClassLoader()); > > If you absolutey need to get your JSP compiler to have a custom > XMLInputFactory, you can copy the JSP compiler into your application and > it will use that ClassLoader instead. > > > What I just don't get is why there's so little online about others > > havingEncodingDetector similar issues. Spring + libs like CXF or > > jackson-dataformat-xml are common, both those libs have transitive > > dependencies on woodstox-core. Throw in ehcache, also common, and your > > webapp won't undeploy if it's the first webapp to load a JSP and thus > > clinit EncodingDetector. Maybe the public has just given up on clean > > undeploying. > > My guess is that most people don't really care too much about clean > shutdowns of their applications. They either bring their applications up > and down along with the whole JVM (either standalone or via Docker, > etc.) or they use an embedded Tomcat where their own application hosts > Tomcat and they start/stop that. > > Or they disable/ignore memory-leak detection log messages. *shrug* > > I wonder if there really are any use-cases for applications wanting > Tomcat to specify the XMLInputFactory *to be used for JSP*. I suspect not. > > -chris > > > On Wed, Mar 20, 2024 at 7:01 PM Christopher Schultz > > wrote: > >> > >> Simon, > >> > >> On 3/20/24 09:59, Simon Niederberger wrote: > >>> The whole thing is caused by Maven dependencies which pull in > >>> com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a > >>> > >>> @ServiceProvider(XMLInputFactory.class) > >>> > >>> annotation, where ServiceProvider is > >>> org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the > >>> key code section, but I do find that the > >>> javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory > >>> as registered service provider. As WstxInputFactory is not on the > >>> common classpath (it's in WEB-INF/lib), I assume it's the webapp > >>> classloader which is used. Below is the stacktrace where > >>> EncodingDetector clinit happens (I defined a > >>> ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get > >>> stacktraces): > >>> > >>> Currently I'm setting > >>> > >>> System.setProperty("javax.xml.stream.XMLInputFactory", > >>> "com.sun.xml.internal.stream.XMLInputFactoryImpl"); > >>> to get a
Re: Memory leak in EncodingDetector?
Simon, On 3/20/24 15:36, Simon Niederberger wrote: What if you create an empty jaxp.properties file and make it available to the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- does that prevent the problem? > I think that boils down to what I'm already doing with the system property, getting to the FactoryFinder before it uses findServiceProvider(). Haven't tried it, but I'm sure it would work. The difference is that Tomcat can ship it as a part of the distribution, while we shouldn't really be setting system properties like that. It might prevent, for example, applications using their own preferred implementation. imho, Tomcat's EncodingDetector should init with either XML_INPUT_FACTORY = XMLInputFactory.newDefaultFactory(); to just always use the JVM XMLInputFactory, or then XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); to honor it's own classloader (maybe EncodingDetector.class.getClassLoader() is the wrong approach, basically something getTomcatCommonClassloader()) Yeah, that's probably better than an empty properties file wrapped in a JAR file wrapped in an enigma. We also have the option of using the JreMemoryLeakPreventionListener for this. There are already some XML-related protections in there, though this one is not specifically there. Do you have a preferred technique for fixing this? All of those suggestions seem equally reasonable. I think I have a slight preference for passing the JSP compiler's ClassLoader in to the factory method: XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); If you absolutey need to get your JSP compiler to have a custom XMLInputFactory, you can copy the JSP compiler into your application and it will use that ClassLoader instead. What I just don't get is why there's so little online about others havingEncodingDetector similar issues. Spring + libs like CXF or jackson-dataformat-xml are common, both those libs have transitive dependencies on woodstox-core. Throw in ehcache, also common, and your webapp won't undeploy if it's the first webapp to load a JSP and thus clinit EncodingDetector. Maybe the public has just given up on clean undeploying. My guess is that most people don't really care too much about clean shutdowns of their applications. They either bring their applications up and down along with the whole JVM (either standalone or via Docker, etc.) or they use an embedded Tomcat where their own application hosts Tomcat and they start/stop that. Or they disable/ignore memory-leak detection log messages. *shrug* I wonder if there really are any use-cases for applications wanting Tomcat to specify the XMLInputFactory *to be used for JSP*. I suspect not. -chris On Wed, Mar 20, 2024 at 7:01 PM Christopher Schultz wrote: Simon, On 3/20/24 09:59, Simon Niederberger wrote: The whole thing is caused by Maven dependencies which pull in com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a @ServiceProvider(XMLInputFactory.class) annotation, where ServiceProvider is org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the key code section, but I do find that the javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory as registered service provider. As WstxInputFactory is not on the common classpath (it's in WEB-INF/lib), I assume it's the webapp classloader which is used. Below is the stacktrace where EncodingDetector clinit happens (I defined a ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get stacktraces): Currently I'm setting System.setProperty("javax.xml.stream.XMLInputFactory", "com.sun.xml.internal.stream.XMLInputFactoryImpl"); to get a XMLInputFactory implementation which is on the common loader's classpath, so the webapp can be undeployed cleanly. So this works, right? What if you create an empty jaxp.properties file and make it available to the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- does that prevent the problem? I'm wondering if Tomcat should simply ship with an empty jaxp.properties file to prevent this kind of thing from happening by default. If someone wants to bundle an XMLInputFactory into Tomcat's lib/ directory and use that, they could remove this file. BTW that's an impressive stack trace. ;) -chris java.lang.RuntimeException: Stracktrace for tracking XMLInputFactory creation at ch.want.funnel.FunnelApp$DelegatingXMLInputFactory.(FunnelApp.java:107) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at
Re: Memory leak in EncodingDetector?
Hi Chris Spring's ObservationFilterChainDecorator is ridiculous, isn't it? > What if you create an empty jaxp.properties file and make it available to the > common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- does that > prevent the problem? I think that boils down to what I'm already doing with the system property, getting to the FactoryFinder before it uses findServiceProvider(). Haven't tried it, but I'm sure it would work. imho, Tomcat's EncodingDetector should init with either XML_INPUT_FACTORY = XMLInputFactory.newDefaultFactory(); to just always use the JVM XMLInputFactory, or then XML_INPUT_FACTORY = XMLInputFactory.newFactory(XMLInputFactory.class.getName(), EncodingDetector.class.getClassLoader()); to honor it's own classloader (maybe EncodingDetector.class.getClassLoader() is the wrong approach, basically something getTomcatCommonClassloader()) What I just don't get is why there's so little online about others havingEncodingDetector similar issues. Spring + libs like CXF or jackson-dataformat-xml are common, both those libs have transitive dependencies on woodstox-core. Throw in ehcache, also common, and your webapp won't undeploy if it's the first webapp to load a JSP and thus clinit EncodingDetector. Maybe the public has just given up on clean undeploying. Simon Mühlegasse 18, 6340 Baar, Switzerland https://www.want.ch https://www.funnel.travel On Wed, Mar 20, 2024 at 7:01 PM Christopher Schultz wrote: > > Simon, > > On 3/20/24 09:59, Simon Niederberger wrote: > > The whole thing is caused by Maven dependencies which pull in > > com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a > > > > @ServiceProvider(XMLInputFactory.class) > > > > annotation, where ServiceProvider is > > org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the > > key code section, but I do find that the > > javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory > > as registered service provider. As WstxInputFactory is not on the > > common classpath (it's in WEB-INF/lib), I assume it's the webapp > > classloader which is used. Below is the stacktrace where > > EncodingDetector clinit happens (I defined a > > ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get > > stacktraces): > > > > Currently I'm setting > > > > System.setProperty("javax.xml.stream.XMLInputFactory", > > "com.sun.xml.internal.stream.XMLInputFactoryImpl"); > > to get a XMLInputFactory implementation which is on the common > > loader's classpath, so the webapp can be undeployed cleanly. > > So this works, right? > > What if you create an empty jaxp.properties file and make it available > to the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) > -- does that prevent the problem? > > I'm wondering if Tomcat should simply ship with an empty jaxp.properties > file to prevent this kind of thing from happening by default. If someone > wants to bundle an XMLInputFactory into Tomcat's lib/ directory and use > that, they could remove this file. > > BTW that's an impressive stack trace. ;) > > -chris > > > java.lang.RuntimeException: Stracktrace for tracking XMLInputFactory > > creation > > at > > ch.want.funnel.FunnelApp$DelegatingXMLInputFactory.(FunnelApp.java:107) > > at > > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method) > > at > > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) > > at > > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at > > java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) > > at > > java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) > > at > > java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:190) > > at > > java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:148) > > at > > java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:261) > > at > > java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:223) > > at > > java.xml/javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:166) > > at > > org.apache.jasper.compiler.EncodingDetector.(EncodingDetector.java:38) > > at > > org.apache.jasper.compiler.ParserController.determineSyntaxAndEncoding(ParserController.java:324) > > at > > org.apache.jasper.compiler.ParserController.doParse(ParserController.java:201) > > at > > org.apache.jasper.compiler.ParserController.parseDirectives(ParserController.java:128) > > at > > org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:207) > > at org.apache.jasper.compiler.Compiler.compile(Compiler.java:396) > > at
Re: Memory leak in EncodingDetector?
Simon, On 3/20/24 09:59, Simon Niederberger wrote: The whole thing is caused by Maven dependencies which pull in com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a @ServiceProvider(XMLInputFactory.class) annotation, where ServiceProvider is org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the key code section, but I do find that the javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory as registered service provider. As WstxInputFactory is not on the common classpath (it's in WEB-INF/lib), I assume it's the webapp classloader which is used. Below is the stacktrace where EncodingDetector clinit happens (I defined a ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get stacktraces): Currently I'm setting System.setProperty("javax.xml.stream.XMLInputFactory", "com.sun.xml.internal.stream.XMLInputFactoryImpl"); to get a XMLInputFactory implementation which is on the common loader's classpath, so the webapp can be undeployed cleanly. So this works, right? What if you create an empty jaxp.properties file and make it available to the common ClassLoader (e.g. in lib/empty-jaxp.jar:/jaxp.properties) -- does that prevent the problem? I'm wondering if Tomcat should simply ship with an empty jaxp.properties file to prevent this kind of thing from happening by default. If someone wants to bundle an XMLInputFactory into Tomcat's lib/ directory and use that, they could remove this file. BTW that's an impressive stack trace. ;) -chris java.lang.RuntimeException: Stracktrace for tracking XMLInputFactory creation at ch.want.funnel.FunnelApp$DelegatingXMLInputFactory.(FunnelApp.java:107) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) at java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:190) at java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:148) at java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:261) at java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:223) at java.xml/javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:166) at org.apache.jasper.compiler.EncodingDetector.(EncodingDetector.java:38) at org.apache.jasper.compiler.ParserController.determineSyntaxAndEncoding(ParserController.java:324) at org.apache.jasper.compiler.ParserController.doParse(ParserController.java:201) at org.apache.jasper.compiler.ParserController.parseDirectives(ParserController.java:128) at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:207) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:396) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:372) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:356) at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:603) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:396) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:380) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:328) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) at
Re: Memory leak in EncodingDetector?
Hi Chris The whole thing is caused by Maven dependencies which pull in com.fasterxml.woodstox:woodstox-core. The WstxInputFactory has a @ServiceProvider(XMLInputFactory.class) annotation, where ServiceProvider is org.ehcache.spi.service.ServiceProvider. I didn't manage to trace the key code section, but I do find that the javax.xml.stream.FactoryFinder then ends up finding WstxInputFactory as registered service provider. As WstxInputFactory is not on the common classpath (it's in WEB-INF/lib), I assume it's the webapp classloader which is used. Below is the stacktrace where EncodingDetector clinit happens (I defined a ch.want.funnel.FunnelApp$DelegatingXMLInputFactory to get stacktraces): Currently I'm setting System.setProperty("javax.xml.stream.XMLInputFactory", "com.sun.xml.internal.stream.XMLInputFactoryImpl"); to get a XMLInputFactory implementation which is on the common loader's classpath, so the webapp can be undeployed cleanly. Best regards Simon -- java.lang.RuntimeException: Stracktrace for tracking XMLInputFactory creation at ch.want.funnel.FunnelApp$DelegatingXMLInputFactory.(FunnelApp.java:107) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) at java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:190) at java.xml/javax.xml.stream.FactoryFinder.newInstance(FactoryFinder.java:148) at java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:261) at java.xml/javax.xml.stream.FactoryFinder.find(FactoryFinder.java:223) at java.xml/javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:166) at org.apache.jasper.compiler.EncodingDetector.(EncodingDetector.java:38) at org.apache.jasper.compiler.ParserController.determineSyntaxAndEncoding(ParserController.java:324) at org.apache.jasper.compiler.ParserController.doParse(ParserController.java:201) at org.apache.jasper.compiler.ParserController.parseDirectives(ParserController.java:128) at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:207) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:396) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:372) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:356) at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:603) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:396) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:380) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:328) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479) at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340) at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128) at
Re: Memory leak in EncodingDetector?
Simon, On 3/18/24 15:17, Simon Niederberger wrote: I'm analyzing a memory leak reported by Tomcat, and have narrowed it down to org.apache.jasper.compiler.EncodingDetector: private static final XMLInputFactory XML_INPUT_FACTORY; static { XML_INPUT_FACTORY = XMLInputFactory.newInstance(); } This class is called by webapp code on a GET request at org.apache.jasper.compiler.EncodingDetector.(EncodingDetector.java:38) at org.apache.jasper.compiler.ParserController.determineSyntaxAndEncoding(ParserController.java:324) at org.apache.jasper.compiler.ParserController.doParse(ParserController.java:201) at org.apache.jasper.compiler.ParserController.parseDirectives(ParserController.java:128) at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:207) ... at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:396) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:380) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:328) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) The EncodingDetector class, if not yet loaded, will be loaded in the common classloader, then continue by loading the XMLInputFactory using the webapp context, and might end up with a XMLInputFactory implementation from a webapp-provided JAR. If that happens, the webapp can't undeploy. (In my case, woodstox WstxInputFactory registers itself as ServiceProvider for XMLInputFactory) For completeness: javax.xml.stream.FactoryFinder.findServiceProvider() is called without classloader (cl = null), and has if (cl == null) { //the current thread's context class loader serviceLoader = ServiceLoader.load(type); } else { serviceLoader = ServiceLoader.load(type, cl); } I can't find anything online about memory leaks from webapp-provided XMLInputFactory implementations, but this must be fairly common. Is my understanding correct, or have I mis-configured something? (I'm mainly wondering whether any XMLInputFactory-implementing JARs belong in tomcat/lib, but again I'm not finding anything online confirming that) Tomcat 10.1.19 JVM 17.0.10+7-Ubuntu-120.04.1 Ubuntu 20.04.6 LTS I'm not sure how many web applications ship with an XMLInputSource, but they definitely do exist. I'm fairly sure most applications won't set a system property or ship with a stax.properties/jaxp.properties file to override the default implementation, but of course if it's possible, someone will eventually do it. I'm curious: in your example, how are you declaring your implementation class, and which implementation are you using? Are you able to log in EncodingDetector. what the value of the thread's context classloader is? I would expect that it's using the common classloader, as you say, and that the implementation class would also be loaded using that same classloader. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Memory leak in EncodingDetector?
Hi I'm analyzing a memory leak reported by Tomcat, and have narrowed it down to org.apache.jasper.compiler.EncodingDetector: private static final XMLInputFactory XML_INPUT_FACTORY; static { XML_INPUT_FACTORY = XMLInputFactory.newInstance(); } This class is called by webapp code on a GET request at org.apache.jasper.compiler.EncodingDetector.(EncodingDetector.java:38) at org.apache.jasper.compiler.ParserController.determineSyntaxAndEncoding(ParserController.java:324) at org.apache.jasper.compiler.ParserController.doParse(ParserController.java:201) at org.apache.jasper.compiler.ParserController.parseDirectives(ParserController.java:128) at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:207) ... at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:396) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:380) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:328) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) The EncodingDetector class, if not yet loaded, will be loaded in the common classloader, then continue by loading the XMLInputFactory using the webapp context, and might end up with a XMLInputFactory implementation from a webapp-provided JAR. If that happens, the webapp can't undeploy. (In my case, woodstox WstxInputFactory registers itself as ServiceProvider for XMLInputFactory) For completeness: javax.xml.stream.FactoryFinder.findServiceProvider() is called without classloader (cl = null), and has if (cl == null) { //the current thread's context class loader serviceLoader = ServiceLoader.load(type); } else { serviceLoader = ServiceLoader.load(type, cl); } I can't find anything online about memory leaks from webapp-provided XMLInputFactory implementations, but this must be fairly common. Is my understanding correct, or have I mis-configured something? (I'm mainly wondering whether any XMLInputFactory-implementing JARs belong in tomcat/lib, but again I'm not finding anything online confirming that) Tomcat 10.1.19 JVM 17.0.10+7-Ubuntu-120.04.1 Ubuntu 20.04.6 LTS Simon - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org