The problem is that values stored in ThreadLocals may live longer than the web application since the threads that hold these values are in a pool and the threads live on after the web application is undeployed. If the value is a JDK class this is not a problem.
If the value is a class that is defined in your webapp (or a library of the web app) then holding on the the value means the class cannot be garbage-collected. And voila, we have a memory leak... Garbage-free logging makes liberal use of ThreadLocals with non-JDK values. If the web container separates the thread pools between applications and restarts the thread pool with the web application, the above memory leak does not occur. (I believe this is what WebSphere does.) Eventually we may be able to address this with LOG4J2-1116 <https://issues.apache.org/jira/browse/LOG4J2-1116>. On Mon, Apr 18, 2016 at 11:04 PM, Matt Sicker <boa...@gmail.com> wrote: > So is this only bad to use when you have multiple wars on a single Tomcat, > or is it bad for when you try to undeploy, or is there another issue here? > > On 18 April 2016 at 09:01, Remko Popma <remko.po...@gmail.com> wrote: > >> Yes but if this is wrong because there is no threadpool or so, then you >> can use System property "log4j2.is.webapp" to tell Log4j that it should >> consider your app a non-webapp. >> >> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker <boa...@gmail.com> wrote: >> >>> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a >>> web application? >>> >>> On 18 April 2016 at 08:56, Remko Popma <remko.po...@gmail.com> wrote: >>> >>>> When I finish performance testing I will add graphs to the Performance >>>> section of the gc-free manual page so that users can make an informed >>>> decision. (The below one is an example.) >>>> >>>> So far, it seems that even with the garbage-free logging enabled, >>>> synchronous logging throughput (the green line) is usually slightly faster >>>> than it was with 2.5 (the purple line). So I think making garbage-free the >>>> default is acceptable. Also, users who care about logging throughput will >>>> be logging asynchronously, not synchronously: this scales much better with >>>> the number of application threads. >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal < >>>> mikael.stal...@magine.com> wrote: >>>> >>>>> Is it wise to enable by default given that "multi-threaded >>>>> applications that use synchronous logging may see worse performance"? >>>>> >>>>> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma <remko.po...@gmail.com> >>>>> wrote: >>>>> >>>>>> Enabled, except for web apps; for those log4j will be low garbage. >>>>>> >>>>>> See the first paragraph of the Configuration section in the manual >>>>>> page. >>>>>> >>>>>> Please let me know if this is not clear. (The fact that you asked >>>>>> means this probably needs improvement...) >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On 2016/04/18, at 20:18, Mikael Ståldal <mikael.stal...@magine.com> >>>>>> wrote: >>>>>> >>>>>> Will garbage-free logging be enabled or disabled by default? >>>>>> >>>>>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) <j...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> [ >>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245018#comment-15245018 >>>>>>> ] >>>>>>> >>>>>>> Remko Popma commented on LOG4J2-1297: >>>>>>> ------------------------------------- >>>>>>> >>>>>>> TODO allocate many temporary objects -> allocate temporary objects >>>>>>> >>>>>>> TODO multi-threaded applications that use synchronous logging may >>>>>>> see worse performance: the lock that was previously only around the IO >>>>>>> operation is widened to include the text formatting and conversion to >>>>>>> bytes. -> _synchronous_ logging performance may be worse for >>>>>>> multi-threaded >>>>>>> applications in this mode due to synchronization on the shared buffer. >>>>>>> If >>>>>>> your application is multi-threaded and logging performance is important, >>>>>>> consider using Async Loggers. >>>>>>> >>>>>>> TODO Caution: Only dates in the predefined formats are garbage-free >>>>>>> -> Caution: Only the predefined date formats are garbage-free >>>>>>> >>>>>>> TODO "(Note that Log4j may call toString() on message and parameter >>>>>>> objects when garbage-free logging is disabled because system property >>>>>>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call >>>>>>> toString() on message and parameter objects when garbage-free logging is >>>>>>> disabled (when system property log4j2.enable.threadlocals is set to >>>>>>> "false".) >>>>>>> >>>>>>> TODO "made an effort to make logging code garbage-free" -> "made an >>>>>>> effort to make logging garbage-free" >>>>>>> >>>>>>> >>>>>>> > Document "gc-free" configuration and performance >>>>>>> > ------------------------------------------------ >>>>>>> > >>>>>>> > Key: LOG4J2-1297 >>>>>>> > URL: >>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1297 >>>>>>> > Project: Log4j 2 >>>>>>> > Issue Type: New Feature >>>>>>> > Components: Documentation >>>>>>> > Affects Versions: 2.5 >>>>>>> > Reporter: Remko Popma >>>>>>> > Assignee: Remko Popma >>>>>>> > Fix For: 2.6 >>>>>>> > >>>>>>> > Attachments: log4j-2.5-FlightRecording.png, >>>>>>> log4j-2.6-FlightRecording.png >>>>>>> > >>>>>>> > >>>>>>> > Update the site with a description of which configurations are >>>>>>> GC-free (i.e., that don't create temporary objects in steady running >>>>>>> state). >>>>>>> > Currently that means >>>>>>> > * Loggers are all asynchronous (Log4jContextSelector is set to >>>>>>> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector). >>>>>>> > * The configuration does not contain a <Properties> section. >>>>>>> > * The "steady-state" appenders are either RandomAccessFile or >>>>>>> RollingRandomAccessFile Appenders (logging to any other appender will >>>>>>> cause >>>>>>> temporary objects to be created - including ConsoleAppender). >>>>>>> > * The Layout is a PatternLayout that uses one of the pre-defined >>>>>>> date formats, does not have any regular expression replacements, and >>>>>>> does >>>>>>> not have lookups (TODO: may need to restrict this further). >>>>>>> > * The thread name is cached (this is the [default| >>>>>>> https://issues.apache.org/jira/browse/LOG4J2-467]). Running with >>>>>>> -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage. >>>>>>> > * In user code, when logging a parameterized message, the number >>>>>>> of parameters is no more than ... (TBD pending discussion in >>>>>>> LOG4J2-1278). >>>>>>> > * In user code, when logging a parameterized message, parameters >>>>>>> of primitive type are boxed in a reused StringBuilder (Log4j provides a >>>>>>> utility to make this relatively painless). >>>>>>> > Even with the above restrictions, Log4j may occasionally create >>>>>>> garbage: >>>>>>> > * Initially StringBuilders are presized to 128 characters. They >>>>>>> may grow for larger messages (contributing to garbage in Old Gen). If >>>>>>> the >>>>>>> StringBuilder grows beyond 512 characters it is trimmed back to 512 >>>>>>> characters to prevent memory leaks from excessively long messages. >>>>>>> (TODO: >>>>>>> the resizing algorithm is {{size = value.length * 2 + 2}}, so a better >>>>>>> cutoff value is 518.) >>>>>>> > * Messages containing {{"$\{"}} will be converted to a String and >>>>>>> StrSubstitutor will be used to replace occurences of variables with >>>>>>> their >>>>>>> matching values. Multiple temporary objects are created during this >>>>>>> process. >>>>>>> > Furthermore, we need to explain that some of this functionality >>>>>>> depends on ThreadLocals and so is disabled by default in web >>>>>>> applications >>>>>>> to prevent memory leaks. The page should also explain how to manually >>>>>>> switch off the use of ThreadLocals. >>>>>>> > Finally, the page should show a performance test comparison >>>>>>> similar to the [performance section| >>>>>>> http://logging.apache.org/log4j/2.x/manual/async.html#Performance] >>>>>>> on the Async Loggers page. I'm thinking a comparison between Logback, >>>>>>> Log4j-1, Log4j-2.0, Log4j-2.6 "classic" and Log4j-2.6 "gc-free" would be >>>>>>> ideal. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> This message was sent by Atlassian JIRA >>>>>>> (v6.3.4#6332) >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org >>>>>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> [image: MagineTV] >>>>>> >>>>>> *Mikael Ståldal* >>>>>> Senior software developer >>>>>> >>>>>> *Magine TV* >>>>>> mikael.stal...@magine.com >>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >>>>>> >>>>>> Privileged and/or Confidential Information may be contained in this >>>>>> message. If you are not the addressee indicated in this message >>>>>> (or responsible for delivery of the message to such a person), you >>>>>> may not copy or deliver this message to anyone. In such case, >>>>>> you should destroy this message and kindly notify the sender by reply >>>>>> email. >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> [image: MagineTV] >>>>> >>>>> *Mikael Ståldal* >>>>> Senior software developer >>>>> >>>>> *Magine TV* >>>>> mikael.stal...@magine.com >>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >>>>> >>>>> Privileged and/or Confidential Information may be contained in this >>>>> message. If you are not the addressee indicated in this message >>>>> (or responsible for delivery of the message to such a person), you may >>>>> not copy or deliver this message to anyone. In such case, >>>>> you should destroy this message and kindly notify the sender by reply >>>>> email. >>>>> >>>> >>>> >>> >>> >>> -- >>> Matt Sicker <boa...@gmail.com> >>> >> >> > > > -- > Matt Sicker <boa...@gmail.com> >
