Great, thanks for the explanation! I can't wait to use this set-up in production.
On 18 April 2016 at 17:48, Remko Popma <remko.po...@gmail.com> wrote: > Yes, in that case the threadpool doesn't outlive the application so > there's no chance of memory leaks. > > Not sure if Tomcat also can be configured to have separate threadpools per > web app and to refresh the thread pool on web app undeploy/redeploy. > > Sent from my iPhone > > On 2016/04/19, at 3:27, Matt Sicker <boa...@gmail.com> wrote: > > But would you say it's safe to use ThreadLocal when you don't undeploy or > redeploy web apps? Like if you always restart Tomcat when you redeploy, > would it be safe to use ThreadLocal in such a scenario? > > On 18 April 2016 at 09:49, Remko Popma <remko.po...@gmail.com> wrote: > >> The problem is that values stored in ThreadLocals may live longer than >> the web application since the threads that hold these values are in a pool >> and the threads live on after the web application is undeployed. If the >> value is a JDK class this is not a problem. >> >> If the value is a class that is defined in your webapp (or a library of >> the web app) then holding on the the value means the class cannot be >> garbage-collected. And voila, we have a memory leak... >> >> Garbage-free logging makes liberal use of ThreadLocals with non-JDK >> values. If the web container separates the thread pools between >> applications and restarts the thread pool with the web application, the >> above memory leak does not occur. (I believe this is what WebSphere does.) >> >> Eventually we may be able to address this with LOG4J2-1116 >> <https://issues.apache.org/jira/browse/LOG4J2-1116>. >> >> >> >> On Mon, Apr 18, 2016 at 11:04 PM, Matt Sicker <boa...@gmail.com> wrote: >> >>> So is this only bad to use when you have multiple wars on a single >>> Tomcat, or is it bad for when you try to undeploy, or is there another >>> issue here? >>> >>> On 18 April 2016 at 09:01, Remko Popma <remko.po...@gmail.com> wrote: >>> >>>> Yes but if this is wrong because there is no threadpool or so, then you >>>> can use System property "log4j2.is.webapp" to tell Log4j that it >>>> should consider your app a non-webapp. >>>> >>>> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker <boa...@gmail.com> wrote: >>>> >>>>> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a >>>>> web application? >>>>> >>>>> On 18 April 2016 at 08:56, Remko Popma <remko.po...@gmail.com> wrote: >>>>> >>>>>> When I finish performance testing I will add graphs to the >>>>>> Performance section of the gc-free manual page so that users can make an >>>>>> informed decision. (The below one is an example.) >>>>>> >>>>>> So far, it seems that even with the garbage-free logging enabled, >>>>>> synchronous logging throughput (the green line) is usually slightly >>>>>> faster >>>>>> than it was with 2.5 (the purple line). So I think making garbage-free >>>>>> the >>>>>> default is acceptable. Also, users who care about logging throughput will >>>>>> be logging asynchronously, not synchronously: this scales much better >>>>>> with >>>>>> the number of application threads. >>>>>> >>>>>> >>>>>> <ThroughputSyncLinux.png> >>>>>> >>>>>> >>>>>> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal < >>>>>> mikael.stal...@magine.com> wrote: >>>>>> >>>>>>> Is it wise to enable by default given that "multi-threaded >>>>>>> applications that use synchronous logging may see worse performance"? >>>>>>> >>>>>>> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma <remko.po...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Enabled, except for web apps; for those log4j will be low garbage. >>>>>>>> >>>>>>>> See the first paragraph of the Configuration section in the manual >>>>>>>> page. >>>>>>>> >>>>>>>> Please let me know if this is not clear. (The fact that you asked >>>>>>>> means this probably needs improvement...) >>>>>>>> >>>>>>>> Sent from my iPhone >>>>>>>> >>>>>>>> On 2016/04/18, at 20:18, Mikael Ståldal <mikael.stal...@magine.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Will garbage-free logging be enabled or disabled by default? >>>>>>>> >>>>>>>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) < >>>>>>>> j...@apache.org> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> [ >>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245018#comment-15245018 >>>>>>>>> ] >>>>>>>>> >>>>>>>>> Remko Popma commented on LOG4J2-1297: >>>>>>>>> ------------------------------------- >>>>>>>>> >>>>>>>>> TODO allocate many temporary objects -> allocate temporary objects >>>>>>>>> >>>>>>>>> TODO multi-threaded applications that use synchronous logging may >>>>>>>>> see worse performance: the lock that was previously only around the IO >>>>>>>>> operation is widened to include the text formatting and conversion to >>>>>>>>> bytes. -> _synchronous_ logging performance may be worse for >>>>>>>>> multi-threaded >>>>>>>>> applications in this mode due to synchronization on the shared >>>>>>>>> buffer. If >>>>>>>>> your application is multi-threaded and logging performance is >>>>>>>>> important, >>>>>>>>> consider using Async Loggers. >>>>>>>>> >>>>>>>>> TODO Caution: Only dates in the predefined formats are >>>>>>>>> garbage-free -> Caution: Only the predefined date formats are >>>>>>>>> garbage-free >>>>>>>>> >>>>>>>>> TODO "(Note that Log4j may call toString() on message and >>>>>>>>> parameter objects when garbage-free logging is disabled because system >>>>>>>>> property log4j2.enable.threadlocals is set to "false".)" -> Log4j may >>>>>>>>> call >>>>>>>>> toString() on message and parameter objects when garbage-free logging >>>>>>>>> is >>>>>>>>> disabled (when system property log4j2.enable.threadlocals is set to >>>>>>>>> "false".) >>>>>>>>> >>>>>>>>> TODO "made an effort to make logging code garbage-free" -> "made >>>>>>>>> an effort to make logging garbage-free" >>>>>>>>> >>>>>>>>> >>>>>>>>> > Document "gc-free" configuration and performance >>>>>>>>> > ------------------------------------------------ >>>>>>>>> > >>>>>>>>> > Key: LOG4J2-1297 >>>>>>>>> > URL: >>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1297 >>>>>>>>> > Project: Log4j 2 >>>>>>>>> > Issue Type: New Feature >>>>>>>>> > Components: Documentation >>>>>>>>> > Affects Versions: 2.5 >>>>>>>>> > Reporter: Remko Popma >>>>>>>>> > Assignee: Remko Popma >>>>>>>>> > Fix For: 2.6 >>>>>>>>> > >>>>>>>>> > Attachments: log4j-2.5-FlightRecording.png, >>>>>>>>> log4j-2.6-FlightRecording.png >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Update the site with a description of which configurations are >>>>>>>>> GC-free (i.e., that don't create temporary objects in steady running >>>>>>>>> state). >>>>>>>>> > Currently that means >>>>>>>>> > * Loggers are all asynchronous (Log4jContextSelector is set to >>>>>>>>> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector). >>>>>>>>> > * The configuration does not contain a <Properties> section. >>>>>>>>> > * The "steady-state" appenders are either RandomAccessFile or >>>>>>>>> RollingRandomAccessFile Appenders (logging to any other appender will >>>>>>>>> cause >>>>>>>>> temporary objects to be created - including ConsoleAppender). >>>>>>>>> > * The Layout is a PatternLayout that uses one of the pre-defined >>>>>>>>> date formats, does not have any regular expression replacements, and >>>>>>>>> does >>>>>>>>> not have lookups (TODO: may need to restrict this further). >>>>>>>>> > * The thread name is cached (this is the [default| >>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-467]). Running with >>>>>>>>> -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage. >>>>>>>>> > * In user code, when logging a parameterized message, the number >>>>>>>>> of parameters is no more than ... (TBD pending discussion in >>>>>>>>> LOG4J2-1278). >>>>>>>>> > * In user code, when logging a parameterized message, parameters >>>>>>>>> of primitive type are boxed in a reused StringBuilder (Log4j provides >>>>>>>>> a >>>>>>>>> utility to make this relatively painless). >>>>>>>>> > Even with the above restrictions, Log4j may occasionally create >>>>>>>>> garbage: >>>>>>>>> > * Initially StringBuilders are presized to 128 characters. They >>>>>>>>> may grow for larger messages (contributing to garbage in Old Gen). If >>>>>>>>> the >>>>>>>>> StringBuilder grows beyond 512 characters it is trimmed back to 512 >>>>>>>>> characters to prevent memory leaks from excessively long messages. >>>>>>>>> (TODO: >>>>>>>>> the resizing algorithm is {{size = value.length * 2 + 2}}, so a better >>>>>>>>> cutoff value is 518.) >>>>>>>>> > * Messages containing {{"$\{"}} will be converted to a String >>>>>>>>> and StrSubstitutor will be used to replace occurences of variables >>>>>>>>> with >>>>>>>>> their matching values. Multiple temporary objects are created during >>>>>>>>> this >>>>>>>>> process. >>>>>>>>> > Furthermore, we need to explain that some of this functionality >>>>>>>>> depends on ThreadLocals and so is disabled by default in web >>>>>>>>> applications >>>>>>>>> to prevent memory leaks. The page should also explain how to manually >>>>>>>>> switch off the use of ThreadLocals. >>>>>>>>> > Finally, the page should show a performance test comparison >>>>>>>>> similar to the [performance section| >>>>>>>>> http://logging.apache.org/log4j/2.x/manual/async.html#Performance] >>>>>>>>> on the Async Loggers page. I'm thinking a comparison between Logback, >>>>>>>>> Log4j-1, Log4j-2.0, Log4j-2.6 "classic" and Log4j-2.6 "gc-free" would >>>>>>>>> be >>>>>>>>> ideal. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> This message was sent by Atlassian JIRA >>>>>>>>> (v6.3.4#6332) >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org >>>>>>>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> [image: MagineTV] >>>>>>>> >>>>>>>> *Mikael Ståldal* >>>>>>>> Senior software developer >>>>>>>> >>>>>>>> *Magine TV* >>>>>>>> mikael.stal...@magine.com >>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >>>>>>>> >>>>>>>> Privileged and/or Confidential Information may be contained in this >>>>>>>> message. If you are not the addressee indicated in this message >>>>>>>> (or responsible for delivery of the message to such a person), you >>>>>>>> may not copy or deliver this message to anyone. In such case, >>>>>>>> you should destroy this message and kindly notify the sender by >>>>>>>> reply email. >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> [image: MagineTV] >>>>>>> >>>>>>> *Mikael Ståldal* >>>>>>> Senior software developer >>>>>>> >>>>>>> *Magine TV* >>>>>>> mikael.stal...@magine.com >>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >>>>>>> >>>>>>> Privileged and/or Confidential Information may be contained in this >>>>>>> message. If you are not the addressee indicated in this message >>>>>>> (or responsible for delivery of the message to such a person), you >>>>>>> may not copy or deliver this message to anyone. In such case, >>>>>>> you should destroy this message and kindly notify the sender by >>>>>>> reply email. >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Matt Sicker <boa...@gmail.com> >>>>> >>>> >>>> >>> >>> >>> -- >>> Matt Sicker <boa...@gmail.com> >>> >> >> > > > -- > Matt Sicker <boa...@gmail.com> > > -- Matt Sicker <boa...@gmail.com>
