Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Remko Popma]

Yes, in that case the threadpool doesn't outlive the application so there's no 
chance of memory leaks. 

Not sure if Tomcat also can be configured to have separate threadpools per web 
app and to refresh the thread pool on web app undeploy/redeploy. 

Sent from my iPhone

> On 2016/04/19, at 3:27, Matt Sicker  wrote:
> 
> But would you say it's safe to use ThreadLocal when you don't undeploy or 
> redeploy web apps? Like if you always restart Tomcat when you redeploy, would 
> it be safe to use ThreadLocal in such a scenario?
> 
>> On 18 April 2016 at 09:49, Remko Popma  wrote:
>> The problem is that values stored in ThreadLocals may live longer than the 
>> web application since the threads that hold these values are in a pool and 
>> the threads live on after the web application is undeployed. If the value is 
>> a JDK class this is not a problem.
>> 
>> If the value is a class that is defined in your webapp (or a library of the 
>> web app) then holding on the the value means the class cannot be 
>> garbage-collected. And voila, we have a memory leak...
>> 
>> Garbage-free logging makes liberal use of ThreadLocals with non-JDK values. 
>> If the web container separates the thread pools between applications and 
>> restarts the thread pool with the web application, the above memory leak 
>> does not occur. (I believe this is what WebSphere does.)
>> 
>> Eventually we may be able to address this with LOG4J2-1116.
>> 
>> 
>> 
>>> On Mon, Apr 18, 2016 at 11:04 PM, Matt Sicker  wrote:
>>> So is this only bad to use when you have multiple wars on a single Tomcat, 
>>> or is it bad for when you try to undeploy, or is there another issue here?
>>> 
 On 18 April 2016 at 09:01, Remko Popma  wrote:
 Yes but if this is wrong because there is no threadpool or so, then you 
 can use System property "log4j2.is.webapp" to tell Log4j that it should 
 consider your app a non-webapp.
 
> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker  wrote:
> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a 
> web application?
> 
>> On 18 April 2016 at 08:56, Remko Popma  wrote:
>> When I finish performance testing I will add graphs to the Performance 
>> section of the gc-free manual page so that users can make an informed 
>> decision. (The below one is an example.)
>> 
>> So far, it seems that even with the garbage-free logging enabled, 
>> synchronous logging throughput (the green line) is usually slightly 
>> faster than it was with 2.5 (the purple line). So I think making 
>> garbage-free the default is acceptable. Also, users who care about 
>> logging throughput will be logging asynchronously, not synchronously: 
>> this scales much better with the number of application threads.
>> 
>> 
>> 
>> ​
>> 
>>> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal 
>>>  wrote:
>>> Is it wise to enable by default given that "multi-threaded applications 
>>> that use synchronous logging may see worse performance"?
>>> 
 On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma  
 wrote:
 Enabled, except for web apps; for those log4j will be low garbage. 
 
 See the first paragraph of the Configuration section in the manual 
 page. 
 
 Please let me know if this is not clear. (The fact that you asked 
 means this probably needs improvement...)
 
 Sent from my iPhone
 
> On 2016/04/18, at 20:18, Mikael Ståldal  
> wrote:
> 
> Will garbage-free logging be enabled or disabled by default?
> 
>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
>>  wrote:
>> 
>> [ 
>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>>  ]
>> 
>> Remko Popma commented on LOG4J2-1297:
>> -
>> 
>> TODO allocate many temporary objects -> allocate temporary objects
>> 
>> TODO multi-threaded applications that use synchronous logging may 
>> see worse performance: the lock that was previously only around the 
>> IO operation is widened to include the text formatting and 
>> conversion to bytes. -> _synchronous_ logging performance may be 
>> worse for multi-threaded applications in this mode due to 
>> synchronization on the shared buffer. If your application is 
>> multi-threaded and logging performance is important, consider using 
>> Async Loggers.
>> 
>> TODO Caution: Only dates in the 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Matt Sicker]

But would you say it's safe to use ThreadLocal when you don't undeploy or
redeploy web apps? Like if you always restart Tomcat when you redeploy,
would it be safe to use ThreadLocal in such a scenario?

On 18 April 2016 at 09:49, Remko Popma  wrote:

> The problem is that values stored in ThreadLocals may live longer than the
> web application since the threads that hold these values are in a pool and
> the threads live on after the web application is undeployed. If the value
> is a JDK class this is not a problem.
>
> If the value is a class that is defined in your webapp (or a library of
> the web app) then holding on the the value means the class cannot be
> garbage-collected. And voila, we have a memory leak...
>
> Garbage-free logging makes liberal use of ThreadLocals with non-JDK
> values. If the web container separates the thread pools between
> applications and restarts the thread pool with the web application, the
> above memory leak does not occur. (I believe this is what WebSphere does.)
>
> Eventually we may be able to address this with LOG4J2-1116
> .
>
>
>
> On Mon, Apr 18, 2016 at 11:04 PM, Matt Sicker  wrote:
>
>> So is this only bad to use when you have multiple wars on a single
>> Tomcat, or is it bad for when you try to undeploy, or is there another
>> issue here?
>>
>> On 18 April 2016 at 09:01, Remko Popma  wrote:
>>
>>> Yes but if this is wrong because there is no threadpool or so, then you
>>> can use System property "log4j2.is.webapp" to tell Log4j that it should
>>> consider your app a non-webapp.
>>>
>>> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker  wrote:
>>>
 Would an embedded Tomcat/Jetty a la Spring Boot really be considered a
 web application?

 On 18 April 2016 at 08:56, Remko Popma  wrote:

> When I finish performance testing I will add graphs to the Performance
> section of the gc-free manual page so that users can make an informed
> decision. (The below one is an example.)
>
> So far, it seems that even with the garbage-free logging enabled,
> synchronous logging throughput (the green line) is usually slightly faster
> than it was with 2.5 (the purple line). So I think making garbage-free the
> default is acceptable. Also, users who care about logging throughput will
> be logging asynchronously, not synchronously: this scales much better with
> the number of application threads.
>
>
>
> ​
>
> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal <
> mikael.stal...@magine.com> wrote:
>
>> Is it wise to enable by default given that "multi-threaded
>> applications that use synchronous logging may see worse performance"?
>>
>> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
>> wrote:
>>
>>> Enabled, except for web apps; for those log4j will be low garbage.
>>>
>>> See the first paragraph of the Configuration section in the manual
>>> page.
>>>
>>> Please let me know if this is not clear. (The fact that you asked
>>> means this probably needs improvement...)
>>>
>>> Sent from my iPhone
>>>
>>> On 2016/04/18, at 20:18, Mikael Ståldal 
>>> wrote:
>>>
>>> Will garbage-free logging be enabled or disabled by default?
>>>
>>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) >> > wrote:
>>>

 [
 https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
 ]

 Remko Popma commented on LOG4J2-1297:
 -

 TODO allocate many temporary objects -> allocate temporary objects

 TODO multi-threaded applications that use synchronous logging may
 see worse performance: the lock that was previously only around the IO
 operation is widened to include the text formatting and conversion to
 bytes. -> _synchronous_ logging performance may be worse for 
 multi-threaded
 applications in this mode due to synchronization on the shared buffer. 
 If
 your application is multi-threaded and logging performance is 
 important,
 consider using Async Loggers.

 TODO Caution: Only dates in the predefined formats are garbage-free
 -> Caution: Only the predefined date formats are garbage-free

 TODO "(Note that Log4j may call toString() on message and parameter
 objects when garbage-free logging is disabled because system property
 log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
 toString() on message and parameter objects when garbage-free logging 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Remko Popma]

The problem is that values stored in ThreadLocals may live longer than the
web application since the threads that hold these values are in a pool and
the threads live on after the web application is undeployed. If the value
is a JDK class this is not a problem.

If the value is a class that is defined in your webapp (or a library of the
web app) then holding on the the value means the class cannot be
garbage-collected. And voila, we have a memory leak...

Garbage-free logging makes liberal use of ThreadLocals with non-JDK values.
If the web container separates the thread pools between applications and
restarts the thread pool with the web application, the above memory leak
does not occur. (I believe this is what WebSphere does.)

Eventually we may be able to address this with LOG4J2-1116
.



On Mon, Apr 18, 2016 at 11:04 PM, Matt Sicker  wrote:

> So is this only bad to use when you have multiple wars on a single Tomcat,
> or is it bad for when you try to undeploy, or is there another issue here?
>
> On 18 April 2016 at 09:01, Remko Popma  wrote:
>
>> Yes but if this is wrong because there is no threadpool or so, then you
>> can use System property "log4j2.is.webapp" to tell Log4j that it should
>> consider your app a non-webapp.
>>
>> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker  wrote:
>>
>>> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a
>>> web application?
>>>
>>> On 18 April 2016 at 08:56, Remko Popma  wrote:
>>>
 When I finish performance testing I will add graphs to the Performance
 section of the gc-free manual page so that users can make an informed
 decision. (The below one is an example.)

 So far, it seems that even with the garbage-free logging enabled,
 synchronous logging throughput (the green line) is usually slightly faster
 than it was with 2.5 (the purple line). So I think making garbage-free the
 default is acceptable. Also, users who care about logging throughput will
 be logging asynchronously, not synchronously: this scales much better with
 the number of application threads.



 ​

 On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal <
 mikael.stal...@magine.com> wrote:

> Is it wise to enable by default given that "multi-threaded
> applications that use synchronous logging may see worse performance"?
>
> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
> wrote:
>
>> Enabled, except for web apps; for those log4j will be low garbage.
>>
>> See the first paragraph of the Configuration section in the manual
>> page.
>>
>> Please let me know if this is not clear. (The fact that you asked
>> means this probably needs improvement...)
>>
>> Sent from my iPhone
>>
>> On 2016/04/18, at 20:18, Mikael Ståldal 
>> wrote:
>>
>> Will garbage-free logging be enabled or disabled by default?
>>
>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
>> wrote:
>>
>>>
>>> [
>>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>>> ]
>>>
>>> Remko Popma commented on LOG4J2-1297:
>>> -
>>>
>>> TODO allocate many temporary objects -> allocate temporary objects
>>>
>>> TODO multi-threaded applications that use synchronous logging may
>>> see worse performance: the lock that was previously only around the IO
>>> operation is widened to include the text formatting and conversion to
>>> bytes. -> _synchronous_ logging performance may be worse for 
>>> multi-threaded
>>> applications in this mode due to synchronization on the shared buffer. 
>>> If
>>> your application is multi-threaded and logging performance is important,
>>> consider using Async Loggers.
>>>
>>> TODO Caution: Only dates in the predefined formats are garbage-free
>>> -> Caution: Only the predefined date formats are garbage-free
>>>
>>> TODO "(Note that Log4j may call toString() on message and parameter
>>> objects when garbage-free logging is disabled because system property
>>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
>>> toString() on message and parameter objects when garbage-free logging is
>>> disabled (when system property log4j2.enable.threadlocals is set to
>>> "false".)
>>>
>>> TODO "made an effort to make logging code garbage-free" -> "made an
>>> effort to make logging garbage-free"
>>>
>>>
>>> > Document "gc-free" configuration and performance
>>> > 
>>> >
>>> > Key: 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Matt Sicker]

So is this only bad to use when you have multiple wars on a single Tomcat,
or is it bad for when you try to undeploy, or is there another issue here?

On 18 April 2016 at 09:01, Remko Popma  wrote:

> Yes but if this is wrong because there is no threadpool or so, then you
> can use System property "log4j2.is.webapp" to tell Log4j that it should
> consider your app a non-webapp.
>
> On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker  wrote:
>
>> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a
>> web application?
>>
>> On 18 April 2016 at 08:56, Remko Popma  wrote:
>>
>>> When I finish performance testing I will add graphs to the Performance
>>> section of the gc-free manual page so that users can make an informed
>>> decision. (The below one is an example.)
>>>
>>> So far, it seems that even with the garbage-free logging enabled,
>>> synchronous logging throughput (the green line) is usually slightly faster
>>> than it was with 2.5 (the purple line). So I think making garbage-free the
>>> default is acceptable. Also, users who care about logging throughput will
>>> be logging asynchronously, not synchronously: this scales much better with
>>> the number of application threads.
>>>
>>>
>>>
>>> ​
>>>
>>> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal <
>>> mikael.stal...@magine.com> wrote:
>>>
 Is it wise to enable by default given that "multi-threaded
 applications that use synchronous logging may see worse performance"?

 On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
 wrote:

> Enabled, except for web apps; for those log4j will be low garbage.
>
> See the first paragraph of the Configuration section in the manual
> page.
>
> Please let me know if this is not clear. (The fact that you asked
> means this probably needs improvement...)
>
> Sent from my iPhone
>
> On 2016/04/18, at 20:18, Mikael Ståldal 
> wrote:
>
> Will garbage-free logging be enabled or disabled by default?
>
> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
> wrote:
>
>>
>> [
>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>> ]
>>
>> Remko Popma commented on LOG4J2-1297:
>> -
>>
>> TODO allocate many temporary objects -> allocate temporary objects
>>
>> TODO multi-threaded applications that use synchronous logging may see
>> worse performance: the lock that was previously only around the IO
>> operation is widened to include the text formatting and conversion to
>> bytes. -> _synchronous_ logging performance may be worse for 
>> multi-threaded
>> applications in this mode due to synchronization on the shared buffer. If
>> your application is multi-threaded and logging performance is important,
>> consider using Async Loggers.
>>
>> TODO Caution: Only dates in the predefined formats are garbage-free
>> -> Caution: Only the predefined date formats are garbage-free
>>
>> TODO "(Note that Log4j may call toString() on message and parameter
>> objects when garbage-free logging is disabled because system property
>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
>> toString() on message and parameter objects when garbage-free logging is
>> disabled (when system property log4j2.enable.threadlocals is set to
>> "false".)
>>
>> TODO "made an effort to make logging code garbage-free" -> "made an
>> effort to make logging garbage-free"
>>
>>
>> > Document "gc-free" configuration and performance
>> > 
>> >
>> > Key: LOG4J2-1297
>> > URL:
>> https://issues.apache.org/jira/browse/LOG4J2-1297
>> > Project: Log4j 2
>> >  Issue Type: New Feature
>> >  Components: Documentation
>> >Affects Versions: 2.5
>> >Reporter: Remko Popma
>> >Assignee: Remko Popma
>> > Fix For: 2.6
>> >
>> > Attachments: log4j-2.5-FlightRecording.png,
>> log4j-2.6-FlightRecording.png
>> >
>> >
>> > Update the site with a description of which configurations are
>> GC-free (i.e., that don't create temporary objects in steady running 
>> state).
>> > Currently that means
>> > * Loggers are all asynchronous (Log4jContextSelector is set to
>> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
>> > * The configuration does not contain a  section.
>> > * The "steady-state" appenders are either RandomAccessFile or
>> RollingRandomAccessFile Appenders (logging to any other appender will 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Remko Popma]

Yes but if this is wrong because there is no threadpool or so, then you can
use System property "log4j2.is.webapp" to tell Log4j that it should
consider your app a non-webapp.

On Mon, Apr 18, 2016 at 10:59 PM, Matt Sicker  wrote:

> Would an embedded Tomcat/Jetty a la Spring Boot really be considered a web
> application?
>
> On 18 April 2016 at 08:56, Remko Popma  wrote:
>
>> When I finish performance testing I will add graphs to the Performance
>> section of the gc-free manual page so that users can make an informed
>> decision. (The below one is an example.)
>>
>> So far, it seems that even with the garbage-free logging enabled,
>> synchronous logging throughput (the green line) is usually slightly faster
>> than it was with 2.5 (the purple line). So I think making garbage-free the
>> default is acceptable. Also, users who care about logging throughput will
>> be logging asynchronously, not synchronously: this scales much better with
>> the number of application threads.
>>
>>
>>
>> ​
>>
>> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal <
>> mikael.stal...@magine.com> wrote:
>>
>>> Is it wise to enable by default given that "multi-threaded applications
>>> that use synchronous logging may see worse performance"?
>>>
>>> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
>>> wrote:
>>>
 Enabled, except for web apps; for those log4j will be low garbage.

 See the first paragraph of the Configuration section in the manual
 page.

 Please let me know if this is not clear. (The fact that you asked means
 this probably needs improvement...)

 Sent from my iPhone

 On 2016/04/18, at 20:18, Mikael Ståldal 
 wrote:

 Will garbage-free logging be enabled or disabled by default?

 On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
 wrote:

>
> [
> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
> ]
>
> Remko Popma commented on LOG4J2-1297:
> -
>
> TODO allocate many temporary objects -> allocate temporary objects
>
> TODO multi-threaded applications that use synchronous logging may see
> worse performance: the lock that was previously only around the IO
> operation is widened to include the text formatting and conversion to
> bytes. -> _synchronous_ logging performance may be worse for 
> multi-threaded
> applications in this mode due to synchronization on the shared buffer. If
> your application is multi-threaded and logging performance is important,
> consider using Async Loggers.
>
> TODO Caution: Only dates in the predefined formats are garbage-free ->
> Caution: Only the predefined date formats are garbage-free
>
> TODO "(Note that Log4j may call toString() on message and parameter
> objects when garbage-free logging is disabled because system property
> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
> toString() on message and parameter objects when garbage-free logging is
> disabled (when system property log4j2.enable.threadlocals is set to
> "false".)
>
> TODO "made an effort to make logging code garbage-free" -> "made an
> effort to make logging garbage-free"
>
>
> > Document "gc-free" configuration and performance
> > 
> >
> > Key: LOG4J2-1297
> > URL:
> https://issues.apache.org/jira/browse/LOG4J2-1297
> > Project: Log4j 2
> >  Issue Type: New Feature
> >  Components: Documentation
> >Affects Versions: 2.5
> >Reporter: Remko Popma
> >Assignee: Remko Popma
> > Fix For: 2.6
> >
> > Attachments: log4j-2.5-FlightRecording.png,
> log4j-2.6-FlightRecording.png
> >
> >
> > Update the site with a description of which configurations are
> GC-free (i.e., that don't create temporary objects in steady running 
> state).
> > Currently that means
> > * Loggers are all asynchronous (Log4jContextSelector is set to
> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
> > * The configuration does not contain a  section.
> > * The "steady-state" appenders are either RandomAccessFile or
> RollingRandomAccessFile Appenders (logging to any other appender will 
> cause
> temporary objects to be created - including ConsoleAppender).
> > * The Layout is a PatternLayout that uses one of the pre-defined
> date formats, does not have any regular expression replacements, and does
> not have lookups (TODO: may need to restrict this further).
> > * The thread name is cached 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Matt Sicker]

Would an embedded Tomcat/Jetty a la Spring Boot really be considered a web
application?

On 18 April 2016 at 08:56, Remko Popma  wrote:

> When I finish performance testing I will add graphs to the Performance
> section of the gc-free manual page so that users can make an informed
> decision. (The below one is an example.)
>
> So far, it seems that even with the garbage-free logging enabled,
> synchronous logging throughput (the green line) is usually slightly faster
> than it was with 2.5 (the purple line). So I think making garbage-free the
> default is acceptable. Also, users who care about logging throughput will
> be logging asynchronously, not synchronously: this scales much better with
> the number of application threads.
>
>
>
> ​
>
> On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal  > wrote:
>
>> Is it wise to enable by default given that "multi-threaded applications
>> that use synchronous logging may see worse performance"?
>>
>> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
>> wrote:
>>
>>> Enabled, except for web apps; for those log4j will be low garbage.
>>>
>>> See the first paragraph of the Configuration section in the manual page.
>>>
>>> Please let me know if this is not clear. (The fact that you asked means
>>> this probably needs improvement...)
>>>
>>> Sent from my iPhone
>>>
>>> On 2016/04/18, at 20:18, Mikael Ståldal 
>>> wrote:
>>>
>>> Will garbage-free logging be enabled or disabled by default?
>>>
>>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
>>> wrote:
>>>

 [
 https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
 ]

 Remko Popma commented on LOG4J2-1297:
 -

 TODO allocate many temporary objects -> allocate temporary objects

 TODO multi-threaded applications that use synchronous logging may see
 worse performance: the lock that was previously only around the IO
 operation is widened to include the text formatting and conversion to
 bytes. -> _synchronous_ logging performance may be worse for multi-threaded
 applications in this mode due to synchronization on the shared buffer. If
 your application is multi-threaded and logging performance is important,
 consider using Async Loggers.

 TODO Caution: Only dates in the predefined formats are garbage-free ->
 Caution: Only the predefined date formats are garbage-free

 TODO "(Note that Log4j may call toString() on message and parameter
 objects when garbage-free logging is disabled because system property
 log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
 toString() on message and parameter objects when garbage-free logging is
 disabled (when system property log4j2.enable.threadlocals is set to
 "false".)

 TODO "made an effort to make logging code garbage-free" -> "made an
 effort to make logging garbage-free"


 > Document "gc-free" configuration and performance
 > 
 >
 > Key: LOG4J2-1297
 > URL:
 https://issues.apache.org/jira/browse/LOG4J2-1297
 > Project: Log4j 2
 >  Issue Type: New Feature
 >  Components: Documentation
 >Affects Versions: 2.5
 >Reporter: Remko Popma
 >Assignee: Remko Popma
 > Fix For: 2.6
 >
 > Attachments: log4j-2.5-FlightRecording.png,
 log4j-2.6-FlightRecording.png
 >
 >
 > Update the site with a description of which configurations are
 GC-free (i.e., that don't create temporary objects in steady running 
 state).
 > Currently that means
 > * Loggers are all asynchronous (Log4jContextSelector is set to
 org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
 > * The configuration does not contain a  section.
 > * The "steady-state" appenders are either RandomAccessFile or
 RollingRandomAccessFile Appenders (logging to any other appender will cause
 temporary objects to be created - including ConsoleAppender).
 > * The Layout is a PatternLayout that uses one of the pre-defined date
 formats, does not have any regular expression replacements, and does not
 have lookups (TODO: may need to restrict this further).
 > * The thread name is cached (this is the [default|
 https://issues.apache.org/jira/browse/LOG4J2-467]). Running with
 -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage.
 > * In user code, when logging a parameterized message, the number of
 parameters is no more than ... (TBD pending discussion in LOG4J2-1278).
 > * In user code, when logging a parameterized message, parameters 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Remko Popma]

When I finish performance testing I will add graphs to the Performance
section of the gc-free manual page so that users can make an informed
decision. (The below one is an example.)

So far, it seems that even with the garbage-free logging enabled,
synchronous logging throughput (the green line) is usually slightly faster
than it was with 2.5 (the purple line). So I think making garbage-free the
default is acceptable. Also, users who care about logging throughput will
be logging asynchronously, not synchronously: this scales much better with
the number of application threads.



​

On Mon, Apr 18, 2016 at 9:57 PM, Mikael Ståldal 
wrote:

> Is it wise to enable by default given that "multi-threaded applications
> that use synchronous logging may see worse performance"?
>
> On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma 
> wrote:
>
>> Enabled, except for web apps; for those log4j will be low garbage.
>>
>> See the first paragraph of the Configuration section in the manual page.
>>
>> Please let me know if this is not clear. (The fact that you asked means
>> this probably needs improvement...)
>>
>> Sent from my iPhone
>>
>> On 2016/04/18, at 20:18, Mikael Ståldal 
>> wrote:
>>
>> Will garbage-free logging be enabled or disabled by default?
>>
>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
>> wrote:
>>
>>>
>>> [
>>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>>> ]
>>>
>>> Remko Popma commented on LOG4J2-1297:
>>> -
>>>
>>> TODO allocate many temporary objects -> allocate temporary objects
>>>
>>> TODO multi-threaded applications that use synchronous logging may see
>>> worse performance: the lock that was previously only around the IO
>>> operation is widened to include the text formatting and conversion to
>>> bytes. -> _synchronous_ logging performance may be worse for multi-threaded
>>> applications in this mode due to synchronization on the shared buffer. If
>>> your application is multi-threaded and logging performance is important,
>>> consider using Async Loggers.
>>>
>>> TODO Caution: Only dates in the predefined formats are garbage-free ->
>>> Caution: Only the predefined date formats are garbage-free
>>>
>>> TODO "(Note that Log4j may call toString() on message and parameter
>>> objects when garbage-free logging is disabled because system property
>>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
>>> toString() on message and parameter objects when garbage-free logging is
>>> disabled (when system property log4j2.enable.threadlocals is set to
>>> "false".)
>>>
>>> TODO "made an effort to make logging code garbage-free" -> "made an
>>> effort to make logging garbage-free"
>>>
>>>
>>> > Document "gc-free" configuration and performance
>>> > 
>>> >
>>> > Key: LOG4J2-1297
>>> > URL: https://issues.apache.org/jira/browse/LOG4J2-1297
>>> > Project: Log4j 2
>>> >  Issue Type: New Feature
>>> >  Components: Documentation
>>> >Affects Versions: 2.5
>>> >Reporter: Remko Popma
>>> >Assignee: Remko Popma
>>> > Fix For: 2.6
>>> >
>>> > Attachments: log4j-2.5-FlightRecording.png,
>>> log4j-2.6-FlightRecording.png
>>> >
>>> >
>>> > Update the site with a description of which configurations are GC-free
>>> (i.e., that don't create temporary objects in steady running state).
>>> > Currently that means
>>> > * Loggers are all asynchronous (Log4jContextSelector is set to
>>> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
>>> > * The configuration does not contain a  section.
>>> > * The "steady-state" appenders are either RandomAccessFile or
>>> RollingRandomAccessFile Appenders (logging to any other appender will cause
>>> temporary objects to be created - including ConsoleAppender).
>>> > * The Layout is a PatternLayout that uses one of the pre-defined date
>>> formats, does not have any regular expression replacements, and does not
>>> have lookups (TODO: may need to restrict this further).
>>> > * The thread name is cached (this is the [default|
>>> https://issues.apache.org/jira/browse/LOG4J2-467]). Running with
>>> -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage.
>>> > * In user code, when logging a parameterized message, the number of
>>> parameters is no more than ... (TBD pending discussion in LOG4J2-1278).
>>> > * In user code, when logging a parameterized message, parameters of
>>> primitive type are boxed in a reused StringBuilder (Log4j provides a
>>> utility to make this relatively painless).
>>> > Even with the above restrictions, Log4j may occasionally create
>>> garbage:
>>> > * Initially StringBuilders are presized to 128 characters. They may
>>> grow for 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Mikael Ståldal]

Is it wise to enable by default given that "multi-threaded applications
that use synchronous logging may see worse performance"?

On Mon, Apr 18, 2016 at 2:22 PM, Remko Popma  wrote:

> Enabled, except for web apps; for those log4j will be low garbage.
>
> See the first paragraph of the Configuration section in the manual page.
>
> Please let me know if this is not clear. (The fact that you asked means
> this probably needs improvement...)
>
> Sent from my iPhone
>
> On 2016/04/18, at 20:18, Mikael Ståldal  wrote:
>
> Will garbage-free logging be enabled or disabled by default?
>
> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA) 
> wrote:
>
>>
>> [
>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>> ]
>>
>> Remko Popma commented on LOG4J2-1297:
>> -
>>
>> TODO allocate many temporary objects -> allocate temporary objects
>>
>> TODO multi-threaded applications that use synchronous logging may see
>> worse performance: the lock that was previously only around the IO
>> operation is widened to include the text formatting and conversion to
>> bytes. -> _synchronous_ logging performance may be worse for multi-threaded
>> applications in this mode due to synchronization on the shared buffer. If
>> your application is multi-threaded and logging performance is important,
>> consider using Async Loggers.
>>
>> TODO Caution: Only dates in the predefined formats are garbage-free ->
>> Caution: Only the predefined date formats are garbage-free
>>
>> TODO "(Note that Log4j may call toString() on message and parameter
>> objects when garbage-free logging is disabled because system property
>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call
>> toString() on message and parameter objects when garbage-free logging is
>> disabled (when system property log4j2.enable.threadlocals is set to
>> "false".)
>>
>> TODO "made an effort to make logging code garbage-free" -> "made an
>> effort to make logging garbage-free"
>>
>>
>> > Document "gc-free" configuration and performance
>> > 
>> >
>> > Key: LOG4J2-1297
>> > URL: https://issues.apache.org/jira/browse/LOG4J2-1297
>> > Project: Log4j 2
>> >  Issue Type: New Feature
>> >  Components: Documentation
>> >Affects Versions: 2.5
>> >Reporter: Remko Popma
>> >Assignee: Remko Popma
>> > Fix For: 2.6
>> >
>> > Attachments: log4j-2.5-FlightRecording.png,
>> log4j-2.6-FlightRecording.png
>> >
>> >
>> > Update the site with a description of which configurations are GC-free
>> (i.e., that don't create temporary objects in steady running state).
>> > Currently that means
>> > * Loggers are all asynchronous (Log4jContextSelector is set to
>> org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
>> > * The configuration does not contain a  section.
>> > * The "steady-state" appenders are either RandomAccessFile or
>> RollingRandomAccessFile Appenders (logging to any other appender will cause
>> temporary objects to be created - including ConsoleAppender).
>> > * The Layout is a PatternLayout that uses one of the pre-defined date
>> formats, does not have any regular expression replacements, and does not
>> have lookups (TODO: may need to restrict this further).
>> > * The thread name is cached (this is the [default|
>> https://issues.apache.org/jira/browse/LOG4J2-467]). Running with
>> -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage.
>> > * In user code, when logging a parameterized message, the number of
>> parameters is no more than ... (TBD pending discussion in LOG4J2-1278).
>> > * In user code, when logging a parameterized message, parameters of
>> primitive type are boxed in a reused StringBuilder (Log4j provides a
>> utility to make this relatively painless).
>> > Even with the above restrictions, Log4j may occasionally create garbage:
>> > * Initially StringBuilders are presized to 128 characters. They may
>> grow for larger messages (contributing to garbage in Old Gen). If  the
>> StringBuilder grows beyond 512 characters it is trimmed back to 512
>> characters to prevent memory leaks from excessively long messages. (TODO:
>> the resizing algorithm is {{size = value.length * 2 + 2}}, so a better
>> cutoff value is 518.)
>> > * Messages containing {{"$\{"}} will be converted to a String and
>> StrSubstitutor will be used to replace occurences of variables with their
>> matching values. Multiple temporary objects are created during this process.
>> > Furthermore, we need to explain that some of this functionality depends
>> on ThreadLocals and so is disabled by default in web applications to
>> prevent memory leaks. The page should also explain how to manually switch
>> off the use 

Re: [jira] [Commented] (LOG4J2-1297) Document "gc-free" configuration and performance

[Remko Popma]

Enabled, except for web apps; for those log4j will be low garbage. 

See the first paragraph of the Configuration section in the manual page. 

Please let me know if this is not clear. (The fact that you asked means this 
probably needs improvement...)

Sent from my iPhone

> On 2016/04/18, at 20:18, Mikael Ståldal  wrote:
> 
> Will garbage-free logging be enabled or disabled by default?
> 
>> On Mon, Apr 18, 2016 at 2:58 AM, Remko Popma (JIRA)  wrote:
>> 
>> [ 
>> https://issues.apache.org/jira/browse/LOG4J2-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245018#comment-15245018
>>  ]
>> 
>> Remko Popma commented on LOG4J2-1297:
>> -
>> 
>> TODO allocate many temporary objects -> allocate temporary objects
>> 
>> TODO multi-threaded applications that use synchronous logging may see worse 
>> performance: the lock that was previously only around the IO operation is 
>> widened to include the text formatting and conversion to bytes. -> 
>> _synchronous_ logging performance may be worse for multi-threaded 
>> applications in this mode due to synchronization on the shared buffer. If 
>> your application is multi-threaded and logging performance is important, 
>> consider using Async Loggers.
>> 
>> TODO Caution: Only dates in the predefined formats are garbage-free -> 
>> Caution: Only the predefined date formats are garbage-free
>> 
>> TODO "(Note that Log4j may call toString() on message and parameter objects 
>> when garbage-free logging is disabled because system property 
>> log4j2.enable.threadlocals is set to "false".)" -> Log4j may call toString() 
>> on message and parameter objects when garbage-free logging is disabled (when 
>> system property log4j2.enable.threadlocals is set to "false".)
>> 
>> TODO "made an effort to make logging code garbage-free" -> "made an effort 
>> to make logging garbage-free"
>> 
>> 
>> > Document "gc-free" configuration and performance
>> > 
>> >
>> > Key: LOG4J2-1297
>> > URL: https://issues.apache.org/jira/browse/LOG4J2-1297
>> > Project: Log4j 2
>> >  Issue Type: New Feature
>> >  Components: Documentation
>> >Affects Versions: 2.5
>> >Reporter: Remko Popma
>> >Assignee: Remko Popma
>> > Fix For: 2.6
>> >
>> > Attachments: log4j-2.5-FlightRecording.png, 
>> > log4j-2.6-FlightRecording.png
>> >
>> >
>> > Update the site with a description of which configurations are GC-free 
>> > (i.e., that don't create temporary objects in steady running state).
>> > Currently that means
>> > * Loggers are all asynchronous (Log4jContextSelector is set to 
>> > org.apache.logging.log4j.core.async.AsyncLoggerContextSelector).
>> > * The configuration does not contain a  section.
>> > * The "steady-state" appenders are either RandomAccessFile or 
>> > RollingRandomAccessFile Appenders (logging to any other appender will 
>> > cause temporary objects to be created - including ConsoleAppender).
>> > * The Layout is a PatternLayout that uses one of the pre-defined date 
>> > formats, does not have any regular expression replacements, and does not 
>> > have lookups (TODO: may need to restrict this further).
>> > * The thread name is cached (this is the 
>> > [default|https://issues.apache.org/jira/browse/LOG4J2-467]). Running with 
>> > -DAsyncLogger.ThreadNameStrategy=UNCACHED will create garbage.
>> > * In user code, when logging a parameterized message, the number of 
>> > parameters is no more than ... (TBD pending discussion in LOG4J2-1278).
>> > * In user code, when logging a parameterized message, parameters of 
>> > primitive type are boxed in a reused StringBuilder (Log4j provides a 
>> > utility to make this relatively painless).
>> > Even with the above restrictions, Log4j may occasionally create garbage:
>> > * Initially StringBuilders are presized to 128 characters. They may grow 
>> > for larger messages (contributing to garbage in Old Gen). If  the 
>> > StringBuilder grows beyond 512 characters it is trimmed back to 512 
>> > characters to prevent memory leaks from excessively long messages. (TODO: 
>> > the resizing algorithm is {{size = value.length * 2 + 2}}, so a better 
>> > cutoff value is 518.)
>> > * Messages containing {{"$\{"}} will be converted to a String and 
>> > StrSubstitutor will be used to replace occurences of variables with their 
>> > matching values. Multiple temporary objects are created during this 
>> > process.
>> > Furthermore, we need to explain that some of this functionality depends on 
>> > ThreadLocals and so is disabled by default in web applications to prevent 
>> > memory leaks. The page should also explain how to manually switch off the 
>> > use of ThreadLocals.
>> > Finally, the page should show a performance test comparison similar to the 
>> > [performance 
>> >