Re: why is AsyncLoggerContextSelector faster?

[Ralph Goers]

Take a look at http://logging.apache.org/log4j/2.x/manual/architecture.html.   
The configuration creates the LoggerConfig objects, not the Loggers. Loggers 
are created by applications and associated with LoggerConfigs. 

XMLConfiguration and JSONConfiguration are responsible for parsing their 
respective configuration files and then turning them into a tree of Node 
objects. AbstractConfiguration then converts the tree into the actual objects 
using the Plugin mechanism.

Ralph


On Jul 21, 2014, at 8:41 AM, Mariano Gonzalez  
wrote:

> Thanks Remko and Matt for your comments.
> 
> I will take your suggestions and let you know how it goes.
> 
> Just out of curiosity (I don't like to use libraries blindly and way too
> curious), where are the loggers instantiated/configured?? I followed the
> code of XmlConfiguration but it seems to only parse configuration items, I
> couldn't find where it parses and configures the loggers. I admit I didn't
> look that hard, but if you could provide a pointer to better understand how
> this all works that'd be great.
> 
> Thanks!
> 
> 
> On Mon, Jul 21, 2014 at 12:12 PM, Remko Popma  wrote:
> 
>> Mariano, to clarify, I would recommend that you keep your custom context
>> selector simple and only use it for the log separation that you mentioned.
>> I would not recommend that you also somehow try to merge the functionality
>> of AsyncLoggerContextSelector in your context selector. From a performance
>> perspective that should not be necessary. If you use  and
>>  in your configuration then I doubt that logging will be a
>> performance bottleneck.
>> 
>> 
>> On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma 
>> wrote:
>> 
>>> Matt, not sure what you mean. Mariano needs to develop his own
>>> ContextSelector to achieve log separation.
>>> So he cannot use the AsyncLoggerContextSelector.
>>> 
>>> Mariano, I would not be overly concerned with the performance results
>>> mentioned on the async logger manual page. Bear in mind that in that
>>> performance test, _all_ that the program does is logging. In the real
>> world
>>> (your application) that will obviously not be true: your application will
>>> do many other things.
>>> 
>>> Also, mixed async loggers performed much, much better than
>> AsyncAppenders.
>>> 
>>> So, unless you have actual measurement numbers for _your_ application,
>> not
>>> the theoretical numbers from the async logger manual page, I would keep
>> it
>>> simple at first, just get your custom selector to work well, and then
>> start
>>> measuring performance.
>>> 
>>> If you are getting strange results, like AsyncAppenders performing better
>>> than AsyncLoggers, then let us know and we'll try to help figure out what
>>> the problem is.
>>> 
>>> 
>>> 
>>> On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker  wrote:
>>> 
 Well, the way to go that's similar to the async version is to use the
 BasicContextSelector which contains a singleton LoggerContext.
>> Otherwise,
 you'll have to keep your own registry that can be looked up dynamically
 such as through reflection of the caller stack and other fun design
 problems. :)
 
 
 On 21 July 2014 09:35, Mariano Gonzalez 
 wrote:
 
> Hello Remko,
> 
> I'm still a couple of days away from starting my own performance
 testing.
> I'm taking about the difference in the async loggers manual page, more
> specifically, the charts that compare sync loggers, to mixed async
 loggers
> against purely async loggers. Since I need to build my own selector,
>> I'm
> trying to be clear on how this works internally in order to implement
 the
> less latency possible selector and try to minimize the performance
 testing
> effort.
> 
> Thanks for the clarifications!
> 
> 
> 
> On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma 
> wrote:
> 
>> Hi,
>> No that is incorrect.
>> If you do not specify AsyncLoggerContextSelector but instead
>> configure
> with
>>  and  loggers, you _do_ need the disruptor
 jar on
>> the classpath and this does _not_ use AsyncAppender. AsyncAppender
>> is
>> completely separate from Async Loggers. Async Loggers (mixed or all
> async)
>> use the disruptor and both need the disruptor jar.
>> 
>> You keep mentioning a performance difference. I was assuming you
>> were
>> talking about the performance test results mentioned on the Async
 Logger
>> manual page, but perhaps I was wrong? Are you experiencing a
 performance
>> difference between the two flavors of Async Loggers in your
 application?
>> 
>> 
>> 
>> 
>> On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
>> mariano.l.gonza...@gmail.com> wrote:
>> 
>>> Hello Remko,
>>> 
>>> I think I found the difference. AsyncLoggerContextSelector always
> returns
>>> the same instance of AsyncLoggerContext, which in turns always
 returns

Re: why is AsyncLoggerContextSelector faster?

[Mariano Gonzalez]

Thanks Remko and Matt for your comments.

I will take your suggestions and let you know how it goes.

Just out of curiosity (I don't like to use libraries blindly and way too
curious), where are the loggers instantiated/configured?? I followed the
code of XmlConfiguration but it seems to only parse configuration items, I
couldn't find where it parses and configures the loggers. I admit I didn't
look that hard, but if you could provide a pointer to better understand how
this all works that'd be great.

Thanks!


On Mon, Jul 21, 2014 at 12:12 PM, Remko Popma  wrote:

> Mariano, to clarify, I would recommend that you keep your custom context
> selector simple and only use it for the log separation that you mentioned.
> I would not recommend that you also somehow try to merge the functionality
> of AsyncLoggerContextSelector in your context selector. From a performance
> perspective that should not be necessary. If you use  and
>  in your configuration then I doubt that logging will be a
> performance bottleneck.
>
>
> On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma 
> wrote:
>
> > Matt, not sure what you mean. Mariano needs to develop his own
> > ContextSelector to achieve log separation.
> > So he cannot use the AsyncLoggerContextSelector.
> >
> > Mariano, I would not be overly concerned with the performance results
> > mentioned on the async logger manual page. Bear in mind that in that
> > performance test, _all_ that the program does is logging. In the real
> world
> > (your application) that will obviously not be true: your application will
> > do many other things.
> >
> > Also, mixed async loggers performed much, much better than
> AsyncAppenders.
> >
> > So, unless you have actual measurement numbers for _your_ application,
> not
> > the theoretical numbers from the async logger manual page, I would keep
> it
> > simple at first, just get your custom selector to work well, and then
> start
> > measuring performance.
> >
> > If you are getting strange results, like AsyncAppenders performing better
> > than AsyncLoggers, then let us know and we'll try to help figure out what
> > the problem is.
> >
> >
> >
> > On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker  wrote:
> >
> >> Well, the way to go that's similar to the async version is to use the
> >> BasicContextSelector which contains a singleton LoggerContext.
> Otherwise,
> >> you'll have to keep your own registry that can be looked up dynamically
> >> such as through reflection of the caller stack and other fun design
> >> problems. :)
> >>
> >>
> >> On 21 July 2014 09:35, Mariano Gonzalez 
> >> wrote:
> >>
> >> > Hello Remko,
> >> >
> >> > I'm still a couple of days away from starting my own performance
> >> testing.
> >> > I'm taking about the difference in the async loggers manual page, more
> >> > specifically, the charts that compare sync loggers, to mixed async
> >> loggers
> >> > against purely async loggers. Since I need to build my own selector,
> I'm
> >> > trying to be clear on how this works internally in order to implement
> >> the
> >> > less latency possible selector and try to minimize the performance
> >> testing
> >> > effort.
> >> >
> >> > Thanks for the clarifications!
> >> >
> >> >
> >> >
> >> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma 
> >> > wrote:
> >> >
> >> > > Hi,
> >> > > No that is incorrect.
> >> > > If you do not specify AsyncLoggerContextSelector but instead
> configure
> >> > with
> >> > >  and  loggers, you _do_ need the disruptor
> >> jar on
> >> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender
> is
> >> > > completely separate from Async Loggers. Async Loggers (mixed or all
> >> > async)
> >> > > use the disruptor and both need the disruptor jar.
> >> > >
> >> > > You keep mentioning a performance difference. I was assuming you
> were
> >> > > talking about the performance test results mentioned on the Async
> >> Logger
> >> > > manual page, but perhaps I was wrong? Are you experiencing a
> >> performance
> >> > > difference between the two flavors of Async Loggers in your
> >> application?
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
> >> > > mariano.l.gonza...@gmail.com> wrote:
> >> > >
> >> > > > Hello Remko,
> >> > > >
> >> > > > I think I found the difference. AsyncLoggerContextSelector always
> >> > returns
> >> > > > the same instance of AsyncLoggerContext, which in turns always
> >> returns
> >> > > > instances of AsyncLogger, which uses disruptor to handle
> >> concurrency.
> >> > > >
> >> > > > However, with any other selector, a standard Logger instance is
> >> > returned
> >> > > > and parallelism is achieved through an AsyncAppender.
> AsyncAppender
> >> > use a
> >> > > > standard blocking queue instead of using disruptor which explains
> >> the
> >> > > > performance difference (there's also the fact that
> >> > > > AsyncLoggerContextSelector always returns the same context
> instance
> >> and
> >> > > > does not spend cycles i

Re: why is AsyncLoggerContextSelector faster?

[Remko Popma]

Mariano, to clarify, I would recommend that you keep your custom context
selector simple and only use it for the log separation that you mentioned.
I would not recommend that you also somehow try to merge the functionality
of AsyncLoggerContextSelector in your context selector. From a performance
perspective that should not be necessary. If you use  and
 in your configuration then I doubt that logging will be a
performance bottleneck.


On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma  wrote:

> Matt, not sure what you mean. Mariano needs to develop his own
> ContextSelector to achieve log separation.
> So he cannot use the AsyncLoggerContextSelector.
>
> Mariano, I would not be overly concerned with the performance results
> mentioned on the async logger manual page. Bear in mind that in that
> performance test, _all_ that the program does is logging. In the real world
> (your application) that will obviously not be true: your application will
> do many other things.
>
> Also, mixed async loggers performed much, much better than AsyncAppenders.
>
> So, unless you have actual measurement numbers for _your_ application, not
> the theoretical numbers from the async logger manual page, I would keep it
> simple at first, just get your custom selector to work well, and then start
> measuring performance.
>
> If you are getting strange results, like AsyncAppenders performing better
> than AsyncLoggers, then let us know and we'll try to help figure out what
> the problem is.
>
>
>
> On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker  wrote:
>
>> Well, the way to go that's similar to the async version is to use the
>> BasicContextSelector which contains a singleton LoggerContext. Otherwise,
>> you'll have to keep your own registry that can be looked up dynamically
>> such as through reflection of the caller stack and other fun design
>> problems. :)
>>
>>
>> On 21 July 2014 09:35, Mariano Gonzalez 
>> wrote:
>>
>> > Hello Remko,
>> >
>> > I'm still a couple of days away from starting my own performance
>> testing.
>> > I'm taking about the difference in the async loggers manual page, more
>> > specifically, the charts that compare sync loggers, to mixed async
>> loggers
>> > against purely async loggers. Since I need to build my own selector, I'm
>> > trying to be clear on how this works internally in order to implement
>> the
>> > less latency possible selector and try to minimize the performance
>> testing
>> > effort.
>> >
>> > Thanks for the clarifications!
>> >
>> >
>> >
>> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma 
>> > wrote:
>> >
>> > > Hi,
>> > > No that is incorrect.
>> > > If you do not specify AsyncLoggerContextSelector but instead configure
>> > with
>> > >  and  loggers, you _do_ need the disruptor
>> jar on
>> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender is
>> > > completely separate from Async Loggers. Async Loggers (mixed or all
>> > async)
>> > > use the disruptor and both need the disruptor jar.
>> > >
>> > > You keep mentioning a performance difference. I was assuming you were
>> > > talking about the performance test results mentioned on the Async
>> Logger
>> > > manual page, but perhaps I was wrong? Are you experiencing a
>> performance
>> > > difference between the two flavors of Async Loggers in your
>> application?
>> > >
>> > >
>> > >
>> > >
>> > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
>> > > mariano.l.gonza...@gmail.com> wrote:
>> > >
>> > > > Hello Remko,
>> > > >
>> > > > I think I found the difference. AsyncLoggerContextSelector always
>> > returns
>> > > > the same instance of AsyncLoggerContext, which in turns always
>> returns
>> > > > instances of AsyncLogger, which uses disruptor to handle
>> concurrency.
>> > > >
>> > > > However, with any other selector, a standard Logger instance is
>> > returned
>> > > > and parallelism is achieved through an AsyncAppender. AsyncAppender
>> > use a
>> > > > standard blocking queue instead of using disruptor which explains
>> the
>> > > > performance difference (there's also the fact that
>> > > > AsyncLoggerContextSelector always returns the same context instance
>> and
>> > > > does not spend cycles in the lookup, but I think that is not a
>> > > significant
>> > > > cost once everything was warmed out).
>> > > >
>> > > > http://logging.apache.org/log4j/2.x/manual/async.html says that
>> when
>> > > using
>> > > > mixed type loggers disruptor is needed on the classpath. That seems
>> to
>> > be
>> > > > an error. For what I see disruptor is only used when setting all
>> > loggers
>> > > > asynchronous.
>> > > >
>> > > > Does this make sense? Anyway around this? Do you have a disruptor
>> > > appender
>> > > > somewhere?
>> > > >
>> > > > Thanks!
>> > > >
>> > > >
>> > > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma <
>> remko.po...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > To be honest, I haven't investigated in detail the reason for the
>> > > > > difference in throughput in the performance tes

Re: why is AsyncLoggerContextSelector faster?

[Remko Popma]

Matt, not sure what you mean. Mariano needs to develop his own
ContextSelector to achieve log separation.
So he cannot use the AsyncLoggerContextSelector.

Mariano, I would not be overly concerned with the performance results
mentioned on the async logger manual page. Bear in mind that in that
performance test, _all_ that the program does is logging. In the real world
(your application) that will obviously not be true: your application will
do many other things.

Also, mixed async loggers performed much, much better than AsyncAppenders.

So, unless you have actual measurement numbers for _your_ application, not
the theoretical numbers from the async logger manual page, I would keep it
simple at first, just get your custom selector to work well, and then start
measuring performance.

If you are getting strange results, like AsyncAppenders performing better
than AsyncLoggers, then let us know and we'll try to help figure out what
the problem is.



On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker  wrote:

> Well, the way to go that's similar to the async version is to use the
> BasicContextSelector which contains a singleton LoggerContext. Otherwise,
> you'll have to keep your own registry that can be looked up dynamically
> such as through reflection of the caller stack and other fun design
> problems. :)
>
>
> On 21 July 2014 09:35, Mariano Gonzalez 
> wrote:
>
> > Hello Remko,
> >
> > I'm still a couple of days away from starting my own performance testing.
> > I'm taking about the difference in the async loggers manual page, more
> > specifically, the charts that compare sync loggers, to mixed async
> loggers
> > against purely async loggers. Since I need to build my own selector, I'm
> > trying to be clear on how this works internally in order to implement the
> > less latency possible selector and try to minimize the performance
> testing
> > effort.
> >
> > Thanks for the clarifications!
> >
> >
> >
> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma 
> > wrote:
> >
> > > Hi,
> > > No that is incorrect.
> > > If you do not specify AsyncLoggerContextSelector but instead configure
> > with
> > >  and  loggers, you _do_ need the disruptor jar
> on
> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender is
> > > completely separate from Async Loggers. Async Loggers (mixed or all
> > async)
> > > use the disruptor and both need the disruptor jar.
> > >
> > > You keep mentioning a performance difference. I was assuming you were
> > > talking about the performance test results mentioned on the Async
> Logger
> > > manual page, but perhaps I was wrong? Are you experiencing a
> performance
> > > difference between the two flavors of Async Loggers in your
> application?
> > >
> > >
> > >
> > >
> > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
> > > mariano.l.gonza...@gmail.com> wrote:
> > >
> > > > Hello Remko,
> > > >
> > > > I think I found the difference. AsyncLoggerContextSelector always
> > returns
> > > > the same instance of AsyncLoggerContext, which in turns always
> returns
> > > > instances of AsyncLogger, which uses disruptor to handle concurrency.
> > > >
> > > > However, with any other selector, a standard Logger instance is
> > returned
> > > > and parallelism is achieved through an AsyncAppender. AsyncAppender
> > use a
> > > > standard blocking queue instead of using disruptor which explains the
> > > > performance difference (there's also the fact that
> > > > AsyncLoggerContextSelector always returns the same context instance
> and
> > > > does not spend cycles in the lookup, but I think that is not a
> > > significant
> > > > cost once everything was warmed out).
> > > >
> > > > http://logging.apache.org/log4j/2.x/manual/async.html says that when
> > > using
> > > > mixed type loggers disruptor is needed on the classpath. That seems
> to
> > be
> > > > an error. For what I see disruptor is only used when setting all
> > loggers
> > > > asynchronous.
> > > >
> > > > Does this make sense? Anyway around this? Do you have a disruptor
> > > appender
> > > > somewhere?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma  >
> > > > wrote:
> > > >
> > > > > To be honest, I haven't investigated in detail the reason for the
> > > > > difference in throughput in the performance test.
> > > > >
> > > > > Are you measuring the performance of your application container,
> and
> > > can
> > > > > you see an improvement when using Async Loggers?
> > > > > Do you see a large difference in performance _in your application_
> > > > between
> > > > > making all loggers Asynchronous and using mixed synchronous and
> > > > > Asynchronous loggers?
> > > > >
> > > > >
> > > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> > > > > mariano.l.gonza...@gmail.com> wrote:
> > > > >
> > > > > > Hello Remko,
> > > > > >
> > > > > > Thanks for the insight. I guess my case falls into the wrong end
> of
> > > the
> > > > > > pareto law. My project is a low

Re: why is AsyncLoggerContextSelector faster?

[Matt Sicker]

Well, the way to go that's similar to the async version is to use the
BasicContextSelector which contains a singleton LoggerContext. Otherwise,
you'll have to keep your own registry that can be looked up dynamically
such as through reflection of the caller stack and other fun design
problems. :)


On 21 July 2014 09:35, Mariano Gonzalez 
wrote:

> Hello Remko,
>
> I'm still a couple of days away from starting my own performance testing.
> I'm taking about the difference in the async loggers manual page, more
> specifically, the charts that compare sync loggers, to mixed async loggers
> against purely async loggers. Since I need to build my own selector, I'm
> trying to be clear on how this works internally in order to implement the
> less latency possible selector and try to minimize the performance testing
> effort.
>
> Thanks for the clarifications!
>
>
>
> On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma 
> wrote:
>
> > Hi,
> > No that is incorrect.
> > If you do not specify AsyncLoggerContextSelector but instead configure
> with
> >  and  loggers, you _do_ need the disruptor jar on
> > the classpath and this does _not_ use AsyncAppender. AsyncAppender is
> > completely separate from Async Loggers. Async Loggers (mixed or all
> async)
> > use the disruptor and both need the disruptor jar.
> >
> > You keep mentioning a performance difference. I was assuming you were
> > talking about the performance test results mentioned on the Async Logger
> > manual page, but perhaps I was wrong? Are you experiencing a performance
> > difference between the two flavors of Async Loggers in your application?
> >
> >
> >
> >
> > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
> > mariano.l.gonza...@gmail.com> wrote:
> >
> > > Hello Remko,
> > >
> > > I think I found the difference. AsyncLoggerContextSelector always
> returns
> > > the same instance of AsyncLoggerContext, which in turns always returns
> > > instances of AsyncLogger, which uses disruptor to handle concurrency.
> > >
> > > However, with any other selector, a standard Logger instance is
> returned
> > > and parallelism is achieved through an AsyncAppender. AsyncAppender
> use a
> > > standard blocking queue instead of using disruptor which explains the
> > > performance difference (there's also the fact that
> > > AsyncLoggerContextSelector always returns the same context instance and
> > > does not spend cycles in the lookup, but I think that is not a
> > significant
> > > cost once everything was warmed out).
> > >
> > > http://logging.apache.org/log4j/2.x/manual/async.html says that when
> > using
> > > mixed type loggers disruptor is needed on the classpath. That seems to
> be
> > > an error. For what I see disruptor is only used when setting all
> loggers
> > > asynchronous.
> > >
> > > Does this make sense? Anyway around this? Do you have a disruptor
> > appender
> > > somewhere?
> > >
> > > Thanks!
> > >
> > >
> > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma 
> > > wrote:
> > >
> > > > To be honest, I haven't investigated in detail the reason for the
> > > > difference in throughput in the performance test.
> > > >
> > > > Are you measuring the performance of your application container, and
> > can
> > > > you see an improvement when using Async Loggers?
> > > > Do you see a large difference in performance _in your application_
> > > between
> > > > making all loggers Asynchronous and using mixed synchronous and
> > > > Asynchronous loggers?
> > > >
> > > >
> > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> > > > mariano.l.gonza...@gmail.com> wrote:
> > > >
> > > > > Hello Remko,
> > > > >
> > > > > Thanks for the insight. I guess my case falls into the wrong end of
> > the
> > > > > pareto law. My project is a low latency application container, so I
> > > need
> > > > to
> > > > > have:
> > > > >
> > > > >
> > > > >- low latency
> > > > >- log separation (I actually had to implement my own context
> > > selector
> > > > >because my logic is more complicated than the standard
> > > > >ClassLoaderContextSelector case)
> > > > >- I want async loggers by default, but deployed apps need to be
> > able
> > > > to
> > > > >specify sync loggers
> > > > >
> > > > >
> > > > > Right now I'm kinda meeting those requirements using config file,
> > > > AsyncRoot
> > > > > and my custom selector, but it'd be really great to achieve a
> > > performance
> > > > > level like the one that AsyncContextSelector promises.
> > > > >
> > > > > Is there a way of doing that? For what I see on the code, the
> > > > > AsyncLoggerContextSelector's secret sauce is just to always return
> an
> > > > > AsyncLogger on the newInstance() method. Why is that so much faster
> > > than
> > > > > what ClassLoaderLoggerContextSelector does?
> > > > >
> > > > > Thanks!
> > > > >
> > > > >
> > > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma <
> remko.po...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > The Async Loggers created with the context s

Re: why is AsyncLoggerContextSelector faster?

[Mariano Gonzalez]

Hello Remko,

I'm still a couple of days away from starting my own performance testing.
I'm taking about the difference in the async loggers manual page, more
specifically, the charts that compare sync loggers, to mixed async loggers
against purely async loggers. Since I need to build my own selector, I'm
trying to be clear on how this works internally in order to implement the
less latency possible selector and try to minimize the performance testing
effort.

Thanks for the clarifications!



On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma  wrote:

> Hi,
> No that is incorrect.
> If you do not specify AsyncLoggerContextSelector but instead configure with
>  and  loggers, you _do_ need the disruptor jar on
> the classpath and this does _not_ use AsyncAppender. AsyncAppender is
> completely separate from Async Loggers. Async Loggers (mixed or all async)
> use the disruptor and both need the disruptor jar.
>
> You keep mentioning a performance difference. I was assuming you were
> talking about the performance test results mentioned on the Async Logger
> manual page, but perhaps I was wrong? Are you experiencing a performance
> difference between the two flavors of Async Loggers in your application?
>
>
>
>
> On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
> mariano.l.gonza...@gmail.com> wrote:
>
> > Hello Remko,
> >
> > I think I found the difference. AsyncLoggerContextSelector always returns
> > the same instance of AsyncLoggerContext, which in turns always returns
> > instances of AsyncLogger, which uses disruptor to handle concurrency.
> >
> > However, with any other selector, a standard Logger instance is returned
> > and parallelism is achieved through an AsyncAppender. AsyncAppender use a
> > standard blocking queue instead of using disruptor which explains the
> > performance difference (there's also the fact that
> > AsyncLoggerContextSelector always returns the same context instance and
> > does not spend cycles in the lookup, but I think that is not a
> significant
> > cost once everything was warmed out).
> >
> > http://logging.apache.org/log4j/2.x/manual/async.html says that when
> using
> > mixed type loggers disruptor is needed on the classpath. That seems to be
> > an error. For what I see disruptor is only used when setting all loggers
> > asynchronous.
> >
> > Does this make sense? Anyway around this? Do you have a disruptor
> appender
> > somewhere?
> >
> > Thanks!
> >
> >
> > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma 
> > wrote:
> >
> > > To be honest, I haven't investigated in detail the reason for the
> > > difference in throughput in the performance test.
> > >
> > > Are you measuring the performance of your application container, and
> can
> > > you see an improvement when using Async Loggers?
> > > Do you see a large difference in performance _in your application_
> > between
> > > making all loggers Asynchronous and using mixed synchronous and
> > > Asynchronous loggers?
> > >
> > >
> > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> > > mariano.l.gonza...@gmail.com> wrote:
> > >
> > > > Hello Remko,
> > > >
> > > > Thanks for the insight. I guess my case falls into the wrong end of
> the
> > > > pareto law. My project is a low latency application container, so I
> > need
> > > to
> > > > have:
> > > >
> > > >
> > > >- low latency
> > > >- log separation (I actually had to implement my own context
> > selector
> > > >because my logic is more complicated than the standard
> > > >ClassLoaderContextSelector case)
> > > >- I want async loggers by default, but deployed apps need to be
> able
> > > to
> > > >specify sync loggers
> > > >
> > > >
> > > > Right now I'm kinda meeting those requirements using config file,
> > > AsyncRoot
> > > > and my custom selector, but it'd be really great to achieve a
> > performance
> > > > level like the one that AsyncContextSelector promises.
> > > >
> > > > Is there a way of doing that? For what I see on the code, the
> > > > AsyncLoggerContextSelector's secret sauce is just to always return an
> > > > AsyncLogger on the newInstance() method. Why is that so much faster
> > than
> > > > what ClassLoaderLoggerContextSelector does?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma 
> > > > wrote:
> > > >
> > > > > The Async Loggers created with the context selector have a slightly
> > > > > different mechanism. One of the differences is that LogEvent
> objects
> > > are
> > > > > re-used.
> > > > >
> > > > > However, unless your application is in the low-latency space, I
> would
> > > not
> > > > > worry too much about the performance difference. Both flavors of
> > Async
> > > > > Loggers are much faster than the alternative (Async Appenders).
> > > > >
> > > > > Your point is valid though. I've been thinking about an alternative
> > way
> > > > to
> > > > > configure Async Loggers than via system properties. The work in
> > > progress
> > > > is
> > > > > tracked in Jira 

Re: why is AsyncLoggerContextSelector faster?

[Matt Sicker]

I think he's talking about the ClassLoaderContextSelector which attempts to
find the logger context through several different ways. Using the
sun.reflect.Reflection class to get the caller class itself is one of those
techniques which might slow things down a little.

I actually have a change to LogManager that can help speed up LoggerContext
resolution when you create a Logger using a Class instead of a String.
The ClassLoader of that Class should be passed to the LoggerContextFactory
to help speed up resolution of the appropriate LoggerContext.


On 21 July 2014 09:22, Remko Popma  wrote:

> Hi,
> No that is incorrect.
> If you do not specify AsyncLoggerContextSelector but instead configure with
>  and  loggers, you _do_ need the disruptor jar on
> the classpath and this does _not_ use AsyncAppender. AsyncAppender is
> completely separate from Async Loggers. Async Loggers (mixed or all async)
> use the disruptor and both need the disruptor jar.
>
> You keep mentioning a performance difference. I was assuming you were
> talking about the performance test results mentioned on the Async Logger
> manual page, but perhaps I was wrong? Are you experiencing a performance
> difference between the two flavors of Async Loggers in your application?
>
>
>
>
> On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
> mariano.l.gonza...@gmail.com> wrote:
>
> > Hello Remko,
> >
> > I think I found the difference. AsyncLoggerContextSelector always returns
> > the same instance of AsyncLoggerContext, which in turns always returns
> > instances of AsyncLogger, which uses disruptor to handle concurrency.
> >
> > However, with any other selector, a standard Logger instance is returned
> > and parallelism is achieved through an AsyncAppender. AsyncAppender use a
> > standard blocking queue instead of using disruptor which explains the
> > performance difference (there's also the fact that
> > AsyncLoggerContextSelector always returns the same context instance and
> > does not spend cycles in the lookup, but I think that is not a
> significant
> > cost once everything was warmed out).
> >
> > http://logging.apache.org/log4j/2.x/manual/async.html says that when
> using
> > mixed type loggers disruptor is needed on the classpath. That seems to be
> > an error. For what I see disruptor is only used when setting all loggers
> > asynchronous.
> >
> > Does this make sense? Anyway around this? Do you have a disruptor
> appender
> > somewhere?
> >
> > Thanks!
> >
> >
> > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma 
> > wrote:
> >
> > > To be honest, I haven't investigated in detail the reason for the
> > > difference in throughput in the performance test.
> > >
> > > Are you measuring the performance of your application container, and
> can
> > > you see an improvement when using Async Loggers?
> > > Do you see a large difference in performance _in your application_
> > between
> > > making all loggers Asynchronous and using mixed synchronous and
> > > Asynchronous loggers?
> > >
> > >
> > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> > > mariano.l.gonza...@gmail.com> wrote:
> > >
> > > > Hello Remko,
> > > >
> > > > Thanks for the insight. I guess my case falls into the wrong end of
> the
> > > > pareto law. My project is a low latency application container, so I
> > need
> > > to
> > > > have:
> > > >
> > > >
> > > >- low latency
> > > >- log separation (I actually had to implement my own context
> > selector
> > > >because my logic is more complicated than the standard
> > > >ClassLoaderContextSelector case)
> > > >- I want async loggers by default, but deployed apps need to be
> able
> > > to
> > > >specify sync loggers
> > > >
> > > >
> > > > Right now I'm kinda meeting those requirements using config file,
> > > AsyncRoot
> > > > and my custom selector, but it'd be really great to achieve a
> > performance
> > > > level like the one that AsyncContextSelector promises.
> > > >
> > > > Is there a way of doing that? For what I see on the code, the
> > > > AsyncLoggerContextSelector's secret sauce is just to always return an
> > > > AsyncLogger on the newInstance() method. Why is that so much faster
> > than
> > > > what ClassLoaderLoggerContextSelector does?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma 
> > > > wrote:
> > > >
> > > > > The Async Loggers created with the context selector have a slightly
> > > > > different mechanism. One of the differences is that LogEvent
> objects
> > > are
> > > > > re-used.
> > > > >
> > > > > However, unless your application is in the low-latency space, I
> would
> > > not
> > > > > worry too much about the performance difference. Both flavors of
> > Async
> > > > > Loggers are much faster than the alternative (Async Appenders).
> > > > >
> > > > > Your point is valid though. I've been thinking about an alternative
> > way
> > > > to
> > > > > configure Async Loggers than via system properties. The work in
> > > p

Re: why is AsyncLoggerContextSelector faster?

[Remko Popma]

Hi,
No that is incorrect.
If you do not specify AsyncLoggerContextSelector but instead configure with
 and  loggers, you _do_ need the disruptor jar on
the classpath and this does _not_ use AsyncAppender. AsyncAppender is
completely separate from Async Loggers. Async Loggers (mixed or all async)
use the disruptor and both need the disruptor jar.

You keep mentioning a performance difference. I was assuming you were
talking about the performance test results mentioned on the Async Logger
manual page, but perhaps I was wrong? Are you experiencing a performance
difference between the two flavors of Async Loggers in your application?




On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
mariano.l.gonza...@gmail.com> wrote:

> Hello Remko,
>
> I think I found the difference. AsyncLoggerContextSelector always returns
> the same instance of AsyncLoggerContext, which in turns always returns
> instances of AsyncLogger, which uses disruptor to handle concurrency.
>
> However, with any other selector, a standard Logger instance is returned
> and parallelism is achieved through an AsyncAppender. AsyncAppender use a
> standard blocking queue instead of using disruptor which explains the
> performance difference (there's also the fact that
> AsyncLoggerContextSelector always returns the same context instance and
> does not spend cycles in the lookup, but I think that is not a significant
> cost once everything was warmed out).
>
> http://logging.apache.org/log4j/2.x/manual/async.html says that when using
> mixed type loggers disruptor is needed on the classpath. That seems to be
> an error. For what I see disruptor is only used when setting all loggers
> asynchronous.
>
> Does this make sense? Anyway around this? Do you have a disruptor appender
> somewhere?
>
> Thanks!
>
>
> On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma 
> wrote:
>
> > To be honest, I haven't investigated in detail the reason for the
> > difference in throughput in the performance test.
> >
> > Are you measuring the performance of your application container, and can
> > you see an improvement when using Async Loggers?
> > Do you see a large difference in performance _in your application_
> between
> > making all loggers Asynchronous and using mixed synchronous and
> > Asynchronous loggers?
> >
> >
> > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> > mariano.l.gonza...@gmail.com> wrote:
> >
> > > Hello Remko,
> > >
> > > Thanks for the insight. I guess my case falls into the wrong end of the
> > > pareto law. My project is a low latency application container, so I
> need
> > to
> > > have:
> > >
> > >
> > >- low latency
> > >- log separation (I actually had to implement my own context
> selector
> > >because my logic is more complicated than the standard
> > >ClassLoaderContextSelector case)
> > >- I want async loggers by default, but deployed apps need to be able
> > to
> > >specify sync loggers
> > >
> > >
> > > Right now I'm kinda meeting those requirements using config file,
> > AsyncRoot
> > > and my custom selector, but it'd be really great to achieve a
> performance
> > > level like the one that AsyncContextSelector promises.
> > >
> > > Is there a way of doing that? For what I see on the code, the
> > > AsyncLoggerContextSelector's secret sauce is just to always return an
> > > AsyncLogger on the newInstance() method. Why is that so much faster
> than
> > > what ClassLoaderLoggerContextSelector does?
> > >
> > > Thanks!
> > >
> > >
> > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma 
> > > wrote:
> > >
> > > > The Async Loggers created with the context selector have a slightly
> > > > different mechanism. One of the differences is that LogEvent objects
> > are
> > > > re-used.
> > > >
> > > > However, unless your application is in the low-latency space, I would
> > not
> > > > worry too much about the performance difference. Both flavors of
> Async
> > > > Loggers are much faster than the alternative (Async Appenders).
> > > >
> > > > Your point is valid though. I've been thinking about an alternative
> way
> > > to
> > > > configure Async Loggers than via system properties. The work in
> > progress
> > > is
> > > > tracked in Jira ticket LOG4J2-321
> > > > . This is still in
> > the
> > > > concept phase though. Meanwhile your best option is probably to use
> > > > ClassLoaderContextSelector and configure with  and
> > > > .
> > > >
> > > >
> > > >
> > > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez <
> > > > mariano.l.gonza...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > According to the performance charts in the documentation, log4j2
> has
> > a
> > > > > significantly higher throughput when using
> AsyncLoggerContextSelector
> > > > than
> > > > > when using all async loggers with any different selector.
> > > > >
> > > > > Why is that? Is it just because the same context is always reused
> and
> > > > > there's no lookup like in the C

Re: why is AsyncLoggerContextSelector faster?

[Mariano Gonzalez]

Hello Remko,

I think I found the difference. AsyncLoggerContextSelector always returns
the same instance of AsyncLoggerContext, which in turns always returns
instances of AsyncLogger, which uses disruptor to handle concurrency.

However, with any other selector, a standard Logger instance is returned
and parallelism is achieved through an AsyncAppender. AsyncAppender use a
standard blocking queue instead of using disruptor which explains the
performance difference (there's also the fact that
AsyncLoggerContextSelector always returns the same context instance and
does not spend cycles in the lookup, but I think that is not a significant
cost once everything was warmed out).

http://logging.apache.org/log4j/2.x/manual/async.html says that when using
mixed type loggers disruptor is needed on the classpath. That seems to be
an error. For what I see disruptor is only used when setting all loggers
asynchronous.

Does this make sense? Anyway around this? Do you have a disruptor appender
somewhere?

Thanks!


On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma  wrote:

> To be honest, I haven't investigated in detail the reason for the
> difference in throughput in the performance test.
>
> Are you measuring the performance of your application container, and can
> you see an improvement when using Async Loggers?
> Do you see a large difference in performance _in your application_ between
> making all loggers Asynchronous and using mixed synchronous and
> Asynchronous loggers?
>
>
> On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
> mariano.l.gonza...@gmail.com> wrote:
>
> > Hello Remko,
> >
> > Thanks for the insight. I guess my case falls into the wrong end of the
> > pareto law. My project is a low latency application container, so I need
> to
> > have:
> >
> >
> >- low latency
> >- log separation (I actually had to implement my own context selector
> >because my logic is more complicated than the standard
> >ClassLoaderContextSelector case)
> >- I want async loggers by default, but deployed apps need to be able
> to
> >specify sync loggers
> >
> >
> > Right now I'm kinda meeting those requirements using config file,
> AsyncRoot
> > and my custom selector, but it'd be really great to achieve a performance
> > level like the one that AsyncContextSelector promises.
> >
> > Is there a way of doing that? For what I see on the code, the
> > AsyncLoggerContextSelector's secret sauce is just to always return an
> > AsyncLogger on the newInstance() method. Why is that so much faster than
> > what ClassLoaderLoggerContextSelector does?
> >
> > Thanks!
> >
> >
> > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma 
> > wrote:
> >
> > > The Async Loggers created with the context selector have a slightly
> > > different mechanism. One of the differences is that LogEvent objects
> are
> > > re-used.
> > >
> > > However, unless your application is in the low-latency space, I would
> not
> > > worry too much about the performance difference. Both flavors of Async
> > > Loggers are much faster than the alternative (Async Appenders).
> > >
> > > Your point is valid though. I've been thinking about an alternative way
> > to
> > > configure Async Loggers than via system properties. The work in
> progress
> > is
> > > tracked in Jira ticket LOG4J2-321
> > > . This is still in
> the
> > > concept phase though. Meanwhile your best option is probably to use
> > > ClassLoaderContextSelector and configure with  and
> > > .
> > >
> > >
> > >
> > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez <
> > > mariano.l.gonza...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > According to the performance charts in the documentation, log4j2 has
> a
> > > > significantly higher throughput when using AsyncLoggerContextSelector
> > > than
> > > > when using all async loggers with any different selector.
> > > >
> > > > Why is that? Is it just because the same context is always reused and
> > > > there's no lookup like in the ClassLoaderContextSelector case?
> > > >
> > > > If I need functionality similar to ClassLoaderContextSelector, is
> there
> > > any
> > > > way to get a throughput similar to AsyncLoggerContextSelector?
> > > >
> > > > Thanks!
> > > >
> > >
> >
>

Re: why is AsyncLoggerContextSelector faster?

[Remko Popma]

To be honest, I haven't investigated in detail the reason for the
difference in throughput in the performance test.

Are you measuring the performance of your application container, and can
you see an improvement when using Async Loggers?
Do you see a large difference in performance _in your application_ between
making all loggers Asynchronous and using mixed synchronous and
Asynchronous loggers?


On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
mariano.l.gonza...@gmail.com> wrote:

> Hello Remko,
>
> Thanks for the insight. I guess my case falls into the wrong end of the
> pareto law. My project is a low latency application container, so I need to
> have:
>
>
>- low latency
>- log separation (I actually had to implement my own context selector
>because my logic is more complicated than the standard
>ClassLoaderContextSelector case)
>- I want async loggers by default, but deployed apps need to be able to
>specify sync loggers
>
>
> Right now I'm kinda meeting those requirements using config file, AsyncRoot
> and my custom selector, but it'd be really great to achieve a performance
> level like the one that AsyncContextSelector promises.
>
> Is there a way of doing that? For what I see on the code, the
> AsyncLoggerContextSelector's secret sauce is just to always return an
> AsyncLogger on the newInstance() method. Why is that so much faster than
> what ClassLoaderLoggerContextSelector does?
>
> Thanks!
>
>
> On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma 
> wrote:
>
> > The Async Loggers created with the context selector have a slightly
> > different mechanism. One of the differences is that LogEvent objects are
> > re-used.
> >
> > However, unless your application is in the low-latency space, I would not
> > worry too much about the performance difference. Both flavors of Async
> > Loggers are much faster than the alternative (Async Appenders).
> >
> > Your point is valid though. I've been thinking about an alternative way
> to
> > configure Async Loggers than via system properties. The work in progress
> is
> > tracked in Jira ticket LOG4J2-321
> > . This is still in the
> > concept phase though. Meanwhile your best option is probably to use
> > ClassLoaderContextSelector and configure with  and
> > .
> >
> >
> >
> > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez <
> > mariano.l.gonza...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > According to the performance charts in the documentation, log4j2 has a
> > > significantly higher throughput when using AsyncLoggerContextSelector
> > than
> > > when using all async loggers with any different selector.
> > >
> > > Why is that? Is it just because the same context is always reused and
> > > there's no lookup like in the ClassLoaderContextSelector case?
> > >
> > > If I need functionality similar to ClassLoaderContextSelector, is there
> > any
> > > way to get a throughput similar to AsyncLoggerContextSelector?
> > >
> > > Thanks!
> > >
> >
>

Re: why is AsyncLoggerContextSelector faster?

[Mariano Gonzalez]

Hello Remko,

Thanks for the insight. I guess my case falls into the wrong end of the
pareto law. My project is a low latency application container, so I need to
have:


   - low latency
   - log separation (I actually had to implement my own context selector
   because my logic is more complicated than the standard
   ClassLoaderContextSelector case)
   - I want async loggers by default, but deployed apps need to be able to
   specify sync loggers


Right now I'm kinda meeting those requirements using config file, AsyncRoot
and my custom selector, but it'd be really great to achieve a performance
level like the one that AsyncContextSelector promises.

Is there a way of doing that? For what I see on the code, the
AsyncLoggerContextSelector's secret sauce is just to always return an
AsyncLogger on the newInstance() method. Why is that so much faster than
what ClassLoaderLoggerContextSelector does?

Thanks!


On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma  wrote:

> The Async Loggers created with the context selector have a slightly
> different mechanism. One of the differences is that LogEvent objects are
> re-used.
>
> However, unless your application is in the low-latency space, I would not
> worry too much about the performance difference. Both flavors of Async
> Loggers are much faster than the alternative (Async Appenders).
>
> Your point is valid though. I've been thinking about an alternative way to
> configure Async Loggers than via system properties. The work in progress is
> tracked in Jira ticket LOG4J2-321
> . This is still in the
> concept phase though. Meanwhile your best option is probably to use
> ClassLoaderContextSelector and configure with  and
> .
>
>
>
> On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez <
> mariano.l.gonza...@gmail.com> wrote:
>
> > Hello,
> >
> > According to the performance charts in the documentation, log4j2 has a
> > significantly higher throughput when using AsyncLoggerContextSelector
> than
> > when using all async loggers with any different selector.
> >
> > Why is that? Is it just because the same context is always reused and
> > there's no lookup like in the ClassLoaderContextSelector case?
> >
> > If I need functionality similar to ClassLoaderContextSelector, is there
> any
> > way to get a throughput similar to AsyncLoggerContextSelector?
> >
> > Thanks!
> >
>

Re: why is AsyncLoggerContextSelector faster?

[Remko Popma]

The Async Loggers created with the context selector have a slightly
different mechanism. One of the differences is that LogEvent objects are
re-used.

However, unless your application is in the low-latency space, I would not
worry too much about the performance difference. Both flavors of Async
Loggers are much faster than the alternative (Async Appenders).

Your point is valid though. I've been thinking about an alternative way to
configure Async Loggers than via system properties. The work in progress is
tracked in Jira ticket LOG4J2-321
. This is still in the
concept phase though. Meanwhile your best option is probably to use
ClassLoaderContextSelector and configure with  and .



On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez <
mariano.l.gonza...@gmail.com> wrote:

> Hello,
>
> According to the performance charts in the documentation, log4j2 has a
> significantly higher throughput when using AsyncLoggerContextSelector than
> when using all async loggers with any different selector.
>
> Why is that? Is it just because the same context is always reused and
> there's no lookup like in the ClassLoaderContextSelector case?
>
> If I need functionality similar to ClassLoaderContextSelector, is there any
> way to get a throughput similar to AsyncLoggerContextSelector?
>
> Thanks!
>