Sure. This will however block by itself and take care of preserving compatibility with the ancient frameworks. With this mentioned, today might be a good day to start a poll on what frameworks log4net should continue to support. In the last days I once more spent numerous hours with the build infrastructure and am fed up by the insane build process caused by the numerous supported frameworks. If we focus on netstandard-1.3 as the base framework almost every recent framework can reference log4net. There was also a proposal to support netstandard-2.0 but thinking about it I see no reason to add another framework if we do not need any of the apis that the framework provides. If we need netstandard-2.0 functionality we might as well provide that functionality as a separate nuget library. Yes, splitting up log4net into several smaller assemblies sounds like a plan to me.
2018-05-10 18:00 GMT+02:00 William Davis <[email protected]>: > Perhaps, but looking at that implementation I see that it is locking in a > few places on append. Could this be made a little better by using built in > ConcurrentCollection types like the ConcurrentQueue? > > On Thu, May 10, 2018 at 1:23 AM, Dominik Psenner <[email protected]> > wrote: > > > This proposal sounds like the bufferingforwardingappender. > > > > On Thu, 10 May 2018, 04:48 William Davis, <[email protected]> > > wrote: > > > > > Agreed, this is what ill be submitting next. > > > > > > On Wed, May 9, 2018, 9:47 PM Remko Popma <[email protected]> > wrote: > > > > > > > Perhaps a reasonable approach would be to work like log4j‘s > > > AsyncAppender: > > > > > > > > This is a class that implements the appender interface by simply > adding > > > > log events to a ConcurrentQueue and returning immediately. When this > > > > appender is started it starts a background thread that blocks until > > > events > > > > become available in the queue. When the queue contains an event, the > > > > background thread pops it off the queue and appends it to one or more > > > > underlying appenders. > > > > > > > > Note that on the producer (application) side, this looks like any > other > > > > appender. The consumer side (the background thread) is likely where > the > > > > async/await api would be used. > > > > > > > > An AsyncAppender must be configured with one or more underlying > > > appenders. > > > > (In log4j these appenders must precede the AsyncAppender in the > > > > configuration so the list of underlying appenders can be immutable). > > > > > > > > Hope this helps, > > > > Remko > > > > > > > > (Shameless plug) Every java main() method deserves > http://picocli.info > > > > > > > > > On May 10, 2018, at 4:04, Matt Sicker <[email protected]> wrote: > > > > > > > > > > One resource I have about fibers is this Java library: > > > > > https://github.com/puniverse/quasar > > > > > > > > > > And the future Java feature: http://openjdk.java.net/ > projects/loom/ > > > > > > > > > > As for continuations, if you're familiar with functional > programming, > > > are > > > > > essentially deferred functions to be executed along with any > curried > > > > state. > > > > > It essentially allows you to pause a computation, but you can't use > > > > things > > > > > like locks and notifications since those are implemented via > threads, > > > and > > > > > fibers don't get their own execution context like threads do (hence > > why > > > > > they're much faster where applicable due to less context switching > > and > > > > data > > > > > copying needed). > > > > > > > > > > > > > > >> On 9 May 2018 at 13:41, Dominik Psenner <[email protected]> > wrote: > > > > >> > > > > >> Btw, here is an example of async file io, note that this is a wpf > > > client > > > > >> application that stays responsive even though there is a > "blocking" > > > > await > > > > >> in the button handler: > > > > >> > > > > >> > > > > > > > https://docs.microsoft.com/en-us/dotnet/standard/io/ > > asynchronous-file-i-o > > > > >> > > > > >> and here is an example of async network io which also explains > more > > > > >> in-depth details of how it works: > > > > >> > > > > >> https://docs.microsoft.com/en-us/dotnet/csharp/programming- > > > > >> guide/concepts/async/ > > > > >> > > > > >> As a starting point it surely takes time to grasp and caused me > some > > > > >> headache. :-) > > > > >> > > > > >> 2018-05-09 20:33 GMT+02:00 Dominik Psenner <[email protected]>: > > > > >> > > > > >>> I don't know about fibers or continuations but am interested. Can > > you > > > > >>> provide me with some link? > > > > >>> > > > > >>> AFAIK, LMAX disruptor intelligently uses hot spins on the cpu > where > > > it > > > > >>> estimates that hot spinning pays off because an async operation > > will > > > > >> finish > > > > >>> soon. When this is not the case after a few hot spins it will > yield > > > and > > > > >>> cause a context switch. When I read this up I quickly came to the > > > > >>> conclusion that such a way makes it very hard to implement > > something > > > > >> that's > > > > >>> reliably fast and stable at the same time. > > > > >>> > > > > >>> I can't provide the following with backup information, but as far > > as > > > I > > > > >>> understood the async/await approach it works so well because the > > > > hardware > > > > >>> provides interrupts to the operating system when data arrives > which > > > in > > > > >> turn > > > > >>> is published to an application via events. As noticed earlier, > > libuv > > > > is a > > > > >>> cross platform library that provides these event api's to an > > > > application. > > > > >>> In the dotnet world since the invention of Task and async/await a > > > libuv > > > > >> has > > > > >>> mostly become futile. The kestrel web server, as far as I know, > > uses > > > > >> libuv > > > > >>> under the hoods and is used by some microsoft devs as playground > to > > > > >>> improve the performance of implementations of the async api's > > > provided > > > > by > > > > >>> netstandard. Future versions of asp.netcore will probably no > longer > > > > >> feature > > > > >>> the kestrel webserver with libuv transports but transports that > are > > > > based > > > > >>> upon .netstandard System.Net.Sockets. > > > > >>> > > > > >>> 2018-05-09 20:17 GMT+02:00 Matt Sicker <[email protected]>: > > > > >>> > > > > >>>> I'm not too familiar with how it's implemented, but that sounds > > > > similar > > > > >> to > > > > >>>> the problems that LMAX was fixing in lock-free queues. The > problem > > > > with > > > > >>>> typical async/await is lock contention which is addressed in a > > lower > > > > >> level > > > > >>>> fashion in disruptor queues. I think this would all be far > easier > > > with > > > > >>>> something like fibers or continuations, but I didn't design > Java. > > :) > > > > >>>> > > > > >>>>> On 9 May 2018 at 13:09, Dominik Psenner <[email protected]> > > > wrote: > > > > >>>>> > > > > >>>>> Disclaimer: so far I never had to use a library like LMAX > > > disruptor. > > > > >>>> After > > > > >>>>> a lot of brain that I spent into the new async/await approach > > > that's > > > > >>>>> available today I even think that a truely high performance > .net > > > > >>>>> application has no need for such library. The following > hopefully > > > > >>>> explains > > > > >>>>> the why's. > > > > >>>>> > > > > >>>>> To me there are mainly two aspects of asynch operations. One is > > the > > > > >>>> asynch > > > > >>>>> nature of multithreading where computational expensive > operations > > > are > > > > >>>>> offloaded to background threads. The other is async io which > > allows > > > > >> the > > > > >>>> cpu > > > > >>>>> to continue doing other tasks when the, compared to the cpu > > cycling > > > > on > > > > >>>> its > > > > >>>>> calculations, veeeery slooooow io like networking is involved. > > > > >>>>> > > > > >>>>> Asyc/await with tasks provides, from an api point of view, > both. > > > > >>>>> Traditionally an io operation would either block the cpu while > > > > waiting > > > > >>>> for > > > > >>>>> the io to complete or be buffered/offloaded to a background > > thread > > > > and > > > > >>>>> finished there. The downside of such an approach is that this > > > > involves > > > > >>>>> cross thread synchronization points. The actual problem we need > > to > > > > >>>> solve is > > > > >>>>> that we do want the cpu to wait for the slow io. This is where > > the > > > > >>>>> async/await comes into play. async/await allows the io > operation > > to > > > > >>>> start > > > > >>>>> and the cpu to continue its task. When the async io is complete > > an > > > > >> event > > > > >>>>> fired by the io will trigger the cpu to continue its work on a > > > > >>>>> synchronization point that is chosen with an await. While this > > > works > > > > >>>> best > > > > >>>>> with io, it also works with cpu intensive tasks that need to be > > run > > > > on > > > > >>>>> background threads. But using this for computational expensive > > cpu > > > > >> tasks > > > > >>>>> only pays off it the costs of synchronization and context > > switches > > > is > > > > >>>>> insignificant with respect to the actual task. > > > > >>>>> > > > > >>>>> This said, if an appender involves IO, a client application > could > > > > >>>>> ultimately choose to either fire and forget, wait for the io or > > > > >> continue > > > > >>>>> and synchronize later if we provided an async api. This however > > > > >>>> requires us > > > > >>>>> to provide both a "normal" api and an async api. But doing so > > > rewards > > > > >>>> with > > > > >>>>> a truely async io. > > > > >>>>> > > > > >>>>> Note that this is something what nginx makes heavy use of. > libuv > > > is a > > > > >>>>> library that provides a few aspects of io as an event based > api. > > > > >>>>> > > > > >>>>>> On Wed, 9 May 2018, 16:56 Matt Sicker, <[email protected]> > > wrote: > > > > >>>>>> > > > > >>>>>> I'd be interesting in hearing about high performant .NET > > > > >> applications > > > > >>>>> that > > > > >>>>>> would necessitate the creation of libraries like LMAX > Disruptor. > > > > >>>> AFAIK, > > > > >>>>>> that's generally a C++ and Java world. > > > > >>>>>> > > > > >>>>>>> On 9 May 2018 at 08:47, Remko Popma <[email protected]> > > > wrote: > > > > >>>>>>> > > > > >>>>>>> In the log4j world, async logging means adding the > information > > to > > > > >> be > > > > >>>>>>> logged to some data structure, whereupon the application > thread > > > > >>>> returns > > > > >>>>>>> immediately to do other work. > > > > >>>>>>> In the background, another thread reads the information to be > > > > >> logged > > > > >>>>> from > > > > >>>>>>> the data structure, potentially transforms it, then renders > it > > to > > > > >>>> the > > > > >>>>>>> configured layout format and writes it to the configured > > > > >>>> appender(s). > > > > >>>>>>> > > > > >>>>>>> The data structure may be a standard queue, in which case the > > > > >>>>>> “information > > > > >>>>>>> to be logged” is often a LogEvent instance, or it could be a > > data > > > > >>>>>> structure > > > > >>>>>>> that is optimized for non-blocking inter-thread handovers, > like > > > > >> the > > > > >>>>> LMAX > > > > >>>>>>> Disruptor. I don’t know what the equivalent of the latter is > in > > > > >> the > > > > >>>>> .NET > > > > >>>>>>> world. > > > > >>>>>>> > > > > >>>>>>> It seems that concurrent queues in .net may use Async/await > > under > > > > >>>> the > > > > >>>>>>> hood. (Based on what I see on SO, like > > > https://stackoverflow.com/ > > > > >>>>>>> questions/7863573/awaitable-task-based-queue) > > > > >>>>>>> > > > > >>>>>>> Not sure if lock-free mechanisms like the lmax disruptor > exist. > > > Be > > > > >>>>> aware > > > > >>>>>>> that the background thread needs to employ some waiting > > strategy > > > > >>>> until > > > > >>>>>> work > > > > >>>>>>> arrives. The simplest thing is to use some block-notify > > > mechanism: > > > > >>>> the > > > > >>>>>>> background thread is suspended and woken up by the operating > > > > >> system > > > > >>>>> when > > > > >>>>>>> notified. I assume this is what async/await uses. To be > > > completely > > > > >>>>>>> lock-free, an alternative wait strategy is to busy-spin but > > this > > > > >>>> means > > > > >>>>>>> dedicating a core to logging which is a hefty price. In the > > > > >>>> disruptor > > > > >>>>>> this > > > > >>>>>>> is configurable so if log4j users really want to they can > have > > > > >>>>> lock-free > > > > >>>>>>> logging in return for dedicating a cpu core. You may not want > > or > > > > >>>> need > > > > >>>>> to > > > > >>>>>> go > > > > >>>>>>> that far. > > > > >>>>>>> > > > > >>>>>>> Remko > > > > >>>>>>> > > > > >>>>>>> (Shameless plug) Every java main() method deserves > > > > >>>> http://picocli.info > > > > >>>>>>> > > > > >>>>>>>> On May 9, 2018, at 22:06, Dominik Psenner < > [email protected] > > > > > > > >>>>> wrote: > > > > >>>>>>>> > > > > >>>>>>>> When implementing the async/await paradigm it would have to > be > > > > >>>>> provided > > > > >>>>>>> as a logging event api and continuously invoked with async > down > > > to > > > > >>>> the > > > > >>>>>>> appender implementations in order for the application code to > > > > >>>> benefit > > > > >>>>>> from > > > > >>>>>>> true async behavior. Or am I wrong here? > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> On 2018-05-09 13:48, William Davis wrote: > > > > >>>>>>>>> Jochen, I dont believe that appender has been ported to > > > > >> Log4Net. > > > > >>>>> Maybe > > > > >>>>>>>>> thats what we should do first? Im sure there are other uses > > > > >> cases > > > > >>>>> out > > > > >>>>>>> there > > > > >>>>>>>>> though, which is why we've seen several people roll async > > > > >>>> appenders > > > > >>>>> in > > > > >>>>>>> the > > > > >>>>>>>>> first place (although it could be a fundamental lack of > > > > >>>>> understanding) > > > > >>>>>>>>> > > > > >>>>>>>>> On Wed, May 9, 2018 at 7:00 AM, Jochen Wiedmann < > > > > >>>>>>> [email protected]> > > > > >>>>>>>>> wrote: > > > > >>>>>>>>> > > > > >>>>>>>>>> On Mon, May 7, 2018 at 2:15 PM William Davis < > > > > >>>>>>> [email protected]> > > > > >>>>>>>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>>>> I've noticed that there are several Async implementations > > of > > > > >>>>>> standard > > > > >>>>>>>>>>> appenders out in the wild. Is there a reason none of > these > > > > >> have > > > > >>>>> made > > > > >>>>>>>>>> there > > > > >>>>>>>>>>> way into the core product? Is it just b/c no one has > taken > > > > >> the > > > > >>>>> time > > > > >>>>>>> to do > > > > >>>>>>>>>> a > > > > >>>>>>>>>>> pull request, or is there some other reason? > > > > >>>>>>>>>> I wonder, why one would create a special async version, > when > > > > >> all > > > > >>>>> you > > > > >>>>>>> need > > > > >>>>>>>>>> to do is to put a standard async logger in front of the > sync > > > > >>>> logger > > > > >>>>>>> [1]? > > > > >>>>>>>>>> > > > > >>>>>>>>>> Jochen > > > > >>>>>>>>>> > > > > >>>>>>>>>> 1: https://logging.apache.org/ > log4j/2.x/manual/async.html# > > > > >>>>>>> MixedSync-Async > > > > >>>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> -- > > > > >>>>>> Matt Sicker <[email protected]> > > > > >>>>>> > > > > >>>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> -- > > > > >>>> Matt Sicker <[email protected]> > > > > >>>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> -- > > > > >>> Dominik Psenner > > > > >>> > > > > >> > > > > >> > > > > >> > > > > >> -- > > > > >> Dominik Psenner > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Matt Sicker <[email protected]> > > > > > > > > > > -- Dominik Psenner
