That's great! I'm looking forward to hearing from you. 2018-05-11 5:10 GMT+02:00 William Davis <william.j.dav...@gmail.com>:
> Well, I cant speak for the entire test suite, but we having been running > log4net in production on dotnet core since it was released and my ELK stack > seems to be getting lots of logs :) I'll reach out to Nick at SE and see if > he can expound upon his previous message. > > On Thu, May 10, 2018 at 5:55 PM, Dominik Psenner <dpsen...@gmail.com> > wrote: > > > Interesting discussion in that pull request, yet it's missing links to > hard > > facts what these functionalities, new features, smaller install area, no > > downsides etc actually are. Have you links to that information? > > > > I've said numerous times now that we don't even run the test suite > against > > netstandard-1.3. It's no point to add netstandard-2.0 if we don't even > know > > whether netstandard-1.3 actually works. And there's an issue somewhere - > > sorry, no link at the moment - that appears to be a regression related to > > that target. > > > > 2018-05-10 23:09 GMT+02:00 William Davis <william.j.dav...@gmail.com>: > > > > > Not a bad conversation to have. I would direct you to read this PR from > > the > > > SE.Redis library where I argued against netstandard 2.0 inclusion at > one > > > point. > > > https://github.com/StackExchange/StackExchange.Redis/pull/767 > > > > > > It would probably be worth while to provide explicit support for it. > > (with > > > out any additional effort) > > > > > > On Thu, May 10, 2018 at 2:42 PM, Dominik Psenner <dpsen...@gmail.com> > > > wrote: > > > > > > > Sure. This will however block by itself and take care of preserving > > > > compatibility with the ancient frameworks. With this mentioned, today > > > might > > > > be a good day to start a poll on what frameworks log4net should > > continue > > > to > > > > support. In the last days I once more spent numerous hours with the > > build > > > > infrastructure and am fed up by the insane build process caused by > the > > > > numerous supported frameworks. If we focus on netstandard-1.3 as the > > base > > > > framework almost every recent framework can reference log4net. There > > was > > > > also a proposal to support netstandard-2.0 but thinking about it I > see > > no > > > > reason to add another framework if we do not need any of the apis > that > > > the > > > > framework provides. If we need netstandard-2.0 functionality we might > > as > > > > well provide that functionality as a separate nuget library. Yes, > > > splitting > > > > up log4net into several smaller assemblies sounds like a plan to me. > > > > > > > > 2018-05-10 18:00 GMT+02:00 William Davis <william.j.dav...@gmail.com > >: > > > > > > > > > Perhaps, but looking at that implementation I see that it is > locking > > > in a > > > > > few places on append. Could this be made a little better by using > > built > > > > in > > > > > ConcurrentCollection types like the ConcurrentQueue? > > > > > > > > > > On Thu, May 10, 2018 at 1:23 AM, Dominik Psenner < > dpsen...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > This proposal sounds like the bufferingforwardingappender. > > > > > > > > > > > > On Thu, 10 May 2018, 04:48 William Davis, < > > > william.j.dav...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Agreed, this is what ill be submitting next. > > > > > > > > > > > > > > On Wed, May 9, 2018, 9:47 PM Remko Popma < > remko.po...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > Perhaps a reasonable approach would be to work like log4j‘s > > > > > > > AsyncAppender: > > > > > > > > > > > > > > > > This is a class that implements the appender interface by > > simply > > > > > adding > > > > > > > > log events to a ConcurrentQueue and returning immediately. > When > > > > this > > > > > > > > appender is started it starts a background thread that blocks > > > until > > > > > > > events > > > > > > > > become available in the queue. When the queue contains an > > event, > > > > the > > > > > > > > background thread pops it off the queue and appends it to one > > or > > > > more > > > > > > > > underlying appenders. > > > > > > > > > > > > > > > > Note that on the producer (application) side, this looks like > > any > > > > > other > > > > > > > > appender. The consumer side (the background thread) is likely > > > where > > > > > the > > > > > > > > async/await api would be used. > > > > > > > > > > > > > > > > An AsyncAppender must be configured with one or more > underlying > > > > > > > appenders. > > > > > > > > (In log4j these appenders must precede the AsyncAppender in > the > > > > > > > > configuration so the list of underlying appenders can be > > > > immutable). > > > > > > > > > > > > > > > > Hope this helps, > > > > > > > > Remko > > > > > > > > > > > > > > > > (Shameless plug) Every java main() method deserves > > > > > http://picocli.info > > > > > > > > > > > > > > > > > On May 10, 2018, at 4:04, Matt Sicker <boa...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > > > > One resource I have about fibers is this Java library: > > > > > > > > > https://github.com/puniverse/quasar > > > > > > > > > > > > > > > > > > And the future Java feature: http://openjdk.java.net/ > > > > > projects/loom/ > > > > > > > > > > > > > > > > > > As for continuations, if you're familiar with functional > > > > > programming, > > > > > > > are > > > > > > > > > essentially deferred functions to be executed along with > any > > > > > curried > > > > > > > > state. > > > > > > > > > It essentially allows you to pause a computation, but you > > can't > > > > use > > > > > > > > things > > > > > > > > > like locks and notifications since those are implemented > via > > > > > threads, > > > > > > > and > > > > > > > > > fibers don't get their own execution context like threads > do > > > > (hence > > > > > > why > > > > > > > > > they're much faster where applicable due to less context > > > > switching > > > > > > and > > > > > > > > data > > > > > > > > > copying needed). > > > > > > > > > > > > > > > > > > > > > > > > > > >> On 9 May 2018 at 13:41, Dominik Psenner < > dpsen...@gmail.com > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> Btw, here is an example of async file io, note that this > is > > a > > > > wpf > > > > > > > client > > > > > > > > >> application that stays responsive even though there is a > > > > > "blocking" > > > > > > > > await > > > > > > > > >> in the button handler: > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > https://docs.microsoft.com/en-us/dotnet/standard/io/ > > > > > > asynchronous-file-i-o > > > > > > > > >> > > > > > > > > >> and here is an example of async network io which also > > explains > > > > > more > > > > > > > > >> in-depth details of how it works: > > > > > > > > >> > > > > > > > > >> https://docs.microsoft.com/en- > us/dotnet/csharp/programming- > > > > > > > > >> guide/concepts/async/ > > > > > > > > >> > > > > > > > > >> As a starting point it surely takes time to grasp and > caused > > > me > > > > > some > > > > > > > > >> headache. :-) > > > > > > > > >> > > > > > > > > >> 2018-05-09 20:33 GMT+02:00 Dominik Psenner < > > > dpsen...@gmail.com > > > > >: > > > > > > > > >> > > > > > > > > >>> I don't know about fibers or continuations but am > > interested. > > > > Can > > > > > > you > > > > > > > > >>> provide me with some link? > > > > > > > > >>> > > > > > > > > >>> AFAIK, LMAX disruptor intelligently uses hot spins on the > > cpu > > > > > where > > > > > > > it > > > > > > > > >>> estimates that hot spinning pays off because an async > > > operation > > > > > > will > > > > > > > > >> finish > > > > > > > > >>> soon. When this is not the case after a few hot spins it > > will > > > > > yield > > > > > > > and > > > > > > > > >>> cause a context switch. When I read this up I quickly > came > > to > > > > the > > > > > > > > >>> conclusion that such a way makes it very hard to > implement > > > > > > something > > > > > > > > >> that's > > > > > > > > >>> reliably fast and stable at the same time. > > > > > > > > >>> > > > > > > > > >>> I can't provide the following with backup information, > but > > as > > > > far > > > > > > as > > > > > > > I > > > > > > > > >>> understood the async/await approach it works so well > > because > > > > the > > > > > > > > hardware > > > > > > > > >>> provides interrupts to the operating system when data > > arrives > > > > > which > > > > > > > in > > > > > > > > >> turn > > > > > > > > >>> is published to an application via events. As noticed > > > earlier, > > > > > > libuv > > > > > > > > is a > > > > > > > > >>> cross platform library that provides these event api's to > > an > > > > > > > > application. > > > > > > > > >>> In the dotnet world since the invention of Task and > > > > async/await a > > > > > > > libuv > > > > > > > > >> has > > > > > > > > >>> mostly become futile. The kestrel web server, as far as I > > > know, > > > > > > uses > > > > > > > > >> libuv > > > > > > > > >>> under the hoods and is used by some microsoft devs as > > > > playground > > > > > to > > > > > > > > >>> improve the performance of implementations of the async > > api's > > > > > > > provided > > > > > > > > by > > > > > > > > >>> netstandard. Future versions of asp.netcore will probably > > no > > > > > longer > > > > > > > > >> feature > > > > > > > > >>> the kestrel webserver with libuv transports but > transports > > > that > > > > > are > > > > > > > > based > > > > > > > > >>> upon .netstandard System.Net.Sockets. > > > > > > > > >>> > > > > > > > > >>> 2018-05-09 20:17 GMT+02:00 Matt Sicker <boa...@gmail.com > >: > > > > > > > > >>> > > > > > > > > >>>> I'm not too familiar with how it's implemented, but that > > > > sounds > > > > > > > > similar > > > > > > > > >> to > > > > > > > > >>>> the problems that LMAX was fixing in lock-free queues. > The > > > > > problem > > > > > > > > with > > > > > > > > >>>> typical async/await is lock contention which is > addressed > > > in a > > > > > > lower > > > > > > > > >> level > > > > > > > > >>>> fashion in disruptor queues. I think this would all be > far > > > > > easier > > > > > > > with > > > > > > > > >>>> something like fibers or continuations, but I didn't > > design > > > > > Java. > > > > > > :) > > > > > > > > >>>> > > > > > > > > >>>>> On 9 May 2018 at 13:09, Dominik Psenner < > > > dpsen...@gmail.com> > > > > > > > wrote: > > > > > > > > >>>>> > > > > > > > > >>>>> Disclaimer: so far I never had to use a library like > LMAX > > > > > > > disruptor. > > > > > > > > >>>> After > > > > > > > > >>>>> a lot of brain that I spent into the new async/await > > > approach > > > > > > > that's > > > > > > > > >>>>> available today I even think that a truely high > > performance > > > > > .net > > > > > > > > >>>>> application has no need for such library. The following > > > > > hopefully > > > > > > > > >>>> explains > > > > > > > > >>>>> the why's. > > > > > > > > >>>>> > > > > > > > > >>>>> To me there are mainly two aspects of asynch > operations. > > > One > > > > is > > > > > > the > > > > > > > > >>>> asynch > > > > > > > > >>>>> nature of multithreading where computational expensive > > > > > operations > > > > > > > are > > > > > > > > >>>>> offloaded to background threads. The other is async io > > > which > > > > > > allows > > > > > > > > >> the > > > > > > > > >>>> cpu > > > > > > > > >>>>> to continue doing other tasks when the, compared to the > > cpu > > > > > > cycling > > > > > > > > on > > > > > > > > >>>> its > > > > > > > > >>>>> calculations, veeeery slooooow io like networking is > > > > involved. > > > > > > > > >>>>> > > > > > > > > >>>>> Asyc/await with tasks provides, from an api point of > > view, > > > > > both. > > > > > > > > >>>>> Traditionally an io operation would either block the > cpu > > > > while > > > > > > > > waiting > > > > > > > > >>>> for > > > > > > > > >>>>> the io to complete or be buffered/offloaded to a > > background > > > > > > thread > > > > > > > > and > > > > > > > > >>>>> finished there. The downside of such an approach is > that > > > this > > > > > > > > involves > > > > > > > > >>>>> cross thread synchronization points. The actual problem > > we > > > > need > > > > > > to > > > > > > > > >>>> solve is > > > > > > > > >>>>> that we do want the cpu to wait for the slow io. This > is > > > > where > > > > > > the > > > > > > > > >>>>> async/await comes into play. async/await allows the io > > > > > operation > > > > > > to > > > > > > > > >>>> start > > > > > > > > >>>>> and the cpu to continue its task. When the async io is > > > > complete > > > > > > an > > > > > > > > >> event > > > > > > > > >>>>> fired by the io will trigger the cpu to continue its > work > > > on > > > > a > > > > > > > > >>>>> synchronization point that is chosen with an await. > While > > > > this > > > > > > > works > > > > > > > > >>>> best > > > > > > > > >>>>> with io, it also works with cpu intensive tasks that > need > > > to > > > > be > > > > > > run > > > > > > > > on > > > > > > > > >>>>> background threads. But using this for computational > > > > expensive > > > > > > cpu > > > > > > > > >> tasks > > > > > > > > >>>>> only pays off it the costs of synchronization and > context > > > > > > switches > > > > > > > is > > > > > > > > >>>>> insignificant with respect to the actual task. > > > > > > > > >>>>> > > > > > > > > >>>>> This said, if an appender involves IO, a client > > application > > > > > could > > > > > > > > >>>>> ultimately choose to either fire and forget, wait for > the > > > io > > > > or > > > > > > > > >> continue > > > > > > > > >>>>> and synchronize later if we provided an async api. This > > > > however > > > > > > > > >>>> requires us > > > > > > > > >>>>> to provide both a "normal" api and an async api. But > > doing > > > so > > > > > > > rewards > > > > > > > > >>>> with > > > > > > > > >>>>> a truely async io. > > > > > > > > >>>>> > > > > > > > > >>>>> Note that this is something what nginx makes heavy use > > of. > > > > > libuv > > > > > > > is a > > > > > > > > >>>>> library that provides a few aspects of io as an event > > based > > > > > api. > > > > > > > > >>>>> > > > > > > > > >>>>>> On Wed, 9 May 2018, 16:56 Matt Sicker, < > > boa...@gmail.com> > > > > > > wrote: > > > > > > > > >>>>>> > > > > > > > > >>>>>> I'd be interesting in hearing about high performant > .NET > > > > > > > > >> applications > > > > > > > > >>>>> that > > > > > > > > >>>>>> would necessitate the creation of libraries like LMAX > > > > > Disruptor. > > > > > > > > >>>> AFAIK, > > > > > > > > >>>>>> that's generally a C++ and Java world. > > > > > > > > >>>>>> > > > > > > > > >>>>>>> On 9 May 2018 at 08:47, Remko Popma < > > > remko.po...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> In the log4j world, async logging means adding the > > > > > information > > > > > > to > > > > > > > > >> be > > > > > > > > >>>>>>> logged to some data structure, whereupon the > > application > > > > > thread > > > > > > > > >>>> returns > > > > > > > > >>>>>>> immediately to do other work. > > > > > > > > >>>>>>> In the background, another thread reads the > information > > > to > > > > be > > > > > > > > >> logged > > > > > > > > >>>>> from > > > > > > > > >>>>>>> the data structure, potentially transforms it, then > > > renders > > > > > it > > > > > > to > > > > > > > > >>>> the > > > > > > > > >>>>>>> configured layout format and writes it to the > > configured > > > > > > > > >>>> appender(s). > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> The data structure may be a standard queue, in which > > case > > > > the > > > > > > > > >>>>>> “information > > > > > > > > >>>>>>> to be logged” is often a LogEvent instance, or it > could > > > be > > > > a > > > > > > data > > > > > > > > >>>>>> structure > > > > > > > > >>>>>>> that is optimized for non-blocking inter-thread > > > handovers, > > > > > like > > > > > > > > >> the > > > > > > > > >>>>> LMAX > > > > > > > > >>>>>>> Disruptor. I don’t know what the equivalent of the > > latter > > > > is > > > > > in > > > > > > > > >> the > > > > > > > > >>>>> .NET > > > > > > > > >>>>>>> world. > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> It seems that concurrent queues in .net may use > > > Async/await > > > > > > under > > > > > > > > >>>> the > > > > > > > > >>>>>>> hood. (Based on what I see on SO, like > > > > > > > https://stackoverflow.com/ > > > > > > > > >>>>>>> questions/7863573/awaitable-task-based-queue) > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Not sure if lock-free mechanisms like the lmax > > disruptor > > > > > exist. > > > > > > > Be > > > > > > > > >>>>> aware > > > > > > > > >>>>>>> that the background thread needs to employ some > waiting > > > > > > strategy > > > > > > > > >>>> until > > > > > > > > >>>>>> work > > > > > > > > >>>>>>> arrives. The simplest thing is to use some > block-notify > > > > > > > mechanism: > > > > > > > > >>>> the > > > > > > > > >>>>>>> background thread is suspended and woken up by the > > > > operating > > > > > > > > >> system > > > > > > > > >>>>> when > > > > > > > > >>>>>>> notified. I assume this is what async/await uses. To > be > > > > > > > completely > > > > > > > > >>>>>>> lock-free, an alternative wait strategy is to > busy-spin > > > but > > > > > > this > > > > > > > > >>>> means > > > > > > > > >>>>>>> dedicating a core to logging which is a hefty price. > In > > > the > > > > > > > > >>>> disruptor > > > > > > > > >>>>>> this > > > > > > > > >>>>>>> is configurable so if log4j users really want to they > > can > > > > > have > > > > > > > > >>>>> lock-free > > > > > > > > >>>>>>> logging in return for dedicating a cpu core. You may > > not > > > > want > > > > > > or > > > > > > > > >>>> need > > > > > > > > >>>>> to > > > > > > > > >>>>>> go > > > > > > > > >>>>>>> that far. > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Remko > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> (Shameless plug) Every java main() method deserves > > > > > > > > >>>> http://picocli.info > > > > > > > > >>>>>>> > > > > > > > > >>>>>>>> On May 9, 2018, at 22:06, Dominik Psenner < > > > > > dpsen...@gmail.com > > > > > > > > > > > > > > > >>>>> wrote: > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>> When implementing the async/await paradigm it would > > have > > > > to > > > > > be > > > > > > > > >>>>> provided > > > > > > > > >>>>>>> as a logging event api and continuously invoked with > > > async > > > > > down > > > > > > > to > > > > > > > > >>>> the > > > > > > > > >>>>>>> appender implementations in order for the application > > > code > > > > to > > > > > > > > >>>> benefit > > > > > > > > >>>>>> from > > > > > > > > >>>>>>> true async behavior. Or am I wrong here? > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>>> On 2018-05-09 13:48, William Davis wrote: > > > > > > > > >>>>>>>>> Jochen, I dont believe that appender has been > ported > > to > > > > > > > > >> Log4Net. > > > > > > > > >>>>> Maybe > > > > > > > > >>>>>>>>> thats what we should do first? Im sure there are > > other > > > > uses > > > > > > > > >> cases > > > > > > > > >>>>> out > > > > > > > > >>>>>>> there > > > > > > > > >>>>>>>>> though, which is why we've seen several people roll > > > async > > > > > > > > >>>> appenders > > > > > > > > >>>>> in > > > > > > > > >>>>>>> the > > > > > > > > >>>>>>>>> first place (although it could be a fundamental > lack > > of > > > > > > > > >>>>> understanding) > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> On Wed, May 9, 2018 at 7:00 AM, Jochen Wiedmann < > > > > > > > > >>>>>>> jochen.wiedm...@gmail.com> > > > > > > > > >>>>>>>>> wrote: > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>>> On Mon, May 7, 2018 at 2:15 PM William Davis < > > > > > > > > >>>>>>> william.j.dav...@gmail.com> > > > > > > > > >>>>>>>>>> wrote: > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>>> I've noticed that there are several Async > > > > implementations > > > > > > of > > > > > > > > >>>>>> standard > > > > > > > > >>>>>>>>>>> appenders out in the wild. Is there a reason none > > of > > > > > these > > > > > > > > >> have > > > > > > > > >>>>> made > > > > > > > > >>>>>>>>>> there > > > > > > > > >>>>>>>>>>> way into the core product? Is it just b/c no one > > has > > > > > taken > > > > > > > > >> the > > > > > > > > >>>>> time > > > > > > > > >>>>>>> to do > > > > > > > > >>>>>>>>>> a > > > > > > > > >>>>>>>>>>> pull request, or is there some other reason? > > > > > > > > >>>>>>>>>> I wonder, why one would create a special async > > > version, > > > > > when > > > > > > > > >> all > > > > > > > > >>>>> you > > > > > > > > >>>>>>> need > > > > > > > > >>>>>>>>>> to do is to put a standard async logger in front > of > > > the > > > > > sync > > > > > > > > >>>> logger > > > > > > > > >>>>>>> [1]? > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>> Jochen > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>> 1: https://logging.apache.org/ > > > > > log4j/2.x/manual/async.html# > > > > > > > > >>>>>>> MixedSync-Async > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> -- > > > > > > > > >>>>>> Matt Sicker <boa...@gmail.com> > > > > > > > > >>>>>> > > > > > > > > >>>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>>> -- > > > > > > > > >>>> Matt Sicker <boa...@gmail.com> > > > > > > > > >>>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> -- > > > > > > > > >>> Dominik Psenner > > > > > > > > >>> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> -- > > > > > > > > >> Dominik Psenner > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Matt Sicker <boa...@gmail.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Dominik Psenner > > > > > > > > > > > > > > > -- > > Dominik Psenner > > > -- Dominik Psenner
