Sure, why not? Ralph
> On Nov 16, 2017, at 11:24 AM, Chandra <[email protected]> > wrote: > > with-in log4j? > > On 16 Nov 2017, 11:46 PM +0530, Ralph Goers <[email protected]>, > wrote: >> I could see forking a process instead of spawning a thread to perform the >> compression as a viable approach. >> >> Ralph >> >>> On Nov 16, 2017, at 10:01 AM, Chandra <[email protected]> >>> wrote: >>> >>> thanks for info matt, was planning to use this on some other project in >>> mind ( might pick your brain on that later ;) ) >>> >>> separating it out the compression to external process isn’t necessarily a >>> bad idea, but having non-reliable scripts is. As it so happened many times >>> before. I’d rather depend on my app than an external process. >>> which is why I was looking for an “agent” . >>> >>> thanks! >>> chandra >>> >>> >>> On 16 Nov 2017, 9:57 PM +0530, Matt Sicker <[email protected]>, wrote: >>>> I brought up Snappy only because I used their off-heap API recently. Snappy >>>> is more about real time compression rather than size (I think snappy files >>>> tend to be larger than gzip files, but take less resources to compress and >>>> decompress). The idea here is to offer support via libraries using native >>>> implementations that can work with direct byte buffers, mmap'd files, or >>>> even just a file name. >>>> >>>> With that in mind, is it so bad to offer the ability to execute an external >>>> process to compress the file? >>>> >>>> On 16 November 2017 at 09:59, Chandra <[email protected] >>>> wrote: >>>> >>>>>> What if compression worked off-heap >>>>> off-heap compression sounds interesting. Let me check if I can find any. >>>>> >>>>> Snappy’s compression is a different altogether, I am not necessarily >>>>> looking for a different compression formats, as I’d have add support for >>>>> it >>>>> in downstream. Standard bz2 , gzip would work in-fact. >>>>> >>>>> >>>>> Ideally a reliable agent something like `FileBeat` would be great for this >>>>> situation. :-/ >>>>> >>>>> Best, >>>>> Chandra >>>>> >>>>> >>>>> On 16 Nov 2017, 9:08 PM +0530, Matt Sicker <[email protected]>, wrote: >>>>>> What if compression worked off-heap like with some of the native >>>>>> implementations of codecs? I'm thinking of this one < >>>>>> https://github.com/xerial/snappy-java> for example. >>>>>> >>>>>> On 15 November 2017 at 23:41, Chandra <chandra.tungathurthi@rwth- >>>>> aachen.de >>>>>> wrote: >>>>>> >>>>>>> Guys, >>>>>>> >>>>>>> I need some input on how this handle situation: >>>>>>> >>>>>>> we are on HP/LL setting where the incoming requests are processed and >>>>>>> logged ( buffered, of course with “async logging”). There are some >>>>>>> situations where due to spikes in the volume of requests, the >>>>> compression >>>>>>> on rolling creates memory starvation. >>>>>>> >>>>>>> Now, a straight forward fix would to remove it from the “context” of >>>>> jvm >>>>>>> and use a script to monitor and compress it timely. as low-hanging the >>>>>>> solution may be, I am skeptical of this solution as the script _may_ >>>>> fail >>>>>>> leading to disk starvation(!) >>>>>>> >>>>>>> I am looking for an alternate and _full-proof_ solution for this >>>>>>> situation. Any thoughts, suggestions would be useful and appreciated. >>>>>>> >>>>>>> thanks! >>>>>>> Chandra >>>>>>> >>>>>>> On 16 Nov 2017, 10:01 AM +0530, Ralph Goers < >>>>> [email protected]>, >>>>>>> wrote: >>>>>>>> Unless someone objects I plan to start the release process for Log4j >>>>>>> 2.10 tomorrow. I had wanted to include a fix for LOG4J2-2106 but the >>>>>>> problem only occurs in rare situations and I am not sure how to fix it >>>>> yet. >>>>>>> There are a lot of other issues that deserve looking at but nothing I >>>>> can >>>>>>> see that warrants waiting on. >>>>>>>> >>>>>>>> Ralph >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Matt Sicker <[email protected] >>>>> >>>> >>>> >>>> >>>> -- >>>> Matt Sicker <[email protected] >> >>
