I should have used “spawn” instead of “fork”. Ralph
> On Nov 16, 2017, at 11:34 AM, Matt Sicker <[email protected]> wrote: > > I don't think Java supports process forking (not even sure if Windows > does), but you can execute and control processes at least. > > On 16 November 2017 at 12:28, Ralph Goers <[email protected]> > wrote: > >> Sure, why not? >> >> Ralph >> >>> On Nov 16, 2017, at 11:24 AM, Chandra <chandra.tungathurthi@rwth- >> aachen.de> wrote: >>> >>> with-in log4j? >>> >>> On 16 Nov 2017, 11:46 PM +0530, Ralph Goers <[email protected]>, >> wrote: >>>> I could see forking a process instead of spawning a thread to perform >> the compression as a viable approach. >>>> >>>> Ralph >>>> >>>>> On Nov 16, 2017, at 10:01 AM, Chandra <chandra.tungathurthi@rwth- >> aachen.de> wrote: >>>>> >>>>> thanks for info matt, was planning to use this on some other project >> in mind ( might pick your brain on that later ;) ) >>>>> >>>>> separating it out the compression to external process isn’t >> necessarily a bad idea, but having non-reliable scripts is. As it so >> happened many times before. I’d rather depend on my app than an external >> process. >>>>> which is why I was looking for an “agent” . >>>>> >>>>> thanks! >>>>> chandra >>>>> >>>>> >>>>> On 16 Nov 2017, 9:57 PM +0530, Matt Sicker <[email protected]>, wrote: >>>>>> I brought up Snappy only because I used their off-heap API recently. >> Snappy >>>>>> is more about real time compression rather than size (I think snappy >> files >>>>>> tend to be larger than gzip files, but take less resources to >> compress and >>>>>> decompress). The idea here is to offer support via libraries using >> native >>>>>> implementations that can work with direct byte buffers, mmap'd files, >> or >>>>>> even just a file name. >>>>>> >>>>>> With that in mind, is it so bad to offer the ability to execute an >> external >>>>>> process to compress the file? >>>>>> >>>>>> On 16 November 2017 at 09:59, Chandra <chandra.tungathurthi@rwth- >> aachen.de >>>>>> wrote: >>>>>> >>>>>>>> What if compression worked off-heap >>>>>>> off-heap compression sounds interesting. Let me check if I can find >> any. >>>>>>> >>>>>>> Snappy’s compression is a different altogether, I am not necessarily >>>>>>> looking for a different compression formats, as I’d have add support >> for it >>>>>>> in downstream. Standard bz2 , gzip would work in-fact. >>>>>>> >>>>>>> >>>>>>> Ideally a reliable agent something like `FileBeat` would be great >> for this >>>>>>> situation. :-/ >>>>>>> >>>>>>> Best, >>>>>>> Chandra >>>>>>> >>>>>>> >>>>>>> On 16 Nov 2017, 9:08 PM +0530, Matt Sicker <[email protected]>, >> wrote: >>>>>>>> What if compression worked off-heap like with some of the native >>>>>>>> implementations of codecs? I'm thinking of this one < >>>>>>>> https://github.com/xerial/snappy-java> for example. >>>>>>>> >>>>>>>> On 15 November 2017 at 23:41, Chandra <chandra.tungathurthi@rwth- >>>>>>> aachen.de >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Guys, >>>>>>>>> >>>>>>>>> I need some input on how this handle situation: >>>>>>>>> >>>>>>>>> we are on HP/LL setting where the incoming requests are processed >> and >>>>>>>>> logged ( buffered, of course with “async logging”). There are some >>>>>>>>> situations where due to spikes in the volume of requests, the >>>>>>> compression >>>>>>>>> on rolling creates memory starvation. >>>>>>>>> >>>>>>>>> Now, a straight forward fix would to remove it from the “context” >> of >>>>>>> jvm >>>>>>>>> and use a script to monitor and compress it timely. as low-hanging >> the >>>>>>>>> solution may be, I am skeptical of this solution as the script >> _may_ >>>>>>> fail >>>>>>>>> leading to disk starvation(!) >>>>>>>>> >>>>>>>>> I am looking for an alternate and _full-proof_ solution for this >>>>>>>>> situation. Any thoughts, suggestions would be useful and >> appreciated. >>>>>>>>> >>>>>>>>> thanks! >>>>>>>>> Chandra >>>>>>>>> >>>>>>>>> On 16 Nov 2017, 10:01 AM +0530, Ralph Goers < >>>>>>> [email protected]>, >>>>>>>>> wrote: >>>>>>>>>> Unless someone objects I plan to start the release process for >> Log4j >>>>>>>>> 2.10 tomorrow. I had wanted to include a fix for LOG4J2-2106 but >> the >>>>>>>>> problem only occurs in rare situations and I am not sure how to >> fix it >>>>>>> yet. >>>>>>>>> There are a lot of other issues that deserve looking at but >> nothing I >>>>>>> can >>>>>>>>> see that warrants waiting on. >>>>>>>>>> >>>>>>>>>> Ralph >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Matt Sicker <[email protected] >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Matt Sicker <[email protected] >>>> >>>> >> >> >> > > > -- > Matt Sicker <[email protected]>
