That's probably why I wasn't able to figure out how you can do this with CronTrigerringPolicy :) Do you think that adding this random support in cron expression is a better solution ?
Regards, Anthony 2017-03-24 18:47 GMT+01:00 Ralph Goers <ralph.go...@dslextreme.com>: > I thought CronTriggeringPolicy would handle this but it is based on > Quartz’ CronExpression which doesn’t support randomizing fields. I was > confused because Jenkins supports this and I had thought it was using > Quartz. Apparently not. > > Ralph > > > > On Mar 24, 2017, at 10:15 AM, Anthony Maire <maire.anth...@gmail.com> > wrote: > > > > Basically we are hosting distributed systems for our clients. So we have > > several dozens of JVM per physical host, and several dozens of physical > > hosts. Moreover some clients are sometimes moved from one machine to > > another so having a single configuration file is a very huge requirement. > > > > Having the file that rolls precisely at midnight is a nice to have. > Having > > the rolling that happen a little later because of randomization can be > > accepted if it solve the performance issue (which stay pretty short, it's > > about 15 seconds during which we can experience a few hundreds of > > milliseconds long stalls but this is already too much in our industry ) > > > > Randomization is a common technique for that kind of issue, used in > > congestion avoidance in some network protocols for example (including > > Ethernet ). Having 2 or 3 jvm that roll at the same time is not a big > deal, > > we have enough free cpu cycles for that. That's why randomization during > a > > few minutes seems OK to me : we don't need to annihilate collisions, > > greatly reduce them is OK > > > > Basically I'm open to any solution as long as it can be deployed > everywhere > > without any OS/shell dependant trick and it does not involve any kind of > > per JVM configuration. The patch I submitted is an option, but I'd be > glad > > to use another option that match these requirements. > > > > Regards, > > Anthony > > > > > > Le 24 mars 2017 15:20, "Remko Popma" <remko.po...@gmail.com> a écrit : > > > > Actually, there is already something like this because Log4j2 > configuration > > has some support for scripting. I would prefer to enhance that over > > building conditional constructs in the configuration. > > > > However, having to specify the exact rollover time for individual servers > > would result in a much more lengthy and involved configuration. > > Randomization is a commonly used technique to get roughly the same result > > without the inconvenience of precise configuration. I don't think it is > > such a bad idea. > > > > Sent from my iPhone > > > >> On Mar 24, 2017, at 19:35, Dominik Psenner <dpsen...@apache.org> wrote: > >> > >> What if a configuration could contain conditional statements? For > > instance: > >> > >> <Config> > >> <Define EnvironmentVariable="Hour" Value="12" /> > >> <Define EnvironmentVariable="Minute" Value="0" /> > >> <If EnvironmentVariable="JVM" Equals="JVM1"> > >> <Define EnvironmentVariable="Minute" Value="0" /> > >> </If> > >> <If EnvironmentVariable="JVM" Equals="JVM2"> > >> <Define EnvironmentVariable="Minute" Value="1" /> > >> </If> > >> <RollingFileAppender> > >> <RollingCondition> > >> <Cron Minute="$Minute" Hour="$Hour" DayOfMonth="*" Month="*" > > DayOfWeek="*" /> > >> </RollingCondition> > >> </RollingFileAppender> > >> </Config> > >> > >> Cheers > >> > >>> On 2017-03-24 11:08, Remko Popma wrote: > >>> I see what you are saying, but the use case is to have a single > > configuration, that is what drives the request. > >>> > >>> Sent from my iPhone > >>> > >>>> On Mar 24, 2017, at 17:22, Dominik Psenner <dpsen...@apache.org> > wrote: > >>>> > >>>> Hi there, > >>>> > >>>> Cron proved itself very to be stable and usable over the past years. > > FWIW, I would not recommend to introduce a randomization algorithm. An > > application that does things random means that the application does > things > > when nobody expects it to do so. Further it does not solve the problem. > One > > just needs enough JVMs to roll around at the same time with a randomizer > > does not produce large enough values to spread the rolling over a larger > > amount of time. The available CPU is then quickly drain out. In my > opinion, > > it's better to configure the several rollings to be delayed, meaning that > > JVM1 rolls at 12:00, JVM2 rolls at 12:01, ... > >>>> > >>>> Cheers, Dominik > >>>> > >>>>> On 2017-03-23 15:04, Anthony Maire wrote: > >>>>> https://issues.apache.org/jira/browse/LOG4J2-1855 > >>>>> https://github.com/apache/logging-log4j2/pull/68 > >>>>> > >>>>> Let me know if you want me to do some changes > >>>>> > >>>>> Regards, > >>>>> Anthony > >>>>> > >>>>> > >>>>> 2017-03-23 13:15 GMT+01:00 Anthony Maire <maire.anth...@gmail.com>: > >>>>> > >>>>>> Ok, I will open a jira ticket and provide a PR. > >>>>>> > >>>>>> Thanks for your input. > >>>>>> > >>>>>> Le 23 mars 2017 13:08, "Remko Popma" <remko.po...@gmail.com> a > écrit : > >>>>>> > >>>>>>> I see what you mean now. I agree it's better to keep the rollover > > concept > >>>>>>> to mean file rename and compression that happen in sequence > > together. So > >>>>>>> the randomization affects when the _sequence_ is triggered, not > just > > one > >>>>>>> part of the sequence. Makes sense. > >>>>>>> > >>>>>>> Sent from my iPhone > >>>>>>> > >>>>>>>> On Mar 23, 2017, at 16:28, Anthony Maire <maire.anth...@gmail.com > > > >>>>>>> wrote: > >>>>>>>> Hi Remko > >>>>>>>> > >>>>>>>> My first idea was to have the rolling that triggers at the > expected > >>>>>>> time, > >>>>>>>> and the compression that will be delayed. That's why I wanted the > >>>>>>> delayed > >>>>>>>> compression to occur before shutdown since the rolling already > > occurred. > >>>>>>>> > >>>>>>>> But I think that's a bad idea. First, it will lead to "fancy code" > > and I > >>>>>>>> would like to avoid it too. But the main issue is that this > behavior > >>>>>>> should > >>>>>>>> impact only the time based triggering when combining several > > policy. So > >>>>>>> the > >>>>>>>> code should be related to the triggering policy and not to the > > rolling > >>>>>>>> strategy. > >>>>>>>> > >>>>>>>> So the best thing to do is to add some property on the timed base > >>>>>>>> triggering policy and let that class handle all the logic and > delay > > the > >>>>>>>> triggering itself instead of the compression. > >>>>>>>> > >>>>>>>> Are you OK with that? > >>>>>>>> > >>>>>>>> Anthony > >>>>>>>> > >>>>>>>> Le 23 mars 2017 00:24, "Remko Popma" <remko.po...@gmail.com> a > > écrit : > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Mar 23, 2017, at 1:06, Anthony Maire <maire.anth...@gmail.com > > > >>>>>>> wrote: > >>>>>>>>> Thanks for these answers > >>>>>>>>> > >>>>>>>>> @Ralph : that was the kind of idea I had in mind : changing the > >>>>>>>>> RollingFileManager.asyncExecutor to be a > > ScheduledThreadPoolExecutor, > >>>>>>> and > >>>>>>>>> based on some configuration, submitting task to be executed > after a > >>>>>>> random > >>>>>>>>> delay. However with this kind of approach, special treatment > > should be > >>>>>>>> made > >>>>>>>>> if the manager is stopped with some pending delayed tasks in it. > >>>>>>>> I'm okay with randomization except for this last bit about > "special > >>>>>>>> treatment...". Let's not make it too fancy. If the manager is > > stopped > >>>>>>>> before it rolled over, then it didn't roll over, just like it > works > >>>>>>>> currently. I don't see the point of adding extra logic to trigger > a > >>>>>>>> rollover when the manager is stopped within the randomized time > > window. > >>>>>>>> > >>>>>>>>> @Matt : Cron policy can be a solution, but I don't know how to > > inject > >>>>>>> some > >>>>>>>>> random element in this to make the file roll at midnight + X > random > >>>>>>>>> seconds. Since there is a lot of JVM to manage and some of them > > can be > >>>>>>>>> moved from a machine to another, I need to have a single > log4j2.xml > >>>>>>> file > >>>>>>>>> for all environments. Moreover, our system administrators are > >>>>>>> reluctant to > >>>>>>>>> have something based on a shell-specific feature (such has the > > $RANDOM > >>>>>>>>> variable from bash) > >>>>>>>>> > >>>>>>>>> Anthony > >>>>>>>>> > >>>>>>>>> 2017-03-22 16:31 GMT+01:00 Ralph Goers < > ralph.go...@dslextreme.com > >> : > >>>>>>>>> > >>>>>>>>>> These are separate JVMs, so having a single executor would be of > > no > >>>>>>> help. > >>>>>>>>>> I believe the only way to do what you are asking for is to add > >>>>>>>>>> configuration so that the asynchronous thread has a semi-random > > delay > >>>>>>>> when > >>>>>>>>>> it starts. > >>>>>>>>>> > >>>>>>>>>> Ralph > >>>>>>>>>> > >>>>>>>>>>> On Mar 22, 2017, at 7:58 AM, Greg Thomas < > greg.d.tho...@gmail.com > >> > >>>>>>>>>> wrote: > >>>>>>>>>>>> One common issue we have with that framework (and I assume we > > can > >>>>>>> have > >>>>>>>>>> the > >>>>>>>>>>>> same with log4j2) is that all of our JVMs (we can have more > > than 50 > >>>>>>>>>> JVMs on > >>>>>>>>>>>> the same server in production) roll their file at midnight. > >>>>>>>>>>>> > >>>>>>>>>>>> When this happens, the system became often not usable for a > few > >>>>>>> seconds > >>>>>>>>>>>> because of the simultaneous zipping of all the rolled files > that > >>>>>>>>>> overload > >>>>>>>>>>>> the CPU (although zipping is done in a specific background > > thread). > >>>>>>>>>>> ISTR that with the most recent versions of log4j, these threads > > are > >>>>>>> in a > >>>>>>>>>>> thread pool so that they are properly shutdown at the right > > time. I > >>>>>>>>>> wonder > >>>>>>>>>>> if it's possible (or could be possible) to somehow inject a > > thread > >>>>>>> pool > >>>>>>>>>> in > >>>>>>>>>>> to log4j for this rollover, so that for you use case you could > >>>>>>> inject a > >>>>>>>>>>> single thread executor, so only one thread is ever compressing > > at a > >>>>>>>> time. > >>>>>>>>>>> Just a thought, anyway, > >>>>>>>>>>> > >>>>>>>>>>> Greg > >>>>>>>>>>> > >>>>>>>>>>> On 22 March 2017 at 13:47, Anthony Maire < > maire.anth...@gmail.com > >> > >>>>>>>>>> wrote: > >>>>>>>>>>>> Hi > >>>>>>>>>>>> > >>>>>>>>>>>> We are currently using another logging framework in > production, > > but > >>>>>>> I'm > >>>>>>>>>>>> pushing to change it for log4j2. > >>>>>>>>>>>> > >>>>>>>>>>>> One common issue we have with that framework (and I assume we > > can > >>>>>>> have > >>>>>>>>>> the > >>>>>>>>>>>> same with log4j2) is that all of our JVMs (we can have more > > than 50 > >>>>>>>>>> JVMs on > >>>>>>>>>>>> the same server in production) roll their file at midnight. > >>>>>>>>>>>> > >>>>>>>>>>>> When this happens, the system became often not usable for a > few > >>>>>>> seconds > >>>>>>>>>>>> because of the simultaneous zipping of all the rolled files > that > >>>>>>>>>> overload > >>>>>>>>>>>> the CPU (although zipping is done in a specific background > > thread). > >>>>>>> To > >>>>>>>>>>>> reduce this effect, we are combining a time based rolling > policy > >>>>>>> with a > >>>>>>>>>>>> sized based policy to zip smaller files, but this is not > enough > > to > >>>>>>> make > >>>>>>>>>> the > >>>>>>>>>>>> system fully responsive at midnight. > >>>>>>>>>>>> > >>>>>>>>>>>> A pretty cool feature for us to avoid this issue is to have > the > >>>>>>>>>> possibility > >>>>>>>>>>>> when a rolling is triggered because of a time based policy to > > change > >>>>>>>>>> file > >>>>>>>>>>>> immediately, but to wait for a random amount of time (within a > >>>>>>>>>> configurable > >>>>>>>>>>>> limit) before starting the compression. This random delay > should > >>>>>>> help a > >>>>>>>>>> lot > >>>>>>>>>>>> to avoid contention on CPU cycles. > >>>>>>>>>>>> > >>>>>>>>>>>> Does log4j2 have something to solve this kind of issue ? If > not, > >>>>>>> would > >>>>>>>>>> you > >>>>>>>>>>>> accept a pull request for this (I will open a Jira if needed) > ? > >>>>>>>>>>>> > >>>>>>>>>>>> Regards, > >>>>>>>>>>>> Anthony > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> ------------------------------------------------------------ > > --------- > >>>>>>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@ > logging.apache.org > >>>>>>>>>> For additional commands, e-mail: log4j-user-help@logging. > > apache.org > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> ------------------------------------------------------------ > > --------- > >>>>>>>> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >>>>>>>> For additional commands, e-mail: log4j-user-help@logging. > apache.org > >>>>>>> ------------------------------------------------------------ > --------- > >>>>>>> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >>>>>>> For additional commands, e-mail: log4j-user-help@logging. > apache.org > >>>>>>> > >>>>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >>>> For additional commands, e-mail: log4j-user-h...@logging.apache.org > >>>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >>> For additional commands, e-mail: log4j-user-h...@logging.apache.org > >>> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >> For additional commands, e-mail: log4j-user-h...@logging.apache.org > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > For additional commands, e-mail: log4j-user-h...@logging.apache.org > >
