AW: Ideas for a new RollingFileAppender
I agree. The user has to be aware of that special file but I wouldn't make
it switchable because it implicitly limits the available configurations in
other places and the user probably does not know. We have a similar magic
toggle in the current RFA which is named staticLogFileName. That toggle
disables rolling over date when set.
Still, optional configuration values are a matter of agreement and we can
find sensible ones by asking the user list with a poll. I've lately spent
some time thinking about how a configuration of the new RFA could look like
(in XML) and this is what my brain came up with:
appender type=RFA2 name=RFA2.mine
logfileFormatlogs/mylogfile.log/logfileFormat
rollingConfiguration
fileCacheFile${appender.name}.cache/fileCacheFile
lockingModel type=minimal /
rollingConditions
rollingCondition
type=startup / !- rolls files on startup; this is equivalent to the
behaviour when one configures appendToFile=false in the current RFA --!
rollingCondition
type=cron
pattern*/5
* * */pattern !-roll every 5 minutes --!
/rollingCondition
rollingCondition
type=cron
pattern0,5,10,15,20,25,30,35,40,45,50,55 * * */pattern !-alternative
but semantically equivalent pattern --!
/rollingCondition
rollingCondition
type=cron
!-yet
another alternative but semantically equivalent pattern --!
pattern
minute=*/5 hour=* day=* dow=* month=* /
/rollingCondition
rollingCondition
type=filesize
maxFileSize
unit=mb5/maxFileSize !-roll if filesize exceeds 5mb --!
/rollingCondition
/rollingConditions
rollingStrategy
type=defaultRollingStrategy
!-rolls a rolled file to the
specified file name format; ${datetime.original} would be the date/time when
the original logfile was first rolled --!
rolledLogfileFormatlogs/mylogfile-${datetime.original}-{$file.index}.log/
rolledLogfileFormat
/rollingStrategy
disposeConditions
disposeCondition
type=filecount
maxFileCount10/maxfileCount !-dispose rolled files if there are 10
newer files --!
/disposeCondition
disposeCondition
type=fileage
maxFileAge
unit=days30/maxFileAge
/disposeCondition
/disposeConditions
disposeStrategy type=moveFile
disposedLogfileFormatlogs/old/mylogfile-${datetime}-${file.index}.log/dis
posedLogfileFormat
/disposeStrategy
/rollingConfiguration
/appender
I would for instance default these properties like this:
fileCachefile: may always default to ${appender.name}.cache
lockingModel: may default to minimal
rollingConditions: may default to file size exceeds 1mb
disposeStrategy: may default to delete file
Cheers
Von: d_k [mailto:mail...@gmail.com]
Gesendet: Sonntag, 25. August 2013 18:41
An: Log4NET Dev
Betreff: Re: Ideas for a new RollingFileAppender
Hard tasks are fun, but I wouldn't want the next version to be stuck forever
because of it.
What if we make it optional?
If the user wishes to roll dynamic paths he should flip a switch and specify
a target path that should be writable.
I don't think you can make it totally headless because the user still needs
to be aware a file is written and he needs write permissions to it, so we
might as well make the default implementation work without creating files
and provide an implementation that persists the data locally.
So applications that upgrade to the new version behave the same.
On Fri, Aug 23, 2013 at 10:54 AM, Dominik Psenner dpsen...@gmail.com
mailto:dpsen...@gmail.com wrote:
Good Morning,
I understand your point. That would be exactly how the current RFA behaves.
However, people are lazy and when something doesn't work they are first
frustrated and then they may throw the framework over board or ask silly
questions. The latter happens about once every month. It is not nice to have
the answer ready this is not supported and it won't ever be because then
discussions
AW: Ideas for a new RollingFileAppender
Good Morning, I understand your point. That would be exactly how the current RFA behaves. However, people are lazy and when something doesn't work they are first frustrated and then they may throw the framework over board or ask silly questions. The latter happens about once every month. It is not nice to have the answer ready this is not supported and it won't ever be because then discussions start about the why-not and may-not-be or could-we-not. As I see it, the time has come that we should provide a solution that works. There are times when documentation does solve an issue but this time it is not. Of course this is not an easy task, but easy tasks are annoying anyway. Hard tasks make the life tasty, don't they? :) Cheers Von: d_k [mailto:mail...@gmail.com] Gesendet: Dienstag, 20. August 2013 21:16 An: Log4NET Dev Betreff: Re: Ideas for a new RollingFileAppender Will it be reasonable to say that as long as the target folder path is static log4net will clean up old files and if the target folder path is dynamic it will clean them as long as the process is up? On Thu, Aug 15, 2013 at 4:44 PM, Stefan Bodewig bode...@apache.org mailto:bode...@apache.org wrote: On 2013-08-14, Dominik Psenner wrote: 2013/8/14 Stefan Bodewig bode...@apache.org mailto:bode...@apache.org The application runs for 3 days and thus log4net rolls the logfile 3 times. Well, I live in a world where all log files of an application are kept inside the same folder :-) For me too, but I've seen enough to know better. Might be a matter of educating the culprits. Yes, I know things like this happen. More seriously, to me it sounds like a bit of feature creep. I wouldn't consider my logging framework to be the one who's responsible for mailing non empty files on rolling. That's the job of logrotate/logwatch or similar specialiced tools IMHO. One could even argue disposal of logfiles should be out of scope. Maybe that's just me. Yes and no. The framework should at least be able to prevent filesystem pollution by itself when told to clean up after itself. I'm not sure I agree with that. But arguing against that is almost arguing against having a RollingFileAppender in the first place. So I'll give in. Stefan
AW: Ideas for a new RollingFileAppender
Another issue that I just remembered now is that currently there's no way
to
roll empty files on date/time boundaries. Rolling only happens when a log
event is generated. If there's no log event, files won't be rolled. Thus
we
should investigate if we should implement the appender as a buffering
async
appender that does the rolling in his async work cycles and whose work
cycles do not depend on the availability of log events.
Making the appender async is a major change and it has ramifications that
might prove problematic like unflushed buffers.
And if we make the buffer of size 1 then its really synchronous again, isn't
it?
Does a 'roll' create a new file? So if there are no log event the folder
will be polluted with empty files?
If a 'roll' does not create a file then we can make the decision to roll or
not as part of the logging process.
Whats wrong with rolling on logging events?
Futher, the rolling file appender should probably receive a persistent
storage where he can keep a set of rolled files he has rolled in the past.
This way we eliminate the dark magic behind detecting rolled files. It
allows the user also to modify or mix the rolling strategies and/or
rolling
filename configuration from one instance to the next. The persistence will
have to include a set of filenames. It would probably be nice to allow
people configuring a relative filepath that will be stripped from the
filenames. Otherwise we would break the usecase where a user moves an
application from, let's say, drive C: to drive D: along with all logfiles
because the rolled filenames would no longer be found and some files would
become zombies that pollute the drive. Thus a persistable rolling history
class could look alike:
Public class RollingHistory {
Public Liststring Files { get; }
I'm up for a clean way to roll but I think that a logging framework that
creates files other than logs is odd. And adding a dependency on some
persistent storage provider is also odd.
I think we can assume that applications that need to perform rolling on size
or date are probably long running which means we can save this information
in memory and in case the application crashed or restarts then yes, the old
files will remain on disk and it should be solved by a local policy because
it the exception, not the rule.
To exchange information between the rolling strategies, rolling conditions
and the rolling file appender itself we should probably use a data class.
The name of it could be RollingContext and basically could be
implemented
like:
public class RollingContext {
public RollingHistory RollingHistory { get; }
public string Logfile { get; }
public long CurrentLogfileSize { get; }
That means that a rolling strategy could be an interface like:
public interface RollingStrategy {
public void DoRolling(RollingContext context);
which either throws exceptions on failure or returns true|false to notify
the caller that something went wrong and log the errors into the log4net
internal debug log.
The next topic is the rolling condition. The rolling condition decides if
rolling should be done right now. Thus its interface should be something
like:
public interface IRollingCondition {
public bool IsRollingConditionMet(RollingContext context);
Concrete rolling conditions could be:
* Roll if filesize exceeds a specific limit
* Roll if a specific date/time condition is met
== this is already kind of supported with an implementation of a
cron-like syntax
Making the rolling condition pluggable would greatly benefit others to
invent their own rolling conditions. For example an implementation of
IRollingCondition like:
public class ManualRollingCondition implements IRollingCondition {
public static bool RollOnNextCycle = false;
public bool IsRollingConditionMet(RollingContext context) {
if(RollOnNextCycle) {
// reset roll on next cycle
RollOnNextCycle = false;
return true;
}
return false;
could let the application that uses log4net decide when a file should be
rolled by a simple invoke to:
ManualRollingCondition.RollOnNextCycle=true;
And now we come to the disposal of old files. Since we have a persistent
set
of old files we should pass it into the disposing logic and the interface
to
it could look like:
Public interface IDisposeStrategy {
Public bool DoDisposal(RollingContext context);
And a concrete implementation of it could be:
Public class LimitNumberOfFilesDisposeStrategy implements IDisposeStrategy
{
Public int MaxNumberOfFiles { get; set; }
Public bool DoDisposal(RollingContext context) {
// delete all files in history
While(MaxNumberOfFiles Context.RollingHistory.Count) {
If(Context.RollingHistory.Count == 0)
Return true;
// delete oldest file and persist the history
// let it
AW: Ideas for a new RollingFileAppender
[good idea :-D]
-Ursprüngliche Nachricht-
Von: Stefan Bodewig [mailto:bode...@apache.org]
Gesendet: Montag, 12. August 2013 10:07
An: Log4NET Dev
Betreff: Ideas for a new RollingFileAppender
[just changing the subject :-)]
On 2013-08-12, Dominik Psenner wrote:
Yesterday it became quite late and I had not the mental force to write
down
my vision of the RFA reimplementation:
The log filename should be determined dynamically to allow users
including
information like %date, %time in the filename. It has often been
requested
and up to now it is supported only marginally.
Another issue that I just remembered now is that currently there's no way
to
roll empty files on date/time boundaries. Rolling only happens when a log
event is generated. If there's no log event, files won't be rolled. Thus
we
should investigate if we should implement the appender as a buffering
async
appender that does the rolling in his async work cycles and whose work
cycles do not depend on the availability of log events.
Futher, the rolling file appender should probably receive a persistent
storage where he can keep a set of rolled files he has rolled in the
past.
This way we eliminate the dark magic behind detecting rolled files. It
allows the user also to modify or mix the rolling strategies and/or
rolling
filename configuration from one instance to the next. The persistence
will
have to include a set of filenames. It would probably be nice to allow
people configuring a relative filepath that will be stripped from the
filenames. Otherwise we would break the usecase where a user moves an
application from, let's say, drive C: to drive D: along with all logfiles
because the rolled filenames would no longer be found and some files
would
become zombies that pollute the drive. Thus a persistable rolling history
class could look alike:
Public class RollingHistory {
Public Liststring Files { get; }
To exchange information between the rolling strategies, rolling
conditions
and the rolling file appender itself we should probably use a data class.
The name of it could be RollingContext and basically could be
implemented
like:
public class RollingContext {
public RollingHistory RollingHistory { get; }
public string Logfile { get; }
public long CurrentLogfileSize { get; }
That means that a rolling strategy could be an interface like:
public interface RollingStrategy {
public void DoRolling(RollingContext context);
which either throws exceptions on failure or returns true|false to notify
the caller that something went wrong and log the errors into the log4net
internal debug log.
The next topic is the rolling condition. The rolling condition decides if
rolling should be done right now. Thus its interface should be something
like:
public interface IRollingCondition {
public bool IsRollingConditionMet(RollingContext context);
Concrete rolling conditions could be:
* Roll if filesize exceeds a specific limit
* Roll if a specific date/time condition is met
== this is already kind of supported with an implementation of a
cron-like syntax
Making the rolling condition pluggable would greatly benefit others to
invent their own rolling conditions. For example an implementation of
IRollingCondition like:
public class ManualRollingCondition implements IRollingCondition {
public static bool RollOnNextCycle = false;
public bool IsRollingConditionMet(RollingContext context) {
if(RollOnNextCycle) {
// reset roll on next cycle
RollOnNextCycle = false;
return true;
}
return false;
could let the application that uses log4net decide when a file should be
rolled by a simple invoke to:
ManualRollingCondition.RollOnNextCycle=true;
And now we come to the disposal of old files. Since we have a persistent
set
of old files we should pass it into the disposing logic and the interface
to
it could look like:
Public interface IDisposeStrategy {
Public bool DoDisposal(RollingContext context);
And a concrete implementation of it could be:
Public class LimitNumberOfFilesDisposeStrategy implements
IDisposeStrategy {
Public int MaxNumberOfFiles { get; set; }
Public bool DoDisposal(RollingContext context) {
// delete all files in history
While(MaxNumberOfFiles Context.RollingHistory.Count) {
If(Context.RollingHistory.Count == 0)
Return true;
// delete oldest file and persist the history
// let it crash if someone has a logfile open and locked
// and let the invoker decide if he wants to retry later
Try {
Delete(Context.RollingHistory.Oldest());
Context.RollingHistory.Remove(Context.RollingHistory.Oldest());
Context.RollingHistory.Persist();
}
Catch
// internal logging
Return false;
}
