RE: Logging help
>Oh hai - as I was reading the documentation, look what I found: > >http://docs.python.org/2/library/logging.html#filter > >Methinks that should do exactly what you want. Hi Wayne, I was too hasty when I looked at filters as I didn't think they could do what I wanted. Turns out a logging object sent the filter method will an exc_info attribute in my case. Thanks! jlc -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Thu, 1 Aug 2013, Joseph L. Casale wrote:
I have a couple handlers applied to a logger for a file and console destination.
Default levels have been set for each, INFO+ to console and anything to file.
How does one prevent logging.exception from going to a specific handler when
it falls within the desired levels?
There is probably a better way, I'm not too familiar with the more
advanced logging, but off the top of my head I'd suggest subclassing the
Handler classes that you're interested in and overriding the Handle method
(I think - you can check the source to be sure) and if the type of message
is exception then just skip handling it.
Alternatively, you could simply create a different logger, e.g.
exc_logger = logging.getLogger('mything.exceptions')
And use that to log exceptions.
Oh hai - as I was reading the documentation, look what I found:
http://docs.python.org/2/library/logging.html#filter
Methinks that should do exactly what you want.
HTH,
W
--
http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
In article , koranthala wrote: > Is it possible somehow to have the logging module rotate the files >every time I start it. If you're on Linux, why not use logrotate? -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 22, 10:25 pm, Vinay Sajip wrote: > On Jan 22, 3:40 pm,koranthala wrote: > > > Thank you very much Vinay. You have been extremely helpful. > > This was my first design - but then I found that log system was taking > > up quite a huge chunk of the memory. > > That is why I went to rotating file handler. > > Using just plain FileHandler takes up less memory than using > RotatingFileHandler (because using the latter imports the "handlers" > module into memory, whereas using the former does not). I'm not aware > of any problem where logging takes up a huge chunk of memory (under > normal usage) - how did you measure the memory usage which was due to > logging? > > > Anyways, now I have modified doRollover to append if file is there, > > so, for me it is working. > > What I was thinking was such an option in the basicloggingsystem > > might be of good help. > > The solution I suggested is IMO better than using a patched > RotatiingFileHandler, both because it avoids importing an extra module > (if memory really is a concern) and because you don't need to change > your code across Python version updates. How does my suggestion to use > FileHandler not meet your use case? > > > Again, Vinay, Thank you very much. You were extremely helpful. > > You're welcome. > > Best wishes, > > Vinay Sajip oops - again I was very unclear in my post. What i meant was that due to the log getting very big - i.e. the log was in 100MB range (since it is a new product - a lot of debugging is done via logs). Not RAM. Having a rotating file handler helped there - because I could say only 5 such logs are needed. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 22, 3:40 pm, koranthala wrote: > Thank you very much Vinay. You have been extremely helpful. > This was my first design - but then I found that log system was taking > up quite a huge chunk of the memory. > That is why I went to rotating file handler. Using just plain FileHandler takes up less memory than using RotatingFileHandler (because using the latter imports the "handlers" module into memory, whereas using the former does not). I'm not aware of any problem where logging takes up a huge chunk of memory (under normal usage) - how did you measure the memory usage which was due to logging? > Anyways, now I have modified doRollover to append if file is there, > so, for me it is working. > What I was thinking was such an option in the basicloggingsystem > might be of good help. The solution I suggested is IMO better than using a patched RotatiingFileHandler, both because it avoids importing an extra module (if memory really is a concern) and because you don't need to change your code across Python version updates. How does my suggestion to use FileHandler not meet your use case? > Again, Vinay, Thank you very much. You were extremely helpful. You're welcome. Best wishes, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 22, 2:14 pm, Vinay Sajip wrote:
> On Jan 22, 6:49 am,koranthala wrote:
>
> > I understand Vinay. But my point is that I wanted a mechanism to
> > rotate the log files based on data - i.e. today one log, tomorrow
>
> Did you mean "based on date"?
>
> > another. This is easier because when trouble tickets are raised, users
> > mention that X failed at this time.
> > Now, timedrotatingfilehandler does it - but only if the program is
> > running the whole length of time.
> > My tool is invoked by a cron job - so the program runs, then stops
> > again and again.
>
> If you just want a log file whose name is date-based, you don't need a
> rotating file handler. Compute the file name from the date and use the
> API to create a FileHandler specifying that file name, and add it to
> your logger. For example:
>
> import logging, time
>
> logging.basicConfig(level=logging.DEBUG, filename=time.strftime("/path/
> to/my/logs/myapp-%Y-%m-%d-%H%M.log", time.localtime()), filemode="w")
> logging.debug("Hello, world!")
>
> Hopefully you can adapt this snippet to your needs.
>
> Regards,
>
> Vinay Sajip
Thank you very much Vinay. You have been extremely helpful.
This was my first design - but then I found that log system was taking
up quite a huge chunk of the memory.
That is why I went to rotating file handler.
Anyways, now I have modified doRollover to append if file is there,
so, for me it is working.
What I was thinking was such an option in the basic logging system
might be of good help.
Again, Vinay, Thank you very much. You were extremely helpful.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 22, 6:49 am, koranthala wrote:
> I understand Vinay. But my point is that I wanted a mechanism to
> rotate the log files based on data - i.e. today one log, tomorrow
Did you mean "based on date"?
> another. This is easier because when trouble tickets are raised, users
> mention that X failed at this time.
> Now, timedrotatingfilehandler does it - but only if the program is
> running the whole length of time.
> My tool is invoked by a cron job - so the program runs, then stops
> again and again.
If you just want a log file whose name is date-based, you don't need a
rotating file handler. Compute the file name from the date and use the
API to create a FileHandler specifying that file name, and add it to
your logger. For example:
import logging, time
logging.basicConfig(level=logging.DEBUG, filename=time.strftime("/path/
to/my/logs/myapp-%Y-%m-%d-%H%M.log", time.localtime()), filemode="w")
logging.debug("Hello, world!")
Hopefully you can adapt this snippet to your needs.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 21, 2:55 pm, Vinay Sajip wrote: > On Jan 20, 10:11 am,koranthala wrote: > > > > > The doRollover method does not append to the earlier file. Rather, it > > creates a new file with the same name. > > Err... that's what rollover means - switching to a new log file (and > renaming the old ones). If it just appended to the old one, why would > it be called doRollover ? ;-) > > Regards, > > Vinay Sajip I understand Vinay. But my point is that I wanted a mechanism to rotate the log files based on data - i.e. today one log, tomorrow another. This is easier because when trouble tickets are raised, users mention that X failed at this time. Now, timedrotatingfilehandler does it - but only if the program is running the whole length of time. My tool is invoked by a cron job - so the program runs, then stops again and again. When I posted the question here, I was forwarded to the doRollover mechanism as a solution. I was just mentioning that doRollover, due to overwriting the files I would lose everything that is stored before. What I did now was to copy the file, and then append to the rolledover file - which does serve the purpose. But it is kludgy, and I dont think my situation is unique. I did see other people asking for the same functionality. So I was wondering whether it would be good idea to have this also in the next version. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 20, 10:11 am, koranthala wrote: > > The doRollover method does not append to the earlier file. Rather, it > creates a new file with the same name. Err... that's what rollover means - switching to a new log file (and renaming the old ones). If it just appended to the old one, why would it be called doRollover ? ;-) Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
En Tue, 20 Jan 2009 08:11:52 -0200, koranthala escribió: On Jan 20, 5:45 am, Chris Rebert wrote: On Mon, Jan 19, 2009 at 11:36 AM, koranthala wrote: > Is it possible somehow to have the logging module rotate the files > every time I start it. > Basically, I can automatically rotate using RotatingFileHandler; > ^^^ > Now I want it rotated every time I start the program too. Just call the .doRollover() method of the RotatingFileHandler at the ^^^ start of the program. Reading The Fine Documentation for the module is helpful. Current doRollover method is not very helpful to rolling over every day if the process starts and stops many times. For example: TimedRotatingFileHandler - when='D' Interval=1. I call handler.doRollover everytime process starts. Why the move to TimedRotatingFileHandler? -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
En Tue, 20 Jan 2009 08:11:52 -0200, koranthala escribió: On Jan 20, 5:45 am, Chris Rebert wrote: On Mon, Jan 19, 2009 at 11:36 AM, koranthala wrote: > Is it possible somehow to have the logging module rotate the files > every time I start it. > Basically, I can automatically rotate using RotatingFileHandler; > ^^^ > Now I want it rotated every time I start the program too. Just call the .doRollover() method of the RotatingFileHandler at the ^^^ start of the program. Reading The Fine Documentation for the module is helpful. Current doRollover method is not very helpful to rolling over every day if the process starts and stops many times. For example: TimedRotatingFileHandler - when='D' Interval=1. I call handler.doRollover everytime process starts. Why the move to TimedRotatingFileHandler? -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Jan 20, 5:45 am, Chris Rebert wrote: > On Mon, Jan 19, 2009 at 11:36 AM, koranthala wrote: > > Hi, > > Is it possible somehow to have the logging module rotate the files > > every time I start it. > > Basically, I can automatically rotate using RotatingFileHandler; > > Now I want it rotated every time I start the program too. > > Just call the .doRollover() method of the RotatingFileHandler at the > start of the program. > Reading The Fine Documentation for the module is helpful. > > Cheers, > Chris > > -- > Follow the path of the Iguana...http://rebertia.com My question was poorly framed. I will try to explain the issue a little more. Current doRollover method is not very helpful to rolling over every day if the process starts and stops many times. For example: TimedRotatingFileHandler - when='D' Interval=1. I call handler.doRollover everytime process starts. First run - the output files are log.txt -- Second run - files are log.txt, log.txt.2009-01-20 -- Till now, fine. Third run - files are log.txt, log.txt.2009-01-20 ***But the earlier file is overwritten*** The doRollover method does not append to the earlier file. Rather, it creates a new file with the same name. Due to this the earlier logs are all gone. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging help
On Mon, Jan 19, 2009 at 11:36 AM, koranthala wrote: > Hi, > Is it possible somehow to have the logging module rotate the files > every time I start it. > Basically, I can automatically rotate using RotatingFileHandler; > Now I want it rotated every time I start the program too. Just call the .doRollover() method of the RotatingFileHandler at the start of the program. Reading The Fine Documentation for the module is helpful. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
