[issue13516] Gzip old log files in rotating handlers

[Mark Grandi]

Mark Grandi added the comment:

While I will say that is slightly easier than subclassing, it still requires a 
function, or a variation of that to be present in every project that wants to 
have a RotatingHandler + compression, and extra lines if you use 
logging.config.dictConfig, as you will need to reference an external function 
rather than just specifying the filename = *.gz

I don't see this this simple feature would add much maintainer overhead, as 
lzma.open, bz2.open and gz.open are all pretty simple to use and are all 
already in the stdlib

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip added the comment:

You don't need a subclass - you can specify your own function to do it, as in 
the cookbook example:

https://docs.python.org/3/howto/logging-cookbook.html#using-a-rotator-and-namer-to-customize-log-rotation-processing

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Mark Grandi]

Mark Grandi added the comment:

I just ran into this myself, and would challenge the notion that just because 
few people complain about the fact that built in compression isn't built in for 
logging handlers (such as TimedRotatingFileHandler), that it isn't a needed 
feature. This is such a common feature in pretty much everything that it would 
be nice to have it built in rather than having a custom RotatingHandler 
subclass that just compresses the logs and does nothing else in every project I 
work with.

While not python, i searched github for logback/slf4j (java logging framework) 
entries for their equivilent of TimedRotatingFileHandler and I found quite a 
few results:

https://github.com/search?utf8=%E2%9C%93=fileNamePattern%2F%3E*.gz+language%3AXML=Code=searchresults

you don't even have to change the API of it, it could act the same way as the 
tarfile module, where if the 'filename' argument ends in one of "bz2", "gz" or 
'xz" then it is compressed with that compression type, and if not, then it 
doesn't compress it (what it does now)

and, that is how logback/slf4j does it as well 
(http://logback.qos.ch/manual/appenders.html#fwrpFileNamePattern)

--
nosy: +markgrandi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

Refactoring in 57295c4d81ac supports this use case.

--
resolution:  - fixed
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Raul Morales]

Raul Morales raul...@gmail.com added the comment:

I have just posted a comment, too.

http://plumberjack.blogspot.com/2011/12/improved-flexibility-for-log-file.html?showComment=1323891345946#c2875224484376643310

With this approach, anyone can implement support for any format easily. It is 
powerful. But IMHO, built-in support for Gzip and Zip formats would cover the 
basic needs of almost all platforms where python run.

Mix-in classes can be implemented in just a few lines of code, and it matches 
with your idea. Of course, IMHO.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

It seems that you agree that the basic mechanism is good enough to build on, so 
I'd rather leave the proposed stdlib change as is, but provide examples of how 
to achieve gzip/zip compression for rotated logs in the Logging Cookbook. The 
reason I don't want to add these in the stdlib is that I have to support 
anything I add indefinitely, so I don't want to add anything which will not be 
often used. This is the first time the issue has come up in all the years since 
the rotating file handlers have been around, so while I want to support your 
use case, I want to minimise stdlib changes and additions as much as I can.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Raul Morales]

Raul Morales raul...@gmail.com added the comment:

Ok, it is reasonable. It has no sense add support for compression since I am 
the only user who want it.

Maybe in the future ;)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

See this for the proposed resolution:

http://plumberjack.blogspot.com/2011/12/improved-flexibility-for-log-file.html

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Raul Morales]

Raul Morales raul...@gmail.com added the comment:

I use a similar code in my scripts, but I thought it could be useful to have 
this feature built into python.

If you prefer subclassing for compression, what about a compressing subclass 
built into logging package?

If you think it is a good feature, I will be able to work on it next week, and 
to add support for other formats.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

I have worked out a possible approach. I will post about it soon, and add a 
link to it from this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Raul Morales]

Raul Morales raul...@gmail.com added the comment:

Interesting, then I will wait your post. Thanks.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

Some people might want other compression methods, e.g. bz2, zip, lzma ...

Have you considered subclassing the existing handler classes as in the 
following example?

http://code.activestate.com/recipes/502265-timedcompressedrotatingfilehandler/

Possibly I could change the implementation in 3.3 to make this easier to do ... 
I will give it some thought.

--
assignee:  - vinay.sajip

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13516] Gzip old log files in rotating handlers

[Raul Morales]

New submission from Raul Morales raul...@gmail.com:

Sometimes log files grow very quickly and consume too much disk space (e.g. 
DEBUG), so compress old log files saves disk space without losing the 
information from log files.

I propose to add a gzip or compress argument to RotatingFileHandler and 
TimedRotatingFileHandler to select the number of old files you want to compress.

For example, if you set the argument to 0 (default) no files are gzipped , but 
if you set it to 3 you get the following log files:
app.log
app.log.1
app.log.2
app.log.3.gz
app.log.4.gz
...
app.log.n.gz

For TimedRotatingFileHandler it works similar, gzipping from the n-th newer log 
file to the oldest log file:
app.log
app.log.2011-12-01
app.log.2011-11-30
app.log.2011-11-29.gz
app.log.2011-11-28.gz
...

A possible patch is attached

--
components: Library (Lib)
files: log.patch
keywords: patch
messages: 148732
nosy: ramhux, vinay.sajip
priority: normal
severity: normal
status: open
title: Gzip old log files in rotating handlers
type: feature request
versions: Python 3.3
Added file: http://bugs.python.org/file23830/log.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13516
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com