[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Vinay Sajip]

Vinay Sajip  added the comment:

Closing, as per last comment.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Berker Peksag]

Change by Berker Peksag :


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Vinay Sajip]

Vinay Sajip  added the comment:

It's not Gunicorn-specific - more a case of multiple processes writing to a 
single log file. This is not supported directly, as documented here:

https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes

As the cookbook says, you can set up a separate process to handle logging to 
file, which listens with a socket server or a multiprocessing QueueListener; 
your worker processes would communicate with it using either a SocketHandler or 
a QueueHandler, as appropriate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Cheryl Sabella]

Change by Cheryl Sabella :


--
nosy: +vinay.sajip
versions: +Python 3.8 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Umesh Pathak]

Change by Umesh Pathak :


--
keywords: +patch
pull_requests: +9027
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29001] logging.handlers.RotatingFileHandler rotation broken under gunicorn

[Bruce Edge]

New submission from Bruce Edge:

I've been seeing some funny behavior in log files form a 
logging.handlers.RotatingFileHandler.

Here's my handler config snippet, in a "log.cfg":

[handlers]
keys=log_file

[formatters]
keys=access

[formatter_access]
format=%(asctime)s - %(name)s(%(funcName)s:%(lineno)s)[%(process)d] - 
%(levelname)s: %(message)s
datefmt=%Y-%m-%d %H:%M:%S
class=logging.Formatter

[handler_log_file]
class=logging.handlers.RotatingFileHandler
formatter=access
args=('/var/log/ml-api/access.log', 'a', 1000, 16, 'utf8')

Specified using:

 /usr/local/bin/gunicorn --worker-class=gevent --bind unix:ml-api.sock -m 007 
--workers=8 --log-config=log.cfg --log-level=DEBUG app

I ran this script to show the progression of log file creation:

while true ; do ls -lrt && sleep 10; done

Initially there's one incrementing log file, fine:

-rw-r--r-- 1 ubuntu www-data 9765042 Dec 16 17:50 access.log
total 9572
-rw-r--r-- 1 ubuntu www-data 9796828 Dec 16 17:50 access.log
total 9656
-rw-r--r-- 1 ubuntu www-data 9881053 Dec 16 17:50 access.log
total 9708
-rw-r--r-- 1 ubuntu www-data 9936776 Dec 16 17:50 access.log
total 9756
-rw-r--r-- 1 ubuntu www-data 9984782 Dec 16 17:50 access.log
total 9816

But as soon as it gets rotated, I immediately get 8 log files:

-rw-r--r-- 1 ubuntu www-data 911 Dec 16 17:50 access.log.7
-rw-r--r-- 1 ubuntu www-data2578 Dec 16 17:50 access.log.5
-rw-r--r-- 1 ubuntu www-data2578 Dec 16 17:50 access.log.3
-rw-r--r-- 1 ubuntu www-data6871 Dec 16 17:50 access.log.2
-rw-r--r-- 1 ubuntu www-data5122 Dec 16 17:50 access.log.1
-rw-r--r-- 1 ubuntu www-data   11165 Dec 16 17:50 access.log.4
-rw-r--r-- 1 ubuntu www-data1718 Dec 16 17:50 access.log
-rw-r--r-- 1 ubuntu www-data2905 Dec 16 17:50 access.log.6
total 9864
-rw-r--r-- 1 ubuntu www-data 911 Dec 16 17:50 access.log.7
-rw-r--r-- 1 ubuntu www-data8921 Dec 16 17:50 access.log.6
-rw-r--r-- 1 ubuntu www-data   15460 Dec 16 17:50 access.log
-rw-r--r-- 1 ubuntu www-data   10313 Dec 16 17:51 access.log.2
-rw-r--r-- 1 ubuntu www-data7699 Dec 16 17:51 access.log.3
-rw-r--r-- 1 ubuntu www-data   21471 Dec 16 17:51 access.log.4
-rw-r--r-- 1 ubuntu www-data6874 Dec 16 17:51 access.log.5
-rw-r--r-- 1 ubuntu www-data   11989 Dec 16 17:51 access.log.1
total 9892
-rw-r--r-- 1 ubuntu www-data 911 Dec 16 17:50 access.log.7
-rw-r--r-- 1 ubuntu www-data7699 Dec 16 17:51 access.log.3
-rw-r--r-- 1 ubuntu www-data   12849 Dec 16 17:51 access.log.1
-rw-r--r-- 1 ubuntu www-data   14936 Dec 16 17:51 access.log.6
-rw-r--r-- 1 ubuntu www-data   30068 Dec 16 17:51 access.log.4
-rw-r--r-- 1 ubuntu www-data   19755 Dec 16 17:51 access.log
-rw-r--r-- 1 ubuntu www-data   11170 Dec 16 17:51 access.log.5
-rw-r--r-- 1 ubuntu www-data   15466 Dec 16 17:51 access.log.2
total 9932

Is this a consequence of the gunicorn --workers=8?

Does logging.handlers.RotatingFileHandler not work with gunicorn workers?

I tried --worker-class=gevent as well with the same result.

--
components: Extension Modules
messages: 283511
nosy: Bruce Edge
priority: normal
severity: normal
status: open
title: logging.handlers.RotatingFileHandler rotation broken under gunicorn
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com