New submission from Mark Dickinson <dicki...@gmail.com>:
(Marking this as 'Interpreter Core' because the issue really seems to come from
Python/import.c rather than from the py_compile module.)
On a Windows 7 VM (64-bit, NTFS) running Python 2.7 (also reproduced on a
non-VM Windows 7 installation), I'm seeing the following surprising behaviour:
C:\Python27\Lib>dir decimal.py*
Volume in drive C is Local
Volume Serial Number is 5083-D43D
Directory of C:\Python27\Lib
06/10/2011 08:46 PM 219,383 decimal.py
01/25/2012 07:05 PM 165,322 decimal.pyc
01/25/2012 04:54 PM 169,386 decimal.pyo
3 File(s) 554,091 bytes
0 Dir(s) 966,266,880 bytes free
C:\Python27\Lib>python
Enthought Python Distribution -- www.enthought.com
Version: 7.2-2 (64-bit)
Python 2.7.2 |EPD 7.2-2 (64-bit)| (default, Sep 14 2011, 11:25:00) [MSC v.1500
64 bit (AMD64)] on win32
Type "packages", "demo" or "enthought" for more information.
>>> import py_compile; py_compile.compile('decimal.py')
>>> import os; os.stat('decimal.pyc').st_mtime
1327518430.306176
>>> import decimal
>>> os.stat('decimal.pyc').st_mtime
1327518443.9094632
Notice that in spite of calling py_compile.compile on decimal.py, the .pyc file
is still regenerated on import.
Relevant details: note that the decimal.py timestamp is in the summer, and that
it's currently winter time. Also, my Windows 7 timezone setting is UTC, with
'automatically adjust for DST' set to true.
This bit me today when working with a 'post-install' script for an MSI
installer for a large Python application, where the post-install script did a
compileall.compile_path to make sure that all the .pyc files were up to date,
and thereby avoid a *long* (> 90 second) application first startup time. It
turned out that on application startup some of the .pyc files got regenerated
while others didn't, and it took me far too long to notice that it was the .py
files with summer timestamps that lead to .pyc regeneration, and that those
with winter timestamps were okay.
When I set my timezone to plain UTC with no DST adjustments, the above issue
disappears. Also, when I reset my timezone to that of Saudi Arabia (no DST),
the issue also disappears.
The cause of the issue seems to be that:
(1) import.c uses an 'fstat' call to get mtime for a .py file.
(2) on my Windows installation, the result of fstat on a file varies (a) with
time of year, and (b) with computer DST and timezone settings. (No such
problems on OS X.)
(3) in contrast, py_compile uses os.stat(...).st_mtime, which appears to be
invariant under time changes.
The difference between the 'fstat' result and the os.stat result can be seen
directly:
Python 2.7.2 |EPD 7.2-2 (64-bit)| (default, Sep 14 2011, 11:25:00) [MSC v.1500
64 bit (AMD64)] on win32
Type "packages", "demo" or "enthought" for more information.
>>> import decimal # make sure .pyc file is up to date
>>> import os; os.stat('decimal.py').st_mtime # actual mtime of .py file
1307738784.0
>>> import struct; struct.unpack('LL', open('decimal.pyc', 'rb').read(8))[1] #
>>> stored mtime from .pyc; uses fstat
1307735184
This presumably also means that in this part of the world, .pyc files will be
regenerated on import after any DST change, since the reported 'fstat' result
used by import.c will no longer match the stored mtime timestamp in the .pyc
file.
----------
components: Interpreter Core
messages: 151969
nosy: mark.dickinson
priority: normal
severity: normal
status: open
title: Using py_compile does not prevent recompilation due to 'bad mtime'.
type: behavior
versions: Python 2.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue13863>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
