[issue11235] Source files with date modifed in 2106 cause OverflowError

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Committed, thanks.

--
resolution:  -> fixed
stage: commit review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Roundup Robot]

Roundup Robot  added the comment:

New changeset a2f3d6986bfa by Antoine Pitrou in branch '3.2':
Issue #11235: Fix OverflowError when trying to import a source file whose 
modification time doesn't fit in a 32-bit timestamp.
http://hg.python.org/cpython/rev/a2f3d6986bfa

New changeset cb13d8cff050 by Antoine Pitrou in branch 'default':
Issue #11235: Fix OverflowError when trying to import a source file whose 
modification time doesn't fit in a 32-bit timestamp.
http://hg.python.org/cpython/rev/cb13d8cff050

New changeset 0499eed74126 by Antoine Pitrou in branch '2.7':
Issue #11235: Fix OverflowError when trying to import a source file whose 
modification time doesn't fit in a 32-bit timestamp.
http://hg.python.org/cpython/rev/0499eed74126

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Brett Cannon]

Brett Cannon  added the comment:

LGTM

--
assignee:  -> pitrou
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
keywords: +patch
Added file: http://bugs.python.org/file24310/imptimestampoverflow.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here is a patch for 3.2. importlib doesn't have the issue, so I just added a 
test.

--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Fixing this is much easier than Victor's suggestion, we just have to ignore the 
higher bits of the timestamp (that is, store it modulo 2**32). This is enough 
for the purposes of testing the freshness of a pyc file.

And by avoiding modifying the pyc format, we can also ship the fix in bugfix 
releases.

--
components: +Interpreter Core -None
nosy: +brett.cannon, pitrou
versions: +Python 2.7, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[STINNER Victor]

STINNER Victor  added the comment:

By the way, a simpler fix would be just to not fail if the .pyc file cannot be 
created (but emit a warning / write an error to stderr).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[STINNER Victor]

STINNER Victor  added the comment:

> To support bigger timestamps, we have to change the file format 
> of .pyc files.

write_compiled_module(), check_compiled_module() and other functions use the 
marshal module to read/write binary file, but marshal has no function for 
int64_t, only for long (which may be 32 bits, especially on Windows).

I don't know if Python has a builtin 64 bits integer type. There is 
PY_LONG_LONG, but this type is optional. A possible solution is to always store 
timestamp as 64 bits signed integer, but reject timestamp > 2^32 (as currently) 
if we don't have 64 bits integer type (like PY_LONG_LONG). Something like:

#ifdef PY_LONG_LONG
   write_timestamp64(t);
#else
   if (t << 32) { error; }
   write_timestamp32(t);
   write_long32(0); /* emulate 64 bits in big endian */
#endif

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[STINNER Victor]

STINNER Victor  added the comment:

The problem occurs on import (import bla reads bla.py), when Python tries to 
create bla.pyc. The problem is that Python stores the timestamp as 4 bytes in 
.pyc files, whereas time_t is 64 bits on Windows (at least on Windows XP with 
Visual Studio).

To support bigger timestamps, we have to change the file format of .pyc files. 
It cannot be done in Python 2.7, I propose to do it in Python 3.3

See also #5537 and #4379: other issues with 64 bits => 32 bits timestamps.

--
nosy: +belopolsky, haypo
versions: +Python 3.3 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +barry, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11235] Source files with date modifed in 2106 cause OverflowError

[Guy Kisel]

New submission from Guy Kisel :

Tested in Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit 
(Intel)] on win32

Exception thrown: OverflowError: modification time overflows a 4 byte field

Steps to reproduce:
1. Set system time to 2/8/2106 or later and modify a .py file (or use a utility 
to change the date modified directly).
2. Try to run or import the .py file.

This date is 2^32 seconds after the Unix epoch.

--
components: None
messages: 128753
nosy: Guy.Kisel
priority: normal
severity: normal
status: open
title: Source files with date modifed in 2106 cause OverflowError
type: crash
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com