[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Marco Buccini marcu...@gmail.com: -- nosy: -markon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Okay just tried it. This is under Windows 7 x64. I created a .py file with read-only permission. When I import the .py module, the created .pyc doesn't have read-only permission. So it works as intended under Windows OS. As long as under posix system the created .pyc files have the same permissions as the .py file, then everything is good. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by desolat nuabara...@web.de: -- nosy: -desolat ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi Nick, Thank you so much for rolling out this change. I think your implementation of the fix is more elegant. I have learned a few things from you. I would love to keep helping to test/fix bugs for Python on Windows. Please contact me if there is anything you would like me to take a look at. There will be many Windows users who will feel very grateful for this change you have implemented. This will actually give many the chance to upgrade to the latest 2.7 when it is released. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: No worries. If you could do a fresh build of the 2.7 branch and confirm this fix works for you, that would be great. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: Interesting - the test case I wrote for this was failing on Linux as well. Same fix worked both places, though (i.e. ensuring S_IWUSR is always set on .pyc files so they can be updated properly) 2.7 update incoming, we'll see what the Windows buildbots have to say before doing anything to 3.x. (Kevin: I didn't end up using your patch, as I decided it was simpler to just create the files with the user's write permission set. This doesn't weaken the security, as any attacker running with the user's privileges could just change the permissions anyway) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: Ah, my test was buggy - the mtime wasn't reliably changing by enough, so the pyc wasn't always being rewritten. I changed it to force a big time delta by backdating the original source file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset 321414874b26 by Nick Coghlan in branch '2.7': Issue #6074: Restore the long-broken support for running with read-only source files on Windows http://hg.python.org/cpython/rev/321414874b26 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: Buildbots look happy, removing 2.7 from affected versions. -- versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset 1856d57abfc8 by Nick Coghlan in branch '3.2': Issue #6074: Forward port Windows read-only source file fix from 2.7 http://hg.python.org/cpython/rev/1856d57abfc8 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset 3f6db10b7a69 by Nick Coghlan in branch '3.3': Dummy merge from 3.2, as that approach to resolving #6074 is not applicable to importlib http://hg.python.org/cpython/rev/3f6db10b7a69 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: OK, it should just be the two importlib based releases affected now. -- versions: +Python 3.4 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset df83d3bbd934 by Nick Coghlan in branch '2.7': Issue #6074: Actually delete the source file in the test as intended http://hg.python.org/cpython/rev/df83d3bbd934 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset bbb3459fbcb8 by Nick Coghlan in branch '3.3': Issue #6074: Apply an appropriate fix for importlib based imports http://hg.python.org/cpython/rev/bbb3459fbcb8 New changeset bedcc790c6e7 by Nick Coghlan in branch 'default': Merge fix from #6074 from 3.3 http://hg.python.org/cpython/rev/bedcc790c6e7 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Roundup Robot added the comment: New changeset 8cf7f6fe4282 by Nick Coghlan in branch '3.2': Issue #6074: Actually delete the source file in the test as intended http://hg.python.org/cpython/rev/8cf7f6fe4282 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: For 3.3+ I ended up using a tweaked version of Eric's patch - I kept the general test structure, but used the behavioural test I had created (i.e. does updating the initially read-only source file affect a subsequent bytecode only import?), as well as the simple rule of just always making the cached files writable by the owning user. On 3.3+, this permissions tweak occurs regardless of OS, because I didn't think it was worth the runtime hit to make it OS specific (whereas the use of the C preprocessor previously made that not an issue). -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Eric Snow added the comment: Sounds good. Thanks for getting this done, Nick. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi Eric, I am happy to install Python 3.3 on Windows and help you test this. Can you please give me some ideas on what I need to do? Do I sync to the latest Python 3.3 branch and compile? Then run some unit-test code? Which test files should I run? What should I be expecting? Also, do you know if anyone is working on adding the fix and test to Python 2.7? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: It's still on my list to follow up (along with the other contributions from the PyCon AU sprints). However, if Eric gets a Windows environment up and running before I get to it, I'm happy to hand the issue over (I rely on the buildbots to sanity check Windows fixes). -- versions: -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Eric Snow added the comment: Well, the workflow on Windows is certainly different. I've run out of time to get this sorted out in the short term. However, the patch should be pretty close to what's needed. Obviously the posix-only bit doesn't fit for a Windows-specific test. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Eric Snow added the comment: Here's a patch for 3.3 with a test. Though, I'm still setting up my windows box for building Python, I wanted to get this patch up. On its own the test should fail on Windows. In that case, it should be backport-able. If no one has a chance to verify the test on windows (or the fix) before I get everything set up, I'll do it. -- Added file: http://bugs.python.org/file27028/issue6074_3.3_eric.snow.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: Adding 3.3 to affected versions, since I'm about to check in a fix for #2051 that will reintroduce this problem. -- nosy: +eric.snow versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: Hi I made the following patch for this issue. It addresses the #2051 security bug. So the .pyc and .pyo files are created using the same permission as .py The MS_WINDOWS version will chmod the permission of .pyc and .pyo files to write accessible before deleting them. I have also created a test for it. This test needs cleaning up, because it is the first time I have written a test for cpython. I would love to have a look at the final committed test so I can learn how to make it better next time. I will also be happy to keep extending that test to make sure it covers more cases. A great big thanks to Nick for your help! -- keywords: +patch versions: +Python 3.1 Added file: http://bugs.python.org/file26912/issue6074.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Nick Coghlan ncogh...@gmail.com: -- assignee: - ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment:
I propose a fix:
static FILE *
open_exclusive(char *filename, mode_t mode)
{
#if defined(O_EXCL)defined(O_CREAT)defined(O_WRONLY)defined(O_TRUNC)
/* Use O_EXCL to avoid a race condition when another process tries to
write the same file. When that happens, our open() call fails,
which is just fine (since it's only a cache).
XXX If the file exists and is writable but the directory is not
writable, the file will never be written. Oh well.
*/
int fd;
(void) unlink(filename);
fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC
#ifdef O_BINARY
|O_BINARY /* necessary for Windows */
#endif
#ifdef __VMS
, mode, ctxt=bin, shr=nil
#elif defined(MS_WINDOWS)
, mode | _S_IWRITE
#else
, mode
#endif
);
if (fd 0 )
return NULL;
return fdopen(fd, wb);
#else
/* Best we can do -- on Windows this can't happen anyway */
return fopen(filename, wb);
#endif
}
--
so doesn't matter what the .py file permission is under windows, the .pyc file
will always have both read and write permissions.
--
nosy: +lowlifer123
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6074
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Kevin Chen added the comment: This is essentially the way things were done in version 2.5.4 of Python. The unlink() always succeeded, because the created .pyc file permission is always set to 0666 in the fd=open() function. This means the .pyc will never be created as read-only, and as long as they are never set to read-only manually by the user, everything will be okay. You might say this will be a problem if someone accidentally set the it to read-only. Well we have been using Python 2.5 for many years, and we have managed quite well, so it is not really going to be a big issue, and can be fixed in the future. Whereas at the moment we have a PROBLEM!! With the current state of the Python 2.6, 2.7 and 3.2 interpreters, many Windows users cannot even think about upgrade from Python 2.5 because source control tools like Perforce will set all .py files to read-only, and so ALL created .pyc files will become read-only every time you run Python. I cannot stress how much pain this causes. PLEASE MAKE IT 2.5 EQUIVALENT ASAP! THANK YOU!! even just this will do the trick: - fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC #ifdef O_BINARY |O_BINARY /* necessary for Windows */ #endif #ifdef __VMS , mode, ctxt=bin, shr=nil #elif defined(MS_WINDOWS) , 0666 #else , mode #endif ); --- And a side note, for the VC8 build for PC, the file random.c was left out from the visual studio project file for pythoncore project. It gives these errors when I try recompile: Error 2 error LNK2019: unresolved external symbol __PyRandom_Init referenced in function _Py_Main main.obj Error 3 error LNK2001: unresolved external symbol __PyRandom_Init pythonrun.obj Error 4 error LNK2019: unresolved external symbol __PyOS_URandom referenced in function _posix_urandom posixmodule.obj This is my first time using bug tracker, so please point me to the appropriate place to put this. Thank you. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Kevin Chen mr.kevin.chen...@gmail.com: -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: OK, I tracked down the original reason for the behavioural change: #2051 The important aspect in that patch is preserving the *read* permissions from the .py file so we don't accidentally expose data to other users. Kevin's patch seems like a good approach for restoring compatibility in 2.7 and 3.2 In 3.3, it appears that #2051 has regressed, since no test was added to confirm the correct behaviour and importlib appears to behave more like 2.5. -- versions: -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Nick Coghlan added the comment: (Note: Kevin is looking into this at the PyConAU sprints) New test case needed (probably in Lib/test/test_import.py): - create .py file as 0400 - import it - check .pyc (or .pyo) file is 0600 That ensures the #2501 security bug remains fixed, while also addressing this problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by desolat nuabara...@web.de: -- nosy: +desolat ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file18739/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Éric Araujo mer...@netwok.org added the comment: Bug fixes go into 2.7 but not 2.6. See msg115664. -- nosy: +eric.araujo versions: -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Steve Thompson steve.f.thomp...@gmail.com added the comment: So what's the current status of this on Windows Platforms? -- nosy: +Steve.Thompson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: This is still the case: on Windows, if foo.py is readonly, python -c import foo generates foo.pyc with the readonly attribute. Tested with 3.1 and current py3k (where the file is named __pycache__\foo.cpython-32.pyc) -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Steve Thompson steve.f.thomp...@gmail.com added the comment: Any idea if this will be fixed (at all) and/or back ported to 2.6.x or 2.7.x? On Fri, Sep 3, 2010 at 3:36 PM, Amaury Forgeot d'Arc rep...@bugs.python.org wrote: Amaury Forgeot d'Arc amaur...@gmail.com added the comment: This is still the case: on Windows, if foo.py is readonly, python -c import foo generates foo.pyc with the readonly attribute. Tested with 3.1 and current py3k (where the file is named __pycache__\foo.cpython-32.pyc) -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ -- Added file: http://bugs.python.org/file18739/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___Any idea if this will be fixed (at all) and/or back ported to 2.6.x or 2.7.x?brbrdiv class=gmail_quoteOn Fri, Sep 3, 2010 at 3:36 PM, Amaury Forgeot d#39;Arc span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/span wrote:br blockquote class=gmail_quote style=margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;br Amaury Forgeot d#39;Arc lt;a href=mailto:amaur...@gmail.com;amaur...@gmail.com/agt; added the comment:br br This is still the case: on Windows, if foo.py is readonly,br   python -c quot;import fooquot;br generates foo.pyc with the readonly attribute.br br Tested with 3.1 and current py3k (where the file is named __pycache__\foo.cpython-32.pyc)br br --br nosy: +amaury.forgeotdarcbr divdiv/divdiv class=h5br ___br Python tracker lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br lt;a href=http://bugs.python.org/issue6074; target=_blankhttp://bugs.python.org/issue6074/agt;br ___br /div/div/blockquote/divbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Antoine Pitrou pit...@free.fr added the comment: Apparently this may have become Windows-specific at some point. I can't reproduce under Linux with Python 2.6.5, 2.7 or 3.2. I get a strange warning with -v under 2.7, though: $ touch b.py ~/cpython/27/python -v a.py 21 | grep b.py # /home/antoine/py3k/__svn__/b.pyc has bad mtime import b # from /home/antoine/py3k/__svn__/b.py # wrote /home/antoine/py3k/__svn__/b.pyc (bad mtime??) -- components: +Windows nosy: +barry, pitrou priority: critical - normal ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Dirk Jülich mucisl...@web.de added the comment: Applies also to Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32. -- nosy: +mucisland versions: +Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Mark Lawrence breamore...@yahoo.co.uk: -- nosy: +brian.curtin versions: +Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: I verified that launching the script from a CMD.EXE window instead of from Cygwin also reproduces the can't overwrite ZZZ.pyc. Here's a sample of python -v showing this: # C:\depot\central\ntf\tools\bin\build1.pyc has bad mtime import build1 # from C:\depot\central\ntf\tools\bin\build1.py # can't create C:\depot\central\ntf\tools\bin\build1.pyc If you would like some help, I can discuss this with Windows expert programmers here and see whether they can suggest a solution you might find acceptable. Thank you. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: Re: chmod 0666 inappropriate: I agree that leaving a file 0666 is inappropriate, however theoretically the code should never leave the file 0666 since the unlink will delete the file. The code was intended to show what the problem is by demonstrating a solution. Note that the line immediately following the chmod 0666 is an unlink call, which will delete the file if the user had sufficient permission to chmod the file to 0666. I suppose the meta-level description of the problem is: [1] Without the chmod 0666, the unlink will fail if the file is 0400 or similar. [2] With the chmod 0666, the unlink succeeds, and the file will be recreated with new permissions (not 0666: I believe my tests show 0444) after the post-unlink file creation. [3] Someone familiar with windows should analyze, propose a solution, test it under all relevant scenarios of file/group ownership (and again the chmod is only required on Windows), and on XP/Vista/Win7. The only problem I can see with the chmod 0666 is if for some reason the file still exists with bad permissions after the unlink. Since this is a low-overhead operation compared to the time to regenerate the .pyc file, I think that it would be reasonable to have an #ifdef MS_WINDOWS, if the file exists after the unlink(), print a big warning message that will always be visible even if -v is not used when invoking Python. This is getting complicated enough that you may want to see if you can get someone else to review it for ideas about how to address the chmod/unlink code. I am certainly not a Windows expert hence I'm making a best Unix-like attempt to analyze and test fixes. Good luck and thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Marco marcu...@gmail.com added the comment: Which version are you using? I've seen the source code of import.c (2.5 and 2.6) and it seems that in the 2.6 some bugfixes/features have been added. However, I've just used the version 2.7 and it works fine (as 2.6, since they have the same code (at least in import.c), until now). bye -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: Oops, I was not accurate in reporting behavior: I now believe that this fails on Windows XP, but may work fine on Linux. Below are my results, showing reproducible failure on XP. The most important thing about this is: on XP, once the .pyc file is broken, Python will not overwrite it, and it must be *manually* removed. This is bad! I will append Ubuntu results in a few minutes. To reproduce the problem, I had to modify the script to include a sleep, see attached showpyreadonlysleep.sh (cygwin bash script). Here's the results of running showpyreadonlysleep.sh, then manually reproducing the problem with touch/python: $ ./showpycreadonlysleep.sh umask = 0022 CYGWIN_NT-5.1 psimanyi_xp 1.5.25(0.156/4/2) 2008-06-12 19:34 i686 Cygwin Python 2.6.2 Running python.exe from: /cygdrive/c/Python26/python -rw-r--r-- 1 psimanyi mkgroup-l-d 10 May 26 12:43 b.py b -rwx--+ 1 psimanyi mkgroup-l-d 137 May 26 12:43 b.pyc b -r-x--+ 1 psimanyi mkgroup-l-d 137 May 26 12:43 b.pyc # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc And now, running manually: $ touch b.py psima...@psimanyi_xp ~ $ python -v a.py 21 | grep 'b\.py' # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc psima...@psimanyi_xp ~ $ touch b.py psima...@psimanyi_xp ~ $ python -v a.py 21 | grep 'b\.py' # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc Again, this shows that without removing the .pyc manually, the pyc will not be overwritten, and therefore program startup will be slow! -- Added file: http://bugs.python.org/file14083/showpycreadonlysleep.sh ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: Re: Ubuntu: I can't reproduce the problem. At this point, I consider this problem Windows XP-only. Re: Windows bash: I decided to verify that this problem occurs even if running outside of Bash in the native (and wonderful) shell CMD.EXE. It *does* occur. Here's the state: [1] Run my script in Bash, which fails as in my previous append from today. [2] Start CMD.EXE. Then run a touch b.py in Bash, then verify using Bash ls -l that b.py is protection 444 (readonly). [3] In CMD.EXE, run python -v a.py (I don't know how to use something like grep in CMD.EXE, so I just manually copy/pasted the python -v lines.) The following lines in CMD.EXE's output show that the problem still occurs in CMD.EXE: # C:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from C:\Documents and Settings\psimanyi\b.py # can't create C:\Documents and Settings\psimanyi\b.pyc b # clear __builtin__._ If you are unable to reproduce this on XP, I will be happy to help. I would not be surprised if this had something to do with the granularity of mtime or some other oddity of Windows, because: if I delete the sleep in the Bash script, then rerun the Bash script very quickly using control-P control-J typed ahead, sometimes the Bash script fails and sometimes it succeeds! (Nice work, Windows.) Thanks much. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: I may have a reason to analyze that would explain why this fails on Windows: the non-Windows code path in import.c's open_exclusive() contians an unlink(filename), but the Windows path does not. I'm going to try to build Python on Windows and add an unlink for both code paths. More later. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment:
I have a fix for this. The code is Windows-only. It has been verified to work
for attached test script
showpycreadonlysleep.sh. It simply adds a chmod call. The issue is that the
unlink() call silently fails
if the file is readonly, but unlink() succeeds if chmod() makes the file
writable.
Out company would really appreciate having this fix integrated into 2.6 since
we use the ActiveState 2.6
builds on Windows. I haven't test this on non-Windows platforms but it should
not change the behavior on
non-Windows platforms if the #ifdef MS_WINDOWS is correct. The diff is below:
$ svn diff
Index: import.c
===
--- import.c(revision 72946)
+++ import.c(working copy)
@@ -840,6 +840,7 @@
static FILE *
open_exclusive(char *filename, mode_t mode)
{
+
#if defined(O_EXCL)defined(O_CREAT)defined(O_WRONLY)defined(O_TRUNC)
/* Use O_EXCL to avoid a race condition when another process tries to
write the same file. When that happens, our open() call fails,
@@ -848,6 +849,9 @@
writable, the file will never be written. Oh well.
*/
int fd;
+#ifdef MS_WINDOWS
+(void) chmod(filename, 0600);
+#endif
(void) unlink(filename);
fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC
#ifdef O_BINARY
It may be appropriate to document that the chmod() is only required on Windows,
and therefore it is only
called on Windows to avoid slowing down non-Windows platforms.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6074
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi psima...@namcobandaigames.com added the comment: I realized that my patch, with a chmod(..., 0600), may not work under certain conditions where the original file is owned by a different user but the directory is group- or world-writable. A more likely correct fix would use chmod(..., 0666). At any rate: someone should test the various combinations of whether this user or another user owns the original .pyc file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Marco marcu...@gmail.com added the comment: TO pdsimanyi: chmod it's not appropriate to change the permissions as you made: 0666 et voilà. However, I don't understand if it can be a problem since you're using Cygwin (and Bash under Windows) or not. I think this because Windows is not POSIX compliant and under Windows the permissions are different than the others under *NIX-like platforms (POSIX compliant). Maybe in these days I can check the permissions on Windows. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Changes by Antoine Pitrou pit...@free.fr: -- priority: - critical stage: - needs patch versions: +Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Marco marcu...@gmail.com added the comment: It seems Python is using a default mask to compile modules. If you open a file b.py as r, and you import it from a.py, you can see that b.pyc is: -rwxr-xr-x 1 marco marco9 24 mag 16:17 a.py -r--r--r-- 1 marco marco 10 24 mag 16:17 b.py -rw-r--r-- 1 marco marco 122 24 mag 16:46 b.pyc If you try to run a.py: -rwxr-xr-x 1 marco marco9 24 mag 16:17 a.py -r-xr-xr-x 1 marco marco 10 24 mag 16:17 b.py you'll get: -rw-r--r-- 1 marco marco 122 24 mag 16:49 b.pyc as you can see, there is a default mode to create compiled modules... -- nosy: +markon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6074 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
New submission from Peter Simanyi psima...@namcobandaigames.com:
Some source code control tools, like Perforce, by default sync files readonly,
and it's useful to
leave them readonly so that you can mark files changed by making them writeable
even if they're not
checked out (e.g. working offline).
When python implicitly compiles a .py file that's readonly, it will create a
.pyc file that's
readonly. This would be fine, except that when the .py file changes, python
*silently* refuses to
overwrite a .pyc file that's readonly, even if the directory is completely
writable by the user
executing Python.
The attached script shows this behavior. Running python -v will produce the
following line
showing this:
import b # from c:\Documents and Settings\psimanyi\tmp\b.py
# can't create c:\Documents and Settings\psimanyi\tmp\b.pyc
And because I hoped this was a Windows-only problem, I tested on OSX (Leopard)
and Ubuntu (9.10
alpha): it occurs on all three platforms.
Thanks!
I fixed showpycreadonly.sh after attaching it, so to be sure you have the
current version, it
follows:
rm -f [ab].py{,c}
echo 'import b' a.py
echo 'print b' b.py
ls -l b.py
python a.py
ls -l b.pyc
rm b.pyc
chmod -w b.py
python a.py
ls -l b.pyc
touch b.py
python -v a.py 21 | grep b.py
--
components: Interpreter Core
files: showpycreadonly.sh
messages: 88138
nosy: pdsimanyi
severity: normal
status: open
title: .pyc files created readonly if .py file is readonly, python won't
overwrite
type: behavior
versions: Python 2.6
Added file: http://bugs.python.org/file14025/showpycreadonly.sh
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6074
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
