Re: [Python-Dev] __pycache__ creation

2010-03-25 Thread Nick Coghlan
Ron Adam wrote: > h... unless there is a __pycache__ *file* located there first. ;-) Just a specific reason why attempting to create __pycache__ can fail (which has defined behaviour in the PEP - running directly from the source without caching the bytecode file). Cheers, Nick. -- Nick Cogh

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Ron Adam
Nick Coghlan wrote: Ron Adam wrote: I think I misunderstood this at first. It looks like, while developing a python 3.2+ program, if you don't create an empty __pycache__ directory, everything will still work, you just won't get the .pyc files. That can be a good thing during development bec

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Stephen J. Turnbull
Barry Warsaw writes: > On Mar 24, 2010, at 11:06 PM, Nick Coghlan wrote: > > >Ah yes, the recollection of seeing such a message is now quite fresh in > >my mind :) > > Just don't tell Guido I borrowed his time machine keys! Wouldn't that be preferable to revealing you've learned to hotwire

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Barry Warsaw
On Mar 24, 2010, at 11:06 PM, Nick Coghlan wrote: >Ah yes, the recollection of seeing such a message is now quite fresh in >my mind :) Just don't tell Guido I borrowed his time machine keys! -Barry signature.asc Description: PGP signature ___ Python-D

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Nick Coghlan
Barry Warsaw wrote: > On Mar 24, 2010, at 10:04 PM, Nick Coghlan wrote: > >> Yep, that's my recollection as well. I don't recall seeing an update to >> state that clearly in the PEP go by on the checkins list though :) > > Check again . Ah yes, the recollection of seeing such a message is now qu

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Barry Warsaw
On Mar 24, 2010, at 10:04 PM, Nick Coghlan wrote: >Yep, that's my recollection as well. I don't recall seeing an update to >state that clearly in the PEP go by on the checkins list though :) Check again . -Barry signature.asc Description: PGP signature _

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Barry Warsaw
On Mar 23, 2010, at 08:42 PM, Ron Adam wrote: >It looks like, while developing a python 3.2+ program, if you don't create >an empty __pycache__ directory, everything will still work, you just won't >get the .pyc files. That can be a good thing during development because >you also will not have

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Nick Coghlan
Ron Adam wrote: > I think I misunderstood this at first. > > It looks like, while developing a python 3.2+ program, if you don't > create an empty __pycache__ directory, everything will still work, you > just won't get the .pyc files. That can be a good thing during > development because you also

Re: [Python-Dev] __pycache__ creation

2010-03-24 Thread Nick Coghlan
Barry Warsaw wrote: > On Mar 22, 2010, at 12:38 PM, Guido van Rossum wrote: > >> Huh? Last time I looked weren't we going to make __pycache__ the >> default (and eventually only) behavior? > > We definitely agreed it would be the default in Python 3.2. > > My recollection is that we agreed it wo

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Ron Adam
Terry Reedy wrote: On 3/22/2010 2:15 PM, Antoine Pitrou wrote: What I am proposing is that the creation of __pycache__ /directories/ be put outside of the core. It can be part of distutils, or of a separate module, or delegated to third-party tools. It could even be as simple as "python -m

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Barry Warsaw
On Mar 23, 2010, at 12:02 AM, Martin v. Löwis wrote: >I think the appropriate action at this point is to record this specific >objection in the PEP. Done. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.or

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Barry Warsaw
On Mar 22, 2010, at 09:57 PM, Antoine Pitrou wrote: >It's especially annoying, of course, if you have to ask someone else to >remove the directories for you (or if you have to write custom code and get >it executed by the Apache or WSGI handler...). compileall should probably grow a --clean optio

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Barry Warsaw
On Mar 22, 2010, at 09:47 PM, Martin v. Löwis wrote: >Therefore, I'm in favor of having it on by default. If certain use cases >make it problematic (e.g. Apache creating directories which you then >cannot delete), there should be a way to turn it *off*. Perhaps the >existing machinery to turn of b

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Barry Warsaw
On Mar 22, 2010, at 12:38 PM, Guido van Rossum wrote: >Huh? Last time I looked weren't we going to make __pycache__ the >default (and eventually only) behavior? We definitely agreed it would be the default in Python 3.2. My recollection is that we agreed it would be the only on-demand way of wri

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Greg Ewing
Steven D'Aprano wrote: If the user has write permission and the __pycache__ folder is created, but the umask is screwy and no .pyc files can be created, no .pyc file is created and the import uses the .py file only despite the existence of an empty __pycache__ folder. Sounds okay to me. --

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Greg Ewing
Antoine Pitrou wrote: Well, if I can create a setuid apache shell, I can probably su as root or apache as well. ("su -c rm -r whatever") Or are you talking about a Web-based shell? I'm just saying that if there is any way of running code of your choice as the apache user, you can get it to ma

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Henning von Bargen
Antoine Pitrou wrote: > > or if a user installs by dragging into > > site-packages instead of using an installer? > > Well... do people actually do this? Why not? And it is also common to just set the PYTHONPATH environment variable instead of using setup.py install or copy the files to the lib/

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Steven D'Aprano
On Tue, 23 Mar 2010 07:38:42 am Guido van Rossum wrote: > But creating it as needed runs into at least similar problems with > ownership as creating .pyc files when first needed (if the parent > directory is root-owned a mere mortal can't create it at all). Isn't that a feature though? I don't se

Re: [Python-Dev] __pycache__ creation

2010-03-23 Thread Martin v. Löwis
> Is there a need for python to use __pycache__ directories 100% of the > time? For 2.x it seems like being flexible would be best, and if 3.x > is going to be strict about it, it should be strict sooner than later > rather than have a lot of 3rd party packages break at some point down > the road

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Terry Reedy
On 3/22/2010 2:15 PM, Antoine Pitrou wrote: What I am proposing is that the creation of __pycache__ /directories/ be put outside of the core. It can be part of distutils, or of a separate module, or delegated to third-party tools. It could even be as simple as "python -m compileall --pycache", i

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Ron Adam
Guido van Rossum wrote: On Mon, Mar 22, 2010 at 12:20 PM, Barry Warsaw wrote: On Mar 22, 2010, at 02:02 PM, Ron Adam wrote: If I understand correctly, we would have the current mode as the default, and can trigger __pycache__ behavior simply by manually creating a __pycache__ directory and d

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Greg Ewing canterbury.ac.nz> writes: > > Antoine Pitrou wrote: > > Having the Web server execute ad hoc system > > administration code is far from elegant and user-friendly. > > With the right piece of code, you could create yourself > a setuid-apache shell and solve this problem once and for >

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Ron Adam
Barry Warsaw wrote: On Mar 22, 2010, at 02:02 PM, Ron Adam wrote: If I understand correctly, we would have the current mode as the default, and can trigger __pycache__ behavior simply by manually creating a __pycache__ directory and deleting any byte-code files in the module/program directory

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Greg Ewing
Antoine Pitrou wrote: Having the Web server execute ad hoc system administration code is far from elegant and user-friendly. With the right piece of code, you could create yourself a setuid-apache shell and solve this problem once and for all. :-) -- Greg __

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Greg Ewing
Antoine Pitrou wrote: Oh, and by the way, there can be a race condition between __pycache__ creation and deletion (if it fails the test) You can check whether the directory would be created with the right user beforehand, and if not, don't create one at all. To exploit a race condition there,

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Martin v. Löwis
> We now end with an alleged complete solution to a problem which doesn't > seem to exist, or is at least vastly overblown (the idea that having pyc > files along their source counterparts is a nuisance doesn't seem to be a > common grief against Python). > > I would really recommend reexamining i

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Le lundi 22 mars 2010 à 23:18 +0100, "Martin v. Löwis" a écrit : > If Apache creates a folder for me that I cannot remove, most likely, > there was a configuration error in the first place: common practice > tells that you should execute user code under user permissions, not as > www-data. I'm sur

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Martin v. Löwis
>> Why is it too late? Fix it, and get on. > > Sure, but it is annoying, and since it's the kind of things that noone > (including sysadmins) ever thinks about in advance, it's bound to repeat > itself > quite often. > > It's especially annoying, of course, if you have to ask someone else to rem

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Martin v. Löwis v.loewis.de> writes: > > Antoine Pitrou wrote: > > Martin v. Löwis v.loewis.de> writes: > >> If certain use cases > >> make it problematic (e.g. Apache creating directories which you then > >> cannot delete), there should be a way to turn it *off*. Perhaps the > >> existing machi

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread R. David Murray
On Mon, 22 Mar 2010 18:15:01 -, Antoine Pitrou wrote: > Isaac Morland uwaterloo.ca> writes: > > > > > IMO, all these issues militate for putting __pycache__ creation out of > > > the interpreter core, and in the hands of third-party package-time/ > > > install-time tools (or distutils). > >

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Martin v. Löwis
Antoine Pitrou wrote: > Martin v. Löwis v.loewis.de> writes: >> If certain use cases >> make it problematic (e.g. Apache creating directories which you then >> cannot delete), there should be a way to turn it *off*. Perhaps the >> existing machinery to turn of byte code generation at all might be

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Martin v. Löwis v.loewis.de> writes: > > If certain use cases > make it problematic (e.g. Apache creating directories which you then > cannot delete), there should be a way to turn it *off*. Perhaps the > existing machinery to turn of byte code generation at all might be > sufficient. Except tha

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Martin v. Löwis
Barry Warsaw wrote: > On Mar 22, 2010, at 06:15 PM, Antoine Pitrou wrote: > >> What I am proposing is that the creation of __pycache__ /directories/ be put >> outside of the core. It can be part of distutils, or of a separate module, or >> delegated to third-party tools. It could even be as simple

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Guido van Rossum
On Mon, Mar 22, 2010 at 12:20 PM, Barry Warsaw wrote: > On Mar 22, 2010, at 02:02 PM, Ron Adam wrote: > >>If I understand correctly, we would have the current mode as the default, and >>can trigger __pycache__ behavior simply by manually creating a __pycache__ >>directory and deleting any byte-cod

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Barry Warsaw
On Mar 22, 2010, at 02:02 PM, Ron Adam wrote: >If I understand correctly, we would have the current mode as the default, and >can trigger __pycache__ behavior simply by manually creating a __pycache__ >directory and deleting any byte-code files in the module/program directory. > >I like this, it i

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Barry Warsaw
On Mar 22, 2010, at 06:15 PM, Antoine Pitrou wrote: >What I am proposing is that the creation of __pycache__ /directories/ be put >outside of the core. It can be part of distutils, or of a separate module, or >delegated to third-party tools. It could even be as simple as >"python -m compileall --p

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Ron Adam
Antoine Pitrou wrote: Isaac Morland uwaterloo.ca> writes: IMO, all these issues militate for putting __pycache__ creation out of the interpreter core, and in the hands of third-party package-time/ install-time tools (or distutils). Speaking only for myself, but really for anybody who likes ti

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Isaac Morland uwaterloo.ca> writes: > > > IMO, all these issues militate for putting __pycache__ creation out of > > the interpreter core, and in the hands of third-party package-time/ > > install-time tools (or distutils). > > Speaking only for myself, but really for anybody who likes tidy sour

Re: [Python-Dev] __pycache__ creation

2010-03-22 Thread Isaac Morland
On Mon, 22 Mar 2010, Antoine Pitrou wrote: Oh, and by the way, there can be a race condition between __pycache__ creation and deletion (if it fails the test), where an attacker can stuff a hostile pyc file in the directory in the meantime (and the deletion then fails because the directory isn't

[Python-Dev] __pycache__ creation

2010-03-22 Thread Antoine Pitrou
Oh, and by the way, there can be a race condition between __pycache__ creation and deletion (if it fails the test), where an attacker can stuff a hostile pyc file in the directory in the meantime (and the deletion then fails because the directory isn't empty). IMO, all these issues militate fo