[Python-Dev] PEP 3147: PYC Repository Directories

PEP: 3147 Title: PYC Repository Directories Version: $Revision$ Last-Modified: $Date$ Author: Barry Warsaw Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2009-12-16 Python-Version: 3.2 Post-History: Abstract This PEP describes an extension to Python's import mech

[Python-Dev] PEP 3147: PYC Repository Directories

That still leaves the question of what to do with __file__ (for which even the solution in the PEP isn't particularly clean). Perhaps the thing to do there is to have __file__ always point to the source file and introduce a __file_cached__ that points to the bytecompiled file on disk (set to Non

[Python-Dev] PEP 3147: PYC Repository Directories

On Mon, Feb 1, 2010 at 11:56 PM, Pablo Mouzo wrote: > On Mon, Feb 1, 2010 at 10:23 PM, Paul Du Bois wrote: > [...] >> Sorry, I'm guilty of having assumed that the POSIX API has an >> operation analogous to win32 CreateFile(GENERIC_WRITE, 0 /* ie, >> "FILE_SHARE_NONE"*/). >> >> If shared-reader/si

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010/1/30 Barry Warsaw : > > Multiple file extensions > > > The PEP author also considered an approach where multiple thin byte > compiled files lived in the same place, but used different file > extensions to designate the Python version.  E.g. foo.pyc25, > foo.pyc26, foo.

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Vitor Bosshard writes: > foo.py > foo.pyc # < 2.7 or < 3.2 > foo.27.pyc > foo.32.pyc > etc. > > > This is simpler and more logical than the current subfolder proposal, > as it is clear which version each file corresponds to. Python can use > all the magic values it wants, but please don't spill t

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sat, Jan 30, 2010 at 8:21 PM, Vitor Bosshard wrote: > Putting the files into a separate dir also makes it much harder to > work with external tools; e.g. VCSes already ignore .pyc and .pyo > files, but not unknown directories. > Can't a VCS be configured to ignore a .pyr directory just as eas

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Ben Finney wrote: Vitor Bosshard writes: foo.py foo.pyc # < 2.7 or < 3.2 foo.27.pyc foo.32.pyc etc. This is simpler and more logical than the current subfolder proposal, as it is clear which version each file corresponds to. Python can use all the magic values it wants, but please don't spil

Re: [Python-Dev] PEP 3147: PYC Repository Directories

MRAB wrote: > One thing that puzzles me is that the PEP shows that "Python -U" has a > different magic number from "Python", but I can't find any reference to > "-U" (the options appear to be case-sensitive, so presumably it's not > the same as "-u"). We deliberate don't document -U because its ty

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Daniel Stutzbach wrote: > On Sat, Jan 30, 2010 at 8:21 PM, Vitor Bosshard > wrote: > > Putting the files into a separate dir also makes it much harder to > work with external tools; e.g. VCSes already ignore .pyc and .pyo > files, but not unknown directories

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Vitor Bosshard wrote: > Why not: > > foo.py > foo.pyc # < 2.7 or < 3.2 > foo.27.pyc > foo.32.pyc > etc. > > > This is simpler and more logical than the current subfolder proposal, > as it is clear which version each file corresponds to. Python can use > all the magic values it wants, but please

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010/1/31 Nick Coghlan : > Vitor Bosshard wrote: >> Why not: >> >> foo.py >> foo.pyc # < 2.7 or < 3.2 >> foo.27.pyc >> foo.32.pyc >> etc. >> >> >> This is simpler and more logical than the current subfolder proposal, >> as it is clear which version each file corresponds to. Python can use >> all th

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Nick Coghlan writes: > This is also the reason why using the version number in the filename > isn't adequate - the magic number can change due to more than just the > Python version changing. Also, if we don't change the bytecode for a > given release, then multiple versions can use the same magi

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Nick Coghlan writes: > It won't be cluttered with subfolders - you will have at most one .pyr > per source .py file. If that doesn't meet your threshold of “cluttered with subfolders”, I'm at a loss for words to think where that threshold might be. It meets, and exceeds by a long shot, my thresh

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010/1/31 Nick Coghlan : >> Can't a VCS be configured to ignore a .pyr directory just as easily as >> it can be configured to ignore a .pyc file? > > Yes they can. Of course they can, but not out of the box. It was just an example off the top of my head. A trickier case: My GUI app offers scrip

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sat, Jan 30, 2010 at 8:22 PM, Vitor Bosshard wrote: > 2010/1/31 Nick Coghlan : > >>> Can't a VCS be configured to ignore a .pyr directory just as easily as >>> it can be configured to ignore a .pyc file? >> >> Yes they can. > > > Of course they can, but not out of the box. It was just an exampl

Re: [Python-Dev] PEP 3147: PYC Repository Directories

+1 overall. I'm certainly not concerned with replacing pyc clutter with pyr clutter. I do like that you haven't _increased_ the number of extraneous siblings of .py files. I have a couple bikesheddy or "why didn't you do this" comments. I'll be perfectly satisfied with an answer or a line in the p

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sat, 30 Jan 2010 19:00:05 -0500, Barry Warsaw wrote: > Linux distributions such as Ubuntu [2]_ and Debian [3]_ provide more > than one Python version at the same time to their users. For example, > Ubuntu 9.10 Karmic Koala can install Python 2.5, 2.6, and 3.1, with > Python 2.6 being the defau

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sat, 30 Jan 2010 20:37:32 -0800, Jeffrey Yasskin wrote: > On Sat, Jan 30, 2010 at 8:22 PM, Vitor Bosshard wrote: > > A trickier case: My GUI app offers scripting facilities. The > > associated open file dialog hides all .pyc files, and users select > > just from .py files. if subfolders are in

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Ben Finney wrote: > Could we instead have a single subdirectory for each tree of module > packages, keeping them tidily out of the way of the source files, while > making them located just as deterministically:: Not easily. With the scheme currently proposed in the PEP, setting a value for __file_

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Vitor Bosshard wrote: >> There is no one-to-one correspondence between Python version and pyc >> magic numbers. Different runtime options may change the magic number and >> different versions may reuse a magic number > > Good point. Runtime options would need to change the version (e.g. > foo.25U.

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Ben Finney wrote: > Nick Coghlan writes: > >> It won't be cluttered with subfolders - you will have at most one .pyr >> per source .py file. > > If that doesn't meet your threshold of “cluttered with subfolders”, I'm > at a loss for words to think where that threshold might be. It meets, > and e

Re: [Python-Dev] PEP 3147: PYC Repository Directories

I like the idea of the PEP. On the other hand, I dislike using directories for it. Others have explained enough reasons for why creating many directories is a bad idea; and there may be other reasons (file-system limits for number of directories, problems when the directories are located on the ne

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Henning von Bargen wrote: > I like the idea of the PEP. > On the other hand, I dislike using directories for it. > Others have explained enough reasons for why creating many > directories is a bad idea; and there may be other reasons > (file-system limits for number of directories, problems when >

Re: [Python-Dev] PEP 3147: PYC Repository Directories

>> Linux distributions such as Ubuntu [2]_ and Debian [3]_ provide more >> than one Python version at the same time to their users. For example, >> Ubuntu 9.10 Karmic Koala can install Python 2.5, 2.6, and 3.1, with >> Python 2.6 being the default. >> >> In order to ease the burden on operating sy

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Nick Coghlan writes: > Actually, this is the first post I've seen noting objective problems > with the use of a subdirectory. The others were just a subjective > difference in perspective that saw subdirectory clutter as somehow > being worse than file clutter. Here's another one, then: The dir

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> Agreed this should be discussed in the PEP, but one obvious problem is > the speed impact. Picking up a file from a subdirectory is going to > introduce less overhead than unpacking it from a zipfile. There is also the issue of race conditions with multiple simultaneous accesses. The original fo

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Barry Warsaw python.org> writes: > > Putting Python version numbers in the packages would be a > maintenance nightmare, since all the packages - *and their > dependencies* - would have to be updated every time a new Python > release was added or removed from the distribution. Because of the > sh

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 07:29, schrieb Nick Coghlan: > Vitor Bosshard wrote: >>> There is no one-to-one correspondence between Python version and pyc >>> magic numbers. Different runtime options may change the magic number and >>> different versions may reuse a magic number >> >> Good point. Runtime options

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 05:18, schrieb Ben Finney: > Nick Coghlan writes: > >> It won't be cluttered with subfolders - you will have at most one .pyr >> per source .py file. > > If that doesn't meet your threshold of “cluttered with subfolders”, I'm > at a loss for words to think where that threshold migh

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 07:18, schrieb Nick Coghlan: > Ben Finney wrote: >> Could we instead have a single subdirectory for each tree of module >> packages, keeping them tidily out of the way of the source files, while >> making them located just as deterministically:: > > Not easily. With the scheme curren

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 10:21, schrieb Ben Finney: > Nick Coghlan writes: > >> Actually, this is the first post I've seen noting objective problems >> with the use of a subdirectory. The others were just a subjective >> difference in perspective that saw subdirectory clutter as somehow >> being worse than

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 6:04 AM, Jeffrey Yasskin wrote: > 2. Vitor's suggestion to make 1 "pyr" directory per directory and > stick all the .pyc's there would solve the "pyc clutter" problem. Any > reason not to do that? Trying to make it 1-pyr-per-directory-hierarchy > as Ben suggested seems unwo

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 1:54 PM, Hanno Schlichting wrote: > I'd be a big +1 to using a single ".pyr" directory per source directory. I don't know whether I in favour of using a single pyr folder or not but if a single folder is used I'd definitely prefer the folder to be called __pyr__ rather tha

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross wrote: > I don't know whether I in favour of using a single pyr folder or not > but if a single folder is used I'd definitely prefer the folder to be > called __pyr__ rather than .pyr. Do you have any specific reason for that? Using the leading dot no

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 13:03, schrieb Simon Cross: > On Sun, Jan 31, 2010 at 1:54 PM, Hanno Schlichting wrote: >> I'd be a big +1 to using a single ".pyr" directory per source directory. > > I don't know whether I in favour of using a single pyr folder or not > but if a single folder is used I'd definitel

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 2:13 PM, Hanno Schlichting wrote: > On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross > wrote: >> I don't know whether I in favour of using a single pyr folder or not >> but if a single folder is used I'd definitely prefer the folder to be >> called __pyr__ rather than .pyr. >

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 13:29, schrieb Simon Cross: > On Sun, Jan 31, 2010 at 2:13 PM, Hanno Schlichting wrote: >> On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross >> wrote: >>> I don't know whether I in favour of using a single pyr folder or not >>> but if a single folder is used I'd definitely prefer the fol

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Georg Brandl wrote: > Am 31.01.2010 07:18, schrieb Nick Coghlan: >> Ben Finney wrote: >>> Could we instead have a single subdirectory for each tree of module >>> packages, keeping them tidily out of the way of the source files, while >>> making them located just as deterministically:: >> Not easily

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Georg Brandl wrote: > Then why did Subversion choose to follow the CVS way and create a > subdirectory in each versioned directory? IMO, this is much more > annoying given the alternative of a single .hg/.bzr/whatever directory. > For .pyc vs .pyr, you didn't have the alternative of putting all th

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010/1/31 Georg Brandl : >> >> foo.py >> foo.pyr/ >>   cpython-25.pyc >>   cpython-25U.pyc >>   cpython-27.pyc >>   cpython-27U.pyc >>   cpython-32.pyc >>   unladen-011.pyc >>   wpython-11.pyc > > +1.  It should be quite easy to assign a new name every time the magic > number is updated. > >> If we

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Georg Brandl wrote: > +1. Having a single (visible) __pyr__ directory is much less clutter than > multiple .pyc files anyway. Also, don't forget Windows users, for whom > the dot convention doesn't mean anything. I must admit I quite like the __pyr__ directory approach as well. Since the interpr

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Vitor Bosshard wrote: > Optimizing disk space (and marginal compile time) is not worth the > mental overhead this would introduce. Better keep it as clear and > simple as possible, i.e. create different .pyc files even if the > bytecode doesn't change between releases. Yeah, makes sense. Given the

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010/1/31 Nick Coghlan : > Georg Brandl wrote: >> Then why did Subversion choose to follow the CVS way and create a >> subdirectory in each versioned directory?  IMO, this is much more >> annoying given the alternative of a single .hg/.bzr/whatever directory. >> For .pyc vs .pyr, you didn't have th

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Am 31.01.2010 14:02, schrieb Nick Coghlan: > Georg Brandl wrote: >> Then why did Subversion choose to follow the CVS way and create a >> subdirectory in each versioned directory? IMO, this is much more >> annoying given the alternative of a single .hg/.bzr/whatever directory. >> For .pyc vs .pyr,

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, 31 Jan 2010 13:13:24 +0100, Georg Brandl wrote: > Am 31.01.2010 13:03, schrieb Simon Cross: > > On Sun, Jan 31, 2010 at 1:54 PM, Hanno Schlichting > > wrote: > >> I'd be a big +1 to using a single ".pyr" directory per source directory. > > > > I don't know whether I in favour of using a

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, 31 Jan 2010 09:50:16 +0100, =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= wrote: > >> Linux distributions such as Ubuntu [2]_ and Debian [3]_ provide more > >> than one Python version at the same time to their users. For example, > >> Ubuntu 9.10 Karmic Koala can install Python 2.5, 2.6, an

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Le Sun, 31 Jan 2010 18:46:54 +1000, Nick Coghlan a écrit : > > Actually, this is the first post I've seen noting objective problems > with the use of a subdirectory. The others were just a subjective > difference in perspective that saw subdirectory clutter as somehow being > worse than file clutt

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Le Sat, 30 Jan 2010 21:04:14 -0800, Jeffrey Yasskin a écrit : > > I have a couple bikesheddy or "why didn't you do this" comments. I'll be > perfectly satisfied with an answer or a line in the pep. > > 1. Why the -R flag? It seems like this is a uniform improvement, so it > should be the default.

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 5:00 AM, "Martin v. Löwis" wrote: >> Agreed this should be discussed in the PEP, but one obvious problem is >> the speed impact. Picking up a file from a subdirectory is going to >> introduce less overhead than unpacking it from a zipfile. > > There is also the issue of rac

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Vitor Bosshard wrote: 2010/1/31 Georg Brandl : foo.py foo.pyr/ cpython-25.pyc cpython-25U.pyc cpython-27.pyc cpython-27U.pyc cpython-32.pyc unladen-011.pyc wpython-11.pyc +1. It should be quite easy to assign a new name every time the magic number is updated. If we don't change

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 8:34 AM, Nick Coghlan wrote: > That still leaves the question of what to do with __file__ (for which > even the solution in the PEP isn't particularly clean). Perhaps the > thing to do there is to have __file__ always point to the source file > and introduce a __file_cached

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> By the way, the part that caused me the most confusion in the language > in the PEP was the emphasized *and their dependencies*, as if a package > having dependencies somehow turned the problem into a factorial explosion. > But there seems to be nothing special, according to your explanation, > a

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 6:04 AM, Jeffrey Yasskin wrote: > 1. Why the -R flag? It seems like this is a uniform improvement, so it > should be the default. Have faith in your design! ;-) +1 for a single strategy that is used in all cases. The current solution could be phased out across multiple rel

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> This would eliminate the read-time race condition but still > potentially allow for a write-time race condition if locking isn't > used. The benefit of this approach is that it is no less clear than > pyc is today and doesn't result in n * versions_of_python pyc files. > There is still the overhe

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Reid Kleckner wrote: On Sun, Jan 31, 2010 at 8:34 AM, Nick Coghlan wrote: That still leaves the question of what to do with __file__ (for which even the solution in the PEP isn't particularly clean). Perhaps the thing to do there is to have __file__ always point to the source file and introduce

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Jan 30, 2010, at 4:00 PM, Barry Warsaw wrote: > Abstract > > > This PEP describes an extension to Python's import mechanism which > improves sharing of Python source code files among multiple installed > different versions of the Python interpreter. +1 > It does this by > allowin

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Sense this is something new, I believe it is helpful to look at all the possibilities so it doesn't become something we regret we did later. This is something that once it gets put in place may be real hard to get rid of. So here are a few questions that I think haven't seen asked yet. Wh

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Martin v. Löwis schrieb: There is also the issue of race conditions with multiple simultaneous accesses. The original format for the PEP had race conditions for multiple simultaneous writers; ZIP will also have race conditions for concurrent readers/writers (as any new writer will have to overwri

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On 1/31/2010 8:58 AM, Vitor Bosshard wrote: Having one single pyr (or__pycache__ or whatever it's called) subfolder per folder is an easy to understand, solid solution. As a user who browses directories to see what is there and to find files to open and look at, I would like this. The near-du

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Whoa. This thread already exploded. I'm picking this message to respond to because it reflects my own view after reading the PEP. On Sun, Jan 31, 2010 at 4:13 AM, Hanno Schlichting wrote: > On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross > wrote: >> I don't know whether I in favour of using a singl

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On 1/31/2010 8:34 AM, Nick Coghlan wrote: Georg Brandl wrote: +1. Having a single (visible) __pyr__ directory is much less clutter than multiple .pyc files anyway. Also, don't forget Windows users, for whom the dot convention doesn't mean anything. I must admit I quite like the __pyr__ dire

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Silke von Bargen wrote: That still leaves the question of what to do with __file__ (for which even the solution in the PEP isn't particularly clean). Perhaps the thing to do there is to have __file__ always point to the source file and introduce a __file_cached__ that points to the bytecompiled

Re: [Python-Dev] PEP 3147: PYC Repository Directories

>> foo.pyr/ >> cpython-25.pyc >> cpython-25U.pyc >> cpython-27.pyc >> cpython-27U.pyc >> cpython-32.pyc >> unladen-011.pyc >> wpython-11.pyc > > +1. It should be quite easy to assign a new name every time the magic > number is updated. It would actually be possible to drop the magi

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> Exactly. How would you define where the pyr folder goes? At the root > of a package? What if I delete the __init__.py file there? Will the > existing pyr folder be orphaned and a new one created in each > subfolder? Unlike VCS working copies, the package / module / script > hierarchy is not forma

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> Not really -- much of the code I've seen that tries to guess the source > file name from a __file__ value just does something like this: > >if fname.lower().endswith(('.pyc', '.pyo')): fname = fname[:-1] > > That's not compatible with using .pyr, either. If a single pyc folder is used, I t

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On 1 February 2010 00:34, Nick Coghlan wrote: > > __file__ would always point to the source files > __file_cached__ would always point to the relevant compiled file (either > pre-existing or newly created) > > > I like this solution combined with having a single cache directory and a few other th

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Tim Delaney: > I like this solution combined with having a single cache directory and a few > other things I've added below. > ... > 2. /tmp is often on non-volatile memory. If it is (e.g. my Windows system > temp dir is on a RAMdisk) then it seems wise to respect the obvious desire > to throw awa

Re: [Python-Dev] PEP 3147: PYC Repository Directories

>I can also see advantages to allowing out of tree compiled cache > directories. For example, you could have a locked down .py tree with > .pycs going into per-user trees. This prevents another user from > spoofing a .pyc I use as well as allowing users to install arbitrary > versions of Python

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On 1/31/2010 2:04 PM, Raymond Hettinger wrote: > On Jan 30, 2010, at 4:00 PM, Barry Warsaw wrote: >> It does this by >> allowing many different byte compilation files (.pyc files) to be >> co-located with the Python source file (.py file). > > It would be nice if all the compilation files could

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 11:16 AM, Terry Reedy wrote: > > 'pycache' would be pretty clear. > Heh -- without the underscores, I read this as "pyc ache". Seems appropriate. -- Curt Hagenlocher c...@hagenlocher.org ___ Python-Dev mailing list Python-Dev@py

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On 1/31/2010 4:26 PM, Tim Delaney wrote: The pyc/pyo files are just an optimisation detail, and are essentially temporary. The .pycs for /Lib and similar are*not* temporarily in the sense you are using. They are effectively permanent for as long as the version is installed. They should *n

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, 31 Jan 2010 19:48:19 +0100, =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= wrote: > > By the way, the part that caused me the most confusion in the language > > in the PEP was the emphasized *and their dependencies*, as if a package > > having dependencies somehow turned the problem into a fa

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Antoine Pitrou wrote: > Le Sat, 30 Jan 2010 21:04:14 -0800, Jeffrey Yasskin a écrit : >> I have a couple bikesheddy or "why didn't you do this" comments. I'll be >> perfectly satisfied with an answer or a line in the pep. >> >> 1. Why the -R flag? It seems like this is a uniform improvement, so it

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Martin v. Löwis wrote: >> Exactly. How would you define where the pyr folder goes? At the root >> of a package? What if I delete the __init__.py file there? Will the >> existing pyr folder be orphaned and a new one created in each >> subfolder? Unlike VCS working copies, the package / module / scri

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Silke von Bargen wrote: > >> That still leaves the question of what to do with __file__ (for which >> even the solution in the PEP isn't particularly clean). Perhaps the >> thing to do there is to have __file__ always point to the source file >> and introduce a __file_cached__ that points to the b

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> But I don't understand how this answers the question. If the > python26-zope.sendmail package doesn't run setup.py, then a > python-zope.sendmail package where you specify at install time which > directory to install the files to isn't going to run setup.py, either. > If the only difference betw

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Nick Coghlan writes: > Would you still be a -1 on making it the new scheme the default if it > used a single cache directory instead? That would actually be cleaner > than the current solution rather than messier. +0 on a default of “store compiled bytecode files in a single cache directory”. It

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> 3. In each top level directory on sys.path, shadow file heirarchy > Major Pro: trivial to separate out all cached files > Major Con: ??? (I got nuthin') The major con of this option (and option 2) is an ambiguity of where to look for in case of packages. In particular for namespace packages

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Hanno Schlichting wrote: >+1 for a single strategy that is used in all cases. The current >solution could be phased out across multiple releases, but in the end >there should be a single approach and no flag. Otherwise some code and >tools will only support one of the approaches, especially if thi

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> Would you still be a -1 on making it the new scheme the default if it > used a single cache directory instead? That would actually be cleaner > than the current solution rather than messier. Well, I guess no, although additional directories are always more intrusive than additional files (visua

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Raymond Hettinger wrote: > > On Jan 30, 2010, at 4:00 PM, Barry Warsaw wrote: >> Abstract >> >> >> This PEP describes an extension to Python's import mechanism which >> improves sharing of Python source code files among multiple installed >> different versions of the Python interpreter. >

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 11:16, Guido van Rossum wrote: > Whoa. This thread already exploded. I'm picking this message to > respond to because it reflects my own view after reading the PEP. > > On Sun, Jan 31, 2010 at 4:13 AM, Hanno Schlichting wrote: >> On Sun, Jan 31, 2010 at 1:03 PM, Simon Cros

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 11:04, Raymond Hettinger wrote: > > On Jan 30, 2010, at 4:00 PM, Barry Warsaw wrote: >> Abstract >> >> >> This PEP describes an extension to Python's import mechanism which >> improves sharing of Python source code files among multiple installed >> different versio

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Le Mon, 01 Feb 2010 11:35:19 -0800, Brett Cannon a écrit : > > As others have said, an uncompressed zip file could work here. Or even a > file format where the first 4 bytes is the timestamp and then after that > are chunks of length-of-bytecode|magic|bytecode. That allows for opening > a file in

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> And I disagree this would be difficult as the PEP suggests given the > proper file format. For zip files zipimport already has the read code > in C; it just would require the code to write to a zip file. And as > for the format I mentioned above, that's dead-simple to implement. How do you write

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Mon, Feb 1, 2010 at 13:19, "Martin v. Löwis" wrote: >> And I disagree this would be difficult as the PEP suggests given the >> proper file format. For zip files zipimport already has the read code >> in C; it just would require the code to write to a zip file. And as >> for the format I mention

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> On Mon, Feb 1, 2010 at 13:19, "Martin v. Löwis" wrote: >> How do you write to a zipfile while others are reading it? On Mon, Feb 1, 2010 at 1:23 PM, Brett Cannon wrote: > By hating concurrency (i.e. I don't have an answer which kills my idea). The python I use (win32 2.6.2) does not complain

Re: [Python-Dev] PEP 3147: PYC Repository Directories

> The python I use (win32 2.6.2) does not complain if it cannot read > from or write to a .pyc; and thus it handles multiple python processes > trying to create .pyc files at the same time. Is the .zip case really > any different? Since .pyc files are an optimization, it seems natural > and correct

Re: [Python-Dev] PEP 3147: PYC Repository Directories

>> The python I use (win32 2.6.2) does not complain if it cannot read >> from or write to a .pyc; and thus it handles multiple python processes >> trying to create .pyc files at the same time. Is the .zip case really >> any different? [ snip discussion of difficulty of writing a sharing-safe updat

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross wrote: I don't know whether I in favour of using a single pyr folder or not but if a single folder is used I'd definitely prefer the folder to be called __pyr__ rather than .pyr. Guido van Rossum wrote: Exactly what I would prefer. I worry that ha

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 12:44:33PM +0100, Georg Brandl wrote: > At least to me, this does not explain why an "unwanted" (why unwanted? If > it's unwanted, set PYTHONDONTWRITEBYTECODE=1) directory is worse than an > "unwanted" file. A directory "feels" different than. For example, typing "ls" in m

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Nick Coghlan wrote: > Henning von Bargen wrote: >> The solution is so obvious: >> >> Why not use a .pyr file that is internally a zip file? I think a Zip file might be the right approach too. Either you could have directories in the zip file for each version, e.g. 2.7/foo.pyc 3.3/foo.p

Re: [Python-Dev] PEP 3147: PYC Repository Directories

Argh. zipfiles are way to complex to be writing. If you want to use zipfiles, compile your whole world ahead of time, stuff it into a zipfile, and install / distribute that. But for the automatic writing of bytecode files as a side effect of importing the source code, please let the filesystem do i

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On approximately 2/2/2010 4:28 PM, came the following characters from the keyboard of Guido van Rossum: Argh. zipfiles are way to complex to be writing. Agreed. But in reading that, it somehow triggered a question: does zipimport only work for zipfiles, or does it work for any archive format

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Tue, Feb 2, 2010 at 5:41 PM, Glenn Linderman wrote: > On approximately 2/2/2010 4:28 PM, came the following characters from the > keyboard of Guido van Rossum: >> >> Argh. zipfiles are way to complex to be writing. > > Agreed.  But in reading that, it somehow triggered a question: does > zipimp

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Sun, Jan 31, 2010 at 11:16 AM, Guido van Rossum wrote: > Whoa. This thread already exploded. I'm picking this message to > respond to because it reflects my own view after reading the PEP. > > On Sun, Jan 31, 2010 at 4:13 AM, Hanno Schlichting wrote: >> On Sun, Jan 31, 2010 at 1:03 PM, Simon C

Re: [Python-Dev] PEP 3147: PYC Repository Directories

I have to say up front that I'm somewhat shocked at how quickly this thread has exploded! Since I'm sprinting this week, I haven't thoroughly read every message and won't have time tonight to answer every question, but I'll try to pick out some common ideas. I really appreciate everyone's input a

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Jan 30, 2010, at 11:21 PM, Vitor Bosshard wrote: >Why not: > >foo.py >foo.pyc # < 2.7 or < 3.2 >foo.27.pyc >foo.32.pyc >etc. Because this clutters the module's directory more than it does today, which I considered to be a negative factor. And as others have pointed out, there isn't a one-to-o

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Jan 31, 2010, at 01:44 PM, Nick Coghlan wrote: >We deliberate don't document -U because its typical effect is "break the >world" - it makes all strings unicode in 2.x. As an aside, I think this should be documented *somewhere* other than just in import.c! I'd totally forgotten about it until

Re: [Python-Dev] PEP 3147: PYC Repository Directories

On Jan 31, 2010, at 03:07 PM, Ben Finney wrote: >In other words, my understanding is that the current PEP would have the >following tree for an example project:: > >foo/ >__init__.py >__init__.pyr/ >deadbeef.pyc >decafbad.pyc >lorem.py >l

  1   2   >