Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 13/08/2010 10:02 PM, Michael Foord wrote:
On 13/08/2010 06:39, Stephen J. Turnbull wrote:
Michael Foord writes:
> How is ~/python not memorable or consistent? (And cross-platform
> memorability and consistency is valuable too.)
But what does "~" mean on Windows?
There is a "user directory" in Windows directly analagous to ~, and this
is the path returned by os.path.expanduser('~') in Python:
Well, see my post early on in the thread about the
various senses of "user directory" under Windows.
The addition of expanduser to include Windows caused
a long debate at the time. (ISTR)
I'm not really all that bothered for this purpose.
There's something to be said for all the suggestions
so far. However, as I've said elsewhere, I'm more
concerned that whatever we end up choosing for
location(s) be clearly documented as such.
TJG
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 13/08/2010 06:39, Stephen J. Turnbull wrote:
Michael Foord writes:
> How is ~/python not memorable or consistent? (And cross-platform
> memorability and consistency is valuable too.)
But what does "~" mean on Windows?
There is a "user directory" in Windows directly analagous to ~, and this
is the path returned by os.path.expanduser('~') in Python:
>>> import os
>>> os.path.expanduser('~')
'C:\\Users\\michael'
(Windows 7)
Inside of Python you can have a
consistent definition, but that doesn't help people whose installer
gets mixed signals so Python itself is in the wrong place, or who are
using external tools (uninstaller, editor, application, etc) that work
on or with Python.
I'm not arguing for use of AppData, but at least it's easy to explain,
and easy to implement, for consistency.
Likewise with the user directory (IMO) - in fact to explain where
AppData is to a Windows user you first explain about the user directory
(AppData is under the user directory).
Personally I would like to see the defaults being:
Linux: ~/.pythonx.y
Mac OS X: ~/.pythonx.y with a fallback of ~/Library/Preferences/.pythonx.y
Windows: ~/pythonx.y perhaps with a backup of AppData/pythonx.y
For both Windows and Mac OS X I would be happy with the fallback /
primary to be the other way round - it doesn't *really* matter. The API
for getting the user config direction should always return a list I
guess if we have fallbacks.
Someone else in this thread (Adal Chiriliuc) expressed a preferences for
the documents folder on Windows over the home directory. This would be
fine as well (with AppData still as a fallback). This is used by other
native windows applications. (In earlier versions of Windows the
documents folder was explicitly called "Documents and Settings".)
We should use pythonx.y rather than just python because (for example)
you will typically have different packages installed when you have
multiple versions of Python, and in unittest would then want / need
different plugins enabled and configured in the unittest config files
for each of the versions of Python.
I've added this text to the issue and Tarek - as bdfl for distutils2
should just make a decision (and the rest of us will have to live with
it). :-)
> Another issue is discoverability. Many users won't know about these
> config files unless they *see* them.
OK, maybe AppData's out.
> In fact for Windows the *correct* thing to do is probably to use
> the registry and then provide a graphical tool for editing the
> values. If you're arguing for consistency why not argue for this?
Sounds reasonable to me. Except Python's standard GUI is probably not
up to Windows standards of beauty
Heh, I was actually being slightly sarcastic - just pointing out that if
we are going to argue that we should do the "correct" thing for the
platform that means the registry on Windows and plists on Mac OS X, with
GUI apps to control them. That is an unnecessary burden for the standard
library and actively makes them harder for developers to maintain
themselves.
All the best,
Michael Foord
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog
READ CAREFULLY. By accepting and reading this email you agree, on behalf of
your employer, to release me from all obligations and waivers arising from any
and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap,
clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and
acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in perpetuity, without
prejudice to my ongoing rights and privileges. You further represent that you
have the authority to release me from any BOGUS AGREEMENTS on behalf of your
employer.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, Aug 13, 2010 at 3:21 PM, John Arbash Meinel wrote: > I don't know what the specific issue is here, but adding entries to > sys.path makes startup time *significantly* slower. > > I happen to use easy_install since Windows doesn't have its own package > manager. Unfortunately the default of creating a new directory and > adding it to easy_install.pth is actually pretty terrible. Adding sys.path entries on Linux isn't free either. Fortunately, this isn't about adding anything to sys.path. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ... > * that said, Windows seems much slower than Linux on equivalent >hardware, perhaps attempting to open files is intrinsically more >expensive there? Certainly it's not safe to assume conclusions drawn >on Linux will apply equally well on Windows, or vice versa. I don't know what the specific issue is here, but adding entries to sys.path makes startup time *significantly* slower. I happen to use easy_install since Windows doesn't have its own package manager. Unfortunately the default of creating a new directory and adding it to easy_install.pth is actually pretty terrible. On my system, 'len(sys.path)' is 72 entries long. 62 of that is from easy-install. A huge amount of that is all the zope and lazr. dependencies that are needed by launchpadlib (not required for bzr itself.) With a fully hot cache, and running the minimal bzr command: time bzr rocks --no-plugins real 0m0.395s vs real 0m0.195s So about 400ms to startup versus 200ms if I use the packaged version of bzr (which has a very small search path). John =:-> -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkxlm0kACgkQJdeBCYSNAAMSEgCfW24XNG3h20UkFdEODNMob6uR nisAoLes/usoHd1YRDIkzxfIJohPjSer =YO9b -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, 13 Aug 2010 11:57:57 -0400 Barry Warsaw wrote: > On Aug 12, 2010, at 09:10 AM, Fred Drake wrote: > > >Perhaps user configuration belongs in ~/.local/, or ~/.local/python/ > >(with attendant Windows & Mac OS noises); I don't really care where it > >lands, because right now we just have a mess. Getting it "right" with > >respect to Window's "roaming" notion and how people expect to work > >with it is... unlikely. Picking a place is better than not, so we > >know where to find things. But that's all it buys us. > > I've missed most of this discussion while on vacation, but if ~/.local is > supposed to mirror /usr/local, then wouldn't a logical place for per-user > configuration files be ~/.local/etc/whatever.cfg? I think this has already been debated, -1 on it. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, Aug 13, 2010 at 11:57 AM, Barry Warsaw wrote: > I've missed most of this discussion while on vacation, but if ~/.local is > supposed to mirror /usr/local, then wouldn't a logical place for per-user > configuration files be ~/.local/etc/whatever.cfg? Maybe it is; I'd hope so. The fd.o spec that describes ~/.local/share/ doesn't say anything *else* about ~/.local/, though. So our expansion of ~/.local/ is very much our own. My own ~/.local/ contains a bunch of gnomish desktop stuff in share/ and Python stuff in lib/, and that's it. I'd be as happy as I'm going to be with configs in ~~/.local/etc/. (Still going to be a grouchy old fart.) -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Aug 12, 2010, at 09:10 AM, Fred Drake wrote: >Perhaps user configuration belongs in ~/.local/, or ~/.local/python/ >(with attendant Windows & Mac OS noises); I don't really care where it >lands, because right now we just have a mess. Getting it "right" with >respect to Window's "roaming" notion and how people expect to work >with it is... unlikely. Picking a place is better than not, so we >know where to find things. But that's all it buys us. I've missed most of this discussion while on vacation, but if ~/.local is supposed to mirror /usr/local, then wouldn't a logical place for per-user configuration files be ~/.local/etc/whatever.cfg? -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Thu, Aug 12, 2010 at 11:48 PM, Nick Coghlan wrote: > 2010/8/12 Éric Araujo : >>> Choosing an arbitrary location we think is good on every system is fine >>> and non risky I think, as long as Python let the various distribution >>> change those paths though configuration. >> >> Don’t you have a bootstrapping problem? How do you know where to look at >> the sysconfig file that tells where to look at config files? Not if located in a place known/owned by the interpreter whatever the layout is. > > Personally, I'm not clear on what a separate syconfig.cfg file offers > over clearly separating the directory configuration settings and > continuing to have distributions patch sysconfig.py directly. The > bootstrapping problem (which would encourage classifying synconfig.cfg > as source code and placing it alongside syscongig.py) is a major part > of that point of view. Sure, sysconfig.cfg would be part of the distribution, and this is not really different from code from our core point of view. But it seems more appealing to give the ability to change installation locations through configuration rather than by patching the code, because the latter also implies that Python behaves differently when patched, and add more maintenance burden for everybody. For us for instance, it would be more comfortable to keep most content in sysconfig private, so we can change them at ease without breaking distributions that patches it. I would hate to have to do a deprecation cycle if we change the way sysconfig internally works. A documented cfg file feels just more standard-ish to me for this. Regards. Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
Michael Foord writes: > How is ~/python not memorable or consistent? (And cross-platform > memorability and consistency is valuable too.) But what does "~" mean on Windows? Inside of Python you can have a consistent definition, but that doesn't help people whose installer gets mixed signals so Python itself is in the wrong place, or who are using external tools (uninstaller, editor, application, etc) that work on or with Python. I'm not arguing for use of AppData, but at least it's easy to explain, and easy to implement, for consistency. > Another issue is discoverability. Many users won't know about these > config files unless they *see* them. OK, maybe AppData's out. > In fact for Windows the *correct* thing to do is probably to use > the registry and then provide a graphical tool for editing the > values. If you're arguing for consistency why not argue for this? Sounds reasonable to me. Except Python's standard GUI is probably not up to Windows standards of beauty ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, Aug 13, 2010 at 7:29 AM, Antoine Pitrou wrote: > On Thu, 12 Aug 2010 18:14:44 -0400 > Glyph Lefkowitz wrote: >> >> On Aug 12, 2010, at 6:30 AM, Tim Golden wrote: >> >> > I don't care how many stats we're doing >> >> You might not, but I certainly do. And I can guarantee you that the >> authors of command-line tools that have to start up in under ten >> seconds, for example 'bzr', care too. > > The idea that import time is dominated by stat() calls sounds rather > undemonstrated (and unlikely) to me. It may be, depending on what you import. I certainly have seen (and profiled) it. In my experience, stat calls and regex compilation often come at the top of the culprits for slow imports. In the case of setuptools namespace package, there was a thread on 23rd april on distutils-sig about this issue: most of the slowdown came from unneeded stat (and symlink translations). cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, Aug 13, 2010 at 03:15:28AM +0200, Éric Araujo wrote:
> > A good alternative would be to make the config file overridable. That way
> > you can have sysconfig.cfg next to sysconfig.py or in a known config
> > directory relative to the python stdlib install but also let the
> > distributions and individual sites override the defaults by making changes
> > to /etc/python3/sysconfig.cfg for instance.
>
> Putting configuration files next to source code does not agree with the
> FHS.
>
That depends -- sometimes a config file is actually a data file. In
following the FHS, a linux distribtuion and system administrators want to
make sure a sysadmin never needs to modify a file outside of /etc for
configuration. However, if you had a config file that is installed with
python and not meant to be modified next to the code and the sysadmin is
able to customize python's behaviour by copying lines from that file to
a file in /etc/ that they do modify, that's perfectly fine. Where
developers often run afoul of the FHS is creating a configuration option
that does not live in a file in /etc/ (either because they think it's data
but system administrators routinely want to modify.it or because they don't
care about the FHS and just want to put everything in a single directory)
and hte system admin has no means to set that option permanently but to
modify the files outside of /etc/.
>
> Your message gave me an idea though: Some function could be called
> in sitecustomize (symlinked from /etc on Debian e.g.) to set the path to
> sysconfig.cfg.
>
I hope that you mean that the actual file lives in /etc/ and then a symlink
in the stdlib points to it. That sounds acceptable from an FHS view but
it's a bad idea. Most of the config files in /etc/ are declarative. A few
of the files in /etc/ are imperative (for instance, init scripts) and
distributions frequently debate whether these are actually config files or
files misplaced in /etc/ for historical reasons. Fedora, for instance, does
not mark initscripts as config -- all configuration of initscripts are done
by setting shell variables in a different file which is marked config.
(Files marked config in a package manager generally are treated differently
on upgrades. If the files have been modified since install, they are either
not replaced or, when replaced, a backup is made so that the system
administrator can merge their local changes.)
The problem with having imperative constructs in /etc/ is that they're much
harder (perhaps impossible) to fix up when changes have to be made at the
system packaging level. Let's say that you hard code this into the
sitecustomize.py when you ship it for the first time on Fedora:
siteconfig_cfg = '/etc/siteconfig.cfg'
The system administrator installs this package and then does some unrelated
customization for his site in the file:
if sys.version_info[:2] == '3.0':
sys.path.append('/opt/python3.0')
siteconfig_cfg = '/etc/siteconfig30.cfg'
elif sys.version_info[:2] == '3.1':
sys.path.append(os.environ['USER'] + '/opt/python3.1')
siteconfig_cfg = '/etc/siteconfig31.cfg'
elif sys.version_info[:2] == '3.2':
sys.path.append(os.environ['USER'] + '/opt/python3.2')
siteconfig_cfg = '/etc/siteconfig.cfg'
# [...] Do other strange stuff because if it's possible, someone will.
Now, when we update to python3.3 we decide to move siteconfig.cfg into
/etc/python3.3/. But, because the system admin has modified that file so
heavily, we don't update that file. This means the updated python is broken
out of the box.
If you're enamoured of symlinks, you can do this directly with the config
file instead of using sitecustomize::
/etc/python3.2/sysconfig.cfg
/usr/lib64/python3.2/sysconfig.cfg => /etc/python3.2/sysconfig.cfg
as a third alternative.
-Toshio
pgpCYTDznfbDR.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
Antoine Pitrou wrote: > On Thu, 12 Aug 2010 18:14:44 -0400 > Glyph Lefkowitz wrote: > > > > On Aug 12, 2010, at 6:30 AM, Tim Golden wrote: > > > > > I don't care how many stats we're doing > > > > You might not, but I certainly do. And I can guarantee you that the > > authors of command-line tools that have to start up in under ten > > seconds, for example 'bzr', care too. > > The idea that import time is dominated by stat() calls sounds rather > undemonstrated (and unlikely) to me. In the case of bzr startup, the exact breakdown varies depending on a range of factors like OS and whether the relevant parts of the filesystem are in the OS cache or not (i.e. is this the first time the user has run bzr since booting?). The short answer is that the number of stat calls isn't really the problem at the moment for bzr, at least not compared to the number of *directories* Python searches, and the amount of non-trivial work done at import time by many modules. But... Your Mileage May Vary. Here's the longer answer: I think some stats about this have been posted to this list before, but some points of interest from the top of my head: * the cost of trying and failing to open foomodule.so + foo.so + foo.pyc + foo.py in a directory isn't much greater than trying to open just one on them. Once the OS has cached the directory entries for that directory subsequent lookups in that directory are fast. The experiement is fairly easy: - strace -e open,stat64 -o py.strace python -c "something..." - by hand, create a .C file that repeats all the stat and open in py.strace (it's pretty easy to munge into valid C) - and also create one with only the *successful* stat and open calls - compare them (using /proc/sys/vm/drop_caches or whatever as appropriate) * each directory probed is a significant cost, especially in the "cold boot" case. So every in sys.path, and every subdirectory of a package. * that said, Windows seems much slower than Linux on equivalent hardware, perhaps attempting to open files is intrinsically more expensive there? Certainly it's not safe to assume conclusions drawn on Linux will apply equally well on Windows, or vice versa. * modules with many class/function definitions are measurably slower than smaller modules. * module-level re.compile calls and other non-trivial operations are to be avoided, but many modules you depend on will do that. This matters so much that bzr monkey-patches the re module to make re.compile lazy. Try grepping the stdlib to see how many modules do re.compile at import time (including as default values of keyword args)! * it's death by a thousand cuts: each module import probably imports a dozen others... by far the simplest way to reduce startup time is to just import less modules. Lazy module imports (bzrlib.lazy_import or hg's demandload or whatever) help a lot, and I wish they were a builtin feature of Python. * I haven't even mentioned NFS or other network filesystems, but you can bet they change the picture significantly. -Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
> A good alternative would be to make the config file overridable. That way > you can have sysconfig.cfg next to sysconfig.py or in a known config > directory relative to the python stdlib install but also let the > distributions and individual sites override the defaults by making changes > to /etc/python3/sysconfig.cfg for instance. Putting configuration files next to source code does not agree with the FHS. Your message gave me an idea though: Some function could be called in sitecustomize (symlinked from /etc on Debian e.g.) to set the path to sysconfig.cfg. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Fri, Aug 13, 2010 at 07:48:22AM +1000, Nick Coghlan wrote:
> 2010/8/12 Éric Araujo :
> >> Choosing an arbitrary location we think is good on every system is fine
> >> and non risky I think, as long as Python let the various distribution
> >> change those paths though configuration.
> >
> > Don’t you have a bootstrapping problem? How do you know where to look at
> > the sysconfig file that tells where to look at config files?
I'd hardcode a list of locations.
[os.path.join(os.path.dirname(__file__), 'sysconfig.cfg'),
os.path.join('/etc', 'sysconfig.cfg')]
The distributor has a limited choice of options on where to look.
A good alternative would be to make the config file overridable. That way
you can have sysconfig.cfg next to sysconfig.py or in a known config
directory relative to the python stdlib install but also let the
distributions and individual sites override the defaults by making changes
to /etc/python3/sysconfig.cfg for instance.
>
> Personally, I'm not clear on what a separate syconfig.cfg file offers
> over clearly separating the directory configuration settings and
> continuing to have distributions patch sysconfig.py directly. The
> bootstrapping problem (which would encourage classifying synconfig.cfg
> as source code and placing it alongside syscongig.py) is a major part
> of that point of view.
>
Here's some advantages but some of them are of dubious worth:
* Allows users/site-administrators to change paths and not have packaging
systems overwrite the changes.
* Makes it conceptually cleaner to make this overridable via user defined
config files since it's now a matter of parsing several config files
instead of having a hardcoded value in the file and overridable values
outside of it.
* Allows sites to add additional paths to the config file.
* Makes it clear to distributions that the values in the config file are
available for making changes to rather than having to look for it in code
and not know the difference between thaat or say, the encoding parameter
in python2.
* Documents the format to use for overriding the paths if individual sites
can override the defaults that are shipped in the system version of
python.
-Toshio
pgpBEZ2XsDBy9.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Thu, 12 Aug 2010 08:18:27 pm Steve Holden wrote: > One might make a case that all configuration data should be stored in > a single SQLite database (with a suitable API to hide the relational > nature of the store). -1 Please don't even *consider* such a thing. Haven't we learned from Firefox? I for one am getting heartily sick and tired of having to "fix" (that is, throw away) corrupted Firefox databases. Please don't go down that path. -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Thu, 12 Aug 2010 18:14:44 -0400 Glyph Lefkowitz wrote: > > On Aug 12, 2010, at 6:30 AM, Tim Golden wrote: > > > I don't care how many stats we're doing > > You might not, but I certainly do. And I can guarantee you that the > authors of command-line tools that have to start up in under ten > seconds, for example 'bzr', care too. The idea that import time is dominated by stat() calls sounds rather undemonstrated (and unlikely) to me. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Aug 12, 2010, at 6:30 AM, Tim Golden wrote: > I don't care how many stats we're doing You might not, but I certainly do. And I can guarantee you that the authors of command-line tools that have to start up in under ten seconds, for example 'bzr', care too. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
2010/8/12 Éric Araujo : >> Choosing an arbitrary location we think is good on every system is fine >> and non risky I think, as long as Python let the various distribution >> change those paths though configuration. > > Don’t you have a bootstrapping problem? How do you know where to look at > the sysconfig file that tells where to look at config files? Personally, I'm not clear on what a separate syconfig.cfg file offers over clearly separating the directory configuration settings and continuing to have distributions patch sysconfig.py directly. The bootstrapping problem (which would encourage classifying synconfig.cfg as source code and placing it alongside syscongig.py) is a major part of that point of view. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 08:26, Tarek Ziadé wrote: [snip...] Choosing an arbitrary location we think is good on every system is fine and non risky I think, as long as Python let the various distribution change those paths though configuration. In fact, that's one of the future goal of the sysconfig module I had in mind. Instead of having it reading paths from python variables, I would like to introduce a configuration file global to Python containing all the installation paths, and the paths we are currently discussing. That file could then be changed by distributions depending on their layouts. Yes - this sounds good. +1 It also provides a single place for users who are unhappy with the defaults that we choose. Michael It will avoid distributions to hack Python to change those paths. For instance, Ubuntu currently patches distutils to change the installation paths. So let's: - define all the default locations for each system - define a sysconfig.cfg file that contains all installation paths, for each target system - change sysconfig.py so it uses it, instead of the global dicts it currently has Regards Tarek -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Thu, Aug 12, 2010 at 6:18 AM, Steve Holden wrote: > Didn't we have this discussion when per-user libraries came up? > Shouldn't we be using a subdirectory of that location? We ruin the risk > of Python becoming distributed so finely it becomes impossible to change > things, what with per-user libraries, per-version shared binaries, > per-user-per-version library configuration files and the rest. Yes, we did. That was in the context of some people's desire for a per-user site-packages directory, and the result is that we're now saddled with a per-user site-packages directory. The structure determined there was the same sort of half-way version-specific mess as so many others are: libraries are per-python-version, and scripts are installed cowboy-style (whoever got there last, wins). Perhaps user configuration belongs in ~/.local/, or ~/.local/python/ (with attendant Windows & Mac OS noises); I don't really care where it lands, because right now we just have a mess. Getting it "right" with respect to Window's "roaming" notion and how people expect to work with it is... unlikely. Picking a place is better than not, so we know where to find things. But that's all it buys us. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
> Ultimately, I don't feel very strongly about this subject. I'm more > concerned that the chosen location (file or registry or whatever) be > documented -- and documented from a Windows perspective as well, so > you don't have to guess what "HOME" means in this context. Rest assured that I don’t want to impose Unix concepts on Windows. My examples have used dotfiles since it’s what I know, but I did search and ask for the right thing to do on other platforms. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12 August 2010 12:59, Tim Golden wrote:
> re: using the Registry: To be honest, I was answering the literal
> question posed by Eric: where to put config files? Not the wider
> question: how should config data be stored? Where the answer to
> the latter question might be: the Registry -- much as I find it
> awkward to work with at times.
>
> One very definite point in its favour as regards this discussion
> is that Python on Windows *already* defines a key in the Registry
> (and has done so since v2.0 at least) and it would make some sense
> to place things under there.
>
> Ultimately, I don't feel very strongly about this subject. I'm more
> concerned that the chosen location (file or registry or whatever) be
> documented -- and documented from a Windows perspective as well, so
> you don't have to guess what "HOME" means in this context.
One potential solution would be to use something like
os.path.expanduser("~/.python"), where os.path.expanduser has a
defined behaviour on Windows (one that could do with a little
clarification, yes I know, doc patch welcomed...), but also add an
additional option, that os.path.expanduser will first look in the
registry for the value HKLM\Software\Python\PythonCore\HOME and use
that if present. Then users could set their own interpretation, in a
Windows-standard manner.
If that would be an acceptable option, I could supply a patch to
os.path.expanduser. I will do a clarifying doc patch regardless.
Paul.
PS We could also support ~user by looking in HKEY_USERS\\Software\Python\... but that's probably YAGNI :-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
Le 12/08/2010 12:18, Steve Holden a écrit : > Didn't we have this discussion when per-user libraries came up? > Shouldn't we be using a subdirectory of that location? As pointed out by Antoine, Georg, Michael and I, the PEP 370 directory for user site-packages is not the right place to put config files. > PS: Then we can start putting the libraries themselves in the database. > That works really well. I don’t know you well enough to tell if this is the final hint that you were sarcastic. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
>> If the files are shared among all users then /usr/local/ >> seems more reasonable. Oh, right, I forgot to think about system-wide config files. They have to be supported by another function in site. A lot of programs have similar-looking code to get a list of filenames and then process it with configparser or something else: find_config_files in Distutils, get_standard_config_files in Docutils, rcpath in Mercurial. There are different namings to support (rc, .cfg, .conf, etc.) and also config directories (for Mercurial), but it does not seem impossible to define a standard function that would benefit Python itself and other programs. > Choosing an arbitrary location we think is good on every system is fine > and non risky I think, as long as Python let the various distribution > change those paths though configuration. Don’t you have a bootstrapping problem? How do you know where to look at the sysconfig file that tells where to look at config files? > In fact, that's one of the future goal of the sysconfig module I had in mind. Seems great. > So let's: > > - define all the default locations for each system > - define a sysconfig.cfg file that contains all installation paths, > for each target system > - change sysconfig.py so it uses it, instead of the global dicts it > currently has I wonder if this is so complex that a wiki page or something else would be better suited than email to iron out the proposal. Or maybe it’s time for that sysconfig.cfg PEP you planned to do. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 12:17, Michael Foord wrote: How is ~/python not memorable or consistent? (And cross-platform memorability and consistency is valuable too.) I was thinking outside Python rather than inside it (where ~ has no meaning on Windows) but you make a good point here. If we were just discussing Python code this would effectively be a winning point IMO. Another issue is discoverability. Many users won't know about these config files unless they *see* them. While I sympathise with this, I'm not sure how much weight one should give it in the context of this discussion. In the Unix world, if I were guessing, I would justifiably look in ~/.myapp followed perhaps by /etc/myapp. On Windows, I might go for the registry, as you mention elsewhere, and look in HKLM\Software\MyApp but for actual files I'm not sure where "discoverable" would be. re: using the Registry: To be honest, I was answering the literal question posed by Eric: where to put config files? Not the wider question: how should config data be stored? Where the answer to the latter question might be: the Registry -- much as I find it awkward to work with at times. One very definite point in its favour as regards this discussion is that Python on Windows *already* defines a key in the Registry (and has done so since v2.0 at least) and it would make some sense to place things under there. Ultimately, I don't feel very strongly about this subject. I'm more concerned that the chosen location (file or registry or whatever) be documented -- and documented from a Windows perspective as well, so you don't have to guess what "HOME" means in this context. TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 11:54, Tim Golden wrote: On 12/08/2010 11:40, Michael Foord wrote: User editable configuration files are very different from libraries. The per user site-packages folder *should* be hidden somewhere out of the way where you can get at them if you want them but won't stumble across them all the time. e.g. AppData on Windows. For files we expect the user to be able to edit we shouldn't go out of our way to make them hard or inconvenient to find. This really comes down -- as with the Mac discussion last week, I think -- to the tension between standard file locations and easy-to-get-to. The difference on Windows, though, is that while on Mac it's either: Apple standard /This/That/TheOther vs Unix standard $HOME/.something on Windows it's %APPDATA%\myprog vs whatever-I-thought-of-at-the-time. APPDATA isn't actually that hard to get to on any version of Windows post 2000. (Just stick %APPDATA% into any Explorer window -- even IE, for goodness sake). Even from the command line it's not hard: notepad "%APPDATA%\python\python26\idle.ini" Ultimately, as a Windows user/developer, I'd vote for a memorable consistency, which for me would be %APPDATA%\python\... If everything's in one place, I'll just create a shortcut / hardlink / whatever to get me there when I want to change things. How is ~/python not memorable or consistent? (And cross-platform memorability and consistency is valuable too.) Another issue is discoverability. Many users won't know about these config files unless they *see* them. Sure if you *know* where they are then editing them is easy - but how often as a user do you browse your AppData folder? (As a windows user the only times I went delving in this mysterious location was trying to diagnose obscure errors.) I'm not aware of any consistent pattern of putting *user editable* files in AppData, and I doubt this is where most users would look for them. In fact for Windows the *correct* thing to do is probably to use the registry and then provide a graphical tool for editing the values. If you're arguing for consistency why not argue for this? If we were to provide native GUI tools for editing values then the location suggested by Ronald for the Mac (which is the standard location for preferences files maintained by GUI tools) would also be the right location. All the best, Michael TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 11:40, Michael Foord wrote: User editable configuration files are very different from libraries. The per user site-packages folder *should* be hidden somewhere out of the way where you can get at them if you want them but won't stumble across them all the time. e.g. AppData on Windows. For files we expect the user to be able to edit we shouldn't go out of our way to make them hard or inconvenient to find. This really comes down -- as with the Mac discussion last week, I think -- to the tension between standard file locations and easy-to-get-to. The difference on Windows, though, is that while on Mac it's either: Apple standard /This/That/TheOther vs Unix standard $HOME/.something on Windows it's %APPDATA%\myprog vs whatever-I-thought-of-at-the-time. APPDATA isn't actually that hard to get to on any version of Windows post 2000. (Just stick %APPDATA% into any Explorer window -- even IE, for goodness sake). Even from the command line it's not hard: notepad "%APPDATA%\python\python26\idle.ini" Ultimately, as a Windows user/developer, I'd vote for a memorable consistency, which for me would be %APPDATA%\python\... If everything's in one place, I'll just create a shortcut / hardlink / whatever to get me there when I want to change things. TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 11:18, Steve Holden wrote: On 8/12/2010 5:50 AM, Tim Golden wrote: On 11/08/2010 16:22, Éric Araujo wrote: It would be nice to define one standard location for config files used by stdlib modules, and maybe also by third-party programs related closely to Python development (testing tools, static code checkers and the like), in a way that doesn’t clutter the user home directory with a dozen dotfiles while still being easily found. (The Unix notions of dotfiles and home directory have to be adapted to use non-dotfiles in some standard place on various Windows. The Mac experts disagree on the right directory to use.) The canonical place on Windows (per Microsoft [1]) is: for roaming data: the Application Data folder (exposed as the APPDATA env var and as CSIDL_APPDATA pre-Vista and the FOLDERID_RoamingAppData Known Folder on Vista+). for non-roaming data: the local app data (CSIDL_LOCAL_APPDATA or FOLDERID_LocalAppData) The obvious difference is whether a user expects the data to be available for the same account on a different computer. At present, the user site packages are held under %APPDATA%\Python\Pythonxx, ie are considered roaming. Unfortunately, the canonical place is not always the place most used. Especially since the convention under *nix is to place dotfile or dotdirs under $HOME. Windows doesn't, by default, have a $HOME so various locations are considered $HOME, including (but not limited to): * the directory pointed to by %HOME% * the directory pointed to by %USERPROFILE% * the "My Documents" shell folder (often, but not always, %USERPROFILE%\My Documents) * c:\ (no: really) * the NT profile home directory (typically a mapped share) Dotfiles are possible on Windows; the issue is that they can't be created within the shell (ie within Windows Explorer). They can be renamed, edited, etc without issue, so it's not much of a problem. They can be created by any program which can call CreateFile -- including Python, obviously. TJG [1] http://support.microsoft.com/kb/310294 Didn't we have this discussion when per-user libraries came up? Shouldn't we be using a subdirectory of that location? We ruin the risk of Python becoming distributed so finely it becomes impossible to change things, what with per-user libraries, per-version shared binaries, per-user-per-version library configuration files and the rest. User editable configuration files are very different from libraries. The per user site-packages folder *should* be hidden somewhere out of the way where you can get at them if you want them but won't stumble across them all the time. e.g. AppData on Windows. For files we expect the user to be able to edit we shouldn't go out of our way to make them hard or inconvenient to find. One might make a case that all configuration data should be stored in a single SQLite database (with a suitable API to hide the relational nature of the store). Yay - let's recreate the Windows registry! A binary blob only editable with special tools, we know how much users love that. Michael Then at least there would only be one file no matter how many versions of Python were installed. Seriously. We are already spending enough time doing stat calls on non-existent directories. regards Steve PS: Then we can start putting the libraries themselves in the database. That works really well. -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 10:50, Tim Golden wrote: Unfortunately, the canonical place is not always the place most used. Especially since the convention under *nix is to place dotfile or dotdirs under $HOME. Windows doesn't, by default, have a $HOME so various locations are considered $HOME, including (but not limited to): * the directory pointed to by %HOME% * the directory pointed to by %USERPROFILE% * the "My Documents" shell folder (often, but not always, %USERPROFILE%\My Documents) * c:\ (no: really) * the NT profile home directory (typically a mapped share) Additionally, on Windows Python expands ~ to the first of: %HOME% %USERPROFILE% %HOMEDRIVE%\%HOMEPATH% all of which are perfectly defensible defintions of "Home" on Windows. But combining that with the Unix-minded concept of "create a dotfile called ~/.myapp" means you get dotapps in non-standard places on Windows. TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 12/08/2010 11:18, Steve Holden wrote: On 8/12/2010 5:50 AM, Tim Golden wrote: [... snip explanation of standard & non-standard locations ...] Didn't we have this discussion when per-user libraries came up? Shouldn't we be using a subdirectory of that location? Yes we should. My explanation above was really just painting the picture. It should have been an Appendix for the Truly Interested. One might make a case that all configuration data should be stored in a single SQLite database (with a suitable API to hide the relational nature of the store). Then at least there would only be one file no matter how many versions of Python were installed. Seriously. We are already spending enough time doing stat calls on non-existent directories. -1 on this. I don't care how many stats we're doing; I want to be able to see my configuration / libraries etc without firing up a sqlite session. TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 8/12/2010 5:50 AM, Tim Golden wrote: > On 11/08/2010 16:22, Éric Araujo wrote: >> It would be nice to define one standard location for config files used >> by stdlib modules, and maybe also by third-party programs related >> closely to Python development (testing tools, static code checkers and >> the like), in a way that doesn’t clutter the user home directory with a >> dozen dotfiles while still being easily found. >> >> (The Unix notions of dotfiles and home directory have to be adapted to >> use non-dotfiles in some standard place on various Windows. The Mac >> experts disagree on the right directory to use.) > > The canonical place on Windows (per Microsoft [1]) is: > > for roaming data: the Application Data folder (exposed as the > APPDATA env var and as CSIDL_APPDATA pre-Vista and the > FOLDERID_RoamingAppData Known Folder on Vista+). > > for non-roaming data: the local app data (CSIDL_LOCAL_APPDATA > or FOLDERID_LocalAppData) > > The obvious difference is whether a user expects the data to be > available for the same account on a different computer. At present, > the user site packages are held under %APPDATA%\Python\Pythonxx, > ie are considered roaming. > > Unfortunately, the canonical place is not always the place most > used. Especially since the convention under *nix is to place dotfile > or dotdirs under $HOME. Windows doesn't, by default, have a $HOME so > various locations are considered $HOME, including (but not limited > to): > > * the directory pointed to by %HOME% > * the directory pointed to by %USERPROFILE% > * the "My Documents" shell folder > (often, but not always, %USERPROFILE%\My Documents) > * c:\ (no: really) > * the NT profile home directory (typically a mapped share) > > Dotfiles are possible on Windows; the issue is that they can't > be created within the shell (ie within Windows Explorer). They > can be renamed, edited, etc without issue, so it's not much of > a problem. They can be created by any program which can call > CreateFile -- including Python, obviously. > > TJG > > [1] http://support.microsoft.com/kb/310294 Didn't we have this discussion when per-user libraries came up? Shouldn't we be using a subdirectory of that location? We ruin the risk of Python becoming distributed so finely it becomes impossible to change things, what with per-user libraries, per-version shared binaries, per-user-per-version library configuration files and the rest. One might make a case that all configuration data should be stored in a single SQLite database (with a suitable API to hide the relational nature of the store). Then at least there would only be one file no matter how many versions of Python were installed. Seriously. We are already spending enough time doing stat calls on non-existent directories. regards Steve PS: Then we can start putting the libraries themselves in the database. That works really well. -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 11/08/2010 16:22, Éric Araujo wrote: It would be nice to define one standard location for config files used by stdlib modules, and maybe also by third-party programs related closely to Python development (testing tools, static code checkers and the like), in a way that doesn’t clutter the user home directory with a dozen dotfiles while still being easily found. (The Unix notions of dotfiles and home directory have to be adapted to use non-dotfiles in some standard place on various Windows. The Mac experts disagree on the right directory to use.) The canonical place on Windows (per Microsoft [1]) is: for roaming data: the Application Data folder (exposed as the APPDATA env var and as CSIDL_APPDATA pre-Vista and the FOLDERID_RoamingAppData Known Folder on Vista+). for non-roaming data: the local app data (CSIDL_LOCAL_APPDATA or FOLDERID_LocalAppData) The obvious difference is whether a user expects the data to be available for the same account on a different computer. At present, the user site packages are held under %APPDATA%\Python\Pythonxx, ie are considered roaming. Unfortunately, the canonical place is not always the place most used. Especially since the convention under *nix is to place dotfile or dotdirs under $HOME. Windows doesn't, by default, have a $HOME so various locations are considered $HOME, including (but not limited to): * the directory pointed to by %HOME% * the directory pointed to by %USERPROFILE% * the "My Documents" shell folder (often, but not always, %USERPROFILE%\My Documents) * c:\ (no: really) * the NT profile home directory (typically a mapped share) Dotfiles are possible on Windows; the issue is that they can't be created within the shell (ie within Windows Explorer). They can be renamed, edited, etc without issue, so it's not much of a problem. They can be created by any program which can call CreateFile -- including Python, obviously. TJG [1] http://support.microsoft.com/kb/310294 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Thu, Aug 12, 2010 at 7:00 AM, Russell E. Owen wrote: ... > > If the files are shared among all users then /usr/local/ > seems more reasonable. > > I also think whatever you choose for linux is also the best choice for > Mac OS X (my preferred platform). While there are other possible > directories, such as ~/Library/Application Support/, all > tools derived from unix that I know about use the unix convention (ssh, > X11, bash...) and I would argue that Python is close enough to count > even though it is a framework build. Put another way, copying the unix > convention is simple, is exactly what power users would expect and I > don't see it would do any harm. Choosing an arbitrary location we think is good on every system is fine and non risky I think, as long as Python let the various distribution change those paths though configuration. In fact, that's one of the future goal of the sysconfig module I had in mind. Instead of having it reading paths from python variables, I would like to introduce a configuration file global to Python containing all the installation paths, and the paths we are currently discussing. That file could then be changed by distributions depending on their layouts. It will avoid distributions to hack Python to change those paths. For instance, Ubuntu currently patches distutils to change the installation paths. So let's: - define all the default locations for each system - define a sysconfig.cfg file that contains all installation paths, for each target system - change sysconfig.py so it uses it, instead of the global dicts it currently has Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
In article <4c62c01d.6000...@netwok.org>, Ãric Araujo wrote: > Hello list > > Tarek opened a distutils bugs in http://bugs.python.org/issue7175 that > evolved into a discussion about the proper location to use for config files. > > Distutils uses [.]pydistutils.cfg and .pypirc, and now unittest2 has a > config file too. > > It would be nice to define one standard location for config files used > by stdlib modules, and maybe also by third-party programs related > closely to Python development (testing tools, static code checkers and > the like), in a way that doesnât clutter the user home directory with a > dozen dotfiles while still being easily found. > > (The Unix notions of dotfiles and home directory have to be adapted to > use non-dotfiles in some standard place on various Windows. The Mac > experts disagree on the right directory to use.) > > Tarek, Antoine, RDM, MAL were +1 on using ~/.python (whether to use > .pythonx.y or .python/x.y is a subissue to discuss after general agreement). > > What do you think about this? I like the idea as long as different versions of python have different directories and the files are intended to be user-specific. If the files are shared among all users then /usr/local/ seems more reasonable. I also think whatever you choose for linux is also the best choice for Mac OS X (my preferred platform). While there are other possible directories, such as ~/Library/Application Support/, all tools derived from unix that I know about use the unix convention (ssh, X11, bash...) and I would argue that Python is close enough to count even though it is a framework build. Put another way, copying the unix convention is simple, is exactly what power users would expect and I don't see it would do any harm. -- Russell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Wed, Aug 11, 2010 at 10:58 PM, Éric Araujo wrote: > Considering the FHS or the XDG Base > Directory specifications, there is a precedent in distinguishing user > config (edited by the user through a text editor or settings graphical > window), program data (state) and cache (files for speedups that can > safely be deleted). Right. The wording in that PEP is ambiguous at best, but the inclusion of the Python version number in the listed path suggests that this is for automatically managed stat (like those trashy *.pth files some tools worry about), not for something the human user is going to manipulate directly. That PEP is particularly concerned with package management per user (a dodgy proposition at best), so everything there is about package management tool support, not user-manipulated configuration data. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
> PEP 370 already specifies a directory for Python config files: > >> user data directory >> >> Usually the parent directory of the user site directory. >> It's meant for Python version specific data like config >> files, docs, images and translations. Thanks for pointing that. However, I have to disagree with the PEP here: config files are not data files. Considering the FHS or the XDG Base Directory specifications, there is a precedent in distinguishing user config (edited by the user through a text editor or settings graphical window), program data (state) and cache (files for speedups that can safely be deleted). Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
> I'd like to see a more complete proposal, including:
Fair enough.
tl;dr: Locating config files is hard.
I have looked at http://github.com/ActiveState/appdirs (MIT) for
OS-specific bits of knowledge. (Note that the directories it uses for
free OSes are not compliant with the freedesktop.org Base Directory
specification, which already did the reflexion and code to separate
config, user data and cache.)
> - what to do with Windows,
appdirs uses pywin32 or ctypes or _winreg to find the correct value for
the user settings directory. There is a new complication in recent
Windows called roaming profiles, which are copied from a server to
multiple clients and need synchronization calls.
> Mac OS X
There is debate between Ned Deily, Ronald Oussoren and Glyph Lefkowitz
as to whether Mac OS is just a Unix or a special beast. appdirs chooses
the Mac-specific directory. There are also probably variations between
system installations, framework things and unix-style installations.
> - backward compatibility considered
> (seems straightforward, but seemings are often wrong)
Are you talking about finding config from older locations, or automatic
renaming of ~/.idlerc to ~/.python/idlerc? The former seems easy; not
sure how to do the latter. Gajim, a PyGTK Jabber client, moved from a
custom layout to BaseDir-compliant directories a few months ago; I think
it looks for the old directories on every startup to see if they need to
be moved. I’m not sure if this affects startup time noticeably, since
it’s after all a graphical network Python app, but it may be a concern
with programs that are run a lot like the unittest2 script (or not,
considering startup time of the interpreter).
> - cross-platform API to locate these files
I was thinking about adding a function to site with this tentative
signature:
def getuserconfig(appname=''):
"""Return the configuration file name for `appname`.
If `appname` is the empty string, return the directory used for
Python configuration files, else returns the file name used for the
configuration file of `appname` (a PyPI project name).
Strings starting with a number ('3.2') are reserved by Python.
"""
I’m not sure the x.y subdirectory is necessary. distutils config files
are compatible since forever and will still be compatible with
distutils2; config files in unittest2 are very recent, they could have a
policy of compatiblity too; I can investigate IDLE config format.
A boolean could control whether to return filenames using the old
conventions. On Windows, this would need application name and
author/vendor name (settings are grouped by author). On Mac, the
application would have to say whether it uses a unix-style dotfile or a
Mac-style directory. Also, capitalization differs across OSes. Can of
worms here.
> Seems like a good idea to have a "registry" of files known to be placed there
The project name registration on PyPI may be enough to avoid duplicates.
Next steps:
1) Find out if winreg is reliable enough, since we don’t want to depend
on ctypes or pywin32 IIRC.
2) Have the Mac people come to an agreement for each type of install.
3) Prototype the new function. It’s probably a good idea to contact the
author of appdirs and work with him on a Python-specific subset of his
module instead of rewriting the same code.
4) Think about the best way to handle b/w compat. Timings could help
deciding for or against automatic renaming.
Thanks for the feedback.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
Hello, Fred Drake fdrake at acm.org wrote: > +0.5 > > I'd like to see a more complete proposal, including: > > - what to do with Windows, Mac OS X PEP 370 already specifies a directory for Python config files: > user data directory > > Usually the parent directory of the user site directory. It's meant for > Python version specific data like config files, docs, > images and > translations. > > Unix (including Mac) > ~/.local/lib/python2.6 > Windows > %APPDATA%/Python/Python26 Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On Wed, Aug 11, 2010 at 11:22 AM, Éric Araujo wrote: > Tarek, Antoine, RDM, MAL were +1 on using ~/.python (whether to use > .pythonx.y or .python/x.y is a subissue to discuss after general agreement). +0.5 I'd like to see a more complete proposal, including: - what to do with Windows, Mac OS X - backward compatibility considered (seems straightforward, but seemings are often wrong) - cross-platform API to locate these files (should be able to address the b/w compat issues) Seems like a good idea to have a "registry" of files known to be placed there (a wiki page should be sufficient; if the API is used to create the directory, it can drop in a README.txt pointing to that page). -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing #7175: a standard location for Python config files
On 11/08/2010 16:22, Éric Araujo wrote: Hello list Tarek opened a distutils bugs in http://bugs.python.org/issue7175 that evolved into a discussion about the proper location to use for config files. Distutils uses [.]pydistutils.cfg and .pypirc, and now unittest2 has a config file too. IDLE has a config directory too - currently .idlerc in ~ (on Mac OS X at any rate). Michael It would be nice to define one standard location for config files used by stdlib modules, and maybe also by third-party programs related closely to Python development (testing tools, static code checkers and the like), in a way that doesn’t clutter the user home directory with a dozen dotfiles while still being easily found. (The Unix notions of dotfiles and home directory have to be adapted to use non-dotfiles in some standard place on various Windows. The Mac experts disagree on the right directory to use.) Tarek, Antoine, RDM, MAL were +1 on using ~/.python (whether to use .pythonx.y or .python/x.y is a subissue to discuss after general agreement). What do you think about this? Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
