At 12:43 AM 10/8/2008 +0200, Josselin Mouette wrote:
Le mardi 07 octobre 2008 à 16:51 -0400, Phillip J. Eby a écrit :
> >If you write a tool to do that, why not make it simply move files
> >properly and let the code locate them, instead of adding yet another
> >layer on top of the existing stuff? The tool will not be more
> >complicated this way.
>
> I'm not sure I follow you. What I'm proposing is that:
>
> 1. A BUILDS "manifest" for a project should list the files included
> in the project by what "kind" of file they are, as well as by
> inter-file relationships (e.g., "these 3 files are part of Python
> package 'foo.bar' within this project, and they are static data used
> at runtime but not read or written").
>
> 2. BUILDS-compatible install tools will be free to install those
> files wherever they like (or create .debs, .rpms, etc. that do), and
> use symlinks so that the installed project can find them in the
> places that the developer wanted to find them.
>
> Is this what you're asking for? Because it's what I've been trying
> to say we should do.
What I am afraid of is that, by adding just another layer, you will
introduce new problems while fixing existing ones. Currently we already
have a hard time maintaining a symlink farm, and adding a second symlink
farm on top of it is not going to make things more understandable
(currently, package maintainers already have a hard time understanding
how to deal with Python modules) nor more reliable.
I guess I'm confused a bit here, since the idea is not to have
maintainers need to understand anything but how to be able to tell
upstream, "hey, you didn't configure file X correctly". At least,
that's what I'd hope would be all that's needed. :-)
The first kind of issues I can think of is that dpkg handles symlinks to
directories very badly, so youâre likely to run into issues that have to
be dealt with by hand by maintainers. I havenât thought about it much,
but I have the feeling that other things will break.
Actually, until I saw the Twisted plugins directory use case, I
hadn't actually seen any use cases for symlinks to directories that
couldn't be handled by using a directory of symlinks instead. Are
there any other Python packages you know of that would need a symlink
to a directory?
In the end, if you are designing a new packaging specification and a new
tool,
The BUILDS manifest is neither packaging nor a tool; it's just
information about a bunch of files. ;-) We don't want to design *a*
tool, we want to make it possible for tools to be
written. Currently, there are a whole bunch of installation and
maintenance tools out there built on top of setuptools, but they
aren't necessarily very compatible with each other, and building
tools that approach things very differently from setuptools is hard.
The idea of a specification is to allow installation tools to be
built that apply different policies on different platforms, not to
build The One True Tool[tm].
In other words, the goal isn't to "do the right thing", but to allow
various useful things to be done.
> For example, if Debian needs metadata that's not in the core BUILDS
> metadata, it should be possible within BUILDS to include it, so that
> it can be contributed upstream. Likewise, it should be possible for
> a BUILDS install tool to use a third-party supplement to get that
> data, so that when you're initially porting a project, you could
> write a file with the additional data while doing the initial port,
> then submit that file upstream.
If there is a need for Debian-specific metadata, it will go in the
debian/ directory. It looks interesting to integrate it upstream only if
it is relevant for other distributors.
If I have to explicitly mark my locale files for Debian to be happy,
then that *is* Debian-specific metadata from my perspective as a
cross-platform developer.
That is, I didn't say anything about metadata that was unique to
Debian, just metadata needed by a Debian BUILDS installer to be
happy, as opposed to a generic BUILDS installer that doesn't care
about the FHS. (For example, an Enthought installer for Windows
that's also installing its own Python interpreter might want to
consume BUILDS but it sure as heck wouldn't be following FHS
standards; it'd have its own.)
There is no use in being compatible with the autotools, the point is
about stealing good ideas.
The first thing that comes to mind, and that we already discussed at
large, is the ability to specify at install time where data,
documentation, manual pages and so on have to go.
The idea of the BUILDS "install locations" (IL) manifest is just that
it lists the contents of the project distribution, and describes the
logical locations that things go in. So, part of the BUILDS spec
would be what categories exist for the above.
It would then be the responsibility of people building
BUILDS-consuming install tools to define the actual locations. I say
it that way mainly because I'm not volunteering my time to actually
build such an install tool, nor am I necessarily volunteering to
build *any* tools. No sense in replacing me as the setuptools
bottleneck to be the bottleneck on new tools. The whole idea is to
get system packagers and major developers together to agree on how to
communicate installation requirements to installation tools, and then
have everybody go back and update their tools to generate or consume
installation data in accordance with the spec.
Of course, it's likely that rather than everybody creating their own
install tools, there will probably be some grouping and alliances
along platform and/or use-case lines. For the stdlib, I expect there
will be a simple installer with configurable locations for installing
things, maybe with some ability to be extended to handle more complex
policies. There would probably also need to be BUILDS-to-wininst,
and BUILDS-to-msi converters.
What some Python projects using autoconf are doing, and Iâd like to see
in BUILDS, is the generation of a config.py file at installation time.
It would be useful to specify install-time stuff like installation
directories, optional features and installation parameters.
For example it could contain:
version = "1.2.3"
builddate = "Tue 7 Oct 2008, 23:12 +0100"
templates_dir = "/usr/share/foobar/templates"
has_libamazing = True # _foobar.so is built with libamazing support
page_footer = "Powered by foobar"
For Python-only packages, you can easily ship a default config.py file
that works without installation:
version = get_version_from_changelog ()
templates_dir =
os.path.join(os.path.basedir(__file__),"templates")
page_footer = "Foobar (development version)"
By doing such a thing, you are guaranteed to obtain easily the data you
need from within the Python application:
import config
template = file(os.path.join(config.templates_dir),
"foo.template")
I may be biased because Iâm used to work like this, but I find this
easier for both the developer and the distributor. In my personal
experience, it is also more reliable.
It looks like a great idea, and I'm all for recommending people use
it, or something like it. I'm also fine with BUILDS having support
for some sort of "install-time editing" like this. I just don't want
it to be a requirement that everybody change their ways. Not
everybody even uses pkg_resources' API for accessing data files yet, after all.
In my experience with setuptools adoption, anything that makes you
have to change something outside of the packaging part is a
significant barrier. People wonder if they can rely on it, whether
it will lock them in to something, they need to do testing, etc.
Sure, these things aren't REALLY that big of a hassle, but the
psychological effect is to create hesitation. But if you can move
your code without changing your code, it creates a sense of safety,
and provides incremental rewards for incremental efforts. Thus,
people who first moved to setuptools without needing to change their
actual package code, were then much more likely to adopt the API
later. The same will be true for this.
Another thing from the autotools that Iâd like to see is seamless
integration with pkg-config, for modules with C extensions. Bonus points
for being able to use not only compilation flags and installation paths,
but also macros you need to use inside the code (they could go in
config.py).
Sorry, what is pkg-config?
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
