Re: [Distutils] conventions or best practice to choose package names?

2012-05-30 Thread Benoît Bryon 

Hello,

Le 14/05/2012 13:12, Jim Fulton a écrit :

+1 for an official document (or addition to an existinhg document)
providing a rational for namespace packages and their naming.


I opened a ticket on CPython issue tracker:
http://bugs.python.org/issue14899

Then started to work in a fork:
https://bitbucket.org/benoitbryon/cpython/src/doc-package-names/Doc/packaging/packagenames.rst

.. but it looks like a PEP. So, I followed PEP 1
and posted the proposal to python-l...@python.org...
where I've been told to post it to distutils-sig :)
So, I'm back here with a proposal...

The document below is the same as current version of
https://bitbucket.org/benoitbryon/cpython/src/doc-package-names/Doc/packaging/packagenames.rst.

Thanks to Martin Aspeli for his article.
Thanks to early reviewers: Alexis Métaireau, Éric Bréhault,
Jean-Philippe Camguilhem and Mathieu Leplâtre.

Benoit



###
Names in packaging: conventions and recipes
###

This document deals with:

* names of Python projects,
* names of distributions in projects,
* names of Python packages or modules being distributed,
* namespace packages.

It provides conventions and recipes for distribution authors.

Main use case is:

* as a developer, I want to create a project in order to distribute a package.
  So I have to choose names.  Which names are "good"?

* `The Zen of Python`_ says:

In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.

* So I need clear and official (i.e. obvious) guidelines or conventions that I
  can follow.

* Here are conventions, guidelines and recipes.

Guidelines for existing projects are also given.


***
Terminology
***

First of all, let's make sure there is no confusion...

Distribution name
  Distribution name is used by packaging utilities:

  * in :doc:`setup script`, it is the value passed as
``name`` to ``packaging.core.setup()``
  * it appears on `PyPI`_ if the package is registered on it
  * it can be used in `pip` requirements files
  * it can be used in `buildout` configuration files.

  Distribution term is introduced in :doc:`packaging docs`.

Egg name
  It is the same concept as distribution name. Technically, the egg is not the
  distribution. But they use the same name: it is declared as
  ``packaging.core.setup()`` name argument.

  "Egg" term is the legacy counterpart to "distribution". It was used by
  distutils and setuptools. It becomes deprecated with the new packaging
  module.

  This document focuses on distributions, not eggs.

Package and module names
  Package and module names are used in Python code. It is the string used in
  :ref:`import statements`.

  Remember that, from a file system perspective, packages are directories and
  modules are files.

  :ref:`Python packaging allows distributions to distribute several packages
  and/or modules`.

Project name
  Usually the name of the repository or folder in which distribution authors put
  their code. It generally is the directory name of the "distribution root",
  as defined in :ref:`packaging-term`.

Namespace packages
  It is common practice to use namespaces in package names. `PEP 420`_ brings
  this concept to core Python. When PEP 420 will be accepted, Python officially
  supports namespace packages.

As an example, consider `django-debug-toolbar`_:

* ``django-debug-toolbar`` is the distribution name. `It is declared in
  setup.py file
  
`_.

* ``debug_toolbar`` is the package name. It is what would appear in Django's
  INSTALLED_APPS setting or be used as ``import debug_toolbar``.

Technically, all those names can be different.


*
Rationale
*

Relationship with other PEPs


* `PEP 8`_ deals with code style guide, including names of Python packages and
  modules. It covers syntax of package/modules names.

* `PEP 345`_ deals with packaging metadata, and defines distribution name.

* `PEP 420`_ deals with namespace packages. It brings support of namespace
  packages to Python core. Before, namespaces packages were implemented by
  external libraries.

* `PEP 3108`_ deals with transition between Python 2.x and Python 3.x applied
  to standard library: some modules to be deleted, some to be renamed.

Other sources of inspiration


* `Martin Aspeli's article about names`_. Some parts of this proposal are
  quotes from this article.

* `The Hitchhiker's Guide to Packaging`_, which has an empty placeholder
  for "naming specification".

* and, of course, `in development official packaging documentation`_.

Facts
=

Before Python version 3.3, there is no official guidelines to name projects,
distributions or packages/modules.
Current PEPs (see `Relationship with oth

Re: [Distutils] Requires-Dist: unittest2; 'test' in extras

2012-05-30 Thread PJ Eby 
On Wed, May 30, 2012 at 3:09 PM, Daniel Holth  wrote:

> It looks like you can only install one extra at a time,


Actually, you can specify more than one, using commas.  e.g. "easy_install
foo[testing,c-extensions,celery-support,...]".
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Problem Installing Pysam Wrapping for SAMtools

2012-05-30 Thread Daniel Holth 
Try

python setup.py --user

It should install somewhere in ~/.local/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Problem Installing Pysam Wrapping for SAMtools

2012-05-30 Thread Underhill, Jennifer M 
To whom it may concern:

I am very new to programming so I apologize if this question is overly simple.

When I attempt to install the wrapping system for SAMtools using pysam, I 
receive this error message:

[Errno 13] Permission denied: 
'/Library/Python/2.6/site-packages/test-easy-install-45999.write-test'

I do not have administrative access to my computer, and I would like to know if 
it would be possible to specify a directory for the package to be installed in.

I am working on a Mac OSX and using the Terminal shell. Thank you.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Requires-Dist: unittest2; 'test' in extras

2012-05-30 Thread Daniel Holth 
It looks like you can only install one extra at a time, so the more pleasing

Requires-Dist: somedist; extra == 'test'

would be correct.

On Wed, May 30, 2012 at 12:05 PM, Daniel Holth  wrote:

> Proposal:
>
> Requires-Dist: unittest2; 'test' in extras
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

2012-05-30 Thread Erik Bray 
On Wed, May 30, 2012 at 12:19 PM, PJ Eby  wrote:
>
>
> On Wed, May 30, 2012 at 11:54 AM, Erik Bray  wrote:
>>
>> On Tue, May 29, 2012 at 7:41 PM, PJ Eby  wrote:
>> >> > I might be confused; I haven't been following the goings-on of late
>> >> > with
>> >> > distutils2.  At one point, I thought the plan was not to bless or
>> >> > include dependency-managing installers with the stdlib, or something
>> >> > like that.  i.e., I thought the plan wasn't to support or bless
>> >> > full-service tools like buildout, easy_install, or pip, or anything
>> >> > comparable to them.
>> >>
>> >> Right, yeah, the plans in this area were fluid for awhile, but the
>> >> eventual conclusion was that the stdlib should have a command-line
>> >> utility capable of installing packages with dependencies. That exists
>> >> in
>> >> default branch now; it's called pysetup. It doesn't have nearly all the
>> >> features of easy_install, buildout, or pip, but it can install packages
>> >> from an index with deps.
>> >>
>> >
>> > In any case, it still doesn't change the part where it's a good idea to
>> > ship
>> > a static setup.cfg, with hooks only needing to run on the sdist-building
>> > machine, unless they are actually part of the build process.  There are
>> > use
>> > cases for calculated data to be in the initial setup.cfg, where the
>> > calculation machinery doesn't need to be on the target (like generating
>> > the
>> > file list or version from revision control info).  So, a setup_requires
>> > (or
>> > maybe better named "build_requires") would still be helpful, but
>> > probably
>> > shouldn't be used for setup.cfg stability.
>>
>> That's not a bad idea for certain kinds of metadata--version/vcs info
>> for example.  I like the idea of including a generated "static"
>> setup.cfg in a source dist as a solution to that kind of problem.  But
>> that doesn't eliminate the need for setup_hooks (or even more
>> complicated objects like custom commands) in an sdist.
>>
>> For example, the majority of projects I work on require Numpy to build
>> one or two extension modules.  They require hooks to check that the
>> numpy package is importable, and then to use numpy's API to get the
>> paths to the numpy headers and and them to the include_dirs for each
>> extension module that requires them.  That's not the only one
>> though--one could have a whole suite of setup_hooks common to a bunch
>> of projects.  Custom Compiler classes are a possibility now too.
>>
>> One could ship a copy of those dependencies with each project, or have
>> some kind of bootstrap script.  But to be able to automatically
>> download and add build dependencies to the path (a la setup_requires)
>> would be much nicer.  And packaging will have pysetup, so it should be
>> doable.  (Having the same capability for test dependencies and doc
>> dependencies would be nice too, but not nearly as important).
>
>
> Certainly.  I was just saying that the generated-metadata cases need
> handling, too, and that people should also be informed that they don't need
> (and shouldn't use) setup_requires for simple metadata generation, and that
> it might be a good idea to call the feature "build_requires", and perhaps
> distinguish between "setup hooks" (for developers to have nice things) and
> "build hooks" (for stuff that absolutely has to run on the install machine).

I think we're on the same page then :)  packaging/d2 also supports
pre/post-command hooks which might, in most cases, be more appropriate
for the "build hooks that absolutely have to run" case.  In fact, I
had completely forgotten this, but my aforementioned Numpy extension
module hook is a pre-build_ext hook.  That makes it absolutely clear
that this is something we have to do before we can build an extension
module, and that it doesn't have any purpose outside that context and
shouldn't be executed ever time I run `pysetup something`.

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

2012-05-30 Thread PJ Eby 
On Wed, May 30, 2012 at 11:54 AM, Erik Bray  wrote:

> On Tue, May 29, 2012 at 7:41 PM, PJ Eby  wrote:
> >> > I might be confused; I haven't been following the goings-on of late
> with
> >> > distutils2.  At one point, I thought the plan was not to bless or
> >> > include dependency-managing installers with the stdlib, or something
> >> > like that.  i.e., I thought the plan wasn't to support or bless
> >> > full-service tools like buildout, easy_install, or pip, or anything
> >> > comparable to them.
> >>
> >> Right, yeah, the plans in this area were fluid for awhile, but the
> >> eventual conclusion was that the stdlib should have a command-line
> >> utility capable of installing packages with dependencies. That exists in
> >> default branch now; it's called pysetup. It doesn't have nearly all the
> >> features of easy_install, buildout, or pip, but it can install packages
> >> from an index with deps.
> >>
> >
> > In any case, it still doesn't change the part where it's a good idea to
> ship
> > a static setup.cfg, with hooks only needing to run on the sdist-building
> > machine, unless they are actually part of the build process.  There are
> use
> > cases for calculated data to be in the initial setup.cfg, where the
> > calculation machinery doesn't need to be on the target (like generating
> the
> > file list or version from revision control info).  So, a setup_requires
> (or
> > maybe better named "build_requires") would still be helpful, but probably
> > shouldn't be used for setup.cfg stability.
>
> That's not a bad idea for certain kinds of metadata--version/vcs info
> for example.  I like the idea of including a generated "static"
> setup.cfg in a source dist as a solution to that kind of problem.  But
> that doesn't eliminate the need for setup_hooks (or even more
> complicated objects like custom commands) in an sdist.
>
> For example, the majority of projects I work on require Numpy to build
> one or two extension modules.  They require hooks to check that the
> numpy package is importable, and then to use numpy's API to get the
> paths to the numpy headers and and them to the include_dirs for each
> extension module that requires them.  That's not the only one
> though--one could have a whole suite of setup_hooks common to a bunch
> of projects.  Custom Compiler classes are a possibility now too.
>
> One could ship a copy of those dependencies with each project, or have
> some kind of bootstrap script.  But to be able to automatically
> download and add build dependencies to the path (a la setup_requires)
> would be much nicer.  And packaging will have pysetup, so it should be
> doable.  (Having the same capability for test dependencies and doc
> dependencies would be nice too, but not nearly as important).
>

Certainly.  I was just saying that the generated-metadata cases need
handling, too, and that people should also be informed that they don't need
(and shouldn't use) setup_requires for simple metadata generation, and that
it might be a good idea to call the feature "build_requires", and perhaps
distinguish between "setup hooks" (for developers to have nice things) and
"build hooks" (for stuff that absolutely has to run on the install machine).
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Requires-Dist: unittest2; 'test' in extras

2012-05-30 Thread Daniel Holth 
Proposal:

Requires-Dist: unittest2; 'test' in extras

...

I noticed very few packages (I counted 644 out of 16k) actually include
Requires: in the sdist PKG-INFO, 5 with Requires-Dist. (requires.txt is
much more common.)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

2012-05-30 Thread Erik Bray 
On Tue, May 29, 2012 at 7:41 PM, PJ Eby  wrote:
>> > I might be confused; I haven't been following the goings-on of late with
>> > distutils2.  At one point, I thought the plan was not to bless or
>> > include dependency-managing installers with the stdlib, or something
>> > like that.  i.e., I thought the plan wasn't to support or bless
>> > full-service tools like buildout, easy_install, or pip, or anything
>> > comparable to them.
>>
>> Right, yeah, the plans in this area were fluid for awhile, but the
>> eventual conclusion was that the stdlib should have a command-line
>> utility capable of installing packages with dependencies. That exists in
>> default branch now; it's called pysetup. It doesn't have nearly all the
>> features of easy_install, buildout, or pip, but it can install packages
>> from an index with deps.
>>
>
> In any case, it still doesn't change the part where it's a good idea to ship
> a static setup.cfg, with hooks only needing to run on the sdist-building
> machine, unless they are actually part of the build process.  There are use
> cases for calculated data to be in the initial setup.cfg, where the
> calculation machinery doesn't need to be on the target (like generating the
> file list or version from revision control info).  So, a setup_requires (or
> maybe better named "build_requires") would still be helpful, but probably
> shouldn't be used for setup.cfg stability.

That's not a bad idea for certain kinds of metadata--version/vcs info
for example.  I like the idea of including a generated "static"
setup.cfg in a source dist as a solution to that kind of problem.  But
that doesn't eliminate the need for setup_hooks (or even more
complicated objects like custom commands) in an sdist.

For example, the majority of projects I work on require Numpy to build
one or two extension modules.  They require hooks to check that the
numpy package is importable, and then to use numpy's API to get the
paths to the numpy headers and and them to the include_dirs for each
extension module that requires them.  That's not the only one
though--one could have a whole suite of setup_hooks common to a bunch
of projects.  Custom Compiler classes are a possibility now too.

One could ship a copy of those dependencies with each project, or have
some kind of bootstrap script.  But to be able to automatically
download and add build dependencies to the path (a la setup_requires)
would be much nicer.  And packaging will have pysetup, so it should be
doable.  (Having the same capability for test dependencies and doc
dependencies would be nice too, but not nearly as important).

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig