Re: [Distutils] command hooks...

[PJ Eby]

On Tue, May 29, 2012 at 5:17 PM, Carl Meyer  wrote:

>
> On 05/29/2012 02:01 PM, PJ Eby wrote:
> > On Tue, May 29, 2012 at 1:24 PM, Carl Meyer  wrote:
> > On 05/29/2012 08:19 AM, PJ Eby wrote:
> > > Right, and I don't think distutils2 can really add setup_requires
> > > without blessing a package manager.
> >
> > I'm confused by this statement. distutils2 _includes_ a package
> manager
> > (pysetup); it has no need to bless an external one. What am I
> missing?
> >
> > I might be confused; I haven't been following the goings-on of late with
> > distutils2.  At one point, I thought the plan was not to bless or
> > include dependency-managing installers with the stdlib, or something
> > like that.  i.e., I thought the plan wasn't to support or bless
> > full-service tools like buildout, easy_install, or pip, or anything
> > comparable to them.
>
> Right, yeah, the plans in this area were fluid for awhile, but the
> eventual conclusion was that the stdlib should have a command-line
> utility capable of installing packages with dependencies. That exists in
> default branch now; it's called pysetup. It doesn't have nearly all the
> features of easy_install, buildout, or pip, but it can install packages
> from an index with deps.
>
>
In any case, it still doesn't change the part where it's a good idea to
ship a static setup.cfg, with hooks only needing to run on the
sdist-building machine, unless they are actually part of the build
process.  There are use cases for calculated data to be in the initial
setup.cfg, where the calculation machinery doesn't need to be on the target
(like generating the file list or version from revision control info).  So,
a setup_requires (or maybe better named "build_requires") would still be
helpful, but probably shouldn't be used for setup.cfg stability.

One use case I ran across rather late in the game with setuptools was the
part where sdist distributions need to be capable of rebuilding an sdist,
for at least the bdist_rpm case, if not others.  That had me tearing my
hair, since I was dependent on what I could do on top of existing
distutils.  But for distutils2, shipping an updated setup.cfg (possibly
*minus* certain setup_hooks) would solve the problem nicely.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

[Carl Meyer]

On 05/29/2012 02:01 PM, PJ Eby wrote:
> On Tue, May 29, 2012 at 1:24 PM, Carl Meyer  wrote:
> On 05/29/2012 08:19 AM, PJ Eby wrote:
> > Right, and I don't think distutils2 can really add setup_requires
> > without blessing a package manager.
> 
> I'm confused by this statement. distutils2 _includes_ a package manager
> (pysetup); it has no need to bless an external one. What am I missing?
> 
> I might be confused; I haven't been following the goings-on of late with
> distutils2.  At one point, I thought the plan was not to bless or
> include dependency-managing installers with the stdlib, or something
> like that.  i.e., I thought the plan wasn't to support or bless
> full-service tools like buildout, easy_install, or pip, or anything
> comparable to them.

Right, yeah, the plans in this area were fluid for awhile, but the
eventual conclusion was that the stdlib should have a command-line
utility capable of installing packages with dependencies. That exists in
default branch now; it's called pysetup. It doesn't have nearly all the
features of easy_install, buildout, or pip, but it can install packages
from an index with deps.

Carl



signature.asc
Description: OpenPGP digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

[PJ Eby]

On Tue, May 29, 2012 at 1:24 PM, Carl Meyer  wrote:

>
>
> On 05/29/2012 08:19 AM, PJ Eby wrote:
> > On Tue, May 29, 2012 at 10:20 AM, Erik Bray  > > wrote:
> >
> > This particular solution works for me.  But the point is that it can
> > be done pretty easily.  However, the lack of a setup_requires-like
> > feature still makes things pretty impossible short of shipping a copy
> > of all the required setup hooks with the projects that use them.
> > Certainly doable, but far from ideal.
> >
> >
> > Right, and I don't think distutils2 can really add setup_requires
> > without blessing a package manager.
>
> I'm confused by this statement. distutils2 _includes_ a package manager
> (pysetup); it has no need to bless an external one. What am I missing?
>

I might be confused; I haven't been following the goings-on of late with
distutils2.  At one point, I thought the plan was not to bless or include
dependency-managing installers with the stdlib, or something like that.
i.e., I thought the plan wasn't to support or bless full-service tools like
buildout, easy_install, or pip, or anything comparable to them.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

[Carl Meyer]

On 05/29/2012 08:19 AM, PJ Eby wrote:
> On Tue, May 29, 2012 at 10:20 AM, Erik Bray  > wrote:
> 
> This particular solution works for me.  But the point is that it can
> be done pretty easily.  However, the lack of a setup_requires-like
> feature still makes things pretty impossible short of shipping a copy
> of all the required setup hooks with the projects that use them.
> Certainly doable, but far from ideal.
> 
> 
> Right, and I don't think distutils2 can really add setup_requires
> without blessing a package manager.

I'm confused by this statement. distutils2 _includes_ a package manager
(pysetup); it has no need to bless an external one. What am I missing?

Carl



signature.asc
Description: OpenPGP digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

[PJ Eby]

On Tue, May 29, 2012 at 10:20 AM, Erik Bray  wrote:

> This particular solution works for me.  But the point is that it can
> be done pretty easily.  However, the lack of a setup_requires-like
> feature still makes things pretty impossible short of shipping a copy
> of all the required setup hooks with the projects that use them.
> Certainly doable, but far from ideal.
>

Right, and I don't think distutils2 can really add setup_requires without
blessing a package manager.  That means the alternative is shipping an
altered setup.cfg with sdist builds, or using tools that generate setup.cfg
in the first place, such that setup.cfg isn't the project's canonical form.

Probably the simplest way to do it would be to just ship setup.cfg in the
sdist -- as modified by the setup hooks, since this keeps the developer
from having to use a different tool to generate the setup.cfg; it can all
be done by setup hooks.

In that case, the setup hooks themselves would need to be idempotent, so
they don't mess things up if they run a second time in an sdist-ed version
of the project.

Another alternative would be to use a package manager bootstrap script in
the project directory (ala ez_setup.py), which could then read and process
a setup_requires from setup.cfg.  But that's a much heavier-weight process;
it would be much preferable if people could make all their setup_requires
stuff run off an original copy only, and not be needed in an sdist copy.
So for example, if you need Pyrex for generating C code, it should run
pre-sdist, not post-sdist.

(OTOH, it's possible that there are some distributions you need for
building the package, for platform-specific build steps.)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] one project into several sub projects???

[Erik Bray]

On Tue, May 29, 2012 at 12:04 AM, Rob Healey  wrote:
> Greetings:
>
> I would like to know what would be the best way to handle this situation???
>
> I have a project that needs to be split into several smaller pieces/
> projects:
> 1) core
> 2) gui
> 3) cli
> 4) web
>
> Could you give me the best case scenario to split up this one project?  It
> would be best if I used DU2 or Packaging...  I need something that will be
> supported for as while to come...
>
> Our project will be moving into Python3 compatibility...

Rob,

That's a big question with no simple answer, and might be better
addressed to comp.lang.python or the like.  How you do this isn't tied
to the packaging framework you use (thought it might be best to keep
the packaging aspect as simple as possible).

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] command hooks...

[Erik Bray]

On Sun, May 27, 2012 at 12:58 PM, PJ Eby  wrote:
> Is setup.cfg already in dist-info?  I guess if it isn't, you could just add
> it, using the same mechanism above.  Hm  (goes to look at packaging
> docs)...  Ouch.  I'm seeing a bigger problem, which is that without either
> the ability to do "setup_requires" or to ship an sdist with a hook-altered
> setup.cfg, it doesn't look like you can actually implement all of
> setuptools' build functionality with just packaging.  (e.g. Setuptools can
> ship an sdist whose contents and version were determined using revision
> control info, but which does not then require the revision control tool when
> installing from the sdist, as the sdist contains a pre-built manifest, and a
> pre-built setup.cfg with hardcoded version numbers copied from the original
> revision control info.)

I've already got sort of a version of that for some of the packages I
maintain here:
https://svn.stsci.edu/trac/ssb/stsci_python/browser/stsci.distutils/trunk/stsci/distutils/hooks.py#L91

It's a basic setup_hook that tacks the SVN revision on to version
string in the metadata (and only if the version contains ".dev").  The
SVN info itself comes from a generated module called 'version.py' that
ships with the sdist and contains hard-coded version info and SVN
revisions.  The main reason it's a Python module is that it also
contains an __version__ that can be imported by the main module of the
package.

This particular solution works for me.  But the point is that it can
be done pretty easily.  However, the lack of a setup_requires-like
feature still makes things pretty impossible short of shipping a copy
of all the required setup hooks with the projects that use them.
Certainly doable, but far from ideal.

> I really hope I'm not going to have to write setuptools2 to work around
> these limitations.  ;-)   (Preferably, if I do write a setuptools2, I hope
> it can all be done using proper setup hooks on top of packaging/distutils2,
> without monkeypatching anything.)
>

Please, please no monkeypatching :)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distutils2: Does uploading to pypi work yet?

[Erik Bray]

On Sun, May 27, 2012 at 11:19 AM, Todd DeLuca  wrote:
> I recently failed to upload a project to pypi using distutils2 and failed to
> install the project using pip.  Are distutils2 and pip not yet supporting
> 'setup.cfg' only projects yet?
>
> The upload failed to upload the tarball to pypi because of what looks like a
> problem encoding a multipart form message with
> distutils2.util.encode_multipart(). I tried to fix that problem (using a
> base64 encoding) but ran into 500 http return code from the pypi server.
> The uploading code seems pretty 'alpha', so I was wondering if anyone is
> successfully using 'pysetup' to upload projects to pypi.
>
> After failing to upload the distribtion, I uploaded it to github and then
> registered my project with a github download url.  Then I tried installing
> my project using pip and that failed too, since I did not have a setup.py,
> only a setup.cfg, in the project.
>
> I'm using python 2.7.3, pip 1.1, and distutils2 from
> hg.python.org/distutils2 (pulled and updated) on Mac OS X 10.6.8.

I can't speak for PyPI or pip, but not a lot of stuff works with
setup.cfg-only distutils2/packaging projects.  Though I'm fairly
certain they shouldn't be expected to work yet.

Shameless plug, but have you tried d2to1
(http://pypi.python.org/pypi/d2to1)?  It allows you to use a setup.cfg
with existing tools like pip by way of a simple stub setup.py.  It
might not suit all your needs, but it also might :)

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] `pysetup create`, but `pysetup run install_dist` - wtf?

[Alexis Métaireau]

Le mar. 29 mai 2012 12:19:33 CEST, Alexis Métaireau a écrit :

Le mar. 29 mai 2012 07:55:51 CEST, anatoly techtonik a écrit :

http://docs.python.org/dev/packaging/tutorial.html#running-commands
What's the point in using 'pysetup run install_dist' instead of just
'pysetup install' ??


They are two different commands which aren't doing the same thing. One
installs the distribution, without dealing with its dependencies (the
install_dist command) while the other fetches and deals with the
dependencies.

This probably should be stated more clearly in the tutorial, thanks
for pointing this out.


I just created a bug requesting this: 
http://bugs.python.org/issue14949. I might take care of that later.


Thanks again,
Alexis
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] `pysetup create`, but `pysetup run install_dist` - wtf?

[Alexis Métaireau]

Le mar. 29 mai 2012 07:55:51 CEST, anatoly techtonik a écrit :

http://docs.python.org/dev/packaging/tutorial.html#running-commands
What's the point in using 'pysetup run install_dist' instead of just
'pysetup install' ??


They are two different commands which aren't doing the same thing. One 
installs the distribution, without dealing with its dependencies (the 
install_dist command) while the other fetches and deals with the 
dependencies.


This probably should be stated more clearly in the tutorial, thanks for 
pointing this out.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig