Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Jeff Rush
Lennart Regebro wrote:
 2009/10/3 Ned Deily n...@acm.org:
 This is not a good experience for users.  Unless I'm missing something
 (and I hope I am), this issue really can't be hand-waved away.
 
 It's unfortunate that this comes in a minor release.

Very unfortunate, as in, it should NOT have happened.  And *especially*
without any announcement on python.org or mention on the
python-committers list of something this major.

 But at the same
 time we can hardly avoid fixing bugs just because setuptools isn't
 getting updated at the moment. It's a lose-lose situation.

Setuptools is hardly a minor software tool.  Considering that setuptools
is a major focus and key technology of this group, those making changes
to distutils should have tested against setuptools and devised a
solution providing backward compatibility, along with deprecation
warnings.  Lennart and Tarek, I know you at least are familiar with this
valuable practice in Zope for years.

At the least, those making changes to Distutils should, after testing
against Setuptools, widely announced this was coming.  It does not
reflect positively on the Distribute team and even hints of
under-the-table intentionality in forcing the community to abandon use
of setuptools.  Distribute should win because of superior technology not
forced migration.


 As I see it this will speed up adaptation of Distribute. Word will
 spread. It's not ideal, but then it's not a perfect world.

Distribute is very new and there are many folk who will not be adopting
it until it has been out for quite some time.  It is a fact of the
conservative nature of some development teams.  If setuptools were an
optional, little-used technology in Python it would not be a big deal
but it isn't.  It is not appropriate to -force- users to adopt a
particular branch of a fork, except perhaps in the move from Python 2.x
to 3.x where major changes are anticipated and there was no setuptools
for 3.x anyway.

Considering that 2.6.3 is messed up in other ways, like displaying the
SVN rc1 tag and a broken logging module, and probably will be
re-released at 2.6.4 very shortly, perhaps we can get this -at least-
into the Python docs and maybe introduce backward compatible hooks into
distutils to support setuptools.

-Jeff

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Lennart Regebro
2009/10/5 Jeff Rush j...@taupro.com:
 Very unfortunate, as in, it should NOT have happened.  And *especially*
 without any announcement on python.org or mention on the
 python-committers list of something this major.

Well this major... It's a bug fix that breaks a setuptools monkey-patch.
But yes, it was discovered before release, and maybe it should have
been discussed, I'm not on python-dev anymore.

 and even hints of
 under-the-table intentionality in forcing the community to abandon use
 of setuptools.

There are no such hints anywhere.

  Distribute should win because of superior technology not
 forced migration.

That makes no sense. You move to distribute because you have to,
because setuptools is buggy and not updated.  Until people encounter
bugs in setuptools, or need Python 3 support, they are not likely to
move to Distribute. There is no other reason than forced migration.

Also, there is no win or lose. This is not a competition.

 Distribute is very new and there are many folk who will not be adopting
 it until it has been out for quite some time.

Nobody will adopt it until they are forced to. This unfortunate bug
means people are forced to quicker than expected. I don't think that's
an actual problem.

 It is a fact of the
 conservative nature of some development teams.

Conservative development teams are not likely to either use Subversion
1.6 or Python 2.6.3, so they are not affected by any of the major
setuptools problems. I would have expected people starting to get
forced to Distribute when major distros where shipping with subversion
1.6. Now it's going to be when they ship with Python 2.6.3 instead.

I fail to see how this is a big disaster in any way. Yes, it's not
perfect, and yeah, maybe there should have been big warning signs
somewhere. But we can NOT leave bugs in Python just because setuptools
isn't getting updated. Setuptools has already been a break on Python 3
development, are we gonna lets it be a break on Python 2 bugfixes too?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] distribution packaging from Distribute

2009-10-05 Thread Rakotomandimby Mihamina

10/05/2009 02:01 PM, Tarek Ziadé:

I have planned to do it on my side once for debian package using a
recipe that was building the debian tree once the buildout was made, but this 
work
was not finished. I can send you what I have if it can help you


Please, do.

--
  Architecte Informatique chez Blueline/Gulfsat:
   Administration Systeme, Recherche  Developpement
   +261 34 29 155 34
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Ronald Oussoren
 
On Monday, 05 October, 2009, at 11:53AM, Lennart Regebro rege...@gmail.com 
wrote:
2009/10/5 Jeff Rush j...@taupro.com:
 Very unfortunate, as in, it should NOT have happened.  And *especially*
 without any announcement on python.org or mention on the
 python-committers list of something this major.

Well this major... It's a bug fix that breaks a setuptools monkey-patch.
But yes, it was discovered before release, and maybe it should have
been discussed, I'm not on python-dev anymore.

I agree with Jeff that this shouldn't have happened, or should at the very least
have been documented in the release notes for 2.6.3. I know of several users
of Python that have been bitten by this (they installed 2.6.3 and now 
easy_install
doesn't work anymore for them).  

For beginners this issue is a showstopper that they cannot resolve without help.



 Distribute is very new and there are many folk who will not be adopting
 it until it has been out for quite some time.

Nobody will adopt it until they are forced to. This unfortunate bug
means people are forced to quicker than expected. I don't think that's
an actual problem.

This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 
but requires the replacement of another component as well.  That can be a 
problem in organizations with strict configuration management where you cannot 
install new software without going to lots of red tape (and that might involve 
lawyers when you install a new package instead of upgrading an existing one).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Tarek Ziadé
On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren ronaldousso...@mac.com wrote:
Nobody will adopt it until they are forced to. This unfortunate bug
means people are forced to quicker than expected. I don't think that's
an actual problem.

 This is a problem, it means 2.6.3 is not a simple drop-in replacement for 
 2.6.2 but requires the replacement of another component as well.  That can be 
 a problem in organizations with strict configuration management where you 
 cannot install new software without going to lots of red tape (and that might 
 involve lawyers when you install a new package instead of upgrading an 
 existing one).

What is your solution ? Setuptools is not part of the standard library
and monkey-patch Distutils.
Its development has been discontinued for over a year.

Everytime Distutils is changed for anything, wether it's a bug fix or
not, Setuptools can be broken.

Now I realize that some folks wants Distutils to be aware of that and
be backward-compatible with
Setuptools monkey-patches ?

If that so, its means that Distutils can't be fixed and its code can't
be changed at all,
if Setuptools is not changed in the meantime. We can't force the
Setuptools maintainer
to do so.

Which package is supposely on the top of which one ?

A drop in replacement from 2.6.2 to 2.6.3 probably breaks some other
softwares out there
that are monkey patching some internals of Python, and the way to go
is to fix those package.

At the end, some folks here sounds like it's Distutils fault if
Setuptools is broken,
and it can't be simply because Setuptools is not maintained anymore
and monkey patches
Distutils.

Like for the svn 1.6 compat problem we had earlier this year,
this problem is a few line changes in Setuptools, it's an 1 hour work.
If your company upgrades to Python 2.6.3 it can also upgrade to an
hypothetical Setuptools 0.6c..10 ?

Tarek

-- 
Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世,因有你参与
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Lennart Regebro
2009/10/5 Ronald Oussoren ronaldousso...@mac.com:
 This is a problem, it means 2.6.3 is not a simple drop-in replacement for 
 2.6.2 but requires the replacement of another component as well.  That can be 
 a problem in organizations with strict configuration management where you 
 cannot install new software without going to lots of red tape (and that might 
 involve lawyers when you install a new package instead of upgrading an 
 existing one).

Sure, but that would have happened sooner or later anyway. Is it
really so bad that it happens now instead of say, in half a year? I
don't see why it's such a big deal. Sorry, but I don't.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Zooko Wilcox-O'Hearn

On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote:

I apologize for my part in this, but moving forward I think that if  
it's possible to patch and release a setuptools that works with  
Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen  
asap.  If that's not possible, then we might need to revert the  
distutils change in a quick Python 2.6.4.


Why not move ahead with both?  Then, for example, people who do have  
Python 2.6.3 installed and get the latest setuptools will be okay,  
and also people who have setuptools-0.6c9 installed and get the  
latest Python.  There are a lot of people who have constraints on  
what they they can deploy and when, so it wouldn't hurt to have both  
fixes available.


(Also, one of the fixes might not get done in a timely way.)

Thanks,

Zooko
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Tarek Ziadé
On Mon, Oct 5, 2009 at 3:38 PM, Barry Warsaw ba...@python.org wrote:
 On Oct 5, 2009, at 5:40 AM, Jeff Rush wrote:

 Very unfortunate, as in, it should NOT have happened.  And *especially*
 without any announcement on python.org or mention on the
 python-committers list of something this major.

 [...]

 Considering that 2.6.3 is messed up in other ways, like displaying the
 SVN rc1 tag and a broken logging module, and probably will be
 re-released at 2.6.4 very shortly, perhaps we can get this -at least-
 into the Python docs and maybe introduce backward compatible hooks into
 distutils to support setuptools.

 I apologize for my part in this, but moving forward I think that if it's
 possible to patch and release a setuptools that works with Python 2.6.3
 /and/ earlier Python 2.6.x's, then that should happen asap.  If that's not
 possible, then we might need to revert the distutils change in a quick
 Python 2.6.4.

It's technically possible to fix Setuptools. We did this fix on Distribute, and
the patch is 2 lines long.

But it's just a matter of having the maintainer doing it. A few months ago we
couldn't make him fix and release the bug that made setuptools fail
with svn 1.6,
and the year before it took several months to get it fixed for svn 1.5
(a one line, not risky change !!!)

That's why we have forked and created Distribute, to provide bug fixes.

If PJE is not concerned anymore by the maintenance, imho he should let someone
that is willing to do it take over the maintenance of his package to
fix this (and the other problems).
That is not a new problem.

Beware that I don't want to run in any new vicious thread here: I had
my share of those.

So about taking over Setuptools maintenance :
1/ I am not saying it should be me, and I am not saying that I am
offended that PJE didn't open the maintenance of setuptools to me.  I
think he should trust the community and let the maintenance of
setuptools be done by all the people that are actively working on the topic.

2/ No, as someone told me in IRC, that's not an evil plan of mine to
make people switch to Distribute. This
is not in our interest, it's a loss-loss situation.

Now I am strongly opposed to revert any bug fix change in Distutils
just because it breaks Setuptools, which is unmaintained since a year.

We have been struggling over a year with this issue. And we are still
struggling because we have
to work in a fork to try to provide solutions for the community, with
a lot of bootstrapping issues.

Now I am astonished that we are talking about reverting changes in
Distutils that were done for bugfixes,
for a third party package that does monkey patches on Distutils.

If this choice wins here, it means that setuptools and the stdlib are
tied together, and that the setuptools package should be integrated to
the stdlib *immediatly*.


Tarek
-- 
Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世,因有你参与
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Tarek Ziadé
On Mon, Oct 5, 2009 at 4:02 PM, Zooko Wilcox-O'Hearn zo...@zooko.com wrote:
 On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote:

 I apologize for my part in this, but moving forward I think that if it's
 possible to patch and release a setuptools that works with Python 2.6.3
 /and/ earlier Python 2.6.x's, then that should happen asap.  If that's not
 possible, then we might need to revert the distutils change in a quick
 Python 2.6.4.

 Why not move ahead with both?  Then, for example, people who do have Python
 2.6.3 installed and get the latest setuptools will be okay, and also people
 who have setuptools-0.6c9 installed and get the latest Python.  There are a
 lot of people who have constraints on what they they can deploy and when, so
 it wouldn't hurt to have both fixes available.

So are you saying that in an environment where you are allowed to
install Python 2.6.3,
you will not be allowed to install an hypothetical setuptools-0.6c10
(or a Distribute 0.6.3) ?

Tarek
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread sstein...@gmail.com


On Oct 5, 2009, at 7:44 AM, Lennart Regebro wrote:


2009/10/5 Ronald Oussoren ronaldousso...@mac.com:
This is a problem, it means 2.6.3 is not a simple drop-in  
replacement for 2.6.2 but requires the replacement of another  
component as well.  That can be a problem in organizations with  
strict configuration management where you cannot install new  
software without going to lots of red tape (and that might involve  
lawyers when you install a new package instead of upgrading an  
existing one).


Sure, but that would have happened sooner or later anyway. Is it
really so bad that it happens now instead of say, in half a year? I
don't see why it's such a big deal. Sorry, but I don't.


It's not the happening, it's the happening without a deprecation,  
warnings, announcements etc. deserved by a change that affects so much  
of the Python eco-system.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread K. Richard Pixley

Ronald Oussoren wrote:

For beginners this issue is a showstopper that they cannot resolve without help.
  
I'm a relative beginner to distutils/setuptools/distribute, but a long 
time configuration/build/packaging professional.  You're mistaken if you 
think that any of these technologies are suitable for beginners.  The 
state of python package distribution resembles the state of linux 
packages circa 1995, except that it isn't very well documented at all.


Easy_install certainly isn't, and never has been.  My first questions 
about easy_install, (how do I get a list of installed packages?  How do 
I upgrade all installed packages?  How do I delete a package using 
easy_install?) are all unanswerable.


I' spent over 20 hours this weekend just reading through documentation 
trying to figure out what the state of the art is expected to be here 
and I still don't know much.  That's far beyond the investment of a 
casual user.  I'm puzzled that people wonder why I haven't used standard 
packaging yet when it's pretty clear that standard packaging and 
distribution doesn't really solve my problems.


Python packaging and distribution right now is not for beginners or the 
faint of heart.


--rich - (who came here recently looking for an opportunity to write 
something akin to stdeb.)

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Packaging Distribute

2009-10-05 Thread K. Richard Pixley

Sridhar Ratnakumar wrote:
On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziadé 
ziade.ta...@gmail.com wrote:



The other way would be to use Distribute instead of Setuptools for
what the packaging system is calling setuptools. That's pretty
much what is happening in Gentoo (arch) and UHU-Linux (dev),
right now

Interesting. Gentoo uses distribute but retains the name 'setuptools'?

  DEPEND=!!dev-python/setuptools-0.6.3-r2

Ah. But what if PJE releases setuptools with the *same* version number 
0.6.3? What would the gentoo folks do in order to get the new 
setuptools release in their packaging system? Or did they make a 
decision of totally dropping setuptools from their repository? 
Debian would probably treat one side or the other as a branch.  Sounds 
like gentoo has decided that distribute is the main line, which would 
define a new release of setuptools as a branch.  So it would probably be 
something like setuptools-0.6.3pje-r0 or the like.  That way it's 
progress from 0.6.3, but would be overwritten by a 0.6.4.  (Or, they'd 
just ignore the pje release).


--rich
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread sstein...@gmail.com


On Oct 5, 2009, at 9:38 AM, Barry Warsaw wrote:


I apologize for my part in this, but moving forward I think that if  
it's possible to patch and release a setuptools that works with  
Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen  
asap.  If that's not possible, then we might need to revert the  
distutils change in a quick Python 2.6.4.


I thought the whole point of the Distribute project was that we  
couldn't _get_ a new setuptools release and, so, had to fork.


Hopefully, with this motivation, we'll get a new setuptools one more  
time! and make the transition to Distribute a little more controlled.


Fast, but controlled.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Barry Warsaw

On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote:

Python packaging and distribution right now is not for beginners or  
the faint of heart.


If we're honest with ourselves, it's not for experienced developers  
either.  Do you really even want to have to /think/ about this stuff?


-Barry



PGP.sig
Description: This is a digitally signed message part
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Lennart Regebro
2009/10/5 Barry Warsaw ba...@python.org:
 I apologize for my part in this, but moving forward I think that if it's
 possible to patch and release a setuptools that works with Python 2.6.3
 /and/ earlier Python 2.6.x's, then that should happen asap.

PJE seems interested in this, as he asked about a patch, so maybe.

 If that's not possible, then we might need to revert the distutils
 change in a quick Python 2.6.4.

That would be a big mistake.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Lennart Regebro
2009/10/5 K. Richard Pixley r...@noir.com:
 This would be a problem if distribute were in general release.  It's not.
  It's clearly a development branch which is intended to move quickly.

No, this is incorrect. The 0.6-branch is not intended to move quickly,
it is in bugfix mode.
It is moving quickly only because some major bugfixes has been made,
but it's not a development branch, that's 0.7, which isn't released.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Tarek Ziadé
On Mon, Oct 5, 2009 at 4:57 PM, Lennart Regebro rege...@gmail.com wrote:
 2009/10/5 Barry Warsaw ba...@python.org:
 I apologize for my part in this, but moving forward I think that if it's
 possible to patch and release a setuptools that works with Python 2.6.3
 /and/ earlier Python 2.6.x's, then that should happen asap.

 PJE seems interested in this, as he asked about a patch, so maybe.

It was pushed in setuptools bug tracker by Ned yesterday IIRC, with
links to the patch that works.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread K. Richard Pixley

Barry Warsaw wrote:

On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote:
Python packaging and distribution right now is not for beginners or 
the faint of heart.
If we're honest with ourselves, it's not for experienced developers 
either.  Do you really even want to have to /think/ about this stuff?
Depends.  It's what I do professionally in slightly different arenas, so 
I think about it anyway.


But I take your point.  The packaging and distribution system should be 
simple, powerful, expressive, reliable, and definitive to the point 
where it's existence fades into the background allowing one to spend 
one's time and focus of attention on less easily solved problems.


--rich
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread K. Richard Pixley

Lennart Regebro wrote:

2009/10/5 K. Richard Pixley r...@noir.com:
  

This would be a problem if distribute were in general release.  It's not.
 It's clearly a development branch which is intended to move quickly.



No, this is incorrect. The 0.6-branch is not intended to move quickly,
it is in bugfix mode.
It is moving quickly only because some major bugfixes has been made,
but it's not a development branch, that's 0.7, which isn't released.
  
I'm recent to python packaging and distribution, so let me see if I've 
put this together right from my reading of the various web pages 
involved over the weekend.


Distutils is currently part of the standard python library.  As such, 
it's released with python, (the reference implementation, anyway).  
Distutils is currently capable of producing only source archives of 
packages.  While it's capable of producing built archives, those 
archives are machine specific, nonrelocatable, untrackable, and have no 
standard method for distribution nor installation nor tracking.


Setuptools was a third party addition to, (and partial replacement of), 
distutils because distutils wasn't suitably usable nor was it moving 
fast enough.  However, since setuptools was initiated, many of the major 
features of setuptools have since been folded back into distutils, 
making setuptools partially redundant and partly colliding.  Setuptools 
provides the ability to produce machine independent built archives and 
a standard method for installing them, (although not for tracking or 
removing them).  And the setuptools approach to installation, 
easy_install, doesn't play nice with the native installers on systems 
that have them like rpm, debian, etc.


However, setuptools has fallen into disrepair and so distribute has been 
created, as a friendly branch off a third party tool which in turn was a 
form of branch off of distutils.  And within distribute, there are two 
lines of development, the 0.7 line, which is intended to replace...  I'm 
confused.  Does it replace setuptools or distutils?  And then there's 
the 0.6 branch, which is a branch off 0.7 which is a branch of 
setuptools which is a branch of distutils which is under recent active 
development and yet it also expected to be stable, as much as such a 
term can be applied to a third party branch off a third party of a 
colliding replacement with a standard facility.


Is that about right?

If I'm anywhere near right, then I can't really imagine what state you 
intend for the 0.6 branch if not development.


--rich
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Lennart Regebro
2009/10/5 K. Richard Pixley r...@noir.com:
 Is that about right?

Nope. 0.6 is a fork of setuptools, providing bugfixes (and also 3.1
support). It's completely backwards compatible with setuptools.

0.7 is a development branch, which aims to refactor setuptools into
something or (rather several somethings) that is simpler and easier to
maintain. It will not be backwards compatible.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Jeremy Sanders
K. Richard Pixley wrote:

 Ronald Oussoren wrote:
 For beginners this issue is a showstopper that they cannot resolve
 without help.
   
 I'm a relative beginner to distutils/setuptools/distribute, but a long
 time configuration/build/packaging professional.  You're mistaken if you
 think that any of these technologies are suitable for beginners.  The
 state of python package distribution resembles the state of linux
 packages circa 1995, except that it isn't very well documented at all.

As a general question, is there any planned project to improve the state of 
distutils or replace it? It appears to be one of the weakest parts of the 
Python system and needs replacing with something much cleaner, better 
documented and more powerful.

Even making something like cmake the standard would help a lot.

Jeremy

-- 
jer...@jeremysanders.net
http://www.jeremysanders.net/

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Alex Grönholm

Jeremy Sanders kirjoitti:

K. Richard Pixley wrote:

  

Ronald Oussoren wrote:


For beginners this issue is a showstopper that they cannot resolve
without help.
  
  

I'm a relative beginner to distutils/setuptools/distribute, but a long
time configuration/build/packaging professional.  You're mistaken if you
think that any of these technologies are suitable for beginners.  The
state of python package distribution resembles the state of linux
packages circa 1995, except that it isn't very well documented at all.



As a general question, is there any planned project to improve the state of 
distutils or replace it? It appears to be one of the weakest parts of the 
Python system and needs replacing with something much cleaner, better 
documented and more powerful.


Even making something like cmake the standard would help a lot.

  
There is a lack of consensus regarding how exactly they should work. If 
we are having this much trouble deciding how a third party tool should 
work, it is certainly not going to be merged into distutils until those 
issues have been resolved. Distutils is what houses (or should) the 
parts we all agree on. That said, I think that plenty of 
setuptools/distribute functionality should be moved to distutils (after 
the code has been cleaned up and the proper unit tests introduced).

Jeremy

  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Zooko Wilcox-O'Hearn

On Monday,2009-10-05, at 8:11 , Tarek Ziadé wrote:

So are you saying that in an environment where you are allowed to  
install Python 2.6.3, you will not be allowed to install an  
hypothetical setuptools-0.6c10 (or a Distribute 0.6.3) ?


Yes, situations like that can come up.  For example, I guess the  
packaging of my own Tahoe-LAFS project is a case in point.  The  
current requirements for Tahoe-LAFS are Python = 2.4.2 and  3.0,  
and the source distribution of Tahoe-LAFS comes with its own bundled  
copy of setuptools.  We haven't finished our qualification of  
Distribute so we're not ready to switch our build system to  
Distribute.  Setuptools-0.6c10 is hypothetical at this point.  What  
do we do?  We could tighten the Python version to = 2.4.2 and =  
2.6.2.  We could develop and test a patch to our bundled copy of  
setuptools to work-around this problem.  We could ask the open source  
volunteer who is testing Distribute for us to hurry up.  We could  
ask PJE to hurry up and release a new version of setuptools.
Perhaps we will end up doing more than one of these things.


Many of our users won't even be able to diagnose the problem if they  
upgrade from Python 2.6.2 to 2.6.3 and the install breaks.  They  
won't know what is wrong or how to work around it, other than by  
writing to our mailing list asking for help.


The faster this gets fixed and the more lenient Python 2.6.x is  
with respect to what versions of setuptools/Distribute it requires  
and the more lenient setuptools and Distribute are with respect to  
what version of Python they require, the better for everyone.


Thank you kindly for your attention.

Regards,

Zooko
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Zooko Wilcox-O'Hearn
I'm sorry to follow-up to my own post, but I realized that I didn't  
make something clear: the current Tahoe-LAFS source distribution  
comes with its own copy of setuptools, so even if PJE releases a new  
version of setuptools or if we patch that copy to work-around this  
problem, we're going to have to re-qualify the combination of the new  
version on our buildbot and make a new stable release of Tahoe-LAFS  
(something that typically only happens every 3 or 4 months) before  
end users will be able to install Tahoe-LAFS on Python 2.6.3.


I don't want to point fingers, because that goes nowhere.  Lots of  
people could have written their software or managed their processes  
differently in order to avoid this situation, including me.  However,  
I do want to emphasize that this is a serious problem.  Backwards  
compatibility on minor releases such as from 2.6.2 to 2.6.3 is a huge  
concern for a lot of people.  If something like this breaks --  
regardless of whose fault(s) it is -- then it reduces people's trust  
in the stability of stable updates of their infrastructural software.


I'm sorry to say that this event has already made me more hesitant to  
jump from setuptools to Distribute, just because some of the  
maintainers of Distribute have posted saying that they don't think  
this kind of thing is such a big deal.  I prefer to use packaging  
tools which are stable.  To achieve that kind of stability sometimes  
requires basically taking responsibility for other people's  
decisions, such as saying Well, setuptools-0.6c9 monkey-patches  
distutils.  We think that's a bad idea and we wish it didn't do  
that.  But, we know that a lot of other people out there are relying  
on the combination of setuptools-0.6c9 and Python 2.6.x, so we're  
going to go the extra mile to make sure that those people don't  
experience disruptions..


Please take this in the constructive spirit that is intended.  We're  
on the same team.  I've already contributed a few patches and a lot  
of bug reports to setuptools, Distribute, and Python, and I'd like to  
contribute more in the future.  Having a policy of actively working  
to maintain stability across stable updates will help everyone.


Regards,

Zooko
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread K. Richard Pixley

Alex Grönholm wrote:
There is a lack of consensus regarding how exactly they should work. 
If we are having this much trouble deciding how a third party tool 
should work, it is certainly not going to be merged into distutils 
until those issues have been resolved. Distutils is what houses (or 
should) the parts we all agree on. That said, I think that plenty of 
setuptools/distribute functionality should be moved to distutils 
(after the code has been cleaned up and the proper unit tests 
introduced). 
I agree there's a lack of consensus.  But I dont' believe that distutils 
is a strong basis for growth.  Distutils may be a reasonable choice of a 
build tool, (I'm not sure yet), but it's packaging and distribution 
support is minimal to nonexistent.


I think what's needed here isn't a consensus, (which we'll never get), 
but rather a generational solution based not on available technology, 
but rather on state of the art requirements.


For example, distutils says nothing about host packaging systems nor 
distribution.  The distutils doc doesn't really even broach the topic of 
impure python packages or distribution.  I don't know, for instance, 
whether C extensions are expected to be contained in a collection of 
different tar.gz archives or if there's expected to be one fat one 
which contains binary C extensions for a collection of different 
operating systems.


The new system needs to support mass updates of all installed packages, 
querying of installed packages, removal of installed packages, recursive 
instances of the packaging database, (perhaps with something like 
virtualenv), as well as all of the things that easy_install supports 
now, and most of the features that debian/rpm/etc support now including 
virtual packages, dependencies, suggestions, conflicts, and both command 
line and gui installers.  It should also be capable of at least playing 
nice with the host packaging system, (eg, rpm, deb, ipk, etc), or 
perhaps of completely excusing itself from the distribution process, 
(other than supporting the native distribution process).  If necessary, 
we can create extra repositories for those systems such that python 
packages can be distributed through the native packaging system, even 
though we're managing our own release processes.  I know that 
debian/ubuntu/rpm/easy_install all allow for this as it's very common 
for enterprises to use such a solution for distributing their own 
packages in-house.


The new system should also support cross compilation to the extent 
that such is needed for C extensions and to the extent that the native 
packaging system supports.  (Eg, debian doesn't, really but bitbake/OE 
virtually requires it).


Most of what I'm talking about here speaks to packaging formats, 
distribution processes, and installation processes.  And this isn't new 
technology.  Both debian, rpm, and several other unix technologies have 
fine systems in operation right now.  Sure, they all have weaknesses, 
but they are much better than easy_install.


--rich


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] [ANN] stdeb 0.3.2 and 0.4.1

2009-10-05 Thread Gerry Reno

Andrew,
I installed stdeb 0.3.2 from PyPi on my Hardy server and ran a couple 
tests just to be sure. stdeb 0.3.2 on Hardy is working fine. I didn't 
see any problems. 'bdist_deb' worked as expected and generated both a 
.dsc and a .deb file for my project which installed perfectly.


Regards,
Gerry

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Hanno Schlichting
On Mon, Oct 5, 2009 at 1:33 PM, Ned Deily n...@acm.org wrote:
 I've opened an issue of the main Python issue tracker outlining the
 problem, primarily for the benefit of affected users who search the
 tracker:

   http://bugs.python.org/issue7064

If I understand the comments on this ticket correctly, Tarek has
changed distutils in a way so the last setuptools release continues to
work, correct?

So based on the current state of Python 2.6.4 will work again with an
unmodified setuptools 0.6c9?

Hanno
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread P.J. Eby

At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote:

How do I delete a package using easy_install?


http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread P.J. Eby

At 04:57 PM 10/5/2009 +0200, Lennart Regebro wrote:

2009/10/5 Barry Warsaw ba...@python.org:
 I apologize for my part in this, but moving forward I think that if it's
 possible to patch and release a setuptools that works with Python 2.6.3
 /and/ earlier Python 2.6.x's, then that should happen asap.

PJE seems interested in this, as he asked about a patch, so maybe.


I expect to have a small amount of time later this week to work on 
setuptools.  No guarantees, but this is certainly one of the bugs on 
my short list for doing something with.




 If that's not possible, then we might need to revert the distutils
 change in a quick Python 2.6.4.

That would be a big mistake.


Actually, I think the mistake here is where non-bugfix code was 
ported from 3.x back to the 2.6.x maintenance branch.  I'm 
particularly concerned about the .compiler / ._compiler change, as it 
seems to me to be responsible for breaking mingw32 compilation on 
Windows, but I haven't had time to investigate thoroughly.  (It 
actually looks like a problem that might be in Python 3 as well.)


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread P.J. Eby

At 06:53 PM 10/5/2009 +0200, Lennart Regebro wrote:

Possibly if you somehow
think it's the Distribute teams fault that a bugfix in Python ended up
breaking setuptools. If it would have been better not to fix that bug,
then the blame reasonably goes to the Python core developers, not the
Distribute team.


In this case, though, the Python core developer is also the 
Distribute lead.  (i.e., it was Tarek who made the changes to the 
distutils.)  So it's a bit understandable that some people might 
wonder if there was a conflict of interest.


I don't personally think that's the case; it's pretty much inevitable 
that the distutils making progress means other things will 
break.  But it's easy to see how others might take the situation 
another way, or treat it as an example of Distribute policy towards 
backward compatibility, or of what kind of breakage is considered 
acceptable in a dot release.


It would be good to bear in mind that extending the distutils (or 
setuptools) is *not* monkeypatching; both libraries provide explicit 
assurance that subclassing is in fact allowed.  And there's nothing 
all that special about setuptools' subclassing of build_ext; in fact, 
if you look back in the archives here, other people have done 
equivalent subclassing to support dynamic library building.  I 
haven't checked their code, but there is a strong possibility that it 
would also fail in the same way.  This is not really about 
monkeypatching, or about special support for setuptools.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread K. Richard Pixley

P.J. Eby wrote:

At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote:

How do I delete a package using easy_install?

http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages
That doesn't remove a package.  It simply removes the package from the 
search path by one method in hopes that no further instances of it will 
be loaded.


To actually remove it, you have to know the format of the library, the 
package formats, local library storage conventions, and you need to be 
an expert user of distutils, setuptools, buildout, etc, in order to 
determine the content of the package itself in order to remove it 
manually.  And even then you'd have to manually search your entire 
system for any packages that might still depend on this package lest you 
break them too.


That's far beyond the scope of expectation for a casual package 
maintainer.  That's more akin to current macosx standards, (install 
only), than debian or rpm, (install, update, query, remove, etc).


I suppose it's not such a bad problem if you reload your OS frequently.  
Or solely work with virtualenv or the like so that you can easily 
discard and rebuild an environment whenever you need one.  Of course, 
that doesn't really help with the task of keeping multiple servers 
current or rolling internal software out to an enterprise.


--rich
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread P.J. Eby

At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote:
If I understand the comments on this ticket correctly, Tarek has 
changed distutils in a way so the last setuptools release continues 
to work, correct?


Yes.  And a very nice fix, done quite quickly.  Thank you Tarek.


So based on the current state of Python 2.6.4 will work again with 
an unmodified setuptools 0.6c9?


AFAICT, that is correct.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Zooko Wilcox-O'Hearn
I'm struggling to articulate something here.  When the maintainer of  
the stable branch of a platform that I rely on says The fact that  
upgrading to our recent stable release will break this critical  
functionality is so-and-so's fault, not ours. this reduces my  
confidence in that maintainer.  Not because he's wrong!  Maybe it  
*is* so-and-so's fault.  But what I'm looking for in the maintainer  
of a stable platform is someone who says Maybe this wasn't our  
fault, but here are the steps we're taking to get you back on your  
feet as soon as possible..


Regards,

Zooko

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Hanno Schlichting
On Mon, Oct 5, 2009 at 8:38 PM, P.J. Eby p...@telecommunity.com wrote:
 At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote:

 If I understand the comments on this ticket correctly, Tarek has changed
 distutils in a way so the last setuptools release continues to work,
 correct?

 Yes.  And a very nice fix, done quite quickly.  Thank you Tarek.

Wonderful. This seems to be the right approach to the current problem
for me. Thank you Tarek indeed!

 So based on the current state of Python 2.6.4 will work again with an
 unmodified setuptools 0.6c9?

 AFAICT, that is correct.

Good. So we can hopefully end this thread and move on to something
more productive.

If anyone wants to step up and provide help in testing pre-releases of
Python, so we can avoid similar situation in the future, that would be
most welcome.

Hanno
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


[Distutils] distutils: packaging a generated configuration file

2009-10-05 Thread Mark Dickinson
I'm trying to use distutils for the first time to package up a
project, and struggling a bit;  I wonder whether some kind soul could
point me in the right direction.

I'm packaging a pure Python project that uses ctypes.  The only
complication is that I'd like the setup script to install a
configuration file alongside the various .py files.  From the
documentation, this sounds like something that would usually be
specified with 'package_data' or 'data_files'.  The catch is that
parts of the configuration file are generated at setup time:  that is,
the setup script gathers various pieces of system information (e.g.,
library locations) and uses those to transform a hard-coded
'config.in' file into the 'config' file that I want to install.

I could just generate the 'config' file in the top-level package
directory, but it seems dangerous to assume that that directory is
writable; it seems better to use one of the build directories that
distutils already knows about.

What's the best way to manage generation and installation of the
config file?  I've got as far as subclassing the build command and
making __run__ create the config file from the config.in file, putting
it into the build_temp directory;  I can't figure out how to tell
distutils to install it in the right place.

It seems like I want a sort of build_data/install_data pair of
commands, that allows creation and installation of *generated* data
files.

Any hints?

Mark
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread P.J. Eby

At 11:29 AM 10/5/2009 -0700, K. Richard Pixley wrote:

P.J. Eby wrote:

At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote:

How do I delete a package using easy_install?

http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages
That doesn't remove a package.  It simply removes the package from 
the search path by one method in hopes that no further instances of 
it will be loaded.


To actually remove it, you have to know the format of the library, 
the package formats, local library storage conventions, and you need 
to be an expert user of distutils, setuptools, buildout, etc, in 
order to determine the content of the package itself in order to 
remove it manually.


As it says at the above link:

After you've done this, you can safely delete the .egg files or 
directories, along with any scripts you wish to remove.


What it doesn't mention (but which should be apparent if you actually 
run the command) is that it will output the locations of the .egg 
files or directories and scripts in question, allowing you to copy 
and paste them to 'rm' or 'del' commands.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Paul Moore
2009/10/5 K. Richard Pixley r...@noir.com:
 I'm recent to python packaging and distribution, so let me see if I've put
 this together right from my reading of the various web pages involved over
 the weekend.

 Distutils is currently part of the standard python library.  As such, it's
 released with python, (the reference implementation, anyway).  Distutils is
 currently capable of producing only source archives of packages.  While it's
 capable of producing built archives, those archives are machine specific,
 nonrelocatable, untrackable, and have no standard method for distribution
 nor installation nor tracking.

 Setuptools was a third party addition to, (and partial replacement of),
 distutils because distutils wasn't suitably usable nor was it moving fast
 enough.  However, since setuptools was initiated, many of the major features
 of setuptools have since been folded back into distutils, making setuptools
 partially redundant and partly colliding.  Setuptools provides the ability
 to produce machine independent built archives and a standard method for
 installing them, (although not for tracking or removing them).  And the
 setuptools approach to installation, easy_install, doesn't play nice with
 the native installers on systems that have them like rpm, debian, etc.

 However, setuptools has fallen into disrepair and so distribute has been
 created, as a friendly branch off a third party tool which in turn was a
 form of branch off of distutils.  And within distribute, there are two lines
 of development, the 0.7 line, which is intended to replace...  I'm
 confused.  Does it replace setuptools or distutils?  And then there's the
 0.6 branch, which is a branch off 0.7 which is a branch of setuptools which
 is a branch of distutils which is under recent active development and yet it
 also expected to be stable, as much as such a term can be applied to a third
 party branch off a third party of a colliding replacement with a standard
 facility.

 Is that about right?

 If I'm anywhere near right, then I can't really imagine what state you
 intend for the 0.6 branch if not development.

I believe that the Distribute 0.6 branch is a stable continuation of
the setuptools (0.6) stable branch. Distribute 0.6 is intended to
include maintenance-only stable fixes (on the basis that setuptools no
longer provides even that minimal level of progress - something on
which I won't comment for fear of starting another long thread).

Distribute 0.7 is the development branch, looking at new features such
as Python 3 support.

The only other point you missed is that there is not universal
approval of setuptools as a solution to the packaging problem. Take-up
of setuptools (and eggs, and easy install, and the various other
aspects of the setuptools ecosystem) is variable, with (as far as I
see it) strong support from the web development community, and mixed
reception elsewhere in the Python community.

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Sridhar Ratnakumar

On Mon, 05 Oct 2009 11:21:28 -0700, P.J. Eby p...@telecommunity.com wrote:

And there's nothing all that special about setuptools' subclassing of  
build_ext; in fact, if you look back in the archives here, other people  
have done equivalent subclassing to support dynamic library building.  I  
haven't checked their code, but there is a strong possibility that it  
would also fail in the same way.


Correct. pywin32-212 broke in similar way -  
http://bugs.python.org/issue7020 - which issue was fixed in pywin32 trunk.


And Tarek commented on the incompleteness of the `get_ext_filename` API:

TAREK: [...] But this API, even if its doctest doesn't make it clear (I  
will change

it to make it clearer) is used for both namespaced names and non
namespaced names by the community.

Personally, I prefer that setuptools is fixed (in accordance with the  
clarified API behavior). Yet, 2.6.3 currently has a critical bug open with  
the logging module that may warrant a quick 2.6.4 release.


-srid


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] distutils: packaging a generated configuration file

2009-10-05 Thread Wolodja Wentland
On Mon, Oct 05, 2009 at 20:02 +0100, Mark Dickinson wrote:
[...]
 specified with 'package_data' or 'data_files'.  The catch is that
 parts of the configuration file are generated at setup time:  that is,
 the setup script gathers various pieces of system information (e.g.,
 library locations) and uses those to transform a hard-coded
 'config.in' file into the 'config' file that I want to install.
[...]

How do you get this data? (just curious)

 What's the best way to manage generation and installation of the
 config file?  I've got as far as subclassing the build command and
 making __run__ create the config file from the config.in file, putting
 it into the build_temp directory;  I can't figure out how to tell
 distutils to install it in the right place.

What do you consider right place? Where do you expect your config file
to be installed and how do you find it from within your code?

 It seems like I want a sort of build_data/install_data pair of
 commands, that allows creation and installation of *generated* data
 files.

I have been in a similar situation some time ago and had one simple, but
unsupported, need when packaging data files. I wanted to access them
from within my code after they have been installed. I detailed the
problems in [1].

The solution I came up with [2] was to subclass the build_py command in
distutils, include a 'build_info.py' at the package root and modify that
file within the subclassed build_py command. I am still not sure if that
is a good approach and haven't got any feedback on it.

If the config file you want to install is *not* a python module and you
expect it to be installed in DATA_PREFIX/config you might have some
success with subclassing the install_data command, modify the config
file before it is copied and include the file with package_data or
data_files. You could also try to modify the data_files attribute of
that problem if you can't specify all generated data files beforehand.
But that might just cause havoc and major breakage...

Note that installing data files within the python package violates the
FHS, so I would recommend using data_files, if you are interested to
have your software included in any *nix or BSD distribution. You will
still have a problem to find it later on though.

I hope that helps and I am *very* interested in any other way to solve
this. 

good luck

Wolodja Wentland

[1] http://mail.python.org/pipermail/distutils-sig/2009-September/013238.html
[2] http://mail.python.org/pipermail/distutils-sig/2009-September/013284.html


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Bill Janssen
Zooko Wilcox-O'Hearn zo...@zooko.com wrote:

 I'm struggling to articulate something here.  When the maintainer of
 the stable branch of a platform that I rely on says The fact that
 upgrading to our recent stable release will break this critical
 functionality is so-and-so's fault, not ours. this reduces my
 confidence in that maintainer.  Not because he's wrong!  Maybe it
 *is* so-and-so's fault.  But what I'm looking for in the maintainer
 of a stable platform is someone who says Maybe this wasn't our
 fault, but here are the steps we're taking to get you back on your
 feet as soon as possible..

Zooko, I've struggled with this over the last year, in integrating a
dozen fairly complicated third-party Python extensions for my system.

I've come to the conclusion that the problem is setuptools, and I'm
trying hard to remove it entirely from my system.

I have no problem with the .egg format or the basic idea, or even the
implementation, which I think is pretty nice.  It's the structure of the
setuptools project that gives me pause.  There seems to be one
developer, and he seems to be too busy to fix the well-known bugs (like
having easy_install ruin the sys.path settings by putting stuff on it in
the wrong place -- is that one fixed yet?).

In addition, I think the mere existence of setuptools stifles progress
on distutils, which is where all the clever tricks of setuptools should
properly appear.  This would let the whole community of Python developers
work on the codebase.

I'd like to see a flag on PyPI marking whether the package relies on
setuptools, in which case I'll avoid it, or voluntarily entangles itself
with setuptools if present (as the only known way to create eggs), or is
setuptools-free (my preferred configuration).  Frankly, I'd also recommend
putting up a warning to developers on PyPI noting these problems with
setuptools.

Bill
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


[Distutils] how to build location-independent bdist packages on OS X?

2009-10-05 Thread Bill Janssen
I'm trying to bundle up a Python package with a C extension in it for
Python 2.6 on an OS X 10.6 system.  I don't want to install it in the
normal /Library/Python/2.6/site-packages/ location.

But when I try python setup.py bdist, it builds a tar file with the
prefix /Library/Python/2.6/site-packages as a prefix to each file.  I
then tried python setup.py bdist_dumb with the same result.  I then
tried python setup.py bdist --prefix=., but --prefix isn't
recognized by bdist.

What's the appropriate magic here?

Oh, and I don't want to have to use setuptools.

TIA,

Bill
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Paul Moore
2009/10/5 Jeremy Sanders jer...@jeremysanders.net:
 As a general question, is there any planned project to improve the state of
 distutils or replace it? It appears to be one of the weakest parts of the
 Python system and needs replacing with something much cleaner, better
 documented and more powerful.

Tarek's working on enhancements to distutils. The current breakage of
setuptools demonstrates why this is a slow, painful, process :-(

 Even making something like cmake the standard would help a lot.

There's no plan for anything like that. I'll leave it to you to
imagine the cries of horror from the community when such a radical
change meant that easy_install no longer worked, and there was no
prospect of fixing it :-)

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Alex Grönholm

Bill Janssen kirjoitti:

Zooko Wilcox-O'Hearn zo...@zooko.com wrote:

  

I'm struggling to articulate something here.  When the maintainer of
the stable branch of a platform that I rely on says The fact that
upgrading to our recent stable release will break this critical
functionality is so-and-so's fault, not ours. this reduces my
confidence in that maintainer.  Not because he's wrong!  Maybe it
*is* so-and-so's fault.  But what I'm looking for in the maintainer
of a stable platform is someone who says Maybe this wasn't our
fault, but here are the steps we're taking to get you back on your
feet as soon as possible..



Zooko, I've struggled with this over the last year, in integrating a
dozen fairly complicated third-party Python extensions for my system.

I've come to the conclusion that the problem is setuptools, and I'm
trying hard to remove it entirely from my system.

I have no problem with the .egg format or the basic idea, or even the
implementation, which I think is pretty nice.  It's the structure of the
setuptools project that gives me pause.  There seems to be one
developer, and he seems to be too busy to fix the well-known bugs (like
having easy_install ruin the sys.path settings by putting stuff on it in
the wrong place -- is that one fixed yet?).

In addition, I think the mere existence of setuptools stifles progress
on distutils, which is where all the clever tricks of setuptools should
properly appear.  This would let the whole community of Python developers
work on the codebase.

I'd like to see a flag on PyPI marking whether the package relies on
setuptools, in which case I'll avoid it, or voluntarily entangles itself
with setuptools if present (as the only known way to create eggs), or is
setuptools-free (my preferred configuration).  Frankly, I'd also recommend
putting up a warning to developers on PyPI noting these problems with
setuptools.

  
Does your bug still exist in Distribute? If so, please report it at 
http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is 
operational, which it currently isn't)

Bill
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] [ANN] stdeb 0.3.2 and 0.4.1

2009-10-05 Thread Gerry Reno

Andrew,
Have you already or are you planning on submitting 'python-stdeb' 
packages to the debian and ubuntu repositories?


Regards,
Gerry

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Bill Janssen
Alex Grönholm alex.gronh...@nextday.fi wrote:

 Does your bug still exist in Distribute? If so, please report it at
 http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is
 operational, which it currently isn't)

Sorry, Alex, I don't know about Distribute, don't (particularly) care.
If you care, test for it and report it if it's there.  It's bug 53 in
the setuptools bug reporter.

Bill
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Alex Grönholm

Bill Janssen kirjoitti:

Alex Grönholm alex.gronh...@nextday.fi wrote:

  

Does your bug still exist in Distribute? If so, please report it at
http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is
operational, which it currently isn't)



Sorry, Alex, I don't know about Distribute, don't (particularly) care.
If you care, test for it and report it if it's there.  It's bug 53 in
the setuptools bug reporter.

Bill
  
If you are seriously expecting setuptools to be fixed, I can only assume 
you haven't been following the conversation on this list.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Ronald Oussoren


On 5 Oct, 2009, at 16:25, K. Richard Pixley wrote:


Ronald Oussoren wrote:
For beginners this issue is a showstopper that they cannot resolve  
without help.


I'm a relative beginner to distutils/setuptools/distribute, but a  
long time configuration/build/packaging professional.  You're  
mistaken if you think that any of these technologies are suitable  
for beginners.  The state of python package distribution resembles  
the state of linux packages circa 1995, except that it isn't very  
well documented at all.


I didn't say that distutils is suitable for beginners, but beginners  
to use them and are confused when they stop working.


Easy_install certainly isn't, and never has been.  My first  
questions about easy_install, (how do I get a list of installed  
packages?  How do I upgrade all installed packages?  How do I delete  
a package using easy_install?) are all unanswerable.


All of these are not yet implemented. PJE has mentioned that he wants  
to work on this in for setuptools 0.7, but development of setuptools  
has completely stalled.   The distribute package was created because  
of this, and development is moving forward again.


Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

2009-10-05 Thread Ronald Oussoren


On 5 Oct, 2009, at 16:37, K. Richard Pixley wrote:


Ronald Oussoren wrote:
This is a problem, it means 2.6.3 is not a simple drop-in  
replacement for 2.6.2 but requires the replacement of another  
component as well.  That can be a problem in organizations with  
strict configuration management where you cannot install new  
software without going to lots of red tape (and that might involve  
lawyers when you install a new package instead of upgrading an  
existing one).
This would be a problem if distribute were in general release.  It's  
not.  It's clearly a development branch which is intended to move  
quickly.  People using distribute are taking development, pre-alpha  
kinds of risk and that has been made pretty clear already.


AFAIK distribute 0.6 is a stable release, basicly setuptools 0.6c9 +  
bugfixes + py3k support.


Installing distribute is therefore not problematic for most people, if  
they know that the project exists.  The fact that distribute is a  
seperate project from setuptools can be a problem for people:  
installing a bugfix release for a software product that we're already  
using at work is significantly easier than introducing a new software  
product.


Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig