Re: [Distutils] Buildout release news
On Fri, Sep 10, 2010 at 2:10 PM, Gary Poster wrote: ... > turns out the Mac system Python does something weird and different itself I'm shocked, shocked! ;) Jim -- Jim Fulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] please fix easy_install shorter URL preference (Was: easy_install wrong download site preference)
Hello Phillip, Again, I've run into completely irrational 'easy_install' behavior that prefers to download from shorter URLs instead of PyPI ones if the same filename exists in both places. `protobuf` project and `gerald` are not installable due to this behavior. http://code.google.com/p/protobuf/issues/detail?id=66 https://sourceforge.net/apps/trac/halfcooked/ticket/29 Gerald is the second package from the last 4 months that I couldn't install due to this very obscure behavior, and I suspect there are more. Phillip, could you, please, still run through your archives to success in explaining why this logic of choosing a shorter download URL is necessary? I'd really to see what packages benefit from it, because I believe Google protocol buffers popularity alone is enough to change the behavior. Thanks. -- anatoly t. On Fri, Jul 2, 2010 at 5:06 AM, P.J. Eby wrote: > At 01:33 AM 7/2/2010 +0300, anatoly techtonik wrote: >> >> On Fri, Jul 2, 2010 at 12:10 AM, P.J. Eby wrote: >> >> > >> >> > It prefers newer packages, or, if the versions are the same, it >> >> > prefers >> >> > the shortest download URL. Â In this case, the Google Code url is >> >> > shorter. >> >> >> >> That's illogical. Better prefer PyPI if versions are the same. >> > >> > The "shortest path" logic is there to avoid certain file recognition >> > problems that occur without it.  The special case of PyPI isn't special >> > enough to break those rules. >> >> Although practicality beats purity. Can you list those "certain file >> recognition problems"? I.e. Explicit is better than implicit. > > I have a vague recollection that it was Fredrik Lundh's website that sparked > the original realization of the need for preferring shorter URLs, but I > wouldn't swear to it. I'd have to dig through years of revision history to > find the original change, assuming I documented it well enough. The choice > of short paths over long was also intended to favor nearby files over > further ones, and local paths over URLs. > > (All that being said, it's still fundamentally a heuristic, and not a very > good one at that. But that doesn't automatically make any other heuristic > *better*; this is one area where status quo bias is a good thing.) > > >> That's why it should use the site where all filenames are Python >> downloads if filenames are the same. > > And how would that work with all the PyPI clones, private indexes, etc.? > > >> > No.  You'd need to remove the current "home_page" setting, or point it >> > elsewhere. >> >> That's very strange. Then what download_url is for? > > The home page and download URLs are simply treated as pages which may > contain links to files, if they are not themselves links to files. That's > the only special status they have. > > >> >>  (I understand that people do not want to touch setuptools code >> >> anymore) >> > >> > That's not really the issue; the issue here is that package precedence >> > is >> > based on a stable comparison scheme, where it doesn't make sense to give >> > one >> > location priority over another, as it will simply lead to someone else >> > complaining about the changed behavior, because they were relying on a >> > different URL having precedence under the current scheme. >> >> These rules need to be described first. What if somebody already broke >> the proper order and now everybody suffers? If autodiscovery rules >> were well described - it was possible to analyse them and propose more >> intuitive approach. Then if "someone else" will attempt to complain - >> you could send them to the PEP or another "how and why" document. > > http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall > > http://peak.telecommunity.com/DevCenter/EasyInstall#package-index-api > > >> I thought it will raise the weight of those links if there could be a >> rel="download" attribute. > > There is no "weighting" of links - what is weighted are distributions, and > distribution objects only have their raw URL available as a basis for > sorting once the version and archive type (the two higher-precedence > attributes) are considered. The place where a URL was retrieved from is not > tracked, and thus can't be used for sorting without a good chunk of > refactoring... which refactoring would likely break tools that build on top > of setuptools' PackageIndex class. > > In short, what you're asking for is a pretty major feature that would be > difficult to implement in a way that would be guaranteed not to break other > things. > > ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [ANN] setuptools post install script for bdist_wininst
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Il 11/09/2010 20:10, P.J. Eby ha scritto: > At 04:42 PM 9/11/2010 +0200, Manlio Perillo wrote: >> This is a problem with the installer, since stdout and stderr are fully >> buffered. > > You probably need to pop up some sort of dialog and direct stdout/stderr > there. Alternately, you could run easy_install in a separate process as > a console script. > By the way, I'm not sure to understand how running easy_install in a separate process would solve the installer GUI freeze problem. It should probably be the installer to execute the post install script in non blocking mode. > [...] Thanks Manlio -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyMwgIACgkQscQJ24LbaUScqQCeK3I7cXfDfuhrdytBmbSuA2bI /QsAoJW1pBgchIjOPvi/S5+TMQHd+I+d =KuaC -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [ANN] setuptools post install script for bdist_wininst
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Il 11/09/2010 23:22, P.J. Eby ha scritto: > At 09:48 PM 9/11/2010 +0200, Manlio Perillo wrote: >> Is it possible to put required DLLs inside an egg? > > Yes, but they have to be adjacent to any Python extensions (.pyd's) that > use them. > Good to know, thanks. But how can I create such a egg? And if this is possible, what do you think is the reasons why packages like pyqt do not release such eggs? >> > 1. A primary reason people have for using bdist_wininst installers is >> > uninstall support, and this approach will leave dependencies behind. >> > >> >> As far as I know, all dependencies are correctly handled. >> Shortcuts are correctly removed by the uninstall. > > But the dependencies installed by easy_install will not get > corresponding uninstall entries in the Windows control panel, and so > will not be uninstalled when the user uninstalls the main application or > library. > Ah, right; I was thinking to created shortcuts. It should not be hard to track installed dependencies, as an example using the --record option of easy_install, and then registering all installed files (and directories) with `file_created` and `directory_created` functions. The only problem is that created directories are not reported in the record, but if files are always listed ordered, it should not be a problem. Some other related questions: * Do you think you can change ez_setup.py so that the code used to check if setuptools must be installed is available as a separate function? I need to check if setuptools must be installed so that I can pop a message box. For normal package dependencies (and assuming setuptools is available) I can just use working_set.__contains__(dist) * Is it safe to call `ez_setup.use_setuptools` setting `to_dir` parameter to `sysconfig.get_python_lib()`? By default the setuptools egg is copied to the same directory where the bdist_wininst file is, and this is not safe since the user may delete these files. * Is it possible to get the ID of the main window of the bdist_wininst application? I need it because MessageBox function requires the ID of the parent window. It is possible to use the Desktop as parent, but this may cause problems (message box displayed in the wrong position, with incorrect focus, or who know) * Is it possible to modify bdist_wininst (in a future release) so that, when executing the post install script, the name of the project being installed is passed in argv[2]? This change does not break existing code. Thanks Manlio -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyMyUsACgkQscQJ24LbaUTAKACfQJDRgrPqkTo+IhTXTZ/zQyop 7Z0AniXpSCvyGpADk87Ybmu1RT0noBO1 =kSwm -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] bdist_wininst and dependencies with setuptools
> By the way, is it possible, from the postinstall script, to know the > Python instance used? Sure: the python interpreter running the postinstall script *is* the one being used. Look at sys.executable. > There is one thing I'm not sure to understand about the bdist_wininst > and bdist_msi commands: what is the reason why they don't simply execute > `python setup.py install` in the background? Primarily because this would users require to have Visual Studio installed, plus potentially tons of header files and other build dependencies (e.g. setuptools). Other reasons are: - with a plain "setup.py install", uninstallation is not supported, but that is a feature of both bdist_wininst and bdist_msi - you need to unpack the files first to run "setup.py install": where would you unpack them to? - it's more efficient: installation runs faster. Regards, Martin ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] please fix easy_install shorter URL preference (Was: easy_install wrong download site preference)
At 12:08 PM 9/12/2010 +0300, anatoly techtonik wrote:
https://sourceforge.net/apps/trac/halfcooked/ticket/29
Gerald is the second package from the last 4 months that I couldn't
install due to this very obscure behavior, and I suspect there are
more.
Gerald installs fine when I try it, and from the same URL (which
isn't the shortest in any case). I suspect you are not using the
current version of easy_install, as the problem shown is due to
Sourceforge's use of '/download' URLs.
That issue was fixed in October 2009, so you are likely using
something <0.6c11.
(Or perhaps you are using distribute, which has still not applied
this bug fix almost a year later.)
Phillip, could you, please, still run through your archives to success
in explaining why this logic of choosing a shorter download URL is
necessary? I'd really to see what packages benefit from it,
It was originally added for PIL and other effbot-released packages;
it's retained because to change it now would introduce random
breakages to other projects, in order to benefit one whose source
filenames are ambiguous anyway.
I believe Google protocol buffers popularity alone is enough to change
the behavior.
Right now, the naming of the two source distributions is ambiguous,
even for humans.
Note, however, that since the full source distribution *also*
includes the python code, all that's needed is to change its setup.py
slightly and move it to the root of the source tree.
(The changes needed would consist of adding
'package_dir={"":"python"}' to the setup() options, and changing
relative path uses in the setup.py so that they don't assume the
current directory is the 'python' subtree... i.e., mainly just
changing all '../src' occurrences to 'src'.)
In the alternative, calling the release that contains all languages
'protobuf-all-2.3.0.*' rather than 'protobuf' would also fix the issue.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [ANN] setuptools post install script for bdist_wininst
At 02:36 PM 9/12/2010 +0200, Manlio Perillo wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Il 11/09/2010 23:22, P.J. Eby ha scritto:
> At 09:48 PM 9/11/2010 +0200, Manlio Perillo wrote:
>> Is it possible to put required DLLs inside an egg?
>
> Yes, but they have to be adjacent to any Python extensions (.pyd's) that
> use them.
>
Good to know, thanks.
But how can I create such a egg?
In the same way that you'd create a distutils-based package that
installs shared libraries next to Python extensions. (i.e., not easily.)
And if this is possible, what do you think is the reasons why packages
like pyqt do not release such eggs?
Because major C extensions (e.g. wx, numpy, pywin32) tend to have
special build requirements for their libraries that aren't handled
well by the distutils, or the build process predated the distutils,
or the build process wasn't originally designed to be used with
Python, etc. etc.
It should not be hard to track installed dependencies, as an example
using the --record option of easy_install, and then registering all
installed files (and directories) with `file_created` and
`directory_created` functions.
The only problem is that created directories are not reported in the
record, but if files are always listed ordered, it should not be a problem.
That's not the only problem: if you install A, then B, and both
depend on C, then if you uninstall A you will break B. You would
have to also track shared use of dependencies.
This situation is a big mess, and I haven't needed it for myself or
any client as yet, so I've avoided the whole subject.
Some other related questions:
* Do you think you can change ez_setup.py so that the code used to
check if setuptools must be installed is available as a separate
function?
Grab the bit that looks like this:
try:
import pkg_resources
except ImportError:
# it's not installed
else:
try:
pkg_resources.require('setuptools>='+version)
except pkg_resources.VersionConflict, e:
# older version is installed
except pkg_resources.DistributionNotFound:
# it's not installed
else:
# it's installed
Season to taste. ;-)
For normal package dependencies (and assuming setuptools is
available) I can just use working_set.__contains__(dist)
No, you need to use require() or resolve() or some other API that:
1. Checks versions, and
2. Handles recursive dependencies
* Is it safe to call `ez_setup.use_setuptools` setting `to_dir`
parameter to `sysconfig.get_python_lib()`?
It's safe for setuptools, but whether it's safe in the context of a
bdist_wininst I'm not sure.
By default the setuptools egg is copied to the same directory where
the bdist_wininst file is, and this is not safe since the user may
delete these files.
You could also fix that by calling easy_install(['--always-copy',
'setuptools']), in the case where you've had to install setuptools.
* Is it possible to get the ID of the main window of the bdist_wininst
application?
I need it because MessageBox function requires the ID of the parent
window.
It is possible to use the Desktop as parent, but this may cause
problems (message box displayed in the wrong position, with incorrect
focus, or who know)
* Is it possible to modify bdist_wininst (in a future release) so that,
when executing the post install script, the name of the project being
installed is passed in argv[2]?
This change does not break existing code.
I don't know the answers to these questions.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
