Re: [Distutils] docs/ vs doc/
Hi, Old thread revival! I was going through my archive and noticed this unanswered message from late September which prompted me to do a bit of research. However, upload_docs never quite works out of the box for me anyway. First, it insists on an index.html file, which my Sphinx builds never seem to write, so I always have to add a symlink. >> Barry, do you mean that you have all files generated by Sphinx but not >> index.html ? That's weird, Sphinx should generate it like the others. > Except that I don't have an index.txt file. It kind of doesn't make sense. I > use buildout and have this in my buildout.cfg: > > [docs] > recipe = z3c.recipe.sphinxdoc > eggs = mailman [docs] > index-doc = docs/README > default.css = > layout.html = > extensions = sphinxconf > > So what I end up doing is symlinking index.html to README.html before I > upload. I don’t know the recipe, but the “index-doc” settings clearly looks like something that controls what the master page is called. I guess it’s because you want your README file to serve double duty as Sphinx front page. Now for the good news: I found you a way to make Sphinx use that file as index.html: http://osdir.com/ml/sphinx-dev/2010-10/msg00082.html One may argue that upload_docs should just bundle *.html files, without imposing the existence of index.html; I think the check is useful. >>> Second, it doesn't seem to play nicely with the location that my buildout-based builds puts the docs, so I always also have to use --upload-dir. Yes, it would be nice if both of these problems were fixed too. Yes, I should report them as bugs (or just fix 'em ;). >> How would you fix the second issue ? e.g. if the docs is not in docs/ >> or doc/ > That's a good question! Maybe it's a buildout or recipe issue? You can make the upload-dir setting persistent by putting it in your setup.cfg. Or if the recipe is used by a large part of the community, open a feature request to have this directory automatically found if neither docs nor doc exist. What is the directory path, btw? Regards ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] zc.buildout/zc, recipe.egg: passing extra options for build_ext
On Fri, Feb 18, 2011 at 11:18 AM, Maurits van Rees wrote: > Hi, > > I see that you can add specify some custom build settings for packages with > C-extensions: > http://pypi.python.org/pypi/zc.recipe.egg#creating-eggs-with-extensions-needing-custom-build-settings > > The options you can pass are limited though. In my case for the InformixDB > package I want to pass '--esql-threadlib = posix' to the build process. > Currently this is not possible. > > Is there a reason why only the current hardcoded options (like include-dirs) > are available for customizing? It looks like it should be possible to > change the zc.recipe.egg code to look for other options and pass them to the > build_ext call. The only thing that might be tricky is to make a difference > between options in the buildout part that should be passed to build_ext and > options that have a totally different meaning and should be ignored by > build_ext. > > Would this be a good addition to zc.recipe egg? Probably. > I could create a branch and > try it. That should be a branch of zc.buildout, right? Yes. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] including optional python packages for testing
On 18/02/2011 15:56, Benji York wrote: You can run bin/buildout install foo, where "foo" is a part to install. It is not required that "foo" be listed in the [buildout] "parts =" line. Therefore you can have buildout actions that aren't run by default but that you can trigger as needed. Ah, now that sounds like what I'm after! thanks, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] zc.buildout/zc, recipe.egg: passing extra options for build_ext
Hi, I see that you can add specify some custom build settings for packages with C-extensions: http://pypi.python.org/pypi/zc.recipe.egg#creating-eggs-with-extensions-needing-custom-build-settings The options you can pass are limited though. In my case for the InformixDB package I want to pass '--esql-threadlib = posix' to the build process. Currently this is not possible. Is there a reason why only the current hardcoded options (like include-dirs) are available for customizing? It looks like it should be possible to change the zc.recipe.egg code to look for other options and pass them to the build_ext call. The only thing that might be tricky is to make a difference between options in the buildout part that should be passed to build_ext and options that have a totally different meaning and should be ignored by build_ext. Would this be a good addition to zc.recipe egg? I could create a branch and try it. That should be a branch of zc.buildout, right? Or are there other ways that currently already work? My workaround for now will likely be to make a copy of the original source dist tarball of InformixDB and add a setup.cfg in there with this contents: [build_ext] --esql-threadlib = posix From a local test this seems to be working. BTW, I did not know that you could specify these options in setup.cfg; I found that out today by looking through the zc.buildout code. :-) Cheers, -- Maurits van Rees Web App Programmer at Zest Software: http://zestsoftware.nl Personal website: http://maurits.vanrees.org/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] including optional python packages for testing
Op 18-02-11 16:37, Chris Withers schreef:
On 18/02/2011 15:28, Maurits van Rees wrote:
So, I'm looking for some way of specifying "optional eggs" when running
the buildout, and not in buildout.cfg so they don't accidentally get
checked in.
How can I achieve this?
Next to the install_requires in setup.py you add an extras_requires;
here is an example from the plone.app.discussion package:
extras_require = {
'test': [
'plone.app.testing',
'interlude',
]
},
If you want to install these extra dependencies in e.g. a buildout
config, you specify plone.app.discussion[test] as an egg.
Hmm, this isn't quite what I'm talking about.
If I did it this way, I'd need to define an extras_require section for
each type of database, so about 4-6 sections.
These also aren't really dependencies of my package, since they're never
imported by my package code, so it feels wrong to declare them as
dependencies...
Ah, I misunderstood you.
Perhaps you could add an extra.cfg that extends buildout.cfg and adds
the optional eggs. Then you need to remember that you have to run
'bin/buildout -c extra.cfg' when you want to test with those extra eggs.
Or maybe let buildout.cfg extend a local.cfg that adds the extra eggs.
Do not add local.cfg in subversion (or whatever VCS you use) but instead
let it ignore that file. Add an instruction in the readme that people
should manually add such a local.cfg and that it can remain empty.
I sometimes have only an almost empty buildout.cfg.in file in a project
that extends a base.cfg file, with instructions to copy it to
buildout.cfg and adapt to your local needs/wishes.
Cheers,
--
Maurits van Rees
Web App Programmer at Zest Software: http://zestsoftware.nl
Personal website: http://maurits.vanrees.org/
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] including optional python packages for testing
On Fri, Feb 18, 2011 at 4:07 AM, Chris Withers wrote: > Hi All, > > I'm using buildout to develop a package that uses sqlalchemy and > sqlalchemy-migrate. > > As a result, I'd like to test with various real database drivers; mysql, > postgres, etc. > > However, I don't want to force anyone who wants to check out and run the > tests to have all supported database drivers available and compiled. > > So, I'm looking for some way of specifying "optional eggs" when running the > buildout, and not in buildout.cfg so they don't accidentally get checked in. If I'm understanding you correctly, buildout's "install" command might help you. You can run bin/buildout install foo, where "foo" is a part to install. It is not required that "foo" be listed in the [buildout] "parts =" line. Therefore you can have buildout actions that aren't run by default but that you can trigger as needed. -- Benji York ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] including optional python packages for testing
On 18/02/2011 15:28, Maurits van Rees wrote:
So, I'm looking for some way of specifying "optional eggs" when running
the buildout, and not in buildout.cfg so they don't accidentally get
checked in.
How can I achieve this?
Next to the install_requires in setup.py you add an extras_requires;
here is an example from the plone.app.discussion package:
extras_require = {
'test': [
'plone.app.testing',
'interlude',
]
},
If you want to install these extra dependencies in e.g. a buildout
config, you specify plone.app.discussion[test] as an egg.
Hmm, this isn't quite what I'm talking about.
If I did it this way, I'd need to define an extras_require section for
each type of database, so about 4-6 sections.
These also aren't really dependencies of my package, since they're never
imported by my package code, so it feels wrong to declare them as
dependencies...
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] including optional python packages for testing
Op 18-02-11 10:07, Chris Withers schreef:
Hi All,
I'm using buildout to develop a package that uses sqlalchemy and
sqlalchemy-migrate.
As a result, I'd like to test with various real database drivers; mysql,
postgres, etc.
However, I don't want to force anyone who wants to check out and run the
tests to have all supported database drivers available and compiled.
So, I'm looking for some way of specifying "optional eggs" when running
the buildout, and not in buildout.cfg so they don't accidentally get
checked in.
How can I achieve this?
Next to the install_requires in setup.py you add an extras_requires;
here is an example from the plone.app.discussion package:
extras_require = {
'test': [
'plone.app.testing',
'interlude',
]
},
If you want to install these extra dependencies in e.g. a buildout
config, you specify plone.app.discussion[test] as an egg.
--
Maurits van Rees
Web App Programmer at Zest Software: http://zestsoftware.nl
Personal website: http://maurits.vanrees.org/
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On Fri, Feb 18, 2011 at 3:04 PM, P.J. Eby wrote: > At 12:41 PM 2/18/2011 +, Daniele Varrazzo wrote: >> >> I've tried to be good and replace the download_url, which used to >> refer to a directory containing all the packages, to a direct link to >> the package to download (see >> http://pypi.python.org/pypi/psycopg2/2.3.2). It wasn't enough as >> probably previous versions listed in /simple/ still refer to the >> directory (see http://pypi.python.org/simple/psycopg2/). >> >> Is there a way to clean the list in /simple/, apart from deleting all >> the previous releases from PyPI, which would be a net loss for PyPI as >> a repository? > > Yes. Just edit those releases. Log in to PyPI, go to your package page, > then click on "Releases". There will be an "edit" link for each release. It didn't work. The url for the beta is probably taken from the home page. Searching for psycopg2 Reading http://pypi.python.org/simple/psycopg2/ Reading http://initd.org/psycopg/ Reading http://initd.org/tracker/psycopg Reading http://initd.org/projects/psycopg2 Best match: psycopg2 2.4-beta2 On the plus side, this means we will have more tests than expected. -- Daniele ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
At 12:41 PM 2/18/2011 +, Daniele Varrazzo wrote: I've tried to be good and replace the download_url, which used to refer to a directory containing all the packages, to a direct link to the package to download (see http://pypi.python.org/pypi/psycopg2/2.3.2). It wasn't enough as probably previous versions listed in /simple/ still refer to the directory (see http://pypi.python.org/simple/psycopg2/). Is there a way to clean the list in /simple/, apart from deleting all the previous releases from PyPI, which would be a net loss for PyPI as a repository? Yes. Just edit those releases. Log in to PyPI, go to your package page, then click on "Releases". There will be an "edit" link for each release. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Tools should (at least optionally) prefer final releases. (was Re: easy_install installing beta version of psycopg2)
On Fri, Feb 18, 2011 at 5:29 AM, Tarek Ziadé wrote: > On Fri, Feb 18, 2011 at 10:00 AM, Chris Withers > wrote: >> On 17/02/2011 00:01, Daniele Varrazzo wrote: >>> >>> I don't think it is fair to ask to stop listing the beta from the >>> homepage and the download page of the 2-pages website of the module: >> >> Indeed, but I think it's totally fair to expect users to pin to the version >> they require using whatever package management policy they choose. >> >> Of course, sure, it would be nice if easy_install had a sane default of >> installing only final releases, but what're the chances of PJE implementing >> that? ;-) >> >> What's the "distribute" take on this? > > If someone wants to contribute a "--prefer-final" option to Distribute > --extracted from zc.buildout-- I am fine adding it, and even > activating this behavior by default, so people opt in to get the non > final releases. Thank you for noting the buildout feature, which has been around since 2007. :) I really think tools should support this feature. I also think the default should be to prefer final. I think this is much better than maintaining separate indexes. > > I think it would be a good way to let the user control the behavior of > the installer better, and that's what zc.buildout wants to do next > AFAIK. Yes. buildout2 will prefer final by default. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On Fri, Feb 18, 2011 at 1:03 PM, Wichert Akkerman wrote: > On 2/18/11 12:30 , Tarek Ziadé wrote: >> >> As a reminder, the current situation in packaging tools is as follows : >> >> - Setuptools development is stalled -- see the attached diagram >> - Distribute development is now also stalled. >> - People that used to work on Distribute are now working on Distutils2 > > My recent experience has been that setuptools is actively maintained, and > PJE is quickly fixing errors where distribute is ignoring bugreports. By "development" I mean adding new features to the project, making it evolve. Now, the rare commits that are done in the two projects are just bug fixes. And stating that "distribute is a year behind setuptools" is a false statement, Maybe this is a more accurate statement: 1 - some rare bugs fixes were not backported from setuptools to Distribute, because missed 2 - the few recents bugfixes were not backported While we can fix 2 -- we cannot fix 1 without some expertise from PJE, because some changes done in giant commits hard to decrypt. > My > only gripe with setuptools at the moment is that long time between releases; > often there are fixes ready in svn waiting for many months before a release > is made. > > Wichert. > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On 2/18/11 12:30 , Tarek Ziadé wrote: As a reminder, the current situation in packaging tools is as follows : - Setuptools development is stalled -- see the attached diagram - Distribute development is now also stalled. - People that used to work on Distribute are now working on Distutils2 My recent experience has been that setuptools is actively maintained, and PJE is quickly fixing errors where distribute is ignoring bugreports. My only gripe with setuptools at the moment is that long time between releases; often there are fixes ready in svn waiting for many months before a release is made. Wichert. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On Thu, Feb 17, 2011 at 7:57 PM, P.J. Eby wrote: ... > > Unfortunately, even if I fixed that today, it wouldn't have ANY effect on > 99% of the field installations of any Python package management tools: there > are still people using 4 or 5 year old versions of easy_install, That's not a good argument imo. If we stop doing any enhancements to installers because some people use old versions, we will never do anything. The people that complain about a behavior in a 4 years-old installer, that was changed/fixed since then, know they can update. > and a lot > of people use Distribute (via their OS install), which is a year behind the > setuptools trunk on various things. If you really believe this statement is true, let us know what are the "various things" so we can fix them for everyone's benefit. As a reminder, the current situation in packaging tools is as follows : - Setuptools development is stalled -- see the attached diagram - Distribute development is now also stalled. - People that used to work on Distribute are now working on Distutils2 I think there are a few things we should all do today to make things better: 1. make sure anything that gets fixed in Setuptools is backported in Distribute 2. stop adding new features in Distribute, or minimal things. I think the py3 support by itself justifies that Distribute should be preferred to Setuptools in OSes. 3. focusing on consolidating the first version of Distutils2 Could you help on 1. ? .. > > Pip is the main package manager that uses its own link-finding algorithm, > but it only supports source installation AFAIK. Distutils2 uses a > link-finding algorithm that was lifted pretty much verbatim from > easy_install, though I think there may have been some additions to it since > I last looked at it. We plan to prefer the final versions, and let the user decides if she wants to use betas, devs etc. Contributions/Feedback/Advices are welcome. Cheers Tarek -- Tarek Ziadé | http://ziade.org <>___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On Fri, Feb 18, 2011 at 10:00 AM, Chris Withers wrote: > On 17/02/2011 00:01, Daniele Varrazzo wrote: >> >> I don't think it is fair to ask to stop listing the beta from the >> homepage and the download page of the 2-pages website of the module: > > Indeed, but I think it's totally fair to expect users to pin to the version > they require using whatever package management policy they choose. > > Of course, sure, it would be nice if easy_install had a sane default of > installing only final releases, but what're the chances of PJE implementing > that? ;-) > > What's the "distribute" take on this? If someone wants to contribute a "--prefer-final" option to Distribute --extracted from zc.buildout-- I am fine adding it, and even activating this behavior by default, so people opt in to get the non final releases. I think it would be a good way to let the user control the behavior of the installer better, and that's what zc.buildout wants to do next AFAIK. I cannot spend time on Distribute anymore because I am spending all my free time on Distutils2 -- but contributions are welcome Cheers Tarek > > cheers, > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > - http://www.simplistix.co.uk > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] [buildout] including optional python packages for testing
Hi All, I'm using buildout to develop a package that uses sqlalchemy and sqlalchemy-migrate. As a result, I'd like to test with various real database drivers; mysql, postgres, etc. However, I don't want to force anyone who wants to check out and run the tests to have all supported database drivers available and compiled. So, I'm looking for some way of specifying "optional eggs" when running the buildout, and not in buildout.cfg so they don't accidentally get checked in. How can I achieve this? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install installing beta version of psycopg2
On 17/02/2011 00:01, Daniele Varrazzo wrote: I don't think it is fair to ask to stop listing the beta from the homepage and the download page of the 2-pages website of the module: Indeed, but I think it's totally fair to expect users to pin to the version they require using whatever package management policy they choose. Of course, sure, it would be nice if easy_install had a sane default of installing only final releases, but what're the chances of PJE implementing that? ;-) What's the "distribute" take on this? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
