Re: [Python-Dev] Identifier API
I like this better too because of the all-caps macro name. Something about seeing Py_identifier look like a function call and having it add the magical PyId_update local bugs me. It just looks wrong, whereas the all-caps is more of a cultural clue that something else is going on. If people think the macro should be all upper-case, I can go through and replace them (but only once). Let me know what the exact spelling should be. Originally, I meant to make the variable name equal the string (e.g. then having a variable named __init__ point to the __init__ string). However, I quickly gave up on that idea, since the strings conflict too often with other identifiers in C. In particular, you couldn't use that approach for calling the fileno, read or write methods. So I think it needs a prefix. If you don't like PyId_, let me know what the prefix should be instead. If there is no fixed prefix (i.e. if you have to specify variable name and string value separately), and if there is no error checking, there is IMO too little gain to make this usable. I'm particularly worried about the error checking: the tricky part in C is to keep track of all the code paths. This API hides this by putting the initialization into the callee (PyObject_GetAttrId and friends), hence the (unlikely) failure to initialize the string is reported in the same code path as the (very plausible) error that the actual attribute access failed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
An alternative I am fond of is to to avoid introducing a new type, and simply initialize a PyObject * and register its address. -1 on that, because of the lack of error checking. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
On Thu, 13 Oct 2011 14:00:38 +0200 Martin v. Löwis mar...@v.loewis.de wrote: I like this better too because of the all-caps macro name. Something about seeing Py_identifier look like a function call and having it add the magical PyId_update local bugs me. It just looks wrong, whereas the all-caps is more of a cultural clue that something else is going on. If people think the macro should be all upper-case, I can go through and replace them (but only once). Let me know what the exact spelling should be. Py_CONST_STRING or Py_IDENTIFIER would be fine with me. Given that everything else uses Id in their name, Py_IDENTIFIER is probably better? Originally, I meant to make the variable name equal the string (e.g. then having a variable named __init__ point to the __init__ string). However, I quickly gave up on that idea, since the strings conflict too often with other identifiers in C. In particular, you couldn't use that approach for calling the fileno, read or write methods. So I think it needs a prefix. If you don't like PyId_, let me know what the prefix should be instead. I agree with that. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
On Oct 13, 2011, at 03:23 PM, Antoine Pitrou wrote: Py_CONST_STRING or Py_IDENTIFIER would be fine with me. Given that everything else uses Id in their name, Py_IDENTIFIER is probably better? I agree that either is fine, with a slight preference for Py_IDENTIFIER for the same reasons. Originally, I meant to make the variable name equal the string (e.g. then having a variable named __init__ point to the __init__ string). However, I quickly gave up on that idea, since the strings conflict too often with other identifiers in C. In particular, you couldn't use that approach for calling the fileno, read or write methods. So I think it needs a prefix. If you don't like PyId_, let me know what the prefix should be instead. I agree with that. I'm fine with that too, as long as it's all well-documented in the C API guide. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Le 09/10/2011 13:54, Paul Moore a écrit : On 9 October 2011 08:15, Éric Araujo mer...@netwok.org wrote: Are there that many distributions with extension modules? sdists should work well even on Windows for pure Python projects. [...] Looking at my installations, I see: - database drivers (cx_Oracle, in my case) - lxml - pywin32 - pyQT - pyzmq (that's just for playing a bit with IPython, so doesn't really count...) - I've also used in the past PIL mod_python (mod_wsgi more recently) and wxPython, These are good examples. Even if the number is not high, they are widely used, so support for binary distributions really seems needed. (When pip switches to distutils2 as underlying lib, they’ll get bdist support for free! IOW, proper bdist support would be another argument to make the world switch to the new standards.) The pysetup features for uninstalling packages aren't going to work with bdist_wininst/bdist_msi (that's an assumption, I haven't tried them but I can't see how they would, and it'd certainly be a lot of marginally-useful effort to do even if it were possible). The virtual environment stuff also wouldn't work that well with the installers, because they wouldn't have any way of finding which environments existed to ask where to install to. The same problem exists with virtualenv. (Again this is speculation backed by a small amount of playing with virtualenv, so I may be wrong here). wininst and msi bdists can continue to be used as previously, for people who want clicky installation to their system Python. With built-in package management and virtual environments in 3.3+, we can just recommend that people publish bdist_simple instead of wininst or eggs. It may be that the bdist_dumb format would be OK. I haven't checked it out (to be honest, I don't think it's ever been used much). It may or may not be. bdist_dumb just makes a tarball or zipfile of Python modules and built extension modules and is supposed to be unpacked under sys.prefix or sys.exec_prefix. However, that won’t play nice with install options (--home, --user, --install-lib or redefined --prefix) or sysconfig categories (i.e. I may want config files under /usr/local/etc, scripts in /usr/local/bin, etc.). I think we could revamp bdist_dumb so that it’s really “sdist with compiled files” and then we let pysetup install things to the right places. Yes! We need feedback to provide a much better tool than distutils, before the API is locked by backward compatibility rules. Always the chicken and egg problem :-) I’d rather say it’s “code in the stdlib has one foot in the grave” + “stdlib code without active maintainer is effectively frozen” (hi asyncore changes in 2.6!). I actually wanted to talk about that, so let me take the opportunity. What if we released packaging in Python 3.3 (and distutils2 1.0 on PyPI) as a not-quite-final release? [...] My immediate thought is that it would actually put people off using packaging for 3.3, they'd wait until it is stable. OK. Too bad. I’ll probably post that question again in its own message to get more feedback. What is the status of distutils2? Is that (still?) intended to be effectively a backport of packaging to earlier Python versions? Yes. It works with 2.4-3.3. I maintain it synchronized with packaging in 3.3. There are a small number of test failures which needs fixing before I release distutils2 1.0a4 on PyPI. I'd suggest getting a distutils2 release available, and promoted, as the early adopter version of packaging. We may do that, but I fear we’re going to lack time for that. As part of the stdlib, the packaging module API will be frozen in June, for the first 3.3 beta. We still have a lot to do: defining __all__ in all public modules, changing some important internals (Tarek wants to kill the subcommands system for example), fixing a number of bugs which may imply incompatible API changes, etc. The pace of development has slowed much these last months, so I’m not sure we’ll reach 1.0 status months before June. Maybe even with an option to install it as packaging so that people can use it in 3.2 and earlier and expect to need no changes when 3.3 is released. Not gonna happen! d2 changed name on purpose when entering the stdlib, so that in the future code can choose to use packaging (in the stdlib, for example 1.0) or distutils2 (external backport, possibly of 2.0). Code wanting to use “packaging if available otherwise distutils2” will use the same import try/except as what’s done with unittest/unittest2, json/simplejson and similar. A python-announce article Python 3.3 new packaging features - early adopter release publicising it, would be what I'm thinking of... Here’s a plan: 1) make the docs usable (I’m on it) 2) fix the three test failures we currently have in d2 3) test on Windows and Mac 4) release 1.0a4 (I’ll do it) 5) announce and request feedback 6) work frantically before Python 3.3b1 to improve stuff and
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Hi Philip, [...] In any case, it definitely wasn't the case that eggs or setuptools were rejected for 2.5; they were withdrawn for reasons that didn't have anything to do with the format itself. Thanks for clarifying. I nonetheless remember strong opposition to pulling the code unmodified, from MvL IIRC. (And, ironically enough, AFAIK the new packaging module uses code that's actually based on the bits of setuptools Fredrik was worried about supporting... setuptools presence in packaging is - packaging.database can read egg/PKG-INFO (zipped and unzipped) and egg-info files - packaging.install can detect that a project uses a setup.py with setuptools, run that setup.py, and convert egg-info to dist-info but at least there now are more people providing that support.) Truth be told, I’m not sure it is so. The student who worked on packaging.database has not remained a member of our group; his mentor is also less active. But that’s not the hardest code in packaging. Regarding installation, we do have people with distribute knowledge and experience, so that’s good. What we can do however is to see what bdist_egg does and define a new bdist command inspired by it, but without zipping, pkg_resource calls, etc. Why? If you just want a dumb bdist format, there's already bdist_dumb. We’re not sure bdist_dumb is what we’re after—see my other messages. Conversely, if you want a smarter format, why reinvent wheels? Recent packaging PEPs and distutils2 are all about reinventing wheels! Or rather standardizing best practices for wheels. Some ideas are taken near-identical from setuptools, other see great changes. In this case, we have to define our requirements, and if bdist_egg can work (as a distribution format, not an installation format!), then we may just take it. If it does not, we’ll have to make a new wheel. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Le 09/10/2011 22:14, Paul Moore a écrit : On 9 October 2011 20:47, Tarek Ziadé ziade.ta...@gmail.com wrote: PEP 376 provide the installation format for the 'future' -- http://www.python.org/dev/peps/pep-0376/ [...] Now for a binary archive, that would get installed ala PEP 376, why not ? I'd just be curious to have someone list the advantage of having a project released that way besides the importable as-is feature. I think I don’t understand you here. IMO, bdists are just intermediary formats that are supposed to be consumed by installation tools. Users are not expected to import from bdists. My expectation would be that the user would type pysetup install some_binary_format_file.zip and have that file unpacked and all the bits put in the appropriate place. Basically just like installing from a source archive - pysetup install project-1.0.tar.gz - but skipping the compile steps because the compiler output files are present. Yep. That may need some extra intelligence in pysetup if it doesn't have this feature already [...] just unzip the bits into the right place, or something similar. Yes. The bdist can be just like an sdist, but it contains compiled files instead of C source files (maybe setuptools bdist_egg is just that), then pysetup uses the setup.cfg file to find files and install them at the right places. Alternatively, the bdist format could put build files into a few top-level directories, using sysconfig names: config, appdata, doc, purelib, platlib, etc. pysetup would then move files to the right target directory. As regards the format, bdist_dumb is about the right level - but having just checked it has some problems (which if I recall, have been known for some time, and are why bdist_dumb doesn't get used). Or maybe because it’s just useless: Windows users want to click many times, Mac OS users want to drag-and-drop things, free OS users are fine with sdists. Specifically, bdist_dumb puts the location of site-packages ON THE BUILD SYSTEM into the archive, making it useless for direct unzipping on a target system which has Python installed somewhere else. pysetup would not just unzip it though, so maybe this limitation is not really one. a new name might be needed if backward compatibility of the old broken format matters... The point of forking distutils under a new name is that we can break compat. I don't know the code at all, and I have little time Your time is best used with giving user expectations and feedback. I hope to get a Windows VM soon-ish, so I should not even pester people for testing my patches :) PS The problem for me is that if pysetup only handles source builds, it's STILL annoyingly incomplete for my requirements (and possibly many Windows users') Agreed. packaging does not want to exclude Windows users. [Nick] bdist_zip, bdist_archive, bdist_simple would all work (bdist_binary is redundant, given what the 'b' stands for). Isn’t calling zipfiles “archives” an abuse? I like bdist_simple. On a related topic, I’m not sure the bdist comand is useful. Its only role is to translate “bdist --formats zip,targz,wininst” to calls to the other bdist_* commands. The 'bdist_dumb' name has always irritated me, since the connotations more strongly favour 'stupid' than they do 'simple' I’ve also recently learned that people with mental illness can be hurt by derogatory uses of “dumb”, so if we change the name to reflect the change in behavior, it’d have the nice side-effect of being nicer. [Vinay] A simple change to packaging will allow an archive containing a setup.cfg-based directory to be installed in the same way as a source directory. Isn’t that already supported, as long as the tarball or zipfile contains source files? In any case, it was intended to be, and there’s still support code around. the current installer stub (wininst-x.y.exe) does not know anything about virtual environments. If we care about virtual environment support (and I think we should), wininst.exe could be enhanced to provide a Browse... button to allow a user to select a virtual environment to install into, Personally, I’ll focus on sdist and bdist_simple support. When pysetup is run from a virtualenv, projects will be installed into the venv. You are free to work on patches for bdist_wininsts, but I’m not sure it will be needed, if we make the pysetup user experience smooth enough. That said, even if Paul was convinced to forsake clicky installers, maybe some Windows users will absolutely refuse to use the command line. [Paul] To summarise, then: 1. By using setup.cfg technology, it would be easy enough to zip up a binary build in a way that pysetup could unpack and install. Correct. I’m still pondering whether I find the idea of registering built files in setup.cfg as elegant or hacky :) We also have the other ideas I wrote to choose from. 1a. A packaging command to build such an archive would be worth providing. Definitely.
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Hi Nick, Le 11/10/2011 03:29, Nick Coghlan a écrit : On Mon, Oct 10, 2011 at 2:29 PM, Paul Moore p.f.mo...@gmail.com wrote: Ideally bdist_wininst and bdist_msi would also integrate with pysetup and with virtual environments, but I imagine that could be pretty hard to make work cleanly, as Windows doesn't really support multiple installations of a software package... That's OK, the package managers get bypassed by pysetup on POSIX systems as well - that's kind of the point of language level virtual environments I’m not sure I follow you. wininst and msi installers are supposed to work with the Windows programs manager, it is not bypassed at all IIUC. That’s the difficulty: How to make the Add/Remove program aware of many Pythons and venvs? There are hard to build packages on POSIX (e.g. PIL) that would also benefit from a good, cross-platform approach to binary installation. We haven’t talked about cross-platform binary installers. The current gist of the discussion seems to point to a world where people continue to release an sdist for capable OSes and also publish one bdist_simple with .pyd for one Windows version and one bdist_simple with .so for one Mac OS X version (but possible many arches). (And given that it’s possible to have one setup.cfg and one setup.py coexisting, maybe eggs will be offered for a while too.) Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Le 13/10/2011 18:25, Éric Araujo a écrit : 2c. There's a question over a GUI install followed by a pysetup uninstall, which wouldn't remove the add/remove entry... I think we could require that a project installed with a clicky wininst_bdist has to be removed via the Add/Remove GUI. (There is support for that in PEP 376: the INSTALLER file.) In case this wasn’t very clear: PEP 376 defines that the name of the installer be recorded in a file in the dist-info dir, and when another installer tries to remove the distribution it will be blocked. So if INSTALLER contains “wininst”, then “pysetup remove” won’t work, so we won’t get imperfect uninstalls. Cheers ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 13 October 2011 17:35, Éric Araujo mer...@netwok.org wrote: Le 11/10/2011 03:29, Nick Coghlan a écrit : On Mon, Oct 10, 2011 at 2:29 PM, Paul Moore p.f.mo...@gmail.com wrote: Ideally bdist_wininst and bdist_msi would also integrate with pysetup and with virtual environments, but I imagine that could be pretty hard to make work cleanly, as Windows doesn't really support multiple installations of a software package... That's OK, the package managers get bypassed by pysetup on POSIX systems as well - that's kind of the point of language level virtual environments I’m not sure I follow you. wininst and msi installers are supposed to work with the Windows programs manager, it is not bypassed at all IIUC. That’s the difficulty: How to make the Add/Remove program aware of many Pythons and venvs? There are 2 separate things here: 1. Native installers, bdist_msi and bdist_wininst. These (currently) integrate with the Add/Remove feature and have a standard platform look feel. The downside is that they don't integrate to the same level with certain Python features like venvs and pysetup, or with non-system Python installations. 2. pysetup, which is Python's native package manager and as such can be assumed to integrate well with any other Python features. The Unix equivalent of (1) would be RPM installers for Python packages. It's not so much about the GUI-ness as the native integration (which on Windows does mean GUI, but less so on other platforms). It's the classic trade-off between integration with the platform vs integration with the language environment. You can't usually have both unless the language is single-platform. There are hard to build packages on POSIX (e.g. PIL) that would also benefit from a good, cross-platform approach to binary installation. We haven’t talked about cross-platform binary installers. The current gist of the discussion seems to point to a world where people continue to release an sdist for capable OSes and also publish one bdist_simple with .pyd for one Windows version and one bdist_simple with .so for one Mac OS X version (but possible many arches). (And given that it’s possible to have one setup.cfg and one setup.py coexisting, maybe eggs will be offered for a while too.) Nick is talking about a cross-platform *approach* - not a single installer that runs on multiple platforms, but rather a common set of instructions (run pysetup run bdist_simple; pysetup upload) which will generate a binary package that can be installed in a common way (pysetup install) on all platforms. The actual bdist_simple file will be version, architecture and platform dependent, just like binary eggs and bdist_winints installers today, but it can be used by people without access to a compiler, or to development packages for supporting libraries, or anything other than Python. This is a good thing for most users (even the RPM for say lxml includes binary so files, and doesn't build from source on the target system), and nigh-on essential for most Windows users (where setting up a compiler is a lot more complex than yum install gcc). It may even be useful enough to persuade Windows users to move away from GUI installers (after all, that's what has happened with setuptools and binary eggs). Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
wininst and msi bdists can continue to be used as previously, for people who want clicky installation to their system Python. With built-in package management and virtual environments in 3.3+, we can just recommend that people publish bdist_simple instead of wininst or eggs. Pardon me for jumping in - but I fail to see why those missing features can't be provided by bdist_wininst and bdist_msi in a straight-forward manner. Giving people even more choice is bad, IMO, as it will confuse users. There should be one obvious way. In particular wrt. virtual environments: I see no need to actually *install* files multiple times. It's rather sufficient that the distributions to be installed are *available* in the virtual env after installation, and unavailable after being removed. Actually copying them into the virtual environment might not be necessary or useful. So I envision a setup where the MSI file puts the binaries into a place on disk where pysetup (or whatever tool) finds them, and links them whereever they need to go (using whatever linking mechanism might work). For MSI in particular, there could be some interaction with pysetup, e.g. to register all virtualenvs that have linked the installation, and warn the user that the file is still in use in certain locations. Likewise, automated download might pick an MSI file, and tell it not to place itself into the actual Python installation, but instead into a location where pysetup will find it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use identifier API for PyObject_GetAttrString.
On Thu, 13 Oct 2011 20:05:06 +0200 Martin v. Löwis mar...@v.loewis.de wrote: - In Modules/_json.c, line 1126, _Py_identifier(strict) is declared but not used, and there are 5 other possible replacements. Antoine reverted this in 8ed6a627a834. I think I started doing them, then noticed that this is an initializer, so it's likely not called that often. That's what I thought too. There didn't seem to be much point in optimizing that code. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
Py_CONST_STRING or Py_IDENTIFIER would be fine with me. Given that everything else uses Id in their name, Py_IDENTIFIER is probably better? I agree that either is fine, with a slight preference for Py_IDENTIFIER for the same reasons. Ok, so it's Py_IDENTIFIER. So I think it needs a prefix. If you don't like PyId_, let me know what the prefix should be instead. I agree with that. I'm fine with that too, as long as it's all well-documented in the C API guide. Hmm. People voted that this should be an internal API, so I'm not sure it should be documented at all outside of the header file, or if, in what document. Currently, this very point is documented in the header file. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 13 October 2011 17:25, Éric Araujo mer...@netwok.org wrote: 1. By using setup.cfg technology, it would be easy enough to zip up a binary build in a way that pysetup could unpack and install. Correct. I’m still pondering whether I find the idea of registering built files in setup.cfg as elegant or hacky :) We also have the other ideas I wrote to choose from. To be honest, I think I prefer the idea of taking the bdist_wininst code which creates a zipped distribution archive with special root directories like PLATLIB, and use it to create a bdist_simple (basically, by removing some of the code to prepend the EXE stub). Then teach pysetup install to install that file format.(likely by just plugging in a new function into install_methods). The benefit is that bdist_wininst installers can be consumed unaltered by this install method. [1] Vinay's suggestion of registering the built files in setup.cfg sounds attractive, as the code is already there, but it seems like it'd just move the complexity from the install code to the process of building the bdist_simple archive. 1a. A packaging command to build such an archive would be worth providing. Definitely. Maybe we could also decide on one of wininst or msi? I was thinking of a new bdist_simple archive format, which is platform-agnostic. bdist_wininst is a compatible superset restricted to the Windows environment, so promoting bdist_wininst over bdist_msi for people who prefer GUIs and platform integration would make sense. 2. A GUI installer would still be valuable for many people 2a. Having the GUI work by doing a pysetup install passing the installer exe (which would have a zipfile as noted in 1 above appended) could make sense to avoid duplicating work. Yes. Or having bdist_wininst usable by pysetup install directly (because it's a bdist_simple compatible format)... 2b. The GUI could do the extra needed to integrate with the OS, which pysetup wouldn't do Nice property. And already present with bdist_wininst. 2c. There's a question over a GUI install followed by a pysetup uninstall, which wouldn't remove the add/remove entry... I think we could require that a project installed with a clicky wininst_bdist has to be removed via the Add/Remove GUI. (There is support for that in PEP 376: the INSTALLER file.) Agreed - and if pysetup install works with bdist_wininst files, the user can choose whether to use a pysetup or a GUI install (and consequently which approach to management/uninstall they prefer). 3. Ideally, the GUI should co-operate with venvs, by offering some form of browse facility. The command line does this automatically. Will Windows users want a GUI to create venvs too? Quite possibly some will. Personally, I don't. And given that virtualenv has managed OK without a GUI interface, I'd say let's assume it's YAGNI for now. [1] Actually, based on the above, I think the pysetup install method that consumes bdist_wininst files as if they were just bdist_simple archives (i.e., assuming the bdist_simple format takes that layout) would be useful even if bdist_simple is never implemented. The bdist_simple version makes the same facilities available for non-Windows users, if they want it. I think I'll look at coding that option, and see where it takes me. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 13 October 2011 18:30, Martin v. Löwis mar...@v.loewis.de wrote: wininst and msi bdists can continue to be used as previously, for people who want clicky installation to their system Python. With built-in package management and virtual environments in 3.3+, we can just recommend that people publish bdist_simple instead of wininst or eggs. Pardon me for jumping in - but I fail to see why those missing features can't be provided by bdist_wininst and bdist_msi in a straight-forward manner. Giving people even more choice is bad, IMO, as it will confuse users. There should be one obvious way. I don't particularly disagree - although I would point out that the two formats bdist_wininst and bdist_msi already offer more than one obvious way... My contention is that there *are* two distinct use cases - platform integrated installation (on Windows that implies a GUI to most people), and Python's native installation process (pysetup). This isn't new, before packaging the python-native form was setuptools/eggs, for better or worse. Ideally, both forms should have full capabilities, making the decision a style/preference choice rather than a functionality choice. But this particular choice is always with us, and people are familiar with it. (Native vs cross-platform GUIs, cygwin vs mingw, etc, etc). So we need two obvious ways, one for each case. (It would be nice if one way could cover both cases, of course - having pysetup consume bdist_wininst files is my attempt to achieve that). I don't really understand the benefits of bdist_msi over bdist_wininst, and I certainly don't understand the MSI technology well enough to comment on what it's capable of, so I'm going to stick to bdist_wininst in the following. My apologies for anything I miss as a result. But it does strike me that the existence of both MSI and wininst is where the confusing duplication exists, rather than having GUI and command line alternatives. The GUI and platform integration aspects of the bdist_wininst format are all part of the executable bit. I haven't looked at that code at all, but I am certain it can be modified to provide whatever user experience is desired. The only real problem here is how many people have the knowledge and/or inclination to work on that code. When it comes to installing the actual package, I don't know how the bdist_wininst code does it - the data is there in zip format, and I suspect that the code simply unzips the data in the expected directories. But the zipped up data in bdist_wininst could be consumed by the packaging module, just by writing a new install method. This would reuse all of the various packaging support routines and infrastructure. The bdist_wininst executable code *could* be modified to invoke that packaging method - whether that's worthwhile isn't clear to me (I don't know how extensive the changes would be to get the benefit reusing the same implementation). As MSI format is a specialised format, I don't believe this option is open for bdist_msi. In particular wrt. virtual environments: I see no need to actually *install* files multiple times. It's rather sufficient that the distributions to be installed are *available* in the virtual env after installation, and unavailable after being removed. Actually copying them into the virtual environment might not be necessary or useful. So I envision a setup where the MSI file puts the binaries into a place on disk where pysetup (or whatever tool) finds them, and links them whereever they need to go (using whatever linking mechanism might work). For MSI in particular, there could be some interaction with pysetup, e.g. to register all virtualenvs that have linked the installation, and warn the user that the file is still in use in certain locations. Likewise, automated download might pick an MSI file, and tell it not to place itself into the actual Python installation, but instead into a location where pysetup will find it. I can't really comment on this. I agree in principle with what you're saying, but I know little about the MSI format so I can't say much more. It feels to me like you're suggesting that the MSI file encapsulate the file layout logic that already has to exist in pysetup, though, which sounds like duplication of effort. Can MSI call out to pysetup to actually install the files and save this duplication? Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 376 - contents of RECORD file
Looking at a RECORD file installed by pysetup (on 3.3 trunk, on Windows) all of the filenames seem to be absolute, even though the package is pure-Python and so everything is under site-packages. Looking at PEP 376, it looks like the paths should be relative to site-packages. Two questions: 1. Am I reading this right? Is it a bug in pysetup? 2. Does it matter? Are relative paths needed, or is it just nice to have? Oh, and a third question - where is the best place to ask these questions? Now that packaging is in core, is python-dev OK? Or should I be asking on the distutils SIG or the packaging developers list? Thanks, Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 13 October 2011 20:28, Tim Golden m...@timgolden.me.uk wrote: On 13/10/2011 19:36, Paul Moore wrote: I don't really understand the benefits of bdist_msi over bdist_wininst Just commenting on this particular issue: in essence, the .MSI format is the Microsoft standard, something which is especially important for corporate rollouts. We're not particularly bureaucratic, but I recently had to bundle a small number of common extensions as .msi packages so they could be deployed easily onto our baseline machines. I'm not saying that Python *must* have .msi support for this reason: if it didn't already, you could argue that it could be provided by corporates who needed this, or by 3rd party service providers, if only by providing light .msi wrappers round standard installers. Thanks for the clarification. I can see why this would be important. But maintaining 3 different interfaces to do essentially the same thing (collect some data from the user, then based on that data put the same set of files in the same places) seems a waste of effort, and a recipe for discrepancies in capabilities. Maybe the wininst and MSI installers should ultimately become simple UIs around a zipfile and an invocation of the packaging APIs? Not that I'm offering to do that work, I'm afraid... Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On Tuesday, October 11, 2011 01:59:45 AM Vinay Sajip wrote: I looked at the dialog resources for wininst-x.y.exe and noticed that there is a Find other ... button which is hidden, and its handler (in PC\bdist_wininst\install.c) is commented out. However, the code called by the handler - GetOtherPythonVersion - is still there. Does anyone here know why the button has been made unavailable? This feature has never been active. It has been commented out since before Distutils was imported into Python proper. -- Jeremy Kloth ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On Thursday, October 13, 2011 01:42:13 PM Paul Moore wrote: Maybe the wininst and MSI installers should ultimately become simple UIs around a zipfile and an invocation of the packaging APIs? Not that I'm offering to do that work, I'm afraid... The bdist_wininst/_msi installers cannot use any of the packaging.* code *runtime* as packaging (or distutils2) isn't necessarily installed on the target machine. I would think that having it as a prerequisite to actually running the installers is a bad thing. Including the required support files within the installers may be doable but could add too much complexity and possibly lead to stale code issues (for the support files). That said, I have been working on a drop-in replacement for the current bdist_wininst executable stub with the following features: - install to 32- or 64-bit Python installations from a single installer; currently one installer for each architechure is required - install to any Python from version 2.4 to the latest; currently one installer is needed for each major version - updated look and feel (Wizard97) with the new (as of Python 2.5!) logo; for some screen shots see: http://www.flickr.com/photos/67460826@N04/sets/72157627653603530/ - unicode metadata support (name, summary, description) - runs on Win95 through Win7 (that is, all support platforms for the supported Python versions for packaging) - per-user installs (as in, setup.py install --user); currently only system-wide or per-user based on permissions and how Python itself was installed Planned Features: - multi-version extension module support; one installer that can install the precompiled extensions to different Python versions - prefix installs (as in, setup.py install --prefix) for virtual environments or other non-standard locations. Current thinking is to *not* track these installations in the Add/Remove programs. -- Jeremy Kloth ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
Martin v. Löwis wrote: So I think it needs a prefix. If you don't like PyId_, let me know what the prefix should be instead. Instead of an explicit prefix, how about a macro, such as Py_ID(__string__)? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
Le jeudi 13 octobre 2011 03:34:00, Victor Stinner a écrit : We would need a new format for Py_BuildValue, e.g. 'a' for ASCII string. Later we can add new functions like _PyDict_GetASCII(). The main difference between my new const ASCII string idea and PyIdentifier is the lifetime of objects. Using const ASCII strings, the strings are destroyed quickly (to not waste memory), whereas PyIdentifiers are intern strings and so they are only destroyed at Python exit. I don't know if const ASCII strings solve a real issue. I implemented my idea. I will do some benchmarks to check if it's useful or not :-) Ok, I did some tests: it is slower with my PyConstASCIIObject. I don't understand why, but it means that the idea is not interesting because the code is not faster. It is also difficult to ensure that the string is constant (test the scope of the string). At least, I found a nice GCC function: __builtin_constant_p(str) can be used to ensure that the string is constant (e.g. abc vs char*). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On Thursday, October 13, 2011 04:02:27 PM Jeremy Kloth wrote: That said, I have been working on a drop-in replacement for the current bdist_wininst executable stub with the following features: - install to 32- or 64-bit Python installations from a single installer; currently one installer for each architechure is required - install to any Python from version 2.4 to the latest; currently one installer is needed for each major version - updated look and feel (Wizard97) with the new (as of Python 2.5!) logo; for some screen shots see: http://www.flickr.com/photos/67460826@N04/sets/72157627653603530/ - unicode metadata support (name, summary, description) - runs on Win95 through Win7 (that is, all support platforms for the supported Python versions for packaging) - per-user installs (as in, setup.py install --user); currently only system-wide or per-user based on permissions and how Python itself was installed I missed a few additional features: - UAC (Vista, Win7) is handled at the install phase depending on the selected Python target (system or user installed Python or user site-packages); currently just at the running of the installer - pre-/post- install and remove script support with the scripts no longer needing to be installed with the distribution - MSVCRT agnostic; built completely with the Windows API meaning only one stub EXE required; currently there is one stub per MSVCRT version -- Jeremy Kloth ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Identifier API
Am 14.10.2011 00:30, schrieb Victor Stinner: Le jeudi 13 octobre 2011 03:34:00, Victor Stinner a écrit : We would need a new format for Py_BuildValue, e.g. 'a' for ASCII string. Later we can add new functions like _PyDict_GetASCII(). The main difference between my new const ASCII string idea and PyIdentifier is the lifetime of objects. Using const ASCII strings, the strings are destroyed quickly (to not waste memory), whereas PyIdentifiers ^ are intern strings and so they are only destroyed at Python exit. I don't know if const ASCII strings solve a real issue. I implemented my idea. I will do some benchmarks to check if it's useful or not :-) Ok, I did some tests: it is slower with my PyConstASCIIObject. I don't understand why, but it means that the idea is not interesting because the code is not faster. I think you've already given the answer above... Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
