Re: [Python-Dev] Shorter release schedule?
Tennessee Leeuwenburg writes: > Can I ask if there's any sense in pursuing a release schedule which > is slow for whatever might be deemed the "most core modules" but > faster for "less core modules"? I think you need to be more specific about how many levels of "fast" there should be, and why some modules might be deemed more or less "core". For example, this is part of why bsddb (sp?) was removed from the stdlib, because its cycle is different from the core (it's heavily torqued by whatever upstream chooses to throw at it, so it has been the devil to test). If you're not familiar with the history, you might try searching the list for "bsddb 'Jesus Cea' stdlib" which should bring up relevant threads. (Make sure you spell the package name right, sorry if I got it wrong!) In short, the answer is "the stuff on a different cycle is already on PyPI". > Something else which would definitely be useful for me personally > would be a kind of update egg which I could apply to, say, Python > 3.0 to bring it up to 3.1 capabilities. But this would have to be considered on a per-feature basis. If that's possible for an individual feature (ie, doesn't involve changes to the interpreter or compiler), almost surely the feature "did hard time" in PyPI. So you can probably get some version there. OTOH, such an egg would have to contain only a subset of features. If there are interdependencies between that subset and those that can't be included, in some sense you will be creating a completely new and *untested* version of Python. So I think that most server admins would really want you to restrict to features you actually need, and therefore the "best" update-egg will be very application-specific. With the new features being proposed for dist-utils, I suppose you (or anybody who feels like doing so) could create a "namespace package" for such updates, pulling in the relevant modules from PyPI. Do you think that could work for you? (See the PEP 382 threads for more info; I think current discussion has moved to distutils-SIG). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
On Wed, May 13, 2009 at 1:26 PM, "Martin v. Löwis" wrote: > > Just food for thought here, but seeing how 3.1 is going to be a real > featureful > > schedule despite being released shortly after 3.0, wouldn't it make sense > to > > tighten future release planning a little? > > Do you have any specific releases in mind that you would like to apply > such a tightened schedule to? > > > I was thinking something like doing a > > major release every 12 months (rather than 18 to 24 months as has been > > heuristically the case lately). > If I can just respond with a bit of feedback from my workplace, I'd say that slower is better. I'm grimacing as I write that :) because I personally love to be able to take advantage of the new capabilities in each release. Can I ask if there's any sense in pursuing a release schedule which is slow for whatever might be deemed the "most core modules" but faster for "less core modules"? This is really a response to my workplace environment. The pro of that is that it's a real example, but the con is that it may not be best practise :) Something else which would definitely be useful for me personally would be a kind of update egg which I could apply to, say, Python 3.0 to bring it up to 3.1 capabilities. Something that already happens now at work reasonably often is that on my PC I have Python 2.4, 2.5, 2.6 and 3.0 installed. I tend to develop under 2.6 from preference. However, server X only has 2.4 installed or worse, 2.3 which I don't even have. Recently I was bitten by this as my code was relying heavily on some functionality in datetime which had changed. I was faced with having to do some re-architecting that I really didn't want to do. Now, I don't know of course (I found another way around the issue), but suppose the changes to Python I needed were relatively cosmetic, i.e. the kind of thing I could maybe install into a virtualenv wrapper, then it would have been quite easy for me to run my scripts written for Python 2.6. To get to the point, I wonder if it would be possible to release new versions alongside a patch or egg which someone with only user-level privileges could use on a server to avoid being held back by a slower server update cycle. A more frequent update cycle would then be easier to deal with. More features would get out into use more quickly, while the pressures of the lowest-common-denominator would be eased. Just some thoughts... Regards, -Tennessee ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Shorter release schedule?
Antoine Pitrou writes: > Just food for thought here, but seeing how 3.1 is going to be a > real featureful schedule despite being released shortly after 3.0, > wouldn't it make sense to tighten future release planning a little? With all due respect, it's easy and natural to have a short, featureful release schedule immediately after a major release (or should I say "complete rewrite"?) The discussion should focus on what happens as people become relatively satisfied with the core, and development shifts to optimizations, (smallish) bug fixes, and features that appeal to specialized audiences. That is when both the costs and the benefits of a tighter and/or time-based releases appear. > I was thinking something like doing a major release every 12 months > (rather than 18 to 24 months as has been heuristically the case > lately). This could also imply switching to some kind of loosely > time-based release system. I don't wish to express an opinion on either of these, as I'm not even in a position to help with release blockers. But I do hope discussion will focus on the implications for Python 3.7, not Python 3.3. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Web-SIG] py3k, cgi, email, and form-data
2009/5/12 Robert Brewer : > There's a major change in functionality in the cgi module between Python > 2 and Python 3 which I've just run across: the behavior of > FieldStorage.read_multi, specifically when an HTTP app accepts a file > upload within a multipart/form-data payload. > > In Python 2, each part would be read in sequence within its own > FieldStorage instance. This allowed file uploads to be shunted to a > TemporaryFile (via make_file) as needed: > > klass = self.FieldStorageClass or self.__class__ > part = klass(self.fp, {}, ib, > environ, keep_blank_values, strict_parsing) > # Throw first part away > while not part.done: > headers = rfc822.Message(self.fp) > part = klass(self.fp, headers, ib, > environ, keep_blank_values, strict_parsing) > self.list.append(part) > > In Python 3 (svn revision 72466), the whole request body is read into > memory first via fp.read(), and then broken into separate parts in a > second step: > > klass = self.FieldStorageClass or self.__class__ > parser = email.parser.FeedParser() > # Create bogus content-type header for proper multipart parsing > parser.feed('Content-Type: %s; boundary=%s\r\n\r\n' % (self.type, ib)) > parser.feed(self.fp.read()) > full_msg = parser.close() > # Get subparts > msgs = full_msg.get_payload() > for msg in msgs: > fp = StringIO(msg.get_payload()) > part = klass(fp, msg, ib, environ, keep_blank_values, > strict_parsing) > self.list.append(part) > > This makes the cgi module in Python 3 somewhat crippled for handling > multipart/form-data file uploads of any significant size (and since > the client is the one determining the size, opens a server up for an > unexpected Denial of Service vector). > > I *think* the FeedParser is designed to accept incremental writes, > but I haven't yet found a way to do any kind of incremental reads > from it in order to shunt the fp.read out to a tempfile again. > I'm secretly hoping Barry has a one-liner fix for this. ;) FWIW, Werkzeug gave up on 'cgi' module for form passing and implements its own. Not sure whether this issue in Python 3.0 was one of the reasons or not. I know one of the reasons was because cgi.FieldStorage is not WSGI 1.0 compliant. One of the main reasons that no one actually adheres to WSGI 1.0 is because of the 'cgi' module. This still hasn't been addressed by a proper amendment to WSGI 1.0 specification or a new WSGI 1.1 specification to allow a hint to readline(). The Werkzeug form processing module is properly WSGI 1.0 compliant, meaning that Wekzeug is possibly the only major WSGI framework to be WSGI compliant. Graham ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
MRAB wrote: Next you'll be saying that they should be named after years. Python 2010, anyone? :-) To keep people on their toes, we should switch to a completely random new naming scheme with every release, like Microsoft has been doing with Windows. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Web-SIG] py3k, cgi, email, and form-data
Graham Dumpleton wrote: > 2009/5/12 Robert Brewer : > > There's a major change in functionality in the cgi module between > Python > > 2 and Python 3 which I've just run across: the behavior of > > FieldStorage.read_multi, specifically when an HTTP app accepts a file > > upload within a multipart/form-data payload. > > > > In Python 2, each part would be read in sequence within its own > > FieldStorage instance. This allowed file uploads to be shunted to a > > TemporaryFile (via make_file) as needed: > > > > klass = self.FieldStorageClass or self.__class__ > > part = klass(self.fp, {}, ib, > > environ, keep_blank_values, strict_parsing) > > # Throw first part away > > while not part.done: > > headers = rfc822.Message(self.fp) > > part = klass(self.fp, headers, ib, > > environ, keep_blank_values, strict_parsing) > > self.list.append(part) > > > > In Python 3 (svn revision 72466), the whole request body is read into > > memory first via fp.read(), and then broken into separate parts in a > > second step: > > > > klass = self.FieldStorageClass or self.__class__ > > parser = email.parser.FeedParser() > > # Create bogus content-type header for proper multipart parsing > > parser.feed('Content-Type: %s; boundary=%s\r\n\r\n' % (self.type, > ib)) > > parser.feed(self.fp.read()) > > full_msg = parser.close() > > # Get subparts > > msgs = full_msg.get_payload() > > for msg in msgs: > > fp = StringIO(msg.get_payload()) > > part = klass(fp, msg, ib, environ, keep_blank_values, > > strict_parsing) > > self.list.append(part) > > > > This makes the cgi module in Python 3 somewhat crippled for handling > > multipart/form-data file uploads of any significant size (and since > > the client is the one determining the size, opens a server up for an > > unexpected Denial of Service vector). > > > > I *think* the FeedParser is designed to accept incremental writes, > > but I haven't yet found a way to do any kind of incremental reads > > from it in order to shunt the fp.read out to a tempfile again. > > I'm secretly hoping Barry has a one-liner fix for this. ;) > > FWIW, Werkzeug gave up on 'cgi' module for form passing and implements > its own. > > Not sure whether this issue in Python 3.0 was one of the reasons or > not. I know one of the reasons was because cgi.FieldStorage is not > WSGI 1.0 compliant. One of the main reasons that no one actually > adheres to WSGI 1.0 is because of the 'cgi' module. This still hasn't > been addressed by a proper amendment to WSGI 1.0 specification or a > new WSGI 1.1 specification to allow a hint to readline(). > > The Werkzeug form processing module is properly WSGI 1.0 compliant, > meaning that Wekzeug is possibly the only major WSGI framework to be > WSGI compliant. FWIW, I just added a replacement for the cgi module to CherryPy over the weekend for the same reasons. It's in the python3 branch but will get backported to CherryPy 3.2 for Python 2.x. Robert Brewer fuman...@aminus.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
> Just food for thought here, but seeing how 3.1 is going to be a real > featureful > schedule despite being released shortly after 3.0, wouldn't it make sense to > tighten future release planning a little? Do you have any specific releases in mind that you would like to apply such a tightened schedule to? > I was thinking something like doing a > major release every 12 months (rather than 18 to 24 months as has been > heuristically the case lately). Such a schedule was initially used for the first 2.x releases. We then switched to 18 months because of user complaints: if releases come too frequently, the users are confused as to what release they should be using. Even 24 months is too frequently for some: some people are only starting to move to 2.5 right now - when we have stopped maintaining it already. One question is what would happen to the old releases: would we still maintain them? If so, how many of them? For how long? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
On Tue, May 12, 2009 at 3:06 PM, Antoine Pitrou wrote: > Hello, > > Just food for thought here, but seeing how 3.1 is going to be a real > featureful > schedule despite being released shortly after 3.0, wouldn't it make sense to > tighten future release planning a little? I was thinking something like doing > a > major release every 12 months (rather than 18 to 24 months as has been > heuristically the case lately). This could also imply switching to some kind > of > loosely time-based release system. I'd be in favor of a shorter, 12-month release cycle. I think the limiting resource would be the time and energy of the release managers and the package builders for Windows, etc. Provided it's not a tax on the release staff, I think shorter release cycles would be a benefit to the community. My own experience with time-based releases at work is that it greatly helps focus energy and attention, knowing that you can't simply delay the release if you slack off on your features/bugs. Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
On May 12, 2009, at 6:06 PM, Antoine Pitrou wrote: Just food for thought here, but seeing how 3.1 is going to be a real featureful schedule despite being released shortly after 3.0, wouldn't it make sense to tighten future release planning a little? I was thinking something like doing a major release every 12 months (rather than 18 to 24 months as has been heuristically the case lately). This could also imply switching to some kind of loosely time-based release system. If I'm wildly off-base, you can either flame me, ignore me, or assign me annoying release blockers involving memoryviews and weird character encodings :-) I've been in favor of that for a while now. With the move to a DVCS (how's that coming along?) I think we can have more solid, releasable trunks for longer in the cycle. Then, we'd have feature branches which wouldn't land in trunk until they too are solid and complete (with docs and tests). If a particular feature doesn't make it, it'll just wait until the next release, which would be only 12 months off instead of almost 2 years off. -Barry PGP.sig Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
MRAB mrabarnett.plus.com> writes: > Next you'll be saying that they should be named after years. Python > 2010, anyone? After py3k, that would be a regression ;) cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shorter release schedule?
Antoine Pitrou wrote: Hello, Just food for thought here, but seeing how 3.1 is going to be a real featureful schedule despite being released shortly after 3.0, wouldn't it make sense to tighten future release planning a little? I was thinking something like doing a major release every 12 months (rather than 18 to 24 months as has been heuristically the case lately). This could also imply switching to some kind of loosely time-based release system. If I'm wildly off-base, you can either flame me, ignore me, or assign me annoying release blockers involving memoryviews and weird character encodings :-) Next you'll be saying that they should be named after years. Python 2010, anyone? :-) I think that releases should depend on whether there are enough changes for one. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Shorter release schedule?
Hello, Just food for thought here, but seeing how 3.1 is going to be a real featureful schedule despite being released shortly after 3.0, wouldn't it make sense to tighten future release planning a little? I was thinking something like doing a major release every 12 months (rather than 18 to 24 months as has been heuristically the case lately). This could also imply switching to some kind of loosely time-based release system. If I'm wildly off-base, you can either flame me, ignore me, or assign me annoying release blockers involving memoryviews and weird character encodings :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CVE-2008-5983 "untrusted python modules search path"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou napsal(a): > Hello, > > I don't think it has already posted to the list, apologies if it has. > > Some Linux tools and vendors have been hit by an alleged "security hole" where > an embedded Python interpreter will prepend the current working directory to > sys.path as soon as PySys_SetArgv() is called by the embedding application. > This > means, for example, that a Python file in the working directory can break > plugins or extensions written for that application if the Python file happens > to > shadow another module. > > Regardless of whether this is a security hole or not, it certainly can make > things disturbingly surprising when the situation arises. In the bug report > (http://bugs.python.org/issue5753), I suggested we add a new function > PySys_SetArgvEx() which would take an additional parameter telling whether to > touch sys.path or not (in the same spirit as Py_InitializeEx() providing a > more > flexible API than Py_Initialize()). > > On the other hand, I don't think we can change the default behaviour of > PySys_SetArgv(), since there are probably tools and applications relying on it > (the obvious use case which comes to my mind is a third-party interactive > interpreter). > > Any opinions? yes! Actually, i wanted to propose and implement something like this back when this vulnerability appeared, but i never got to it. I'd propose to create a whole new function, called, say, PySys_FillArgv() (no, i don't think that's a very good name) that would - -only- fill sys.argv and not touch sys.path. In addition to that, there would be a function like PySys_SetScriptPath() that would not fill sys.argv, but prepend the script's directory to sys.path Then i'd reimplement PySys_SetArgv as { PySys_FillArgv(); PySys_SetScriptPath(); } And as a final killing step, i would never ever mention PySys_SetArgv anywhere but in its own documentation ;e) And especially not in the first page of "Embedding Python". My rationale is that the only application deliberately using PySys_SetArgv the way it's written is a Python interpreter. For that, it's desirable to have '.' in sys.path _when no script is being executed_. For *all other applications*, this makes no sense ;e) regards m. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jmatejek%40suse.cz -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkoJwywACgkQjBrWA+AvBr8UQwCgmLdu+aq9pYUxbSn/7i7hF1dK lw0AnRo0UCBbszxtzeXNcmmdO7d9sYx4 =0tU7 -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switchover: mail.python.org
On May 11, 2009, at 1:29 PM, Jeroen Ruigrok van der Werven wrote: -On [20090511 14:47], Aahz (a...@pythoncraft.com) wrote: On Monday 2009-05-11, mail.python.org will be switched to another machine starting roughly at 14:00 UTC. This should be invisible (expected downtime is less than ten minutes). The headers for the python checkins mails are apparently different now. So people might want to adjust any filtering. Somehow, personalization got turned off for python-checkins. This disables VERPing of the headers. I've turned it back on, so please let me know if that fixes the issue. This did not appear to happen site-wide, just for python-checkins AFAICT. -Barry PGP.sig Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switchover: mail.python.org
-On [20090512 18:41], Barry Warsaw (ba...@python.org) wrote: >Somehow, personalization got turned off for python-checkins. This >disables VERPing of the headers. I've turned it back on, so please >let me know if that fixes the issue. This did not appear to happen >site-wide, just for python-checkins AFAICT. Yes, the current batches are arriving with personilization again. I don't mind either way, just thought a heads up was warranted. ;) Thanks Barry, -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B The Idea does not replace the work... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A wordcode-based Python
On Tue, May 12, 2009 05:27 PM, Collin Winter wrote: > On Tue, May 12, 2009 at 4:45 AM, Cesare Di Mauro > wrote: >> Another note. Fredrik Johansson let me note just few minutes ago that >> I've >> compiled my sources without PGO optimizations enabled. >> >> That's because I used Visual Studio Express Edition. >> >> So another gain in performances can be obtained. :) > > FWIW, Unladen Swallow experimented with gcc 4.4's FDO and got an > additional 10-30% (depending on the benchmark). The training load is > important, though: some training sets offered better performance than > others. I'd be interested in how MSVC's PGO compares to gcc's FDO in > terms of overall effectiveness. The results for gcc FDO with our > 2009Q1 release are at the bottom of > http://code.google.com/p/unladen-swallow/wiki/Releases. > > Collin Winter Unfortunately I can't test PGO, since I use the Express Editions of VS. May be Martin or othe mainteners of the Windows versions can help here. However it'll be difficult to find a good enough profile for the binaries distributed for the official Python. FDO brings to quite different results based on the profile selected. cheers, Cesare ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A wordcode-based Python
On Tue, May 12, 2009 at 4:45 AM, Cesare Di Mauro wrote: > Another note. Fredrik Johansson let me note just few minutes ago that I've > compiled my sources without PGO optimizations enabled. > > That's because I used Visual Studio Express Edition. > > So another gain in performances can be obtained. :) FWIW, Unladen Swallow experimented with gcc 4.4's FDO and got an additional 10-30% (depending on the benchmark). The training load is important, though: some training sets offered better performance than others. I'd be interested in how MSVC's PGO compares to gcc's FDO in terms of overall effectiveness. The results for gcc FDO with our 2009Q1 release are at the bottom of http://code.google.com/p/unladen-swallow/wiki/Releases. Collin Winter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A wordcode-based Python
On Thu, May 12, 2009 01:40PM, Antoine Pitrou wrote: > > Hi Cesare, > > Cesare Di Mauro a-tono.com> writes: >> >> It was my idea too, but first I need to take a deep look at what parts >> of code are changed from 2.6 to 3.0. >> That's because I don't know how much work is required for this >> "forward" port. > > If you have some questions or need some help, send me a message. > > Regards > > Antoine. OK, thanks. :) Another note. Fredrik Johansson let me note just few minutes ago that I've compiled my sources without PGO optimizations enabled. That's because I used Visual Studio Express Edition. So another gain in performances can be obtained. :) cheers Cesare ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A wordcode-based Python
Hi Cesare, Cesare Di Mauro a-tono.com> writes: > > It was my idea too, but first I need to take a deep look at what parts > of code are changed from 2.6 to 3.0. > That's because I don't know how much work is required for this > "forward" port. If you have some questions or need some help, send me a message. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com