Re: [Python-Dev] Coroutines and PEP 380

2012-01-18 Thread Mark Shannon

Matt Joiner wrote:
Just to clarify, this differs in functionality from enhanced generators 
by allowing you to yield from an arbitrary call depth rather than having 
to yield from through a chain of calling generators? Furthermore 
there's no syntactical change except to the bottommost frame doing a 
co_yield? Does this capture the major differences?



Yes.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Coroutines and PEP 380

2012-01-18 Thread Mark Shannon

Glyph wrote:

On Jan 17, 2012, at 5:03 PM, Mark Shannon wrote:


Lets start controversially: I don't like PEP 380, I think it's a kludge.


Too late; it's already accepted.  There's not much point in making 
controversial statements about it now.


Why is it too late? Presenting this as a fait accompli does not make it 
any better. The PEP mailing list is closed to most people, so what forum 
for debate is there?




I think that CPython should have proper coroutines, rather than add 
more bits and pieces to generators in an attempt to make them more 
like coroutines.


By proper coroutines, you mean implicit coroutines (cooperative 
threads) rather than explicit coroutines (cooperative generators). 

Nothing implicit about it.
 Python has been going in the explicit direction on this question for 
a long time.  (And, in my opinion, this is the right direction to go, 
but that's not really relevant here.)


You can use asymmetric coroutines with a scheduler to provide 
cooperative threads if you want, but coroutines not have to be used as 
threads.


The key advantages of my coroutine implmentation over PEP 380 are:

1. No syntax change.
2. Code can be used in coroutines without modification.
3. No stack unwinding is required at a yield point.



I think this discussion would be more suitable for python-ideas though, 
since you have a long row to hoe here.  There's already a PEP - 
http://www.python.org/dev/peps/pep-0219/ - apparently deferred and not 
rejected, which you may want to revisit.


There are several libraries which can give you cooperative threading 
already; I assume you're already aware of greenlet and stackless, but I 
didn't see what advantages your proposed implementation provides over 
those.  I would guess that one of the first things you should address on 
python-ideas is why adopting your implementation would be a better idea 
than just bundling one of those with the standard library :).


Already been discussed:
http://mail.python.org/pipermail/python-ideas/2011-October/012571.html

All of the objections to coroutines (as I propose) also apply to PEP 380.

The advantage of my implementation over greenlets is portability.

I suspect stackless is actually fairly similar to what I have done,
I haven't checked in detail.

Cheers,
Mark.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-18 Thread Victor Stinner
2012/1/18 Martin v. Löwis mar...@v.loewis.de:
 For 3.3 onwards, I'm skeptical whether all this configuration support is
 really necessary. I think a much smaller patch which leaves no choice
 would be more appropriate.

The configuration helps unit testing: see changes on Lib/test/*.py in
my last patch. I hesitate to say that the configuration is required
for tests. Anyway, users upgrading from Python 3.2 to 3.3 may need to
keep the same hash function and don't care of security (e.g. programs
running locally with trusted data).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-18 Thread Hrvoje Niksic

On 01/17/2012 09:29 PM, Martin v. Löwis wrote:

I(0) = H  MASK
PERTURB(0) = H
I(n+1) = (5*I(n) + 1 + PERTURB(n))  MASK
PERTURN(n+1) = PERTURB(n)  5

So if two objects O1 and O2 have the same hash value H, the sequence of
probed indices is the same for any MASK value. It will be a different
sequence, yes, but they will still collide on each and every slot.

This is the very nature of open addressing.


Open addressing can still deploy a collision resolution mechanism 
without this property. For example, double hashing uses a different hash 
function (applied to the key) to calculate PERTURB(0). To defeat it, the 
attacker would have to produce keys that hash the same using both hash 
functions.


Double hashing is not a good general solution for Python dicts because 
it complicates the interface of hash tables that support arbitrary keys. 
Still, it could be considered for dicts with known key types (built-ins 
could hardcode the alternative hash function) or for SafeDicts, if they 
are still considered.


Hrvoje
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Antoine Pitrou
On Wed, 18 Jan 2012 07:52:20 +
Paul Moore p.f.mo...@gmail.com wrote:
 On 18 January 2012 07:46, Georg Brandl g.bra...@gmx.net wrote:
  But I am dubious that releases that are obsolete in 6 months and lack
  3rd party support will see much production use.
 
  Whether people would use the releases is probably something that only
  they can tell us -- that's why a community survey is mentioned in the
  PEP.
 
 The class of people who we need to consider carefully is those who
 want to use the latest release, but are limited by the need for other
 parties to release stuff that works with that release (usually, this
 means Windows binaries of extensions, or platform vendor packaged
 releases of modules/packages).

Well, do consider, though, that anyone not using third-party C
extensions under Windows (either Windows users that are content with
pure Python libs, or users of other platforms) won't have that problem.
That should be quite a lot of people already.

As for vendors, they have their own release management independent of
ours already, so this PEP wouldn't change anything for them.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Antoine Pitrou
On Wed, 18 Jan 2012 11:37:08 +0900
Stephen J. Turnbull step...@xemacs.org wrote:
   availability of release management volunteers,
 
 Dramatic increase here.  It may look like RM is not so demanding --
 run a few scripts to put out the alphas/betas/releases.  But the RM
 needs to stay on top of breaking news, make decisions.  That takes
 time, interrupts other work, etc.

Georg and Barry may answer you here: they are release managers and PEP
co-authors.

   quick availability of new features (and behavioural changes),
 
 These are already *available*, just not *tested*.
 
 Since testing is the bottleneck on what users consider to be
 available for me, you cannot decrease the amount of testing (alpha,
 beta releases) by anywhere near the amount you're increasing
 frequency, or you're just producing as is snapshots.

The point is to *increase* the amount of testing by making features
available in stable releases on a more frequent basis. Not decrease it.

Alphas and betas never produce much feedback, because people are
reluctant to install them for anything else than toying around. Python
is not emacs or Firefox, you don't use it in a vacuum and therefore
installing non-stable versions is dangerous.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Nick Coghlan
This won't be a surprise to Antoine or Georg (since I've already
expressed the same opinion privately), but I'm -1 on the idea of
official releases of the whole shebang every 6 months. We're not
Ubuntu, Fedora, Chrome or Firefox with a for-profit company (or large
foundation) with multiple paid employees kicking around to really
drive the QA process. If we had official support from Red Hat or
Canonical promising to devote paid QA and engineering resources to
keeping things on track my opinion might be different, but that is
highly unlikely. I'm also wholly in agreement with Ezio that using the
same versioning scheme for both full releases and interim releases is
thoroughly confusing for users (for example, I consider Red Hat's
completely separate branding and versioning for Fedora and RHEL a
better model for end users than Canonical's more subtle 'Ubuntu' and
'Ubuntu LTS' distinction, and that's been my opinion since long before
I started working for RH).

My original suggestion to Antoine and Georg for 3.4 was that we simply
propose to Larry Hastings (the 3.4 RM) that we spread out the release
cycle, releasing the first alpha after ~6 months, the second after
about ~12, then rolling into the regular release cycle of a final
alpha, some beta releases, one or two release candidates and then the
actual release. However, I'm sympathetic to Antoine's point that early
alphas aren't likely to be at all interesting to folks that would like
a fully supported stdlib update to put into production and no longer
think that suggestion makes much sense on its own.

Instead, if the proposal involves instituting a PEP 3003 style
moratorium (i.e. stdlib changes only) for all interim releases, then
we're essentially talking about splitting the versioning of the core
language (and the CPython C API) and the standard library. If we're
going to discuss that, we may as well go a bit further and just split
development of the two out onto separate branches, with the current
numbering scheme applying to full language version releases and
switching to a date-based versioning scheme for the standard library
(i.e. if 3.3 goes out in August as planned, then it would be Python
3.3 with the 12.08 stdlib release).

What might such a change mean?


1. For 3.3, the following releases would be made:
- 3.2.x is cut from the 3.2 branch (1 rc + 1 release)
- 3.3.0 + PyStdlib 12.08 is created from the default branch (1
alpha, 2 betas, 1+ rc, 1 release)
- the 3.3 maintenance branch is created
- the stdlib development branch is created

2. Once 3.2 goes into security-fix only mode, this would then leave us
with 4 active branches:
- 2.7 (maintenance)
- 3.3 (maintenance)
- stdlib (Python 3.3 compatible, PEP 3003 compliant updates)
- default (3.4 development)

The 2.7 branch would remain a separate head of development, but for
3.x development the update flow would become:
Bug fixes: 3.3-stdlib-default
Stdlib features: stdlib-default
Language changes: default

3. Somewhere around February 2013, we prepare to release Python 3.4a1
and 3.3.1, along with PyStdlib 13.02:
- 3.3.1 + PyStdlib 12.08 is cut from the 3.3 branch (1 rc + 1 release)
- 3.3.1 + PyStdlib 13.02 comes from the stdlib branch (1 alpha, 1
beta, 1+ rc, 1 release)
- 3.4.0a1 comes from the default branch (may include additional
stdlib changes)

4. Around August 2013 this process repeats:
- 3.3.2 + PyStdlib 12.08 is cut from the 3.3 branch
- 3.3.2 + PyStdlib 13.08 comes from the stdlib branch (final 3.3
compatible stdlib release)
- 3.4.0a2 comes from the default branch

5. And then in February 2014, we gear up for a new major release:
- 3.3.3 is cut from the 3.3 branch and the 3.3 branch enters
security fix only mode
- 3.4.0 + PyStdlib 14.02 is created from the default branch (1
alpha, 2 betas, 1+ rc, 1 release)
- the 3.4 maintenance branch is created and merged into the stdlib branch

(alternatively, Feb 2014 could be another interim release of 3.4 alpha
and a 3.3 compatible stdlib updated, with 3.4 delayed until August
2014)

I believe this approach would get to the core of what the PEP authors
want (i.e. more frequent releases of the standard library), while
being quite explicit in *avoiding* the concerns associated with more
frequent releases of the core language itself. The rate of updates on
the language spec, the C API (and ABI), the bytecode format and the
AST would remain largely unchanged at 18-24 months. Other key
protocols (e.g. default pickle formats) could also be declared
ineligible for changes in interim releases.

If a critical security problem is found, then additional releases may
be cut for the maintenance branch and for the stdlib branch.

There's a slight annoyance in having all development filtered through
an additional branch, but there's a large advantage in that having a
stable core in the stdlib branch makes it more likely we'll be able to
use it as a venue for collaboration with the 

Re: [Python-Dev] Coroutines and PEP 380

2012-01-18 Thread Glyph
On Jan 18, 2012, at 4:23 AM, Mark Shannon wrote:

 Glyph wrote:
 On Jan 17, 2012, at 5:03 PM, Mark Shannon wrote:
 Lets start controversially: I don't like PEP 380, I think it's a kludge.
 Too late; it's already accepted.  There's not much point in making 
 controversial statements about it now.
 
 Why is it too late?

Because discussion happens before the PEP is accepted.  See the description of 
the workflow in http://www.python.org/dev/peps/pep-0001/.  The time to object 
to PEP 380 was when those threads were going on.

 Presenting this as a fait accompli does not make it any better.

But it is[1] a fait accompli, whether you like it or not; I'm first and 
foremost informing you of the truth, not trying to make you feel better (or 
worse).  Secondly, I am trying to forestall a long and ultimately pointless 
conversation :).

 The PEP mailing list is closed to most people,

The PEP mailing list is just where you submit your PEPs, and where the PEP 
editors do their work.  I'm not on it, but to my understanding of the process, 
there's not really any debate there.

 so what forum for debate is there?

python-ideas, and then this mailing list, in that order.  Regarding PEP 380 
specifically, there's been quite a bit.  See for example 
http://thread.gmane.org/gmane.comp.python.devel/102161/focus=102164.  Keep in 
mind that the purpose of debate in this context is to inform Guido's opinion.  
There's no voting involved, although he will occasionally delegate decisions 
about particular PEPs to people knowledgeable in a relevant area.

 I think this discussion would be more suitable for python-ideas though [...]
 Already been discussed:
 http://mail.python.org/pipermail/python-ideas/2011-October/012571.html

If you're following the PEP process, then the next step would be for you 
(having built some support) to author a new PEP, or to resurrect the deferred 
Stackless PEP with some new rationale - personally I'd recommend the latter.

My brief skimming of the linked thread doesn't indicate you have a lot of 
strong support though, just some people who would be somewhat interested.  So I 
still think it bears more discussion there, especially on the motivation / 
justification side of things.

 All of the objections to coroutines (as I propose) also apply to PEP 380.

You might want to see the video of Guido's Fireside Chat last year 
http://pycon.tv/#/video/100.  Skip to a little before 15:00.  He mentions the 
point that coroutines that can implicitly switch out from under you have the 
same non-deterministic property as threads: you don't know where you're going 
to need a lock or lock-like construct to update any variables, so you need to 
think about concurrency more deeply than if you could explicitly always see a 
'yield'.  I have more than one painful event in my past (as he refers to it) 
indicating that microthreads have the same problem as real threads :).

(And yes, they're microthreads, even if you don't have an elaborate scheduling 
construct.  If you can switch to another stack by making a function call, then 
you are effectively context switching, and it can become arbitrarily complex.  
Any coroutine in a system may introduce an arbitrarily complex microthread 
scheduler just by calling a function that yields to it.)

-glyph

([1]: Well actually it isn't, note the dashed line from Accepted to 
Rejected in the workflow diagram.  But you have to have a really darn good 
reason, and championing the rejection of a pep that Guido has explicitly 
accepted and has liked from pretty much the beginning is going to be very, very 
hard.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-18 Thread Barry Warsaw
On Jan 18, 2012, at 08:19 AM, Martin v. Löwis wrote:

My concern is not about breaking doctests: this proposal will also break
them. My concern is about applications that assume that hash(s) is
stable across runs, and we do have reports that it will break
applications.

I am a proponent of doctests, and thus use them heavily.  I can tell you that
the issue of dict hashing (non-)order has been well known for *years* and I
have convenience functions in my own doctests to sort and print dict
elements.  Back in my Launchpad days (which has oodles of doctests), many
years ago we went on a tear to fix dict printing when some change in Python
caused them to break.  So I'm not personally worried that such a change would
break any of my own code.

Even though I hope anybody who uses doctests has their own workarounds for
this, I still support being conservative in default behavior for stable
releases, because it's the right thing to do for our users.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Antoine Pitrou

Le mercredi 18 janvier 2012 à 21:26 +1000, Nick Coghlan a écrit :
 I'm also wholly in agreement with Ezio that using the
 same versioning scheme for both full releases and interim releases is
 thoroughly confusing for users 

It's a straight-forward way to track the feature support of a release.
How do you suggest all these sys.version_info = (3, 2) - and the
corresponding documentation snippets a.k.a versionadded or
versionchanged tags - be spelt otherwise?

 for example, I consider Red Hat's
 completely separate branding and versioning for Fedora and RHEL a
 better model for end users

It's not only branding and versioning, is it? They're completely
different projects with different goals (and different commercial
support).

If you're suggesting we do only short-term releases and leave the
responsibility of long-term support to another project or entity, I'm
not against it, but it's far more radical than what we are proposing in
the PEP :-)

 Instead, if the proposal involves instituting a PEP 3003 style
 moratorium (i.e. stdlib changes only) for all interim releases, then
 we're essentially talking about splitting the versioning of the core
 language (and the CPython C API) and the standard library. If we're
 going to discuss that, we may as well go a bit further and just split
 development of the two out onto separate branches, with the current
 numbering scheme applying to full language version releases and
 switching to a date-based versioning scheme for the standard library
 (i.e. if 3.3 goes out in August as planned, then it would be Python
 3.3 with the 12.08 stdlib release).

Well, you're opposing the PEP on the basis that it's workforce-intensive
but you're proposing something much more workforce-intensive :-)

Splitting the stdlib:
- requires someone to do the splitting (highly non-trivial given the
interactions of some modules with interpreter details or low-level C
code)
- requires setting up separate resources (continuous integration with N
stdlib versions and M interpreter versions, for example)
- requires separate maintenance and releases for the stdlib (but with
non-trivial interaction with interpreter maintenance, since they will
affect each other and must be synchronized for Python to be usable at
all)
- requires more attention by users since there are now *two* release
schedules and independent version numbers to track

The former two are one-time costs, but the latter two are recurring
costs.

Therefore, splitting the stdlib is much more complicated and involved
than many people think; it's not just move a few directories around and
be done.
And it's not even obvious it would have an actual benefit, since
developers of other implementations are busy doing just that (see Jeff
Hardy's message in this thread).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Stephen J. Turnbull
Antoine Pitrou writes:

   Since testing is the bottleneck on what users consider to be
   available for me, you cannot decrease the amount of testing (alpha,
   beta releases) by anywhere near the amount you're increasing
   frequency, or you're just producing as is snapshots.
  
  The point is to *increase* the amount of testing by making features
  available in stable releases on a more frequent basis. Not decrease
  it.

We're talking about different kinds of testing.  You're talking about
(what old-school commercial software houses meant by beta) testing
in a production or production prototype environment.  I'd love to see
more of that, too!  My claim is that I don't expect much uptake if you
don't do close to as many of what are called alpha and beta tests
on python-dev as are currently done.

  Alphas and betas never produce much feedback, because people are
  reluctant to install them for anything else than toying around. Python
  is not emacs or Firefox, you don't use it in a vacuum
  and therefore installing non-stable versions is dangerous.

Exactly my point, except that the PEP authors seem to think that we
can cut back on the number of alpha and beta prereleases and still
achieve the stability that such users expect from a Python release.  I
don't think that's right.  I expect that unless quite substantial
resources (far more than proportional to 1/frequency) are devoted to
each non-LTS release, a large fraction of such users to avoid non-LTS
releases the way they avoid betas now.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Antoine Pitrou

Le mercredi 18 janvier 2012 à 21:48 +0900, Stephen J. Turnbull a écrit :
 My claim is that I don't expect much uptake if you
 don't do close to as many of what are called alpha and beta tests
 on python-dev as are currently done.

You claim people won't use stable releases because of not enough alphas?
That sounds completely unrelated. I don't know of any users who would
bother about that.
(you can produce flimsy software with many alphas, too)

   Alphas and betas never produce much feedback, because people are
   reluctant to install them for anything else than toying around. Python
   is not emacs or Firefox, you don't use it in a vacuum
   and therefore installing non-stable versions is dangerous.
 
 Exactly my point, except that the PEP authors seem to think that we
 can cut back on the number of alpha and beta prereleases and still
 achieve the stability that such users expect from a Python release.  I
 don't think that's right.

Sure, and we think it is :)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Nick Coghlan
On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou solip...@pitrou.net wrote:
 Splitting the stdlib:
 - requires someone to do the splitting (highly non-trivial given the
 interactions of some modules with interpreter details or low-level C
 code)
 - requires setting up separate resources (continuous integration with N
 stdlib versions and M interpreter versions, for example)
 - requires separate maintenance and releases for the stdlib (but with
 non-trivial interaction with interpreter maintenance, since they will
 affect each other and must be synchronized for Python to be usable at
 all)
 - requires more attention by users since there are now *two* release
 schedules and independent version numbers to track

Did you read what I actually proposed? I specifically *didn't* propose
separate stdlib releases (for all the reasons you point out), only
separate date based stdlib *versioning*. Distribution of the CPython
interpreter + stdlib would remain monolithic, as it is today. Any
given stdlib release would only be supported for the most recent
language release. The only difference is that between language
releases, where we currently only release maintenance builds, we'd
*also* release a second version of each maintenance build with an
updated standard library, along with an alpha release of the next
language version (with the last part being entirely optional, but I
figured I may as well make the suggestion since I like the idea to
encourage getting syntax updates and the like out for earlier
experimentation).

When you initially pitched the proposal via email, you didn't include
the language moratarium applies to interim releases idea. That one
additional suggestion makes the whole concept *much* more appealing to
me, but I only like it on the condition that we decouple the stdlib
versioning from the language definition versioning (even though I
recommend we only officially support very specific combinations of the
two). My suggestion is really just a concrete proposal for
implementing Ezio's idea of only bumping the Python version for
releases with long term support, and using some other mechanism to
distinguish the interim releases.

So, assuming a 2 year LTS cycle, the released versions up to February
2015 with my suggestion would end up being:

From the default branch:
Python 3.3.0 + stdlib 12.08.0  (~August 2012)
Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013)
Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)
Python 3.4.0a4 + stdlib 14.08.0a4  (~2014)
Python 3.4.0b1 + stdlib 14.08.0b1  (~2014)
Python 3.4.0b2 + stdlib 14.08.0b2  (~2014)
Python 3.4.0c1 + stdlib 14.08.0c1  (~2014)
Python 3.4.0 + stdlib 14.08  (~August 2014)
Python 3.5.0a1 + stdlib 16.08.0a1  (~February 2015)

From the 3.3 maintenance branch (these are maintenance updates to the
LTS release):
Python 3.3.1 + stdlib 12.08.1  (~February 2013)
Python 3.3.2 + stdlib 12.08.2  (~August 2013)
Python 3.3.3 + stdlib 12.08.3  (~February 2014)
Python 3.3.4 + stdlib 12.08.4  (~August 2014) (and 3.3 branch enters
security patch only mode)

From the 3.4 maintenance branch (these are maintenance updates to the
LTS release):
Python 3.4.1 + stdlib 14.08.1  (~February 2015)

From the stdlib feature development branch (these are the new interim
releases with standard library updates only as proposed by PEP 407):
Python 3.3.1 + stdlib 13.02.0  (~February 2013)
Python 3.3.2 + stdlib 13.08.0  (~August 2013)
Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path
from here is to make the jump to 3.4.0)
-- 3.4.0 + 12.08.0 is released from default branch --
Python 3.4.1 + stdlib 15.02.0  (~February 2015)

If we have to make brown paper bag releases for the maintenance or
stdlib branches then the micro versions get bumped - the date based
version of the standard library versions relates to when that
particular *API* was realised, not when bugs were last fixed in it. If
a target release date slips, then the stdlib version would be
increased accordingly (cf. Ubuntu 6.06).

Yes, we'd have an extra set of active buildbots to handle the stdlib
branch, but a) that's no harder than creating the buildbots for a new
maintenance branch and b) the interim release proposal will need to
separate language level changes from stdlib level changes *anyway*.

As far as how sys.version checks would be updated, I would propose a
simple API addition to track the new date-based standard lib
versioning: sys.stdlib_version. People could choose to just depend on
a specific Python version (implicitly depending on the stdlib version
that was originally shipped with that version of CPython), or they may
instead decide to depend on a specific stdlib version (implicitly
depending on the first Python version that was shipped with that
stdlib).

The reason I like this scheme is that it allows us (and users) to
precisely track the things that can vary at the two different rates.
At least the following would still be governed by 

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Stephen J. Turnbull
Antoine Pitrou writes:

  You claim people won't use stable releases because of not enough
  alphas?  That sounds completely unrelated.

Surely testing is related to user perceptions of stability.  More
testing helps reduce bugs in released software, which improves user
perception of stability, encouraging them to use the software in
production.  Less testing, then, will have the opposite effect.  But
you understand that theory, I'm sure.  So what do you mean to say?

  (you can produce flimsy software with many alphas, too)

The problem is the converse: can you produce Python-release-quality
software with much less pre-release testing than current feature
releases get?

  Sure, and we think it is [possible to do that] :)

Given the relative risk of rejecting PEP 407 and me being wrong (the
status quo really isn't all that bad AFAICS), vs. accepting PEP 407
and you being wrong, I don't find a smiley very convincing.  In fact,
I don't find the PEP itself convincing -- and I'm not the only one.

We'll see what Barry and Georg have to say.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Antoine Pitrou

Le jeudi 19 janvier 2012 à 00:25 +0900, Stephen J. Turnbull a écrit :
 
   You claim people won't use stable releases because of not enough
   alphas?  That sounds completely unrelated.
 
 Surely testing is related to user perceptions of stability.  More
 testing helps reduce bugs in released software, which improves user
 perception of stability, encouraging them to use the software in
 production.

I have asked a practical question, a theoretical answer isn't exactly
what I was waiting for.

   Sure, and we think it is [possible to do that] :)
 
 Given the relative risk of rejecting PEP 407 and me being wrong (the
 status quo really isn't all that bad AFAICS), vs. accepting PEP 407
 and you being wrong, I don't find a smiley very convincing.

I don't care to convince *you*, since you are not involved in Python
development and release management (you haven't ever been a contributor
AFAIK). Unless you produce practical arguments, saying I don't think
you can do it is plain FUD and certainly not worth answering to.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Senthil Kumaran
On Wed, Jan 18, 2012 at 09:26:19PM +1000, Nick Coghlan wrote:
 My original suggestion to Antoine and Georg for 3.4 was that we simply
 propose to Larry Hastings (the 3.4 RM) that we spread out the release
 cycle, releasing the first alpha after ~6 months, the second after
 about ~12, then rolling into the regular release cycle of a final
 alpha, some beta releases, one or two release candidates and then the
 actual release. However, I'm sympathetic to Antoine's point that early
 alphas aren't likely to be at all interesting to folks that would like
 a fully supported stdlib update to put into production and no longer
 think that suggestion makes much sense on its own.

This looks like a 'good bridge' of suggestion between rapid releases
and stable releases. 

What would be purpose of alpha release. Would we encourage people to
use it or test it? Which the rapid relase cycle, the encouragement is
to use rather than test.

-- 
Senthil
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Antoine Pitrou
On Thu, 19 Jan 2012 01:06:07 +1000
Nick Coghlan ncogh...@gmail.com wrote:
 On Wed, Jan 18, 2012 at 2:31 PM,  solip...@pitrou.net wrote:
  results for 12de1ad1cee8 on branch default
  
 
  test_capi leaked [2008, 2008, 2008] references, sum=6024
 
 Yikes, you weren't kidding about that new subinterpreter code
 execution test upsetting the refleak detection...

Well, these are real leaks, but I expect them to be quite difficult to
track (I've found a couple of them), because they can be scattered
around in C module initialization routines and the like. I suggest we
skip this test on refleak runs.

cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-18 Thread PJ Eby
On Tue, Jan 17, 2012 at 7:58 PM, Martin v. Löwis mar...@v.loewis.dewrote:

 Am 17.01.2012 22:26, schrieb Antoine Pitrou:
  Only 2 bits are used in ob_sstate, meaning 30 are left. These 30 bits
  could cache a hash perturbation computed from the string and the
  random bits:
 
  - hash() would use ob_shash
  - dict_lookup() would use ((ob_shash * 103) ^ (ob_sstate  ~3))
 
  This way, you cache almost all computations, adding only a computation
  and a couple logical ops when looking up a string in a dict.

 That's a good idea. For Unicode, it might be best to add another slot
 into the object, even though this increases the object size.


Wouldn't that break the ABI in 2.x?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Brett Cannon
On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou solip...@pitrou.net wrote:

 On Thu, 19 Jan 2012 01:06:07 +1000
 Nick Coghlan ncogh...@gmail.com wrote:
  On Wed, Jan 18, 2012 at 2:31 PM,  solip...@pitrou.net wrote:
   results for 12de1ad1cee8 on branch default
   
  
   test_capi leaked [2008, 2008, 2008] references, sum=6024
 
  Yikes, you weren't kidding about that new subinterpreter code
  execution test upsetting the refleak detection...

 Well, these are real leaks, but I expect them to be quite difficult to
 track (I've found a couple of them), because they can be scattered
 around in C module initialization routines and the like. I suggest we
 skip this test on refleak runs.


Do we have any general strategy to help make it more fine-grained to detect
where the leak might be coming from? We could then maybe try to get some
people pound on this at the PyCon sprints. Otherwise I'm reluctant to skip
it since they are legitimate leaks that should be get fixed.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Antoine Pitrou
On Wed, 18 Jan 2012 11:14:50 -0500
Brett Cannon br...@python.org wrote:

 On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou solip...@pitrou.net wrote:
 
  On Thu, 19 Jan 2012 01:06:07 +1000
  Nick Coghlan ncogh...@gmail.com wrote:
   On Wed, Jan 18, 2012 at 2:31 PM,  solip...@pitrou.net wrote:
results for 12de1ad1cee8 on branch default

   
test_capi leaked [2008, 2008, 2008] references, sum=6024
  
   Yikes, you weren't kidding about that new subinterpreter code
   execution test upsetting the refleak detection...
 
  Well, these are real leaks, but I expect them to be quite difficult to
  track (I've found a couple of them), because they can be scattered
  around in C module initialization routines and the like. I suggest we
  skip this test on refleak runs.
 
 
 Do we have any general strategy to help make it more fine-grained to detect
 where the leak might be coming from?

Unfortunately not. I've tried to track down the remaining leaks (*) by
using gc.get_objects(), but apart from a couple of false positives
(dead weakrefs lingering in some tp_subclasses slots until the next
subclasses take their place ;-)), most refleaks seem to be either on
long-lived objects (meaning the leaks are not severe) or on
non-gc-tracked objects.

(*)

$ ./python -m test -R 3:2 test_capi
[1/1] test_capi
beginning 5 repetitions
12345
.
test_capi leaked [152, 152] references, sum=304


 We could then maybe try to get some
 people pound on this at the PyCon sprints. Otherwise I'm reluctant to skip
 it since they are legitimate leaks that should be get fixed.

Well it's the old well-known issue with pseudo-permanent references
not being appropriately managed/cleaned up. Which only shows when
calling Py_Initialize/Py_Finalize multiple times, or using
sub-interpreters.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Brett Cannon
On Wed, Jan 18, 2012 at 11:27, Antoine Pitrou solip...@pitrou.net wrote:

 On Wed, 18 Jan 2012 11:14:50 -0500
 Brett Cannon br...@python.org wrote:

  On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou solip...@pitrou.net
 wrote:
 
   On Thu, 19 Jan 2012 01:06:07 +1000
   Nick Coghlan ncogh...@gmail.com wrote:
On Wed, Jan 18, 2012 at 2:31 PM,  solip...@pitrou.net wrote:
 results for 12de1ad1cee8 on branch default
 

 test_capi leaked [2008, 2008, 2008] references, sum=6024
   
Yikes, you weren't kidding about that new subinterpreter code
execution test upsetting the refleak detection...
  
   Well, these are real leaks, but I expect them to be quite difficult to
   track (I've found a couple of them), because they can be scattered
   around in C module initialization routines and the like. I suggest we
   skip this test on refleak runs.
  
 
  Do we have any general strategy to help make it more fine-grained to
 detect
  where the leak might be coming from?

 Unfortunately not. I've tried to track down the remaining leaks (*) by
 using gc.get_objects(), but apart from a couple of false positives
 (dead weakrefs lingering in some tp_subclasses slots until the next
 subclasses take their place ;-)), most refleaks seem to be either on
 long-lived objects (meaning the leaks are not severe) or on
 non-gc-tracked objects.

 (*)

 $ ./python -m test -R 3:2 test_capi
 [1/1] test_capi
 beginning 5 repetitions
 12345
 .
 test_capi leaked [152, 152] references, sum=304


  We could then maybe try to get some
  people pound on this at the PyCon sprints. Otherwise I'm reluctant to
 skip
  it since they are legitimate leaks that should be get fixed.

 Well it's the old well-known issue with pseudo-permanent references
 not being appropriately managed/cleaned up. Which only shows when
 calling Py_Initialize/Py_Finalize multiple times, or using
 sub-interpreters.


Could we tweak the report to somehow ignore the permanent refcounts for
just this test? If not then we might as well leave it out since that number
will never hit 0.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Antoine Pitrou
On Wed, 18 Jan 2012 11:39:42 -0500
Brett Cannon br...@python.org wrote:
 
   We could then maybe try to get some
   people pound on this at the PyCon sprints. Otherwise I'm reluctant to
  skip
   it since they are legitimate leaks that should be get fixed.
 
  Well it's the old well-known issue with pseudo-permanent references
  not being appropriately managed/cleaned up. Which only shows when
  calling Py_Initialize/Py_Finalize multiple times, or using
  sub-interpreters.
 
 
 Could we tweak the report to somehow ignore the permanent refcounts for
 just this test? If not then we might as well leave it out since that number
 will never hit 0.

I can't think of any way to specifically ignore them (if we knew where
they are we could just fix the refleaks :-)).

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Stephen J. Turnbull
Nick Coghlan writes:

  From the stdlib feature development branch (these are the new interim
  releases with standard library updates only as proposed by PEP 407):
  Python 3.3.1 + stdlib 13.02.0  (~February 2013)
  Python 3.3.2 + stdlib 13.08.0  (~August 2013)
  Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path
  from here is to make the jump to 3.4.0)
  -- 3.4.0 + 12.08.0 is released from default branch --

Typo? - 3.4.0 + 14.08.0, right?

  Python 3.4.1 + stdlib 15.02.0  (~February 2015)

It seems to me there could be considerable divergence between the
stdlib code in

  the default branch:
  Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
  Python 3.4.0a2 + stdlib 14.08.0a2  (~August 2013)
  Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)

and

  the stdlib feature development branch
  Python 3.3.1 + stdlib 13.02.0  (~February 2013)
  Python 3.3.2 + stdlib 13.08.0  (~August 2013)
  Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path

because 14.08.0a* will be targeting 3.4, and *should* use new language
constructs and APIs where they are appropriate, while 13.02.0 ...
14.02.0 will be targeting the 3.3 API, and mustn't use them.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Dirkjan Ochtman
On Tuesday, January 17, 2012, Antoine Pitrou solip...@pitrou.net wrote:
 We would like to propose the following PEP to change (C)Python's release
 cycle. Discussion is welcome, especially from people involved in the
 release process, and maintainers from third-party distributions of
 Python.

As a Gentoo packager, this would mean much more work for us, unless all the
non-LTS releases promised to be backwards compatible. I.e. the hard part
for us is managing all the incompatibilities in other packages,
compatibility with Python.

As a user of Python, I would rather dislike the change from 18 to 24 months
for LTS release cycles. And the limiting factor for my use of Python
features is largely old Python versions still in use, not the availability
of newer features in the newest Python. So I'm much more interested in
finding ways of improving 2.7/3.2 uptake than adding more feature releases.
I also think that it would be sensible to wait with something like this
process change until the 3.x adoption curve is much further along.

Cheers,

Dirkjan




 Regards

 Antoine.


 PEP: 407
 Title: New release cycle and introducing long-term support versions
 Version: $Revision$
 Last-Modified: $Date$
 Author: Antoine Pitrou solip...@pitrou.net,
Georg Brandl ge...@python.org,
Barry Warsaw ba...@python.org
 Status: Draft
 Type: Process
 Content-Type: text/x-rst
 Created: 2012-01-12
 Post-History:
 Resolution: TBD


 Abstract
 

 Finding a release cycle for an open-source project is a delicate
 exercise in managing mutually contradicting constraints: developer
 manpower, availability of release management volunteers, ease of
 maintenance for users and third-party packagers, quick availability of
 new features (and behavioural changes), availability of bug fixes
 without pulling in new features or behavioural changes.

 The current release cycle errs on the conservative side.  It is
 adequate for people who value stability over reactivity.  This PEP is
 an attempt to keep the stability that has become a Python trademark,
 while offering a more fluid release of features, by introducing the
 notion of long-term support versions.


 Scope
 =

 This PEP doesn't try to change the maintenance period or release
 scheme for the 2.7 branch.  Only 3.x versions are considered.


 Proposal
 

 Under the proposed scheme, there would be two kinds of feature
 versions (sometimes dubbed minor versions, for example 3.2 or 3.3):
 normal feature versions and long-term support (LTS) versions.

 Normal feature versions would get either zero or at most one bugfix
 release; the latter only if needed to fix critical issues.  Security
 fix handling for these branches needs to be decided.

 LTS versions would get regular bugfix releases until the next LTS
 version is out.  They then would go into security fixes mode, up to a
 termination date at the release manager's discretion.

 Periodicity
 ---

 A new feature version would be released every X months.  We
 tentatively propose X = 6 months.

 LTS versions would be one out of N feature versions.  We tentatively
 propose N = 4.

 With these figures, a new LTS version would be out every 24 months,
 and remain supported until the next LTS version 24 months later.  This
 is mildly similar to today's 18 months bugfix cycle for every feature
 version.

 Pre-release versions
 

 More frequent feature releases imply a smaller number of disruptive
 changes per release.  Therefore, the number of pre-release builds
 (alphas and betas) can be brought down considerably.  Two alpha builds
 and a single beta build would probably be enough in the regular case.
 The number of release candidates depends, as usual, on the number of
 last-minute fixes before final release.


 Effects
 ===

 Effect on development cycle
 ---

 More feature releases might mean more stress on the development and
 release management teams.  This is quantitatively alleviated by the
 smaller number of pre-release versions; and qualitatively by the
 lesser amount of disruptive changes (meaning less potential for
 breakage).  The shorter feature freeze period (after the first beta
 build until the final release) is easier to accept.  The rush for
 adding features just before feature freeze should also be much
 smaller.

 Effect on bugfix cycle
 --

 The effect on fixing bugs should be minimal with the proposed figures.
 The same number of branches would be simultaneously open for regular
 maintenance (two until 2.x is terminated, then one).

 Effect on workflow
 --

 The workflow for new features would be the same: developers would only
 commit them on the ``default`` branch.

 The workflow for bug fixes would be slightly updated: developers would
 commit bug fixes to the current LTS branch (for example ``3.3``) and
 then merge them into ``default``.

___
Python-Dev 

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-18 Thread Martin v. Löwis
Am 18.01.2012 17:01, schrieb PJ Eby:
 On Tue, Jan 17, 2012 at 7:58 PM, Martin v. Löwis mar...@v.loewis.de
 mailto:mar...@v.loewis.de wrote:
 
 Am 17.01.2012 22:26, schrieb Antoine Pitrou:
  Only 2 bits are used in ob_sstate, meaning 30 are left. These 30 bits
  could cache a hash perturbation computed from the string and the
  random bits:
 
  - hash() would use ob_shash
  - dict_lookup() would use ((ob_shash * 103) ^ (ob_sstate  ~3))
 
  This way, you cache almost all computations, adding only a computation
  and a couple logical ops when looking up a string in a dict.
 
 That's a good idea. For Unicode, it might be best to add another slot
 into the object, even though this increases the object size.
 
 
 Wouldn't that break the ABI in 2.x?

I was thinking about adding the field at the end, so I thought it
shouldn't. However, if somebody inherits from PyUnicodeObject, it still
might - so my new proposal is to add the extra hash into the str block,
either at str[-1], or after the terminating 0. This would cause an
average increase of four bytes of the storage (0 bytes in 50% of the
cases, 8 bytes because of padding in the other 50%).

What do you think?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-18 Thread Martin v. Löwis
Am 18.01.2012 13:30, schrieb Barry Warsaw:
 On Jan 18, 2012, at 08:19 AM, Martin v. Löwis wrote:
 
 My concern is not about breaking doctests: this proposal will also break
 them. My concern is about applications that assume that hash(s) is
 stable across runs, and we do have reports that it will break
 applications.
 
 I am a proponent of doctests, and thus use them heavily.  I can tell you that
 the issue of dict hashing (non-)order has been well known for *years* and I
 have convenience functions in my own doctests to sort and print dict
 elements.

Indeed. So that breakage may actually be less than people expect.

As for cases that still rely on dict order: none of the proposed
solutions preserve full compatibility in dict order. The only solution
(not actually proposed so far) is to add an AVL tree into the hash
table, to track keys that collide on hash values (rather than hash
slots). Such a tree would be only used if there is an actual collision,
which, in practical dict usage, never occurs.

I've been seriously considering implementing a balanced tree inside
the dict (again for string-only dicts, as ordering can't be guaranteed
otherwise). However, this would be a lot of code for a security fix.
It *would* solve the issue for good, though.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Brett Cannon
On Wed, Jan 18, 2012 at 09:08, Nick Coghlan ncogh...@gmail.com wrote:

 On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou solip...@pitrou.net
 wrote:
  Splitting the stdlib:
  - requires someone to do the splitting (highly non-trivial given the
  interactions of some modules with interpreter details or low-level C
  code)
  - requires setting up separate resources (continuous integration with N
  stdlib versions and M interpreter versions, for example)
  - requires separate maintenance and releases for the stdlib (but with
  non-trivial interaction with interpreter maintenance, since they will
  affect each other and must be synchronized for Python to be usable at
  all)
  - requires more attention by users since there are now *two* release
  schedules and independent version numbers to track

 Did you read what I actually proposed? I specifically *didn't* propose
 separate stdlib releases (for all the reasons you point out), only
 separate date based stdlib *versioning*. Distribution of the CPython
 interpreter + stdlib would remain monolithic, as it is today. Any
 given stdlib release would only be supported for the most recent
 language release. The only difference is that between language
 releases, where we currently only release maintenance builds, we'd
 *also* release a second version of each maintenance build with an
 updated standard library, along with an alpha release of the next
 language version (with the last part being entirely optional, but I
 figured I may as well make the suggestion since I like the idea to
 encourage getting syntax updates and the like out for earlier
 experimentation).


 When you initially pitched the proposal via email, you didn't include
 the language moratarium applies to interim releases idea. That one
 additional suggestion makes the whole concept *much* more appealing to
 me, but I only like it on the condition that we decouple the stdlib
 versioning from the language definition versioning (even though I
 recommend we only officially support very specific combinations of the
 two). My suggestion is really just a concrete proposal for
 implementing Ezio's idea of only bumping the Python version for
 releases with long term support, and using some other mechanism to
 distinguish the interim releases.


IOW we would have a language moratorium every 2 years (i.e. between LTS
releases) while switching to a 6 month release cycle for language/VM
bugfixes and full stdlib releases? I would support that as it has several
benefits from several angles.

From a VM perspective, it gives other VMs 2 years to catch up to the next
release instead of 18 months; not a big switch, but still better than
shortening it.

It also makes disruptive language changes less frequent so people have more
time to catch up, update books/docs, etc. We can also let them bake longer
and we all get more experience with them.

Doing a release every 6 months that includes updates to the stdlib and
bugfixes to the language/VM also benefits other VMs by getting
compatibility fixes in faster. All of the other VM maintainers have told me
that keeping the stdlib non-CPython compliant is the biggest hurdle. This
kind of switch means they could release a VM that supports a release 6
months or a year after a language change release (e.g. 1 to 2 releases in)
so as to get changes in faster and lower the need to keep their own fork.

It should also increase the chances of external developers of projects
being willing to become core developers and contributing their project to
Python. If they get to keep a 6 month release cycle we could consider
pulling in project like httplib2 and others that have resisted inclusion in
the stdlib because painfully long (for them) wait between releases.



 So, assuming a 2 year LTS cycle, the released versions up to February
 2015 with my suggestion would end up being:

 From the default branch:
 Python 3.3.0 + stdlib 12.08.0  (~August 2012)
 Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
 Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013)
 Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)
 Python 3.4.0a4 + stdlib 14.08.0a4  (~2014)
 Python 3.4.0b1 + stdlib 14.08.0b1  (~2014)
 Python 3.4.0b2 + stdlib 14.08.0b2  (~2014)
 Python 3.4.0c1 + stdlib 14.08.0c1  (~2014)
 Python 3.4.0 + stdlib 14.08  (~August 2014)
 Python 3.5.0a1 + stdlib 16.08.0a1  (~February 2015)

 From the 3.3 maintenance branch (these are maintenance updates to the
 LTS release):
 Python 3.3.1 + stdlib 12.08.1  (~February 2013)
 Python 3.3.2 + stdlib 12.08.2  (~August 2013)
 Python 3.3.3 + stdlib 12.08.3  (~February 2014)
 Python 3.3.4 + stdlib 12.08.4  (~August 2014) (and 3.3 branch enters
 security patch only mode)

 From the 3.4 maintenance branch (these are maintenance updates to the
 LTS release):
 Python 3.4.1 + stdlib 14.08.1  (~February 2015)

 From the stdlib feature development branch (these are the new interim
 releases with standard library updates only as proposed by PEP 407):
 Python 3.3.1 + 

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Antoine Pitrou

Hello Dirkjan,

On Wed, 18 Jan 2012 18:32:22 +0100
Dirkjan Ochtman dirk...@ochtman.nl wrote:
 On Tuesday, January 17, 2012, Antoine Pitrou solip...@pitrou.net wrote:
  We would like to propose the following PEP to change (C)Python's release
  cycle. Discussion is welcome, especially from people involved in the
  release process, and maintainers from third-party distributions of
  Python.
 
 As a Gentoo packager, this would mean much more work for us, unless all the
 non-LTS releases promised to be backwards compatible. I.e. the hard part
 for us is managing all the incompatibilities in other packages,
 compatibility with Python.

It might need to be spelt clearly in the PEP, but one of my assumptions
is that packagers choose on what release series they want to
synchronize. So packagers can synchronize on the LTS releases if it's
more practical for them, or if it maps better to their own release
model (e.g. Debian).

Do you think that's a valid answer to Gentoo's concerns?

 So I'm much more interested in
 finding ways of improving 2.7/3.2 uptake than adding more feature releases.

That would be nice as well, but I think it's orthogonal to the PEP.
Besides, I'm afraid there's not much we (python-dev) can do about it.
Some vendors (Debian, Redhat) will always lag behind the bleeding-edge
feature releases.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Georg Brandl
Am 18.01.2012 16:25, schrieb Stephen J. Turnbull:
 Antoine Pitrou writes:
 
   You claim people won't use stable releases because of not enough
   alphas?  That sounds completely unrelated.
 
 Surely testing is related to user perceptions of stability.  More
 testing helps reduce bugs in released software, which improves user
 perception of stability, encouraging them to use the software in
 production.  Less testing, then, will have the opposite effect.  But
 you understand that theory, I'm sure.  So what do you mean to say?
 
   (you can produce flimsy software with many alphas, too)
 
 The problem is the converse: can you produce Python-release-quality
 software with much less pre-release testing than current feature
 releases get?

   Sure, and we think it is [possible to do that] :)
 
 Given the relative risk of rejecting PEP 407 and me being wrong (the
 status quo really isn't all that bad AFAICS), vs. accepting PEP 407
 and you being wrong, I don't find a smiley very convincing.

The status quo really isn't all that bad applies to any PEP.  Also,
compared to most PEPs, it is quite easy to revert to the previous
state of things if they don't work out as wanted.

 In fact,
 I don't find the PEP itself convincing -- and I'm not the only one.

That is noted.  And I think Antoine was a little harsh earlier; of
course we also need to convince users that the new cycle is
advantageous and not detrimental.

 We'll see what Barry and Georg have to say.

Two things:

a) The release manager's job is not as bad as you might believe.  We
have an incredibly helpful and active core of developers which means
that the RM job is more or less reduced to pronouncing on changes
during the rc phase, and actually producing the releases.

b) I did not have the impression (maybe someone can underline that
with tracker stats?) that there were a lot more bug reports than
usual during the alpha and early beta stages of Python 3.2.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Georg Brandl
Am 18.01.2012 18:56, schrieb Brett Cannon:

 IOW we would have a language moratorium every 2 years (i.e. between LTS
 releases) while switching to a 6 month release cycle for language/VM bugfixes
 and full stdlib releases?

That is certainly a possibility (it's listed as an open issue in the PEP).

 I would support that as it has several benefits from
 several angles.
 
 From a VM perspective, it gives other VMs 2 years to catch up to the next
 release instead of 18 months; not a big switch, but still better than 
 shortening it.
 
 It also makes disruptive language changes less frequent so people have more 
 time
 to catch up, update books/docs, etc. We can also let them bake longer and we 
 all
 get more experience with them.

Yes.  In the end, the moratorium really was a good idea, and this would be
carrying on the spirit.

 Doing a release every 6 months that includes updates to the stdlib and 
 bugfixes
 to the language/VM also benefits other VMs by getting compatibility fixes in
 faster. All of the other VM maintainers have told me that keeping the stdlib
 non-CPython compliant is the biggest hurdle. This kind of switch means they
 could release a VM that supports a release 6 months or a year after a language
 change release (e.g. 1 to 2 releases in) so as to get changes in faster and
 lower the need to keep their own fork.
 
 It should also increase the chances of external developers of projects being
 willing to become core developers and contributing their project to Python. If
 they get to keep a 6 month release cycle we could consider pulling in project
 like httplib2 and others that have resisted inclusion in the stdlib because
 painfully long (for them) wait between releases.

Exactly!

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-18 Thread Glenn Linderman

On 1/18/2012 9:52 AM, Martin v. Löwis wrote:

I've been seriously considering implementing a balanced tree inside
the dict (again for string-only dicts, as ordering can't be guaranteed
otherwise). However, this would be a lot of code for a security fix.
It*would*  solve the issue for good, though.


To handle keys containing non-orderable keys along with strings, which 
are equally vulnerable to string-only keys, especially if the non-string 
components can have fixed values during an attack, you could simply use 
their hash value as an orderable proxy for the non-orderable key components.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Antoine Pitrou

Well, they should be fixed now :-)

Regards

Antoine.


On Wed, 18 Jan 2012 17:42:15 +0100
Antoine Pitrou solip...@pitrou.net wrote:
 On Wed, 18 Jan 2012 11:39:42 -0500
 Brett Cannon br...@python.org wrote:
  
We could then maybe try to get some
people pound on this at the PyCon sprints. Otherwise I'm reluctant to
   skip
it since they are legitimate leaks that should be get fixed.
  
   Well it's the old well-known issue with pseudo-permanent references
   not being appropriately managed/cleaned up. Which only shows when
   calling Py_Initialize/Py_Finalize multiple times, or using
   sub-interpreters.
  
  
  Could we tweak the report to somehow ignore the permanent refcounts for
  just this test? If not then we might as well leave it out since that number
  will never hit 0.
 
 I can't think of any way to specifically ignore them (if we knew where
 they are we could just fix the refleaks :-)).
 
 Regards
 
 Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread fwierzbi...@gmail.com
On Wed, Jan 18, 2012 at 9:56 AM, Brett Cannon br...@python.org wrote:

 Doing a release every 6 months that includes updates to the stdlib and
 bugfixes to the language/VM also benefits other VMs by getting compatibility
 fixes in faster. All of the other VM maintainers have told me that keeping
 the stdlib non-CPython compliant is the biggest hurdle. This kind of switch
 means they could release a VM that supports a release 6 months or a year
 after a language change release (e.g. 1 to 2 releases in) so as to get
 changes in faster and lower the need to keep their own fork.
As one of the other VM maintainers I agree with everything Brett has
said here. The proposal sounds very good to me from that perspective.

-Frank
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Steven D'Aprano

Antoine Pitrou wrote:

Le jeudi 19 janvier 2012 à 00:25 +0900, Stephen J. Turnbull a écrit :

  You claim people won't use stable releases because of not enough
  alphas?  That sounds completely unrelated.

Surely testing is related to user perceptions of stability.  More
testing helps reduce bugs in released software, which improves user
perception of stability, encouraging them to use the software in
production.


I have asked a practical question, a theoretical answer isn't exactly
what I was waiting for.

[...]

I don't care to convince *you*, since you are not involved in Python
development and release management (you haven't ever been a contributor
AFAIK). Unless you produce practical arguments, saying I don't think
you can do it is plain FUD and certainly not worth answering to.


Pardon me, but people like Stephen Turnbull are *users* of Python, exactly the 
sort of people you DO have to convince that moving to an accelerated or more 
complex release process will result in a better product. The risk is that you 
will lose users, or fragment the user base even more than it is now with 2.x 
vs 3.x.


Quite frankly, I like the simplicity and speed of the current release cycle. 
All this talk about separate LTS releases and parallel language releases and 
library releases makes my head spin. I fear the day that people asking 
questions on the tutor or python-list mailing lists will have to say (e.g.) 
I'm using Python 3.4.1 and standard library 1.2.7 in order to specify the 
version they're using.


I fear change, because the current system works well and for every way to make 
it better there are a thousand ways to make it worse. Dismissing fears like 
this as FUD doesn't do anyone any favours.


One on-going complaint is that Python-Dev doesn't have the manpower or time to 
do everything that needs to be done. Bugs languish for months or years because 
nobody has the time to look at it. Will going to a more rapid release cycle 
give people more time, or just increase their workload? You're hoping that a 
more rapid release cycle will attract more developers, and there is a chance 
that you could be right; but a more rapid release cycle WILL increase the 
total work load. So you're betting that this change will attract enough new 
developers that the work load per person will decrease even as the total work 
load increases. I don't think that's a safe bet.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Steven D'Aprano

Brett Cannon wrote:


And honestly, if we don't go with this I'm with Georg's comment in another
email of beginning to consider stripping the stdlib down to core libraries
to help stop with the bitrot (sorry, Paul). If we can't attract new
replacements for modules we can't ditch because of backwards compatibility
I start to wonder if I should even care about improving the stdlib outside
of core code required to make Python simply function.



Do we have any evidence of this alleged bitrot? I spend a lot of time on the 
comp.lang.python newsgroup and I see no evidence that people using Python 
believe the standard library is rotting from lack of attention.


I do see people having trouble with installing third party packages. I see 
that stripping back the standard library and forcing people to rely more on 
external libraries will hurt, rather than help, the experience they have with 
Python.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Matt Joiner
On Wed, Jan 18, 2012 at 6:55 PM, Georg Brandl g.bra...@gmx.net wrote:
 The main reason is changes in the library.  We have been getting complaints
 about the standard library bitrotting for years now, and one of the main
 reasons it's so hard to a) get decent code into the stdlib and b) keep it
 maintained is that the release cycles are so long.  It's a tough thing for
 contributors to accept that the feature you've just implemented will only
 be in a stable release in 16 months.

 If the stdlib does not get more reactive, it might just as well be cropped
 down to a bare core, because 3rd-party libraries do everything as well and
 do it before we do.  But you're right that if Python came without batteries,
 the current release cycle would be fine.

I think this is the real issue here. The batteries in Python are so
important because:
1) The stability and quality of 3rd party libraries is not guaranteed.
2) The mechanism used to obtain 3rd party libraries, is not popular or
considered reliable.

Much of the bitrot is that standard library modules have been
deprecated by third party ones that are of a much higher
functionality. Rather than importing these libraries, it needs to be
trivial to obtain them.

Putting some of these higher quality 3rd party modules into lock step
with Python is an unpopular move, and hampers their future growth.

From the top of my head, libraries such as LXML, argparse, and
requests are such popular libraries that shouldn't be baked in. In the
long term, it would be nice to see these kinds of libraries dropped
from the standard installation, and made available through the new
distribute package systems etc.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Writable __doc__

2012-01-18 Thread Ethan Furman
Is there a reason why normal classes can't have their __doc__ strings 
rewritten?  Creating a do-nothing metaclass seems like overkill for such 
a simple operation.


Python 3.2 ... on win32
-- class Test():
...   __doc__ = 'am I permanent?'
...
-- Test.__doc__
'am I permanent?'
-- Test.__doc__ = 'yes'
Traceback (most recent call last):
  File stdin, line 1, in module
AttributeError: attribute '__doc__' of 'type' objects is not writable
-- type(Test)
class 'type'

-- class Meta(type):
...   only for exists to allow writable __doc__
...
-- class Test(metaclass=Meta):
...   __doc__ = 'am I permanent?'
...
-- Test.__doc__
'am I permanent?'
-- Test.__doc__ = 'No!'
-- Test.__doc__
'No!'
-- type(Test)
class '__main__.Meta'

Should I create a bug report?

~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Writable __doc__

2012-01-18 Thread Benjamin Peterson
2012/1/18 Ethan Furman et...@stoneleaf.us:
 Is there a reason why normal classes can't have their __doc__ strings
 rewritten?  Creating a do-nothing metaclass seems like overkill for such a
 simple operation.

 Python 3.2 ... on win32
 -- class Test():
 ...   __doc__ = 'am I permanent?'
 ...
 -- Test.__doc__
 'am I permanent?'
 -- Test.__doc__ = 'yes'
 Traceback (most recent call last):
  File stdin, line 1, in module
 AttributeError: attribute '__doc__' of 'type' objects is not writable
 -- type(Test)
 class 'type'

 -- class Meta(type):
 ...   only for exists to allow writable __doc__
 ...
 -- class Test(metaclass=Meta):
 ...   __doc__ = 'am I permanent?'
 ...
 -- Test.__doc__
 'am I permanent?'
 -- Test.__doc__ = 'No!'
 -- Test.__doc__
 'No!'
 -- type(Test)
 class '__main__.Meta'

 Should I create a bug report?

 $ ./python
Python 3.3.0a0 (default:095de2293f39, Jan 18 2012, 10:34:18)
[GCC 4.5.3] on linux
Type help, copyright, credits or license for more information.
 class Test:
... __doc__ = time machine
...
 Test.__doc__ = strikes again
 Test.__doc__
'strikes again'




-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Coroutines and PEP 380

2012-01-18 Thread Matt Joiner
PEP380 and Mark's coroutines could coexist, so I really don't it's
too late matters. Furthermore, PEP380 has utility in its own right
without considering its use for explicit coroutines.

I would like to see these coroutines considered, but as someone else
mentioned, coroutines via PEP380 enhanced generators have some
interesting characteristics, from my experimentations they feel
monadic.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Nick Coghlan
On Thu, Jan 19, 2012 at 7:31 AM, fwierzbi...@gmail.com
fwierzbi...@gmail.com wrote:
 On Wed, Jan 18, 2012 at 9:56 AM, Brett Cannon br...@python.org wrote:

 Doing a release every 6 months that includes updates to the stdlib and
 bugfixes to the language/VM also benefits other VMs by getting compatibility
 fixes in faster. All of the other VM maintainers have told me that keeping
 the stdlib non-CPython compliant is the biggest hurdle. This kind of switch
 means they could release a VM that supports a release 6 months or a year
 after a language change release (e.g. 1 to 2 releases in) so as to get
 changes in faster and lower the need to keep their own fork.
 As one of the other VM maintainers I agree with everything Brett has
 said here. The proposal sounds very good to me from that perspective.

Yes, with the addition of the idea of a PEP 3003 style language change
moratorium for interim releases, I've been converted from an initial
opponent of the idea (since we don't want to give the wider community
whiplash) to a supporter (since some parts of the community,
especially web service developers that deploy to tightly controlled
environments, aren't well served by the standard library's inability
to keep up with externally maintained standards and recommended
development practices).

It means PEP 407 can end up serving two goals:

1. Speeding up the rate of release for the standard library, allowing
enhanced features to be made available to end users sooner.
2. Slowing down (slightly) the rate of release of changes to the core
language and builtins, providing more time for those changes to filter
out through the wider Python ecosystem.

Agreeing with those goals in principle then leaves two key questions
to be addressed:

1. How would we have to update our development practices to make such
a dual versioning scheme feasible?
2. How can we best communicate a new approach to versioning without
unduly confusing developers that have built up certain expectations
about Python's release cycle over the past 20+ years?

For the first point, I think having two active development branches
(one for stdlib updates, one for language updates) will prove to be
absolutely essential. Otherwise all language updates would have to be
landed in the 6 month window between the last stdlib release for a
given language version and the next language release, which seems to
me a crazy way to go about things. As a consequence, I think we'd be
obliged to do something to avoid conflicts on Misc/NEWS (this could be
as simple as splitting it out into NEWS and NEWS_STDLIB, but if we're
restructuring those files anyway, we may also want to do something
about the annoying conflicts between maintenance releases and
development releases).

That then leaves the question of how to best communicate such a change
to the rest of the Python community. This is more a political and
educational question than it is a technical one. A few different
approaches have already been suggested:

1. I believe the PEP currently proposes just taking the no more than
9 limit off the minor version of the language. Feature releases would
just come out every 6 months, with every 4th release flagged as a
language release. This could even be conveyed programmatically by
offering sys.lang_version and sys.lang_version_info attributes
that define the *language* version of a given release - 3.3, 3.4, 3.5
and 3.6 would all have something like sys.lang_version == '3.3', and
then in 3.7 (the next language release) it would be updated to say
sys.lang_version == '3.7'.

This approach would require that some policies (such as the
deprecation cycle) by updated to refer to changes in the language
version (sys.lang_version) rather than change in the stdlib version
(sys.version).

I don't like this scheme because it tries to use one number (the minor
version field) to cover two very different concepts (stdlib updates
and language updates). While technically feasible, this is
unnecessarily obscure and confusing for end users.

2. Brett's alternative proposal is that we switch to using the major
version for language releases and the minor version for stdlib
releases. We would then release 3.3, 3.4, 3.5 and 3.6 at 6 month
intervals, with 4.0 then being released in August 2014 as a new
language version.

Without taking recent history into acount, I actually like this scheme
- it fits well with traditional usage of major.minor.micro version
numbering. However, I'm not confident that the python name will
refer to Python 3 on a majority of systems by 2014 and accessing
Python 4.0 through the python3 name would just be odd.

It also means we lose our ability to signal to the community when we
plan to make a backwards incompatible language release (making the
assumtion that we're never going to want to do that again would be
incredibly naive). On a related note, we'd also be setting ourselves
to have to explain to everyone that no, no, Python 3 - 4 is like
upgrading from Python 3.2 - 3.3, 

Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Nick Coghlan
On Thu, Jan 19, 2012 at 10:19 AM, Steven D'Aprano st...@pearwood.info wrote:
 Brett Cannon wrote:
 Do we have any evidence of this alleged bitrot? I spend a lot of time on the
 comp.lang.python newsgroup and I see no evidence that people using Python
 believe the standard library is rotting from lack of attention.

IMO, it's a problem mainly with network (especially web) protocols and
file formats. It can take the stdlib a long time to catch up with
external developments due to the long release cycle, so people are
often forced to switch to third party libraries that better track the
latest versions of relevant standards (de facto or otherwise).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407 / splitting the stdlib

2012-01-18 Thread Terry Reedy

On 1/18/2012 8:06 PM, Nick Coghlan wrote:

On Thu, Jan 19, 2012 at 10:19 AM, Steven D'Apranost...@pearwood.info  wrote:

Do we have any evidence of this alleged bitrot? I spend a lot of time on the
comp.lang.python newsgroup and I see no evidence that people using Python
believe the standard library is rotting from lack of attention.


IMO, it's a problem mainly with network (especially web) protocols and
file formats. It can take the stdlib a long time to catch up with
external developments due to the long release cycle, so people are
often forced to switch to third party libraries that better track the
latest versions of relevant standards (de facto or otherwise).


Some of those modules are more that 2 years out of date and I guess what 
Brett is saying is that the people interested and able to update them 
will not do so in the stdlib because they want to be able to push out 
feature updates whenever they are needed and available and not be tied 
to a slow release schedule. Morever, since the external standards will 
continue to evolve for the foreseeable future, the need to track them 
more quickly will also continue.


We could relax the ban on new features in micro releases and designate 
such modules as volatile and let them get new features in each x.y.z 
release. In a sense, this would be less drastic than inventing a new 
type of release. Code can require an x.y.z release, as it must if it 
depends on a bug fix not in x.y.0.


I also like the idea of stretching out the alpha release cycle. I would 
like to see 3.3.0a1 appear along with 3.2.3 (in February?). If alpha 
releases are released with all buildbots green, they are as good, at 
least with respect to old features, as a corresponding bugfix release. 
All releases will become more dependable as test coverage improves. 
Again, this idea avoids inventing a new type of release with new release 
designations.


I think one reason people avoid alpha releases is that they so quickly 
become obsolete. If one sat for 3 to 6 months, it might get more 
attention. As for any alpha stigma, we should emphasize that alpha only 
mean not feature frozen.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Writable __doc__

2012-01-18 Thread Eric Snow
On Wed, Jan 18, 2012 at 5:01 PM, Ethan Furman et...@stoneleaf.us wrote:
 Is there a reason why normal classes can't have their __doc__ strings
 rewritten?  Creating a do-nothing metaclass seems like overkill for such a
 simple operation.

 Python 3.2 ... on win32
 -- class Test():
 ...   __doc__ = 'am I permanent?'
 ...
 -- Test.__doc__
 'am I permanent?'
 -- Test.__doc__ = 'yes'
 Traceback (most recent call last):
  File stdin, line 1, in module
 AttributeError: attribute '__doc__' of 'type' objects is not writable
 -- type(Test)
 class 'type'

 -- class Meta(type):
 ...   only for exists to allow writable __doc__
 ...
 -- class Test(metaclass=Meta):
 ...   __doc__ = 'am I permanent?'
 ...
 -- Test.__doc__
 'am I permanent?'
 -- Test.__doc__ = 'No!'
 -- Test.__doc__
 'No!'
 -- type(Test)
 class '__main__.Meta'

 Should I create a bug report?

http://bugs.python.org/issue12773  :)

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Writable __doc__

2012-01-18 Thread Noufal Ibrahim
Sreenivas Reddy T thatiparthysreeni...@gmail.com writes:

 this is happening on python 2.6 too.

 Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
 [GCC 4.4.3] on linux2
 Type help, copyright, credits or license for more information.
 class Test(type):
 ...   __doc__=
   File stdin, line 2
 __doc__=
^
 SyntaxError: invalid syntax
 class Test(type):
 ...   __doc__='asasdas'
 ...



I don't get any syntax errors (Python2.7 and 2.6)

 class Test(object):
...   __doc__ = Something
... 
 
 help(Test)

 class Test(type):
...   __doc__ = something
... 
 help(Test)

 Test.__doc__
'something'

 Test.__doc__='sadfsdff'
 Traceback (most recent call last):
   File stdin, line 1, in module
 AttributeError: attribute '__doc__' of 'type' objects is not writable
 type(Test)
 type 'type'


The __name__, __bases__, __module__, __abstractmethods__, __dict__ and
__doc__ attributes have custom getters and setters in the type object
definition. __doc__ has only a getter. No setter and no deleter.

http://hg.python.org/cpython/file/0b5ce36a7a24/Objects/typeobject.c#l658

That is why you're seeing this. What's the question here?

[...]


-- 
~noufal
http://nibrahim.net.in

May I ask a question?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Writable __doc__

2012-01-18 Thread Noufal Ibrahim
Noufal Ibrahim nou...@nibrahim.net.in writes:

[...]

 That is why you're seeing this. What's the question here?

[...]

My apologies. I didn't read the whole thread. 

-- 
~noufal
http://nibrahim.net.in

Some bird populations soaring down -Headline of an article in Science News, 
page 126, February 20, 1993.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Stephen J. Turnbull
Georg Brandl writes:

  The status quo really isn't all that bad applies to any PEP.  Also,
  compared to most PEPs, it is quite easy to revert to the previous
  state of things if they don't work out as wanted.

That depends on how doesn't work out plays out.  If meeting the
schedule *and* producing a good release regularly is just more work
than expected, of course you're right.

If you stick to the schedule with insufficient resources, and lack of
testing produces a really bad release (or worse, a couple of sorta bad
releases in succession), reverting Python's reputation for stability
is going to be non-trivial.

  a) The release manager's job is not as bad as you might believe.  We
  have an incredibly helpful and active core of developers which means
  that the RM job is more or less reduced to pronouncing on changes
  during the rc phase, and actually producing the releases.

I've done release management and I've been watching Python do release
management since PEP 263; I'm well aware that Python has a truly
excellent process in place, and I regularly recommend studying to
friends interested in improving their own projects' processes.

But I've also (twice) been involved (as RM) in a major revision of RM
procedures, and both times it was a lot more work than anybody
expected.  Finally, the whole point of this exercise is to integrate a
lot more stdlib changes (including whole packages) than in the past on
a much shorter timeline, and to do it repeatedly.  Every six months
still sounds like a long time if you are a leaf project still
working on your changes on your own schedule and chafing at the bit
waiting to get them in to the core project's releases, but it's
actually quite short for the RM.

I'm not against this change (especially since, as Antoine so
graciously pointed out, I'm not going to be actually doing the work in
the foreseeable future), but I do advise that the effort required
seemed to be dramatically underestimated.

  b) I did not have the impression (maybe someone can underline that
  with tracker stats?) that there were a lot more bug reports than
  usual during the alpha and early beta stages of Python 3.2.

Yeah, but the question for Python's stability reputation is were
there more than zero?  Every bug that gets through is a risk.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-18 Thread Stephen J. Turnbull
Steven D'Aprano writes:

  Pardon me, but people like Stephen Turnbull are *users* of Python, exactly 
  the 
  sort of people you DO have to convince that moving to an accelerated or more 
  complex release process will result in a better product.

Well, to be fair, Antoine is right in excluding me from the user base
he's trying to attract (as I understand it).  I do not maintain
products or systems that depend on Python working 99.9% of the
time, and in fact in many of my personal projects I use trunk.

One of the problems with this kind of discussion is that the targets
of the new procedures are not clear in everybody's mind, but all of us
tend to use generic terms like users when we mean to discuss
benefits or costs to a specific class of users.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

2012-01-18 Thread Gregory P. Smith
On Wed, Jan 18, 2012 at 12:50 PM, Antoine Pitrou solip...@pitrou.net wrote:

 Well, they should be fixed now :-)

 Regards

 Antoine.

awesome! :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com