Re: [Python-Dev] Moving Python 2.7 [was: 3.5] on Windows to a new compiler

2014-06-06 Thread Stephen J. Turnbull
Brian Curtin writes:

 > Adding features into 3.x is already not enough of a carrot on the
 > stick for many users. Intentionally leaving 2.7 on a dead compiler is
 > like beating them with the stick.

No, it's like a New Year's resolution to stop self-flagellating, and
handing the whip to the users to use on themselves, or not, as they
choose.

Remember, the users *chose* to remain locked-in to 2.7, hoping that we
would continue to provide support, maybe 2.8.  They had alternatives:
contributing resources (in full-time developer support units!) to the
PSF earmarked for Python 2, porting their dependencies to Python 3,
etc.  All expensive, yes, but eventually they need to pay the price of
support or switching.  Staying with Python 2 was always a bet that
switching would be cheaper in the future, or that they'd have more
resources in the future, or both.  Who knows about the private
resources, but not only does Python 3 acquire more features steadily,
but efforts in core by folks like Ethan, distutils, and Nick (just to
name those I've followed personally), along with steadily and
expanding ports of 3rd party libraries, are quickly making switching
cheaper.

Cheap *enough*?  That's for the users themselves to decide.  So I'm
not arguing against support; this kind of support (*and* the people
who argue that it's worth doing, and then *do* it!) is one reason I
have *no* hesitation in recommending Python (3!) vs. any comparable
language.[1]  But whatever is decided here, we're doing it for pride
or for our own use, not because we owe the users anything.

Footnotes: 
[1]  I don't know enough about languages like Ruby or Perl to say
Python provides strictly better support.  I just can't imagine that it
gets better than this!

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nick Coghlan
On 7 June 2014 01:41, Steve Dower  wrote:
>
> What this means for Python is that C extensions for Python 3.5 and later can 
> be built using any version of MSVC from 14.0 and later. Those who are aware 
> of the current state of affairs where you need to use a matching compiler 
> will hopefully see how big an improvement this will be. It is also likely 
> that other compilers will have an easier time providing compatibility with 
> this new CRT, making it simpler and more reliable to build extensions with 
> LLVM or GCC against an MSVC CPython.

\o/ That's great news.

(I'm assuming that change in policy includes figuring out a solution
to the file descriptor problem, since we determined during the
Stackless 2.8 discussion that file descriptor mismatches were actually
our biggest stumbling block when it came to mixing and matching
different CRT versions in one process)

> Basically, what I am offering to do is:
>
> * Update the files in PCBuild to work with Visual Studio "14"
> * Make any code changes necessary to build with VC14
> * Regularly test the latest Python source with the latest MSVC builds and 
> report issues/suggestions to the MSVC team
> * Keep all changes in a separate (public) repo until early next year when 
> we're getting close to the final VS "14" release
>
> What I am asking anyone else to do is:
>
> * Nothing
>
> Thoughts/comments/concerns?

As long as we're also keeping the VS10 files up to date as a fallback
option, which we will be, since the VS14 work will be in a separate
repo, this sounds like a fine idea to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nick Coghlan
On 7 June 2014 15:05, Donald Stufft  wrote:
> I don’t particularly care too much though, I just think that bumping
> the compiler in a 2.7.Z release is a really bad idea and that either
> of the other two options are massively better.

It is *incredibly* unlikely that backwards compatibility with binary
extensions will be broken within the Python 2.7 series - there's a
reason we said "No" when the Stackless folks were asking about it a
while back. Instead, the toolchain availability problem is currently
being tackled by trying to make suitable build toolchains more readily
available (both the official VS 2008 toolchain and alternative open
source toolchains), and by reducing the reliance on building from
source for end users.

Both of those courses of action are likely to bear fruit. It's only in
the case where those approaches *don't* solve the problem that we'll
need to come back and revisit the question of a compatibility break
for binary extensions - it is, as you say, a really bad idea, and
hence not something we would pursue when there are better options
available (I think a Python 2.8 release would be an *even worse* idea
in terms of souring our relationships with redistributors, but
fortunately, those aren't our only two choices).

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 7, 2014, at 12:58 AM, Nick Coghlan  wrote:

> On 7 June 2014 14:47, Donald Stufft  wrote:
>> On Jun 7, 2014, at 12:41 AM, Nick Coghlan  wrote:
>>> 
>>> Words like "just", or "simple", or "easy" really have no place being
>>> applied to a task where the time required to fully execute it with *no
>>> significant problems* is still measured in years.
>> 
>> How much of that time exists because there were actual significant
>> changes from 2.6 to 2.7 and how much of it would not need to exist
>> if 2.8 was literally 2.7.Z with a new compiler on Windows. IOW is it
>> the *version* number that causes the slow upgrade, or is it the fact
>> that there are enough changes that it can’t be safely applied
>> automatically.
> 
> It's the version number change itself. Python 2.7 was covered by the
> language moratorium, so it consists almost entirely of standard
> library changes, and the porting notes are minimal:
> https://docs.python.org/2/whatsnew/2.7.html#porting-to-python-2-7

I’m not sure I agree, the porting docs only show a subset of changes,
you also have a lot of new stuff like OrderedDict, dict comprehensions,
set literals, argparse, dict views, memory views, etc. AFAIK stable
releases don’t jump versions because all of these new features are
risks, not because a number didn’t change.

I don’t particularly care too much though, I just think that bumping
the compiler in a 2.7.Z release is a really bad idea and that either
of the other two options are massively better.

> 
> We didn't even switch compilers on Windows (both 2.6 and 2.7 use VS 2008).
> 
> I can't think of a better demonstration than the slow pace of the
> Python 2.7 rollout that the challenges with doing a new minor release
> of Python really aren't technical ones at the language level - they're
> technical and administrative challenges in the way the language
> version number interacts with the broader Python ecosystem, especially
> the various redistribution channels.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nick Coghlan
On 7 June 2014 14:47, Donald Stufft  wrote:
> On Jun 7, 2014, at 12:41 AM, Nick Coghlan  wrote:
>>
>> Words like "just", or "simple", or "easy" really have no place being
>> applied to a task where the time required to fully execute it with *no
>> significant problems* is still measured in years.
>
> How much of that time exists because there were actual significant
> changes from 2.6 to 2.7 and how much of it would not need to exist
> if 2.8 was literally 2.7.Z with a new compiler on Windows. IOW is it
> the *version* number that causes the slow upgrade, or is it the fact
> that there are enough changes that it can’t be safely applied
> automatically.

It's the version number change itself. Python 2.7 was covered by the
language moratorium, so it consists almost entirely of standard
library changes, and the porting notes are minimal:
https://docs.python.org/2/whatsnew/2.7.html#porting-to-python-2-7

We didn't even switch compilers on Windows (both 2.6 and 2.7 use VS 2008).

I can't think of a better demonstration than the slow pace of the
Python 2.7 rollout that the challenges with doing a new minor release
of Python really aren't technical ones at the language level - they're
technical and administrative challenges in the way the language
version number interacts with the broader Python ecosystem, especially
the various redistribution channels.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nick Coghlan
On 7 June 2014 14:01, Chris Barker  wrote:
>>
>> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
>> I cannot see why that would be massive undertaking, if changing compiler
>> for 2.7 is neccesary anyway.
>
>
> A reminder that this was brought up a few months ago, as a proposal by the
> stackless team, as they wanted to use a newer compiler for binaries. IIRC,
> there was a pretty resounding "don't do that" from this list. Makes sense to
> me -- we have how many different binaries of 2.7 on how many platforms, with
> how many compilers? Sure, python.org has been nicely consistent about what
> compiler (run time, really) they use to distribute Windows binaries, but the
> python version has NOTHING to do with what compiler is used. (for hat matter
> there is 32 bit and 64 bit 2.7 on Windows ...)

Supported by python-dev? We have two: 32-bit and 64-bit, both
depending on the Microsoft C runtime, and both published as binary
installers on python.org. That's it.

> I think, at the time, it was thought that pip, wheel, and the metadata
> standards should be extended to allow multiple binaries of the same version
> with different compilers to be in the wild. those projects have had bigger
> fish to fry, but maybe it's time to get ahead of the game with that, so we
> can accommodate this change. It's already getting hard to find VS2008
> Express, and building 64 bit extensions is s serious pain.

That was a largely independent discussion, noting that if we come up
with a mechanism for dealing with Linux distro variances, it may also
be useful for dealing with Windows C runtime variances.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 7, 2014, at 12:41 AM, Nick Coghlan  wrote:

> On 7 June 2014 08:43, Sturla Molden  wrote:
>> Brett Cannon  wrote:
>> 
>>> Nope. A new minor release of Python is a massive undertaking which is why
>>> we have saved ourselves the hassle of doing a Python 2.8 or not giving a
>>> clear signal as to when Python 2.x will end as a language.
>> 
>> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
>> I cannot see why that would be massive undertaking, if changing compiler
>> for 2.7 is neccesary anyway.
> 
> It's honestly astonishing the number of people that tell us doing a
> new minor release of Python 2 is easy, and then refuse to believe us
> when we tell them it isn't.
> 
> It's 2014 and Python *2.7*, which was released in *2010*, is STILL
> BEING ROLLED OUT. One part of the rollout that is near & dear to my
> own heart is the fact that Red Hat Enterprise Linux 7 and CentOS 7 are
> still in their respective release candidate phases, and it is the 6 ->
> 7 transition that finally upgrades their system Pythons from 2.6 to
> 2.7. Maya 2014 & MotionBuilder 2014 are also the first versions
> Autodesk are shipping that use 2.7 rather than 2.6 as the scripting
> engine (although my understanding is that Autodesk don't guarantee
> compatibility with Python C extensions that aren't built specifically
> for use with their products, so they already use a newer C runtime on
> Windows than we do).
> 
> And once those two dominoes fall, then there'll be some additional
> follow on upgrade work in some parts of the developer community as the
> *users* that receive their Python through those channels rather than
> directly from upstream switch from 2.6 to 2.7 and stumble over the
> small compatibility breaks between those two releases.
> 
> Words like "just", or "simple", or "easy" really have no place being
> applied to a task where the time required to fully execute it with *no
> significant problems* is still measured in years.

How much of that time exists because there were actual significant
changes from 2.6 to 2.7 and how much of it would not need to exist
if 2.8 was literally 2.7.Z with a new compiler on Windows. IOW is it
the *version* number that causes the slow upgrade, or is it the fact
that there are enough changes that it can’t be safely applied
automatically.

> 
> That said, there are definitely problems with toolchain availability
> on Windows for Python 2, and it isn't clear yet how that will be
> addressed in the long run. Steve is working on ensuring the official
> toolchain and C runtime binaries are more readily available from MS.
> Other folks are independently looking into ensuring that open source
> toolchains (like mingw) can be used effectively to at least build
> Python C extensions for Windows (and ironing out some of the glitches
> with that approach that others have mentioned). The Python Packaging
> Authority are continuing to work on the wheel based infrastructure to
> help avoid end users having to compile anything in the first place,
> and redistributors like ActiveState, Enthought & Continuum Analytics
> also make it possible for many end users to just ignore these upstream
> concerns. An extension compatibility break would be an absolutely last
> resort, pursued only if all other attempts at resolving the challenges
> have demonstrably failed - even at the best of times it can take
> months for C extension authors to start publishing compatible binaries
> for a new minor release, so we'd have to assume that time would be
> even longer for a Python 2.7 maintenance release, if they published
> updated binaries at all.
> 
> Regards,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nick Coghlan
On 7 June 2014 08:43, Sturla Molden  wrote:
> Brett Cannon  wrote:
>
>> Nope. A new minor release of Python is a massive undertaking which is why
>> we have saved ourselves the hassle of doing a Python 2.8 or not giving a
>> clear signal as to when Python 2.x will end as a language.
>
> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
> I cannot see why that would be massive undertaking, if changing compiler
> for 2.7 is neccesary anyway.

It's honestly astonishing the number of people that tell us doing a
new minor release of Python 2 is easy, and then refuse to believe us
when we tell them it isn't.

It's 2014 and Python *2.7*, which was released in *2010*, is STILL
BEING ROLLED OUT. One part of the rollout that is near & dear to my
own heart is the fact that Red Hat Enterprise Linux 7 and CentOS 7 are
still in their respective release candidate phases, and it is the 6 ->
7 transition that finally upgrades their system Pythons from 2.6 to
2.7. Maya 2014 & MotionBuilder 2014 are also the first versions
Autodesk are shipping that use 2.7 rather than 2.6 as the scripting
engine (although my understanding is that Autodesk don't guarantee
compatibility with Python C extensions that aren't built specifically
for use with their products, so they already use a newer C runtime on
Windows than we do).

And once those two dominoes fall, then there'll be some additional
follow on upgrade work in some parts of the developer community as the
*users* that receive their Python through those channels rather than
directly from upstream switch from 2.6 to 2.7 and stumble over the
small compatibility breaks between those two releases.

Words like "just", or "simple", or "easy" really have no place being
applied to a task where the time required to fully execute it with *no
significant problems* is still measured in years.

That said, there are definitely problems with toolchain availability
on Windows for Python 2, and it isn't clear yet how that will be
addressed in the long run. Steve is working on ensuring the official
toolchain and C runtime binaries are more readily available from MS.
Other folks are independently looking into ensuring that open source
toolchains (like mingw) can be used effectively to at least build
Python C extensions for Windows (and ironing out some of the glitches
with that approach that others have mentioned). The Python Packaging
Authority are continuing to work on the wheel based infrastructure to
help avoid end users having to compile anything in the first place,
and redistributors like ActiveState, Enthought & Continuum Analytics
also make it possible for many end users to just ignore these upstream
concerns. An extension compatibility break would be an absolutely last
resort, pursued only if all other attempts at resolving the challenges
have demonstrably failed - even at the best of times it can take
months for C extension authors to start publishing compatible binaries
for a new minor release, so we'd have to assume that time would be
even longer for a Python 2.7 maintenance release, if they published
updated binaries at all.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Barker
>
>
> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
> I cannot see why that would be massive undertaking, if changing compiler
> for 2.7 is neccesary anyway.
>

A reminder that this was brought up a few months ago, as a proposal by the
stackless team, as they wanted to use a newer compiler for binaries. IIRC,
there was a pretty resounding "don't do that" from this list. Makes sense
to me -- we have how many different binaries of 2.7 on how many platforms,
with how many compilers? Sure, python.org has been nicely consistent about
what compiler (run time, really) they use to distribute Windows binaries,
but the python version has NOTHING to do with what compiler is used. (for
hat matter there is 32 bit and 64 bit 2.7 on Windows ...)

I think, at the time, it was thought that pip, wheel, and the metadata
standards should be extended to allow multiple binaries of the same version
with different compilers to be in the wild. those projects have had bigger
fish to fry, but maybe it's time to get ahead of the game with that, so we
can accommodate this change. It's already getting hard to find VS2008
Express, and building 64 bit extensions is s serious pain.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Terry Reedy

On 6/6/2014 9:13 PM, Donald Stufft wrote:


On Jun 6, 2014, at 9:05 PM, Terry Reedy  wrote:



If you are suggesting that a Windows compiler change should be
invisible to non-Windows users, I agree.

Let us assume that /pcbuild remains for those who have vc2008 and
that /pcbuild14 is added (and everything else remains as is). Then
the only other thing that would change is the Windows installer
released on Python.org. Call than 2.7.9W or whatever on the
download site and interactive startup message to signal that
something is different.



How are packaging tools supposed to cope with this? AFAIK there is
nothing in most of them to deal with a X.Y.Z release suddenly dealing
with a different compiler.


For this option, packaging tools on Windows would have to gain a special 
rule to cope with a special, hopefully unique, not-to-be-repeated, 
series of releases. If VC2008 ceases to become available to those who do 
not already have it, and who machines do not break or get replaced, 
dealing with a different easily available compiler would be easier than 
dealing with having no compiler.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Sturla Molden
Nathaniel Smith  wrote:

>> with numpy.accelerate:
>> x = 
>> y = 
>> z = 
>> # evaluation of x,y,z happens here
> 
> Using an alternative evaluation engine is indeed another way to
> optimize execution, which is why projects like numexpr, numba, theano,
> etc. exist. But this is basically switching to a different language in
> a different VM.

I was not thinking that complicated. Let us focus on what an unmodified
CPython can do.

A compound expression with arrays can also be seen as a pipeline. Imagine
what would happen if in "NumPy 2.0" arithmetic operators returned
coroutines instead of temporary arrays. That way an expression could be
evaluated chunkwise, and the chunks would be small enough to fit in cache.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Sturla Molden
Greg Ewing  wrote:
> Julian Taylor wrote:
>> tp_can_elide receives two objects and returns one of three values:
>> * can work inplace, operation is associative
>> * can work inplace but not associative
>> * cannot work inplace
> 
> Does it really need to be that complicated? Isn't it
> sufficient just to ask the object potentially being
> overwritten whether it's okay to overwrite it?

How can it know this without help from the interpreter? 

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Sturla Molden
Brian Curtin  wrote:

>> If Python 2.7 users are left with a dead compiler on Windows, they will
>> find a solution. For example, Enthought is already bundling their Python
>> distribution with gcc 2.8.1 on Windows.
> 
> Again, not something I think we should depend on. A lot of people use
> python.org installers.

I am not talking about changing the python.org installers. Let it remain on
VS2008 for Python 2.7. I am only suggesting we make it easier to find a
free C compiler compatible with the python.org installers. 

The NumPy/SciPy dev team have taken the burden to build a MinGW toolchain
that is configured to be 100 % ABI compatible with the python.org
installer. I am only suggesting a link to it or something like that,
perhaps even host it as a separate download. (It is GPL, so anyone can do
that.) That way it would be easy to find a compatible C compiler. We have
to consider that VS2008 will be unobtainable abandonware long before the
promised Python 2.7 support expires. When that happens, users of Python 2.7
will need to find another compiler to build C extensions. If Python.org
makes this easier it would hurt less to have Python 2.7 remain on VS2008
forever.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 6, 2014, at 9:05 PM, Terry Reedy  wrote:

> On 6/6/2014 6:47 PM, Nathaniel Smith wrote:
>> On Fri, Jun 6, 2014 at 11:43 PM, Sturla Molden  
>> wrote:
>>> Brett Cannon  wrote:
>>> 
 Nope. A new minor release of Python is a massive undertaking which is why
 we have saved ourselves the hassle of doing a Python 2.8 or not giving a
 clear signal as to when Python 2.x will end as a language.
>>> 
>>> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
>>> I cannot see why that would be massive undertaking, if changing compiler
>>> for 2.7 is neccesary anyway.
>> 
>> This would require recompiling all packages on OS X and Linux, even
>> though nothing had changed.
> 
> If you are suggesting that a Windows compiler change should be invisible to 
> non-Windows users, I agree.
> 
> Let us assume that /pcbuild remains for those who have vc2008 and that 
> /pcbuild14 is added (and everything else remains as is). Then the only other 
> thing that would change is the Windows installer released on Python.org. Call 
> than 2.7.9W or whatever on the download site and interactive startup message 
> to signal that something is different.
> 
> -- 
> Terry Jan Reedy
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

How are packaging tools supposed to cope with this? AFAIK there is nothing in 
most of them to deal with a X.Y.Z release suddenly dealing with a different 
compiler.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Jun 6, 2014 6:33 PM, "Sturla Molden"  wrote:
>
> Brian Curtin  wrote:
>
> > Well we're certainly not going to assume such a thing. I know people do
> > that, but many don't (I never have).
>
> If Python 2.7 users are left with a dead compiler on Windows, they will
> find a solution. For example, Enthought is already bundling their Python
> distribution with gcc 2.8.1 on Windows.

Again, not something I think we should depend on. A lot of people use
python.org installers.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Terry Reedy

On 6/6/2014 6:47 PM, Nathaniel Smith wrote:

On Fri, Jun 6, 2014 at 11:43 PM, Sturla Molden  wrote:

Brett Cannon  wrote:


Nope. A new minor release of Python is a massive undertaking which is why
we have saved ourselves the hassle of doing a Python 2.8 or not giving a
clear signal as to when Python 2.x will end as a language.


Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
I cannot see why that would be massive undertaking, if changing compiler
for 2.7 is neccesary anyway.


This would require recompiling all packages on OS X and Linux, even
though nothing had changed.


If you are suggesting that a Windows compiler change should be invisible 
to non-Windows users, I agree.


Let us assume that /pcbuild remains for those who have vc2008 and that 
/pcbuild14 is added (and everything else remains as is). Then the only 
other thing that would change is the Windows installer released on 
Python.org. Call than 2.7.9W or whatever on the download site and 
interactive startup message to signal that something is different.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Nick Coghlan
On 7 Jun 2014 00:53, "Paul Sokolovsky"  wrote:
>
> Yes. Except for one small detail - Python3 specifies these code points
> to be Unicode code points. And Unicode is a very bloated thing.

I rather suspect users of East Asian & African scripts might have a
different notion of what constitutes "bloated" vs "can actually represent
this language properly, unlike 8-bit code spaces".

> But if we drop that "Unicode" stipulation, then it's also exactly what
> MicroPython implements. Its "str" type consists of codepoints, we don't
> have pet names for them yet, like Unicode does, but their numeric
> values are 0-255. Note that it in no way limits encodings, characters,
> or scripts which can be used with MicroPython, because just like
> Unicode, it support concept of "surrogate pairs" (but we don't call it
> like that) - specifically, smaller code points may comprise bigger
> groupings. But unlike Unicode, we don't stipulate format, value or
> other constraints on how these "surrogate pairs"-alikes are formed,
> leaving that to users.

This is effectively what the Python 2 str type does, and it's a recipe for
data driven latent defects. You inevitably end up concatenating strings
using different code spaces, or else splitting strings between surrogate
pairs rather than on the proper boundaries, etc.

The abstraction presented to users by the str type *must* be the full range
of Unicode code points as atomic units. Storing those internally as UTF-8
rather than as fixed width code points as CPython does is an experiment
worth trying, since you don't have the same C level backwards compatibility
constraints we do. But limiting the str type to a single code page per
process is not an acceptable constraint in a Python 3 implementation.

Regards,
Nick.

>
>
> --
> Best regards,
>  Paul  mailto:pmis...@gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Greg Ewing

Julian Taylor wrote:

tp_can_elide receives two objects and returns one of three values:
* can work inplace, operation is associative
* can work inplace but not associative
* cannot work inplace


Does it really need to be that complicated? Isn't it
sufficient just to ask the object potentially being
overwritten whether it's okay to overwrite it?
I.e. a parameterless method returning a boolean.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Sturla Molden
Eli Bendersky  wrote:

> While we're at it, Clang in nearing a stage where it can compile C and C++
> on Windows *with ABI-compatibility to MSVC* (yes, even C++) -- see
>  href="http://clang.llvm.org/docs/MSVCCompatibility.html";>http://clang.llvm.org/docs/MSVCCompatibility.html
> for more details. Could
> this help?

Possibly. "cl-clang" is exciting and I hope distutils will support it one
day. Clang is not well known among Windows users as it is among users of
"Unix" (Apple, Linux, FreeBSD, et al.) It would be even better if Python
were bundled with Clang on Windows. 

The MinGW-based "SciPy toolchain" has ABI compatibility with MSVC only for
C (and Fortran), not C++. Differences from vanilla MinGW is mainly static
linkage of the MinGW runtime, different stack alignment (4 bytes instead of
16), and it links with msvcr91.dll instead of msvcrt.dll. 

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Eli Bendersky
On Fri, Jun 6, 2014 at 4:32 PM, Sturla Molden 
wrote:

> Brian Curtin  wrote:
>
> > Well we're certainly not going to assume such a thing. I know people do
> > that, but many don't (I never have).
>
> If Python 2.7 users are left with a dead compiler on Windows, they will
> find a solution. For example, Enthought is already bundling their Python
> distribution with gcc 2.8.1 on Windows.
>

While we're at it, Clang in nearing a stage where it can compile C and C++
on Windows *with ABI-compatibility to MSVC* (yes, even C++) -- see
http://clang.llvm.org/docs/MSVCCompatibility.html for more details. Could
this help?

Eli
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Nathaniel Smith
On Fri, Jun 6, 2014 at 11:33 PM, Sturla Molden  wrote:
> Nathaniel Smith  wrote:
>
>> The proposal in my initial email requires zero pthreads, and is
>> substantially more effective. (Your proposal reduces only the alloc
>> overhead for large arrays; mine reduces both alloc and memory access
>> overhead for boyh large and small arrays.)
>
> My suggestion prevents the kernel from zeroing pages in the middle of a
> computation, which is an important part. It would also be an optimiation
> the Python interpreter could benefit from indepently of NumPy, by allowing
> reuse of allocated memory pages within CPU bound portions of the Python
> code. And no, the method I suggested does not only work for large arrays.

Small allocations are already recycled within process and don't touch
the kernel, so your method doesn't affect them at all.

My guess is that PyMalloc is unlikely to start spawning background
threads any time soon, but if you'd like to propose it maybe you
should start a new thread for that?

> If we really want to take out the memory access overhead, we need to
> consider lazy evaluation. E.g. a context manager that collects a symbolic
> expression and triggers evaluation on exit:
>
> with numpy.accelerate:
> x = 
> y = 
> z = 
> # evaluation of x,y,z happens here

Using an alternative evaluation engine is indeed another way to
optimize execution, which is why projects like numexpr, numba, theano,
etc. exist. But this is basically switching to a different language in
a different VM. I think people will keep running plain-old-CPython
code for some time yet, and the point of this thread is that there's
some low-hanging fruit for making all *that* code faster.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nathaniel Smith
On Fri, Jun 6, 2014 at 11:43 PM, Sturla Molden  wrote:
> Brett Cannon  wrote:
>
>> Nope. A new minor release of Python is a massive undertaking which is why
>> we have saved ourselves the hassle of doing a Python 2.8 or not giving a
>> clear signal as to when Python 2.x will end as a language.
>
> Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
> I cannot see why that would be massive undertaking, if changing compiler
> for 2.7 is neccesary anyway.

This would require recompiling all packages on OS X and Linux, even
though nothing had changed.

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Sturla Molden
Brian Curtin  wrote:

> Well we're certainly not going to assume such a thing. I know people do
> that, but many don't (I never have).

If Python 2.7 users are left with a dead compiler on Windows, they will
find a solution. For example, Enthought is already bundling their Python
distribution with gcc 2.8.1 on Windows.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Jun 6, 2014 6:01 PM, "Sturla Molden"  wrote:
>
> Brian Curtin  wrote:
>
> > Adding features into 3.x is already not enough of a carrot on the
> > stick for many users. Intentionally leaving 2.7 on a dead compiler is
> > like beating them with the stick.
>
> Those who want to build extensions on Windows will just use MinGW
> (currently GCC 2.8.2) instead.

Well we're certainly not going to assume such a thing. I know people do
that, but many don't (I never have).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Steve Dower
Martin v. Löwis wrote:
> Am 06.06.14 22:13, schrieb Paul Moore:
>> From
>> http://www.visualstudio.com/en-us/downloads/visual-studio-14-ctp-vs
>>
>> """
>> Currently, Visual Studio "14" CTPs have known compatibility issues
>> with previous releases of Visual Studio and should not be installed
>> side-by-side on the same computer.
>> """
> 
> I also found
> 
> http://support.microsoft.com/kb/2967191
> 
> which is more specific about this issue:
> 
> '''There are known issues when you install Visual Studio "14" CTP
> 14.0.21730.1 DP on the same computer as Visual Studio 2013. While we expect 
> that
> an uninstallation of Visual Studio "14" and then a repair of Visual Studio 
> 2013
> should fix these issues, our safest recommendation is to install Visual Studio
> "14" in a VM, a VHD, a fresh computer, or another non-production test-only
> computer that does not have Visual Studio 2013 on it. All of these Visual 
> Studio
> side-by-side issues are expected to be fixed soon.

Somebody ran a test to see how well the install/uninstall/repair scenario 
works, and it isn't that great. There are a lot of teams who contribute to 
Visual Studio, and not all of them have updated their installers yet (my team 
included...). Unfortunately, it all happened too close to the release to fix it 
for this version, hence the recommendation.

Eventually, VS 14 will be safe to install side-by-side with earlier versions. 
Chances are it is safe enough with VS 2010 or VS 2012 - it's the 
one-version-prior that's causing the most trouble.

> There is an installation block in this Visual Studio "14" CTP that will 
> prevent
> installation on a computer where an earlier version of Visual Studio is 
> already
> installed. To disable the block that will put the computer in an 
> un-recommended
> state, add the value "BlockerOverride" to the registry:
> HKLM\SOFTWARE\Microsoft\DevDiv\vs\Servicing'''
> 
> So it seems to me that switching to VS 14 at this point in time is not 
> possible.
> 
> Of course, Steve could certainly maintain a Mercurial clone in his 
> hg.python.org
> sandbox that has all the necessary changes done, so people won't have to redo
> the porting over and over.

That's what I had in mind.

[Earlier post]
> 1. what is the availability of the compiler during the testing phase,
>and what will it be immediately after the testing ends (where
>traditionally people would have to buy licenses, or wait for VS
>Express to be released)?

It's freely available now as part of Visual Studio, and all the pre-release 
releases will include everything. The last release (RC or whatever they decide 
to call it this time) should have a go-live license, though it will also be 
time bombed. I believe Express will be released at the same time as the paid 
versions.

> 2. what is the risk of installing a beta compiler on what might
>otherwise be a "production" developer system? In particular, could
>it interfere with other VS installations, and could it require a
>complete system reinstall when the final release of VC 14 is
>available?

Answered above. It's as risky as it always is, though as I mentioned, VC 14 may 
well be fine against VC 10.

Build-to-build upgrades may not be supported between pre-release versions, but 
typically RC to RTM upgrades are supported.

> 3. what is the chance of the final release being delayed beyond
>the planned release date of Python 3.5? Microsoft has a bad
>track record of meeting release dates (or the tradition of not
>announcing any for that reason); the blog says that it will
>be available "sometime in 2015". Now, Python 3.5 might appear
>November 2015, so what do we do if VS 2015 is not released
>by the time 3.5b1 is planned?

We keep the VS 2010 files around and make sure they keep working. This is the 
biggest risk of the whole plan, but I believe that there's enough of a gap 
between when VS 14 is planned to release (which I know, but can't share) and 
when Python 3.5 is planned (which I don't know, but have a semi-informed guess).

Is Python 3.5b1 being built with VS 14 RC (hypothetically) a blocking issue? Do 
we need to resolve that now or can it wait until it happens?

> Regards,
> Martin

Cheers,
Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Sturla Molden
Brian Curtin  wrote:

> Adding features into 3.x is already not enough of a carrot on the
> stick for many users. Intentionally leaving 2.7 on a dead compiler is
> like beating them with the stick.

Those who want to build extensions on Windows will just use MinGW
(currently GCC 2.8.2) instead. 

NumPy and SciPy are planning a switch to a GCC based toolchain with static
linkage of the MinGW runtime on Windows.  It is carefully configured to be
binary compatible with VS2008 on Python 2.7. The major reason for this is
to use gfortran also on Windows. But the result will be a GCC based
toolchain that anyone can use to build extensions on Windows.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Sturla Molden
Brett Cannon  wrote:

> Nope. A new minor release of Python is a massive undertaking which is why
> we have saved ourselves the hassle of doing a Python 2.8 or not giving a
> clear signal as to when Python 2.x will end as a language.

Why not just define Python 2.8 as Python 2.7 except with a newer compiler?
I cannot see why that would be massive undertaking, if changing compiler
for 2.7 is neccesary anyway.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Sturla Molden
Nathaniel Smith  wrote:

> The proposal in my initial email requires zero pthreads, and is
> substantially more effective. (Your proposal reduces only the alloc
> overhead for large arrays; mine reduces both alloc and memory access
> overhead for boyh large and small arrays.)

My suggestion prevents the kernel from zeroing pages in the middle of a
computation, which is an important part. It would also be an optimiation
the Python interpreter could benefit from indepently of NumPy, by allowing
reuse of allocated memory pages within CPU bound portions of the Python
code. And no, the method I suggested does not only work for large arrays.

If we really want to take out the memory access overhead, we need to
consider lazy evaluation. E.g. a context manager that collects a symbolic
expression and triggers evaluation on exit:

with numpy.accelerate:
x = 
y = 
z = 
# evaluation of x,y,z happens here

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Tim Delaney
On 7 June 2014 00:52, Paul Sokolovsky  wrote:

> > At heart, this is exactly what the Python 3 "str" type is. The
> > universal convention is "code points".
>
> Yes. Except for one small detail - Python3 specifies these code points
> to be Unicode code points. And Unicode is a very bloated thing.
>
> But if we drop that "Unicode" stipulation, then it's also exactly what
> MicroPython implements. Its "str" type consists of codepoints, we don't
> have pet names for them yet, like Unicode does, but their numeric
> values are 0-255. Note that it in no way limits encodings, characters,
> or scripts which can be used with MicroPython, because just like
> Unicode, it support concept of "surrogate pairs" (but we don't call it
> like that) - specifically, smaller code points may comprise bigger
> groupings. But unlike Unicode, we don't stipulate format, value or
> other constraints on how these "surrogate pairs"-alikes are formed,
> leaving that to users.


I think you've missed my point.

There is absolutely nothing conceptually bloaty about what a Python 3
string is. It's just like a 7-bit ASCII string, except each entry can be
from a larger table. When you index into a Python 3 string, you get back
exactly *one valid entry* from the Unicode code point table. That plus the
length of the string, plus the guarantee of immutability gives everything
needed to layer the rest of the string functionality on top.

There are no surrogate pairs - each code point is standalone (unlike code
*units*). It is conceptually very simple. The implementation may be
difficult (if you're trying to do better than 4 bytes per code point) but
the concept is dead simple.

If the MicroPython string type requires people *using* it to deal with
surrogates (i.e. indexing could return a value that is not a valid Unicode
code point) then it will have broken the conceptual simplicity of the
Python 3 string type (and most certainly can't be considered in any way
compatible).

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 11:42 PM,   wrote:
> On Sat, Jun 07, 2014 at 05:33:45AM +1000, Chris Angelico wrote:
>
>> > Is it really any difference in maintenance if you just stop applying
>> > updates to 2.7 and switch to 2.8? If 2.8 is really just 2.7 with a
>> > new compiler then there should be no functional difference between
>> > doing that and doing a 2.7.whatever except all of the tooling that
>> > relies on the compiler not to change in micro releases won’t
>> > suddenly break and freak out.
>
>> If the only difference between 2.7 and 2.8 is the compiler used on
>> Windows, what happens on Linux and other platforms? A Python 2.8 would
>> have to be materially different from Python 2.7, not just binarily
>> incompatible on one platform.
>
> Grrmph, that's fair. Perhaps a final alternative is simply continuing
> the 2.7 series with a stale compiler, as a kind of carrot on a stick to
> encourage users to upgrade? Gating 2.7 life on the natural decline of
> its supported compiler/related ecosystem seems somehow quite a gradual
> and natural demise.. :)

Adding features into 3.x is already not enough of a carrot on the
stick for many users. Intentionally leaving 2.7 on a dead compiler is
like beating them with the stick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Jurko Gospodnetić

  Hi.

On 6.6.2014. 21:46, Guido van Rossum wrote:

A reminder:
https://lh5.googleusercontent.com/-d4rF0qJPskQ/U0qpNjP5GoI/PW0/4RF_7zy3esY/w1118-h629-no/Python28.jpg


  *ROFL*

  Subtle, ain't he? *gdr*

  Best regards,
Jurko Gospodnetić


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Martin v. Löwis
Am 06.06.14 22:13, schrieb Paul Moore:
> From http://www.visualstudio.com/en-us/downloads/visual-studio-14-ctp-vs
> 
> """
> Currently, Visual Studio "14" CTPs have known compatibility issues
> with previous releases of Visual Studio and should not be installed
> side-by-side on the same computer.
> """

I also found

http://support.microsoft.com/kb/2967191

which is more specific about this issue:

'''There are known issues when you install Visual Studio "14" CTP
14.0.21730.1 DP on the same computer as Visual Studio 2013. While we
expect that an uninstallation of Visual Studio "14" and then a repair of
Visual Studio 2013 should fix these issues, our safest recommendation is
to install Visual Studio "14" in a VM, a VHD, a fresh computer, or
another non-production test-only computer that does not have Visual
Studio 2013 on it. All of these Visual Studio side-by-side issues are
expected to be fixed soon.

There is an installation block in this Visual Studio "14" CTP that will
prevent installation on a computer where an earlier version of Visual
Studio is already installed. To disable the block that will put the
computer in an un-recommended state, add the value "BlockerOverride" to
the registry:
HKLM\SOFTWARE\Microsoft\DevDiv\vs\Servicing'''

So it seems to me that switching to VS 14 at this point in time is
not possible.

Of course, Steve could certainly maintain a Mercurial clone
in his hg.python.org sandbox that has all the necessary changes
done, so people won't have to redo the porting over and over.

Regards,
Marti

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Angelico
On Sat, Jun 7, 2014 at 5:42 AM,   wrote:
> Perhaps a final alternative is simply continuing
> the 2.7 series with a stale compiler, as a kind of carrot on a stick to
> encourage users to upgrade?

More likely, what would happen is that there'd be an alternate
distribution of Python 2.7 (eg ActiveState), which would be
language-compatible with python.org 2.7, but built with a different
compiler, and therefore unable to use extensions built for
python.org's 2.7. One way or another, pain will happen.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread dw+python-dev
On Sat, Jun 07, 2014 at 05:33:45AM +1000, Chris Angelico wrote:

> > Is it really any difference in maintenance if you just stop applying
> > updates to 2.7 and switch to 2.8? If 2.8 is really just 2.7 with a
> > new compiler then there should be no functional difference between
> > doing that and doing a 2.7.whatever except all of the tooling that
> > relies on the compiler not to change in micro releases won’t
> > suddenly break and freak out.

> If the only difference between 2.7 and 2.8 is the compiler used on
> Windows, what happens on Linux and other platforms? A Python 2.8 would
> have to be materially different from Python 2.7, not just binarily
> incompatible on one platform.

Grrmph, that's fair. Perhaps a final alternative is simply continuing
the 2.7 series with a stale compiler, as a kind of carrot on a stick to
encourage users to upgrade? Gating 2.7 life on the natural decline of
its supported compiler/related ecosystem seems somehow quite a gradual
and natural demise.. :)


David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Martin v. Löwis
Am 06.06.14 21:20, schrieb "Martin v. Löwis":
> 2. what is the risk of installing a beta compiler on what might
>otherwise be a "production" developer system? In particular, could
>it interfere with other VS installations, and could it require a
>complete system reinstall when the final release of VC 14 is
>available?

I found an official answer here:

http://www.visualstudio.com/en-us/downloads/visual-studio-14-ctp-vs

"Installing a CTP release will place a computer in an unsupported state.
For that reason, we recommend only installing CTP releases in a virtual
machine, or on a computer that is available for reformatting."

So there is no promise that you will not need to reformat the system
during the evolution of the compiler.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Paul Moore
On 6 June 2014 20:20, "Martin v. Löwis"  wrote:
> 2. what is the risk of installing a beta compiler on what might
>otherwise be a "production" developer system? In particular, could
>it interfere with other VS installations, and could it require a
>complete system reinstall when the final release of VC 14 is
>available?

From http://www.visualstudio.com/en-us/downloads/visual-studio-14-ctp-vs

"""
Currently, Visual Studio "14" CTPs have known compatibility issues
with previous releases of Visual Studio and should not be installed
side-by-side on the same computer.
"""

It also states that installing the CTP on a PC puts that PC into
"Unsupported" state.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Guido van Rossum
A reminder:
https://lh5.googleusercontent.com/-d4rF0qJPskQ/U0qpNjP5GoI/PW0/4RF_7zy3esY/w1118-h629-no/Python28.jpg

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Angelico
On Sat, Jun 7, 2014 at 5:36 AM, Donald Stufft  wrote:
> Well it’d contain bug fixes and whatever other sorts of things you’d put
> into a 2.7.whatever release. So they’d still want to upgrade to 2.8 since
> that’ll have bug fixes.

But it's not a potentially-breaking change. For example, on Debian
Wheezy, there are a huge number of packages that depend on "python (<<
2.8)", because they expect Python 2.7 and *not* Python 2.8. A newer
version 2.7 will satisfy that; a version 2.8 won't, because it's
entirely possible that 2.8 will have something that's significantly
different. That's what version numbers mean; Python follows the
standard three-part convention, where you upgrade automatically only
within the last part of the number.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Martin v. Löwis
Am 06.06.14 20:25, schrieb Brian Curtin:
> We're going to have to change it at some point, otherwise we're going
> to have people in 2018 scrambling to find VS2008, which will be 35
> versions too old by then.

Not sure whether you picked 2018 deliberately: extended support for
VS2008 Professional ends on April 10, 2018.

In any case, the extension problem will occur regardless of what
you do:
- if you switch compilers within 2.7, applications may crash
- if you switch compilers and declare it 2.8, you extensions
  might not be available precompiled for some time (in particular
  if the developers of some package have abandoned 2.7)
- if you don't switch compilers, availability of the tool chain
  will be terrible.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 6, 2014, at 3:33 PM, Chris Angelico  wrote:

> On Sat, Jun 7, 2014 at 5:11 AM, Donald Stufft  wrote:
>> Is it really any difference in maintenance if you just stop applying updates 
>> to
>> 2.7 and switch to 2.8? If 2.8 is really just 2.7 with a new compiler then 
>> there
>> should be no functional difference between doing that and doing a 
>> 2.7.whatever
>> except all of the tooling that relies on the compiler not to change in micro
>> releases won’t suddenly break and freak out.
> 
> If the only difference between 2.7 and 2.8 is the compiler used on
> Windows, what happens on Linux and other platforms? A Python 2.8 would
> have to be materially different from Python 2.7, not just binarily
> incompatible on one platform.
> 
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Well it’d contain bug fixes and whatever other sorts of things you’d put
into a 2.7.whatever release. So they’d still want to upgrade to 2.8 since
that’ll have bug fixes.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Angelico
On Sat, Jun 7, 2014 at 5:11 AM, Donald Stufft  wrote:
> Is it really any difference in maintenance if you just stop applying updates 
> to
> 2.7 and switch to 2.8? If 2.8 is really just 2.7 with a new compiler then 
> there
> should be no functional difference between doing that and doing a 2.7.whatever
> except all of the tooling that relies on the compiler not to change in micro
> releases won’t suddenly break and freak out.

If the only difference between 2.7 and 2.8 is the compiler used on
Windows, what happens on Linux and other platforms? A Python 2.8 would
have to be materially different from Python 2.7, not just binarily
incompatible on one platform.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Martin v. Löwis
Am 06.06.14 19:31, schrieb Brian Curtin:

>> If that's a non-issue, or if we can actually drop XP support, I'm all for it.
> 
> Extended support ended in April of this year, so I think we should put
> XP as unsupported for 3.5 in PEP 11 -
> http://legacy.python.org/dev/peps/pep-0011/
> 
> I seem to remember that we were waiting for this anyway.

We don't actually need to explicitly put XP there, as PEP 11 ties our
support to the Microsoft product life cycle. XP is not supported by
Python anymore.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Martin v. Löwis
Am 06.06.14 17:41, schrieb Steve Dower:
> Hi all
> 
> I would like to propose moving Python 3.5 to use Visual C++ 14.0 as
> the main compiler.

This is fine with me, but I'm worried about the precise timing of doing
so. I assume that you would plan to do this moving before VC++ 14 is
actually released. This worries me for three reasons:
1. what is the availability of the compiler during the testing phase,
   and what will it be immediately after the testing ends (where
   traditionally people would have to buy licenses, or wait for VS
   Express to be released)?
2. what is the risk of installing a beta compiler on what might
   otherwise be a "production" developer system? In particular, could
   it interfere with other VS installations, and could it require a
   complete system reinstall when the final release of VC 14 is
   available?
3. what is the chance of the final release being delayed beyond
   the planned release date of Python 3.5? Microsoft has a bad
   track record of meeting release dates (or the tradition of not
   announcing any for that reason); the blog says that it will
   be available "sometime in 2015". Now, Python 3.5 might appear
   November 2015, so what do we do if VS 2015 is not released
   by the time 3.5b1 is planned?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 6, 2014, at 3:09 PM, Brian Curtin  wrote:

> On Fri, Jun 6, 2014 at 11:08 PM, Donald Stufft  wrote:
>> 
>> On Jun 6, 2014, at 3:04 PM, Brian Curtin  wrote:
>> 
>>> On Fri, Jun 6, 2014 at 10:56 PM,   wrote:
 On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:
 
> None of the options are particularly good, but yes, I think that's an
> option we have to consider. We're supporting 2.7.x for 6 more years on
> a compiler that is already 6 years old.
 
 Surely that is infinitely less desirable than simply bumping the minor
 version?
>>> 
>>> It's definitely not desirable, but "simply" bumping the minor version
>>> is not A Thing.
>> 
>> Why? I mean even if it’s the same thing as 2.7 just with an updated
>> compiler that seems like a better answer than having to deal with
>> 2.7.whatever suddenly breaking all C exts.
> 
> Because then we have to maintain 2.8 at a time when no one even wants
> to maintain 2.7?

Is it really any difference in maintenance if you just stop applying updates to
2.7 and switch to 2.8? If 2.8 is really just 2.7 with a new compiler then there
should be no functional difference between doing that and doing a 2.7.whatever
except all of the tooling that relies on the compiler not to change in micro
releases won’t suddenly break and freak out.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 11:08 PM, Donald Stufft  wrote:
>
> On Jun 6, 2014, at 3:04 PM, Brian Curtin  wrote:
>
>> On Fri, Jun 6, 2014 at 10:56 PM,   wrote:
>>> On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:
>>>
 None of the options are particularly good, but yes, I think that's an
 option we have to consider. We're supporting 2.7.x for 6 more years on
 a compiler that is already 6 years old.
>>>
>>> Surely that is infinitely less desirable than simply bumping the minor
>>> version?
>>
>> It's definitely not desirable, but "simply" bumping the minor version
>> is not A Thing.
>
> Why? I mean even if it’s the same thing as 2.7 just with an updated
> compiler that seems like a better answer than having to deal with
> 2.7.whatever suddenly breaking all C exts.

Because then we have to maintain 2.8 at a time when no one even wants
to maintain 2.7?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft

On Jun 6, 2014, at 3:04 PM, Brian Curtin  wrote:

> On Fri, Jun 6, 2014 at 10:56 PM,   wrote:
>> On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:
>> 
>>> None of the options are particularly good, but yes, I think that's an
>>> option we have to consider. We're supporting 2.7.x for 6 more years on
>>> a compiler that is already 6 years old.
>> 
>> Surely that is infinitely less desirable than simply bumping the minor
>> version?
> 
> It's definitely not desirable, but "simply" bumping the minor version
> is not A Thing.

Why? I mean even if it’s the same thing as 2.7 just with an updated
compiler that seems like a better answer than having to deal with
2.7.whatever suddenly breaking all C exts.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brett Cannon
On Fri Jun 06 2014 at 2:59:24 PM,  wrote:

> On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:
>
> > None of the options are particularly good, but yes, I think that's an
> > option we have to consider. We're supporting 2.7.x for 6 more years on
> > a compiler that is already 6 years old.
>
> Surely that is infinitely less desirable than simply bumping the minor
> version?
>

Nope. A new minor release of Python is a massive undertaking which is why
we have saved ourselves the hassle of doing a Python 2.8 or not giving a
clear signal as to when Python 2.x will end as a language.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 10:56 PM,   wrote:
> On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:
>
>> None of the options are particularly good, but yes, I think that's an
>> option we have to consider. We're supporting 2.7.x for 6 more years on
>> a compiler that is already 6 years old.
>
> Surely that is infinitely less desirable than simply bumping the minor
> version?

It's definitely not desirable, but "simply" bumping the minor version
is not A Thing.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Division of tool labour in porting Python 2 code to 2/3

2014-06-06 Thread Brett Cannon
On Fri Jun 06 2014 at 2:29:13 PM, Terry Reedy  wrote:

> On 6/6/2014 12:37 PM, Brett Cannon wrote:
> > After Glyph and Alex's email about their asks for assisting in writing
> > Python 2/3 code, it got me thinking about where in the toolchain various
> > warnings and such should go in order to help direct energy to help
> > develop whatever future toolchain to assist in porting.
> >
> > There seems to be three places where issues are/can be caught once a
> > project has embarked down the road of 2/3 source compatibility:
> >
> >  1. -3 warnings
> >  2. Some linter tool
> >  3. Failing tests
> >
> > -3 warnings are things that we know are flat-out wrong and do not cause
> > massive compatibility issues in the stdlib. For instance, warning that
> > buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake
> > when he asked for it as a new warning -- is a perfect example of
> > something that isn't excessively noisy and won't cause issues when
> > people run with it.
> >
> > But what about warning about classic classes? The stdlib is full of them
> > and they were purposefully left alone for compatibility reasons. But
> > there is a subtle semantic difference between classic and new-style
> > classes,
>
> A non-subtle difference is that old style classes do not have .__new__.
> I just ran into this when backporting an Idle test to 2.7. (I rewrote
> the test to avoid diverging the code). In retrospect, perhaps we should
> have added a global 'new-class future' -C switch, like -Q, and made sure
> that stdlib worked either way. People running 2and3 code could then run
> 2.x with the switch. Is is possible to add this now?
>

I consider changing the CLI out of bounds in a bugfix release as it's part
of the API of Python.


>
>  > and so 2/3 code should consider switching
>
> I do not not understand what you mean without a switch or furture
> statement available to switch from old to new in 2.7.
>

Run a 2to3 fixer that changes all of their classes to new-style.


>
> > (this is when people
> > chime in saying "this is why we want a 2.8 release!", but that still
> > isn't happening). If this were made a py3k warning in 2.7 then the
> > stdlib itself would spew out warnings which we can't change due to
> > compatibility, so that makes it not useful
> > (http://bugs.python.org/issue21231).
>
> Don't issue the warning if the class is in the stdlib.
> If the warning is issued *after* creating class C:
>
> f = C.__module__.__file__
> if classic(C) and (not 'lib' in f or 'site_packages' in f):
>warn(...)
>
> On Windows, the directory is 'Lib'; I presume it is lowercased
> everywhere. If not, adjust.
>

That's just asking for trouble. I don't want to be import-dependent like
that in the stdlib.


>
>  > But as part of a lint tool specific
> > to Python 2.7 that kind of warning would not be an issue and is easily
> > managed and integrated into CI setups to make sure there are no
> regressions.
> >
> > Lastly, there are things like string/unicode comparisons.
> > http://bugs.python.org/issue21401 has a patch from VIctor which warns
> > when comparing strings and unicode in Python 2.7. Much like the classic
> > classes example, the stdlib becomes rather noisy due to APIs that handle
> > either/or, etc. But unlike the classic classes example, you just can't
> > systematically verify that two variables are always going to be str vs.
> > unicode in Python 2.7 if they aren't literals. If people want to
> > implement type constraint graphs for 2.7 code to help find them then
> > that's great, but I personally don't have that kind of time. In this
> > instance it would seem like relying on a project's unit tests to find
> > this sort of problem is the best option.
> >
> > With those three levels in mind, where do we draw the line between these
> > levels? Take for instance the print statement. Right now there is no
> > warning with -3. Do we add one and then update the 2.7 stdlib to prevent
> > warnings being generated by the stdlib?
>
> Make conditional as with class.
>
> We *could* change 'print s' to the exactly equivalent 'print(s)'
> (perhaps half the cases); 'print r, s' to "print('%s %s' % (r,s)),
> 'print '', y' to "print(' %s' % y), and so on. However, 'print
>  >>self.stdout, x', etc, does not translate to a pseudo-call. It would
> need transltion to "self.stdout.write(x+'\n')".  Grepping 2.7.6 lib/*.py
> for print gives 1341 hits, with at least 1000 being actual print
> statements.
>

Yep, which is why I don't want to do a 2to3 run on the stdlib to get rid of
them. I also want to minimize conditional checks as it leads to potential
issues of people thinking it's okay not to change things when there are in
actually differences (e.g. I don't want to promote classic classes or
native strings if it can be helped for the vast majority of users).

-Brett


>
>  > Or do we add it to some linter
>  > tool to pick up when people accidentally leave one in their code?
>
> > The reason I ask is since this 

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread dw+python-dev
On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote:

> None of the options are particularly good, but yes, I think that's an
> option we have to consider. We're supporting 2.7.x for 6 more years on
> a compiler that is already 6 years old.

Surely that is infinitely less desirable than simply bumping the minor
version?


David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread M.-A. Lemburg
On 06.06.2014 20:49, Brian Curtin wrote:
> On Fri, Jun 6, 2014 at 10:41 PM, M.-A. Lemburg  wrote:
>> On 06.06.2014 20:25, Brian Curtin wrote:
>>> On Fri, Jun 6, 2014 at 10:19 PM, Chris Angelico  wrote:
 On Sat, Jun 7, 2014 at 4:12 AM, Steve Dower  
 wrote:
> Chris Angelico wrote:
>> On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  
>> wrote:
>>> What this means for Python is that C extensions for Python 3.5 and 
>>> later can be built using any version of MSVC from 14.0 and later.
>>
>> Oh, if only this had been available for 2.7!! Actually... this means 
>> that 14.0 would be a good target for a compiler change for 2.7.x, if 
>> such a change is ever acceptable.
>
> Maybe, but I doubt it will ever be acceptable :)

 Well, there were discussions. Since Python 2.7's support is far
 exceeding the Microsoft promise of support for the compiler it was
 built on, there's going to be a problem, one way or the other. I don't
 know how that's going to end up being resolved.
>>>
>>> We're going to have to change it at some point, otherwise we're going
>>> to have people in 2018 scrambling to find VS2008, which will be 35
>>> versions too old by then. No matter what we do here, we're going to
>>> have a tough PR situation, but we have to make something workable. I'd
>>> rather cause a hassle than outright kill extensions.
>>>
>>> I would probably prefer we aim for VS 14 for 3.5, and then explore
>>> making the same change for the 2.7.x release that comes after 3.5.0
>>> comes out. Lessons learned and all that.
>>
>> Are you sure that's an option ? Changing the compiler the stock
>> Python from python.org is built with will most likely render
>> existing Python extensions built for 2.7.x with x < (release that comes
>> after 3.5.0) broken, so users and installation tools will end up
>> having to pay close attention to the patch level version of Python
>> they are using... which is something we wanted to avoid after
>> we ran into this situation with 1.5.1 and 1.5.2 a few years ago.
> 
> None of the options are particularly good, but yes, I think that's an
> option we have to consider. We're supporting 2.7.x for 6 more years on
> a compiler that is already 6 years old. Something less than awesome
> for everyone involved is going to have to happen to make that
> possible.

Perhaps we could combine this with the breakage that a Python 2.7.10
would introduce due to the two digit patch level release version ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-05-28: Released mxODBC.Connect 2.1.0 ... http://egenix.com/go56
2014-07-02: Python Meeting Duesseldorf ... 26 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 10:41 PM, M.-A. Lemburg  wrote:
> On 06.06.2014 20:25, Brian Curtin wrote:
>> On Fri, Jun 6, 2014 at 10:19 PM, Chris Angelico  wrote:
>>> On Sat, Jun 7, 2014 at 4:12 AM, Steve Dower  
>>> wrote:
 Chris Angelico wrote:
> On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  
> wrote:
>> What this means for Python is that C extensions for Python 3.5 and later 
>> can be built using any version of MSVC from 14.0 and later.
>
> Oh, if only this had been available for 2.7!! Actually... this means that 
> 14.0 would be a good target for a compiler change for 2.7.x, if such a 
> change is ever acceptable.

 Maybe, but I doubt it will ever be acceptable :)
>>>
>>> Well, there were discussions. Since Python 2.7's support is far
>>> exceeding the Microsoft promise of support for the compiler it was
>>> built on, there's going to be a problem, one way or the other. I don't
>>> know how that's going to end up being resolved.
>>
>> We're going to have to change it at some point, otherwise we're going
>> to have people in 2018 scrambling to find VS2008, which will be 35
>> versions too old by then. No matter what we do here, we're going to
>> have a tough PR situation, but we have to make something workable. I'd
>> rather cause a hassle than outright kill extensions.
>>
>> I would probably prefer we aim for VS 14 for 3.5, and then explore
>> making the same change for the 2.7.x release that comes after 3.5.0
>> comes out. Lessons learned and all that.
>
> Are you sure that's an option ? Changing the compiler the stock
> Python from python.org is built with will most likely render
> existing Python extensions built for 2.7.x with x < (release that comes
> after 3.5.0) broken, so users and installation tools will end up
> having to pay close attention to the patch level version of Python
> they are using... which is something we wanted to avoid after
> we ran into this situation with 1.5.1 and 1.5.2 a few years ago.

None of the options are particularly good, but yes, I think that's an
option we have to consider. We're supporting 2.7.x for 6 more years on
a compiler that is already 6 years old. Something less than awesome
for everyone involved is going to have to happen to make that
possible.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread M.-A. Lemburg
On 06.06.2014 20:25, Brian Curtin wrote:
> On Fri, Jun 6, 2014 at 10:19 PM, Chris Angelico  wrote:
>> On Sat, Jun 7, 2014 at 4:12 AM, Steve Dower  
>> wrote:
>>> Chris Angelico wrote:
 On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  
 wrote:
> What this means for Python is that C extensions for Python 3.5 and later 
> can be built using any version of MSVC from 14.0 and later.

 Oh, if only this had been available for 2.7!! Actually... this means that 
 14.0 would be a good target for a compiler change for 2.7.x, if such a 
 change is ever acceptable.
>>>
>>> Maybe, but I doubt it will ever be acceptable :)
>>
>> Well, there were discussions. Since Python 2.7's support is far
>> exceeding the Microsoft promise of support for the compiler it was
>> built on, there's going to be a problem, one way or the other. I don't
>> know how that's going to end up being resolved.
> 
> We're going to have to change it at some point, otherwise we're going
> to have people in 2018 scrambling to find VS2008, which will be 35
> versions too old by then. No matter what we do here, we're going to
> have a tough PR situation, but we have to make something workable. I'd
> rather cause a hassle than outright kill extensions.
> 
> I would probably prefer we aim for VS 14 for 3.5, and then explore
> making the same change for the 2.7.x release that comes after 3.5.0
> comes out. Lessons learned and all that.

Are you sure that's an option ? Changing the compiler the stock
Python from python.org is built with will most likely render
existing Python extensions built for 2.7.x with x < (release that comes
after 3.5.0) broken, so users and installation tools will end up
having to pay close attention to the patch level version of Python
they are using... which is something we wanted to avoid after
we ran into this situation with 1.5.1 and 1.5.2 a few years ago.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-05-28: Released mxODBC.Connect 2.1.0 ... http://egenix.com/go56
2014-07-02: Python Meeting Duesseldorf ... 26 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Division of tool labour in porting Python 2 code to 2/3

2014-06-06 Thread Terry Reedy

On 6/6/2014 12:37 PM, Brett Cannon wrote:

After Glyph and Alex's email about their asks for assisting in writing
Python 2/3 code, it got me thinking about where in the toolchain various
warnings and such should go in order to help direct energy to help
develop whatever future toolchain to assist in porting.

There seems to be three places where issues are/can be caught once a
project has embarked down the road of 2/3 source compatibility:

 1. -3 warnings
 2. Some linter tool
 3. Failing tests

-3 warnings are things that we know are flat-out wrong and do not cause
massive compatibility issues in the stdlib. For instance, warning that
buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake
when he asked for it as a new warning -- is a perfect example of
something that isn't excessively noisy and won't cause issues when
people run with it.

But what about warning about classic classes? The stdlib is full of them
and they were purposefully left alone for compatibility reasons. But
there is a subtle semantic difference between classic and new-style
classes,


A non-subtle difference is that old style classes do not have .__new__. 
I just ran into this when backporting an Idle test to 2.7. (I rewrote 
the test to avoid diverging the code). In retrospect, perhaps we should 
have added a global 'new-class future' -C switch, like -Q, and made sure 
that stdlib worked either way. People running 2and3 code could then run 
2.x with the switch. Is is possible to add this now?


> and so 2/3 code should consider switching

I do not not understand what you mean without a switch or furture 
statement available to switch from old to new in 2.7.



(this is when people
chime in saying "this is why we want a 2.8 release!", but that still
isn't happening). If this were made a py3k warning in 2.7 then the
stdlib itself would spew out warnings which we can't change due to
compatibility, so that makes it not useful
(http://bugs.python.org/issue21231).


Don't issue the warning if the class is in the stdlib.
If the warning is issued *after* creating class C:

f = C.__module__.__file__
if classic(C) and (not 'lib' in f or 'site_packages' in f):
  warn(...)

On Windows, the directory is 'Lib'; I presume it is lowercased 
everywhere. If not, adjust.


> But as part of a lint tool specific

to Python 2.7 that kind of warning would not be an issue and is easily
managed and integrated into CI setups to make sure there are no regressions.

Lastly, there are things like string/unicode comparisons.
http://bugs.python.org/issue21401 has a patch from VIctor which warns
when comparing strings and unicode in Python 2.7. Much like the classic
classes example, the stdlib becomes rather noisy due to APIs that handle
either/or, etc. But unlike the classic classes example, you just can't
systematically verify that two variables are always going to be str vs.
unicode in Python 2.7 if they aren't literals. If people want to
implement type constraint graphs for 2.7 code to help find them then
that's great, but I personally don't have that kind of time. In this
instance it would seem like relying on a project's unit tests to find
this sort of problem is the best option.

With those three levels in mind, where do we draw the line between these
levels? Take for instance the print statement. Right now there is no
warning with -3. Do we add one and then update the 2.7 stdlib to prevent
warnings being generated by the stdlib?


Make conditional as with class.

We *could* change 'print s' to the exactly equivalent 'print(s)' 
(perhaps half the cases); 'print r, s' to "print('%s %s' % (r,s)), 
'print '', y' to "print(' %s' % y), and so on. However, 'print 
>>self.stdout, x', etc, does not translate to a pseudo-call. It would 
need transltion to "self.stdout.write(x+'\n')".  Grepping 2.7.6 lib/*.py 
for print gives 1341 hits, with at least 1000 being actual print statements.


> Or do we add it to some linter
> tool to pick up when people accidentally leave one in their code?


The reason I ask is since this is clear I'm willing to spearhead the
tooling work we talked about at the language summit to make sure there's
a clear path for people wanting to port which is as easy as (reasonably)
possible, but I don't want to start on it until I have a clear
indication of what people are going to be okay with.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 10:19 PM, Chris Angelico  wrote:
> On Sat, Jun 7, 2014 at 4:12 AM, Steve Dower  wrote:
>> Chris Angelico wrote:
>>> On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  
>>> wrote:
 What this means for Python is that C extensions for Python 3.5 and later 
 can be built using any version of MSVC from 14.0 and later.
>>>
>>> Oh, if only this had been available for 2.7!! Actually... this means that 
>>> 14.0 would be a good target for a compiler change for 2.7.x, if such a 
>>> change is ever acceptable.
>>
>> Maybe, but I doubt it will ever be acceptable :)
>
> Well, there were discussions. Since Python 2.7's support is far
> exceeding the Microsoft promise of support for the compiler it was
> built on, there's going to be a problem, one way or the other. I don't
> know how that's going to end up being resolved.

We're going to have to change it at some point, otherwise we're going
to have people in 2018 scrambling to find VS2008, which will be 35
versions too old by then. No matter what we do here, we're going to
have a tough PR situation, but we have to make something workable. I'd
rather cause a hassle than outright kill extensions.

I would probably prefer we aim for VS 14 for 3.5, and then explore
making the same change for the 2.7.x release that comes after 3.5.0
comes out. Lessons learned and all that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Angelico
On Sat, Jun 7, 2014 at 4:12 AM, Steve Dower  wrote:
> Chris Angelico wrote:
>> On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  
>> wrote:
>>> What this means for Python is that C extensions for Python 3.5 and later 
>>> can be built using any version of MSVC from 14.0 and later.
>>
>> Oh, if only this had been available for 2.7!! Actually... this means that 
>> 14.0 would be a good target for a compiler change for 2.7.x, if such a 
>> change is ever acceptable.
>
> Maybe, but I doubt it will ever be acceptable :)

Well, there were discussions. Since Python 2.7's support is far
exceeding the Microsoft promise of support for the compiler it was
built on, there's going to be a problem, one way or the other. I don't
know how that's going to end up being resolved.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Steve Dower
Chris Angelico wrote:
> On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  wrote:
>> What this means for Python is that C extensions for Python 3.5 and later can 
>> be built using any version of MSVC from 14.0 and later.
>
> Oh, if only this had been available for 2.7!! Actually... this means that 
> 14.0 would be a good target for a compiler change for 2.7.x, if such a change 
> is ever acceptable.

Maybe, but I doubt it will ever be acceptable :)

> To what extent is this compatibility going to be maintained? Is there a 
> guarantee that there'll be X versions (or X years) of cross-compilation 
> support?

There are a few breaking changes in this version that are designed to 
standardize on a function-call based ABI, which should effectively be a 
life-long guarantee.

The only promise I can make is this: when cross-compilation support is 
eventually broken, it will be due to something that nobody has been able to 
predict up until now. (Hopefully that's better than promising that it will be 
broken in the very next release.)

> ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Steve Dower
Stefan Krah wrote:
>Stefan Krah  wrote:
>> > * Will VS 14 be golden prior to Python 3.5's release? It would suck to
>> >   rely on a beta compiler.. :)
>> 
>> This is my only concern, too. Otherwise, +1 for the switch.
>
>One more thing: Will the SDK 64-bit tools be available for the Express 
>Versions?

They should be. If they're not, I'll certainly be making a noise about it 
(unless there's another, easier way to get the tools by then...)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Steve Dower
dw+python-...@hmmz.org wrote:
> Speaking as a third party who aims to provide binary distributions for recent
> Python releases on Windows, every new compiler introduces a licensing and
> configuration headache. So I guess the questions are:
> 
> * Does the ABI stability address some historical real world problem with
> Python binary builds? (I guess possibly)

Yes. It's very hard to explain to users that even though they've gone out and 
paid for Visual Studio 2013 Ultimate, they don't really have a C compiler that 
works with Python. This stability will eventually get us to a place where it 
doesn't matter what version of the compiler you have, though for a while people 
will obviously need the latest. (Another thing I'm working on is making sure 
that it's really easy to get the latest... lots of pieces to this puzzle.)

> * Is the existing solution of third parties building under e.g. Mingw as
> an option of last resort causing real world issues? It seems to work
> for a lot of people, although I personally avoid it.

I think it actually tends to solve more issues than it causes :(  I want to fix 
that by making MSVC better for Python, rather than switching away to another 
toolset.

> * Have other compiler vendors indicated they will change their ABI
> environment to match VS under this new stability guarantee? If not,
> then as yet there is no real world benefit here.

I have no idea, but I hope they do (eventually they almost certainly will). 
I've already mentioned to our team that they should reach out to the other 
projects and try to help them move it along, though I have no idea if they have 
the time or contacts to manage that.

FWIW, the stability guarantee was only announced this week, so there's a good 
chance that the gcc/clang/etc. teams aren't even aware of it yet.

> * Has Python ever hit a showstopper release issue as a result of a bug
> in MSVC? (I guess probably not).

Not to my knowledge, and I'm certainly hoping to avoid it by keeping the builds 
coming regularly. I can't do an official buildbot for it (and probably can't 
even reuse the infrastructure) since I'm going to work against the latest 
internal version as much as I can and we get new builds almost daily. More 
likely, building Python will reveal showstopper issues that actually get fixed 
(and it has done in the past, though that was never publicised :) )

> * Will VS 14 be golden prior to Python 3.5's release? It would suck to
> rely on a beta compiler.. :)

I sure hope so. The current planning looks like it will (I'm assuming that 
Python 3.5 is going to be late next year, but I couldn't find a good reference).

If things slip here, I'm going to be surrounded by very stressed people, which 
is not much fun. So I hope it'll be done!

At worst, VS 14 RC (or whatever label it gets) will probably be released under 
a "go live" licence. If anything is dramatically broken at that point, we'll 
know and it should be fixed, or we know that it's going to be around for a 
while regardless and we can make the decision to either stick with VC10 or work 
around the issues.

> Sorry for dunking water on this, but I've recently spent a ton of time 
> getting a
> Microsoft build environment running, and it seems possible a new compiler may
> not yet justify more effort if there is little tangible benefit.

Not at all. I've spent far more time than I wanted to getting a build 
environment running for producing the Python 2.7 installers, and I spent just 
as long getting an environment for default going too. I'm personally a big fan 
of automating things like this, so you can also expect scripts (probably 
PowerShell) that will configure as much as possible.

Cheers,
Steve

> David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Brian Curtin
On Fri, Jun 6, 2014 at 8:22 PM, Zachary Ware
 wrote:
> On Fri, Jun 6, 2014 at 10:41 AM, Steve Dower  
> wrote:
>> Thoughts/comments/concerns?
>
> My only concern is support for elderly versions of Windows, in
> particular: XP.  I seem to recall the last "let's update our MSVC
> version" discussion dying off because of XP support.  Even though MS
> has abandoned it, I'm not sure whether we can yet.
>
> If that's a non-issue, or if we can actually drop XP support, I'm all for it.

Extended support ended in April of this year, so I think we should put
XP as unsupported for 3.5 in PEP 11 -
http://legacy.python.org/dev/peps/pep-0011/

I seem to remember that we were waiting for this anyway.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Julian Taylor
On 06.06.2014 04:26, Greg Ewing wrote:
> Nathaniel Smith wrote:
> 
>> I'd be a little nervous about whether anyone has implemented, say, an
>> iadd with side effects such that you can tell whether a copy was made,
>> even if the object being copied is immediately destroyed.
> 
> I can think of at least one plausible scenario where
> this could occur: the operand is a view object that
> wraps another object, and its __iadd__ method updates
> that other object.
> 
> In fact, now that I think about it, exactly this
> kind of thing happens in numpy when you slice an
> array!
> 
> So the opt-in indicator would need to be dynamic, on
> a per-object basis, rather than a type flag.
> 

yes an opt-in indicator would need to receive both operand objects so it
would need to be a slot in the object or number type object.
Would the addition of a tp_can_elide slot to the object types be
acceptable for this rather specialized case?


tp_can_elide receives two objects and returns one of three values:
* can work inplace, operation is associative
* can work inplace but not associative
* cannot work inplace

Implementation could e.g. look about like this:

TARGET(BINARY_SUBTRACT) {
   fl = left->obj_type->tp_can_elide
   fr = right->obj_type->tp_can_elide
   elide = 0
   if (unlikely(fl)) {
  elide = fl(left, right)
   }
   else if (unlikely(fr)) {
  elide = fr(left, right)
   }
   if (unlikely(elide == YES) && left->refcnt == 1) {
   PyNumber_InPlaceSubtract(left, right)
   }
   else if (unlikely(elide == SWAPPABLE) && right->refcnt == 1) {
   PyNumber_InPlaceSubtract(right, left)
   }
   else {
   PyNumber_Subtract(left, right)
   }
}
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Stefan Krah
Stefan Krah  wrote:
> > * Will VS 14 be golden prior to Python 3.5's release? It would suck to
> >   rely on a beta compiler.. :)
> 
> This is my only concern, too. Otherwise, +1 for the switch.

One more thing: Will the SDK 64-bit tools be available for the Express
Versions?



Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Nathaniel Smith
On 6 Jun 2014 17:07, "Sturla Molden"  wrote:
> We would in total need two mutexes, one condition variable, a pthread, and
> a heap.

The proposal in my initial email requires zero pthreads, and is
substantially more effective. (Your proposal reduces only the alloc
overhead for large arrays; mine reduces both alloc and memory access
overhead for boyh large and small arrays.)

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread R. David Murray
On Fri, 06 Jun 2014 16:37:01 -, dw+python-...@hmmz.org wrote:
> On Fri, Jun 06, 2014 at 03:41:22PM +, Steve Dower wrote:
> 
> > [snip]
> 
> Speaking as a third party who aims to provide binary distributions for
> recent Python releases on Windows, every new compiler introduces a
> licensing and configuration headache. So I guess the questions are:
> 
> * Does the ABI stability address some historical real world problem with
>   Python binary builds? (I guess possibly)
> 
> * Is the existing solution of third parties building under e.g. Mingw as
>   an option of last resort causing real world issues? It seems to work
>   for a lot of people, although I personally avoid it.
> 
> * Have other compiler vendors indicated they will change their ABI
>   environment to match VS under this new stability guarantee? If not,
>   then as yet there is no real world benefit here.
> 
> * Has Python ever hit a showstopper release issue as a result of a bug
>   in MSVC? (I guess probably not).
> 
> * Will VS 14 be golden prior to Python 3.5's release? It would suck to
>   rely on a beta compiler.. :)
> 
> 
> Sorry for dunking water on this, but I've recently spent a ton of time
> getting a Microsoft build environment running, and it seems possible a
> new compiler may not yet justify more effort if there is little tangible
> benefit.

If I understand correctly (but I may not as I'm not a windows dev) we're
going to want to switch VS versions for 3.5 anyway, so switching to the
cutting edge one but where Steve can be and is willing to be in a tight
feedback loop with the developers sounds like a win to me.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Stefan Krah
dw+python-...@hmmz.org  wrote:
> * Has Python ever hit a showstopper release issue as a result of a bug
>   in MSVC? (I guess probably not).

Yes, a PGO issue:

http://bugs.python.org/issue15993


To be fair, in that issue I did not look if there's some undefined behavior in
longobject.c.


> * Will VS 14 be golden prior to Python 3.5's release? It would suck to
>   rely on a beta compiler.. :)

This is my only concern, too. Otherwise, +1 for the switch.



Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Division of tool labour in porting Python 2 code to 2/3

2014-06-06 Thread Brett Cannon
After Glyph and Alex's email about their asks for assisting in writing
Python 2/3 code, it got me thinking about where in the toolchain various
warnings and such should go in order to help direct energy to help develop
whatever future toolchain to assist in porting.

There seems to be three places where issues are/can be caught once a
project has embarked down the road of 2/3 source compatibility:

   1. -3 warnings
   2. Some linter tool
   3. Failing tests

-3 warnings are things that we know are flat-out wrong and do not cause
massive compatibility issues in the stdlib. For instance, warning that
buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake when
he asked for it as a new warning -- is a perfect example of something that
isn't excessively noisy and won't cause issues when people run with it.

But what about warning about classic classes? The stdlib is full of them
and they were purposefully left alone for compatibility reasons. But there
is a subtle semantic difference between classic and new-style classes, and
so 2/3 code should consider switching (this is when people chime in saying
"this is why we want a 2.8 release!", but that still isn't happening). If
this were made a py3k warning in 2.7 then the stdlib itself would spew out
warnings which we can't change due to compatibility, so that makes it not
useful (http://bugs.python.org/issue21231). But as part of a lint tool
specific to Python 2.7 that kind of warning would not be an issue and is
easily managed and integrated into CI setups to make sure there are no
regressions.

Lastly, there are things like string/unicode comparisons.
http://bugs.python.org/issue21401 has a patch from VIctor which warns when
comparing strings and unicode in Python 2.7. Much like the classic classes
example, the stdlib becomes rather noisy due to APIs that handle either/or,
etc. But unlike the classic classes example, you just can't systematically
verify that two variables are always going to be str vs. unicode in Python
2.7 if they aren't literals. If people want to implement type constraint
graphs for 2.7 code to help find them then that's great, but I personally
don't have that kind of time. In this instance it would seem like relying
on a project's unit tests to find this sort of problem is the best option.

With those three levels in mind, where do we draw the line between these
levels? Take for instance the print statement. Right now there is no
warning with -3. Do we add one and then update the 2.7 stdlib to prevent
warnings being generated by the stdlib? Or do we add it to some linter tool
to pick up when people accidentally leave one in their code?

The reason I ask is since this is clear I'm willing to spearhead the
tooling work we talked about at the language summit to make sure there's a
clear path for people wanting to port which is as easy as (reasonably)
possible, but I don't want to start on it until I have a clear indication
of what people are going to be okay with.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread dw+python-dev
On Fri, Jun 06, 2014 at 03:41:22PM +, Steve Dower wrote:

> [snip]

Speaking as a third party who aims to provide binary distributions for
recent Python releases on Windows, every new compiler introduces a
licensing and configuration headache. So I guess the questions are:

* Does the ABI stability address some historical real world problem with
  Python binary builds? (I guess possibly)

* Is the existing solution of third parties building under e.g. Mingw as
  an option of last resort causing real world issues? It seems to work
  for a lot of people, although I personally avoid it.

* Have other compiler vendors indicated they will change their ABI
  environment to match VS under this new stability guarantee? If not,
  then as yet there is no real world benefit here.

* Has Python ever hit a showstopper release issue as a result of a bug
  in MSVC? (I guess probably not).

* Will VS 14 be golden prior to Python 3.5's release? It would suck to
  rely on a beta compiler.. :)


Sorry for dunking water on this, but I've recently spent a ton of time
getting a Microsoft build environment running, and it seems possible a
new compiler may not yet justify more effort if there is little tangible
benefit.


David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Fri, 06 Jun 2014 11:59:31 -0400
Terry Reedy  wrote:

[]

> The other problem is that a small slice view of a large object keeps
> the large object alive, so a view user needs to think carefully about 
> whether to make a copy or create a view, and later to copy views to 
> delete the base object. This is not for beginners.

Yes, so it doesn't make sense to add such feature to any of existing
APIs. However, as I pointed in another mail, it would make lot of sense
to add iterator-based string API (because if dict methods were
*switched* to iterators, why can't string have them *as alternative*),
and for their return values, it would be ~ natural to return "string
views", especially if it's clearly and explicitly described that if
user wants to store them, they should be explicitly copied via
str(view).

One reason against this would be of course API bloat. But API bloat
happens all the time, for example compare this modest proposal
http://bugs.python.org/issue21180 with what's going to be actually
implemented:
http://legacy.python.org/dev/peps/pep-0467/#alternate-constructors .


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Zachary Ware
On Fri, Jun 6, 2014 at 10:41 AM, Steve Dower  wrote:
> Thoughts/comments/concerns?

My only concern is support for elderly versions of Windows, in
particular: XP.  I seem to recall the last "let's update our MSVC
version" discussion dying off because of XP support.  Even though MS
has abandoned it, I'm not sure whether we can yet.

If that's a non-issue, or if we can actually drop XP support, I'm all for it.

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Paul Moore
On 6 June 2014 16:41, Steve Dower  wrote:
> Basically, what I am offering to do is:
>
> * Update the files in PCBuild to work with Visual Studio "14"
> * Make any code changes necessary to build with VC14
> * Regularly test the latest Python source with the latest MSVC builds and 
> report issues/suggestions to the MSVC team
> * Keep all changes in a separate (public) repo until early next year when 
> we're getting close to the final VS "14" release
>
> What I am asking anyone else to do is:
>
> * Nothing

+1 from me.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Hrvoje Niksic

On 06/06/2014 05:59 PM, Terry Reedy wrote:

The other problem is that a small slice view of a large object keeps the
large object alive, so a view user needs to think carefully about
whether to make a copy or create a view, and later to copy views to
delete the base object. This is not for beginners.


And this was important enough that Java 7 actually removed the 
long-standing feature of String.substring creating a string that shares 
the character array with the original.


http://java-performance.info/changes-to-string-java-1-7-0_06/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread Barry Warsaw
On Jun 06, 2014, at 04:47 PM, MRAB wrote:

>Isn't this a little like when bool, True and False were added to
>Python 2.2.1, a bugfix release, an act that is, I believe, now regarded
>as a mistake not to be repeated?

Yes, that was a mistake, but the case under discussion is different.  With
True/False, it was a runtime-wide change that affected every Python program,
and there was no such "special dispensation".

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread Guido van Rossum
On Fri, Jun 6, 2014 at 8:47 AM, MRAB  wrote:

> On 2014-06-06 10:31, Victor Stinner wrote:
>
>> Hi,
>>
>> I added a new BaseEventLoop.is_closed() method to Tulip and Python
>> 3.5 to fix an issue (see Tulip issue 169 for the detail). The problem
>> is that I don't want to add this method to Python 3.4 because usually
>> we don't add new methods in minor versions of Python (future version
>> 3.4.2 in this case).
>>
>> Guido just wrote in the issue: "Actually for asyncio we have special
>> dispensation to push new features to minor releases (until 3.5).
>> Please push to 3.4 so the source code is the same everywhere (except
>> selectors.py, which is not covered by the exception)."
>>
>> I disagree with Guido. I would prefer to start to maintain a
>> different branch for Python 3.4, because I consider that only
>> bugfixes should be applied to Python 3.4.
>>
>>  [snip]
>
> Isn't this a little like when bool, True and False were added to
> Python 2.2.1, a bugfix release, an act that is, I believe, now regarded
> as a mistake not to be repeated?
>

It's a little like that, but it's also a little unlike that -- asyncio is
explicitly accepted in the stdlib with "provisional" status which allows
changes like this.

Regarding the workflow, I'd really like asyncio to be able to move faster
than the rest of the stdlib, at least until 3.5 is fixed. Working in the
Tulip repo is much easier for me than working in the CPython repo, so I'd
like to keep the workflow of Tulip -> 3.4 -> 3.5 as long as possible. I
also specifically consider selectors.py subject to a *different* workflow
-- for that module the workflow should be 3.5 -> Tulip. If Tulip's
update_stdlib.sh script's prompts to copy this file are too distracting, I
can hack the script to be silent about this file if it detects that the
CPython repo is 3.4.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Summary of Python tracker Issues

2014-06-06 Thread Python tracker

ACTIVITY SUMMARY (2014-05-30 - 2014-06-06)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open4650 (+15)
  closed 28802 (+52)
  total  33452 (+67)

Open issues with patches: 2127 


Issues opened (48)
==

#21614: Case sensitivity problem in multiprocessing.
http://bugs.python.org/issue21614  opened by ColinPDavidson

#21615: Curses bug report for Python 2.7 and Python 3.2
http://bugs.python.org/issue21615  opened by eclectic9509

#21616: argparse explodes with nargs='*' and a tuple metavar
http://bugs.python.org/issue21616  opened by vvas

#21617: importlib reload can fail with AttributeError if module remove
http://bugs.python.org/issue21617  opened by ned.deily

#21619: Cleaning up a subprocess with a broken pipe
http://bugs.python.org/issue21619  opened by vadmium

#21621: Add note to 3.x What's New re Idle changes in bugfix releases
http://bugs.python.org/issue21621  opened by terry.reedy

#21622: ctypes.util incorrectly fails for libraries without DT_SONAME
http://bugs.python.org/issue21622  opened by Jeremy.Huntwork

#21623: build ssl failed use vs2010 express
http://bugs.python.org/issue21623  opened by Mo.Jia

#21624: Idle: polish htests
http://bugs.python.org/issue21624  opened by terry.reedy

#21625: help()'s more-mode is frustrating
http://bugs.python.org/issue21625  opened by nedbat

#21626: Add options width and compact to pickle cli
http://bugs.python.org/issue21626  opened by barcc

#21627: Concurrently closing files and iterating over the open files d
http://bugs.python.org/issue21627  opened by sstewartgallus

#21629: clinic.py --converters fails
http://bugs.python.org/issue21629  opened by serhiy.storchaka

#21632: Idle: sychronize text files across versions as appropriate.
http://bugs.python.org/issue21632  opened by terry.reedy

#21633: Argparse does not propagate HelpFormatter class to subparsers
http://bugs.python.org/issue21633  opened by Michael.Cohen

#21635: difflib.SequenceMatcher stores matching blocks as tuples, not 
http://bugs.python.org/issue21635  opened by drevicko

#21642: "_ if 1else _" does not compile
http://bugs.python.org/issue21642  opened by Joshua.Landau

#21644: Optimize bytearray(int) constructor to use calloc()
http://bugs.python.org/issue21644  opened by haypo

#21645: test_read_all_from_pipe_reader() of test_asyncio hangs on Free
http://bugs.python.org/issue21645  opened by haypo

#21646: Add tests for turtle.ScrolledCanvas
http://bugs.python.org/issue21646  opened by ingrid

#21647: Idle unittests: make gui, mock switching easier.
http://bugs.python.org/issue21647  opened by terry.reedy

#21648: urllib urlopener leaves open sockets for FTP connection
http://bugs.python.org/issue21648  opened by Claudiu.Popa

#21649: Mention "Recommendations for Secure Use of TLS and DTLS"
http://bugs.python.org/issue21649  opened by pitrou

#21650: add json.tool option to avoid alphabetic sort of fields
http://bugs.python.org/issue21650  opened by Pavel.Kazlou

#21652: Python 2.7.7 regression in mimetypes module on Windows
http://bugs.python.org/issue21652  opened by foom

#21655: Write Unit Test for Vec2 class in the Turtle Module
http://bugs.python.org/issue21655  opened by Lita.Cho

#21656: Create test coverage for TurtleScreenBase in Turtle
http://bugs.python.org/issue21656  opened by Lita.Cho

#21657: pip.get_installed_distributions() Does not return packages in 
http://bugs.python.org/issue21657  opened by Adam.Matan

#21658: __m128, can't build 3.4.1 with intel 14.0.0
http://bugs.python.org/issue21658  opened by aom

#21659: IDLE: One corner calltip case
http://bugs.python.org/issue21659  opened by serhiy.storchaka

#21660: Substitute @TOKENS@ from sysconfig variables, for python-confi
http://bugs.python.org/issue21660  opened by haubi

#21664: multiprocessing leaks temporary directories pymp-xxx
http://bugs.python.org/issue21664  opened by yjhong

#21665: 2.7.7 ttk widgets not themed
http://bugs.python.org/issue21665  opened by les.bothwell

#21666: Argparse exceptions should include which argument has a proble
http://bugs.python.org/issue21666  opened by v+python

#21667: Clarify status of O(1) indexing semantics of str objects
http://bugs.python.org/issue21667  opened by ncoghlan

#21668: The select and time modules uses libm functions without linkin
http://bugs.python.org/issue21668  opened by fornwall

#21669: Custom error messages when print & exec are used as statements
http://bugs.python.org/issue21669  opened by ncoghlan

#21670: Add repr to shelve.Shelf
http://bugs.python.org/issue21670  opened by Claudiu.Popa

#21671: CVE-2014-0224: OpenSSL upgrade to 1.0.1h on Windows required
http://bugs.python.org/issue21671  opened by lambacck

#21672: Python for Windows 2.7.7: Path Configuration File No Longer Wo
http://bugs.python.org/issue21672  opened by jblairpdx

#21673: Idle: hilite search terms in hits in Find in

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Donald Stufft
On Jun 6, 2014, at 11:41 AM, Steve Dower  wrote:

> words

+1 from me.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Sturla Molden
Julian Taylor  wrote:
 
> The problem with this approach is that it is already difficult enough to
> handle memory in numpy.

I would not do this in a way that complicates memory management in NumPy. I
would just replace malloc and free with temporarily cached versions. From
the perspective of NumPy the API should be the same.

> Having a cache that potentially stores gigabytes
> of memory out of the users sight will just make things worse.

Buffer don't need to stay in cache forver, just long enough to allow resue
within an expression. We are probably talking about delaying the call to
free with just a few microseconds.

We could e.g. have a setup like this:

NumPy thread on "malloc":
- tries to grab memory off the internal heap
- calls system malloc on failure

NumPy thread on "free":
- returns a buffer to the internal heap
- signals a condition

Background daemonic GC thread:
- wakes after sleeping on the condition
- sleeps for another N microseconds (N = magic number)
- flushes or shrinks the internal heap with system free
- goes back to sleeping on the condition 
 
It can be implemented with the same API as malloc and free, and plugged
directly into the existing NumPy code. 

We would in total need two mutexes, one condition variable, a pthread, and
a heap.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Chris Angelico
On Sat, Jun 7, 2014 at 1:41 AM, Steve Dower  wrote:
> What this means for Python is that C extensions for Python 3.5 and later can 
> be built using any version of MSVC from 14.0 and later.

Oh, if only this had been available for 2.7!! Actually... this means
that 14.0 would be a good target for a compiler change for 2.7.x, if
such a change is ever acceptable.

To what extent is this compatibility going to be maintained? Is there
a guarantee that there'll be X versions (or X years) of
cross-compilation support?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Terry Reedy

On 6/6/2014 4:53 AM, Hrvoje Niksic wrote:

On 06/04/2014 05:52 PM, Mark Lawrence wrote:



Out of idle curiosity is there anything that stops MicroPython, or any
other implementation for that matter, from providing views of a string
rather than copying every time?  IIRC memoryviews in CPython rely on the
buffer protocol at the C API level, so since strings don't support this
protocol you can't take a memoryview of them.  Could this actually be
implemented in the future, is the underlying C code just too
complicated, or what?



Memory view of Unicode strings is controversial for two reasons:

1. It exposes the internal representation of the string. If memoryviews
of strings were supported in Python 3, PEP 393 would not have been
possible (without breaking that feature).

2. Even if it were OK to expose the internal representation, it might
not be what the users expect. For example, memoryview("Hrvoje") would
return a view of a 6-byte buffer, while memoryview("Nikšić") would
return a view of a 12-byte UCS-2 buffer. The user of a memory view might
expect to get UCS-2 (or UCS-4, or even UTF-8) in all cases.

An implementation that decided to export strings as memory views might
be forced to make a decision about internal representation of strings,
and then stick to it.

The byte objects don't have these issues, which is why in Python 2.7
memoryview("foo") works just fine, as does memoryview(b"foo") in Python 3.


The other problem is that a small slice view of a large object keeps the 
large object alive, so a view user needs to think carefully about 
whether to make a copy or create a view, and later to copy views to 
delete the base object. This is not for beginners.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Steve Dower
Hi all

I would like to propose moving Python 3.5 to use Visual C++ 14.0 as the main 
compiler. The first CTP of Visual Studio "14" was released earlier this week: 
http://blogs.msdn.com/b/vcblog/archive/2014/06/03/visual-studio-14-ctp.aspx

The major feature of interest in this version of MSVC is a new policy to 
maintain binary compatibility for the CRT into the future. (There will be a 
blog about this soon, but I didn't want to hold up getting the discussion 
started here.)

What this means for Python is that C extensions for Python 3.5 and later can be 
built using any version of MSVC from 14.0 and later. Those who are aware of the 
current state of affairs where you need to use a matching compiler will 
hopefully see how big an improvement this will be. It is also likely that other 
compilers will have an easier time providing compatibility with this new CRT, 
making it simpler and more reliable to build extensions with LLVM or GCC 
against an MSVC CPython. 

The other major benefit is that both products are at points in their 
development where changes can be made. Being a Microsoft employee, I have the 
ability to test Python builds regularly against the daily MSVC builds and to 
file bugs directly to the VC team (crashes, incorrect code generation, 
incorrect linking, performance regressions, etc.). This is a great opportunity 
to make sure that our needs are covered by the compiler team - it's also a good 
chance to raise any particular missing features that would be beneficial.

My internal testing shows that the core code is almost fully compatible and 
builds successfully with only trivial modifications (some CRT variables are now 
macros with a leading underscore). The project files need updating, but I am 
willing to do this as part of any migration. There may also be some work 
required for external dependencies, since I did not test these, but I am also 
willing to do that.

Basically, what I am offering to do is:

* Update the files in PCBuild to work with Visual Studio "14"
* Make any code changes necessary to build with VC14
* Regularly test the latest Python source with the latest MSVC builds and 
report issues/suggestions to the MSVC team
* Keep all changes in a separate (public) repo until early next year when we're 
getting close to the final VS "14" release 

What I am asking anyone else to do is:

* Nothing

Thoughts/comments/concerns?

Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread MRAB

On 2014-06-06 10:31, Victor Stinner wrote:

Hi,

I added a new BaseEventLoop.is_closed() method to Tulip and Python
3.5 to fix an issue (see Tulip issue 169 for the detail). The problem
is that I don't want to add this method to Python 3.4 because usually
we don't add new methods in minor versions of Python (future version
3.4.2 in this case).

Guido just wrote in the issue: "Actually for asyncio we have special
dispensation to push new features to minor releases (until 3.5).
Please push to 3.4 so the source code is the same everywhere (except
selectors.py, which is not covered by the exception)."

I disagree with Guido. I would prefer to start to maintain a
different branch for Python 3.4, because I consider that only
bugfixes should be applied to Python 3.4.


[snip]

Isn't this a little like when bool, True and False were added to
Python 2.2.1, a bugfix release, an act that is, I believe, now regarded
as a mistake not to be repeated?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Chris Angelico
On Fri, Jun 6, 2014 at 8:15 PM, Paul Sokolovsky  wrote:
> I'm sorry if I was somehow related to that, my
> bringing in the formal language spec was more a rhetorical figure, a
> response to people claiming O(1) requirement.

This was exactly why this whole discussion came up, though. We were
debating on the uPy bug tracker about how important O(1) indexing is;
I then came to python-list to try to get some solid data from which to
debate; and then the discussion jumped here to python-dev for more
solid explanations. The spec wasn't perfectly clear, and now it's
being made clearer: O(N) indexing does not violate Python's spec, ergo
uPy is allowed to use UTF-8 as its internal representation, as long as
script-visible behaviour is correct. It'll be interesting to see when
it's done (I'm currently working on that implementation, bit by bit)
and to run the CPython benchmarks on it.

It's been a fruitful and interesting discussion, and the formal
language spec is key to it. No need to apologize!

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Fri, 6 Jun 2014 21:48:41 +1000
Tim Delaney  wrote:

> On 6 June 2014 21:34, Paul Sokolovsky  wrote:
> 
> >
> > On Fri, 06 Jun 2014 20:11:27 +0900
> > "Stephen J. Turnbull"  wrote:
> >
> > > Paul Sokolovsky writes:
> > >
> > >  > That kinda means "string is atomic", instead of your
> > >  > "characters are atomic".
> > >
> > > I would be very surprised if a language that behaved that way was
> > > called a "Python subset".  No indexing, no slicing, no regexps, no
> > > .split(), no .startswith(), no sorted() or .sort(), ...!?
> > >
> > > If that's not what you mean by "string is atomic", I think you're
> > > using very confusing terminology.
> >
> > I'm sorry if I didn't mention it, or didn't make it clear enough -
> > it's all about layering.
> >
> > On level 0, you treat strings verbatim, and can write some subset of
> > apps (my point is that even this level allows to write lot enough
> > apps). Let's call this set A0.
> >
> > On level 1, you accept that there's some universal enough
> > conventions for some chars, like space or newline. And you can
> > write set of apps A1 > A0.
> >
> 
> At heart, this is exactly what the Python 3 "str" type is. The
> universal convention is "code points". 

Yes. Except for one small detail - Python3 specifies these code points
to be Unicode code points. And Unicode is a very bloated thing.

But if we drop that "Unicode" stipulation, then it's also exactly what
MicroPython implements. Its "str" type consists of codepoints, we don't
have pet names for them yet, like Unicode does, but their numeric
values are 0-255. Note that it in no way limits encodings, characters,
or scripts which can be used with MicroPython, because just like
Unicode, it support concept of "surrogate pairs" (but we don't call it
like that) - specifically, smaller code points may comprise bigger
groupings. But unlike Unicode, we don't stipulate format, value or
other constraints on how these "surrogate pairs"-alikes are formed,
leaving that to users.


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread R. David Murray
On Fri, 06 Jun 2014 10:05:52 -0400, Antoine Pitrou  wrote:
> Le 06/06/2014 07:00, R. David Murray a écrit :
> >
> > I don't have any opinion on the workflow.
> >
> > My understanding is that part of the purpose of the "provisional"
> > designation is to allow faster evolution (read: fixing) of an API before
> > the library becomes non-provisional.  Thus I agree with Guido here, and
> > will be doing something similar with at least one of the minor provisional
> > email API features in 3.4.2 (unless I miss the cutoff again ... :(
> 
> I would personally distinguish API fixes (compatibility-breaking 
> changes) from feature additions (new APIs).

It doesn't look like the PEP directly addresses API changes in maintenance
releases, and I suppose that should be fixed.

I specifically want to fix this API before someone depends on it working
the wrong way, which they would have to if I left it alone for the whole
of the 3.4 series.  (Issue 21091 for the curious.)

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread Antoine Pitrou

Le 06/06/2014 07:00, R. David Murray a écrit :


I don't have any opinion on the workflow.

My understanding is that part of the purpose of the "provisional"
designation is to allow faster evolution (read: fixing) of an API before
the library becomes non-provisional.  Thus I agree with Guido here, and
will be doing something similar with at least one of the minor provisional
email API features in 3.4.2 (unless I miss the cutoff again ... :(


I would personally distinguish API fixes (compatibility-breaking 
changes) from feature additions (new APIs).


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Mark Lawrence

On 06/06/2014 09:53, Hrvoje Niksic wrote:

On 06/04/2014 05:52 PM, Mark Lawrence wrote:

On 04/06/2014 16:32, Steve Dower wrote:


If copying into a separate list is a problem (memory-wise),
re.finditer('\\S+', string) also provides the same behaviour and
gives me the sliced string, so there's no need to index for anything.



Out of idle curiosity is there anything that stops MicroPython, or any
other implementation for that matter, from providing views of a string
rather than copying every time?  IIRC memoryviews in CPython rely on the
buffer protocol at the C API level, so since strings don't support this
protocol you can't take a memoryview of them.  Could this actually be
implemented in the future, is the underlying C code just too
complicated, or what?



Memory view of Unicode strings is controversial for two reasons:

1. It exposes the internal representation of the string. If memoryviews
of strings were supported in Python 3, PEP 393 would not have been
possible (without breaking that feature).

2. Even if it were OK to expose the internal representation, it might
not be what the users expect. For example, memoryview("Hrvoje") would
return a view of a 6-byte buffer, while memoryview("Nikšić") would
return a view of a 12-byte UCS-2 buffer. The user of a memory view might
expect to get UCS-2 (or UCS-4, or even UTF-8) in all cases.

An implementation that decided to export strings as memory views might
be forced to make a decision about internal representation of strings,
and then stick to it.

The byte objects don't have these issues, which is why in Python 2.7
memoryview("foo") works just fine, as does memoryview(b"foo") in Python 3.



Thanks for the explanation :)

--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Fri, 06 Jun 2014 09:32:25 +0100
Mark Lawrence  wrote:

> On 04/06/2014 16:52, Mark Lawrence wrote:
> > On 04/06/2014 16:32, Steve Dower wrote:
> >>
> >> If copying into a separate list is a problem (memory-wise),
> >> re.finditer('\\S+', string) also provides the same behaviour and
> >> gives me the sliced string, so there's no need to index for
> >> anything.
> >>
> >
> > Out of idle curiosity is there anything that stops MicroPython, or
> > any other implementation for that matter, from providing views of a
> > string rather than copying every time?  IIRC memoryviews in CPython
> > rely on the buffer protocol at the C API level, so since strings
> > don't support this protocol you can't take a memoryview of them.
> > Could this actually be implemented in the future, is the underlying
> > C code just too complicated, or what?
> >
> 
> Anybody?

I'd like to address this, and other, buffer manipulation
optimization ideas I have for MicroPython at some time later. But as
you suggest, it would possible to transparently have
"strings-by-reference". The reasons MicroPython doesn't have such so
far (and why I'm, as a uPy contributor, not ready to discuss them) is
because they're optimization, and everyone knows what premature
optimization is.

[]

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Tim Delaney
On 6 June 2014 21:34, Paul Sokolovsky  wrote:

>
> On Fri, 06 Jun 2014 20:11:27 +0900
> "Stephen J. Turnbull"  wrote:
>
> > Paul Sokolovsky writes:
> >
> >  > That kinda means "string is atomic", instead of your "characters
> >  > are atomic".
> >
> > I would be very surprised if a language that behaved that way was
> > called a "Python subset".  No indexing, no slicing, no regexps, no
> > .split(), no .startswith(), no sorted() or .sort(), ...!?
> >
> > If that's not what you mean by "string is atomic", I think you're
> > using very confusing terminology.
>
> I'm sorry if I didn't mention it, or didn't make it clear enough - it's
> all about layering.
>
> On level 0, you treat strings verbatim, and can write some subset of
> apps (my point is that even this level allows to write lot enough
> apps). Let's call this set A0.
>
> On level 1, you accept that there's some universal enough conventions
> for some chars, like space or newline. And you can write set of
> apps A1 > A0.
>

At heart, this is exactly what the Python 3 "str" type is. The universal
convention is "code points". It's got nothing to do with encodings, or
bytes. A Python string is simply a finite sequence of atomic code points -
it is indexable, and it has a length. Once you have that, everything is
layered on top of it. How the code points themselves are implemented is
opaque and irrelevant other than the memory and performance consequences of
the implementation decisions (for example, a string could be indexable by
iterating from the start until you find the nth code point).

Similarly the "bytes" type is a sequence of 8-bit bytes.

Encodings are simply a way to transport code points via a byte-oriented
transport.

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Nick Coghlan
On 6 June 2014 21:15, Paul Sokolovsky  wrote:
> Hello,
>
> On Thu, 5 Jun 2014 23:15:54 +1000
> Nick Coghlan  wrote:
>
>> On 5 June 2014 22:37, Paul Sokolovsky  wrote:
>> > On Thu, 5 Jun 2014 22:20:04 +1000
>> > Nick Coghlan  wrote:
>> >> problems caused by trusting the locale encoding to be correct, but
>> >> the startup code will need non-trivial changes for that to happen
>> >> - the C.UTF-8 locale may even become widespread before we get
>> >> there).
>> >
>> > ... And until those golden times come, it would be nice if Python
>> > did not force its perfect world model, which unfortunately is not
>> > based on surrounding reality, and let users solve their encoding
>> > problems themselves - when they need, because again, one can go
>> > quite a long way without dealing with encodings at all. Whereas now
>> > Python3 forces users to deal with encoding almost universally, but
>> > forcing a particular for all strings (which is again, doesn't
>> > correspond to the state of surrounding reality). I already hear
>> > response that it's good that users taught to deal with encoding,
>> > that will make them write correct programs, but that's a bit far
>> > away from the original aim of making it write "correct" programs
>> > easy and pleasant. (And definition of "correct" vary.)
>>
>> As I've said before in other contexts, find me Windows, Mac OS X and
>> JVM developers, or educators and scientists that are as concerned by
>> the text model changes as folks that are primarily focused on Linux
>> system (including network) programming, and I'll be more willing to
>> concede the point.
>
> Well, but this question reduces to finding out (or specifying) who are
> target audiences of Python. It always has been (with a bow to Guido)
> forpost of scientific users (and probably even if there was mass exodus
> of other categories of users will remain prominent in that role). But
> Python has always had its share as system scripting language among
> Perl-haters, and with Perl going flatline, I guess it's fair to say
> that Python is major system scripting and service implementation
> language.

Correct - and the efforts of a number of core developers are focused
on getting the Linux distros and major projects like OpenStack
migrated. If other Linux users say "I'm not switching to Python 3
until after my distro has switched their own Python applications
over", that's a perfectly reasonable course of action for them to
take. After all, that approach to the adoption of new Python versions
is a large part of why Python 2.6 is still so widely supported by
library and framework developers: enterprise Linux distros haven't
even finished migrating to Python 2.7 yet, let alone Python 3. (The
other reason is that the language moratorium that was applied to
Python 2.7 and 3.2 means that supporting back to Python 2.6 isn't that
much harder than supporting 2.7 at this point in time).

That said, the feedback from the early adopters of Python 3 on Linux
is proving invaluable, and Linux users in general will benefit from
their work as the distros move their infrastructure applications over.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Fri, 06 Jun 2014 20:11:27 +0900
"Stephen J. Turnbull"  wrote:

> Paul Sokolovsky writes:
> 
>  > That kinda means "string is atomic", instead of your "characters
>  > are atomic".
> 
> I would be very surprised if a language that behaved that way was
> called a "Python subset".  No indexing, no slicing, no regexps, no
> .split(), no .startswith(), no sorted() or .sort(), ...!?
> 
> If that's not what you mean by "string is atomic", I think you're
> using very confusing terminology.

I'm sorry if I didn't mention it, or didn't make it clear enough - it's
all about layering.

On level 0, you treat strings verbatim, and can write some subset of
apps (my point is that even this level allows to write lot enough
apps). Let's call this set A0.

On level 1, you accept that there's some universal enough conventions
for some chars, like space or newline. And you can write set of 
apps A1 > A0.

On level 2, you add len(), and - oh magic - you now can center a string
within fixed-size field, something you probably to as often as once a
month, so hopefully that will keep you busy for few.

On level 3, it indeed starts to smell Unicode, we get isdigit(),
isalpha(), which require long boring tables, which hopefully can be
compressed enough to fit in your pocket.

On level 4, it's pumping up, with tolower() and friends, tables for
which you carry around in suitcase.

On level 5, everything is Unicode, what a bliss! You can even start
pretending that no other levels exist (God created Unicode on a second
day).

On level 6, there're mind-boggling, ugly manual-use utilities to deal
with internals of "magic" "working on its own for everyone" encoding to
deal with stuff like code-point vs charecters vs surrogate pair
vs grapheme separation, etc.



So, once again, for me and some other people, it's not that bright idea
to shoot for level 5 if levels 0-4 exist and well-proven pragmatic
model. And level 6 is still there anyway.


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Thu, 5 Jun 2014 23:15:54 +1000
Nick Coghlan  wrote:

> On 5 June 2014 22:37, Paul Sokolovsky  wrote:
> > On Thu, 5 Jun 2014 22:20:04 +1000
> > Nick Coghlan  wrote:
> >> problems caused by trusting the locale encoding to be correct, but
> >> the startup code will need non-trivial changes for that to happen
> >> - the C.UTF-8 locale may even become widespread before we get
> >> there).
> >
> > ... And until those golden times come, it would be nice if Python
> > did not force its perfect world model, which unfortunately is not
> > based on surrounding reality, and let users solve their encoding
> > problems themselves - when they need, because again, one can go
> > quite a long way without dealing with encodings at all. Whereas now
> > Python3 forces users to deal with encoding almost universally, but
> > forcing a particular for all strings (which is again, doesn't
> > correspond to the state of surrounding reality). I already hear
> > response that it's good that users taught to deal with encoding,
> > that will make them write correct programs, but that's a bit far
> > away from the original aim of making it write "correct" programs
> > easy and pleasant. (And definition of "correct" vary.)
> 
> As I've said before in other contexts, find me Windows, Mac OS X and
> JVM developers, or educators and scientists that are as concerned by
> the text model changes as folks that are primarily focused on Linux
> system (including network) programming, and I'll be more willing to
> concede the point.

Well, but this question reduces to finding out (or specifying) who are
target audiences of Python. It always has been (with a bow to Guido)
forpost of scientific users (and probably even if there was mass exodus
of other categories of users will remain prominent in that role). But
Python has always had its share as system scripting language among
Perl-haters, and with Perl going flatline, I guess it's fair to say
that Python is major system scripting and service implementation
language.

To whom all features like memoryview, array.array, in-place
input operations, etc. cater? To scientists? I'm sure most of them are
just happy with stuffing "@jit" for their kernel functions. And
scientist who bother with memoryviews for their data structures are
system-level-ish programmers too.

So, no wonder that Linux crowd cries at Python3 - it makes doing simple
things unnecessarily complicated.

> Windows, Mac OS X, and the JVM are all opinionated about the text
> encodings to be used at platform boundaries (using UTF-16, UTF-8 and
> UTF-16, respectively). By contrast, Linux (or, more accurately, POSIX)
> says "well, it's configurable, but we won't provide a reliable
> mechanism for finding out what the encoding is. So either guess as

[]

Yes, I understand complexity of developing cross-platform language with
advanced features. By I may offer another look at all this activity:
Python3 was brave enough to do revolution in its own world (catching a
lot of its users by surprise), but surely not brave enough to do
revolution around itself, by saying something like "We choose ONE, the
most right, and even the most used (per bytes transferred) encoding as
our standard I/O encoding. Grow up or explicitly specify encoding which
you personally need.".

Surely, it didn't to that - it makes no sense to fight the world. But
then Python3 is sympathetic about Java's desire to use "UTF-16" instead
of "right" encoding, and no so about Unix desire to treat encodings
as a separate level from content (and treating Unicode by nothing else
as yet another arbitrary encoding, which it is formally, and will be
for a long time de-facto, however sad it is). So, maybe "cross-platform"
should have mean "don't do implicit conversions". Because see, Python2
had a problem with implicit encoding conversion when str and unicode
objects were mixed, and Python3 has problem with implicit conversions
whenever str is used at all.


Anyway, I appreciate detailed responses, and understand what you
(Python3 developers) are trying to achieve, and appreciate your work,
and hope it all work out. Each user has own concerns about Unicode.
Mine are efficiency and layering. But once MicroPython has UTF-8 support
I will be much more relaxed about it. Layering is harder to accept, but
hopefully can be tackled too both on own mind's and technical sides. I
hope other users will find their peace with Unicode too!


[]


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Stephen J. Turnbull
Paul Sokolovsky writes:

 > That kinda means "string is atomic", instead of your "characters are
 > atomic".

I would be very surprised if a language that behaved that way was
called a "Python subset".  No indexing, no slicing, no regexps, no
.split(), no .startswith(), no sorted() or .sort(), ...!?

If that's not what you mean by "string is atomic", I think you're
using very confusing terminology.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread Nick Coghlan
On 6 June 2014 19:31, Victor Stinner  wrote:

> Guido just wrote in the issue: "Actually for asyncio we have special
> dispensation to push new features to minor releases (until 3.5).
> Please push to 3.4 so the source code is the same everywhere (except
> selectors.py, which is not covered by the exception)."
>
> I disagree with Guido. I would prefer to start to maintain a different
> branch for Python 3.4, because I consider that only bugfixes should be
> applied to Python 3.4.

This is why PEP 411 was thrashed out: to let us split the dates of
"make broadly available in the standard library" and "get ultra
conservative with API changes". asyncio was added as a provisional
module, so it can still get new features in 3.4.x maintenance releases
- that's a far more minor change than the backwards compatibility
breaks permitted by the PEP.

The difference with selectors is that it was *not* added as a
provisional module - it's subject to all the normal stability
requirements.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread R. David Murray
On Fri, 06 Jun 2014 11:31:23 +0200, Victor Stinner  
wrote:
> Hi,
> 
> I added a new BaseEventLoop.is_closed() method to Tulip and Python 3.5
> to fix an issue (see Tulip issue 169 for the detail). The problem is
> that I don't want to add this method to Python 3.4 because usually we
> don't add new methods in minor versions of Python (future version
> 3.4.2 in this case).
> 
> Guido just wrote in the issue: "Actually for asyncio we have special
> dispensation to push new features to minor releases (until 3.5).
> Please push to 3.4 so the source code is the same everywhere (except
> selectors.py, which is not covered by the exception)."
> 
> I disagree with Guido. I would prefer to start to maintain a different
> branch for Python 3.4, because I consider that only bugfixes should be
> applied to Python 3.4.
> 
> It's not the first change that cannot be applied on Python 3.4 (only
> in Tulip and Python 3.5): the selectors module now also supports
> devpoll on Solaris. It's annoying because the Tulip script
> "update_stdlib.sh" used to synchronize Tulip and Python wants to
> replace Lib/selectors.py in Python 3.4. I have to revert the change each time.
> 
> I propose a new workflow: use Python default (future version 3.5) as
> the new asyncio "upstream". Bugfixes would be applied as other Python
> bugfixes: first in Python 3.4, than in Python 3.5. The
> "update_stdlib.sh" script of Tulip should be modified to copy files
> from Python default to Tulip (opposite of the current direction).
> 
> Workflow:
> 
> New feature: Python 3.5 => Tulip => Trollius
> Bugfix: Python 3.4 => Python 3.5 => Tulip => Trollius
> 
> I don't think that Tulip should have minor release just for bugfixes,
> it would be a pain to maintain. Tulip is a third party module, it
> doesn't have the same constraints than Python stdlib.
> 
> What do you think?

I don't have any opinion on the workflow.

My understanding is that part of the purpose of the "provisional"
designation is to allow faster evolution (read: fixing) of an API before
the library becomes non-provisional.  Thus I agree with Guido here, and
will be doing something similar with at least one of the minor provisional
email API features in 3.4.2 (unless I miss the cutoff again ... :(

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Greg Ewing

Steven D'Aprano wrote:
I don't know about car engine controllers, but presumably they have 
diagnostic ports, and they may sometimes output text. If they output 
text, then at least hypothetically car mechanics in Russia might prefer 
their car to output "правда" and "ложный" rather than "true" and 
"false".


From a bit of googling, it seems that engine controller
diagnostic ports typically speak some kind of binary
protocol. So it would be up to the software running on
whatever was plugged into the port to display the
information in the user's native language.

E.g. this document lists a big pile of hex byte values
and little or no text that I can see:

https://law.resource.org/pub/us/cfr/ibr/005/sae.j1979.2002.pdf

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Thu, 5 Jun 2014 22:38:13 +1000
Nick Coghlan  wrote:

> On 5 June 2014 22:10, Stefan Krah  wrote:
> > Paul Sokolovsky  wrote:
> >> In this regard, I'm glad to participate in mind-resetting
> >> discussion. So, let's reiterate - there's nothing like "the best",
> >> "the only right", "the only correct", "righter than", "more
> >> correct than" in CPython's implementation of Unicode storage. It
> >> is *arbitrary*. Well, sure, it's not arbitrary, but based on
> >> requirements, and these requirements match CPython's (implied)
> >> usage model well enough. But among all possible sets of
> >> requirements, CPython's requirements are no more valid that other
> >> possible. And other set of requirement fairly clearly lead to
> >> situation where CPython implementation is rejected as not correct
> >> for those requirements at all.
> >
> > Several core-devs have said that using UTF-8 for MicroPython is
> > perfectly okay. I also think it's the right choice and I hope that
> > you guys come up with a very efficient implementation.
> 
> Based on this discussion , I've also posted a draft patch aimed at
> clarifying the relevant aspects of the data model section of the
> language reference (http://bugs.python.org/issue21667).

Thanks, it's very much appreciated. Though, the discussion there opened
another can of worms. I'm sorry if I was somehow related to that, my
bringing in the formal language spec was more a rhetorical figure, a
response to people claiming O(1) requirement. So, it either should be
in spec, or spec should be treated as such - something not specified
means it's underspecified and implementation-dependent. I'm glad that
the last point now explicitly pronounced by BDFL in the last comment
of that ticket (http://bugs.python.org/issue21667#msg219824)

> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/pmiscml%40gmail.com



-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] asyncio/Tulip: use CPython as the new upstream

2014-06-06 Thread Victor Stinner
Hi,

I added a new BaseEventLoop.is_closed() method to Tulip and Python 3.5
to fix an issue (see Tulip issue 169 for the detail). The problem is
that I don't want to add this method to Python 3.4 because usually we
don't add new methods in minor versions of Python (future version
3.4.2 in this case).

Guido just wrote in the issue: "Actually for asyncio we have special
dispensation to push new features to minor releases (until 3.5).
Please push to 3.4 so the source code is the same everywhere (except
selectors.py, which is not covered by the exception)."

I disagree with Guido. I would prefer to start to maintain a different
branch for Python 3.4, because I consider that only bugfixes should be
applied to Python 3.4.

It's not the first change that cannot be applied on Python 3.4 (only
in Tulip and Python 3.5): the selectors module now also supports
devpoll on Solaris. It's annoying because the Tulip script
"update_stdlib.sh" used to synchronize Tulip and Python wants to
replace Lib/selectors.py in Python 3.4. I have to revert the change each time.

I propose a new workflow: use Python default (future version 3.5) as
the new asyncio "upstream". Bugfixes would be applied as other Python
bugfixes: first in Python 3.4, than in Python 3.5. The
"update_stdlib.sh" script of Tulip should be modified to copy files
from Python default to Tulip (opposite of the current direction).

Workflow:

New feature: Python 3.5 => Tulip => Trollius
Bugfix: Python 3.4 => Python 3.5 => Tulip => Trollius

I don't think that Tulip should have minor release just for bugfixes,
it would be a pain to maintain. Tulip is a third party module, it
doesn't have the same constraints than Python stdlib.

What do you think?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Paul Sokolovsky
Hello,

On Thu, 5 Jun 2014 22:21:30 +1000
Tim Delaney  wrote:

> On 5 June 2014 22:01, Paul Sokolovsky  wrote:
> 
> >
> > All these changes are what let me dream on and speculate on
> > possibility that Python4 could offer an encoding-neutral string type
> > (which means based on bytes)
> >
> 
> To me, an "encoding neutral string type" means roughly "characters are
> atomic", and the best representation we have for a "character" is a

And for me it means exactly what "encoding neutral string type" moniker
promises - that you should not make any assumption about its encoding.
That kinda means "string is atomic", instead of your "characters are
atomic". That's the most basic level, and you can write a big enough
set of applications using it - for example, get some information from
user, store in database, then show back to user at later time.

[]

> 
> Cheers,
> 
> Tim Delaney



-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Hrvoje Niksic

On 06/04/2014 05:52 PM, Mark Lawrence wrote:

On 04/06/2014 16:32, Steve Dower wrote:


If copying into a separate list is a problem (memory-wise), re.finditer('\\S+', 
string) also provides the same behaviour and gives me the sliced string, so 
there's no need to index for anything.



Out of idle curiosity is there anything that stops MicroPython, or any
other implementation for that matter, from providing views of a string
rather than copying every time?  IIRC memoryviews in CPython rely on the
buffer protocol at the C API level, so since strings don't support this
protocol you can't take a memoryview of them.  Could this actually be
implemented in the future, is the underlying C code just too
complicated, or what?



Memory view of Unicode strings is controversial for two reasons:

1. It exposes the internal representation of the string. If memoryviews 
of strings were supported in Python 3, PEP 393 would not have been 
possible (without breaking that feature).


2. Even if it were OK to expose the internal representation, it might 
not be what the users expect. For example, memoryview("Hrvoje") would 
return a view of a 6-byte buffer, while memoryview("Nikšić") would 
return a view of a 12-byte UCS-2 buffer. The user of a memory view might 
expect to get UCS-2 (or UCS-4, or even UTF-8) in all cases.


An implementation that decided to export strings as memory views might 
be forced to make a decision about internal representation of strings, 
and then stick to it.


The byte objects don't have these issues, which is why in Python 2.7 
memoryview("foo") works just fine, as does memoryview(b"foo") in Python 3.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Julian Taylor
On 06.06.2014 04:18, Sturla Molden wrote:
> On 05/06/14 22:51, Nathaniel Smith wrote:
> 
>> This gets evaluated as:
>>
>> tmp1 = a + b
>> tmp2 = tmp1 + c
>> result = tmp2 / c
>>
>> All these temporaries are very expensive. Suppose that a, b, c are
>> arrays with N bytes each, and N is large. For simple arithmetic like
>> this, then costs are dominated by memory access. Allocating an N byte
>> array requires the kernel to clear the memory, which incurs N bytes of
>> memory traffic.
> 
> It seems to be the case that a large portion of the run-time in Python
> code using NumPy can be spent in the kernel zeroing pages (which the
> kernel does for security reasons).
> 
> I think this can also be seen as a 'malloc problem'. It comes about
> because each new NumPy array starts with a fresh buffer allocated by
> malloc. Perhaps buffers can be reused?
> 
> Sturla
> 
> 

Caching memory inside of numpy would indeed solve this issue too. There
has even been a paper written on this which contains some more serious
benchmarks than the laplace case which runs on very old hardware (and
the inplace and out of place cases are actually not the same, one
computes array/scalar the other array * (1 / scalar)):

hiperfit.dk/pdf/Doubling.pdf
"The result is an improvement of as much as 2.29 times speedup, on
average 1.32 times speedup across a benchmark suite of 15 applications"

The problem with this approach is that it is already difficult enough to
handle memory in numpy. Having a cache that potentially stores gigabytes
of memory out of the users sight will just make things worse.

This would not be needed if we can come up with a way on how python can
help out numpy in eliding the temporaries.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Internal representation of strings and Micropython

2014-06-06 Thread Mark Lawrence

On 04/06/2014 16:52, Mark Lawrence wrote:

On 04/06/2014 16:32, Steve Dower wrote:


If copying into a separate list is a problem (memory-wise),
re.finditer('\\S+', string) also provides the same behaviour and gives
me the sliced string, so there's no need to index for anything.



Out of idle curiosity is there anything that stops MicroPython, or any
other implementation for that matter, from providing views of a string
rather than copying every time?  IIRC memoryviews in CPython rely on the
buffer protocol at the C API level, so since strings don't support this
protocol you can't take a memoryview of them.  Could this actually be
implemented in the future, is the underlying C code just too
complicated, or what?



Anybody?

--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


  1   2   >