Re: [Python-Dev] [Python-checkins] cpython (3.3): return NULL here

[Ronald Oussoren]

On 24 Jul, 2013, at 8:43, Gregory P. Smith  wrote:

> 
> On Tue, Jul 23, 2013 at 8:46 AM, Ronald Oussoren  
> wrote:
> 
> On 23 Jul, 2013, at 17:36, Christian Heimes  wrote:
> 
> > Am 23.07.2013 17:10, schrieb Benjamin Peterson:
> >>> PyErr_SetFromErrno() already and always returns NULL. Or do you prefer
> >>> to return NULL explicitly?
> >>
> >> It might always return NULL, but the compiler sees (PyObject *)NULL
> >> when this function returns dl_funcptr.
> >
> > Oh, you are right. I must have missed the compiler warning. How about we
> > turn type return and type assignment warnings into fatal errors?
> 
> That's probably possible with a '-Werror=' argument. But please consider
> issue 18211 before unconditionally adding such a flag, as that issue mentions
> new compiler flags also get used when compiling 3th-party extensions.
> 
> I guess there needs to be (yet) another CFLAGS_xxx variable in the Makefile 
> that
> gets added to $(CFLAGS) during the build of Python itself, but is ignored by
> distutils and sysconfig.
> 
> It seems fair to turn those on in 3.4 and require that third party extensions 
> clean up their code when porting from 3.3 to 3.4.

In this case its "just" code cleanup, the issue I filed (see above) is for 
another -Werror flag
that causes compile errors with some valid C99 code that isn't valid C89. 
That's good for
CPython itself because its source code is explicitly C89, but is not good when 
building 3th-party
extensions.

A proper fix requires tweaking the configure script, Makefile and distutils and 
that's not really
a fun prospect ;-)

Ronald

> 
> 
> Ronald
> 
> >
> > Christian
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com
> 
> ___
> Python-checkins mailing list
> python-check...@python.org
> http://mail.python.org/mailman/listinfo/python-checkins
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): return NULL here

[Antoine Pitrou]

Le Wed, 24 Jul 2013 09:01:30 +0200,
Ronald Oussoren  a écrit :
> 
> On 24 Jul, 2013, at 8:43, Gregory P. Smith  wrote:
> 
> > 
> > On Tue, Jul 23, 2013 at 8:46 AM, Ronald Oussoren
> >  wrote:
> > 
> > On 23 Jul, 2013, at 17:36, Christian Heimes 
> > wrote:
> > 
> > > Am 23.07.2013 17:10, schrieb Benjamin Peterson:
> > >>> PyErr_SetFromErrno() already and always returns NULL. Or do you
> > >>> prefer to return NULL explicitly?
> > >>
> > >> It might always return NULL, but the compiler sees (PyObject
> > >> *)NULL when this function returns dl_funcptr.
> > >
> > > Oh, you are right. I must have missed the compiler warning. How
> > > about we turn type return and type assignment warnings into fatal
> > > errors?
> > 
> > That's probably possible with a '-Werror=' argument. But please
> > consider issue 18211 before unconditionally adding such a flag, as
> > that issue mentions new compiler flags also get used when compiling
> > 3th-party extensions.
> > 
> > I guess there needs to be (yet) another CFLAGS_xxx variable in the
> > Makefile that gets added to $(CFLAGS) during the build of Python
> > itself, but is ignored by distutils and sysconfig.
> > 
> > It seems fair to turn those on in 3.4 and require that third party
> > extensions clean up their code when porting from 3.3 to 3.4.
> 
> In this case its "just" code cleanup, the issue I filed (see above)
> is for another -Werror flag that causes compile errors with some
> valid C99 code that isn't valid C89. That's good for CPython itself
> because its source code is explicitly C89, but is not good when
> building 3th-party extensions.

Agreed. We shouldn't impose specific error flags on 3rd-party extension
writers.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 3 as a Default in Linux Distros

[Bohuslav Kabrda]

Hi all,
in recent days, there has been a discussion on fedora-devel (see thread [1]) 
about moving to Python 3 as a default.
I'd really love to hear opinions on the matter from the upstream, mainly 
regarding these two points (that are not that clearly defined in my original 
proposal and have been formed during the discussion):

- Should we point /usr/bin/python to Python 3 when we make the move?
I know that pep 394 [2] deals with this and it says that /usr/bin/python may 
refer to Python 3 on some bleeding edge distributions - supposedly, this was 
added to the pep because of what Arch Linux did, not the other way round.
As the pep says, the recommendation of pointing /usr/bin/python to Python 2 may 
be changed after the Python 3 ecosystem is sufficiently mature. I'm wondering 
if there are any more specific criteria - list of big projects migrated/ported 
or something like that - or will this be judged by what I'd call "overall 
spirit" in Python community (I hope you know what I mean by this)?
In Fedora, we have two concerns that clash in this decision - being "First" 
(e.g. actively promote and use new technologies and also suggest them to our 
users) vs. not breaking user expectations. So we figured it'd be a good idea to 
ask upstream to get more opinions on this.

- What should user get after using "yum install python"?
There are basically few ways of coping with this:
1) Just keep doing what we do, eventually far in the future drop "python" 
package and never provide it again (= go on only with python3/python4/... while 
having "yum install python" do nothing).
2) Do what is in 1), but when "python" is dropped, use virtual provide (*) 
"python" for python3 package, so that "yum install python" installs python3.
3), 4) Rename python to python2 and {don't add, add} virtual provide "python" 
in the same way that is in 1), 2)
5) Rename python to python2 and python3 to python at one point. This makes 
sense to me from the traditional "one version in distro + possibly compat 
package shipping the old" approach in Linux, but some say that Python 2 and 
Python 3 are just different languages [3] and this should never be done.
All of the approaches have their pros and cons, but generally it is all about 
what user should get when he tries to install python - either nothing or 
python2 for now and python3 in future - and how we as a distro cope with that 
on the technical side (and when we should actually do the switch).
Just as a sidenote, IMO the package that gets installed as "python" (if any) 
should point to /usr/bin/python, which makes consider these two points very 
closely coupled.

Thank you all for your suggestions and opinions.
Slavek.

-- 
Regards,
Bohuslav "Slavek" Kabrda.

(*) You can think of a virtual provide as of a "packaging symlink" - adding 
"Provides: foo" to package makes "yum install foo" install this package, even 
if this package name is "bar".

[1] https://lists.fedoraproject.org/pipermail/devel/2013-July/186098.html
[2] http://www.python.org/dev/peps/pep-0394/
[3] https://lists.fedoraproject.org/pipermail/devel/2013-July/186186.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Chris Angelico]

On Wed, Jul 24, 2013 at 7:12 PM, Bohuslav Kabrda  wrote:
> - What should user get after using "yum install python"?
> There are basically few ways of coping with this:
> 1) Just keep doing what we do, eventually far in the future drop "python" 
> package and never provide it again (= go on only with python3/python4/... 
> while having "yum install python" do nothing).
> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*) 
> "python" for python3 package, so that "yum install python" installs python3.
> 3), 4) Rename python to python2 and {don't add, add} virtual provide "python" 
> in the same way that is in 1), 2)
> 5) Rename python to python2 and python3 to python at one point. This makes 
> sense to me from the traditional "one version in distro + possibly compat 
> package shipping the old" approach in Linux, but some say that Python 2 and 
> Python 3 are just different languages [3] and this should never be done.


If a user's (rather than dev's) perspective is of value, I'm a Debian
and Ubuntu user who wields Python as a multi-context weapon. Typing
"packagemanager install python" (for me that's apt-get, for you yum,
same diff) should get the one against which most other packages are
going to depend. Since your policy is to move people forward, that's
most likely going to be Python 3, unless you have a lot of packages
that simply haven't yet moved.

Your fifth option seems messy. I'd rather see python2 and python3 as
viable packages, and then use the virtual provide to symlink 'python'
to one of them, probably py3. So I'd vote for option 4. Python 2 and 3
aren't so different as to be called separate languages, there just
happen to be some backward-incompatible changes.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Dirkjan Ochtman]

On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:
> - What should user get after using "yum install python"?
> There are basically few ways of coping with this:
> 1) Just keep doing what we do, eventually far in the future drop "python" 
> package and never provide it again (= go on only with python3/python4/... 
> while having "yum install python" do nothing).
> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*) 
> "python" for python3 package, so that "yum install python" installs python3.
> 3), 4) Rename python to python2 and {don't add, add} virtual provide "python" 
> in the same way that is in 1), 2)
> 5) Rename python to python2 and python3 to python at one point. This makes 
> sense to me from the traditional "one version in distro + possibly compat 
> package shipping the old" approach in Linux, but some say that Python 2 and 
> Python 3 are just different languages [3] and this should never be done.
> All of the approaches have their pros and cons, but generally it is all about 
> what user should get when he tries to install python - either nothing or 
> python2 for now and python3 in future - and how we as a distro cope with that 
> on the technical side (and when we should actually do the switch).
> Just as a sidenote, IMO the package that gets installed as "python" (if any) 
> should point to /usr/bin/python, which makes consider these two points very 
> closely coupled.

On Gentoo we get python2 and python3 executables and have user-level
tools to change what the 'python' symlink points to. I think we
default to only having Python 3 installed (and Python 3 is even made
the default Python), though it's currently always the case that Python
2 gets pulled in by some core dependencies.

Cheers,

Dirkjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Paul Moore]

On 24 July 2013 10:12, Bohuslav Kabrda  wrote:

> - What should user get after using "yum install python"?
> There are basically few ways of coping with this:
> 1) Just keep doing what we do, eventually far in the future drop "python"
> package and never provide it again (= go on only with python3/python4/...
> while having "yum install python" do nothing).
> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*)
> "python" for python3 package, so that "yum install python" installs python3.
> 3), 4) Rename python to python2 and {don't add, add} virtual provide
> "python" in the same way that is in 1), 2)
> 5) Rename python to python2 and python3 to python at one point. This makes
> sense to me from the traditional "one version in distro + possibly compat
> package shipping the old" approach in Linux, but some say that Python 2 and
> Python 3 are just different languages [3] and this should never be done.
>

I'm not a Unix user, but IMO any approach that has the simple package name
and command "python" disappear or become deprecated, makes me sad. I think
that "yum install python" should install Python, and "python" should run
Python. Frankly, the differences between Python 2 and Python 3 are *not*
big enough that users writing simple scripts and applications for personal
or local use can't cope with this. Not everyone writes major libraries, and
it's the casual users you should be looking at.

There may be a debate over *when* this usage should give the user Python 3
rather than Python 2 (I'm very much an early adopter by instinct, so I'd
say sooner rather than later, but others may differ) but that's secondary.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Maciej Fijalkowski]

On Wed, Jul 24, 2013 at 12:36 PM, Paul Moore  wrote:
> On 24 July 2013 10:12, Bohuslav Kabrda  wrote:
>>
>> - What should user get after using "yum install python"?
>> There are basically few ways of coping with this:
>> 1) Just keep doing what we do, eventually far in the future drop "python"
>> package and never provide it again (= go on only with python3/python4/...
>> while having "yum install python" do nothing).
>> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*)
>> "python" for python3 package, so that "yum install python" installs python3.
>> 3), 4) Rename python to python2 and {don't add, add} virtual provide
>> "python" in the same way that is in 1), 2)
>> 5) Rename python to python2 and python3 to python at one point. This makes
>> sense to me from the traditional "one version in distro + possibly compat
>> package shipping the old" approach in Linux, but some say that Python 2 and
>> Python 3 are just different languages [3] and this should never be done.
>
>
> I'm not a Unix user, but IMO any approach that has the simple package name
> and command "python" disappear or become deprecated, makes me sad. I think
> that "yum install python" should install Python, and "python" should run
> Python. Frankly, the differences between Python 2 and Python 3 are *not* big
> enough that users writing simple scripts and applications for personal or
> local use can't cope with this. Not everyone writes major libraries, and
> it's the casual users you should be looking at.

the problem is not that. The problem is that people *use* major libraries a lot.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Brett Cannon]

On Wed, Jul 24, 2013 at 5:12 AM, Bohuslav Kabrda  wrote:

> Hi all,
> in recent days, there has been a discussion on fedora-devel (see thread
> [1]) about moving to Python 3 as a default.
> I'd really love to hear opinions on the matter from the upstream, mainly
> regarding these two points (that are not that clearly defined in my
> original proposal and have been formed during the discussion):
>
> - Should we point /usr/bin/python to Python 3 when we make the move?
> I know that pep 394 [2] deals with this and it says that /usr/bin/python
> may refer to Python 3 on some bleeding edge distributions - supposedly,
> this was added to the pep because of what Arch Linux did, not the other way
> round.
> As the pep says, the recommendation of pointing /usr/bin/python to Python
> 2 may be changed after the Python 3 ecosystem is sufficiently mature. I'm
> wondering if there are any more specific criteria - list of big projects
> migrated/ported or something like that - or will this be judged by what I'd
> call "overall spirit" in Python community (I hope you know what I mean by
> this)?
> In Fedora, we have two concerns that clash in this decision - being
> "First" (e.g. actively promote and use new technologies and also suggest
> them to our users) vs. not breaking user expectations. So we figured it'd
> be a good idea to ask upstream to get more opinions on this.
>
> - What should user get after using "yum install python"?
> There are basically few ways of coping with this:
> 1) Just keep doing what we do, eventually far in the future drop "python"
> package and never provide it again (= go on only with python3/python4/...
> while having "yum install python" do nothing).
> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*)
> "python" for python3 package, so that "yum install python" installs python3.
> 3), 4) Rename python to python2 and {don't add, add} virtual provide
> "python" in the same way that is in 1), 2)
> 5) Rename python to python2 and python3 to python at one point. This makes
> sense to me from the traditional "one version in distro + possibly compat
> package shipping the old" approach in Linux, but some say that Python 2 and
> Python 3 are just different languages [3] and this should never be done.
> All of the approaches have their pros and cons, but generally it is all
> about what user should get when he tries to install python - either nothing
> or python2 for now and python3 in future - and how we as a distro cope with
> that on the technical side (and when we should actually do the switch).
> Just as a sidenote, IMO the package that gets installed as "python" (if
> any) should point to /usr/bin/python, which makes consider these two points
> very closely coupled.
>

A similar discussion broke out when Arch Linux switched python to point to
python3. This led to http://www.python.org/dev/peps/pep-0394/ which says
have python2/python3, and have python point at whatever makes the most
sense to you based on your users and version uptake (option 3/4). The key,
though, is adding python2 and getting your code to use that
binary  specifically so that shifting the default name is more of a
convenience than something which might break existing code not ready for
the switch.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Lennart Regebro]

On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:
> - Should we point /usr/bin/python to Python 3 when we make the move?

No.

> - What should user get after using "yum install python"?

Will a base install include Python 3? If it does, I think yum install
python should mean python3, and hence already be installed.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[David Hutto]

Haven't been over to yum land in a while, but usually they stabilize the
distro in synergy to the python version. Ubuntu came with 2.7, and I just
use sudo-apt get install, for 3.2(or a gui package manager), and then can
use 3.2 with using the command python3.2 in command line, or you can
compile from source with a newer python not available with your distro's
package manager/repository.

It's really up to your distro's base linux, and python version decided on
top of a specific distro they define as compatible with the distros system.



On Wed, Jul 24, 2013 at 9:54 AM, Lennart Regebro  wrote:

> On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda 
> wrote:
> > - Should we point /usr/bin/python to Python 3 when we make the move?
>
> No.
>
> > - What should user get after using "yum install python"?
>
> Will a base install include Python 3? If it does, I think yum install
> python should mean python3, and hence already be installed.
>
> //Lennart
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dwightdhutto%40gmail.com
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com*
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Martin v. Löwis]

Am 24.07.13 11:12, schrieb Bohuslav Kabrda:
> - Should we point /usr/bin/python to Python 3 when we make the move?

This should depend on the answer to this question:
- for how long have you been providing /usr/bin/python2 binaries?

Users "should" have been explicit in declaring scripts as
/usr/bin/python2 for quite some time now, except that this would break
on distros which don't provide a python2 symlink.

Ideally, you "should" have warned users to be explicit with python2 if
the script wouldn't work on python3. But I'd wave this requirement, as
there is already the upstream PEP.

So (IMO) if it the last three Fedora releases had been providing python2
binaries, it should be allowed to switch python to be python3. The exact
number can be debated; it should depend on what releases are still in
active use so that a script would run on multiple releases.

Then you can tell users that possible breakage is easy to fix: just
install python2, and change the shebang line.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Toshio Kuratomi]

Note: I'm the opposite number to bkabrda in the discussion on the Fedora
Lists about how quickly we should be breaking end-user expectations of what
"python" means.

On Wed, Jul 24, 2013 at 09:34:11AM -0400, Brett Cannon wrote:
> 
> 
> 
> On Wed, Jul 24, 2013 at 5:12 AM, Bohuslav Kabrda  wrote:
> 
> Hi all,
> in recent days, there has been a discussion on fedora-devel (see thread
> [1]) about moving to Python 3 as a default.
> I'd really love to hear opinions on the matter from the upstream, mainly
> regarding these two points (that are not that clearly defined in my
> original proposal and have been formed during the discussion):
> 
Note that the proposal is for Fedora 22.  So the timeframe for making the
switch in development is approximately 8 months from now.  Timeframe for
that release to be public is October 2014.

> - Should we point /usr/bin/python to Python 3 when we make the move?
> I know that pep 394 [2] deals with this and it says that /usr/bin/python
> may refer to Python 3 on some bleeding edge distributions - supposedly,
> this was added to the pep because of what Arch Linux did, not the other 
> way
> round.
> As the pep says, the recommendation of pointing /usr/bin/python to Python 
> 2
> may be changed after the Python 3 ecosystem is sufficiently mature. I'm
> wondering if there are any more specific criteria - list of big projects
> migrated/ported or something like that - or will this be judged by what 
> I'd
> call "overall spirit" in Python community (I hope you know what I mean by
> this)?
> In Fedora, we have two concerns that clash in this decision - being 
> "First"
> (e.g. actively promote and use new technologies and also suggest them to
> our users) vs. not breaking user expectations. So we figured it'd be a 
> good
> idea to ask upstream to get more opinions on this.
> 
> - What should user get after using "yum install python"?
> There are basically few ways of coping with this:
> 1) Just keep doing what we do, eventually far in the future drop "python"
> package and never provide it again (= go on only with python3/python4/...
> while having "yum install python" do nothing).
> 2) Do what is in 1), but when "python" is dropped, use virtual provide (*)
> "python" for python3 package, so that "yum install python" installs
> python3.
> 3), 4) Rename python to python2 and {don't add, add} virtual provide
> "python" in the same way that is in 1), 2)
>
4) Is my preference: python package becomes python2; Virtual Provide: python
means you'd get the python package is what I'd promote for now.  Users still
expect python2 when they talk about "python".  At some point in the future,
people will come to pycon and talks will apply to python3 unless otherwise
specified.  People writing new blog posts will say "python" and the code
they use as samples won't run on the python2 interpreter.  Expecting for
that to be the case in 12 months seems premature.

> 5) Rename python to python2 and python3 to python at one point. This makes
> sense to me from the traditional "one version in distro + possibly compat
> package shipping the old" approach in Linux, but some say that Python 2 
> and
> Python 3 are just different languages [3] and this should never be done.
> All of the approaches have their pros and cons, but generally it is all
> about what user should get when he tries to install python - either 
> nothing
> or python2 for now and python3 in future - and how we as a distro cope 
> with
> that on the technical side (and when we should actually do the switch).
> Just as a sidenote, IMO the package that gets installed as "python" (if
> any) should point to /usr/bin/python, which makes consider these two 
> points
> very closely coupled.
> 
> 
> A similar discussion broke out when Arch Linux switched python to point to
> python3. This led to http://www.python.org/dev/peps/pep-0394/ which says have
> python2/python3, and have python point at whatever makes the most sense to you
> based on your users and version uptake (option 3/4).

  I think bkabrda is looking for some clarification on PEP-394.  My
reading and participation in the previous discussions lead me to believe
that while PEP-394 wants to be diplomatic, the message it wants to get
across is:

1) warn distributions that what Arch was doing was premature.
2) provide a means to get them to switch at roughly the same time (when the
   recommendation in the PEP is flipped to suggest linking /usr/bin/python
   to /usr/bin/python3)

This is especially my reading from the Recommendations section of the PEP.
Unfortunately, we're getting stuck in the Abstract section which has this
bullet point:

* python should refer to the same target as python2 but may refer to python3
on some bleeding edge distributions

Knowing the history, I read this in two parts:
* Recommendation to dist

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Nick Coghlan]

On 25 July 2013 01:26, Toshio Kuratomi  wrote:
> On Wed, Jul 24, 2013 at 09:34:11AM -0400, Brett Cannon wrote:
>> A similar discussion broke out when Arch Linux switched python to point to
>> python3. This led to http://www.python.org/dev/peps/pep-0394/ which says have
>> python2/python3, and have python point at whatever makes the most sense to 
>> you
>> based on your users and version uptake (option 3/4).
>
>   I think bkabrda is looking for some clarification on PEP-394.  My
> reading and participation in the previous discussions lead me to believe
> that while PEP-394 wants to be diplomatic, the message it wants to get
> across is:
>
> 1) warn distributions that what Arch was doing was premature.
> 2) provide a means to get them to switch at roughly the same time (when the
>recommendation in the PEP is flipped to suggest linking /usr/bin/python
>to /usr/bin/python3)
>
> This is especially my reading from the Recommendations section of the PEP.
> Unfortunately, we're getting stuck in the Abstract section which has this
> bullet point:
>
> * python should refer to the same target as python2 but may refer to python3
> on some bleeding edge distributions
>
> Knowing the history, I read this in two parts:
> * Recommendation to distributions: "python should refer to the same target
>   as python2".
> * Statement of fact: "but may refer to python3 on some bleeding edge
>   *ahem*Arch*ahem* distributions"
>
> However, other people are reading this as one recommendation for
> distributions:
>
> If you are a conservative, slow moving distro (like RHEL or Debian Stable)
> then you should point to python2.  If you are a fast moving distro like
> Arch or Fedora or Ubuntu then you should point to python3.
>
> So -- is there some opinion on which of these is correct?

It's the first one, but I can definitely see how it could be read as
the second one.

How's this for an updated wording in the abstract:

  * for the time being, all distributions should ensure that python
refers to the same target as python2
  * however, users should be aware that python refers to python3 on at
least Arch Linux (that change is
what prompted the creation of this PEP), so "python" should be
used in the shebang line only for
scripts that are source compatible with both Python 2 and 3

That's what the previous bullet point was *trying* to say, but it
wasn't very clear, since the first half of the sentence was aimed at
distributions, while the second was aimed at end users.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Nick Coghlan]

On 25 July 2013 01:41, Nick Coghlan  wrote:
> How's this for an updated wording in the abstract:
>
>   * for the time being, all distributions should ensure that python
> refers to the same target as python2
>   * however, users should be aware that python refers to python3 on at
> least Arch Linux (that change is
> what prompted the creation of this PEP), so "python" should be
> used in the shebang line only for
> scripts that are source compatible with both Python 2 and 3
>
> That's what the previous bullet point was *trying* to say, but it
> wasn't very clear, since the first half of the sentence was aimed at
> distributions, while the second was aimed at end users.

I'll also note that this is deliberately consistent with the behaviour
of the "py" launcher on Windows (as defined in PEP 397): if you invoke
that from the command with both Python 2.x and Python 3.x installed,
it will default to using the most recently released version of Python
2, *unless* you pass a "-3" on the command line, or the shebang line
indicates the script is a Python 3 script.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Lennart Regebro]

On Wed, Jul 24, 2013 at 3:54 PM, Lennart Regebro  wrote:
> On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:
>> - Should we point /usr/bin/python to Python 3 when we make the move?
>
> No.

To be more explicit. I think it's perfectly fine to not provide a
/usr/bin/python at all, but I think pointing it to Python 3 will
provide unhelpful error messages. Also having a /usr/bin/python that
does nothing but say "You should use either /usr/bin/python2 or
/usr/bin/python3" would work.

But pointing it to Python 3 will make for unhelpful and confusing
error messages for anyone having Python scripts using /usr/bin/python,
which is the majority of people having python scripts at all.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Barry Warsaw]

On Jul 24, 2013, at 05:12 AM, Bohuslav Kabrda wrote:

>in recent days, there has been a discussion on fedora-devel (see thread [1])
>about moving to Python 3 as a default.

I've been lurking via Gmane. :)

>- Should we point /usr/bin/python to Python 3 when we make the move?

Over in Debian (inherited by Ubuntu), we have no plans to do this.  Currently,
/usr/bin/python points to pythonX.Y which for us is 2.7.  Our /usr/bin/python2
symlink also points to python2.7, and our /usr/bin/python3 points to python3.3.

I'm very much in favor of promoting Python 3 adoption through the entire
ecosystem.  We've been working in Ubuntu to banish Python 2 off the images,
eventually only in universe by the next LTS (14.04 - in April 2014).  For
practical reasons, I don't think we'll quite get there, but it's good to have
goals, and I've been really impressed with the progress made by distro
developers and upstreams.

"Porting is one honking great idea -- let's do more of those!"

We have a mandate that all new work (certainly where Canonical is the upstream
for Ubuntu packages) should be Python 3 unless there are specific dependencies
for which porting is impractical and there are no alternatives.  We want no
Python 2 on the Ubuntu Touch images.  Some of the new work will retain Python
2 compatibility for now, again for practical reasons.

Code we control is the easy part.  There is just so much out there that isn't
even in the distro, stuff that no distro maintainer will ever even know about
(unless we break stuff ;), and all of that is expecting /usr/bin/python to be
Python 2.  If they want Python 3, they are explicitly shebanging
/usr/bin/python3.  I think Nick in one of the Fedora threads correctly
identified the vast number of one-off scripts out there.

At this time, I think keeping /usr/bin/python pointing to Python 2 (2.7
preferably) strikes the right balance between what we in Python land would
like to see happen, with the least amount of pain, and friendly encouragement
to our distro users.

I hope someday we can remember Python 2 as fondly as we remember 1.5.2
(1.6.1).  I think there's a ton of things we can do to get us
there, but forcing our users through unnecessary pain too early won't help the
cause.  The last year or so has really shown a very positive shift toward
Python 3, in code, mindshare, and enthusiasm.

>I know that pep 394 [2] deals with this and it says that /usr/bin/python may
>refer to Python 3 on some bleeding edge distributions - supposedly, this was
>added to the pep because of what Arch Linux did, not the other way round.

Correct.  I think we should consider rewording the text to make this more
clear.

>As the pep says, the recommendation of pointing /usr/bin/python to Python 2
>may be changed after the Python 3 ecosystem is sufficiently mature. I'm
>wondering if there are any more specific criteria - list of big projects
>migrated/ported or something like that - or will this be judged by what I'd
>call "overall spirit" in Python community (I hope you know what I mean by
>this)?

In the discussions I've had at Pycon and various other forum, it seems like
Python 2.7's scheduled security-only transition in May 2015 would be an
appropriate time to make the switch.  If there's broad agreement to do this,
we have plenty of time to prepare our users, and all of our software stacks.

>- What should user get after using "yum install python"?

I can only say that for Debian and Ubuntu, `apt-get install python` will get
you the default Python 2 version (e.g. 2.7).  `apt-get install python3` will
get you the default Python 3 version (e.g. 3.3).  I don't think changing this
really provides much user benefit, and I've not seen any complaints/bugs
indicating user confusion or frustration.  I'd advocate for being user driven
here, i.e. IIABDFI[1].

>All of the approaches have their pros and cons, but generally it is all about
>what user should get when he tries to install python - either nothing or
>python2 for now and python3 in future - and how we as a distro cope with that
>on the technical side (and when we should actually do the switch).

I'm glad Arch is out there hacking at the vines.  For whatever part I can play
and for whatever my opinion matters, I would like to keep the Fedora/RH and
Debian/Ubuntu ecosystem in sync.  PEP 394 is the instrument we should rally
around.

Cheers,
-Barry

[1] The secret 20th Zen of Python?  If It Ain't Broke Don't Fix It.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Laurent Gautier]

On 07/24/2013 05:56 PM, Lennart Regebro wrote:

On Wed, Jul 24, 2013 at 3:54 PM, Lennart Regebro  wrote:

On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:

- Should we point /usr/bin/python to Python 3 when we make the move?

No.

To be more explicit. I think it's perfectly fine to not provide a
/usr/bin/python at all, but I think pointing it to Python 3 will
provide unhelpful error messages. Also having a /usr/bin/python that
does nothing but say "You should use either /usr/bin/python2 or
/usr/bin/python3" would work.

But pointing it to Python 3 will make for unhelpful and confusing
error messages for anyone having Python scripts using /usr/bin/python,
which is the majority of people having python scripts at all.


I guess that the underlying idea is to make the majority of python 
scripts working with Python 3.


What about the following idea ?

- errors that are typical of "Python 2 script running with Python 
3"-specific are probably limited (e.g., use of unicode, use of xrange, 
etc...)


- /usr/bin/python could be a wrapper that tries to run it with Python 3 
and display an informative message if the errors above are occurring
(e.g., "Python 3 is the current release, and this script might be 
designed for the older Python 2 - please explicitly use python2 if you 
need it"


- /usr/bin/python could issue a warning in any case about the need to be 
explicit about python2/python3




//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Martin v. Löwis]

Am 24.07.13 17:56, schrieb Lennart Regebro:> On Wed, Jul 24, 2013 at
3:54 PM, Lennart Regebro  wrote:
>> On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda
 wrote:
>>> - Should we point /usr/bin/python to Python 3 when we make the move?
>>
>> No.
>
> To be more explicit. I think it's perfectly fine to not provide a
> /usr/bin/python at all

I disagree. For interactive use, it's good if there is something called
"python" in the path.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Chris Angelico]

On Thu, Jul 25, 2013 at 2:21 AM, Laurent Gautier  wrote:
> - errors that are typical of "Python 2 script running with Python
> 3"-specific are probably limited (e.g., use of unicode, use of xrange,
> etc...)
>

The most common, in interactive scripts at least, is likely to be:

>>> print "Hello, world!"
SyntaxError: invalid syntax

How helpful it's possible to make that one, I don't know. Is it safe
to presume that it's more likely a syntax error will come from an
interpreter version mismatch than a code bug?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Brian Curtin]

On Wed, Jul 24, 2013 at 10:56 AM, Lennart Regebro  wrote:
> On Wed, Jul 24, 2013 at 3:54 PM, Lennart Regebro  wrote:
>> On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:
>>> - Should we point /usr/bin/python to Python 3 when we make the move?
>>
>> No.
>
> To be more explicit. I think it's perfectly fine to not provide a
> /usr/bin/python at all, but I think pointing it to Python 3 will
> provide unhelpful error messages.

-1. For one example, this is going to add non-productive and
potentially confusing steps before any educator can use Python to
teach anything.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Barry Warsaw]

On Jul 24, 2013, at 08:26 AM, Toshio Kuratomi wrote:

>  I think bkabrda is looking for some clarification on PEP-394.  My
>reading and participation in the previous discussions lead me to believe
>that while PEP-394 wants to be diplomatic, the message it wants to get
>across is:
>
>1) warn distributions that what Arch was doing was premature.
>2) provide a means to get them to switch at roughly the same time (when the
>   recommendation in the PEP is flipped to suggest linking /usr/bin/python
>   to /usr/bin/python3)

+1

>This is especially my reading from the Recommendations section of the PEP.
>Unfortunately, we're getting stuck in the Abstract section which has this
>bullet point:
>
>* python should refer to the same target as python2 but may refer to python3
>on some bleeding edge distributions

Yes, that wording is confusing and should be fixed.

>Knowing the history, I read this in two parts:
>* Recommendation to distributions: "python should refer to the same target
>  as python2".
>* Statement of fact: "but may refer to python3 on some bleeding edge
>  *ahem*Arch*ahem* distributions"
>
>However, other people are reading this as one recommendation for
>distributions:
>
>If you are a conservative, slow moving distro (like RHEL or Debian Stable)
>then you should point to python2.  If you are a fast moving distro like
>Arch or Fedora or Ubuntu then you should point to python3.
>
>So -- is there some opinion on which of these is correct?

You know how I'm going to answer. :)

I like the way you've split that out into recommendations and
statements-of-fact.

>> The key, though, is adding python2 and getting your code to use that
>> binary  specifically so that shifting the default name is more of a
>> convenience than something which might break existing code not ready for
>> the switch.
>
>  yeah, this is something that I think should be in the distribution's
>one year plan regardless of whether the binary is switched now or not.

We need to do that in Debian.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Barry Warsaw]

On Jul 25, 2013, at 01:41 AM, Nick Coghlan wrote:

>How's this for an updated wording in the abstract:
>
>  * for the time being, all distributions should ensure that python
>refers to the same target as python2
>  * however, users should be aware that python refers to python3 on at
>least Arch Linux (that change is
>what prompted the creation of this PEP), so "python" should be
>used in the shebang line only for
>scripts that are source compatible with both Python 2 and 3

+1

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Terry Reedy]

On 7/24/2013 5:12 AM, Bohuslav Kabrda wrote:

Hi all, in recent days, there has been a discussion on fedora-devel
(see thread [1]) about moving to Python 3 as a default.


Default-shift is a known natural language phenomenon.
https://en.wikipedia.org/wiki/Retronym
It is inevitably messy in the middle. Even worse for us, "Python' is 
both a natural and artificial language term and the default meaning will 
not necessarily be synchronized across the two domains.


Not being a current *nix user, I have no opinion on the specific 
alternatives. However, I am concerned about newcomers being mis-served 
by being mislead into thinking that 'Python' is python 2 rather python 
3. We see this in some newbie posts on Python list. (This is especially 
a concern when default Python is still, say, 2.5.)  I have no idea, 
though, how to avoid breaking scripts while directing newcomers in the 
proper direction.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Gustavo Carneiro]

On Wed, Jul 24, 2013 at 6:04 PM, Terry Reedy  wrote:

> On 7/24/2013 5:12 AM, Bohuslav Kabrda wrote:
>
>> Hi all, in recent days, there has been a discussion on fedora-devel
>> (see thread [1]) about moving to Python 3 as a default.
>>
>
> Default-shift is a known natural language phenomenon.
> https://en.wikipedia.org/wiki/**Retronym
> It is inevitably messy in the middle. Even worse for us, "Python' is both
> a natural and artificial language term and the default meaning will not
> necessarily be synchronized across the two domains.
>
> Not being a current *nix user, I have no opinion on the specific
> alternatives. However, I am concerned about newcomers being mis-served by
> being mislead into thinking that 'Python' is python 2 rather python 3. We
> see this in some newbie posts on Python list. (This is especially a concern
> when default Python is still, say, 2.5.)  I have no idea, though, how to
> avoid breaking scripts while directing newcomers in the proper direction.


+1

Maybe we could make interactive python2 scripts print a big warning
advising users to use python 3 instead of 2, if possible?  I mean, if
stdout is a TTY.  This way, even if 'python' is python2, newbies will see
the warning.  I think every developer (newbie or not) sooner or later will
try a python interactive shell, and so will see the warning.  It is
important also for experienced python developers to be told explicitly that
they should be thinking about switching to python 3, at some point.



>
>
> --
> Terry Jan Reedy
>
>
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> gjcarneiro%40gmail.com
>



-- 
Gustavo J. A. M. Carneiro
"The universe is always one step beyond logic." -- Frank Herbert
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Laurent Gautier]

On 07/24/2013 06:30 PM, Chris Angelico wrote:

On Thu, Jul 25, 2013 at 2:21 AM, Laurent Gautier  wrote:

- errors that are typical of "Python 2 script running with Python
3"-specific are probably limited (e.g., use of unicode, use of xrange,
etc...)


The most common, in interactive scripts at least, is likely to be:


print "Hello, world!"

SyntaxError: invalid syntax

How helpful it's possible to make that one, I don't know. Is it safe
to presume that it's more likely a syntax error will come from an
interpreter version mismatch than a code bug?


The wrapper in /usr/bin/python:
- could use what is in 2to3. I think that most of the cases are solved 
there.
- whenever interactive, could have an intermediate layer between the 
input in the console and execution.





ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Stefan Behnel]

Gustavo Carneiro, 24.07.2013 19:16:
> On Wed, Jul 24, 2013 at 6:04 PM, Terry Reedy wrote:
>> On 7/24/2013 5:12 AM, Bohuslav Kabrda wrote:
>>> Hi all, in recent days, there has been a discussion on fedora-devel
>>> (see thread [1]) about moving to Python 3 as a default.
>>
>> Not being a current *nix user, I have no opinion on the specific
>> alternatives. However, I am concerned about newcomers being mis-served by
>> being mislead into thinking that 'Python' is python 2 rather python 3. We
>> see this in some newbie posts on Python list. (This is especially a concern
>> when default Python is still, say, 2.5.)  I have no idea, though, how to
>> avoid breaking scripts while directing newcomers in the proper direction.
> 
> Maybe we could make interactive python2 scripts print a big warning
> advising users to use python 3 instead of 2, if possible?

Well, what I think could be done is to have the Py2 interpreter output a
warning if an unpostfixed shebang is found in an executed script. That
would eventually push users into either using the explicit "/usr/bin/env
python2", or port to Py3 straight away (or both).

Stefan


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

On Tue, Jul 23, 2013 at 4:21 PM, Richard Oudkerk  wrote:
> On 23/07/2013 11:45pm, Victor Stinner wrote:
>>
>> Said differently: the HANDLE_FLAG_INHERIT flag only has an effect on
>> *handles*, as indicated in its name. On Windows, file*descriptors*
>> are never inherited (are always closed) in child processes. I don't
>> think that it is possible to inherit file descriptors on Windows.
>
>
> Actually, you can inherit fds if you use os.spawnv() instead of
> subprocess.Popen().

Wow. Indeed you can -- I just tested this myself. How is this
accomplished? I guess the CRT has a backchannel to talk to itself when
it creates a process using spawn*?

This is the only reason I can think of for the odd default in the CRT
of opening file descriptors inheritable by default, which Victor
discovered. (But it doesn't explain why os.pipe() creates
uninheritable fds.)

If it weren't for this I would definitely vote to change the default
on Windows throughout the stdlib to create file descriptors whose
handles aren't inheritable. (Perhaps with a different policy for
stdin/stdout/stderr, which seem to be treated specially at the handle
level.)

I'm about ready to give up hope that we'll ever have a decent way to
deal with this.

But I'm also ready to propose that all this is such a mess that we
*should* change the default fd/handle inheritance to False, *across
platforms*, and damn the torpedoes -- i.e. accept breaking all
existing 3rd party UNIX code for subprocess creation that bypasses the
subprocess module, as well as breaking uses of os.spawn*() on both
platforms that depend on FD inheritance beyond stdin/stdout/stderr).

With the new, sane default, all we need instead of PEP 446 is a way to
make an FD inheritable after it's been created, which can be a single
os.make_inheritable(fd) call that you must apply to the fileno() of
the stream or socket object you want inherited (or directly to a FD
you created otherwise, e.g. with os.pipe()). On Windows, this should
probably only work with os.spawn*(), since otherwise you need *handle*
inheritance, not *FD* inheritance, and that's a non-portable concept
anyway.

We can fix multiprocessing any anything else in the stdlib that this
breaks, I presume.

To reduce the need for 3rd party subprocess creation code, we should
have better daemon creation code in the stdlib -- I wrote some damn
robust code for this purpose in my previous job, but it never saw the
light of day.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Richard Oudkerk]

> Wow. Indeed you can -- I just tested this myself. How is this
> accomplished? I guess the CRT has a backchannel to talk to itself when
> it creates a process using spawn*?

CreateProcess() takes a STARTUPINFO argument with undocumented fields 
cbReserved2, lpReserved2.  They are used to pass an array of fds.


> But I'm also ready to propose that all this is such a mess that we

*should* change the default fd/handle inheritance to False, *across
platforms*, and damn the torpedoes -- i.e. accept breaking all
existing 3rd party UNIX code for subprocess creation that bypasses the
subprocess module, as well as breaking uses of os.spawn*() on both
platforms that depend on FD inheritance beyond stdin/stdout/stderr).


+1


We can fix multiprocessing any anything else in the stdlib that this
breaks, I presume.


In the experimental branch of multiprocessing, child processes no longer 
inherit unnecessary handles.


--
Richard
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

On Wed, Jul 24, 2013 at 11:13 AM, Richard Oudkerk  wrote:
>> Wow. Indeed you can -- I just tested this myself. How is this
>> accomplished? I guess the CRT has a backchannel to talk to itself when
>> it creates a process using spawn*?
>
> CreateProcess() takes a STARTUPINFO argument with undocumented fields
> cbReserved2, lpReserved2.  They are used to pass an array of fds.

Does it also inherit sockets (which take up a different namespace than
regular FDs in CRT, unlike UNIX)?

>> But I'm also ready to propose that all this is such a mess that we
>>
>> *should* change the default fd/handle inheritance to False, *across
>> platforms*, and damn the torpedoes -- i.e. accept breaking all
>> existing 3rd party UNIX code for subprocess creation that bypasses the
>> subprocess module, as well as breaking uses of os.spawn*() on both
>> platforms that depend on FD inheritance beyond stdin/stdout/stderr).
>
> +1

Thanks! This was a difficult conclusion to come to. "Damn the
torpedoes" is occasionally a nice meme. :-(

>> We can fix multiprocessing any anything else in the stdlib that this
>> breaks, I presume.
>
> In the experimental branch of multiprocessing, child processes no longer
> inherit unnecessary handles.

And if we default individual handles to uninheritable, we can
presumably fix the ones that multiprocessing creates with the express
purpose of being inherited easily. (If it even uses that -- I haven't
read the source code, maybe it uses named pipes? :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Richard Oudkerk]

On 24/07/2013 7:17pm, Guido van Rossum wrote:
> Does it also inherit sockets (which take up a different namespace than
> regular FDs in CRT, unlike UNIX)?

Not reliably.  Processes created with CreateProcess() seems to inherit 
socket handles just like normal handles on my computer, but on some 
other computers -- with the same Windows version! -- it appears not to 
work.  See http://bugs.python.org/issue17399.


I think WSADuplicateSocket() should be used instead.

> And if we default individual handles to uninheritable, we can
> presumably fix the ones that multiprocessing creates with the express
> purpose of being inherited easily. (If it even uses that -- I haven't
> read the source code, maybe it uses named pipes?

multiprocessing never really needs to create any inheritable handles: it 
can use DuplicateHandle() to transfer each handle directly to the child 
process.


--
Richard

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Toshio Kuratomi]

On Wed, Jul 24, 2013 at 12:42:09PM -0400, Barry Warsaw wrote:
> On Jul 25, 2013, at 01:41 AM, Nick Coghlan wrote:
> 
> >How's this for an updated wording in the abstract:
> >
> >  * for the time being, all distributions should ensure that python
> >refers to the same target as python2
> >  * however, users should be aware that python refers to python3 on at
> >least Arch Linux (that change is
> >what prompted the creation of this PEP), so "python" should be
> >used in the shebang line only for
> >scripts that are source compatible with both Python 2 and 3
> 
> +1
> 
+1 as well.  Much clearer.

-Toshio


pgpgh3MdAJ43H.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Lennart Regebro]

On Wed, Jul 24, 2013 at 7:22 PM, Laurent Gautier  wrote:
> The wrapper in /usr/bin/python:
> - could use what is in 2to3. I think that most of the cases are solved
> there.

Only the most trivial cases are solved completely by 2to3, and the
error messages you would get would be hard to understand, and running
the scripts would be very slow.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Victor Stinner]

2013/7/24 Richard Oudkerk :
>> Wow. Indeed you can -- I just tested this myself. How is this
>> accomplished? I guess the CRT has a backchannel to talk to itself when
>> it creates a process using spawn*?
>
> CreateProcess() takes a STARTUPINFO argument with undocumented fields
> cbReserved2, lpReserved2.  They are used to pass an array of fds.

So would it be possible to implement the pass_fds parameter of
subprocess using spawnl() or the undocumented fields?

And is it possible to close all handles except one (implement "pass_handles")?

The idea would be to use something subprocess.Popen(cmd,
pass_fds=[pipe_rfd], close_fds=True) or subprocess.Popen(cmd,
pass_handles=[pipe_rhandle], close_handles=True) instead of
subprocess.Popen(cmd, close_fds=False).

> In the experimental branch of multiprocessing, child processes
> no longer inherit unnecessary handles.

Where is this branch? How did you create the channel between the
manager and the worker?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Antoine Pitrou]

On Wed, 24 Jul 2013 10:56:05 -0700
Guido van Rossum  wrote:
> 
> But I'm also ready to propose that all this is such a mess that we
> *should* change the default fd/handle inheritance to False, *across
> platforms*, and damn the torpedoes -- i.e. accept breaking all
> existing 3rd party UNIX code for subprocess creation that bypasses the
> subprocess module, as well as breaking uses of os.spawn*() on both
> platforms that depend on FD inheritance beyond stdin/stdout/stderr).

So I suppose you mean "change it to False except for
stdin/stdout/stderr"?

> With the new, sane default, all we need instead of PEP 446 is a way to
> make an FD inheritable after it's been created, which can be a single
> os.make_inheritable(fd) call that you must apply to the fileno() of
> the stream or socket object you want inherited (or directly to a FD
> you created otherwise, e.g. with os.pipe()). On Windows, this should
> probably only work with os.spawn*(), since otherwise you need *handle*
> inheritance, not *FD* inheritance, and that's a non-portable concept
> anyway.

I'm not sure how *fd* inheritance could work without *handle*
inheritance.
(since a fd seems to just be a proxy for a handle)

> To reduce the need for 3rd party subprocess creation code, we should
> have better daemon creation code in the stdlib -- I wrote some damn
> robust code for this purpose in my previous job, but it never saw the
> light of day.

What do you call "daemon"? An actual Unix-like daemon?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

On Wed, Jul 24, 2013 at 2:57 PM, Antoine Pitrou  wrote:
> On Wed, 24 Jul 2013 10:56:05 -0700
> Guido van Rossum  wrote:
>>
>> But I'm also ready to propose that all this is such a mess that we
>> *should* change the default fd/handle inheritance to False, *across
>> platforms*, and damn the torpedoes -- i.e. accept breaking all
>> existing 3rd party UNIX code for subprocess creation that bypasses the
>> subprocess module, as well as breaking uses of os.spawn*() on both
>> platforms that depend on FD inheritance beyond stdin/stdout/stderr).
>
> So I suppose you mean "change it to False except for
> stdin/stdout/stderr"?

Hm, that's a tricky detail. I expect that on UNIX those are pre-opened
and inherited from the shell, and we should not change their cloexec
status; IIRC on Windows inheritance of the stdin/out/err handles is
guided by a separate flag to CreateProcess().

>> With the new, sane default, all we need instead of PEP 446 is a way to
>> make an FD inheritable after it's been created, which can be a single
>> os.make_inheritable(fd) call that you must apply to the fileno() of
>> the stream or socket object you want inherited (or directly to a FD
>> you created otherwise, e.g. with os.pipe()). On Windows, this should
>> probably only work with os.spawn*(), since otherwise you need *handle*
>> inheritance, not *FD* inheritance, and that's a non-portable concept
>> anyway.
>
> I'm not sure how *fd* inheritance could work without *handle*
> inheritance.
> (since a fd seems to just be a proxy for a handle)

The MS spawn() implementation takes care of this -- I presume they
make the handles inheritable (I think some of the bugs probably refer
to this fact) and then use the "internal" interface to pass the
mapping from FDs to handles to the subprocess.

>> To reduce the need for 3rd party subprocess creation code, we should
>> have better daemon creation code in the stdlib -- I wrote some damn
>> robust code for this purpose in my previous job, but it never saw the
>> light of day.
>
> What do you call "daemon"? An actual Unix-like daemon?

Yeah, a background process with parent PID 1 and not associated with
any terminal group.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Antoine Pitrou]

On Wed, 24 Jul 2013 15:25:50 -0700
Guido van Rossum  wrote:
> >> To reduce the need for 3rd party subprocess creation code, we should
> >> have better daemon creation code in the stdlib -- I wrote some damn
> >> robust code for this purpose in my previous job, but it never saw the
> >> light of day.
> >
> > What do you call "daemon"? An actual Unix-like daemon?
> 
> Yeah, a background process with parent PID 1 and not associated with
> any terminal group.

But is that relevant to the PEP? A daemon only uses fork(), not exec().
Or have I misunderstood your concern?

Regards

Antoine.
 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Richard Oudkerk]

On 24/07/2013 10:50pm, Victor Stinner wrote:

So would it be possible to implement the pass_fds parameter of
subprocess using spawnl() or the undocumented fields?


Not in a non-racy way.

spawnv() calls CreateProcess() with bInheritHandles=TRUE, so *all* 
inheritable handles are inherited by the child.


The passing of the arrays of fds just makes the fds in the child process 
match the fds in the parent.


If you have a Visual Studio installed then the relevant code is in

   .../Microsoft Visual Studio 10.0/VC/crt/src/dospawn.c


And is it possible to close all handles except one (implement "pass_handles")?


I don't know how to do that.


In the experimental branch of multiprocessing, child processes
no longer inherit unnecessary handles.


Where is this branch? How did you create the channel between the
manager and the worker?


http://hg.python.org/sandbox/sbt/

The parent creates a pipe and starts the child process.  The pid of the 
parent, and the handle for the read end of the pipe are passed on the 
command line.  Then the child "steals" the handle from the parent using 
OpenProcess() and DuplicateHandle() using the DUPLICATE_CLOSE_SOURCE flag.


--
Richard

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Greg Ewing]

Chris Angelico wrote:

print "Hello, world!"

SyntaxError: invalid syntax

Is it safe
to presume that it's more likely a syntax error will come from an
interpreter version mismatch than a code bug?


Maybe look at sys.args[0], and if it ends in "python" with
no version number, add something to the error message about
possibly using the wrong version?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Eric V. Smith]

On 7/24/2013 6:25 PM, Guido van Rossum wrote:
>>> To reduce the need for 3rd party subprocess creation code, we should
>>> have better daemon creation code in the stdlib -- I wrote some damn
>>> robust code for this purpose in my previous job, but it never saw the
>>> light of day.
>>
>> What do you call "daemon"? An actual Unix-like daemon?
> 
> Yeah, a background process with parent PID 1 and not associated with
> any terminal group.

There's PEP 3143 and https://pypi.python.org/pypi/python-daemon. I've
used it often, with great success.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Nick Coghlan]

On 25 Jul 2013 05:30, "Toshio Kuratomi"  wrote:
>
> On Wed, Jul 24, 2013 at 12:42:09PM -0400, Barry Warsaw wrote:
> > On Jul 25, 2013, at 01:41 AM, Nick Coghlan wrote:
> >
> > >How's this for an updated wording in the abstract:
> > >
> > >  * for the time being, all distributions should ensure that python
> > >refers to the same target as python2
> > >  * however, users should be aware that python refers to python3 on at
> > >least Arch Linux (that change is
> > >what prompted the creation of this PEP), so "python" should be
> > >used in the shebang line only for
> > >scripts that are source compatible with both Python 2 and 3
> >
> > +1
> >
> +1 as well.  Much clearer.

OK, unless someone gets to it before me, I'll update it tonight.

Cheers,
Nick.

>
> -Toshio
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Cameron Simpson]

On 25Jul2013 00:35, Antoine Pitrou  wrote:
| On Wed, 24 Jul 2013 15:25:50 -0700
| Guido van Rossum  wrote:
| > >> To reduce the need for 3rd party subprocess creation code, we should
| > >> have better daemon creation code in the stdlib -- I wrote some damn
| > >> robust code for this purpose in my previous job, but it never saw the
| > >> light of day.
| > >
| > > What do you call "daemon"? An actual Unix-like daemon?
| > 
| > Yeah, a background process with parent PID 1 and not associated with
| > any terminal group.
| 
| But is that relevant to the PEP? A daemon only uses fork(), not exec().

Not necessarily.

I routinely run other commands in a daemonlike mode, without the
command itself having "make myself a daemon" functionality. If I
were writing the launcher in Python (and my "setsid" script is
Python already), I would be doing all the daemon bits in Python
including the fork, then execing the other tool.

Cheers,
-- 

If it can't be turned off, it's not a feature. - Karl Heuer
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

On Wed, Jul 24, 2013 at 3:35 PM, Antoine Pitrou  wrote:
> On Wed, 24 Jul 2013 15:25:50 -0700
> Guido van Rossum  wrote:
>> >> To reduce the need for 3rd party subprocess creation code, we should
>> >> have better daemon creation code in the stdlib -- I wrote some damn
>> >> robust code for this purpose in my previous job, but it never saw the
>> >> light of day.
>> >
>> > What do you call "daemon"? An actual Unix-like daemon?
>>
>> Yeah, a background process with parent PID 1 and not associated with
>> any terminal group.
>
> But is that relevant to the PEP? A daemon only uses fork(), not exec().
> Or have I misunderstood your concern?

Actually it's common for "daemonization" libraries to run an arbitrary
executable as a daemon, and then it would be relevant. E.g. zdaemon
(https://pypi.python.org/pypi/zdaemon) does this. (Disclosure: I wrote
the first version of zdaemon when I worked for Zope Corp over a decade
ago. :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Daemon creation code in the standard library (was: Inherance of file descriptor and handles on Windows (PEP 446))

[Ben Finney]

Guido van Rossum  writes:

> To reduce the need for 3rd party subprocess creation code, we should
> have better daemon creation code in the stdlib -- I wrote some damn
> robust code for this purpose in my previous job, but it never saw the
> light of day.

Work continues on the PEP 3143-compatible ‘python-daemon’, porting it to
Python 3 and aiming for inclusion in the standard library.

Interested parties are invited to join us on the discussion forums
http://lists.alioth.debian.org/cgi-bin/mailman/admin/python-daemon-devel>.

-- 
 \“Politics is not the art of the possible. It consists in |
  `\   choosing between the disastrous and the unpalatable.” —John |
_o__)Kenneth Galbraith, 1962-03-02 |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daemon creation code in the standard library

[Ben Finney]

Ben Finney  writes:

> Work continues on the PEP 3143-compatible ‘python-daemon’, porting it to
> Python 3 and aiming for inclusion in the standard library.

At PyPI http://pypi.python.org/pypi/python-daemon/>, and
development co-ordinated at Alioth
https://alioth.debian.org/projects/python-daemon/>.

> Interested parties are invited to join us on the discussion forums

The correct link for the ‘python-daemon-devel’ forum is
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/python-daemon-devel>.
For announcements only, we have 
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/python-daemon-announce>.

-- 
 \“This sentence contradicts itself — no actually it doesn't.” |
  `\   —Douglas Hofstadter |
_o__)  |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Bohuslav Kabrda]

- Original Message -
> On Wed, Jul 24, 2013 at 11:12 AM, Bohuslav Kabrda  wrote:
> > - Should we point /usr/bin/python to Python 3 when we make the move?
> 
> No.
> 
> > - What should user get after using "yum install python"?
> 
> Will a base install include Python 3? If it does, I think yum install
> python should mean python3, and hence already be installed.
> 

Yep, base will include Python 3.
This is not just about python package, but also all other python extension 
modules (e.g. python-foo vs. python2-foo vs. python3-foo...).

> //Lennart
> 

-- 
Regards,
Bohuslav "Slavek" Kabrda.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 as a Default in Linux Distros

[Bohuslav Kabrda]

- Original Message -
> Am 24.07.13 11:12, schrieb Bohuslav Kabrda:
> > - Should we point /usr/bin/python to Python 3 when we make the move?
> 
> This should depend on the answer to this question:
> - for how long have you been providing /usr/bin/python2 binaries?
> 

Huh, I don't know exactly, but quick git blame of our python.spec shows that 
/usr/bin/python2 has been there since July 2009, but probably even longer.

> Users "should" have been explicit in declaring scripts as
> /usr/bin/python2 for quite some time now, except that this would break
> on distros which don't provide a python2 symlink.
> 
> Ideally, you "should" have warned users to be explicit with python2 if
> the script wouldn't work on python3. But I'd wave this requirement, as
> there is already the upstream PEP.
> 
> So (IMO) if it the last three Fedora releases had been providing python2
> binaries, it should be allowed to switch python to be python3. The exact
> number can be debated; it should depend on what releases are still in
> active use so that a script would run on multiple releases.
> 

Yep, it seems like 6+ fedora releases and certainly the currently active ones 
have that. We will however need to make it clear that everyone, especially 
package maintainers, points to /usr/bin/python{2,3}.

> Then you can tell users that possible breakage is easy to fix: just
> install python2, and change the shebang line.
> 
> Regards,
> Martin
> 
> 

-- 
Regards,
Bohuslav "Slavek" Kabrda.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com