[Python-Dev] [Visualpython-users] How VPython 6 differs from VPython 5

[Mark Janssen]

> Since this was copied to the Python-Dev list, I want to go on record as
> stating firmly that there is no evidence whatsoever to substantiate claims
> that there has ever been some kind of conflict between VPython and Python.

My apologies, Bruce, I didn't mean for that second message to go to
they python-dev mailing list (and sorry python-dev list... :).

> Since __future__ was also mentioned, I'll take the opportunity to say that
> I've been very impressed by the way the Python community has handled the
> difficult 2->3 transition. For example, it has been a big help to the
> educational uses of VPython that we could tell students simply to start
with
> "from __future__ import division, print_function", put parens around print
> arguments, and thereby make it irrelevant whether they used Python 2 or
> Python 3. Many thanks to the Python development community!

Yes, it is/was relatively seemless *syntactically*, but it hasn't been
seemless *semanticly*.  from __future__ still does something very odd
as far as the program language definition -- it modifies the way the
interpreter interprets a syntactic construct -- a sort of
meta-linguistic construct.

mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Glenn Linderman]

On 1/13/2013 5:49 AM, Nick Coghlan wrote:

I agree it makes it obvious what the right default behaviour should
be: flag every FD as sensitive by default, and pass an argument to say
"sensitive=False" when you want to disable Python's automatic
protections.


"sensitive" is a bad name... because the data passed via an open 
descriptor that _should_ be passed may well be sensitive, so saying 
sensitive=False is false... it's just necessary to leave the descriptor 
open to make things work...
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Visualpython-users] How VPython 6 differs from VPython 5

[Mark Janssen]

On Sun, Jan 13, 2013 at 12:14 PM, Bruce Sherwood
 wrote:
> For the record, I do not know of any evidence whatsoever for a supposed
> "split" between the tiny VPython community and the huge Python community
> concerning floating point variables. Nor do I see anything in Python that
> needs to be "fixed".

Well it was bit enough that the python community created a brand-new
language construct called import __future__ -- something never
considered before then and which actually changes the behavior of
python unlike any other module.  And perhaps I've just felt it more
because I was a big proponent of both 3-d graphics as a way to keep
python a draw for beginning programmers and also a big fan of
scientific simulation.  No one had anything near vpython back then.
(But ultimately I need to stop mentioning this issue to this vpython
list because it's really the Python group which need to get back in
sync.)

> The new (currently experimental) version of VPython based on wxPython must,
> in order to run in a Cocoa environment on the Mac, make the interact loop be
> the primary thread, with the user's Python calculations at worst a secondary
> thread, which is the opposite of the older VPython, where the
> interval-driven rendering thread was secondary to the user's calculations.
> By the unusual stratagem of having VPython import the user's program it has
> been possible to make this work, and at the same time greatly simplify the
> C++ component of VPython by eliminating threading, with additional important
> simplification from eliminating essentially all platform-dependent code
> thanks to the multiplatform character of wxPython. The result is that nearly
> all existing VPython programs will work without change, at the very small
> cost of a few marginal cases requiring minor tweaking. I should alter the
> documentation to make this important property of the new version more
> salient.

I need to analyze this more carefully before commenting further

mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Greg Ewing]

Nick Coghlan wrote:

Agreed, but it's the security implications that let us even
contemplate the backwards compatibility break.


I don't think that's a sufficient criterion for choosing a
name. The backwards compatibility issue is a transitional
thing, but we'll be stuck with the name forever.

IMO the name should simply describe the mechanism, and be
neutral as to what the reason might be for using the
mechanism.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Cleanup the docs ElementTree a bit and describe the default_namespace

[Serhiy Storchaka]

On 13.01.13 19:28, Eli Bendersky wrote:

1. A better place to "park" them is an issue (the original issue for
which the commits were made, or even a new issue).


I agree. Usually I do this, except for the cases when the issue number 
is not specified in the commit message.



2. Feel free to use your newly gained superpowers to help out with such
fixes. Often a quick fix + commit doesn't take much longer than to write
an email.


I didn't want to come in your heels. Now, when you have approved it and 
left it for me, I'll do it.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Cleanup the docs ElementTree a bit and describe the default_namespace

[Eli Bendersky]

On Sun, Jan 13, 2013 at 7:31 AM, Serhiy Storchaka wrote:

> And perhaps it will be better if parameters will be described in same
> order as they enumerated in the signature (now 'default_namespace'
> described after 'method').
>
>
>
Serhiy, both you comments make sense. But:

1. A better place to "park" them is an issue (the original issue for which
the commits were made, or even a new issue).
2. Feel free to use your newly gained superpowers to help out with such
fixes. Often a quick fix + commit doesn't take much longer than to write an
email.

Eli





> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> eliben%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Visualpython-users] How VPython 6 differs from VPython 5

[Mark Janssen]

On Sun, Jan 13, 2013 at 1:15 AM, Bruce Sherwood  wrote:
> Here is detailed information on how VPython 6 differs from VPython 5, which
> will be incorporated in the Help for upcoming releases of VPython 6. Note
> that the fact that in a main program __name__ isn't '__main__' is an
> unavoidable "feature", not a bug. That is, there doesn't seem to be a way to
> make this work.

Hi, Bruce.  I think there are two ways of fixing this.  One is to get
down to the C language level, and share the C main() loop, which will
percolate back up into the CPython loop.  The other involves mending
the split that I've been mentioning that happened over floating point
between the VPython developers and the Python-dev group.  Then perhaps
Guido will accept VPython using/sharing the __main__ "loop".  (I'm
CC'ing him in this message.)  So, either of these tracks should fix
this problem.  This is why I keep mentioning the important of healing
that split between the parties arguing in the two camps.  Perhaps Tim
Peters will be able help bridge this gap.

> Changes from VPython 5
>
> The new version makes an essential change to the syntax of VPython programs.
> Now, an animation loop MUST contain a rate or sleep statement, which limits
> the number of loop iterations per second as before but also when appropriate
> (about 30 times per second) updates the 3D scene and handles mouse and
> keyboard events. Without a rate or sleep statement, the scene will not be
> updated until and unless the loop is completed.

I think this is perfectly acceptible and is just a necessary
restriction wherefrom the OS *must* maintain ultimate control of I/O.
Python sits in userspace surrounded by the larger computation
environment, so this is only fair that an OS call is made so that it
can keep things in "check".   Naming it "sleep" is okay, but makes it
sound more like a  voluntary thing, and as you probably know, is the
traditional name for relinquishng some control to the OS.  (cf. Unix
Systems Programming, Robbins, et al.)

> You should use the new function sleep rather than time.sleep. The new
> function periodically renders the scene and processes mouse events, making
> it possible to continue using zoom and rotate, whereas time.sleep does not
> do this.

This is strange to me.  I think this is where VPython must be ahead of
the curve and not close enough to the Linux development communities.

> For technical reasons, it is necessary for VPython 6 to do something rather
> unusual. When you import visual (or vis), your own program is in turn
> imported by the visual module.

Again this is because of the faction that was created by in 200?,
regarding the Vpython community vs. the python-dev.  Really, this
should be mended.

Anyway, I hope that any of this made sense.

Thanks for your help!

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Cleanup the docs ElementTree a bit and describe the default_namespace

[Serhiy Storchaka]

And perhaps it will be better if parameters will be described in same 
order as they enumerated in the signature (now 'default_namespace' 
described after 'method').


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Cleanup the docs ElementTree a bit and describe the default_namespace

[Serhiy Storchaka]

On 13.01.13 16:28, eli.bendersky wrote:

http://hg.python.org/cpython/rev/f0e80c7404a5
changeset:   81490:f0e80c7404a5
user:Eli Bendersky 
date:Sun Jan 13 06:27:51 2013 -0800
summary:
   Cleanup the docs ElementTree a bit and describe the default_namespace 
parameter. In the code, replace the old outdated Doxygen-ish comment above 
ElementTree.write by a proper docstring.


I think that wrong documentation of ElementTree.write is a bug and 
should be fixed in early Python versions too.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Victor Stinner]

> Alternatives
> 
>
> Always set close-on-exec flag
> -

Oops, this is completly wrong. This section is basically exactly the
same than the PEP proposal, except that the default value of cloexec
is set to True. The correct title is: "Set the close-on-exec flag by
default" (I just fixed it).

Victort
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Antoine Pitrou]

On Sun, 13 Jan 2013 23:49:32 +1000
Nick Coghlan  wrote:
> 
> > (it's not even just a security issue: letting a bound socket open and
> > therefore being unable to re-use the same port is a bug even when
> > security is not a concern)
> 
> Agreed, but it's the security implications that let us even
> contemplate the backwards compatibility break. We either let
> inexperienced users continue to write insecure software by default, or
> we close the loophole and tell experienced users "hey, to upgrade to
> Python 3.4, you will need to address this change in behaviour".
> 
> The nice thing is that with enough advance warning, they should be
> able to update their code to forcibly clear the flag in a way that
> works even on earlier Python versions.
> 
> A more conservative approach, based on the steps taken in introducing
> hash randomisation, would be to expose the setting as an environment
> variable in 3.4, and then switch the default behaviour in 3.5.

The "more conservative approach" sounds good to me :-)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Nick Coghlan]

On Sun, Jan 13, 2013 at 11:22 PM, Antoine Pitrou  wrote:
> On Sun, 13 Jan 2013 22:44:06 +1000
> Nick Coghlan  wrote:
>> On Sun, Jan 13, 2013 at 9:43 PM, Antoine Pitrou  wrote:
>> > As for the opacity, feel free to propose something better
>> > ("close_on_spawn", whatever). But I'm definitely and strongly -1
>> > on "noinherit".
>>
>> That's the main reason I quite like "sensitive" as a term for this,
>> since it decouples the user statement ("this file descriptor provides
>> access to potentially sensitive information") from the steps the
>> interpreter promises to take to protect that information (such as
>> closing it before executing a different program or ensuring it isn't
>> inherited by child processes).
>
> This assumes that some file descriptors are not "sensitive", which
> sounds a bit weird to me (since a fd will by definition give access
> to a system resource). What should happen is that *no* file descriptors
> are inherited on exec(), except for those few ones which are necessary
> for proper operation of the exec()ed process.

I agree it makes it obvious what the right default behaviour should
be: flag every FD as sensitive by default, and pass an argument to say
"sensitive=False" when you want to disable Python's automatic
protections.

However, there's a pragmatic question of whether we're willing to eat
the backwards incompatibility of such a change in a feature release,
or if it's something that has to wait until Python 4.0, or else be
selected via a global flag (in line with the hash randomisation
change).

> (it's not even just a security issue: letting a bound socket open and
> therefore being unable to re-use the same port is a bug even when
> security is not a concern)

Agreed, but it's the security implications that let us even
contemplate the backwards compatibility break. We either let
inexperienced users continue to write insecure software by default, or
we close the loophole and tell experienced users "hey, to upgrade to
Python 3.4, you will need to address this change in behaviour".

The nice thing is that with enough advance warning, they should be
able to update their code to forcibly clear the flag in a way that
works even on earlier Python versions.

A more conservative approach, based on the steps taken in introducing
hash randomisation, would be to expose the setting as an environment
variable in 3.4, and then switch the default behaviour in 3.5.

So, 3.3: FDs non-sensitive by default, use O_CLOEXEC, noinherit, etc
directly for sensitive ones
3.4: FDs non-sensitive by default, set "sensitive=True" to enable
selectively, or set PYTHONSENSITIVEFDS to enable globally and
"sensitive=False" to disable selectively
3.5: FDs sensitive by default, set "sensitive=False" to enable
selectively, no PYTHONSENSITIVEFDS setting

(To accelerate the adoption schedule, replace 3.5 with 3.4, and 3.4
with 3.3.1, but require that people set or clear the platform specific
flags to work around the default behaviour)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Antoine Pitrou]

On Sun, 13 Jan 2013 22:44:06 +1000
Nick Coghlan  wrote:
> On Sun, Jan 13, 2013 at 9:43 PM, Antoine Pitrou  wrote:
> > As for the opacity, feel free to propose something better
> > ("close_on_spawn", whatever). But I'm definitely and strongly -1
> > on "noinherit".
> 
> That's the main reason I quite like "sensitive" as a term for this,
> since it decouples the user statement ("this file descriptor provides
> access to potentially sensitive information") from the steps the
> interpreter promises to take to protect that information (such as
> closing it before executing a different program or ensuring it isn't
> inherited by child processes).

This assumes that some file descriptors are not "sensitive", which
sounds a bit weird to me (since a fd will by definition give access
to a system resource). What should happen is that *no* file descriptors
are inherited on exec(), except for those few ones which are necessary
for proper operation of the exec()ed process.

(it's not even just a security issue: letting a bound socket open and
therefore being unable to re-use the same port is a bug even when
security is not a concern)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Nick Coghlan]

On Sun, Jan 13, 2013 at 9:43 PM, Antoine Pitrou  wrote:
> As for the opacity, feel free to propose something better
> ("close_on_spawn", whatever). But I'm definitely and strongly -1
> on "noinherit".

That's the main reason I quite like "sensitive" as a term for this,
since it decouples the user statement ("this file descriptor provides
access to potentially sensitive information") from the steps the
interpreter promises to take to protect that information (such as
closing it before executing a different program or ensuring it isn't
inherited by child processes).

We can then define a glossary entry for "sensitive" that explains the
consequences of flagging a descriptor as sensitive on the various
operating systems (i.e. setting cloexec on POSIX and noinherit on
Windows). As the platforms provide additional security mechanisms, we
can provide them without needing to change the user facing APIs.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Antoine Pitrou]

On Sun, 13 Jan 2013 21:33:30 +1000
Nick Coghlan  wrote:
> On Sun, Jan 13, 2013 at 9:15 PM, Antoine Pitrou  wrote:
> >> It also means we're free to implement a mechanism that tries to close
> >> all sensitive file descriptors in _PyOS_AfterFork.
> >
> > Ouch! This actually shows that "noinherit" is a very bad name. The PEP
> > is about closing fds after exec(), *not* after fork(). So "cloexec" is
> > really the right, precise, non-ambiguous name here.
> 
> No, 'cloexec' is a terrible name, because, aside from the cryptic
> opacity of it, it's also wrong on Windows, which doesn't have the
> fork() vs exec() distinction.

Just because Windows doesn't have the fork() vs exec() distinction
doesn't mean "cloexec" is a bad description of the flag: it describes
exactly what happens (the fd gets closed when executing an external
program).

As for the opacity, feel free to propose something better
("close_on_spawn", whatever). But I'm definitely and strongly -1
on "noinherit".

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Nick Coghlan]

On Sun, Jan 13, 2013 at 9:15 PM, Antoine Pitrou  wrote:
>> It also means we're free to implement a mechanism that tries to close
>> all sensitive file descriptors in _PyOS_AfterFork.
>
> Ouch! This actually shows that "noinherit" is a very bad name. The PEP
> is about closing fds after exec(), *not* after fork(). So "cloexec" is
> really the right, precise, non-ambiguous name here.

No, 'cloexec' is a terrible name, because, aside from the cryptic
opacity of it, it's also wrong on Windows, which doesn't have the
fork() vs exec() distinction.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Antoine Pitrou]

On Sun, 13 Jan 2013 21:13:42 +1000
Nick Coghlan  wrote:
> >
> > Also, ISTM that Windows also supports this flag. If it does, then
> > "cloexec" might not be the best name, because it refers to the
> > execve() Unix system call. Maybe something like "noinherit" would be
> > clearer (although coming from a Unix background "cloexec" is
> > crystal-clear to me :-).
> 
> Indeed, this may be an area where following the underlying standards
> too closely may not be a good idea. In particular, a *descriptive*
> flag may be better choice than an imperative one.
> 
> For example, if we make the flag "sensitive", then the programmer is
> telling us "this file descriptor is sensitive" and then we get to
> decide what that means in terms of the underlying OS behaviours like
> "close-on-exec" and "no-inherit" (as well as deciding whether or not
> file descriptors are considered sensitive by default).
> 
> It also means we're free to implement a mechanism that tries to close
> all sensitive file descriptors in _PyOS_AfterFork.

Ouch! This actually shows that "noinherit" is a very bad name. The PEP
is about closing fds after exec(), *not* after fork(). So "cloexec" is
really the right, precise, non-ambiguous name here.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Nick Coghlan]

On Sun, Jan 13, 2013 at 8:40 PM, Charles-François Natali
 wrote:
>> XXX Should ``subprocess.Popen`` set the close-on-exec flag on file XXX
>> XXX descriptors of the constructor the ``pass_fds`` argument?  XXX
>
> What?
> Setting them cloexec would prevent them from being inherited in the
> child process!

Perhaps Victor meant "clear" rather than "set" in this comment?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Nick Coghlan]

On Sun, Jan 13, 2013 at 8:40 PM, Charles-François Natali
 wrote:
> Hello,
>
>> PEP: 433
>> Title: Add cloexec argument to functions creating file descriptors
>
> I'm not a native English speaker, but it seems to me that the correct
> wording should be "parameter" (part of the function
> definition/prototype, whereas "argument" refers to the actual value
> supplied).

This is correct (although it's a subtle distinction that even many
native English speakers get wrong).

For the PEP, I'd actually avoid assuming the solution in the title and
instead say something like "Easier suppression of file descriptor
inheritance"

>> This PEP proposes to add a new optional argument ``cloexec`` on
>> functions creating file descriptors in the Python standard library. If
>> the argument is ``True``, the close-on-exec flag will be set on the
>> new file descriptor.
>
> It would probably be useful to recap briefly what the close-on-exec flag does.
>
> Also, ISTM that Windows also supports this flag. If it does, then
> "cloexec" might not be the best name, because it refers to the
> execve() Unix system call. Maybe something like "noinherit" would be
> clearer (although coming from a Unix background "cloexec" is
> crystal-clear to me :-).

Indeed, this may be an area where following the underlying standards
too closely may not be a good idea. In particular, a *descriptive*
flag may be better choice than an imperative one.

For example, if we make the flag "sensitive", then the programmer is
telling us "this file descriptor is sensitive" and then we get to
decide what that means in terms of the underlying OS behaviours like
"close-on-exec" and "no-inherit" (as well as deciding whether or not
file descriptors are considered sensitive by default).

It also means we're free to implement a mechanism that tries to close
all sensitive file descriptors in _PyOS_AfterFork.

Alternatively, you could flip the sense of the flag and use
"inherited" - the scope is then clearly limited to indicating whether
or not the file descriptor should be inherited by child processes.

Either way, if done with sufficient advance warning and clear advice
to users, the "It's a systematic fix to adopt secure-by-default
behaviour" excuse may actually let it get by, especially if the
appropriate stdlib APIs adopt "inherited-by-default" for the FDs where
it is needed (such as subprocess pass_fds).


>> Add a new optional ``cloexec`` argument to:
>>
>>  * Maybe also: ``os.open()``, ``os.openpty()``
>
> Open can be passed O_CLOEXEC directly.

Indeed, people playing at the os module layer are already expected to
have to deal with platform specific behaviour. That's probably a
reasonable model to continue here - if people want cross-platform
handling of sensitive file descriptors, they either have to handle it
themselves or step up a layer of abstraction.

> Also, it has the same drawbacks as global variables: not thread-safe,
> not library-safe (i.e. if two libraries set it to conflicting values,
> you don't know which one is picked up).

I think it makes sense to consider the two issues separately, while
making sure the design in this PEP supports both "given inherited FD
by default, mark individual ones that shouldn't be inherited" and
"given non-inherited FD by default, mark individual ones for
inheritance"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Chris Jerdonek]

On Sun, Jan 13, 2013 at 2:40 AM, Charles-François Natali
 wrote:
> Hello,
>
>> PEP: 433
>> Title: Add cloexec argument to functions creating file descriptors
>
> I'm not a native English speaker, but it seems to me that the correct
> wording should be "parameter" (part of the function
> definition/prototype, whereas "argument" refers to the actual value
> supplied).

Yes, this distinction is now reflected in our glossary as of a month
or two ago.  Let's try to be consistent with that. :)

--Chris
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

[Charles-François Natali]

Hello,

> PEP: 433
> Title: Add cloexec argument to functions creating file descriptors

I'm not a native English speaker, but it seems to me that the correct
wording should be "parameter" (part of the function
definition/prototype, whereas "argument" refers to the actual value
supplied).

> This PEP proposes to add a new optional argument ``cloexec`` on
> functions creating file descriptors in the Python standard library. If
> the argument is ``True``, the close-on-exec flag will be set on the
> new file descriptor.

It would probably be useful to recap briefly what the close-on-exec flag does.

Also, ISTM that Windows also supports this flag. If it does, then
"cloexec" might not be the best name, because it refers to the
execve() Unix system call. Maybe something like "noinherit" would be
clearer (although coming from a Unix background "cloexec" is
crystal-clear to me :-).

> On UNIX, subprocess closes file descriptors greater than 2 by default
> since Python 3.2 [#subprocess_close]_. All file descriptors created by
> the parent process are automatically closed.

("in the child process")

> ``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of
> the listening socket, the parent class ``socketserver.BaseServer``
> does not set this flag.

As has been discussed earlier, the real issue is that the server
socket is not closed in the child process. Setting it cloexec would
only add an extra security for multi-threaded programs.

> Inherited file descriptors issues
> -
>
> Closing the file descriptor in the parent process does not close the
> related resource (file, socket, ...) because it is still open in the
> child process.

You might want to go through the bug tracker to find examples of such
issues, and list them:
http://bugs.python.org/issue7213
http://bugs.python.org/issue12786
http://bugs.python.org/issue2320
http://bugs.python.org/issue3006

The list goes on.
Some of those examples resulted in deadlocks.

> The listening socket of TCPServer is not closed on ``exec()``: the
> child process is able to get connection from new clients; if the
> parent closes the listening socket and create a new listening socket
> on the same address, it would get an "address already is used" error.

See above for the real cause.

> Not closing file descriptors can lead to resource exhaustion: even if
> the parent closes all files, creating a new file descriptor may fail
> with "too many files" because files are still open in the child
> process.

You might want to detail the course of events (a child if forked
before the parent gets a chance to close the file descriptors...
EMFILE).

> Leaking file descriptors is a major security vulnerability. An
> untrusted child process can read sensitive data like passwords and
> take control of the parent process though leaked file descriptors. It
> is for example a known vulnerability to escape from a chroot.

You might add a link to this:
https://www.securecoding.cert.org/confluence/display/seccode/FIO42-C.+Ensure+files+are+properly+closed+when+they+are+no+longer+needed

It can also result in DoS (if the child process highjacks the server
socket and accepts connections).

Example of vulnerabilities:
http://www.openssh.com/txt/portable-keysign-rand-helper.adv
http://www.securityfocus.com/archive/1/348368
http://cwe.mitre.org/data/definitions/403.html

> The problem is that these flags and functions are not portable: only
> recent versions of operating systems support them. ``O_CLOEXEC`` and
> ``SOCK_CLOEXEC`` flags are ignored by old Linux versions and so
> ``FD_CLOEXEC`` flag must be checked using ``fcntl(fd, F_GETFD)``.  If
> the kernel ignores ``O_CLOEXEC`` or ``SOCK_CLOEXEC`` flag, a call to
> ``fcntl(fd, F_SETFD, flags)`` is required to set close-on-exec flag.
>
> .. note::
>OpenBSD older 5.2 does not close the file descriptor with
>close-on-exec flag set if ``fork()`` is used before ``exec()``, but
>it works correctly if ``exec()`` is called without ``fork()``.

That would be *really* surprising, are your sure your test case is correct?
Otherwise it could be a compilation issue, because I simply can't
believe OpenBSD would ignore the close-on-exec flag.

> This PEP only change the close-on-exec flag of file descriptors
> created by the Python standard library, or by modules using the
> standard library.  Third party modules not using the standard library
> should be modified to conform to this PEP. The new
> ``os.set_cloexec()`` function can be used for example.
>
> Impacted functions:
>
>  * ``os.forkpty()``
>  * ``http.server.CGIHTTPRequestHandler.run_cgi()``

I've opened http://bugs.python.org/issue16945 to rewrite this to use subprocess.

> Impacted modules:
>
>  * ``multiprocessing``
>  * ``socketserver``
>  * ``subprocess``
>  * ``tempfile``

Hum, I thought temporay file are already created with the close-on-exec flag.

>  * ``xmlrpc.server``
>  * Maybe: ``signal``, ``threading``
>
> XXX Should ``subprocess.Popen

Re: [Python-Dev] cpython (3.3): clean trailing whitespace

[Serhiy Storchaka]

On 12.01.13 17:45, eli.bendersky wrote:

http://hg.python.org/cpython/rev/bbfc8f62cb67
changeset:   81452:bbfc8f62cb67
branch:  3.3
parent:  81450:f9d1d120c19e
user:Eli Bendersky 
date:Sat Jan 12 07:44:32 2013 -0800
summary:
   clean trailing whitespace


Can you please clean trailing whitespace in Modules/_elementtree.c too? 
It's very annoying manually edit every patch for Modules/_elementtree.c 
to remove unrelated changes (my editors remove trailing spaces 
automatically).


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com