Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Steve Dower
On 16Apr2019 0532, Christian Heimes wrote: Sorry, I forgot to reply. Do you think it would make sense to split the PEP into two PEPs? The auditing hook and import opener hook are related, but distinct improvements. The auditing part looks solid and ready now. The import opener may need some more

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Christian Heimes
On 16/04/2019 14.57, Victor Stinner wrote: > Le mar. 16 avr. 2019 à 14:35, Christian Heimes a écrit > : >> * Linux: readlink("/proc/self/fd/%i") > > That doens't work if /proc is not mounted, which can occur in a > container (where /proc is not mounted nor binded to host /proc). No, it won't wo

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Victor Stinner
Le mar. 16 avr. 2019 à 14:35, Christian Heimes a écrit : > * Linux: readlink("/proc/self/fd/%i") That doens't work if /proc is not mounted, which can occur in a container (where /proc is not mounted nor binded to host /proc). Victor ___ Python-Dev mail

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Christian Heimes
Sorry, I forgot to reply. Do you think it would make sense to split the PEP into two PEPs? The auditing hook and import opener hook are related, but distinct improvements. The auditing part looks solid and ready now. The import opener may need some more refinement. I would also like to get feedbac

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Christian Heimes
On 15/04/2019 23.17, Steve Dower wrote: > On 15Apr2019 1344, Christian Heimes wrote: >> Hi Steve, >> >> (memory dump before I go to bed) >> >> Steve Grubb from Red Hat security pointed me to some interesting things >> [1]. For instance there is some work on a new O_MAYEXEC flag for open(). >> Steve

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-15 Thread Steve Dower
On 15Apr2019 1344, Christian Heimes wrote: Hi Steve, (memory dump before I go to bed) Steve Grubb from Red Hat security pointed me to some interesting things [1]. For instance there is some work on a new O_MAYEXEC flag for open(). Steve came to similar conclusions like we, e.g. streaming code f

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-15 Thread Christian Heimes
On 28/03/2019 23.35, Steve Dower wrote: > Hi all > > Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into > Python 3.8. Here's the current text for review and comment before I > submit to the Steering Council. > > The formatted text is at https://www.python.org/dev/peps/pep-0578/

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-01 Thread Steve Dower
On 30Mar2019 0913, Steve Dower wrote: On 30Mar.2019 0747, Nick Coghlan wrote: I like this PEP in principle, but the specific "open_for_import" name bothers me a lot, as it implies that "importing" is the only situation where a file will be opened for code execution. If this part of the API were

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-01 Thread Steve Dower
On 31Mar2019 0538, Christian Heimes wrote: I don't like the fact that the PEP requires users to learn and use an additional layer to handle native code. Although we cannot provide a fully secure hook for native code, we could at least try to provide a best effort hook and document the limitations

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-31 Thread Christian Heimes
On 29/03/2019 21.10, Steve Dower wrote: >> For example how does the importhook work in regarding of alternative >> importers like zipimport? What does the import hook 'see' for an import >> from a zipfile? > > Yes, good point. I think opening the zip file with open_for_import() is > the right plac

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-30 Thread Steve Dower
On 30Mar.2019 0747, Nick Coghlan wrote: > I like this PEP in principle, but the specific "open_for_import" name > bothers me a lot, as it implies that "importing" is the only situation > where a file will be opened for code execution. > > If this part of the API were lower down the stack (e.g. >

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-30 Thread Steve Dower
On 29Mar.2019 2020, Inada Naoki wrote: > I don't like adding more Python callback from low level. > > Python runtime is very complicated already, especially __del__, > shutdown process, and multi threading. Python callback from low level > is source of very difficult bugs always. Asynchronous ca

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-30 Thread Nick Coghlan
On Sat, 30 Mar 2019 at 06:14, Steve Dower wrote: > On 29Mar2019 1218, Christian Heimes wrote: > > On 28/03/2019 23.35, Steve Dower wrote: > >> There is no Python API provided for changing the open hook. To modify > >> import behavior from Python code, use the existing functionality > >> provided b

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Inada Naoki
I don't like adding more Python callback from low level. Python runtime is very complicated already, especially __del__, shutdown process, and multi threading. Python callback from low level is source of very difficult bugs always. Additionally, if we used the PEP for logging complex application

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
On 29Mar2019 1218, Christian Heimes wrote: On 28/03/2019 23.35, Steve Dower wrote: The ``importlib.util.open_for_import()`` function is a drop-in replacement for ``open(str(pathlike), 'rb')``. Its default behaviour is to open a file for raw, binary access. To change the behaviour a new handler s

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Christian Heimes
On 28/03/2019 23.35, Steve Dower wrote: > Hi all > > Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into > Python 3.8. Here's the current text for review and comment before I > submit to the Steering Council. > > The formatted text is at https://www.python.org/dev/peps/pep-0578/

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
PEP 551 (referenced from this one) contains information about using these hooks for security purposes, along with other approaches to minimize the risk of having Python in your production environments. Threat models have to be designed by the user; we can't predict what it looks like for the in

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Ivan Pozdeev via Python-Dev
Like in the mktemp thread earlier, I would request a threat model (what use cases are supposed to be protected (in this case, by reporting rather than preventing) and from what threats) -- in the discussion, and eventually, in the PEP. Without one, any claims and talks about whether something woul

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
Thanks Christian for responding - I endorse and support all your comments. (I'd hoped that by explicitly saying "this is not a sandbox" it would avoid people thinking it was a sandbox, but apparently I would have been better just to avoid the keyword completely...) On 29Mar2019 0324, Christian

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
On 29Mar2019 0334, Christian Heimes wrote: On 28/03/2019 23.35, Steve Dower wrote: Audit Hook -- In order to observe actions taken by the runtime (on behalf of the caller), an API is required to raise messages from within certain operations. These operations are typically deep within th

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Christian Heimes
On 28/03/2019 23.35, Steve Dower wrote: > Audit Hook > -- > > In order to observe actions taken by the runtime (on behalf of the > caller), an API is required to raise messages from within certain > operations. These operations are typically deep within the Python > runtime or standard lib

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Christian Heimes
On 29/03/2019 01.02, Victor Stinner wrote: > Hi, > > I read quickly the PEP, I'm not sure that I understood it correctly, > so here are some early questions more about the usage of the PEP, than > its implementation. > >> This is not sandboxing, as this proposal does not attempt to prevent >> mal

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Victor Stinner
Hi, I read quickly the PEP, I'm not sure that I understood it correctly, so here are some early questions more about the usage of the PEP, than its implementation. > This is not sandboxing, as this proposal does not attempt to prevent > malicious behavior (though it enables some new options to do

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower
The implementation can be viewed as a PR at https://github.com/python/cpython/pull/12613 On 28Mar2019 1535, Steve Dower wrote: Hi all Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into Python 3.8. Here's the current text for review and comment before I submit to the Steerin

[Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower
Hi all Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into Python 3.8. Here's the current text for review and comment before I submit to the Steering Council. The formatted text is at https://www.python.org/dev/peps/pep-0578/ (update just pushed, so give it an hour or so, bu