Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Mariatta Wijaya]

Thanks all.

Documentation has been updated in  https://bugs.python.org/issue30470


On May 23, 2017 9:13 PM, "Victor Stinner"  wrote:

Sure, make your change and then update libffi!

Victor

Le 23 mai 2017 18:19, "Steve Dower"  a écrit :

> On 23May2017 1212, Victor Stinner wrote:
>
>> 2017-05-22 13:17 GMT-05:00 Steve Dower :
>>
>>> Once the special protection is removed, most of these cases will become
>>> OSError due to the general protection against segmentation faults.
>>>
>>
>> It didn't know that ctypes on Windows had a special protection against
>> programming errors. I'm not aware of such protection Linux. If you
>> call a function with the wrong number of arguments, it's likely to
>> crash or return random data.
>>
>> I guess that the point is to help debugging. But since Python 3.6,
>> faulthandler now registers a Windows exception handler and so it able
>> to dump the Python traceback on any Windows exception:
>> https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable
>>
>> So I think that it's now fine to remove the ctypes protection. Just
>> advice (remind? ;-)) users to enable faulthandler: python3 -X
>> faulthandler, or call faulthandler.enable(). (You might want to use a
>> log file for that on Windows, depends on the use case.)
>>
>
> faulthandler is already recommended in the docs, and the existing SEH
> protection for access violations will remain (since that is independent of
> libffi).
>
> I'll be honest, I have appreciated the functionality in the past, but it
> really isn't good practice and getting rid of it will be an overall
> benefit. Technically even the segfault protection isn't a great idea, since
> you really do end up in an unknown state with regards to memory page
> allocations, but it's better than crashing all the way out.
>
> Cheers,
> Steve
>

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/
mariatta.wijaya%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Victor Stinner]

Sure, make your change and then update libffi!

Victor

Le 23 mai 2017 18:19, "Steve Dower"  a écrit :

> On 23May2017 1212, Victor Stinner wrote:
>
>> 2017-05-22 13:17 GMT-05:00 Steve Dower :
>>
>>> Once the special protection is removed, most of these cases will become
>>> OSError due to the general protection against segmentation faults.
>>>
>>
>> It didn't know that ctypes on Windows had a special protection against
>> programming errors. I'm not aware of such protection Linux. If you
>> call a function with the wrong number of arguments, it's likely to
>> crash or return random data.
>>
>> I guess that the point is to help debugging. But since Python 3.6,
>> faulthandler now registers a Windows exception handler and so it able
>> to dump the Python traceback on any Windows exception:
>> https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable
>>
>> So I think that it's now fine to remove the ctypes protection. Just
>> advice (remind? ;-)) users to enable faulthandler: python3 -X
>> faulthandler, or call faulthandler.enable(). (You might want to use a
>> log file for that on Windows, depends on the use case.)
>>
>
> faulthandler is already recommended in the docs, and the existing SEH
> protection for access violations will remain (since that is independent of
> libffi).
>
> I'll be honest, I have appreciated the functionality in the past, but it
> really isn't good practice and getting rid of it will be an overall
> benefit. Technically even the segfault protection isn't a great idea, since
> you really do end up in an unknown state with regards to memory page
> allocations, but it's better than crashing all the way out.
>
> Cheers,
> Steve
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Steve Dower]

On 23May2017 1212, Victor Stinner wrote:

2017-05-22 13:17 GMT-05:00 Steve Dower :

Once the special protection is removed, most of these cases will become
OSError due to the general protection against segmentation faults.


It didn't know that ctypes on Windows had a special protection against
programming errors. I'm not aware of such protection Linux. If you
call a function with the wrong number of arguments, it's likely to
crash or return random data.

I guess that the point is to help debugging. But since Python 3.6,
faulthandler now registers a Windows exception handler and so it able
to dump the Python traceback on any Windows exception:
https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable

So I think that it's now fine to remove the ctypes protection. Just
advice (remind? ;-)) users to enable faulthandler: python3 -X
faulthandler, or call faulthandler.enable(). (You might want to use a
log file for that on Windows, depends on the use case.)


faulthandler is already recommended in the docs, and the existing SEH 
protection for access violations will remain (since that is independent 
of libffi).


I'll be honest, I have appreciated the functionality in the past, but it 
really isn't good practice and getting rid of it will be an overall 
benefit. Technically even the segfault protection isn't a great idea, 
since you really do end up in an unknown state with regards to memory 
page allocations, but it's better than crashing all the way out.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Victor Stinner]

2017-05-22 13:17 GMT-05:00 Steve Dower :
> Once the special protection is removed, most of these cases will become
> OSError due to the general protection against segmentation faults.

It didn't know that ctypes on Windows had a special protection against
programming errors. I'm not aware of such protection Linux. If you
call a function with the wrong number of arguments, it's likely to
crash or return random data.

I guess that the point is to help debugging. But since Python 3.6,
faulthandler now registers a Windows exception handler and so it able
to dump the Python traceback on any Windows exception:
https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable

So I think that it's now fine to remove the ctypes protection. Just
advice (remind? ;-)) users to enable faulthandler: python3 -X
faulthandler, or call faulthandler.enable(). (You might want to use a
log file for that on Windows, depends on the use case.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Steve Dower]

On 23May2017 0844, Mariatta Wijaya wrote:

+1

My understanding is this is a documentation change, marking it as
deprecated in 3.6.2 and a Misc News entry.
No actual code change.
Correct?


That's correct. I'm thinking just a :note: box in that section, and then 
you can keep working on the 3.7 change. I'll let you file the bugs.


Cheers,
Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Mariatta Wijaya]

+1

My understanding is this is a documentation change, marking it as
deprecated in 3.6.2 and a Misc News entry.
No actual code change.
Correct?



Mariatta Wijaya

On Tue, May 23, 2017 at 8:28 AM, Antoine Pitrou  wrote:

> On Mon, 22 May 2017 11:17:18 -0700
> Steve Dower  wrote:
> >
> > I'd like to propose a highly-accelerated deprecation period for this
> > specific feature, starting in CPython 3.6.2 and being "completed" in
> > 3.7.0, when we will hopefully move onto a newer libffi.
> >
> > In general, the "feature" is a misfeature anyway, since calling a native
> > function with incorrect arguments is unsupported and a very easy way to
> > cause information leakage or code execution vulnerabilities.
>
> Agreed.
>
> > Does anyone have any reasons to oppose this? It already has votes from
> > another Windows expert and the 3.6/3.7 Release Manager, but we wanted to
> > see if anyone has a concern we haven't thought of.
>
> +1 from me.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> mariatta.wijaya%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Antoine Pitrou]

On Mon, 22 May 2017 11:17:18 -0700
Steve Dower  wrote:
> 
> I'd like to propose a highly-accelerated deprecation period for this 
> specific feature, starting in CPython 3.6.2 and being "completed" in 
> 3.7.0, when we will hopefully move onto a newer libffi.
> 
> In general, the "feature" is a misfeature anyway, since calling a native 
> function with incorrect arguments is unsupported and a very easy way to 
> cause information leakage or code execution vulnerabilities.

Agreed.

> Does anyone have any reasons to oppose this? It already has votes from 
> another Windows expert and the 3.6/3.7 Release Manager, but we wanted to 
> see if anyone has a concern we haven't thought of.

+1 from me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Paul Moore]

On 22 May 2017 at 19:17, Steve Dower  wrote:
> One of the main reasons we are stuck with an old libffi fork in CPython is
> because the newer versions do not support protection from calling functions
> with too few/many arguments:
>
> https://docs.python.org/3/library/ctypes.html?highlight=ctypes#calling-functions
>
> There are a number of caveats here, including "this only works on Windows",
> but since it is documented we cannot just remove the behaviour without a
> deprecation period.
>
> I'd like to propose a highly-accelerated deprecation period for this
> specific feature, starting in CPython 3.6.2 and being "completed" in 3.7.0,
> when we will hopefully move onto a newer libffi.
>
> In general, the "feature" is a misfeature anyway, since calling a native
> function with incorrect arguments is unsupported and a very easy way to
> cause information leakage or code execution vulnerabilities. There may be an
> argument for removing the functionality immediately, but honestly I think
> changing libffi in a point release is higher risk.
>
> Once the special protection is removed, most of these cases will become
> OSError due to the general protection against segmentation faults. Some will
> undoubtedly fall through the cracks and crash the entire interpreter, but
> these are unavoidable (and really ought to crash to avoid potential
> exploits).
>
> Does anyone have any reasons to oppose this? It already has votes from
> another Windows expert and the 3.6/3.7 Release Manager, but we wanted to see
> if anyone has a concern we haven't thought of.

+1 from me.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Guido van Rossum]

Sounds good to me.

On Mon, May 22, 2017 at 11:17 AM, Steve Dower 
wrote:

> One of the main reasons we are stuck with an old libffi fork in CPython is
> because the newer versions do not support protection from calling functions
> with too few/many arguments:
>
> https://docs.python.org/3/library/ctypes.html?highlight=ctyp
> es#calling-functions
>
> There are a number of caveats here, including "this only works on
> Windows", but since it is documented we cannot just remove the behaviour
> without a deprecation period.
>
> I'd like to propose a highly-accelerated deprecation period for this
> specific feature, starting in CPython 3.6.2 and being "completed" in 3.7.0,
> when we will hopefully move onto a newer libffi.
>
> In general, the "feature" is a misfeature anyway, since calling a native
> function with incorrect arguments is unsupported and a very easy way to
> cause information leakage or code execution vulnerabilities. There may be
> an argument for removing the functionality immediately, but honestly I
> think changing libffi in a point release is higher risk.
>
> Once the special protection is removed, most of these cases will become
> OSError due to the general protection against segmentation faults. Some
> will undoubtedly fall through the cracks and crash the entire interpreter,
> but these are unavoidable (and really ought to crash to avoid potential
> exploits).
>
> Does anyone have any reasons to oppose this? It already has votes from
> another Windows expert and the 3.6/3.7 Release Manager, but we wanted to
> see if anyone has a concern we haven't thought of.
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Deprecate invalid ctypes call protection on Windows

[Steve Dower]

One of the main reasons we are stuck with an old libffi fork in CPython 
is because the newer versions do not support protection from calling 
functions with too few/many arguments:


https://docs.python.org/3/library/ctypes.html?highlight=ctypes#calling-functions

There are a number of caveats here, including "this only works on 
Windows", but since it is documented we cannot just remove the behaviour 
without a deprecation period.


I'd like to propose a highly-accelerated deprecation period for this 
specific feature, starting in CPython 3.6.2 and being "completed" in 
3.7.0, when we will hopefully move onto a newer libffi.


In general, the "feature" is a misfeature anyway, since calling a native 
function with incorrect arguments is unsupported and a very easy way to 
cause information leakage or code execution vulnerabilities. There may 
be an argument for removing the functionality immediately, but honestly 
I think changing libffi in a point release is higher risk.


Once the special protection is removed, most of these cases will become 
OSError due to the general protection against segmentation faults. Some 
will undoubtedly fall through the cracks and crash the entire 
interpreter, but these are unavoidable (and really ought to crash to 
avoid potential exploits).


Does anyone have any reasons to oppose this? It already has votes from 
another Windows expert and the 3.6/3.7 Release Manager, but we wanted to 
see if anyone has a concern we haven't thought of.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com