Re: Issue sending data from C++ to Python

[Dieter Maurer]

Pablo Martinez Ulloa wrote at 2022-5-18 15:08 +0100:
>I have been using your C++ Python API, in order to establish a bridge from
>C++ to Python.

Do you know `cython`?
It can help very much in the implementation of bridges between
Python and C/C++.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Issue sending data from C++ to Python

[Christian Gollwitzer]

Am 18.05.22 um 16:08 schrieb Pablo Martinez Ulloa:

I have been using your C++ Python API, in order to establish a bridge from
C++ to Python. We want to do this, as we have a tactile sensor, which only
has a library developed in C++, but we want to obtain the data in real time
in Python to perform tests with a robotic arm and gripper. The problem we
are facing is with transmitting the data into Python. We are using the
function Py_BuildValue, and it seems to be working, but only for sending
one value at a time, and we want to transmit 54 numbers


The usual way to pass an array of numbers into Python is by means of a 
Numpy Array. In order to do that, you need to include arrayobject.h and 
then use PyArray_SimpleNew to create an array of numbers as a Python 
object which you can return 
https://numpy.org/devdocs/reference/c-api/array.html#c.PyArray_SimpleNew



 Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Issue sending data from C++ to Python

[MRAB]

On 2022-05-18 15:08, Pablo Martinez Ulloa wrote:

Hello,

I have been using your C++ Python API, in order to establish a bridge from
C++ to Python. We want to do this, as we have a tactile sensor, which only
has a library developed in C++, but we want to obtain the data in real time
in Python to perform tests with a robotic arm and gripper. The problem we
are facing is with transmitting the data into Python. We are using the
function Py_BuildValue, and it seems to be working, but only for sending
one value at a time, and we want to transmit 54 numbers. When
transmitting a single value retrieved from the sensor into Python in a
double or string format, it works, but when we make the string bigger it
gives us an error with the utf-8 encoding. Also when sending a tuple of
doubles or strings, it works when the tuple is composed of a single
element, but not when we send 2 or more values. Is there any way of fixing
this issue? Or is it just not possible to transmit such a large amount of
data?


I find it easier if I have some code in front of me to 'criticise'!

Have you thought about building and returning, say, a list instead?

/* Error checking omitted. */
PyObject* list;
list = PyList_New(0);
PyList_Append(list, PyLong_FromSsize_t(1));
PyList_Append(list, PyFloat_FromDouble(2.0));
...
--
https://mail.python.org/mailman/listinfo/python-list

Issue sending data from C++ to Python

[Pablo Martinez Ulloa]

Hello,

I have been using your C++ Python API, in order to establish a bridge from
C++ to Python. We want to do this, as we have a tactile sensor, which only
has a library developed in C++, but we want to obtain the data in real time
in Python to perform tests with a robotic arm and gripper. The problem we
are facing is with transmitting the data into Python. We are using the
function Py_BuildValue, and it seems to be working, but only for sending
one value at a time, and we want to transmit 54 numbers. When
transmitting a single value retrieved from the sensor into Python in a
double or string format, it works, but when we make the string bigger it
gives us an error with the utf-8 encoding. Also when sending a tuple of
doubles or strings, it works when the tuple is composed of a single
element, but not when we send 2 or more values. Is there any way of fixing
this issue? Or is it just not possible to transmit such a large amount of
data?

Thank you.
Best regards,

*Pablo Martinez Ulloa*
PhD Candidate,
School of Electrical and Electronic Engineering, R324A
University College Dublin, Belfield, Dublin 4, Ireland
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45814] datetime.time.strftime: use the same function signature for C and Python implementations

[Paul Ganssle]

Paul Ganssle  added the comment:

I think this is mostly a duplicate of bpo-41260, which has an open PR on it. I 
think that got lost in the shuffle, I'm sad we didn't fix it in Python 3.10. I 
think we should migrate all of these signatures that differ to whichever one 
the C implementation is using (I believe that's 3.11).

I'm going to close that one and edit the other one to cover `time` and `date` 
as well. Thanks for the report Yevhenii!

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> datetime: strftime method takes different keyword argument: fmt 
(pure) or format (C)

___
Python tracker 
<https://bugs.python.org/issue45814>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45814] datetime.time.strftime: use the same function signature for C and Python implementations

[Alex Waygood]

Change by Alex Waygood :


--
title: Use the same function signature for datetime.time.strftime -> 
datetime.time.strftime: use the same function signature for C and Python 
implementations

___
Python tracker 
<https://bugs.python.org/issue45814>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41031] Inconsistency in C and python traceback printers

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41031] Inconsistency in C and python traceback printers

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 9e31b3952f6101ef71ec029481b972169ab0e0f1 by Irit Katriel in 
branch 'main':
bpo-41031: Match C and Python code formatting of unprintable exceptions and 
exceptions in the __main__ module. (GH-28139)
https://github.com/python/cpython/commit/9e31b3952f6101ef71ec029481b972169ab0e0f1


--
nosy: +serhiy.storchaka

___
Python tracker 
<https://bugs.python.org/issue41031>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41031] Inconsistency in C and python traceback printers

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41031] Inconsistency in C and python traceback printers

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +patch
nosy: +iritkatriel
nosy_count: 1.0 -> 2.0
pull_requests: +26578
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28139

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Sergey B Kirpichev]

Sergey B Kirpichev  added the comment:

On Sun, May 30, 2021 at 08:58:56AM +, Mark Dickinson wrote:
> Yep, you're absolutely right. I should have said "after the PEP is final"

Unfortunately, neither correction can fix that the PEP does not
"accurately describes the state of the implementation at the point where
it is marked Final."

> It would also be a backwards incompatible change at this point to
> start refusing strings that were previously accepted.

I'm not sure...

Well, it's not so clear which strings are accepted previously (i.e.
what's was documented).  PEP 515 claims one.  The docs says something
different:
>8---
If value is a string, it should conform to the decimal numeric string
syntax after leading and trailing whitespace characters, as well as
underscores throughout, are removed
>8---

and
--->8--
Underscores are allowed for grouping, _as with integral and
floating-point literals in code_.
-->8---

The 1-st sentence doesn't specify the way underscores are removed.  And
given the 2-nd sentence: it's clearly can't be like the current
behaviour of the Decimal constructor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Mark Dickinson]

Change by Mark Dickinson :


--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Mark Dickinson]

Mark Dickinson  added the comment:

> Regarding mail thread: I don't think that following the PEP will
slow down string conversion.

Sorry, I just don't think it's worth re-opening this discussion; Stefan Krah 
had good reasons (not just speed) to avoid implementing a precise 
interpretation of PEP 515 for Decimal.

It would also be a backwards incompatible change at this point to start 
refusing strings that were previously accepted. As I said, it's probably best 
left alone at this point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Mark Dickinson]

Mark Dickinson  added the comment:

> Well, then I something misunderstood in PEP 0

Yep, you're absolutely right. I should have said "after the PEP is final", not 
"after the PEP is accepted". PEP 515 was marked final on April 28th, 2017.

> The current behaviour is documented.

Thanks; I missed that. In that case, I don't think there's anything to do here 
documentation-wise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Sergey B Kirpichev]

Sergey B Kirpichev  added the comment:

On Sun, May 30, 2021 at 08:32:40AM +, Mark Dickinson wrote:
> Standards Track PEPs are historical documents; it's quite common that an
> actual implementation ends up diverging from a PEP in small ways after
> the PEP is accepted, and we don't usually do post-hoc updates in those 
> situations.

Well, then I something misunderstood in PEP 0:
--->8--
If changes based on implementation experience and user feedback are made
to Standards track PEPs while in the Accepted or Provisional State,
those changes should be noted in the PEP, such that the PEP accurately
describes the state of the implementation at the point where it is
marked Final.
>8
I don't think that PEP describes the state of art in the decimal module.

> Possibly the decimal documentation could be updated, though.

The current behaviour is documented.  Do you mean we should document
disagreement with PEP as well?

Regarding mail thread: I don't think that following the PEP will
slow down string conversion.  Also, probably we want that strings
that supported by float() and Decimal() were interchangeable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Mark Dickinson]

Mark Dickinson  added the comment:

> If this is not a bug - it should be adjusted

Standards Track PEPs are historical documents; it's quite common that an actual 
implementation ends up diverging from a PEP in small ways after the PEP is 
accepted, and we don't usually do post-hoc updates in those situations.

Possibly the decimal documentation could be updated, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Sergey B Kirpichev]

Sergey B Kirpichev  added the comment:

On Sun, May 30, 2021 at 08:20:14AM +, Mark Dickinson wrote:
> There was some discussion of this on the python-dev mailing list at the time

I see.

> It's probably best left alone.

PEP 515 is clear.  If this is not a bug - it should be adjusted (as it
claims to cover Decimal's among other stuff).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Mark Dickinson]

Mark Dickinson  added the comment:

There was some discussion of this on the python-dev mailing list at the time - 
see https://mail.python.org/pipermail/python-dev/2016-March/143556.html and the 
surrounding thread. It's probably best left alone.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Sergey B Kirpichev]

Change by Sergey B Kirpichev :


Added file: 
https://bugs.python.org/file50072/0001-bpo-44267-fix-parsing-Decimal-s-with-underscores.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +facundobatista, mark.dickinson, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44267] Wrong PEP 515 parsing in decimal module (both C and Python versions)

[Sergey B Kirpichev]

New submission from Sergey B Kirpichev :

While working on issue44258 I discover that the decimal module doesn't follow 
specification in PEP 515: "The current proposal is to allow one underscore 
between digits, and after base specifiers in numeric literals." (c)

For example:
>>> float("1.1__1")
Traceback (most recent call last):
  File "", line 1, in 
ValueError: could not convert string to float: '1.1__1'

but
>>> from decimal import Decimal as C
>>> from _pydecimal import Decimal as P
>>> C("1.1__1")
Decimal('1.11')
>>> P("1.1__1")
Decimal('1.11')

Maybe this requirement could be relaxed in PEP, but it seems - this was already 
discussed (see Alternative Syntax section).  Hence, I think this is a bug.

Patch for _pydecimal attached.

--
components: Extension Modules, Library (Lib)
files: _pydecimal-pep515.diff
keywords: patch
messages: 394750
nosy: Sergey.Kirpichev
priority: normal
severity: normal
status: open
title: Wrong PEP 515 parsing in decimal module (both C and Python versions)
versions: Python 3.11
Added file: https://bugs.python.org/file50071/_pydecimal-pep515.diff

___
Python tracker 
<https://bugs.python.org/issue44267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43559] ctypes: Heap Pointer is damaged between C and Python

[Canberk Sönmez]

Canberk Sönmez  added the comment:

Alright, I solved the problem. It was simply a typo: "restypes" instead of 
"res_types". It didn't cause a problem in Python 3.6 but obviously, something 
was changed. A nice error message might be nice when setting these attributes.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43559] ctypes: Heap Pointer is damaged between C and Python

[Eric V. Smith]

Eric V. Smith  added the comment:

Are you using a 64-bit version of python? What is sys.maxsize?

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43559] ctypes: Heap Pointer is damaged between C and Python

[Canberk Sönmez]

New submission from Canberk Sönmez :

Please see the SO post:

https://stackoverflow.com/questions/66713071/ctypes-heap-pointer-is-damaged-between-c-and-python-linux-x86-64

In summary, when I return a pointer to a heap-allocated memory location from a 
C function, its most significant 32 bits are chopped off for some reason.

I observed this behavior in Python 3.7 and Python 3.8, on Ubuntu 18.04 and 
Centos 7 (x86_64).

--
messages: 389107
nosy: canberk.sonmez.409
priority: normal
severity: normal
status: open
title: ctypes: Heap Pointer is damaged between C and Python

___
Python tracker 
<https://bugs.python.org/issue43559>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Universal compiler that runs Java, Ruby, C++, and Python in a single VM

[James Lu]

It's called Oracle's Truffle. Truffle runs all those languages with an
autogenerated JIT.

This is my response to the neos drama.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue41031] Inconsistency in C and python traceback printers

[Michael Simacek]

New submission from Michael Simacek :

I belive the python traceback module was designed to produce the same output as 
the internal exception printer (sys.__excepthook__), but this is not the case 
when the exception's __str__ raises an exception.

Given an exception of the following class:
class E(Exception):
   def __str__(self):
  raise RuntimeError

Internal printer output:
Traceback (most recent call last):
  File "inconsistent.py", line 6, in 
raise E()
__main__.E: 

traceback.print_exc output:
Traceback (most recent call last):
  File "inconsistent.py", line 6, in 
raise E()
E: 

--
components: Library (Lib)
messages: 371855
nosy: msimacek
priority: normal
severity: normal
status: open
title: Inconsistency in C and python traceback printers
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue41031>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add private _PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 regression)

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add private _PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0b72b23fb0c130279f65f3bcd23521acf4a98c88 by Victor Stinner in 
branch 'master':
bpo-38500: Add _PyInterpreterState_SetEvalFrameFunc() (GH-17340)
https://github.com/python/cpython/commit/0b72b23fb0c130279f65f3bcd23521acf4a98c88


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add private _PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

Mark: Would you mind to open a separated issue for your following idea?

> I propose a new method on code objects `withCallAtLine(callable: Callable[], 
> line:int)` which returns a new code object with calls to the given callable 
> at the given line.
> A breakpoint can then be inserted at line L in function f with `f.__code__ = 
> f.__code__.withCallAtLine(sys.breakpoint, L)`.

I prefer to restrict this issue to the PEP 523.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add private _PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

I took Mark's concerns in account and I updated my PR to only add *private* 
functions to the C API, not *public* functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add private _PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 regression)

[STINNER Victor]

Change by STINNER Victor :


--
title: PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API 
(Python 3.8 regression) -> PEP 523: Add private 
_PyInterpreterState_SetEvalFrameFunc() function to the C API (Python 3.8 
regression)

___
Python tracker 
<https://bugs.python.org/issue38500>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

Mark:
> PEP 523 is quite vague, but the rationale indicates that exposing 
> `eval_frame` is for "a method-level JIT". PEP 523 did not suggest adding an 
> API.

I disagree, the PEP is quite explicit: "Third-party code may then set their own 
frame evaluation function instead to control the execution of Python code." 
That's the whole point of the PEP: let third-party code set eval_frame to use 
the feature.

The PEP was written in 2016, when the PyInterpreterState structure was part of 
the public C API. But PyInterpreterState was moved to the internal C API, after 
the PEP was approved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

Mark Shannon: `PyThreadState` is an internal opaque data structure, which means 
we are free to change it.

I guess that you mean PyInterpreterState which was moved to the internal C API 
in Python 3.8. It was part of public C API in Python 3.7. See commit 
be3b295838547bba267eb08434b418ef0df87ee0 of bpo-35886.

By the way, I just created bpo-39947: "Make the PyThreadState structure opaque 
(move it to the internal C API)".

--

_______
Python tracker 
<https://bugs.python.org/issue38500>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

Somehow related, I just created bpo-39946: "Is it time to remove 
_PyThreadState_GetFrame() hook?".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

For me, the *short term* goal is to find a way to limit the number of broken C 
extension module while we modify the C API to make it more opaque.

We cannot reach all goals at once (opaque C API, subinterpreter, more 
optimizations, etc). We have to move step by step.

For me it's ok to deprecate or even remove 
PyInterpreterState_SetEvalFrameFunc() later, once we will have a good reason 
for that.

I'm also ok with having an alternative Python implementation which doesn't 
support PyInterpreterState_SetEvalFrameFunc(). Users would be able to make a 
choice: faster Python without PyInterpreterState_SetEvalFrameFunc(), or regular 
"slow" Python with PyInterpreterState_SetEvalFrameFunc(). That's part of my 
larger https://pythoncapi.readthedocs.io/ goal: the ability to use different 
Python "runtimes".

>From what I understood, to be able to provide multiple Python runtimes, we 
>have first to "fix" the C API. The HPy project is another approach to this 
>problem. Making the C API opaque makes CPython closer to this goal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

> Why not do so by reverting the change that caused it?

Making PyInterpreterState structure private was motivated by the 
subinterpreters work but also by the work on cleaning the C API.

Over time, PyInterpreterState became more and more complex because many other 
structures have been moved there. For example, the GIL, GC state, warnings 
state, parser state, etc.

I would really avoid exposing the GIL state in the C API since it uses the 
"pycore_atomic.h" header which caused a *lot* of compiler errors in the past. 
Most errors were on including the header file, even if the C extension didn't 
use any atomic variable.

I'm really happy that we managed to move atomic variables into the internal C 
API: we got less error coming from that. I'm strongly opposed to move 
PyInterpreterState structure back into the Include/cpython/ C API. That would 
be a big mistake for various reasons.

Even in CPython, pycore_pystate.h is causing a lot of troubles since 
PyInterpreterState became very complex. Example in posixmodule.c:
---
#include "Python.h"
#ifdef MS_WINDOWS
   /* include  early to avoid conflict with pycore_condvar.h:

#define WIN32_LEAN_AND_MEAN
#include 

  FSCTL_GET_REPARSE_POINT is not exported with WIN32_LEAN_AND_MEAN. */
#  include 
#endif

#include "pycore_ceval.h" /* _PyEval_ReInitThreads() */
#include "pycore_import.h"/* _PyImport_ReInitLock() */
#include "pycore_pystate.h"   /* _PyRuntime */
---

pycore_condvar.h is used by the GIL state.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[Mark Shannon]

Mark Shannon  added the comment:

I'm not suggesting immediate removal of PEP 532.

I am requesting that you don't add a new function to the C API that will 
prevent implementation of many meaningful optimizations to CPython.
PEP 532 does not add any API functions.

I understand that you want to fix the regression.
Why not do so by reverting the change that caused it?

Adding a new API function is not fixing a regression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

STINNER Victor  added the comment:

> I think you mean PEP 352? And a more formal proposal to withdraw would need 
> to be made and that has not happened, so assume the PEP is still accepted and 
> final.

I mean PEP 523 "Adding a frame evaluation API to CPython" as I wrote in the 
title, sorry for the typo :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[Brett Cannon]

Brett Cannon  added the comment:

> Mark Shannon listed flaws in the PEP 532 and suggest to withdraw this PEP.

I think you mean PEP 352? And a more formal proposal to withdraw would need to 
be made and that has not happened, so assume the PEP is still accepted and 
final.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] PEP 523: Add PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 regression)

[STINNER Victor]

Change by STINNER Victor :


--
title: Provide a way to get/set PyInterpreterState.frame_eval without needing 
to access interpreter internals -> PEP 523: Add 
PyInterpreterState_SetEvalFrameFunc() to the public C API (Python 3.8 
regression)
versions:  -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38500>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See issue37685.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[miss-islington]

miss-islington  added the comment:


New changeset c6b31061997526b31961ec34328408ca421f51fc by Miss Islington (bot) 
(Xtreak) in branch '3.7':
[3.7] bpo-37579: Improve equality behavior for pure Python datetime and time 
(GH-14726) (GH-14745)
https://github.com/python/cpython/commit/c6b31061997526b31961ec34328408ca421f51fc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Paul Ganssle]

Change by Paul Ganssle :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[miss-islington]

miss-islington  added the comment:


New changeset 143672cf028740fc549e532c049559c522930c95 by Miss Islington (bot) 
in branch '3.8':
bpo-37579: Improve equality behavior for pure Python datetime and time 
(GH-14726)
https://github.com/python/cpython/commit/143672cf028740fc549e532c049559c522930c95


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests: +14541
pull_request: https://github.com/python/cpython/pull/14745

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[miss-islington]

Change by miss-islington :


--
pull_requests: +14539
pull_request: https://github.com/python/cpython/pull/14743

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Paul Ganssle]

Paul Ganssle  added the comment:


New changeset e6b46aafad3427463d6264a68824df4797e682f1 by Paul Ganssle (Xtreak) 
in branch 'master':
bpo-37579: Improve equality behavior for pure Python datetime and time 
(GH-14726)
https://github.com/python/cpython/commit/e6b46aafad3427463d6264a68824df4797e682f1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Do you mind to fix also other similar cases if they are? __lt__ and others 
should have the same property.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
keywords: +patch
pull_requests: +14522
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14726

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37579] Difference in equality check between C and Python implementation for datetime module's timedelta and time

[Karthikeyan Singaravelan]

New submission from Karthikeyan Singaravelan :

As reported by Serhiy on https://bugs.python.org/issue37555#msg347733 there is 
a difference in __eq__ definition in datetime module's C and Python 
implementation for timedelta and time. When the other in __eq__ is not of the 
same type NotImplemented is not returned in Python implementation like C 
causing equality to behave differently.

Difference in C implementation and Python implementation for 
datetime.timedelta.__eq__

https://github.com/python/cpython/blob/c8e7146de257930ea8d0d4aa74b3a64fcaa79d4b/Modules/_datetimemodule.c#L2152

static PyObject *
delta_richcompare(PyObject *self, PyObject *other, int op)
{
if (PyDelta_Check(other)) {
int diff = delta_cmp(self, other);
return diff_to_bool(diff, op);
}
else {
Py_RETURN_NOTIMPLEMENTED;
}
}


https://github.com/python/cpython/blob/c8e7146de257930ea8d0d4aa74b3a64fcaa79d4b/Lib/datetime.py#L732

def __eq__(self, other):
if isinstance(other, timedelta):
return self._cmp(other) == 0
else:
return False


I will add a PR for this with test.

--
components: Library (Lib)
messages: 347773
nosy: belopolsky, p-ganssle, serhiy.storchaka, xtreak
priority: normal
severity: normal
status: open
title: Difference in equality check between C and Python implementation for 
datetime module's timedelta and time
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37579>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Paul Ganssle]

Paul Ganssle  added the comment:

I've left a mostly finished PR on #8983, but I've decided it's not really worth 
continuing to work on. Anyone can feel free to take it and run with it if they 
want to implement the fix for this.

As Alexey mentions, that PR indeed already fixes this bug, I mainly blocked on 
getting consistent behavior across all platforms. Currently that PR has 
consistent behavior across all platforms if you disable wcsftime *except* 
Ubuntu 14.0, which seems to have some kind of funky bug in wcstombs, though for 
the life of me I don't know how to consistently trigger it. I suspect that 
using strftime instead of wcsftime on all platforms would lead to simpler code 
that works most consistently, but I am guessing wcsftime is faster.

Another option is to use wsftime but fall back to strftime on MacOS in the 
event that the error condition (blank output from non-blank input) is detected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

> if we can't make assertions about the behavior of strftime outputs, I think 
> it makes it hard to prevent regressions.

Ironically, some of the changes we may have to make to fix time.strftime() 
inconsistencies may appear like regressions to some users. For example, 
regarding dropping wcsftime() on all platforms, 'man strftime' on macOS 
contains the following warning in BUGS section: "The strftime() function does 
not correctly handle multibyte characters in the format argument.". (I'm not 
sure what "incorrect handling" means here). That's not to discourage you, just 
to point out that we should be extra careful at this point when there might be 
users relying on every variety of the current behavior. And that's also why I 
don't want to mix the fix for C vs. Python issue (low risk) with changes in 
time.strftime() (high risk).

Also, while I certainly agree with "the outsized effort" point, it seems that 
your PR goes far beyond supporting surrogates because falling back to escaping 
allows one to bypass checks in wcsftime()/strftime() and round-trip any code 
point regardless of the current locale.

--

___
Python tracker 
<https://bugs.python.org/issue34481>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Paul Ganssle]

Paul Ganssle  added the comment:

@izbyshev That's totally fair and I wouldn't want to make it a condition of 
merging the existing fixes - I've already made great progress in fixing the 
time.strftime part as well.

The main reason it relates here is that I generally find the tests to be among 
the hardest part about writing a good PR, and if we can't make assertions about 
the behavior of strftime outputs, I think it makes it hard to prevent 
regressions. I figured if I can solve the problem all the way down the stack in 
one go, I might as well.

That said, Victor makes an *extremely* good point that this is an outsized 
effort for the bug it's fixing. No one really *needs* support for unpaired 
surrogates in their strftime as far as I can tell. The main reason I'm still 
working on it is that I'm curious to see if it's even possible to fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

@p-ganssle

> I'm finding it very difficult to reconcile these things.

Paul, this issue was only about reconciling C and Python implementations of 
datetime module -- not fixing time.strftime(), which both of them delegate to. 
While the latter is certainly more important, it's also much more difficult, 
and IMO needs another issue report (maybe merged with #34512). As for the 
former, it seems that you've already fixed it in your PR.

--

___
Python tracker 
<https://bugs.python.org/issue34481>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[STINNER Victor]

STINNER Victor  added the comment:

> Because caring of surrogates,

Before*

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[STINNER Victor]

STINNER Victor  added the comment:

Because caring of surrogates, I would prefer to first fix time.strftime() and 
time.strptime() for legit time zone names. Currently, Python uses the C 
function strfime() because the string produced by wcsftime() cannot be parsed 
later. We should use the native Windows API to get the time zone name properly 
encoded. I don't recall the details, sorry. There is maybe an open issue for 
that one.

--

___
Python tracker 
<https://bugs.python.org/issue34481>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Paul Ganssle]

Paul Ganssle  added the comment:

I'm finding it very difficult to reconcile these things. I'm not entirely sure, 
but we may need to take a performance hit in normal strftime if we want to make 
this work with surrogate characters, which really does not appeal to me (though 
we can certainly improve to some degree).

One major question here: Is anyone (@vstinner, @belopolsky?) know why time's 
strftime opportunistically uses wcsftime instead of strftime? It makes the code 
*way* more complicated and difficult to read / maintain - are there platforms 
that provide wcstrftime and not strftime?

Also, related, it seems according to 
https://bugs.python.org/issue10653#msg243660 that there may have been a 
regression in issue #10653, which may be related here.

Either way, some note should probably be made in the code to clarify exactly 
*why* these choices were made in the code, in case the situation has changed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Paul Ganssle]

Change by Paul Ganssle :


--
keywords: +patch
pull_requests: +8457
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Paul Ganssle]

Change by Paul Ganssle :


--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34481] Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings

[Alexey Izbyshev]

New submission from Alexey Izbyshev :

The C datetime implementation uses PyUnicode_AsUTF8AndSize() in wrap_strftime() 
and rejects strings containing surrogate code points (0xD800 - 0xDFFF) since 
they can't be encoded in UTF-8. On the other hand, the pure-Python datetime 
implementation doesn't have this restriction:

>>> import sys
>>> sys.modules['_datetime'] = None # block C implementation
>>> from datetime import time
>>> time().strftime('\ud800')
'\ud800'
>>> del sys.modules['datetime']
>>> del sys.modules['_datetime']
>>> from datetime import time
>>> time().strftime('\ud800')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 
0: surrogates not allowed

--
components: Extension Modules
messages: 323963
nosy: belopolsky, izbyshev, pitrou, serhiy.storchaka, taleinat
priority: normal
severity: normal
status: open
title: Different behavior of C and Python impls of datetime.strftime with 
non-UTF-8-encodable strings
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue34481>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Handle SIGINT in C and Python (Posting On Python-List Prohibited)

[eryk sun]

On Thu, Feb 1, 2018 at 4:57 AM, Victor Porton <por...@narod.ru> wrote:
> Lawrence D’Oliveiro wrote:
>
>> On Thursday, February 1, 2018 at 8:10:24 AM UTC+13, Victor Porton wrote:
>>> Lawrence D’Oliveiro wrote:
>>>
>>>> The usual behaviour for POSIX is that the call is aborted with EINTR
>>>> after you get the signal.
>>>
>>> That poll() is interrupted does not imply that Python will run its
>>> pythonic signal handler at the point of interruption. That is a problem.
>>
>> * Python calls poll()
>> * poll() aborted with EINTR
>> * Python runs your signal handler
>>
>> Versus native C code:
>>
>> * your code calls poll()
>> * poll() aborted with EINTR
>> * your signal handler is run
>>
>> Where is there a fundamental difference?
>
> I meant to call poll() from C code, not Python code. In this case when
> poll() is aborted with EINTR, the pythonic signal handler does not run.

An extension module should call PyErr_CheckSignals [1] when
interrupted by EINTR. For example, here's the loop used to call poll()
in the standard-library select module:

async_err = 0;
do {
Py_BEGIN_ALLOW_THREADS
errno = 0;
poll_result = poll(self->ufds, self->ufd_len, (int)ms);
Py_END_ALLOW_THREADS

if (errno != EINTR)
break;

/* poll() was interrupted by a signal */
if (PyErr_CheckSignals()) {
async_err = 1;
break;
}

if (timeout >= 0) {
timeout = deadline - _PyTime_GetMonotonicClock();
if (timeout < 0) {
poll_result = 0;
break;
}
ms = _PyTime_AsMilliseconds(timeout, _PyTime_ROUND_CEILING);
/* retry poll() with the recomputed timeout */
}
} while (1);

self->poll_running = 0;

if (poll_result < 0) {
if (!async_err)
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}

[1]: https://docs.python.org/3/c-api/exceptions.html#c.PyErr_CheckSignals
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handle SIGINT in C and Python (Posting On Python-List Prohibited)

[Victor Porton]

Lawrence D’Oliveiro wrote:

> On Thursday, February 1, 2018 at 5:57:58 PM UTC+13, Victor Porton wrote:
>> I meant to call poll() from C code, not Python code.
> 
> Do you need to use C code at all? Python is quite capable of handling this
> <https://docs.python.org/3/library/select.html>.

I already concluded that I can use Popen.communicate() instead of my 
library.

So the issue is closed.

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handle SIGINT in C and Python (Posting On Python-List Prohibited)

[Victor Porton]

Lawrence D’Oliveiro wrote:

> On Thursday, February 1, 2018 at 8:10:24 AM UTC+13, Victor Porton wrote:
>> Lawrence D’Oliveiro wrote:
>> 
>>> The usual behaviour for POSIX is that the call is aborted with EINTR
>>> after you get the signal.
>> 
>> That poll() is interrupted does not imply that Python will run its
>> pythonic signal handler at the point of interruption. That is a problem.
> 
> * Python calls poll()
> * poll() aborted with EINTR
> * Python runs your signal handler
> 
> Versus native C code:
> 
> * your code calls poll()
> * poll() aborted with EINTR
> * your signal handler is run
> 
> Where is there a fundamental difference?

I meant to call poll() from C code, not Python code. In this case when 
poll() is aborted with EINTR, the pythonic signal handler does not run.

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handle SIGINT in C and Python (Posting On Python-List Prohibited)

[Victor Porton]

Lawrence D’Oliveiro wrote:

> On Wednesday, January 31, 2018 at 9:55:45 PM UTC+13, Victor Porton wrote:
>> Lawrence D’Oliveiro wrote:
>> 
>>> On Wednesday, January 31, 2018 at 8:58:18 PM UTC+13, Victor Porton
>>> wrote:
 For this reason I
 cannot use Python signals because "A Python signal handler does not get
 executed inside the low-level (C) signal handler. Instead, the
 low-level signal handler sets a flag which tells the virtual machine to
 execute the corresponding Python signal handler at a later point(for
 example at the next bytecode instruction)."
>>> 
>>> Why is that a problem?
>> 
>> As I already said, I need to handle the signal (SIGCHLD) while poll()
>> waits for i/o.
> 
> The usual behaviour for POSIX is that the call is aborted with EINTR after
> you get the signal.

That poll() is interrupted does not imply that Python will run its pythonic 
signal handler at the point of interruption. That is a problem.

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handle SIGINT in C and Python (Posting On Python-List Prohibited)

[Victor Porton]

Lawrence D’Oliveiro wrote:

> On Wednesday, January 31, 2018 at 8:58:18 PM UTC+13, Victor Porton wrote:
>> For this reason I
>> cannot use Python signals because "A Python signal handler does not get
>> executed inside the low-level (C) signal handler. Instead, the low-level
>> signal handler sets a flag which tells the virtual machine to execute the
>> corresponding Python signal handler at a later point(for example at the
>> next bytecode instruction)."
> 
> Why is that a problem?

As I already said, I need to handle the signal (SIGCHLD) while poll() waits 
for i/o.

You can read the sources (and the file HACKING) of the C library which I am 
wrapping into Python here:

https://github.com/vporton/libcomcom

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handle SIGINT in C and Python

[Victor Porton]

Victor Porton wrote:
> I need to assign a real C signal handler to SIGINT.
> 
> This handler may be called during poll() waiting for data. For this reason
> I cannot use Python signals because "A Python signal handler does not get
> executed inside the low-level (C) signal handler. Instead, the low-level
> signal handler sets a flag which tells the virtual machine to execute the
> corresponding Python signal handler at a later point(for example at the
> next bytecode instruction)."
> 
> I want after my signal handler for SIGINT executed to raise
> KeyboardInterrupt (as if I didn't installed my own signal handler).
> 
> Is this possible? How?

I've found a solution: I can use PyOS_setsig() from Python implementation to 
get the old (Python default) OS-level signal handler, while I assign the new 
one.

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Handle SIGINT in C and Python

[Victor Porton]

I need to assign a real C signal handler to SIGINT.

This handler may be called during poll() waiting for data. For this reason I 
cannot use Python signals because "A Python signal handler does not get 
executed inside the low-level (C) signal handler. Instead, the low-level 
signal handler sets a flag which tells the virtual machine to execute the 
corresponding Python signal handler at a later point(for example at the next 
bytecode instruction)."

I want after my signal handler for SIGINT executed to raise 
KeyboardInterrupt (as if I didn't installed my own signal handler).

Is this possible? How?

-- 
Victor Porton - http://portonvictor.org
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28706] msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0

[Stefan Krah]

Change by Stefan Krah :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> setup.py cannot find vcversall.bat on MSWin 8.1 if installed in 
user AppData

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28706] msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0

[Jiří Hofman]

New submission from Jiří Hofman:

When running
"c:\Program Files\Python27\Scripts\pip2.7.exe" install wx
following error occurs:

Complete output from command "c:\program files\python27\python.exe" -u -c 
"import setuptools, 
tokenize;__file__='c:\\users\\jiri\\appdata\\local\\temp\\pip-build-lqokio\\wx\\setup.py';f=getattr(tokenize,
 'open', open)(__file__);code=f.read().replace('\r\n', 
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record 
c:\users\jiri\appdata\local\temp\pip-51m45m-record\install-record.txt 
--single-version-externally-managed --compile:
running install
running build
WARNING: Building this way assumes that all generated files have been
generated already.  If that is not the case then use build.py directly
to generate the source and perform the build stage.  You can use
--skip-build with the bdist_* or install commands to avoid this
message and the wxWidgets and Phoenix build steps in the future.

"c:\program files\python27\python.exe" -u build.py build
Will build using: "c:\program files\python27\python.exe"
2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit 
(AMD64)]
Python's architecture is 64bit
cfg.VERSION: 3.0.3

Running command: build
Running command: build_wx
Command '"c:\program files\python27\python.exe" -c "import 
distutils.msvc9compiler as msvc; mc = msvc.MSVCCompiler(); mc.initialize(); 
print(mc.cc)"' failed with exit code 1.
Traceback (most recent call last):
  File "", line 1, in 
  File "c:\program files\python27\lib\distutils\msvc9compiler.py", line 
385, in initialize
vc_env = query_vcvarsall(VERSION, plat_spec)
  File "c:\program files\python27\lib\distutils\msvc9compiler.py", line 
273, in query_vcvarsall
raise DistutilsPlatformError("Unable to find vcvarsall.bat")
distutils.errors.DistutilsPlatformError: Unable to find vcvarsall.bat
Finished command: build_wx (0.138s)
Finished command: build (0.139s)
Command '"c:\program files\python27\python.exe" -u build.py build' failed 
with exit code 1.

I found out that the problem is in msvc9compiler. The dir where vcvarsall.bat 
is installed is not found. It is installed (by default) in 
C:\Users\Jiri\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0
Needed registry keys do not exist after installation of the Visual C++ for 
Python 9.0.

My OS is Windows 10 Home (64bit).

--
components: Distutils
messages: 280894
nosy: dstufft, eric.araujo, jyrkih
priority: normal
severity: normal
status: open
title: msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0
versions: Python 2.7

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28706>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

PyEmbedC 1.0 - Embed C in Python code

[Fen Trias]

Python module for embedding C/C++ code within Python code. It is
designed to allow coders to easily speed up slow code by replacing small
portions of Python with compiled C/C++ that can transparently read
and modify Python variables. Keeping all the code in the same source
file simplifies development and improves code readability. All the
compiling and linking is done dynamically by the module to further
simplify development.

* Embed C/C++ within Python source code
* Automatically compile and dynamically link using GCC
* Access and modify Python variables as C variables
* Support arrays and strings
* Use ctypes for maximum portability

Simple Example:

from embedc import C
a=[1,4,9]
alen=len(a)
C("""
  for (int i=0; i<alen; i++) {
a[i] = a[i]*a[i];
  }
""")
print a

Output: [1,16,81]

http://pyembedc.sourceforge.net/;>PyEmbedC 1.0 -
Module to embed C/C++ code within Python code and read/modify
variables natively. (16-Jan-2016)
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

[issue16050] ctypes: callback from C++ to Python fails with Illegal Instruction call

[R. David Murray]

R. David Murray added the comment:

Arnon also checked that our local copy of libffi contains the upstream fix.  In 
the absence of a way to test this, we'll just have to assume it is fixed.

--
nosy: +r.david.murray
resolution:  - out of date
stage:  - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16050
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16050] ctypes: callback from C++ to Python fails with Illegal Instruction call

[Arnon Yaari]

Arnon Yaari added the comment:

libffi seem to have fixed this issue in their commit 4acf005 - Build fix for 
soft-float power targets. The fix is different than what Pavel suggested, but 
sounds like it should do the same.

--
nosy: +wiggin15

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16050
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

how to create an external string when binding C to Python?

[Simon]

I'd like to make a C memory buffer available inside Python via the Python C/API 
without copying that memory into Python. How to do this? I've read [1] but it's 
not clear that this functionality exists. In javascript it's possible using 
String::NewExternal() [2].

Creates a new external string using the ASCII data defined in the given 
resource.

When the external string is no longer live on V8's heap the resource will be 
disposed by calling its Dispose method. The caller of this function should not 
otherwise delete or modify the resource. Neither should the underlying buffer 
be deallocated or modified except through the destructor of the external string 
resource.

[1] https://docs.python.org/2/c-api/buffer.html
[2] 
http://izs.me/v8-docs/classv8_1_1String.html#a07c47bf675b802c550984fa24511a589
-- 
https://mail.python.org/mailman/listinfo/python-list

how to create a zero copy external string when binding C to Python?

[Simon]

I'd like to make a C memory buffer available inside Python via the Python C/API 
without copying that memory into Python. How to do this? I've read [1] but it's 
not clear that this functionality exists. In javascript it's possible using 
String::NewExternal() [2].

Creates a new external string using the ASCII data defined in the given 
resource.

When the external string is no longer live on V8's heap the resource will be 
disposed by calling its Dispose method. The caller of this function should not 
otherwise delete or modify the resource. Neither should the underlying buffer 
be deallocated or modified except through the destructor of the external string 
resource.

[1] https://docs.python.org/2/c-api/buffer.html
[2] 
http://izs.me/v8-docs/classv8_1_1String.html#a07c47bf675b802c550984fa24511a589
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to create an external string when binding C to Python?

[Mark Lawrence]

On 09/05/2014 00:10, Simon wrote:

I'd like to make a C memory buffer available inside Python via the Python C/API 
without copying that memory into Python. How to do this? I've read [1] but it's 
not clear that this functionality exists. In javascript it's possible using 
String::NewExternal() [2].

Creates a new external string using the ASCII data defined in the given 
resource.

When the external string is no longer live on V8's heap the resource will be 
disposed by calling its Dispose method. The caller of this function should not 
otherwise delete or modify the resource. Neither should the underlying buffer be 
deallocated or modified except through the destructor of the external string 
resource.

[1] https://docs.python.org/2/c-api/buffer.html
[2] 
http://izs.me/v8-docs/classv8_1_1String.html#a07c47bf675b802c550984fa24511a589



Does this help 
http://jakevdp.github.io/blog/2014/05/05/introduction-to-the-python-buffer-protocol/ 
?


Also note that the Python 3 buffer protocol differs from that of Python 
2, so make sure that you check out PEP 3118 that's referenced in the 
above link.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: how to create an external string when binding C to Python?

[Ian Kelly]

On Thu, May 8, 2014 at 5:10 PM, Simon simo...@gmail.com wrote:
 I'd like to make a C memory buffer available inside Python via the Python 
 C/API without copying that memory into Python. How to do this? I've read [1] 
 but it's not clear that this functionality exists. In javascript it's 
 possible using String::NewExternal() [2].

 Creates a new external string using the ASCII data defined in the given 
 resource.

 When the external string is no longer live on V8's heap the resource will be 
 disposed by calling its Dispose method. The caller of this function should 
 not otherwise delete or modify the resource. Neither should the underlying 
 buffer be deallocated or modified except through the destructor of the 
 external string resource.

 [1] https://docs.python.org/2/c-api/buffer.html
 [2] 
 http://izs.me/v8-docs/classv8_1_1String.html#a07c47bf675b802c550984fa24511a589

In 3.3 or greater you can create a memoryview directly from a char*
using this this call:
https://docs.python.org/3/c-api/memoryview.html#c.PyMemoryView_FromMemory

Otherwise, you can do it using the buffer protocol but it takes a
little more work than just a function call.  You'll need to define in
C an extension type that will wrap your C buffer and that implements
the buffer protocol, and then objects of that type can be used as the
basis for memoryviews (or other buffer consumers).
-- 
https://mail.python.org/mailman/listinfo/python-list

C++ to python for LED Matrix

[Liam Knott]

Hey folks,

So the last week or so I've been searching this site for information on how to 
control and program a LED Matrix (or a number of them) for a project. A few 
Topics have caught my eye, with me originally having in mind using a Maxim 
MAX7221 to control the matrix, but none more than Klaas's: 
http://www.raspberrypi.org/phpBB3/viewtopic.php?t=41713.
This was almost perfect for me, until I saw the code was in C++ and from the 
title all I know is Python. So after checking the code out and trying to figure 
how this language works I just couldn't break it down and Unfortunately I don't 
have a lot of time to learn C++ ( would be great if I could but I don't have 
that luxury right now).
So if anyone is willing to check Klaas's topic and code and convert it into 
python that would be awesome, even if someone would explain how the code is 
working that would be great, oh and don't be scared to comment on ways of going 
about this project.

Note: Hoping I could achieve something like this but with the Pi: 
http://www.youtube.com/watch?v=FzHT-L-7jIA

Cheers!

Lknott :-D
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C++ to python for LED Matrix

[Michael Torrie]

Yes you could use Python for this sort of thing.  The link you posted is
just using a kernel spi driver that Python can write to just as well as
C++ can (via it's /dev/spidev0.0 file).  There is a python library that
can talk to SPI in Python on the pi:

http://www.100randomtasks.com/simple-spi-on-raspberry-pi

You still need to know some low-level stuff though.  Like hexadecimal,
binary bit-wise operations, etc.

Definitely talk to people on the Raspberry Pi forum.  They are doing
this stuff frequently.

Also don't be afraid of C.  Learn it. You'll be glad.  The code you
linked to looks more complicated than it really is. The ioctl stuff
looks complicated. But everything else is easy.  If it weren't for the
ioctl stuff, which I know can be translated to Python directly but I'm
not quite sure how at the moment, the rest of that code could be
transliterated into Python in very short order.  The trick is to make
the code more pythonic, and use classes when appropriate to encapsulate
things. I'd make a class that talks to SPI, for example. It would open
the file, set the ioctls, and then provide a basic interface for writing
to the bus.  Then from that I'd build another class that implements the
matrix abstraction, using SPI class for the low-level stuff.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 And yet, in Python 2, people could do that, and Python didn't care.
 *That's* the regression I'm worried about. If it hadn't round-tripped
 cleanly in Python 2, I wouldn't care here either.

$ python2.7 -c print u'\u20ac'
€
$ LANG=C python2.7 -c print u'\u20ac'
Traceback (most recent call last):
  File string, line 1, in module
UnicodeEncodeError: 'ascii' codec can't encode character u'\u20ac' in position 
0: ordinal not in range(128)

And even worse:

$ python2.7 -c print u'\u20ac' /dev/null
Traceback (most recent call last):
  File string, line 1, in module
UnicodeEncodeError: 'ascii' codec can't encode character u'\u20ac' in position 
0: ordinal not in range(128)

What the wart!

Other program can produces wrong (or even absolutely senseless) output with C 
locale.

$ LANG=C ls
   
 ??
?? ??  
?? 
??  
??
?? ??   


What is better, silently produce corrupted output or raise an exception? If 
first, then let just set the replace or backslashreplace error handler for 
sys.stdout by default.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 sworddragon@ubuntu:~$ LANG=C
 sworddragon@ubuntu:~$ ä
 bash: $'\303\244': command not found
 
 - The terminal doesn't pseudo-crash with an exception because it doesn't
 matter about encodings. - It allows to change the encoding at runtime.

This is not a locale of your terminal. Try `LANG=C xterm`.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

The C locale is part of the ANSI C standard. The POSIX locale is an alias 
for the C locale and a POSIX standard, so we cannot just replace the ASCII 
encoding with UTF-8 as we wish, so Antoine's patch won't work.

See e.g. http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html

The C and POSIX locale settings are the only locale settings that are 
guaranteed to always exist in C libraries. Python 3 should work with such 
locale settings. It doesn't have to be able to output non-ASCII code points, 
but it should run with ASCII data.

AFAIK, Python 3 does work with ASCII data in the C locale, so I'm not sure 
whether this is a bug at all.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[STINNER Victor]

STINNER Victor added the comment:

I didn't understand Serhiy's ls example. I tried:

$ mkdir unicode
$ cd unicode
$ python3 -c 'open(ab\xe9.txt, w).close()'
$ python3 -c 'open(euro\u20ac.txt, w).close()'
$ ls
abé.txt  euro€.txt
$ LANG=C ls
ab??.txt  euro???.txt


Ah yes, I didn't remember that ls is aware of the locale encoding.

printf() and wprintf() behave differently on unencodable/undecoable characters:
http://unicodebook.readthedocs.org/en/latest/programming_languages.html#printf-functions-family

Again, the issue is not specific to Python. So it's time to learn how to 
configure correctly your locales.

About the interoperability point I mentionned in my first message (This 
encoding is the best choice for interopability with other (python2 or non 
python) programs.): if you work around the annoying ASCII encoding by forcing 
UTF-8 encoding, Python may produce data which would be incompatible with other 
applications following POSIX and so using the ASCII encoding.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[STINNER Victor]

STINNER Victor added the comment:

Nick testing applications for POSIX compliance

Sorry but what do you mean by POSIX compliance? The POSIX standard only 
specify the ASCII encoding.

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html
The tables in Locale Definition describe the characteristics and behavior of 
the POSIX locale for data consisting entirely of characters from the portable 
character set and the control character set. For other characters, the behavior 
is unspecified. For C-language programs, the POSIX locale shall be the default 
locale when the setlocale() function is not called.

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap06.html#tagtcjh_3
Portable character set = ASCII

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[STINNER Victor]

STINNER Victor added the comment:

Marc-Andre AFAIK, Python 3 does work with ASCII data in the C locale, so I'm 
not sure whether this is a bug at all.

What do you mean? Python uses the surrogateescape encoding since Python 3.1, 
undecodable bytes are stored as surrogate characters.

Many bugs related to locales were fixed in Python 3.2, 3.3 and 3.4.

There are remaining bugs?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 09.12.2013 11:19, STINNER Victor wrote:
 
 STINNER Victor added the comment:
 
 Marc-Andre AFAIK, Python 3 does work with ASCII data in the C locale, so I'm 
 not sure whether this is a bug at all.
 
 What do you mean? Python uses the surrogateescape encoding since Python 3.1, 
 undecodable bytes are stored as surrogate characters.
 
 Many bugs related to locales were fixed in Python 3.2, 3.3 and 3.4.
 
 There are remaining bugs?

I was referring to the original bug report on this ticket.

FWIW: I don't think you can expect Python to work without exceptions
if you use a C locale and write non-ASCII data to stdout. I also
don't think that the new ticket title is correct - or at least,
I fail to see which aspect of Python breaks with LANG=C :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3

[Nick Coghlan]

Changes by Nick Coghlan ncogh...@gmail.com:


--
title: print() and write() are relying on sys.getfilesystemencoding() instead 
of sys.getdefaultencoding() - Setting LANG=C breaks Python 3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
title: print() and write() are relying on sys.getfilesystemencoding() instead 
of sys.getdefaultencoding() - Setting LANG=C breaks Python 3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3

[STINNER Victor]

STINNER Victor added the comment:

 Or said differently, the filesystem encoding is different than the
 locale encoding.

 Indeed, but the FS encoding and the IO encoding are the same.
 locale encoding doesn't really matter here, as we are assuming that
 it's wrong.

Oh, I realized that FS encoding term in not clear. When I wrote FS 
encoding, I mean sys.getfilesystemencoding() which is mbcs on Windows, UTF-8 
on Mac OS X and (currently) the locale encoding on other platforms (UNIX, ex: 
Linux/FreeBSD/Solaris/AIX).

--

IMO there are two different points in this issue:

(a) which encoding should be used when the C locale is used: the encoding 
announced by the OS using nl_langinfo(CODESET) (current choice) or use an 
arbitrary optimistic utf-8 encoding?

(b) for technical reasons, Python reuses the C codec during Python 
initialization to decode and encode OS data, and so currently Python *must* use 
the locale encoding for its filesystem encoding

Before being able to pronounce me on the point (a), I would like to see a patch 
fixing the point (b). I'm not against fixing point (b). I'm just saying that 
it's not trivial and obviously it must be fixed to change the status of point 
(a). I even gave clues to fix point (b).

--

asciilocale.patch has many issues. Try to run the Python test suite using this 
patch to see what I mean. Example of failures:

==
FAIL: test_non_ascii (test.test_cmd_line.CmdLineTest)
--
Traceback (most recent call last):
  File /home/haypo/prog/python/default/Lib/test/test_cmd_line.py, line 140, 
in test_non_ascii
assert_python_ok('-c', command)
  File /home/haypo/prog/python/default/Lib/test/script_helper.py, line 69, in 
assert_python_ok
return _assert_python(True, *args, **env_vars)
  File /home/haypo/prog/python/default/Lib/test/script_helper.py, line 55, in 
_assert_python
stderr follows:\n%s % (rc, err.decode('ascii', 'ignore')))
AssertionError: Process return code is 1, stderr follows:
Unable to decode the command from the command line:
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' in position 
12: surrogates not allowed

==
FAIL: test_ioencoding_nonascii (test.test_sys.SysModuleTest)
--
Traceback (most recent call last):
  File /home/haypo/prog/python/default/Lib/test/test_sys.py, line 603, in 
test_ioencoding_nonascii
self.assertEqual(out, os.fsencode(test.support.FS_NONASCII))
AssertionError: b'' != b'\xc3\xa6'

==
FAIL: test_nonascii (test.test_warnings.CEnvironmentVariableTests)
--
Traceback (most recent call last):
  File /home/haypo/prog/python/default/Lib/test/test_warnings.py, line 774, 
in test_nonascii
['ignore:Deprecaci\xf3nWarning'].encode('utf-8'))
AssertionError: b['ignore:Deprecaci\\udcc3\\udcb3nWarning'] != 
b['ignore:Deprecaci\xc3\xb3nWarning']

==
FAIL: test_nonascii (test.test_warnings.PyEnvironmentVariableTests)
--
Traceback (most recent call last):
  File /home/haypo/prog/python/default/Lib/test/test_warnings.py, line 774, 
in test_nonascii
['ignore:Deprecaci\xf3nWarning'].encode('utf-8'))
AssertionError: b['ignore:Deprecaci\\udcc3\\udcb3nWarning'] != 
b['ignore:Deprecaci\xc3\xb3nWarning']


test_warnings is probably #9988, test_cmd_line failure is maybe #9992.

There are maybe other issues, the Python test suite only have a few tests for 
non-ASCII characters.

--

If anything is changed, I would prefer to have more than a few months of test 
to make sure that it doesn't break anything. So I set the version field to 
Python 3.5.

--
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3

[Antoine Pitrou]

Antoine Pitrou added the comment:

On dim., 2013-12-08 at 22:22 +, STINNER Victor wrote:
 (b) for technical reasons, Python reuses the C codec during Python
 initialization to decode and encode OS data, and so currently Python
 *must* use the locale encoding for its filesystem encoding

Ahhh! Well indeed that's a bummer :-)

 asciilocale.patch has many issues. Try to run the Python test suite
 using this patch to see what I mean.

I'm assuming much of this is due to (b) (all those tests seem to spawn
external processes).

It seems there is more work to do to get this right, but I'm not
terribly interested either. Feel free to take over.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3

[STINNER Victor]

STINNER Victor added the comment:

 It seems there is more work to do to get this right, but I'm not
 terribly interested either. Feel free to take over.

If you are talking to me: I'm currently opposed to change anything, so I'm not 
interested to work on a patch. IMO Python works fine and you should try to 
workaround the current limitations :-)

If someone is interested to write an huge patch fixing all these issues, I 
would be able to reconsider my opinion on point (a).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Nick Coghlan]

Nick Coghlan added the comment:

End users tripping over this by setting LANG=C is one of the pain points of 
Python 3 relative to Python 2 for Fedora, so I've added a couple of Fedora 
folks to the nosy list.

My current understanding of the situation:

- we should leave Windows and Mac OS X alone, since they ignore the locale when 
choosing the OS API encoding anyway

- the main problem is on Linux (but potentially other *nix systems as well), 
where people set LANG=C for a variety of reasons, but this has the side 
effect of Python 3 choosing an inappropriate encoding (ASCII rather than UTF-8) 
when talking to the OS APIs.

Given the initialisation problems, this may be something that PEP 432 (the 
initialisation process rewrite) can help with (since it changes the 
initialisation order to create a more complete Python runtime before it starts 
to configure the OS interfaces).

Tangentially related, we may want to consider aliasing 
sys.getfilesystemencoding, os.fsencode and os.fsdecode as something like 
sys.getosapiencoding, os.apiencode and os.apidecode, since the current naming 
is misleading (the value is based on the platform and environment, not any 
particular filesystem, and is used for almost all bytes-based OS APIs, not just 
filesystem metadata)

--
nosy: +a.badger, bkabrda
title: Setting LANG=C breaks Python 3 - Setting LANG=C breaks Python 3 on Linux

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[STINNER Victor]

STINNER Victor added the comment:

 End users tripping over this by setting LANG=C is one of the pain points of 
 Python 3 relative to Python 2 for Fedora, so I've added a couple of Fedora 
 folks to the nosy list.

Sorry, I'm not aware of such issue. Do you have examples?

 - the main problem is on Linux (but potentially other *nix systems as well), 
 where people set LANG=C for a variety of reasons, but this has the side 
 effect of Python 3 choosing an inappropriate encoding (ASCII rather than 
 UTF-8) when talking to the OS APIs.

Why do you think that the issue is specific to Python 3? Try to open a
terminal with LC_ALL=C and try to type non-ASCII characters with your
keyboard. You can't because your terminal uses ASCII. Did you
applications written in another language handling Unicode, like Perl?
(Perl with Unicode support correctly enabled, it's use utf8; if I
remember correctly).

Can you explain the various reasons why users explictly force the
encoding to ASCII?

I use LANG=C to get manual pages and error messages in english. But
LANG=en_US man ls would be more correct, or LC_MESSAGES=en_US man
ls to be pedantic. (Env var priority: LC_ALL  LANG  LC_xxx).

IMO if you use LANG=C, you must not complain that Unicode stopped
working, but you should learn how to configure locales. Trivial
examples like the one which can be found in the initial message
(msg204849) are wrong: why would you force all locales to C and use
non-ASCII characters?

 Given the initialisation problems, this may be something that PEP 432 (the 
 initialisation process rewrite) can help with (since it changes the 
 initialisation order to create a more complete Python runtime before it 
 starts to configure the OS interfaces).

I don't see how it would help to solve my point (b).

Technically, this issue cannot be fixed. Or to be more specific, I
don't want to fix it, it's a waste of time. So I don't understand what
do you expect from this open issue?

I would prefer to close it as invalid or wontfix to be clear.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Nick Coghlan]

Nick Coghlan added the comment:

On 9 December 2013 12:08, STINNER Victor rep...@bugs.python.org wrote:

 STINNER Victor added the comment:

 End users tripping over this by setting LANG=C is one of the pain points of 
 Python 3 relative to Python 2 for Fedora, so I've added a couple of Fedora 
 folks to the nosy list.

 Sorry, I'm not aware of such issue. Do you have examples?

Armin's travails with remote shell access and Python 3 are just as
likely today as they were a couple of years ago:
http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3/

(although technically that was a terminal ending up with the POSIX
locale, rather than specifically LANG=C)

 - the main problem is on Linux (but potentially other *nix systems as well), 
 where people set LANG=C for a variety of reasons, but this has the side 
 effect of Python 3 choosing an inappropriate encoding (ASCII rather than 
 UTF-8) when talking to the OS APIs.

 Why do you think that the issue is specific to Python 3? Try to open a
 terminal with LC_ALL=C and try to type non-ASCII characters with your
 keyboard. You can't because your terminal uses ASCII. Did you
 applications written in another language handling Unicode, like Perl?
 (Perl with Unicode support correctly enabled, it's use utf8; if I
 remember correctly).

It's the fact this used to work transparently in Python 2 (since all
these interfaces were just bytes based on the Python side as well)
that's a problem. That makes the new sensitivity to the locale
encoding a usability regression, and that's a concern for distros that
are considering switching their default Python version.

 Can you explain the various reasons why users explictly force the
 encoding to ASCII?

- testing applications for POSIX compliance
- default settings on servers where you don't control the environment
- because they never previously had to care, and it's only Python 3
deciding to pay attention to it which makes it relevent for them

 I use LANG=C to get manual pages and error messages in english. But
 LANG=en_US man ls would be more correct, or LC_MESSAGES=en_US man
 ls to be pedantic. (Env var priority: LC_ALL  LANG  LC_xxx).

 IMO if you use LANG=C, you must not complain that Unicode stopped
 working, but you should learn how to configure locales. Trivial
 examples like the one which can be found in the initial message
 (msg204849) are wrong: why would you force all locales to C and use
 non-ASCII characters?

And yet, in Python 2, people could do that, and Python didn't care.
*That's* the regression I'm worried about. If it hadn't round-tripped
cleanly in Python 2, I wouldn't care here either.

 Given the initialisation problems, this may be something that PEP 432 (the 
 initialisation process rewrite) can help with (since it changes the 
 initialisation order to create a more complete Python runtime before it 
 starts to configure the OS interfaces).

 I don't see how it would help to solve my point (b).

Having a Python runtime available makes things that are currently
tediously painful to deal with during startup easier to tweak. I'm not
sure it *will* help in this particular case, but it's now one I'm
going to keep an eye on.

 Technically, this issue cannot be fixed. Or to be more specific, I
 don't want to fix it, it's a waste of time. So I don't understand what
 do you expect from this open issue?

A way to get Python 3 to cope as well with a misconfigured OS
environment as Python 2 did.

 I would prefer to close it as invalid or wontfix to be clear.

It's a usability regression from Python 2, so I don't want to give up
on it. It may be that we just implement a ignore what the OS claims,
it's misconfigured, just use UTF-8 for everything flag. But OS
configuration errors shouldn't cripple the Python runtime.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19846] Setting LANG=C breaks Python 3 on Linux

[Sworddragon]

Sworddragon added the comment:

You should keep things more simple:

- Python and the operation system/filesystem are in a client-server 
relationship and Python should validate all.
- It doesn't matter what you will finally decide to be the default encoding on 
various places - all will provide race-conditions with no exception.
- The easiest way to fix this is to give the developer the ability to make a 
decision (like sys.use_strict_encoding(), sys.setfilesystemencoding(), 
sys.setdefaultencoding() etc.).
* For example giving the developer control is especially needed if he wants to 
handle multiple different filesystems.


 Why do you think that the issue is specific to Python 3? Try to open a
 terminal with LC_ALL=C and try to type non-ASCII characters with your
 keyboard. You can't because your terminal uses ASCII.

sworddragon@ubuntu:~$ LANG=C
sworddragon@ubuntu:~$ ä
bash: $'\303\244': command not found

- The terminal doesn't pseudo-crash with an exception because it doesn't matter 
about encodings.
- It allows to change the encoding at runtime.


 Did you
 applications written in another language handling Unicode, like Perl?

Compare C: It wouldn't matter like the terminal. For example fopen will simply 
return NULL if it can't open the file 'ä' because the filesystem is endoded 
with ISO-8859-1 and we wanted to open the utf-8 counterpart.


 Can you explain the various reasons why users explictly force the
 encoding to ASCII?

For example I'm using this for testcases to set the language uncomplicated to 
english.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19846
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Implementing #define macros similar to C on python

[Serhiy Storchaka]

15.11.13 06:57, Chris Angelico написав(ла):

On Fri, Nov 15, 2013 at 3:10 PM, Roy Smith r...@panix.com wrote:

Why would you want to?  One of the most horrible things about C/C++ is
the preprocessor.


Hey, that's not fair! Without the preprocessor, how would you be able
to do this:

//Hide this part away in a header file somewhere
struct b0rkb0rk
{
 float value;
 b0rkb0rk(float v):value(v) {}
 operator float() {return value;}
 float operator +(float other) {return value+other-0.1;}
};
//Behold the power of the preprocessor!
#define float b0rkb0rk

//Okay, now here's your application
#include iostream

int main()
{
 std::cout  Look how stupidly inaccurate float is!\n;
 float x = 123.0f;
 std::cout  123.0 + 2.0 =   x + 2.0f  \n;
 std::cout  See? You should totally use double instead.\n;
}

(Anybody got a cheek de-tonguer handy? I think it's stuck.)


 class b0rkb0rk(float):
... def __add__(self, other):
... return super().__add__(other) - 0.1
...
 import builtins
 builtins.float = b0rkb0rk
 float(123) + 2
124.9


--
https://mail.python.org/mailman/listinfo/python-list

Re: Implementing #define macros similar to C on python

[Mark Lawrence]

On 16/11/2013 05:38, JL wrote:

On Saturday, November 16, 2013 8:22:25 AM UTC+8, Mark Lawrence wrote:


Yes but please don't top post.  Actually print is a statement in Python
2 so your code should work if you use
from __future__ import print_function
at the top of your code.
Would you also be kind enough to read and action this
https://wiki.python.org/moin/GoogleGroupsPython to prevent the double
line spacing shown above, thanks.


Thank you for the tip. Will try that out. Hope I get the posting etiquette 
right this time.



No problem.  It's not a matter of etiquette, it's using a tool that's 
not flawed :)


--
Python is the second best programming language in the world.
But the best has yet to be invented.  Christian Tismer

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Implementing #define macros similar to C on python

[Irmen de Jong]

On 15-11-2013 3:29, JL wrote:
 One of my favorite tools in C/C++ language is the preprocessor macros.
 
 One example is switching certain print messages for debugging use only
 
 #ifdef DEBUG_ENABLE
 DEBUG_PRINT   print
 #else
 DEBUG_PRINT
 
 Is it possible to implement something similar in python? Thank you.
 

You could just run cpp (or gcc -E) on your python-with-macros-file to generate 
the final
.py file. But: yuck, eww, gross.

Irmen


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Implementing #define macros similar to C on python

[JL]

Thanks! This is the answer which I am seeking. However, I am not able to get 
the following line to work. I am using python 2.7.5

debug_print = print

Can we assign a function into a variable in this manner?

On Friday, November 15, 2013 11:49:52 AM UTC+8, Chris Angelico wrote:
 On Fri, Nov 15, 2013 at 1:29 PM, JL lightai...@gmail.com wrote:
 
  One of my favorite tools in C/C++ language is the preprocessor macros.
 
 
 
  One example is switching certain print messages for debugging use only
 
 
 
  #ifdef DEBUG_ENABLE
 
  DEBUG_PRINT   print
 
  #else
 
  DEBUG_PRINT
 
 
 
  Is it possible to implement something similar in python? Thank you.
 
 
 
 There are usually other ways to do things. For instance, you can
 
 define a function to either do something or do nothing:
 
 
 
 if debug_mode:
 
 debug_print = print
 
 else:
 
 debug_print = lambda: None
 
 
 
 debug_print(This won't be shown unless we're in debug mode!)
 
 
 
 But as Dave says, you could write a preprocessor if you need one.
 
 
 
 ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list