[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 5f622f1d5c5425ed1e992da6611edfb486a9bf7c by Miss Islington (bot) 
in branch '3.9':
bpo-45822: Minor cleanups to the test_Py_CompileString test (GH-29750) 
(GH-29759)
https://github.com/python/cpython/commit/5f622f1d5c5425ed1e992da6611edfb486a9bf7c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset e1e3f648ad6ab467356d2d14e1d208583c1a76c6 by Miss Islington (bot) 
in branch '3.10':
bpo-45822: Minor cleanups to the test_Py_CompileString test (GH-29750) 
(GH-29758)
https://github.com/python/cpython/commit/e1e3f648ad6ab467356d2d14e1d208583c1a76c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[miss-islington]

Change by miss-islington :


--
pull_requests: +27997
pull_request: https://github.com/python/cpython/pull/29759

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset abfc794bbf2c6a0939ddd81b6e700c46944ba87a by Pablo Galindo Salgado 
in branch 'main':
bpo-45822: Minor cleanups to the test_Py_CompileString test (GH-29750)
https://github.com/python/cpython/commit/abfc794bbf2c6a0939ddd81b6e700c46944ba87a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[miss-islington]

Change by miss-islington :


--
pull_requests: +27996
pull_request: https://github.com/python/cpython/pull/29758

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +27987
pull_request: https://github.com/python/cpython/pull/29750

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 0ef308a2890571c850c624fb99ac00f8951363c6 by Pablo Galindo Salgado 
in branch '3.9':
bpo-45822: Respect PEP 263's coding cookies in the parser even if flags are not 
provided (GH-29582) (GH-29585)
https://github.com/python/cpython/commit/0ef308a2890571c850c624fb99ac00f8951363c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset e3aa9fd77bf474bb3e8a7a1d1bd1ebf45147945a by Pablo Galindo Salgado 
in branch '3.10':
[3.10] bpo-45822: Respect PEP 263's coding cookies in the parser even if flags 
are not provided (GH-29582) (GH-29586)
https://github.com/python/cpython/commit/e3aa9fd77bf474bb3e8a7a1d1bd1ebf45147945a


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +27829
pull_request: https://github.com/python/cpython/pull/29586

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +27828
pull_request: https://github.com/python/cpython/pull/29585

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[miss-islington]

miss-islington  added the comment:


New changeset da20d7401de97b425897d3069f71f77b039eb16f by Pablo Galindo Salgado 
in branch 'main':
bpo-45822: Respect PEP 263's coding cookies in the parser even if flags are not 
provided (GH-29582)
https://github.com/python/cpython/commit/da20d7401de97b425897d3069f71f77b039eb16f


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Thomas Wouters]

Thomas Wouters  added the comment:

Py_CompileString() in Python 3.9 and later, using the PEG parser, appears to no 
longer honours source encoding cookies. A reduced test case:

#include "Python.h"
#include 

const char *src = (
"# -*- coding: Latin-1 -*-\n"
"'''\xc3'''\n");

int main(int argc, char **argv)
{
Py_Initialize();
PyObject *res = Py_CompileString(src, "some_path", Py_file_input);
if (res) {
fprintf(stderr, "Compile succeeded.\n");
return 0;
} else {
fprintf(stderr, "Compile failed.\n");
PyErr_Print();
return 1;
}
}

Compiling and running the resulting binary with Python 3.8 (or earlier):

% ./encoding_bug
Compile succeeded.

With 3.9 and PYTHONOLDPARSER=1:

% PYTHONOLDPARSER=1 ./encoding_bug
Compile succeeded.

With 3.9 (without the env var) or 3.10:
% ./encoding_bug
Compile failed.
  File "some_path", line 2
'''�'''
 ^
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xc3 in 
position 0: unexpected end of data

Writing the same bytes to a file and making python3.9 or python3.10 import them 
works fine, as does passing the bytes to compile():

Python 3.10.0+ (heads/3.10-dirty:7bac598819, Nov 16 2021, 20:35:12) [GCC 
8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> b = open('encoding_bug.py', 'rb').read()
>>> b
b"# -*- coding: Latin-1 -*-\n'''\xc3'''\n"
>>> import encoding_bug
>>> encoding_bug.__doc__
'Ã'
>>> co = compile(b, 'some_path', 'exec')
>>> co
 at 0x7f447e1b0c90, file "some_path", line 1>
>>> co.co_consts[0]
'Ã'


It's just Py_CompileString() that fails. I don't understand why, and I do 
believe it's a regression.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +27825
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29582

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45822] Py_CompileString does not respect the coding cookie with the new parser if flags are empty

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

When executing Py_CompileString with a source string that has a coding cookie, 
this is not respected as with the old parser.

--
components: Parser
messages: 406425
nosy: lys.nikolaou, pablogsal, twouters
priority: normal
severity: normal
status: open
title: Py_CompileString does not respect the coding cookie with the new parser 
if flags are empty
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com