[Python-checkins] gh-117606: Truncate extremely long error message in `test_exceptions` (#117670)
https://github.com/python/cpython/commit/02f1385f8ad6bf45376377c41f106b386d3a7eb0
commit: 02f1385f8ad6bf45376377c41f106b386d3a7eb0
branch: main
author: Nice Zombies
committer: AlexWaygood
date: 2024-04-11T08:37:01+01:00
summary:
gh-117606: Truncate extremely long error message in `test_exceptions` (#117670)
Co-authored-by: Alex Waygood
files:
M Lib/test/test_exceptions.py
diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
index 36fd89dbb8896c..1224f143b5441f 100644
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -1451,7 +1451,8 @@ def test_recursion_normalizing_infinite_exception(self):
"""
rc, out, err = script_helper.assert_python_failure("-c", code)
self.assertEqual(rc, 1)
-self.assertIn(b'RecursionError: maximum recursion depth exceeded', err)
+expected = b'RecursionError: maximum recursion depth exceeded'
+self.assertTrue(expected in err, msg=f"{expected!r} not found in
{err[:3_000]!r}... (truncated)")
self.assertIn(b'Done.', out)
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-117606: Truncate extremely long error message in `test_exceptions` (GH-117670) (#117745)
https://github.com/python/cpython/commit/2a508572ab86320c80edfa829329232ad5a4b8e9 commit: 2a508572ab86320c80edfa829329232ad5a4b8e9 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: AlexWaygood date: 2024-04-11T07:53:26Z summary: [3.12] gh-117606: Truncate extremely long error message in `test_exceptions` (GH-117670) (#117745) gh-117606: Truncate extremely long error message in `test_exceptions` (GH-117670) (cherry picked from commit 02f1385f8ad6bf45376377c41f106b386d3a7eb0) Co-authored-by: Nice Zombies Co-authored-by: Alex Waygood files: M Lib/test/test_exceptions.py diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 6c09c1793c8d7f..b738ec6a0324dd 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -1448,7 +1448,8 @@ def test_recursion_normalizing_infinite_exception(self): """ rc, out, err = script_helper.assert_python_failure("-c", code) self.assertEqual(rc, 1) -self.assertIn(b'RecursionError: maximum recursion depth exceeded', err) +expected = b'RecursionError: maximum recursion depth exceeded' +self.assertTrue(expected in err, msg=f"{expected!r} not found in {err[:3_000]!r}... (truncated)") self.assertIn(b'Done.', out) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-117711: Only check for 'test/wheeldata' when it's actually used (#117712)
https://github.com/python/cpython/commit/d4963871b03cc76fe7d9648d022d12007585beae commit: d4963871b03cc76fe7d9648d022d12007585beae branch: main author: Karolina Surma <[email protected]> committer: vstinner date: 2024-04-11T11:37:28+02:00 summary: gh-117711: Only check for 'test/wheeldata' when it's actually used (#117712) It's possible to build Python with option `--with-wheel-pkg-dir` pointing to a custom wheel directory. Don't include the directory in the test set if the wheels are used from a different location. Co-authored-by: Miro Hrončok files: M Lib/test/test_tools/test_makefile.py diff --git a/Lib/test/test_tools/test_makefile.py b/Lib/test/test_tools/test_makefile.py index 29f5c28e33bb2b..48a7c1a773bb83 100644 --- a/Lib/test/test_tools/test_makefile.py +++ b/Lib/test/test_tools/test_makefile.py @@ -67,6 +67,10 @@ def test_makefile_test_folders(self): ) used.append(relpath) +# Don't check the wheel dir when Python is built --with-wheel-pkg-dir +if sysconfig.get_config_var('WHEEL_PKG_DIR'): +test_dirs.remove('test/wheeldata') + # Check that there are no extra entries: unique_test_dirs = set(test_dirs) self.assertSetEqual(unique_test_dirs, set(used)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113317: Add Codegen class to Argument Clinic (#117626)
https://github.com/python/cpython/commit/a2ae84726b8b46e6970ae862244dad1a82cf5d19
commit: a2ae84726b8b46e6970ae862244dad1a82cf5d19
branch: main
author: Victor Stinner
committer: vstinner
date: 2024-04-11T12:15:48+02:00
summary:
gh-113317: Add Codegen class to Argument Clinic (#117626)
* Move ifndef_symbols, includes and add_include() from Clinic to
Codegen. Add a 'codegen' (Codegen) attribute to Clinic.
* Remove libclinic.crenderdata module: move code to libclinic.codegen.
* BlockPrinter.print_block(): remove unused 'limited_capi' argument.
Remove also 'core_includes' parameter.
* Add get_includes() methods.
* Make Codegen.ifndef_symbols private.
* Make Codegen.includes private.
* Make CConverter.includes private.
files:
D Tools/clinic/libclinic/crenderdata.py
M Lib/test/test_clinic.py
M Tools/clinic/libclinic/app.py
M Tools/clinic/libclinic/clanguage.py
M Tools/clinic/libclinic/codegen.py
M Tools/clinic/libclinic/converter.py
M Tools/clinic/libclinic/converters.py
M Tools/clinic/libclinic/return_converters.py
diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py
index e3ba3d943216de..e8c638ea610782 100644
--- a/Lib/test/test_clinic.py
+++ b/Lib/test/test_clinic.py
@@ -877,9 +877,8 @@ def _test(self, input, output):
blocks = list(BlockParser(input, language))
writer = BlockPrinter(language)
-c = _make_clinic()
for block in blocks:
-writer.print_block(block, limited_capi=c.limited_capi,
header_includes=c.includes)
+writer.print_block(block)
output = writer.f.getvalue()
assert output == input, "output != input!\n\noutput " + repr(output) +
"\n\n input " + repr(input)
diff --git a/Tools/clinic/libclinic/app.py b/Tools/clinic/libclinic/app.py
index 47a897712d053e..62e9892eb95a2b 100644
--- a/Tools/clinic/libclinic/app.py
+++ b/Tools/clinic/libclinic/app.py
@@ -9,8 +9,7 @@
from libclinic import fail, warn
from libclinic.function import Class
from libclinic.block_parser import Block, BlockParser
-from libclinic.crenderdata import Include
-from libclinic.codegen import BlockPrinter, Destination
+from libclinic.codegen import BlockPrinter, Destination, Codegen
from libclinic.parser import Parser, PythonParser
from libclinic.dsl_parser import DSLParser
if TYPE_CHECKING:
@@ -102,8 +101,7 @@ def __init__(
self.modules: ModuleDict = {}
self.classes: ClassDict = {}
self.functions: list[Function] = []
-# dict: include name => Include instance
-self.includes: dict[str, Include] = {}
+self.codegen = Codegen(self.limited_capi)
self.line_prefix = self.line_suffix = ''
@@ -132,7 +130,6 @@ def __init__(
DestBufferList = list[DestBufferType]
self.destination_buffers_stack: DestBufferList = []
-self.ifndef_symbols: set[str] = set()
self.presets: dict[str, dict[Any, Any]] = {}
preset = None
@@ -159,24 +156,6 @@ def __init__(
assert name in self.destination_buffers
preset[name] = buffer
-def add_include(self, name: str, reason: str,
-*, condition: str | None = None) -> None:
-try:
-existing = self.includes[name]
-except KeyError:
-pass
-else:
-if existing.condition and not condition:
-# If the previous include has a condition and the new one is
-# unconditional, override the include.
-pass
-else:
-# Already included, do nothing. Only mention a single reason,
-# no need to list all of them.
-return
-
-self.includes[name] = Include(name, reason, condition)
-
def add_destination(
self,
name: str,
@@ -212,9 +191,7 @@ def parse(self, input: str) -> str:
self.parsers[dsl_name] = parsers[dsl_name](self)
parser = self.parsers[dsl_name]
parser.parse(block)
-printer.print_block(block,
-limited_capi=self.limited_capi,
-header_includes=self.includes)
+printer.print_block(block)
# these are destinations not buffers
for name, destination in self.destinations.items():
@@ -229,9 +206,7 @@ def parse(self, input: str) -> str:
block.input = "dump " + name + "\n"
warn("Destination buffer " + repr(name) + " not empty at
end of file, emptying.")
printer.write("\n")
-printer.print_block(block,
-limited_capi=self.limited_capi,
-header_includes=self.includes)
+printer.print_block(block)
continue
if destination.type == 'file':
@@ -255,11 +230,10 @@ def parse(self, input: str) -> str:
pass
[Python-checkins] [3.12] gh-117711: Only check for 'test/wheeldata' when it's actually used (GH-117712) (#117749)
https://github.com/python/cpython/commit/5b681d60a6501e6c733546bc87c8631b23b80e45 commit: 5b681d60a6501e6c733546bc87c8631b23b80e45 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: vstinner date: 2024-04-11T10:22:17Z summary: [3.12] gh-117711: Only check for 'test/wheeldata' when it's actually used (GH-117712) (#117749) gh-117711: Only check for 'test/wheeldata' when it's actually used (GH-117712) It's possible to build Python with option `--with-wheel-pkg-dir` pointing to a custom wheel directory. Don't include the directory in the test set if the wheels are used from a different location. (cherry picked from commit d4963871b03cc76fe7d9648d022d12007585beae) Co-authored-by: Karolina Surma <[email protected]> Co-authored-by: Miro Hrončok files: M Lib/test/test_tools/test_makefile.py diff --git a/Lib/test/test_tools/test_makefile.py b/Lib/test/test_tools/test_makefile.py index 17a1a6d0d38d7d..e253bd0049ffa6 100644 --- a/Lib/test/test_tools/test_makefile.py +++ b/Lib/test/test_tools/test_makefile.py @@ -66,6 +66,10 @@ def test_makefile_test_folders(self): ) used.append(relpath) +# Don't check the wheel dir when Python is built --with-wheel-pkg-dir +if sysconfig.get_config_var('WHEEL_PKG_DIR'): +test_dirs.remove('test/wheeldata') + # Check that there are no extra entries: unique_test_dirs = set(test_dirs) self.assertSetEqual(unique_test_dirs, set(used)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113317: Add ParseArgsCodeGen class (#117707)
https://github.com/python/cpython/commit/671cb22094df5b645f65c383213bfda17c8003c5
commit: 671cb22094df5b645f65c383213bfda17c8003c5
branch: main
author: Victor Stinner
committer: vstinner
date: 2024-04-11T13:49:07Z
summary:
gh-113317: Add ParseArgsCodeGen class (#117707)
files:
A Tools/clinic/libclinic/parse_args.py
M Tools/clinic/libclinic/app.py
M Tools/clinic/libclinic/clanguage.py
M Tools/clinic/libclinic/codegen.py
diff --git a/Tools/clinic/libclinic/app.py b/Tools/clinic/libclinic/app.py
index 62e9892eb95a2b..632bed3ce53dde 100644
--- a/Tools/clinic/libclinic/app.py
+++ b/Tools/clinic/libclinic/app.py
@@ -9,7 +9,7 @@
from libclinic import fail, warn
from libclinic.function import Class
from libclinic.block_parser import Block, BlockParser
-from libclinic.codegen import BlockPrinter, Destination, Codegen
+from libclinic.codegen import BlockPrinter, Destination, CodeGen
from libclinic.parser import Parser, PythonParser
from libclinic.dsl_parser import DSLParser
if TYPE_CHECKING:
@@ -101,7 +101,7 @@ def __init__(
self.modules: ModuleDict = {}
self.classes: ClassDict = {}
self.functions: list[Function] = []
-self.codegen = Codegen(self.limited_capi)
+self.codegen = CodeGen(self.limited_capi)
self.line_prefix = self.line_suffix = ''
diff --git a/Tools/clinic/libclinic/clanguage.py
b/Tools/clinic/libclinic/clanguage.py
index 5d1a273312dec5..10efedd5cb9cea 100644
--- a/Tools/clinic/libclinic/clanguage.py
+++ b/Tools/clinic/libclinic/clanguage.py
@@ -8,93 +8,19 @@
import libclinic
from libclinic import (
-unspecified, fail, warn, Sentinels, VersionTuple)
-from libclinic.function import (
-GETTER, SETTER, METHOD_INIT, METHOD_NEW)
-from libclinic.codegen import CRenderData, TemplateDict, Codegen
+unspecified, fail, Sentinels, VersionTuple)
+from libclinic.codegen import CRenderData, TemplateDict, CodeGen
from libclinic.language import Language
from libclinic.function import (
Module, Class, Function, Parameter,
-permute_optional_groups)
-from libclinic.converters import (
-defining_class_converter, object_converter, self_converter)
+permute_optional_groups,
+GETTER, SETTER, METHOD_INIT)
+from libclinic.converters import self_converter
+from libclinic.parse_args import ParseArgsCodeGen
if TYPE_CHECKING:
from libclinic.app import Clinic
-def declare_parser(
-f: Function,
-*,
-hasformat: bool = False,
-codegen: Codegen,
-) -> str:
-"""
-Generates the code template for a static local PyArg_Parser variable,
-with an initializer. For core code (incl. builtin modules) the
-kwtuple field is also statically initialized. Otherwise
-it is initialized at runtime.
-"""
-limited_capi = codegen.limited_capi
-if hasformat:
-fname = ''
-format_ = '.format = "{format_units}:{name}",'
-else:
-fname = '.fname = "{name}",'
-format_ = ''
-
-num_keywords = len([
-p for p in f.parameters.values()
-if not p.is_positional_only() and not p.is_vararg()
-])
-if limited_capi:
-declarations = """
-#define KWTUPLE NULL
-"""
-elif num_keywords == 0:
-declarations = """
-#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
-# define KWTUPLE (PyObject *)&_Py_SINGLETON(tuple_empty)
-#else
-# define KWTUPLE NULL
-#endif
-"""
-else:
-declarations = """
-#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
-
-#define NUM_KEYWORDS %d
-static struct {{
-PyGC_Head _this_is_not_used;
-PyObject_VAR_HEAD
-PyObject *ob_item[NUM_KEYWORDS];
-}} _kwtuple = {{
-.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
-.ob_item = {{ {keywords_py} }},
-}};
-#undef NUM_KEYWORDS
-#define KWTUPLE (&_kwtuple.ob_base.ob_base)
-
-#else // !Py_BUILD_CORE
-# define KWTUPLE NULL
-#endif // !Py_BUILD_CORE
-""" % num_keywords
-
-condition = '#if defined(Py_BUILD_CORE) &&
!defined(Py_BUILD_CORE_MODULE)'
-codegen.add_include('pycore_gc.h', 'PyGC_Head', condition=condition)
-codegen.add_include('pycore_runtime.h', '_Py_ID()',
condition=condition)
-
-declarations += """
-static const char * const _keywords[] = {{{keywords_c} NULL}};
-static _PyArg_Parser _parser = {{
-.keywords = _keywords,
-%s
-.kwtuple = KWTUPLE,
-}};
-#undef KWTUPLE
-""" % (format_ or fname)
-return libclinic.normalize_snippet(declarations)
-
-
class CLanguage(Language):
body_prefix = "#"
@@ -104,99 +30,6 @@ class CLanguage(Language):
stop_line = "[{dsl_name} start generated code]*/"
[Python-checkins] gh-117709: Add vectorcall support for str() with positional-only arguments (#117746)
https://github.com/python/cpython/commit/044dc496e06843cd3eb30a32f34d9d080635875a
commit: 044dc496e06843cd3eb30a32f34d9d080635875a
branch: main
author: Erlend E. Aasland
committer: erlend-aasland
date: 2024-04-11T13:55:37Z
summary:
gh-117709: Add vectorcall support for str() with positional-only arguments
(#117746)
Fall back to tp_call() for cases when arguments are passed by name.
Co-authored-by: Donghee Na
Co-authored-by: Victor Stinner
files:
A Misc/NEWS.d/next/Core and
Builtins/2024-04-10-22-16-18.gh-issue-117709.-_1YL0.rst
M Lib/test/test_str.py
M Objects/unicodeobject.c
diff --git a/Lib/test/test_str.py b/Lib/test/test_str.py
index b4927113db44e3..ea37eb5d96457d 100644
--- a/Lib/test/test_str.py
+++ b/Lib/test/test_str.py
@@ -2651,6 +2651,24 @@ def test_check_encoding_errors(self):
proc = assert_python_failure('-X', 'dev', '-c', code)
self.assertEqual(proc.rc, 10, proc)
+def test_str_invalid_call(self):
+check = lambda *a, **kw: self.assertRaises(TypeError, str, *a, **kw)
+
+# too many args
+check(1, "", "", 1)
+
+# no such kw arg
+check(test=1)
+
+# 'encoding' must be str
+check(1, encoding=1)
+check(1, 1)
+
+# 'errors' must be str
+check(1, errors=1)
+check(1, "", errors=1)
+check(1, 1, 1)
+
class StringModuleTest(unittest.TestCase):
def test_formatter_parser(self):
diff --git a/Misc/NEWS.d/next/Core and
Builtins/2024-04-10-22-16-18.gh-issue-117709.-_1YL0.rst b/Misc/NEWS.d/next/Core
and Builtins/2024-04-10-22-16-18.gh-issue-117709.-_1YL0.rst
new file mode 100644
index 00..2216b53688c378
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and
Builtins/2024-04-10-22-16-18.gh-issue-117709.-_1YL0.rst
@@ -0,0 +1,3 @@
+Speed up calls to :func:`str` with positional-only argument,
+by using the :pep:`590` ``vectorcall`` calling convention.
+Patch by Erlend Aasland.
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
index 5f15071d7d54ef..2c259b7e869efe 100644
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -14617,6 +14617,56 @@ unicode_new_impl(PyTypeObject *type, PyObject *x,
const char *encoding,
return unicode;
}
+static const char *
+arg_as_utf8(PyObject *obj, const char *name)
+{
+if (!PyUnicode_Check(obj)) {
+PyErr_Format(PyExc_TypeError,
+ "str() argument '%s' must be str, not %T",
+ name, obj);
+return NULL;
+}
+return _PyUnicode_AsUTF8NoNUL(obj);
+}
+
+static PyObject *
+unicode_vectorcall(PyObject *type, PyObject *const *args,
+ size_t nargsf, PyObject *kwnames)
+{
+assert(Py_Is(_PyType_CAST(type), &PyUnicode_Type));
+
+Py_ssize_t nargs = PyVectorcall_NARGS(nargsf);
+if (kwnames != NULL && PyTuple_GET_SIZE(kwnames) != 0) {
+// Fallback to unicode_new()
+PyObject *tuple = _PyTuple_FromArray(args, nargs);
+if (tuple == NULL) {
+return NULL;
+}
+PyObject *dict = _PyStack_AsDict(args + nargs, kwnames);
+if (dict == NULL) {
+Py_DECREF(tuple);
+return NULL;
+}
+PyObject *ret = unicode_new(_PyType_CAST(type), tuple, dict);
+Py_DECREF(tuple);
+Py_DECREF(dict);
+return ret;
+}
+if (!_PyArg_CheckPositional("str", nargs, 0, 3)) {
+return NULL;
+}
+if (nargs == 0) {
+return unicode_get_empty();
+}
+PyObject *object = args[0];
+if (nargs == 1) {
+return PyObject_Str(object);
+}
+const char *encoding = arg_as_utf8(args[1], "encoding");
+const char *errors = (nargs == 3) ? arg_as_utf8(args[2], "errors") : NULL;
+return PyUnicode_FromEncodedObject(object, encoding, errors);
+}
+
static PyObject *
unicode_subtype_new(PyTypeObject *type, PyObject *unicode)
{
@@ -14758,6 +14808,7 @@ PyTypeObject PyUnicode_Type = {
0,/* tp_alloc */
unicode_new, /* tp_new */
PyObject_Del, /* tp_free */
+.tp_vectorcall = unicode_vectorcall,
};
/* Initialize the Unicode implementation */
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] Fix markup of `win32_ver` in `platform.rst` (#116492)
https://github.com/python/cpython/commit/be82058064f34f3f3b9736927da2d3057838e70c commit: be82058064f34f3f3b9736927da2d3057838e70c branch: main author: Nikita Sobolev committer: hugovk <[email protected]> date: 2024-04-11T17:13:53+03:00 summary: Fix markup of `win32_ver` in `platform.rst` (#116492) files: M Doc/library/platform.rst diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index 069dab791dcbe5..66af37e3073852 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -219,8 +219,8 @@ Windows Platform default to an empty string). As a hint: *ptype* is ``'Uniprocessor Free'`` on single processor NT machines - and ``'Multiprocessor Free'`` on multi processor machines. The *'Free'* refers - to the OS version being free of debugging code. It could also state *'Checked'* + and ``'Multiprocessor Free'`` on multi processor machines. The ``'Free'`` refers + to the OS version being free of debugging code. It could also state ``'Checked'`` which means the OS version uses debugging code, i.e. code that checks arguments, ranges, etc. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-115142: Skip `test_capi.test_dict.py` if `_testcapi` is not available (GH-117588) (GH-117718)
https://github.com/python/cpython/commit/8c1c71ee3819ef9bcf106f39206335de45e3d11a
commit: 8c1c71ee3819ef9bcf106f39206335de45e3d11a
branch: 3.12
author: Kirill Podoprigora
committer: encukou
date: 2024-04-11T16:18:28+02:00
summary:
[3.12] gh-115142: Skip `test_capi.test_dict.py` if `_testcapi` is not available
(GH-117588) (GH-117718)
(cherry picked from commit dfcae4379f2cc4d352a180f9fef2381570aa9bcb)
files:
M Lib/test/test_capi/test_dict.py
diff --git a/Lib/test/test_capi/test_dict.py b/Lib/test/test_capi/test_dict.py
index 0717708fcce8ef..3ec2374a901794 100644
--- a/Lib/test/test_capi/test_dict.py
+++ b/Lib/test/test_capi/test_dict.py
@@ -4,7 +4,9 @@
from types import MappingProxyType
from test import support
from test.support import import_helper
-import _testcapi
+
+
+_testcapi = import_helper.import_module("_testcapi")
NULL = None
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-115142: Skip ``test_capi/test_opt.py`` if ``_testinternalcapi`` is not available (GH-117574)
https://github.com/python/cpython/commit/91d7605ac3c15a185fd67921907cebaf0def1674
commit: 91d7605ac3c15a185fd67921907cebaf0def1674
branch: main
author: Kirill Podoprigora
committer: encukou
date: 2024-04-11T16:19:19+02:00
summary:
gh-115142: Skip ``test_capi/test_opt.py`` if ``_testinternalcapi`` is not
available (GH-117574)
files:
M Lib/test/test_capi/test_opt.py
diff --git a/Lib/test/test_capi/test_opt.py b/Lib/test/test_capi/test_opt.py
index 7ca0f6927fe4a1..28d18739b6d4a5 100644
--- a/Lib/test/test_capi/test_opt.py
+++ b/Lib/test/test_capi/test_opt.py
@@ -6,9 +6,10 @@
import os
import _opcode
-import _testinternalcapi
-from test.support import script_helper, requires_specialization
+from test.support import script_helper, requires_specialization, import_helper
+
+_testinternalcapi = import_helper.import_module("_testinternalcapi")
from _testinternalcapi import TIER2_THRESHOLD
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] Fix markup of `win32_ver` in `platform.rst` (GH-116492) (#117753)
https://github.com/python/cpython/commit/a4541a60807af2166811e01437477efcbf9d7b86 commit: a4541a60807af2166811e01437477efcbf9d7b86 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: hugovk <[email protected]> date: 2024-04-11T14:20:29Z summary: [3.12] Fix markup of `win32_ver` in `platform.rst` (GH-116492) (#117753) Co-authored-by: Nikita Sobolev files: M Doc/library/platform.rst diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index ec2a7ebd5d6e0b..e0c539fa3da084 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -210,8 +210,8 @@ Windows Platform default to an empty string). As a hint: *ptype* is ``'Uniprocessor Free'`` on single processor NT machines - and ``'Multiprocessor Free'`` on multi processor machines. The *'Free'* refers - to the OS version being free of debugging code. It could also state *'Checked'* + and ``'Multiprocessor Free'`` on multi processor machines. The ``'Free'`` refers + to the OS version being free of debugging code. It could also state ``'Checked'`` which means the OS version uses debugging code, i.e. code that checks arguments, ranges, etc. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-101549: fix documentation of xml.etree.ElementInclude (#101550)
https://github.com/python/cpython/commit/898f6de63fd5285006ee0f4993aeb8ed3e8f97f9 commit: 898f6de63fd5285006ee0f4993aeb8ed3e8f97f9 branch: main author: Mikhail B <[email protected]> committer: hugovk <[email protected]> date: 2024-04-11T14:23:52Z summary: gh-101549: fix documentation of xml.etree.ElementInclude (#101550) Co-authored-by: Serhiy Storchaka Co-authored-by: Adam Turner <[email protected]> files: M Doc/library/xml.etree.elementtree.rst M Lib/xml/etree/ElementInclude.py diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 7d721f7633899e..30a7b653f940e9 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -840,33 +840,28 @@ Functions .. module:: xml.etree.ElementInclude -.. function:: xml.etree.ElementInclude.default_loader( href, parse, encoding=None) - :module: +.. function:: default_loader(href, parse, encoding=None) - Default loader. This default loader reads an included resource from disk. *href* is a URL. - *parse* is for parse mode either "xml" or "text". *encoding* - is an optional text encoding. If not given, encoding is ``utf-8``. Returns the - expanded resource. If the parse mode is ``"xml"``, this is an ElementTree - instance. If the parse mode is "text", this is a Unicode string. If the - loader fails, it can return None or raise an exception. + Default loader. This default loader reads an included resource from disk. + *href* is a URL. *parse* is for parse mode either "xml" or "text". + *encoding* is an optional text encoding. If not given, encoding is ``utf-8``. + Returns the expanded resource. + If the parse mode is ``"xml"``, this is an :class:`~xml.etree.ElementTree.Element` instance. + If the parse mode is ``"text"``, this is a string. + If the loader fails, it can return ``None`` or raise an exception. -.. function:: xml.etree.ElementInclude.include( elem, loader=None, base_url=None, \ -max_depth=6) - :module: +.. function:: include(elem, loader=None, base_url=None, max_depth=6) - This function expands XInclude directives. *elem* is the root element. *loader* is - an optional resource loader. If omitted, it defaults to :func:`default_loader`. + This function expands XInclude directives in-place in tree pointed by *elem*. + *elem* is either the root :class:`~xml.etree.ElementTree.Element` or an + :class:`~xml.etree.ElementTree.ElementTree` instance to find such element. + *loader* is an optional resource loader. If omitted, it defaults to :func:`default_loader`. If given, it should be a callable that implements the same interface as :func:`default_loader`. *base_url* is base URL of the original file, to resolve relative include file references. *max_depth* is the maximum number of recursive - inclusions. Limited to reduce the risk of malicious content explosion. Pass a - negative value to disable the limitation. - - Returns the expanded resource. If the parse mode is - ``"xml"``, this is an ElementTree instance. If the parse mode is "text", - this is a Unicode string. If the loader fails, it can return None or - raise an exception. + inclusions. Limited to reduce the risk of malicious content explosion. + Pass ``None`` to disable the limitation. .. versionchanged:: 3.9 Added the *base_url* and *max_depth* parameters. diff --git a/Lib/xml/etree/ElementInclude.py b/Lib/xml/etree/ElementInclude.py index 40a9b22292479f..986e6c3bbe90f6 100644 --- a/Lib/xml/etree/ElementInclude.py +++ b/Lib/xml/etree/ElementInclude.py @@ -79,8 +79,8 @@ class LimitedRecursiveIncludeError(FatalIncludeError): # @param parse Parse mode. Either "xml" or "text". # @param encoding Optional text encoding (UTF-8 by default for "text"). # @return The expanded resource. If the parse mode is "xml", this -#is an ElementTree instance. If the parse mode is "text", this -#is a Unicode string. If the loader fails, it can return None +#is an Element instance. If the parse mode is "text", this +#is a string. If the loader fails, it can return None #or raise an OSError exception. # @throws OSError If the loader fails to load the resource. @@ -98,7 +98,7 @@ def default_loader(href, parse, encoding=None): ## # Expand XInclude directives. # -# @param elem Root element. +# @param elem Root Element or any ElementTree of a tree to be expanded # @param loader Optional resource loader. If omitted, it defaults # to {@link default_loader}. If given, it should be a callable # that implements the same interface as default_loader. @@ -106,12 +106,13 @@ def default_loader(href, parse, encoding=None): # relative include file references. # @param max_depth The maximum number of recursive inclusions. # Limited to reduce the risk of malicious co
[Python-checkins] gh-117722: Fix Stream.readuntil with non-bytes buffer objects (#117723)
https://github.com/python/cpython/commit/01a51f949475f1590eb5899f3002304060501ab2 commit: 01a51f949475f1590eb5899f3002304060501ab2 branch: main author: Bruce Merry <[email protected]> committer: gvanrossum date: 2024-04-11T07:41:55-07:00 summary: gh-117722: Fix Stream.readuntil with non-bytes buffer objects (#117723) gh-16429 introduced support for an iterable of separators in Stream.readuntil. Since bytes-like types are themselves iterable, this can introduce ambiguities in deciding whether the argument is an iterator of separators or a singleton separator. In gh-16429, only 'bytes' was considered a singleton, but this will break code that passes other buffer object types. Fix it by only supporting tuples rather than arbitrary iterables. Closes gh-117722. files: A Misc/NEWS.d/next/Library/2024-04-10-20-59-10.gh-issue-117722.oxIUEI.rst M Doc/library/asyncio-stream.rst M Doc/whatsnew/3.13.rst M Lib/asyncio/streams.py M Lib/test/test_asyncio/test_streams.py diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 6231b49b1e2431..3fdc79b3c6896c 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -260,7 +260,7 @@ StreamReader buffer is reset. The :attr:`IncompleteReadError.partial` attribute may contain a portion of the separator. - The *separator* may also be an :term:`iterable` of separators. In this + The *separator* may also be a tuple of separators. In this case the return value will be the shortest possible that has any separator as the suffix. For the purposes of :exc:`LimitOverrunError`, the shortest possible separator is considered to be the one that @@ -270,7 +270,7 @@ StreamReader .. versionchanged:: 3.13 - The *separator* parameter may now be an :term:`iterable` of + The *separator* parameter may now be a :class:`tuple` of separators. .. method:: at_eof() diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index d394fbe3b0c357..65985ddc65a86f 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -324,6 +324,10 @@ asyncio :exc:`asyncio.QueueShutDown`) for queue termination. (Contributed by Laurie Opperman and Yves Duprat in :gh:`104228`.) +* Accept a tuple of separators in :meth:`asyncio.StreamReader.readuntil`, + stopping when one of them is encountered. + (Contributed by Bruce Merry in :gh:`81322`.) + base64 -- diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index 4517ca22d74637..64aac4cc50d15a 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -591,17 +591,17 @@ async def readuntil(self, separator=b'\n'): LimitOverrunError exception will be raised, and the data will be left in the internal buffer, so it can be read again. -The ``separator`` may also be an iterable of separators. In this +The ``separator`` may also be a tuple of separators. In this case the return value will be the shortest possible that has any separator as the suffix. For the purposes of LimitOverrunError, the shortest possible separator is considered to be the one that matched. """ -if isinstance(separator, bytes): -separator = [separator] -else: -# Makes sure shortest matches wins, and supports arbitrary iterables +if isinstance(separator, tuple): +# Makes sure shortest matches wins separator = sorted(separator, key=len) +else: +separator = [separator] if not separator: raise ValueError('Separator should contain at least one element') min_seplen = len(separator[0]) diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py index 792e88761acdc2..ae943f39869815 100644 --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -384,9 +384,9 @@ def test_readuntil_separator(self): with self.assertRaisesRegex(ValueError, 'Separator should be'): self.loop.run_until_complete(stream.readuntil(separator=b'')) with self.assertRaisesRegex(ValueError, 'Separator should be'): -self.loop.run_until_complete(stream.readuntil(separator=[b''])) +self.loop.run_until_complete(stream.readuntil(separator=(b'',))) with self.assertRaisesRegex(ValueError, 'Separator should contain'): -self.loop.run_until_complete(stream.readuntil(separator=[])) +self.loop.run_until_complete(stream.readuntil(separator=())) def test_readuntil_multi_chunks(self): stream = asyncio.StreamReader(loop=self.loop) @@ -475,15 +475,15 @@ def test_readuntil_multi_separator(self): # Simple case stream.feed_data(b'line 1\nline 2\r') -data = self.loop.run_until_complete(stream.readuntil([b'\r', b'\n'])) +data = self.loop.run_until_complete
[Python-checkins] gh-117233: Detect support for several hashes at hashlib build time (GH-117234)
https://github.com/python/cpython/commit/b8eaad30090b46f115dfed23266305b6546fb364
commit: b8eaad30090b46f115dfed23266305b6546fb364
branch: main
author: Will Childs-Klein
committer: encukou
date: 2024-04-11T16:49:41+02:00
summary:
gh-117233: Detect support for several hashes at hashlib build time (GH-117234)
Detect libcrypto BLAKE2, Shake, SHA3, and Truncated-SHA512 support at hashlib
build time
## BLAKE2
While OpenSSL supports both "b" and "s" variants of the BLAKE2 hash
function, other cryptographic libraries may lack support for one or both
of the variants. This commit modifies `hashlib`'s C code to detect
whether or not the linked libcrypto supports each BLAKE2 variant, and
elides references to each variant's NID accordingly. In cases where the
underlying libcrypto doesn't fully support BLAKE2, CPython's
`./configure` script can be given the following flag to use CPython's
interned BLAKE2 implementation: `--with-builtin-hashlib-hashes=blake2`.
## SHA3, Shake, & truncated SHA512.
Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the
OpenSSL-ish libcrypto library at build time. This helps allow hashlib's
`_hashopenssl` to be used with libraries that do not to support every
algorithm that upstream OpenSSL does. Such as AWS-LC & BoringSSL.
Co-authored-by: Gregory P. Smith [Google LLC]
files:
A Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst
M Modules/_hashopenssl.c
diff --git
a/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst
b/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst
new file mode 100644
index 00..a4142ec21b7e5d
--- /dev/null
+++ b/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst
@@ -0,0 +1,3 @@
+Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the OpenSSL-ish
+libcrypto library at build time. This allows :mod:`hashlib` to be used with
+libraries that do not to support every algorithm that upstream OpenSSL does.
diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c
index 0e230f332ff6cb..d0b46810dc1489 100644
--- a/Modules/_hashopenssl.c
+++ b/Modules/_hashopenssl.c
@@ -45,9 +45,15 @@
#define MUNCH_SIZE INT_MAX
#define PY_OPENSSL_HAS_SCRYPT 1
+#if defined(NID_sha3_224) && defined(NID_sha3_256) && defined(NID_sha3_384) &&
defined(NID_sha3_512)
#define PY_OPENSSL_HAS_SHA3 1
+#endif
+#if defined(NID_shake128) || defined(NID_shake256)
#define PY_OPENSSL_HAS_SHAKE 1
+#endif
+#if defined(NID_blake2s256) || defined(NID_blake2b512)
#define PY_OPENSSL_HAS_BLAKE2 1
+#endif
#if OPENSSL_VERSION_NUMBER >= 0x3000L
#define PY_EVP_MD EVP_MD
@@ -88,22 +94,45 @@ typedef struct {
PY_EVP_MD *evp_nosecurity;
} py_hashentry_t;
+// Fundamental to TLS, assumed always present in any libcrypto:
#define Py_hash_md5 "md5"
#define Py_hash_sha1 "sha1"
#define Py_hash_sha224 "sha224"
#define Py_hash_sha256 "sha256"
#define Py_hash_sha384 "sha384"
#define Py_hash_sha512 "sha512"
-#define Py_hash_sha512_224 "sha512_224"
-#define Py_hash_sha512_256 "sha512_256"
-#define Py_hash_sha3_224 "sha3_224"
-#define Py_hash_sha3_256 "sha3_256"
-#define Py_hash_sha3_384 "sha3_384"
-#define Py_hash_sha3_512 "sha3_512"
-#define Py_hash_shake_128 "shake_128"
-#define Py_hash_shake_256 "shake_256"
-#define Py_hash_blake2s "blake2s"
-#define Py_hash_blake2b "blake2b"
+
+// Not all OpenSSL-like libcrypto libraries provide these:
+#if defined(NID_sha512_224)
+# define Py_hash_sha512_224 "sha512_224"
+#endif
+#if defined(NID_sha512_256)
+# define Py_hash_sha512_256 "sha512_256"
+#endif
+#if defined(NID_sha3_224)
+# define Py_hash_sha3_224 "sha3_224"
+#endif
+#if defined(NID_sha3_256)
+# define Py_hash_sha3_256 "sha3_256"
+#endif
+#if defined(NID_sha3_384)
+# define Py_hash_sha3_384 "sha3_384"
+#endif
+#if defined(NID_sha3_512)
+# define Py_hash_sha3_512 "sha3_512"
+#endif
+#if defined(NID_shake128)
+# define Py_hash_shake_128 "shake_128"
+#endif
+#if defined(NID_shake256)
+# define Py_hash_shake_256 "shake_256"
+#endif
+#if defined(NID_blake2s256)
+# define Py_hash_blake2s "blake2s"
+#endif
+#if defined(NID_blake2b512)
+# define Py_hash_blake2b "blake2b"
+#endif
#define PY_HASH_ENTRY(py_name, py_alias, ossl_name, ossl_nid) \
{py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
@@ -119,19 +148,39 @@ static const py_hashentry_t py_hashes[] = {
PY_HASH_ENTRY(Py_hash_sha384, "SHA384", SN_sha384, NID_sha384),
PY_HASH_ENTRY(Py_hash_sha512, "SHA512", SN_sha512, NID_sha512),
/* truncated sha2 */
+#ifdef Py_hash_sha512_224
PY_HASH_ENTRY(Py_hash_sha512_224, "SHA512_224", SN_sha512_224,
NID_sha512_224),
+#endif
+#ifdef Py_hash_sha512_256
PY_HASH_ENTRY(Py_hash_sha512_256, "SHA512_256", SN_sha512_256,
NID_sha512_256),
+#endif
/* sha3 */
+#ifdef Py_hash_sha3_224
PY_HASH_ENTRY(Py_hash_sha3_224, NULL, SN_sha3_224, NID_sha3_224),
+#endif
+#ifdef Py_hash_sha3_256
PY_HASH_ENTRY(Py_hash_sha3_256, NULL, SN_sha3_256, NID_sha3_256),
+#endif
+#i
[Python-checkins] [3.12] gh-101549: fix documentation of xml.etree.ElementInclude (GH-101550) (#117754)
https://github.com/python/cpython/commit/a3f1e980c3c09d499cfa5c633c9187ccad410986 commit: a3f1e980c3c09d499cfa5c633c9187ccad410986 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: hugovk <[email protected]> date: 2024-04-11T14:53:27Z summary: [3.12] gh-101549: fix documentation of xml.etree.ElementInclude (GH-101550) (#117754) Co-authored-by: Mikhail B <[email protected]> Co-authored-by: Serhiy Storchaka Co-authored-by: Adam Turner <[email protected]> files: M Doc/library/xml.etree.elementtree.rst M Lib/xml/etree/ElementInclude.py diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 09aa81cc47facd..87c57db923c0d3 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -835,33 +835,28 @@ Functions .. module:: xml.etree.ElementInclude -.. function:: xml.etree.ElementInclude.default_loader( href, parse, encoding=None) - :module: +.. function:: default_loader(href, parse, encoding=None) - Default loader. This default loader reads an included resource from disk. *href* is a URL. - *parse* is for parse mode either "xml" or "text". *encoding* - is an optional text encoding. If not given, encoding is ``utf-8``. Returns the - expanded resource. If the parse mode is ``"xml"``, this is an ElementTree - instance. If the parse mode is "text", this is a Unicode string. If the - loader fails, it can return None or raise an exception. + Default loader. This default loader reads an included resource from disk. + *href* is a URL. *parse* is for parse mode either "xml" or "text". + *encoding* is an optional text encoding. If not given, encoding is ``utf-8``. + Returns the expanded resource. + If the parse mode is ``"xml"``, this is an :class:`~xml.etree.ElementTree.Element` instance. + If the parse mode is ``"text"``, this is a string. + If the loader fails, it can return ``None`` or raise an exception. -.. function:: xml.etree.ElementInclude.include( elem, loader=None, base_url=None, \ -max_depth=6) - :module: +.. function:: include(elem, loader=None, base_url=None, max_depth=6) - This function expands XInclude directives. *elem* is the root element. *loader* is - an optional resource loader. If omitted, it defaults to :func:`default_loader`. + This function expands XInclude directives in-place in tree pointed by *elem*. + *elem* is either the root :class:`~xml.etree.ElementTree.Element` or an + :class:`~xml.etree.ElementTree.ElementTree` instance to find such element. + *loader* is an optional resource loader. If omitted, it defaults to :func:`default_loader`. If given, it should be a callable that implements the same interface as :func:`default_loader`. *base_url* is base URL of the original file, to resolve relative include file references. *max_depth* is the maximum number of recursive - inclusions. Limited to reduce the risk of malicious content explosion. Pass a - negative value to disable the limitation. - - Returns the expanded resource. If the parse mode is - ``"xml"``, this is an ElementTree instance. If the parse mode is "text", - this is a Unicode string. If the loader fails, it can return None or - raise an exception. + inclusions. Limited to reduce the risk of malicious content explosion. + Pass ``None`` to disable the limitation. .. versionchanged:: 3.9 Added the *base_url* and *max_depth* parameters. diff --git a/Lib/xml/etree/ElementInclude.py b/Lib/xml/etree/ElementInclude.py index 40a9b22292479f..986e6c3bbe90f6 100644 --- a/Lib/xml/etree/ElementInclude.py +++ b/Lib/xml/etree/ElementInclude.py @@ -79,8 +79,8 @@ class LimitedRecursiveIncludeError(FatalIncludeError): # @param parse Parse mode. Either "xml" or "text". # @param encoding Optional text encoding (UTF-8 by default for "text"). # @return The expanded resource. If the parse mode is "xml", this -#is an ElementTree instance. If the parse mode is "text", this -#is a Unicode string. If the loader fails, it can return None +#is an Element instance. If the parse mode is "text", this +#is a string. If the loader fails, it can return None #or raise an OSError exception. # @throws OSError If the loader fails to load the resource. @@ -98,7 +98,7 @@ def default_loader(href, parse, encoding=None): ## # Expand XInclude directives. # -# @param elem Root element. +# @param elem Root Element or any ElementTree of a tree to be expanded # @param loader Optional resource loader. If omitted, it defaults # to {@link default_loader}. If given, it should be a callable # that implements the same interface as default_loader. @@ -106,12 +106,13 @@ def default_loader(href, parse, encoding=None): # relative include file references. # @p
[Python-checkins] gh-104269: Document `glob.glob` duplicates when using multiple `**` patterns (#105406)
https://github.com/python/cpython/commit/c06be6bbb8d138dde50c0a07cbd64496bee537c5 commit: c06be6bbb8d138dde50c0a07cbd64496bee537c5 branch: main author: Tomas R committer: barneygale date: 2024-04-11T15:16:39Z summary: gh-104269: Document `glob.glob` duplicates when using multiple `**` patterns (#105406) files: M Doc/library/glob.rst diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index 15fef747296ed4..ab6da98bc74ad2 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -75,6 +75,10 @@ The :mod:`glob` module defines the following functions: Using the "``**``" pattern in large directory trees may consume an inordinate amount of time. + .. note:: + This function may return duplicate path names if *pathname* + contains multiple "``**``" patterns and *recursive* is true. + .. versionchanged:: 3.5 Support for recursive globs using "``**``". @@ -94,6 +98,10 @@ The :mod:`glob` module defines the following functions: .. audit-event:: glob.glob pathname,recursive glob.iglob .. audit-event:: glob.glob/2 pathname,recursive,root_dir,dir_fd glob.iglob + .. note:: + This function may return duplicate path names if *pathname* + contains multiple "``**``" patterns and *recursive* is true. + .. versionchanged:: 3.5 Support for recursive globs using "``**``". ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-104269: Document `glob.glob` duplicates when using multiple `**` patterns (GH-105406) (#117757)
https://github.com/python/cpython/commit/84fb531fc2323cf3196366c3d0fbc0f43df99045 commit: 84fb531fc2323cf3196366c3d0fbc0f43df99045 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: barneygale date: 2024-04-11T18:44:46+01:00 summary: [3.12] gh-104269: Document `glob.glob` duplicates when using multiple `**` patterns (GH-105406) (#117757) gh-104269: Document `glob.glob` duplicates when using multiple `**` patterns (GH-105406) (cherry picked from commit c06be6bbb8d138dde50c0a07cbd64496bee537c5) Co-authored-by: Tomas R files: M Doc/library/glob.rst diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index 0e4cfe7ebed797..293c2c4fed62fd 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -77,6 +77,10 @@ For example, ``'[?]'`` matches the character ``'?'``. Using the "``**``" pattern in large directory trees may consume an inordinate amount of time. + .. note:: + This function may return duplicate path names if *pathname* + contains multiple "``**``" patterns and *recursive* is true. + .. versionchanged:: 3.5 Support for recursive globs using "``**``". @@ -96,6 +100,10 @@ For example, ``'[?]'`` matches the character ``'?'``. .. audit-event:: glob.glob pathname,recursive glob.iglob .. audit-event:: glob.glob/2 pathname,recursive,root_dir,dir_fd glob.iglob + .. note:: + This function may return duplicate path names if *pathname* + contains multiple "``**``" patterns and *recursive* is true. + .. versionchanged:: 3.5 Support for recursive globs using "``**``". ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-111506: Error if the limited API is used in free-threaded build (#117762)
https://github.com/python/cpython/commit/39d381f91e93559011587d764c1895ee30efb741 commit: 39d381f91e93559011587d764c1895ee30efb741 branch: main author: Sam Gross committer: colesbury date: 2024-04-11T13:49:52-04:00 summary: gh-111506: Error if the limited API is used in free-threaded build (#117762) Issue a build time error if both `Py_LIMITED_API` and `Py_GIL_DISABLED` are defined. files: M Include/Python.h diff --git a/Include/Python.h b/Include/Python.h index ca38a98d8c4eca..bb771fb3aec980 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -45,6 +45,11 @@ # endif #endif +// gh-111506: The free-threaded build is not compatible with the limited API +// or the stable ABI. +#if defined(Py_LIMITED_API) && defined(Py_GIL_DISABLED) +# error "The limited API is not currently supported in the free-threaded build" +#endif // Include Python header files #include "pyport.h" ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-117233: Detect support for several hashes at hashlib build time (GH-117234) (#117767)
https://github.com/python/cpython/commit/9376a9f0d85d0e7cd54f5aa010ff057700916c93 commit: 9376a9f0d85d0e7cd54f5aa010ff057700916c93 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: gpshead date: 2024-04-11T18:36:31Z summary: [3.12] gh-117233: Detect support for several hashes at hashlib build time (GH-117234) (#117767) gh-117233: Detect support for several hashes at hashlib build time (GH-117234) Detect libcrypto BLAKE2, Shake, SHA3, and Truncated-SHA512 support at hashlib build time GH-GH- BLAKE2 While OpenSSL supports both "b" and "s" variants of the BLAKE2 hash function, other cryptographic libraries may lack support for one or both of the variants. This commit modifies `hashlib`'s C code to detect whether or not the linked libcrypto supports each BLAKE2 variant, and elides references to each variant's NID accordingly. In cases where the underlying libcrypto doesn't fully support BLAKE2, CPython's `./configure` script can be given the following flag to use CPython's interned BLAKE2 implementation: `--with-builtin-hashlib-hashes=blake2`. GH-GH- SHA3, Shake, & truncated SHA512. Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the OpenSSL-ish libcrypto library at build time. This helps allow hashlib's `_hashopenssl` to be used with libraries that do not to support every algorithm that upstream OpenSSL does. Such as AWS-LC & BoringSSL. (cherry picked from commit b8eaad30090b46f115dfed23266305b6546fb364) Co-authored-by: Will Childs-Klein Co-authored-by: Gregory P. Smith [Google LLC] files: A Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst M Modules/_hashopenssl.c diff --git a/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst b/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst new file mode 100644 index 00..a4142ec21b7e5d --- /dev/null +++ b/Misc/NEWS.d/next/Security/2024-03-25-21-25-28.gh-issue-117233.E4CyI_.rst @@ -0,0 +1,3 @@ +Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the OpenSSL-ish +libcrypto library at build time. This allows :mod:`hashlib` to be used with +libraries that do not to support every algorithm that upstream OpenSSL does. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index af6d1b23d3ae91..2998820953bda9 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -45,9 +45,15 @@ #define MUNCH_SIZE INT_MAX #define PY_OPENSSL_HAS_SCRYPT 1 +#if defined(NID_sha3_224) && defined(NID_sha3_256) && defined(NID_sha3_384) && defined(NID_sha3_512) #define PY_OPENSSL_HAS_SHA3 1 +#endif +#if defined(NID_shake128) || defined(NID_shake256) #define PY_OPENSSL_HAS_SHAKE 1 +#endif +#if defined(NID_blake2s256) || defined(NID_blake2b512) #define PY_OPENSSL_HAS_BLAKE2 1 +#endif #if OPENSSL_VERSION_NUMBER >= 0x3000L #define PY_EVP_MD EVP_MD @@ -88,22 +94,45 @@ typedef struct { PY_EVP_MD *evp_nosecurity; } py_hashentry_t; +// Fundamental to TLS, assumed always present in any libcrypto: #define Py_hash_md5 "md5" #define Py_hash_sha1 "sha1" #define Py_hash_sha224 "sha224" #define Py_hash_sha256 "sha256" #define Py_hash_sha384 "sha384" #define Py_hash_sha512 "sha512" -#define Py_hash_sha512_224 "sha512_224" -#define Py_hash_sha512_256 "sha512_256" -#define Py_hash_sha3_224 "sha3_224" -#define Py_hash_sha3_256 "sha3_256" -#define Py_hash_sha3_384 "sha3_384" -#define Py_hash_sha3_512 "sha3_512" -#define Py_hash_shake_128 "shake_128" -#define Py_hash_shake_256 "shake_256" -#define Py_hash_blake2s "blake2s" -#define Py_hash_blake2b "blake2b" + +// Not all OpenSSL-like libcrypto libraries provide these: +#if defined(NID_sha512_224) +# define Py_hash_sha512_224 "sha512_224" +#endif +#if defined(NID_sha512_256) +# define Py_hash_sha512_256 "sha512_256" +#endif +#if defined(NID_sha3_224) +# define Py_hash_sha3_224 "sha3_224" +#endif +#if defined(NID_sha3_256) +# define Py_hash_sha3_256 "sha3_256" +#endif +#if defined(NID_sha3_384) +# define Py_hash_sha3_384 "sha3_384" +#endif +#if defined(NID_sha3_512) +# define Py_hash_sha3_512 "sha3_512" +#endif +#if defined(NID_shake128) +# define Py_hash_shake_128 "shake_128" +#endif +#if defined(NID_shake256) +# define Py_hash_shake_256 "shake_256" +#endif +#if defined(NID_blake2s256) +# define Py_hash_blake2s "blake2s" +#endif +#if defined(NID_blake2b512) +# define Py_hash_blake2b "blake2b" +#endif #define PY_HASH_ENTRY(py_name, py_alias, ossl_name, ossl_nid) \ {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL} @@ -119,19 +148,39 @@ static const py_hashentry_t py_hashes[] = { PY_HASH_ENTRY(Py_hash_sha384, "SHA384", SN_sha384, NID_sha384), PY_HASH_ENTRY(Py_hash_sha512, "SHA512", SN_sha512, NID_sha512), /* truncated sha2 */ +#ifdef Py_hash_sha512_224 PY_HASH_ENTRY(Py_hash_sha512_224, "SHA512_224", SN_sha512_224, NID_sha512_224), +#endif +#ifdef Py_hash_sha512_256 PY_HASH_ENTRY(Py_hash_sha512_256, "SHA512_256", SN_sha512
[Python-checkins] gh-117649: Raise ImportError for unsupported modules in free-threaded build (#117651)
https://github.com/python/cpython/commit/25f6ff5d3e92305659db62e7f7545f823f0dbd05
commit: 25f6ff5d3e92305659db62e7f7545f823f0dbd05
branch: main
author: Sam Gross
committer: colesbury
date: 2024-04-11T15:00:54-04:00
summary:
gh-117649: Raise ImportError for unsupported modules in free-threaded build
(#117651)
The free-threaded build does not currently support the combination of
single-phase init modules and non-isolated subinterpreters. Ensure that
`check_multi_interp_extensions` is always `True` for subinterpreters in
the free-threaded build so that importing these modules raises an
`ImportError`.
files:
M Include/cpython/pylifecycle.h
M Lib/test/support/__init__.py
M Lib/test/test_capi/test_misc.py
M Lib/test/test_import/__init__.py
M Lib/test/test_importlib/test_util.py
M Lib/test/test_interpreters/test_api.py
M Lib/test/test_threading.py
M Python/import.c
M Python/pylifecycle.c
diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h
index d425a233f71000..e46dfe59ec4630 100644
--- a/Include/cpython/pylifecycle.h
+++ b/Include/cpython/pylifecycle.h
@@ -63,6 +63,15 @@ typedef struct {
.gil = PyInterpreterConfig_OWN_GIL, \
}
+// gh-117649: The free-threaded build does not currently support single-phase
+// init extensions in subinterpreters. For now, we ensure that
+// `check_multi_interp_extensions` is always `1`, even in the legacy config.
+#ifdef Py_GIL_DISABLED
+# define _PyInterpreterConfig_LEGACY_CHECK_MULTI_INTERP_EXTENSIONS 1
+#else
+# define _PyInterpreterConfig_LEGACY_CHECK_MULTI_INTERP_EXTENSIONS 0
+#endif
+
#define _PyInterpreterConfig_LEGACY_INIT \
{ \
.use_main_obmalloc = 1, \
@@ -70,7 +79,7 @@ typedef struct {
.allow_exec = 1, \
.allow_threads = 1, \
.allow_daemon_threads = 1, \
-.check_multi_interp_extensions = 0, \
+.check_multi_interp_extensions =
_PyInterpreterConfig_LEGACY_CHECK_MULTI_INTERP_EXTENSIONS, \
.gil = PyInterpreterConfig_SHARED_GIL, \
}
diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
index 4bf2d7b5142da9..be3f93ab2e5fd1 100644
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -842,6 +842,12 @@ def requires_gil_enabled(msg="needs the GIL enabled"):
"""Decorator for skipping tests on the free-threaded build."""
return unittest.skipIf(Py_GIL_DISABLED, msg)
+def expected_failure_if_gil_disabled():
+"""Expect test failure if the GIL is disabled."""
+if Py_GIL_DISABLED:
+return unittest.expectedFailure
+return lambda test_case: test_case
+
if Py_GIL_DISABLED:
_header = 'PHBBInP'
else:
diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py
index 35d6a209122a99..8cdecaf3626401 100644
--- a/Lib/test/test_capi/test_misc.py
+++ b/Lib/test/test_capi/test_misc.py
@@ -26,6 +26,8 @@
from test.support import threading_helper
from test.support import warnings_helper
from test.support import requires_limited_api
+from test.support import requires_gil_enabled, expected_failure_if_gil_disabled
+from test.support import Py_GIL_DISABLED
from test.support.script_helper import assert_python_failure,
assert_python_ok, run_python_until_end
try:
import _posixsubprocess
@@ -2023,15 +2025,30 @@ def test_configured_settings(self):
kwlist[-2] = 'check_multi_interp_extensions'
kwlist[-1] = 'own_gil'
-# expected to work
-for config, expected in {
+expected_to_work = {
(True, True, True, True, True, True, True):
(ALL_FLAGS, True),
(True, False, False, False, False, False, False):
(OBMALLOC, False),
(False, False, False, True, False, True, False):
(THREADS | EXTENSIONS, False),
-}.items():
+}
+
+expected_to_fail = {
+(False, False, False, False, False, False, False),
+}
+
+# gh-117649: The free-threaded build does not currently allow
+# setting check_multi_interp_extensions to False.
+if Py_GIL_DISABLED:
+for config in list(expected_to_work.keys()):
+kwargs = dict(zip(kwlist, config))
+if not kwargs['check_multi_interp_extensions']:
+del expected_to_work[config]
+expected_to_fail.add(config)
+
+# expected to work
+for config, expected in expected_to_work.items():
kwargs = dict(zip(kwlist, config))
exp_flags, exp_gil = expected
expected = {
@@ -2055,9 +2072,7 @@ def test_configured_settings(self):
self.assertEqual(settings, expected)
# expected to fail
-for config in [
-(False, False, False, False, False, False, False),
-]:
+for config in expected_to_fail:
kwargs = dict(zip(kwlist, config))
with self.subTest(config):
script = textwrap.d
[Python-checkins] gh-117649: Fix file descriptor leak in (expected) failing test case (#117780)
https://github.com/python/cpython/commit/1b10efad66e9a0b72aae82de23074eafa49ec4db
commit: 1b10efad66e9a0b72aae82de23074eafa49ec4db
branch: main
author: Sam Gross
committer: colesbury
date: 2024-04-11T21:35:46Z
summary:
gh-117649: Fix file descriptor leak in (expected) failing test case (#117780)
The test case is currently expected to fail in the free-threaded build.
However, it fails before it gets a chance to close the write end of
the pipe.
files:
M Lib/test/test_capi/test_misc.py
diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py
index 8cdecaf3626401..9c24ec8fd05b12 100644
--- a/Lib/test/test_capi/test_misc.py
+++ b/Lib/test/test_capi/test_misc.py
@@ -2139,6 +2139,9 @@ def check(enabled, override):
}
r, w = os.pipe()
+if Py_GIL_DISABLED:
+# gh-117649: The test fails before `w` is closed
+self.addCleanup(os.close, w)
script = textwrap.dedent(f'''
from test.test_capi.check_config import run_singlephase_check
run_singlephase_check({override}, {w})
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-116738: Make _abc module thread-safe (#117488)
https://github.com/python/cpython/commit/f268e328ed5d7b2df5bdad39691f6e4789a2fcde
commit: f268e328ed5d7b2df5bdad39691f6e4789a2fcde
branch: main
author: Brett Simmers
committer: colesbury
date: 2024-04-11T18:13:25-04:00
summary:
gh-116738: Make _abc module thread-safe (#117488)
A collection of small changes aimed at making the `_abc` module safe to
use in a free-threaded build.
files:
M Include/internal/pycore_typeobject.h
M Modules/_abc.c
M Objects/typeobject.c
diff --git a/Include/internal/pycore_typeobject.h
b/Include/internal/pycore_typeobject.h
index 8a25935f308178..1693119ffece03 100644
--- a/Include/internal/pycore_typeobject.h
+++ b/Include/internal/pycore_typeobject.h
@@ -152,6 +152,18 @@ PyAPI_FUNC(PyObject*) _PySuper_Lookup(PyTypeObject
*su_type, PyObject *su_obj,
extern PyObject* _PyType_GetFullyQualifiedName(PyTypeObject *type, char sep);
+// Perform the following operation, in a thread-safe way when required by the
+// build mode.
+//
+// self->tp_flags = (self->tp_flags & ~mask) | flags;
+extern void _PyType_SetFlags(PyTypeObject *self, unsigned long mask,
+ unsigned long flags);
+
+// Like _PyType_SetFlags(), but apply the operation to self and any of its
+// subclasses without Py_TPFLAGS_IMMUTABLETYPE set.
+extern void _PyType_SetFlagsRecursive(PyTypeObject *self, unsigned long mask,
+ unsigned long flags);
+
#ifdef __cplusplus
}
diff --git a/Modules/_abc.c b/Modules/_abc.c
index 399ecbbd6a2172..ad28035843fd32 100644
--- a/Modules/_abc.c
+++ b/Modules/_abc.c
@@ -21,7 +21,7 @@ PyDoc_STRVAR(_abc__doc__,
typedef struct {
PyTypeObject *_abc_data_type;
-unsigned long long abc_invalidation_counter;
+uint64_t abc_invalidation_counter;
} _abcmodule_state;
static inline _abcmodule_state*
@@ -32,17 +32,61 @@ get_abc_state(PyObject *module)
return (_abcmodule_state *)state;
}
+static inline uint64_t
+get_invalidation_counter(_abcmodule_state *state)
+{
+#ifdef Py_GIL_DISABLED
+return _Py_atomic_load_uint64(&state->abc_invalidation_counter);
+#else
+return state->abc_invalidation_counter;
+#endif
+}
+
+static inline void
+increment_invalidation_counter(_abcmodule_state *state)
+{
+#ifdef Py_GIL_DISABLED
+_Py_atomic_add_uint64(&state->abc_invalidation_counter, 1);
+#else
+state->abc_invalidation_counter++;
+#endif
+}
+
/* This object stores internal state for ABCs.
Note that we can use normal sets for caches,
since they are never iterated over. */
typedef struct {
PyObject_HEAD
+/* These sets of weak references are lazily created. Once created, they
+ will point to the same sets until the ABCMeta object is destroyed or
+ cleared, both of which will only happen while the object is visible to a
+ single thread. */
PyObject *_abc_registry;
-PyObject *_abc_cache; /* Normal set of weak references. */
-PyObject *_abc_negative_cache; /* Normal set of weak references. */
-unsigned long long _abc_negative_cache_version;
+PyObject *_abc_cache;
+PyObject *_abc_negative_cache;
+uint64_t _abc_negative_cache_version;
} _abc_data;
+static inline uint64_t
+get_cache_version(_abc_data *impl)
+{
+#ifdef Py_GIL_DISABLED
+return _Py_atomic_load_uint64(&impl->_abc_negative_cache_version);
+#else
+return impl->_abc_negative_cache_version;
+#endif
+}
+
+static inline void
+set_cache_version(_abc_data *impl, uint64_t version)
+{
+#ifdef Py_GIL_DISABLED
+_Py_atomic_store_uint64(&impl->_abc_negative_cache_version, version);
+#else
+impl->_abc_negative_cache_version = version;
+#endif
+}
+
static int
abc_data_traverse(_abc_data *self, visitproc visit, void *arg)
{
@@ -90,7 +134,7 @@ abc_data_new(PyTypeObject *type, PyObject *args, PyObject
*kwds)
self->_abc_registry = NULL;
self->_abc_cache = NULL;
self->_abc_negative_cache = NULL;
-self->_abc_negative_cache_version = state->abc_invalidation_counter;
+self->_abc_negative_cache_version = get_invalidation_counter(state);
return (PyObject *) self;
}
@@ -130,8 +174,12 @@ _get_impl(PyObject *module, PyObject *self)
}
static int
-_in_weak_set(PyObject *set, PyObject *obj)
+_in_weak_set(_abc_data *impl, PyObject **pset, PyObject *obj)
{
+PyObject *set;
+Py_BEGIN_CRITICAL_SECTION(impl);
+set = *pset;
+Py_END_CRITICAL_SECTION();
if (set == NULL || PySet_GET_SIZE(set) == 0) {
return 0;
}
@@ -168,16 +216,19 @@ static PyMethodDef _destroy_def = {
};
static int
-_add_to_weak_set(PyObject **pset, PyObject *obj)
+_add_to_weak_set(_abc_data *impl, PyObject **pset, PyObject *obj)
{
-if (*pset == NULL) {
-*pset = PySet_New(NULL);
-if (*pset == NULL) {
-return -1;
-}
+PyObject *set;
+Py_BEGIN_CRITICAL_SECTION(impl);
+set = *pset;
+if (set == NULL) {
+set = *pset = PySet_New(NULL);
+}
+Py_END_CRITICAL_SECTION();
+if (set == NULL) {
+
[Python-checkins] gh-117787: Autoconf: fix bashisms/semantic breakage of iOS checks (#117788)
https://github.com/python/cpython/commit/fd2bab9d287ef0879568662d4fedeae0a0c61d43 commit: fd2bab9d287ef0879568662d4fedeae0a0c61d43 branch: main author: Eli Schwartz committer: erlend-aasland date: 2024-04-12T01:03:54+02:00 summary: gh-117787: Autoconf: fix bashisms/semantic breakage of iOS checks (#117788) files: M configure M configure.ac diff --git a/configure b/configure index f9647566636e4c..f9c181ac3805e1 100755 --- a/configure +++ b/configure @@ -24334,7 +24334,7 @@ if test "$PY_ENABLE_SHARED" = "1" && ( test -n "$ANDROID_API_LEVEL" || test "$MA fi # On iOS the shared libraries must be linked with the Python framework -if test "$ac_sys_system" == "iOS"; then +if test "$ac_sys_system" = "iOS"; then MODULE_DEPS_SHARED="$MODULE_DEPS_SHARED \$(PYTHONFRAMEWORKDIR)/\$(PYTHONFRAMEWORK)" fi diff --git a/configure.ac b/configure.ac index e195e15b39ed21..47c084d92a30e8 100644 --- a/configure.ac +++ b/configure.ac @@ -6053,7 +6053,7 @@ if test "$PY_ENABLE_SHARED" = "1" && ( test -n "$ANDROID_API_LEVEL" || test "$MA fi # On iOS the shared libraries must be linked with the Python framework -if test "$ac_sys_system" == "iOS"; then +if test "$ac_sys_system" = "iOS"; then MODULE_DEPS_SHARED="$MODULE_DEPS_SHARED \$(PYTHONFRAMEWORKDIR)/\$(PYTHONFRAMEWORK)" fi ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-76785: Handle Legacy Interpreters Properly (gh-117490)
https://github.com/python/cpython/commit/fd259fdabe95329768eb3e90039c77888c130a4c
commit: fd259fdabe95329768eb3e90039c77888c130a4c
branch: main
author: Eric Snow
committer: ericsnowcurrently
date: 2024-04-11T23:23:25Z
summary:
gh-76785: Handle Legacy Interpreters Properly (gh-117490)
This is similar to the situation with threading._DummyThread. The methods
(incl. __del__()) of interpreters.Interpreter objects must be careful with
interpreters not created by interpreters.create(). The simplest thing to start
with is to disable any method that modifies or runs in the interpreter. As
part of this, the runtime keeps track of where an interpreter was created. We
also handle interpreter "refcounts" properly.
files:
M Include/internal/pycore_crossinterp.h
M Include/internal/pycore_interp.h
M Lib/test/support/interpreters/__init__.py
M Lib/test/test_interpreters/test_api.py
M Lib/test/test_interpreters/utils.py
M Modules/_testinternalcapi.c
M Modules/_xxsubinterpretersmodule.c
M Python/crossinterp.c
M Python/pystate.c
diff --git a/Include/internal/pycore_crossinterp.h
b/Include/internal/pycore_crossinterp.h
index 64d881dbab7357..2dd165eae74850 100644
--- a/Include/internal/pycore_crossinterp.h
+++ b/Include/internal/pycore_crossinterp.h
@@ -325,6 +325,7 @@ PyAPI_FUNC(int) _PyXI_HasCapturedException(_PyXI_session
*session);
// Export for _testinternalcapi shared extension
PyAPI_FUNC(PyInterpreterState *) _PyXI_NewInterpreter(
PyInterpreterConfig *config,
+long *maybe_whence,
PyThreadState **p_tstate,
PyThreadState **p_save_tstate);
PyAPI_FUNC(void) _PyXI_EndInterpreter(
diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h
index c96a9e40e4274a..d38959e3ce4ec5 100644
--- a/Include/internal/pycore_interp.h
+++ b/Include/internal/pycore_interp.h
@@ -109,7 +109,8 @@ struct _is {
#define _PyInterpreterState_WHENCE_LEGACY_CAPI 2
#define _PyInterpreterState_WHENCE_CAPI 3
#define _PyInterpreterState_WHENCE_XI 4
-#define _PyInterpreterState_WHENCE_MAX 4
+#define _PyInterpreterState_WHENCE_STDLIB 5
+#define _PyInterpreterState_WHENCE_MAX 5
long _whence;
/* Has been initialized to a safe state.
@@ -316,6 +317,8 @@ PyAPI_FUNC(int)
_PyInterpreterState_IDInitref(PyInterpreterState *);
PyAPI_FUNC(int) _PyInterpreterState_IDIncref(PyInterpreterState *);
PyAPI_FUNC(void) _PyInterpreterState_IDDecref(PyInterpreterState *);
+PyAPI_FUNC(int) _PyInterpreterState_IsReady(PyInterpreterState *interp);
+
PyAPI_FUNC(long) _PyInterpreterState_GetWhence(PyInterpreterState *interp);
extern void _PyInterpreterState_SetWhence(
PyInterpreterState *interp,
diff --git a/Lib/test/support/interpreters/__init__.py
b/Lib/test/support/interpreters/__init__.py
index 60323c9874f9a0..0a5a9259479be4 100644
--- a/Lib/test/support/interpreters/__init__.py
+++ b/Lib/test/support/interpreters/__init__.py
@@ -74,51 +74,77 @@ def __str__(self):
def create():
"""Return a new (idle) Python interpreter."""
id = _interpreters.create(reqrefs=True)
-return Interpreter(id)
+return Interpreter(id, _ownsref=True)
def list_all():
"""Return all existing interpreters."""
-return [Interpreter(id)
-for id, in _interpreters.list_all()]
+return [Interpreter(id, _whence=whence)
+for id, whence in _interpreters.list_all(require_ready=True)]
def get_current():
"""Return the currently running interpreter."""
-id, = _interpreters.get_current()
-return Interpreter(id)
+id, whence = _interpreters.get_current()
+return Interpreter(id, _whence=whence)
def get_main():
"""Return the main interpreter."""
-id, = _interpreters.get_main()
-return Interpreter(id)
+id, whence = _interpreters.get_main()
+assert whence == _interpreters.WHENCE_RUNTIME, repr(whence)
+return Interpreter(id, _whence=whence)
_known = weakref.WeakValueDictionary()
class Interpreter:
-"""A single Python interpreter."""
+"""A single Python interpreter.
-def __new__(cls, id, /):
+Attributes:
+
+"id" - the unique process-global ID number for the interpreter
+"whence" - indicates where the interpreter was created
+
+If the interpreter wasn't created by this module
+then any method that modifies the interpreter will fail,
+i.e. .close(), .prepare_main(), .exec(), and .call()
+"""
+
+_WHENCE_TO_STR = {
+ _interpreters.WHENCE_UNKNOWN: 'unknown',
+ _interpreters.WHENCE_RUNTIME: 'runtime init',
+ _interpreters.WHENCE_LEGACY_CAPI: 'legacy C-API',
+ _interpreters.WHENCE_CAPI: 'C-API',
+ _interpreters.WHENCE_XI: 'cross-interpreter C-API',
+ _interpreters.WHENCE_STDLIB: '_interpreters module',
+}
+
+def __new__(cls, id, /, _whence=None, _ownsref=None):
# There is only one instance for any given ID.
if not isinstance(id, int):
raise TypeError(f'id must be an int, got {id!r}')
id = int(id)
+
[Python-checkins] gh-117752: Autoconf: store all LLVM profile data in the build directory (#117790)
https://github.com/python/cpython/commit/396b831850f0f364d584db4407a5d633f33e571c
commit: 396b831850f0f364d584db4407a5d633f33e571c
branch: main
author: Erlend E. Aasland
committer: erlend-aasland
date: 2024-04-12T07:23:39+02:00
summary:
gh-117752: Autoconf: store all LLVM profile data in the build directory
(#117790)
This prevents spurious 'env changed' and llvm-profdata merge errors.
files:
M configure
M configure.ac
diff --git a/configure b/configure
index f9c181ac3805e1..15a645ae84618a 100755
--- a/configure
+++ b/configure
@@ -8828,7 +8828,7 @@ case "$CC_BASENAME" in
PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd"
LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd
*.profclangr"
-LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\""
+LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\""
if test $LLVM_PROF_FOUND = not-found
then
LLVM_PROF_ERR=yes
@@ -8844,7 +8844,7 @@ case "$CC_BASENAME" in
PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd"
LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd
*.profclangr"
-LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\""
+LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\""
if test "${LLVM_PROF_FOUND}" = "not-found"
then
LLVM_PROF_ERR=yes
diff --git a/configure.ac b/configure.ac
index 47c084d92a30e8..359ac2bd8393a9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2013,7 +2013,7 @@ case "$CC_BASENAME" in
PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd"
LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd
*.profclangr"
-LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\""
+LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\""
if test $LLVM_PROF_FOUND = not-found
then
LLVM_PROF_ERR=yes
@@ -2029,7 +2029,7 @@ case "$CC_BASENAME" in
PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd"
LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd
*.profclangr"
-LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\""
+LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\""
if test "${LLVM_PROF_FOUND}" = "not-found"
then
LLVM_PROF_ERR=yes
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-117752: Autoconf: store all LLVM profile data in the build directory (GH-117790) (#117795)
https://github.com/python/cpython/commit/c64b65e3aa24478ab0ced14ff0c01ff962d7623c commit: c64b65e3aa24478ab0ced14ff0c01ff962d7623c branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: erlend-aasland date: 2024-04-12T05:39:14Z summary: [3.12] gh-117752: Autoconf: store all LLVM profile data in the build directory (GH-117790) (#117795) This prevents spurious 'env changed' and llvm-profdata merge errors. (cherry picked from commit 396b831850f0f364d584db4407a5d633f33e571c) Co-authored-by: Erlend E. Aasland files: M configure M configure.ac diff --git a/configure b/configure index be783faa248c14..a755c078a25755 100755 --- a/configure +++ b/configure @@ -8786,7 +8786,7 @@ case $CC in PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" -LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" +LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes @@ -8802,7 +8802,7 @@ case $CC in PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" -LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" +LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\"" if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes diff --git a/configure.ac b/configure.ac index 8be26cc0ab77a0..4aad4ae2ebfff6 100644 --- a/configure.ac +++ b/configure.ac @@ -1989,7 +1989,7 @@ case $CC in PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" -LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" +LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes @@ -2005,7 +2005,7 @@ case $CC in PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" -LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" +LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"\$(shell pwd)/code-%p.profclangr\"" if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
