[Python-checkins] [3.13] gh-123110: correct note about _Bool in the struct module docs (GH-123111) (#123126)
https://github.com/python/cpython/commit/cf444d851555ec77ad0a8e9266a09958398da689 commit: cf444d851555ec77ad0a8e9266a09958398da689 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-19T13:00:54+05:30 summary: [3.13] gh-123110: correct note about _Bool in the struct module docs (GH-123111) (#123126) gh-123110: correct note about _Bool in the struct module docs (GH-123111) (cherry picked from commit b0f462d4c808d6fb1d381bba4932acd8309c1f3b) Co-authored-by: Sergey B Kirpichev files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index a2c293443e23d3..4769affdf1d666 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -279,9 +279,9 @@ Notes: (1) .. index:: single: ? (question mark); in struct format strings - The ``'?'`` conversion code corresponds to the :c:expr:`_Bool` type defined by - C99. If this type is not available, it is simulated using a :c:expr:`char`. In - standard mode, it is always represented by one byte. + The ``'?'`` conversion code corresponds to the :c:expr:`_Bool` type + defined by C standards since C99. In standard mode, it is + represented by one byte. (2) When attempting to pack a non-integer using any of the integer conversion ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-123110: correct note about _Bool in the struct module docs (GH-123111) (#123127)
https://github.com/python/cpython/commit/8edfa0b0b4ae4235bb3262d952c23e7581516d4f commit: 8edfa0b0b4ae4235bb3262d952c23e7581516d4f branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-19T13:01:12+05:30 summary: [3.12] gh-123110: correct note about _Bool in the struct module docs (GH-123111) (#123127) gh-123110: correct note about _Bool in the struct module docs (GH-123111) (cherry picked from commit b0f462d4c808d6fb1d381bba4932acd8309c1f3b) Co-authored-by: Sergey B Kirpichev files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 346784d86e3831..29bce521ee1a09 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -275,9 +275,9 @@ Notes: (1) .. index:: single: ? (question mark); in struct format strings - The ``'?'`` conversion code corresponds to the :c:expr:`_Bool` type defined by - C99. If this type is not available, it is simulated using a :c:expr:`char`. In - standard mode, it is always represented by one byte. + The ``'?'`` conversion code corresponds to the :c:expr:`_Bool` type + defined by C standards since C99. In standard mode, it is + represented by one byte. (2) When attempting to pack a non-integer using any of the integer conversion ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-122081: fixed crash in decimal.IEEEContext() (#122082)
https://github.com/python/cpython/commit/b9e10d1a0fc4d8428d4b36eb127570a832c26b6f commit: b9e10d1a0fc4d8428d4b36eb127570a832c26b6f branch: main author: Sergey B Kirpichev committer: kumaraditya303 date: 2024-08-19T07:51:38Z summary: gh-122081: fixed crash in decimal.IEEEContext() (#122082) * gh-122081: fixed crash in decimal.IEEEContext() Now $ ./configure CFLAGS=-DEXTRA_FUNCTIONALITY -q && make -s && \ ./python -m test test_decimal - PASS * Apply suggestions from code review Co-authored-by: Bénédikt Tran <[email protected]> * Update Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst * Apply suggestions from code review - Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst M Modules/_decimal/_decimal.c diff --git a/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst b/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst new file mode 100644 index 00..4c988b16168047 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst @@ -0,0 +1,2 @@ +Fix a crash in the :func:`!decimal.IEEEContext` optional function +available via the ``EXTRA_FUNCTIONALITY`` configuration flag. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 94a2cc2c8e5f8a..15855e2cf1bb32 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1519,7 +1519,7 @@ init_extended_context(PyObject *v) #ifdef EXTRA_FUNCTIONALITY /* Factory function for creating IEEE interchange format contexts */ static PyObject * -ieee_context(PyObject *dummy UNUSED, PyObject *v) +ieee_context(PyObject *module, PyObject *v) { PyObject *context; mpd_ssize_t bits; @@ -1536,7 +1536,7 @@ ieee_context(PyObject *dummy UNUSED, PyObject *v) goto error; } -decimal_state *state = get_module_state_by_def(Py_TYPE(v)); +decimal_state *state = get_module_state(module); context = PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL); if (context == NULL) { return NULL; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-122081: fixed crash in decimal.IEEEContext() (GH-122082) (#123136)
https://github.com/python/cpython/commit/032b6467fab8dc9b9bd97a418c15fe388f4a18b9 commit: 032b6467fab8dc9b9bd97a418c15fe388f4a18b9 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-19T14:14:11+05:30 summary: [3.13] gh-122081: fixed crash in decimal.IEEEContext() (GH-122082) (#123136) gh-122081: fixed crash in decimal.IEEEContext() (GH-122082) * gh-122081: fixed crash in decimal.IEEEContext() Now $ ./configure CFLAGS=-DEXTRA_FUNCTIONALITY -q && make -s && \ ./python -m test test_decimal - PASS * Apply suggestions from code review * Update Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst * Apply suggestions from code review - (cherry picked from commit b9e10d1a0fc4d8428d4b36eb127570a832c26b6f) Co-authored-by: Sergey B Kirpichev Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst M Modules/_decimal/_decimal.c diff --git a/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst b/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst new file mode 100644 index 00..4c988b16168047 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst @@ -0,0 +1,2 @@ +Fix a crash in the :func:`!decimal.IEEEContext` optional function +available via the ``EXTRA_FUNCTIONALITY`` configuration flag. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 2daa24c823a542..81e548f3b48245 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1519,7 +1519,7 @@ init_extended_context(PyObject *v) #ifdef EXTRA_FUNCTIONALITY /* Factory function for creating IEEE interchange format contexts */ static PyObject * -ieee_context(PyObject *dummy UNUSED, PyObject *v) +ieee_context(PyObject *module, PyObject *v) { PyObject *context; mpd_ssize_t bits; @@ -1536,7 +1536,7 @@ ieee_context(PyObject *dummy UNUSED, PyObject *v) goto error; } -decimal_state *state = get_module_state_by_def(Py_TYPE(v)); +decimal_state *state = get_module_state(module); context = PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL); if (context == NULL) { return NULL; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] Fix typo mentioning threads instead of tasks (#123203)
https://github.com/python/cpython/commit/58fdb169c8a93925541fecc74ba73c566147f2ca commit: 58fdb169c8a93925541fecc74ba73c566147f2ca branch: main author: Kevin Evans committer: kumaraditya303 date: 2024-08-23T21:54:12+05:30 summary: Fix typo mentioning threads instead of tasks (#123203) files: M Lib/asyncio/locks.py diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py index aaee8ff0702923..f2f8b7ec858096 100644 --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -341,9 +341,9 @@ def _notify(self, n): fut.set_result(False) def notify_all(self): -"""Wake up all threads waiting on this condition. This method acts -like notify(), but wakes up all waiting threads instead of one. If the -calling thread has not acquired the lock when this method is called, +"""Wake up all tasks waiting on this condition. This method acts +like notify(), but wakes up all waiting tasks instead of one. If the +calling task has not acquired the lock when this method is called, a RuntimeError is raised. """ self.notify(len(self._waiters)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-123243: Fix reference leak in `_decimal` (#123244)
https://github.com/python/cpython/commit/5ff638f1b53587b9f912a18fc776a2a141fd7bed commit: 5ff638f1b53587b9f912a18fc776a2a141fd7bed branch: main author: neonene <[email protected]> committer: kumaraditya303 date: 2024-08-24T09:37:01+05:30 summary: gh-123243: Fix reference leak in `_decimal` (#123244) files: A Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst M Modules/_decimal/_decimal.c diff --git a/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst b/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst new file mode 100644 index 00..cf52585020111f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst @@ -0,0 +1 @@ +Fix memory leak in :mod:`!_decimal`. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 15855e2cf1bb32..0e743a609d33d6 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1390,6 +1390,10 @@ context_new(PyTypeObject *type, PyObject *args UNUSED, PyObject *kwds UNUSED) CtxCaps(self) = 1; self->tstate = NULL; +if (type == state->PyDecContext_Type) { +PyObject_GC_Track(self); +} +assert(PyObject_GC_IsTracked((PyObject *)self)); return (PyObject *)self; } @@ -2038,6 +2042,10 @@ PyDecType_New(PyTypeObject *type) MPD(dec)->alloc = _Py_DEC_MINALLOC; MPD(dec)->data = dec->data; +if (type == state->PyDec_Type) { +PyObject_GC_Track(dec); +} +assert(PyObject_GC_IsTracked((PyObject *)dec)); return (PyObject *)dec; } #define dec_alloc(st) PyDecType_New((st)->PyDec_Type) @@ -6143,8 +6151,22 @@ decimal_clear(PyObject *module) Py_CLEAR(state->SignalTuple); Py_CLEAR(state->PyDecimal); -PyMem_Free(state->signal_map); -PyMem_Free(state->cond_map); +if (state->signal_map != NULL) { +for (DecCondMap *cm = state->signal_map; cm->name != NULL; cm++) { +Py_DECREF(cm->ex); +} +PyMem_Free(state->signal_map); +state->signal_map = NULL; +} + +if (state->cond_map != NULL) { +// cond_map[0].ex has borrowed a reference from signal_map[0].ex +for (DecCondMap *cm = state->cond_map + 1; cm->name != NULL; cm++) { +Py_DECREF(cm->ex); +} +PyMem_Free(state->cond_map); +state->cond_map = NULL; +} return 0; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-123243: Fix reference leak in `_decimal` (GH-123244) (#123280)
https://github.com/python/cpython/commit/a65fe07db4aa57b4e67f2f9e914db7ea281f127f commit: a65fe07db4aa57b4e67f2f9e914db7ea281f127f branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-24T04:33:49Z summary: [3.13] gh-123243: Fix reference leak in `_decimal` (GH-123244) (#123280) gh-123243: Fix reference leak in `_decimal` (GH-123244) (cherry picked from commit 5ff638f1b53587b9f912a18fc776a2a141fd7bed) Co-authored-by: neonene <[email protected]> files: A Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst M Modules/_decimal/_decimal.c diff --git a/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst b/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst new file mode 100644 index 00..cf52585020111f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-08-22-20-10-13.gh-issue-123243.Kifj1L.rst @@ -0,0 +1 @@ +Fix memory leak in :mod:`!_decimal`. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 81e548f3b48245..05664bf9ed38ae 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1390,6 +1390,10 @@ context_new(PyTypeObject *type, PyObject *args UNUSED, PyObject *kwds UNUSED) CtxCaps(self) = 1; self->tstate = NULL; +if (type == state->PyDecContext_Type) { +PyObject_GC_Track(self); +} +assert(PyObject_GC_IsTracked((PyObject *)self)); return (PyObject *)self; } @@ -2038,6 +2042,10 @@ PyDecType_New(PyTypeObject *type) MPD(dec)->alloc = _Py_DEC_MINALLOC; MPD(dec)->data = dec->data; +if (type == state->PyDec_Type) { +PyObject_GC_Track(dec); +} +assert(PyObject_GC_IsTracked((PyObject *)dec)); return (PyObject *)dec; } #define dec_alloc(st) PyDecType_New((st)->PyDec_Type) @@ -6143,8 +6151,22 @@ decimal_clear(PyObject *module) Py_CLEAR(state->SignalTuple); Py_CLEAR(state->PyDecimal); -PyMem_Free(state->signal_map); -PyMem_Free(state->cond_map); +if (state->signal_map != NULL) { +for (DecCondMap *cm = state->signal_map; cm->name != NULL; cm++) { +Py_DECREF(cm->ex); +} +PyMem_Free(state->signal_map); +state->signal_map = NULL; +} + +if (state->cond_map != NULL) { +// cond_map[0].ex has borrowed a reference from signal_map[0].ex +for (DecCondMap *cm = state->cond_map + 1; cm->name != NULL; cm++) { +Py_DECREF(cm->ex); +} +PyMem_Free(state->cond_map); +state->cond_map = NULL; +} return 0; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-123089: Make weakref.WeakSet safe against concurrent mutations while it is being iterated (#123279)
https://github.com/python/cpython/commit/03f5abf15a20f6e623282a393bc2a0affac69bb0 commit: 03f5abf15a20f6e623282a393bc2a0affac69bb0 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2024-08-27T13:04:03Z summary: gh-123089: Make weakref.WeakSet safe against concurrent mutations while it is being iterated (#123279) * Make `weakref.WeakSet` safe against concurrent mutations while it is being iterated. `_IterationGuard` is no longer used for `WeakSet`, it now relies on copying the underlying set which is an atomic operation while iterating so that it can be modified by other threads. files: A Misc/NEWS.d/next/Library/2024-08-27-12-38-42.gh-issue-123089.vA7iFR.rst M Lib/_weakrefset.py diff --git a/Lib/_weakrefset.py b/Lib/_weakrefset.py index 489eec714e0d48..2071755d71dfc8 100644 --- a/Lib/_weakrefset.py +++ b/Lib/_weakrefset.py @@ -36,41 +36,26 @@ def __exit__(self, e, t, b): class WeakSet: def __init__(self, data=None): self.data = set() + def _remove(item, selfref=ref(self)): self = selfref() if self is not None: -if self._iterating: -self._pending_removals.append(item) -else: -self.data.discard(item) +self.data.discard(item) + self._remove = _remove -# A list of keys to be removed -self._pending_removals = [] -self._iterating = set() if data is not None: self.update(data) -def _commit_removals(self): -pop = self._pending_removals.pop -discard = self.data.discard -while True: -try: -item = pop() -except IndexError: -return -discard(item) - def __iter__(self): -with _IterationGuard(self): -for itemref in self.data: -item = itemref() -if item is not None: -# Caveat: the iterator will keep a strong reference to -# `item` until it is resumed or closed. -yield item +for itemref in self.data.copy(): +item = itemref() +if item is not None: +# Caveat: the iterator will keep a strong reference to +# `item` until it is resumed or closed. +yield item def __len__(self): -return len(self.data) - len(self._pending_removals) +return len(self.data) def __contains__(self, item): try: @@ -83,21 +68,15 @@ def __reduce__(self): return self.__class__, (list(self),), self.__getstate__() def add(self, item): -if self._pending_removals: -self._commit_removals() self.data.add(ref(item, self._remove)) def clear(self): -if self._pending_removals: -self._commit_removals() self.data.clear() def copy(self): return self.__class__(self) def pop(self): -if self._pending_removals: -self._commit_removals() while True: try: itemref = self.data.pop() @@ -108,18 +87,12 @@ def pop(self): return item def remove(self, item): -if self._pending_removals: -self._commit_removals() self.data.remove(ref(item)) def discard(self, item): -if self._pending_removals: -self._commit_removals() self.data.discard(ref(item)) def update(self, other): -if self._pending_removals: -self._commit_removals() for element in other: self.add(element) @@ -136,8 +109,6 @@ def difference(self, other): def difference_update(self, other): self.__isub__(other) def __isub__(self, other): -if self._pending_removals: -self._commit_removals() if self is other: self.data.clear() else: @@ -151,8 +122,6 @@ def intersection(self, other): def intersection_update(self, other): self.__iand__(other) def __iand__(self, other): -if self._pending_removals: -self._commit_removals() self.data.intersection_update(ref(item) for item in other) return self @@ -184,8 +153,6 @@ def symmetric_difference(self, other): def symmetric_difference_update(self, other): self.__ixor__(other) def __ixor__(self, other): -if self._pending_removals: -self._commit_removals() if self is other: self.data.clear() else: diff --git a/Misc/NEWS.d/next/Library/2024-08-27-12-38-42.gh-issue-123089.vA7iFR.rst b/Misc/NEWS.d/next/Library/2024-08-27-12-38-42.gh-issue-123089.vA7iFR.rst new file mode 100644 index 00..74cbdd551350f7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-08-27-12-38-42.gh-issue-123089.vA7iFR.rst @@ -0,0 +1 @@ +Make :class:`weakref.WeakSet` safe against
[Python-checkins] gH-80788: remove old weakset workaround for thread safety (#123388)
https://github.com/python/cpython/commit/460ee5b994335994d4b5186c08f44e775b3e55fa
commit: 460ee5b994335994d4b5186c08f44e775b3e55fa
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-08-27T19:12:44+05:30
summary:
gH-80788: remove old weakset workaround for thread safety (#123388)
files:
M Lib/asyncio/tasks.py
diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py
index 2f8f4f08a1e111..2112dd4b99d17f 100644
--- a/Lib/asyncio/tasks.py
+++ b/Lib/asyncio/tasks.py
@@ -48,23 +48,8 @@ def all_tasks(loop=None):
# capturing the set of eager tasks first, so if an eager task "graduates"
# to a regular task in another thread, we don't risk missing it.
eager_tasks = list(_eager_tasks)
-# Looping over the WeakSet isn't safe as it can be updated from another
-# thread, therefore we cast it to list prior to filtering. The list cast
-# itself requires iteration, so we repeat it several times ignoring
-# RuntimeErrors (which are not very likely to occur).
-# See issues 34970 and 36607 for details.
-scheduled_tasks = None
-i = 0
-while True:
-try:
-scheduled_tasks = list(_scheduled_tasks)
-except RuntimeError:
-i += 1
-if i >= 1000:
-raise
-else:
-break
-return {t for t in itertools.chain(scheduled_tasks, eager_tasks)
+
+return {t for t in itertools.chain(_scheduled_tasks, eager_tasks)
if futures._get_loop(t) is loop and not t.done()}
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-123919: Fix null handling in `_freeze_module.c` (#123920)
https://github.com/python/cpython/commit/c8d1dbef5b770b647aa7ff45fd5b269bc7629d0b
commit: c8d1dbef5b770b647aa7ff45fd5b269bc7629d0b
branch: main
author: sobolevn
committer: kumaraditya303
date: 2024-09-11T19:07:35+05:30
summary:
gh-123919: Fix null handling in `_freeze_module.c` (#123920)
files:
M Programs/_freeze_module.c
diff --git a/Programs/_freeze_module.c b/Programs/_freeze_module.c
index 2a462a42cdad7c..891e4256e897ab 100644
--- a/Programs/_freeze_module.c
+++ b/Programs/_freeze_module.c
@@ -110,6 +110,9 @@ static PyObject *
compile_and_marshal(const char *name, const char *text)
{
char *filename = (char *) malloc(strlen(name) + 10);
+if (filename == NULL) {
+return PyErr_NoMemory();
+}
sprintf(filename, "", name);
PyObject *code = Py_CompileStringExFlags(text, filename,
Py_file_input, NULL, 0);
@@ -133,6 +136,9 @@ get_varname(const char *name, const char *prefix)
{
size_t n = strlen(prefix);
char *varname = (char *) malloc(strlen(name) + n + 1);
+if (varname == NULL) {
+return NULL;
+}
(void)strcpy(varname, prefix);
for (size_t i = 0; name[i] != '\0'; i++) {
if (name[i] == '.') {
@@ -178,6 +184,11 @@ write_frozen(const char *outpath, const char *inpath,
const char *name,
fprintf(outfile, "%s\n", header);
char *arrayname = get_varname(name, "_Py_M__");
+if (arrayname == NULL) {
+fprintf(stderr, "memory error: could not allocate varname\n");
+fclose(outfile);
+return -1;
+}
write_code(outfile, marshalled, arrayname);
free(arrayname);
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-123919: Fix null handling in `_freeze_module.c` (GH-123920) (#123949)
https://github.com/python/cpython/commit/cbfeb6a7fe89101e24ba7f61b5d42e12b79f1c9d commit: cbfeb6a7fe89101e24ba7f61b5d42e12b79f1c9d branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-09-11T13:56:00Z summary: [3.12] gh-123919: Fix null handling in `_freeze_module.c` (GH-123920) (#123949) gh-123919: Fix null handling in `_freeze_module.c` (GH-123920) (cherry picked from commit c8d1dbef5b770b647aa7ff45fd5b269bc7629d0b) Co-authored-by: sobolevn files: M Programs/_freeze_module.c diff --git a/Programs/_freeze_module.c b/Programs/_freeze_module.c index e55f1d56745c4d..c669d96c17351b 100644 --- a/Programs/_freeze_module.c +++ b/Programs/_freeze_module.c @@ -123,6 +123,9 @@ static PyObject * compile_and_marshal(const char *name, const char *text) { char *filename = (char *) malloc(strlen(name) + 10); +if (filename == NULL) { +return PyErr_NoMemory(); +} sprintf(filename, "", name); PyObject *code = Py_CompileStringExFlags(text, filename, Py_file_input, NULL, 0); @@ -146,6 +149,9 @@ get_varname(const char *name, const char *prefix) { size_t n = strlen(prefix); char *varname = (char *) malloc(strlen(name) + n + 1); +if (varname == NULL) { +return NULL; +} (void)strcpy(varname, prefix); for (size_t i = 0; name[i] != '\0'; i++) { if (name[i] == '.') { @@ -191,6 +197,11 @@ write_frozen(const char *outpath, const char *inpath, const char *name, fprintf(outfile, "%s\n", header); char *arrayname = get_varname(name, "_Py_M__"); +if (arrayname == NULL) { +fprintf(stderr, "memory error: could not allocate varname\n"); +fclose(outfile); +return -1; +} write_code(outfile, marshalled, arrayname); free(arrayname); ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120121: Add InvalidStateError to concurrent.futures.__all__ (#120123)
https://github.com/python/cpython/commit/5d59b870effa0f576acf7264cfcbfca2b36e34e3 commit: 5d59b870effa0f576acf7264cfcbfca2b36e34e3 branch: main author: AN Long committer: kumaraditya303 date: 2024-06-08T21:41:19+05:30 summary: gh-120121: Add InvalidStateError to concurrent.futures.__all__ (#120123) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst M Lib/concurrent/futures/__init__.py diff --git a/Lib/concurrent/futures/__init__.py b/Lib/concurrent/futures/__init__.py index 292e886d5a88ac..72de617a5b6f61 100644 --- a/Lib/concurrent/futures/__init__.py +++ b/Lib/concurrent/futures/__init__.py @@ -23,6 +23,7 @@ 'ALL_COMPLETED', 'CancelledError', 'TimeoutError', +'InvalidStateError', 'BrokenExecutor', 'Future', 'Executor', diff --git a/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst new file mode 100644 index 00..4f3526477c8cce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst @@ -0,0 +1 @@ +Add :exc:`concurrent.futures.InvalidStateError` to module's ``__all__``. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (#120274)
https://github.com/python/cpython/commit/58657d0dc160b8dea60cb5e5a51c5c925995cc36 commit: 58657d0dc160b8dea60cb5e5a51c5c925995cc36 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-08T16:28:21Z summary: [3.12] gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (#120274) gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (cherry picked from commit 5d59b870effa0f576acf7264cfcbfca2b36e34e3) Co-authored-by: AN Long Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst M Lib/concurrent/futures/__init__.py diff --git a/Lib/concurrent/futures/__init__.py b/Lib/concurrent/futures/__init__.py index 292e886d5a88ac..72de617a5b6f61 100644 --- a/Lib/concurrent/futures/__init__.py +++ b/Lib/concurrent/futures/__init__.py @@ -23,6 +23,7 @@ 'ALL_COMPLETED', 'CancelledError', 'TimeoutError', +'InvalidStateError', 'BrokenExecutor', 'Future', 'Executor', diff --git a/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst new file mode 100644 index 00..4f3526477c8cce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst @@ -0,0 +1 @@ +Add :exc:`concurrent.futures.InvalidStateError` to module's ``__all__``. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (#120273)
https://github.com/python/cpython/commit/4bca4e58af2e8cf6ce78151d3ae73bd254a4e64c commit: 4bca4e58af2e8cf6ce78151d3ae73bd254a4e64c branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-08T16:35:10Z summary: [3.13] gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (#120273) gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123) (cherry picked from commit 5d59b870effa0f576acf7264cfcbfca2b36e34e3) Co-authored-by: AN Long Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst M Lib/concurrent/futures/__init__.py diff --git a/Lib/concurrent/futures/__init__.py b/Lib/concurrent/futures/__init__.py index 292e886d5a88ac..72de617a5b6f61 100644 --- a/Lib/concurrent/futures/__init__.py +++ b/Lib/concurrent/futures/__init__.py @@ -23,6 +23,7 @@ 'ALL_COMPLETED', 'CancelledError', 'TimeoutError', +'InvalidStateError', 'BrokenExecutor', 'Future', 'Executor', diff --git a/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst new file mode 100644 index 00..4f3526477c8cce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-05-16-30-28.gh-issue-120121.9dz8i7.rst @@ -0,0 +1 @@ +Add :exc:`concurrent.futures.InvalidStateError` to module's ``__all__``. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120568: fix file leak in PyUnstable_CopyPerfMapFile (#120569)
https://github.com/python/cpython/commit/92cebaa4911786683e87841bf7788351e7595ac2
commit: 92cebaa4911786683e87841bf7788351e7595ac2
branch: main
author: Carson Radtke
committer: kumaraditya303
date: 2024-06-16T17:51:52Z
summary:
gh-120568: fix file leak in PyUnstable_CopyPerfMapFile (#120569)
files:
M Python/sysmodule.c
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
index 00aa95531026b5..3bb7b4d59b94e2 100644
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -2509,16 +2509,16 @@ PyAPI_FUNC(void) PyUnstable_PerfMapState_Fini(void) {
PyAPI_FUNC(int) PyUnstable_CopyPerfMapFile(const char* parent_filename) {
#ifndef MS_WINDOWS
-FILE* from = fopen(parent_filename, "r");
-if (!from) {
-return -1;
-}
if (perf_map_state.perf_map == NULL) {
int ret = PyUnstable_PerfMapState_Init();
if (ret != 0) {
return ret;
}
}
+FILE* from = fopen(parent_filename, "r");
+if (!from) {
+return -1;
+}
char buf[4096];
PyThread_acquire_lock(perf_map_state.map_lock, 1);
int fflush_result = 0, result = 0;
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-114091: Reword error message for unawaitable types (#114090)
https://github.com/python/cpython/commit/2c7209a3bdf81a289ccd6b80a77497cfcd5732de
commit: 2c7209a3bdf81a289ccd6b80a77497cfcd5732de
branch: main
author: Steele Farnsworth
committer: kumaraditya303
date: 2024-06-17T20:18:17+05:30
summary:
gh-114091: Reword error message for unawaitable types (#114090)
Reword error message for unawaitable types.
files:
A Misc/NEWS.d/next/Core and
Builtins/2024-01-15-21-51-26.gh-issue-114091.VOtSJl.rst
M Lib/test/test_asyncio/test_locks.py
M Lib/test/test_asyncio/test_pep492.py
M Lib/test/test_coroutines.py
M Objects/genobject.c
diff --git a/Lib/test/test_asyncio/test_locks.py
b/Lib/test/test_asyncio/test_locks.py
index c961dadff9fef2..34509717f2872a 100644
--- a/Lib/test/test_asyncio/test_locks.py
+++ b/Lib/test/test_asyncio/test_locks.py
@@ -39,7 +39,7 @@ async def test_lock(self):
with self.assertRaisesRegex(
TypeError,
-"object Lock can't be used in 'await' expression"
+"'Lock' object can't be awaited"
):
await lock
@@ -77,7 +77,7 @@ async def test_lock_by_with_statement(self):
self.assertFalse(lock.locked())
with self.assertRaisesRegex(
TypeError,
-r"object \w+ can't be used in 'await' expression"
+r"'\w+' object can't be awaited"
):
with await lock:
pass
@@ -941,7 +941,7 @@ async def test_semaphore(self):
with self.assertRaisesRegex(
TypeError,
-"object Semaphore can't be used in 'await' expression",
+"'Semaphore' object can't be awaited",
):
await sem
@@ -1270,7 +1270,7 @@ async def test_barrier(self):
self.assertIn("filling", repr(barrier))
with self.assertRaisesRegex(
TypeError,
-"object Barrier can't be used in 'await' expression",
+"'Barrier' object can't be awaited",
):
await barrier
diff --git a/Lib/test/test_asyncio/test_pep492.py
b/Lib/test/test_asyncio/test_pep492.py
index dc25a46985e349..033784bc7aec05 100644
--- a/Lib/test/test_asyncio/test_pep492.py
+++ b/Lib/test/test_asyncio/test_pep492.py
@@ -77,7 +77,7 @@ async def test(lock):
self.assertFalse(lock.locked())
with self.assertRaisesRegex(
TypeError,
-"can't be used in 'await' expression"
+"can't be awaited"
):
with await lock:
pass
diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
index f705f4f5bfbd88..a677301c62becc 100644
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -974,13 +974,13 @@ def test_await_1(self):
async def foo():
await 1
-with self.assertRaisesRegex(TypeError, "object int can.t.*await"):
+with self.assertRaisesRegex(TypeError, "'int' object can.t be
awaited"):
run_async(foo())
def test_await_2(self):
async def foo():
await []
-with self.assertRaisesRegex(TypeError, "object list can.t.*await"):
+with self.assertRaisesRegex(TypeError, "'list' object can.t be
awaited"):
run_async(foo())
def test_await_3(self):
@@ -1040,7 +1040,7 @@ class Awaitable:
async def foo(): return await Awaitable()
with self.assertRaisesRegex(
-TypeError, "object Awaitable can't be used in 'await' expression"):
+TypeError, "'Awaitable' object can't be awaited"):
run_async(foo())
diff --git a/Misc/NEWS.d/next/Core and
Builtins/2024-01-15-21-51-26.gh-issue-114091.VOtSJl.rst b/Misc/NEWS.d/next/Core
and Builtins/2024-01-15-21-51-26.gh-issue-114091.VOtSJl.rst
new file mode 100644
index 00..55b7d9104baed9
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and
Builtins/2024-01-15-21-51-26.gh-issue-114091.VOtSJl.rst
@@ -0,0 +1 @@
+Changed the error message for awaiting something that can't be awaited from
"object can't be used in an await expression" to "'' object can't
be awaited".
diff --git a/Objects/genobject.c b/Objects/genobject.c
index 92cd8c61e7e9ca..662e644f8aeddd 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1047,7 +1047,7 @@ _PyCoro_GetAwaitableIter(PyObject *o)
}
PyErr_Format(PyExc_TypeError,
- "object %.100s can't be used in 'await' expression",
+ "'%.100s' object can't be awaited",
ot->tp_name);
return NULL;
}
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-115649: Copy the filename into main interpreter before intern in import.c (#120315)
https://github.com/python/cpython/commit/28140d1f2da1766bfbb83f58779f15255c73c871
commit: 28140d1f2da1766bfbb83f58779f15255c73c871
branch: main
author: AN Long
committer: kumaraditya303
date: 2024-06-17T21:27:22+05:30
summary:
gh-115649: Copy the filename into main interpreter before intern in import.c
(#120315)
Co-authored-by: Kumar Aditya
files:
M Lib/test/test_import/__init__.py
M Python/import.c
diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py
index 97e262cfdb9023..e29097baaf53ae 100644
--- a/Lib/test/test_import/__init__.py
+++ b/Lib/test/test_import/__init__.py
@@ -2138,6 +2138,8 @@ def test_single_init_extension_compat(self):
self.check_incompatible_here(module)
with self.subTest(f'{module}: strict, fresh'):
self.check_incompatible_fresh(module)
+with self.subTest(f'{module}: isolated, fresh'):
+self.check_incompatible_fresh(module, isolated=True)
@unittest.skipIf(_testmultiphase is None, "test requires _testmultiphase
module")
def test_multi_init_extension_compat(self):
diff --git a/Python/import.c b/Python/import.c
index 2c7a461ac786c8..932881950d7baa 100644
--- a/Python/import.c
+++ b/Python/import.c
@@ -1969,7 +1969,17 @@ import_run_extension(PyThreadState *tstate,
PyModInitFunction p0,
if (info->filename != NULL) {
// XXX There's a refleak somewhere with the filename.
// Until we can track it down, we intern it.
-PyObject *filename = Py_NewRef(info->filename);
+PyObject *filename = NULL;
+if (switched) {
+// The original filename may be allocated by
subinterpreter's
+// obmalloc, so we create a copy here.
+filename = _PyUnicode_Copy(info->filename);
+if (filename == NULL) {
+return NULL;
+}
+} else {
+filename = Py_NewRef(info->filename);
+}
PyUnicode_InternInPlace(&filename);
if (PyModule_AddObjectRef(mod, "__file__", filename) < 0) {
PyErr_Clear(); /* Not important enough to report */
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] fix enum doc typo (#120091)
https://github.com/python/cpython/commit/0506f75c53ce0895408050ededcfb66bc07eb19b commit: 0506f75c53ce0895408050ededcfb66bc07eb19b branch: main author: Saul Pwanson committer: kumaraditya303 date: 2024-06-17T21:31:15+05:30 summary: fix enum doc typo (#120091) files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 9cf94e342dad28..8b3f397ea862f4 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -629,7 +629,7 @@ Data Types of two, starting with ``1``. .. versionchanged:: 3.11 The *repr()* of zero-valued flags has changed. It - is now:: + is now: >>> Color(0) # doctest: +SKIP ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-115649: Copy the filename into main interpreter before intern in import.c (GH-120315) (#120652)
https://github.com/python/cpython/commit/9172bc35c669b7706d7122d4f0a90f065700a474 commit: 9172bc35c669b7706d7122d4f0a90f065700a474 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-17T22:30:26+05:30 summary: [3.13] gh-115649: Copy the filename into main interpreter before intern in import.c (GH-120315) (#120652) gh-115649: Copy the filename into main interpreter before intern in import.c (GH-120315) (cherry picked from commit 28140d1f2da1766bfbb83f58779f15255c73c871) Co-authored-by: AN Long Co-authored-by: Kumar Aditya files: M Lib/test/test_import/__init__.py M Python/import.c diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index b09065f812c0e4..f9e8558d1a71c8 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -2157,6 +2157,8 @@ def test_single_init_extension_compat(self): self.check_incompatible_here(module) with self.subTest(f'{module}: strict, fresh'): self.check_incompatible_fresh(module) +with self.subTest(f'{module}: isolated, fresh'): +self.check_incompatible_fresh(module, isolated=True) @unittest.skipIf(_testmultiphase is None, "test requires _testmultiphase module") def test_multi_init_extension_compat(self): diff --git a/Python/import.c b/Python/import.c index 10ac49fd95fc59..8cf97b51f65fec 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1969,7 +1969,17 @@ import_run_extension(PyThreadState *tstate, PyModInitFunction p0, if (info->filename != NULL) { // XXX There's a refleak somewhere with the filename. // Until we can track it down, we intern it. -PyObject *filename = Py_NewRef(info->filename); +PyObject *filename = NULL; +if (switched) { +// The original filename may be allocated by subinterpreter's +// obmalloc, so we create a copy here. +filename = _PyUnicode_Copy(info->filename); +if (filename == NULL) { +return NULL; +} +} else { +filename = Py_NewRef(info->filename); +} PyUnicode_InternInPlace(&filename); if (PyModule_AddObjectRef(mod, "__file__", filename) < 0) { PyErr_Clear(); /* Not important enough to report */ ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-119960: Add information about regex flags in re module functions (#119978)
https://github.com/python/cpython/commit/a86e6255c371e14cab8680dee979a7393b339ce5 commit: a86e6255c371e14cab8680dee979a7393b339ce5 branch: main author: Awbert <[email protected]> committer: kumaraditya303 date: 2024-06-19T09:42:01Z summary: gh-119960: Add information about regex flags in re module functions (#119978) files: M Doc/library/re.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 39788de76b558b..bce36660afcada 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -911,6 +911,10 @@ Functions ``None`` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: match(pattern, string, flags=0) @@ -925,6 +929,10 @@ Functions If you want to locate a match anywhere in *string*, use :func:`search` instead (see also :ref:`search-vs-match`). + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: fullmatch(pattern, string, flags=0) @@ -932,6 +940,10 @@ Functions corresponding :class:`~re.Match`. Return ``None`` if the string does not match the pattern; note that this is different from a zero-length match. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionadded:: 3.4 @@ -974,6 +986,10 @@ Functions >>> re.split(r'(\W*)', '...words...') ['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', ''] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -1004,6 +1020,10 @@ Functions >>> re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10') [('width', '20'), ('height', '10')] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1015,6 +1035,10 @@ Functions is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1070,6 +1094,10 @@ Functions character ``'0'``. The backreference ``\g<0>`` substitutes in the entire substring matched by the RE. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -1102,6 +1130,10 @@ Functions Perform the same operation as :func:`sub`, but return a tuple ``(new_string, number_of_subs_made)``. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: escape(pattern) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (#119964)
https://github.com/python/cpython/commit/9be94f9ce6dbc944e9807534664dbde28077f737 commit: 9be94f9ce6dbc944e9807534664dbde28077f737 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-19T10:11:07Z summary: [3.13] gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (#119964) gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (cherry picked from commit 52586f930f62bd80374f0f240a4ecce0c0238174) Co-authored-by: Radislav Chugunov <[email protected]> Co-authored-by: Inada Naoki files: A Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst M Lib/test/test_io.py M Modules/_io/textio.c diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index e5cb08c2cdd04c..1ca3edac8c8dc9 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -4016,6 +4016,28 @@ def write(self, data): t.write("x"*chunk_size) self.assertEqual([b"abcdef", b"ghi", b"x"*chunk_size], buf._write_stack) +def test_issue119506(self): +chunk_size = 8192 + +class MockIO(self.MockRawIO): +written = False +def write(self, data): +if not self.written: +self.written = True +t.write("middle") +return super().write(data) + +buf = MockIO() +t = self.TextIOWrapper(buf) +t.write("abc") +t.write("def") +# writing data which size >= chunk_size cause flushing buffer before write. +t.write("g" * chunk_size) +t.flush() + +self.assertEqual([b"abcdef", b"middle", b"g"*chunk_size], + buf._write_stack) + class PyTextIOWrapperTest(TextIOWrapperTest): io = pyio diff --git a/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst b/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst new file mode 100644 index 00..f9b764ae0c49b3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst @@ -0,0 +1 @@ +Fix :meth:`!io.TextIOWrapper.write` method breaks internal buffer when the method is called again during flushing internal buffer. diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index 9dff8eafb2560f..c162d8106ec1fd 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -1719,16 +1719,26 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) bytes_len = PyBytes_GET_SIZE(b); } -if (self->pending_bytes == NULL) { -self->pending_bytes_count = 0; -self->pending_bytes = b; -} -else if (self->pending_bytes_count + bytes_len > self->chunk_size) { -// Prevent to concatenate more than chunk_size data. -if (_textiowrapper_writeflush(self) < 0) { -Py_DECREF(b); -return NULL; +// We should avoid concatinating huge data. +// Flush the buffer before adding b to the buffer if b is not small. +// https://github.com/python/cpython/issues/87426 +if (bytes_len >= self->chunk_size) { +// _textiowrapper_writeflush() calls buffer.write(). +// self->pending_bytes can be appended during buffer->write() +// or other thread. +// We need to loop until buffer becomes empty. +// https://github.com/python/cpython/issues/118138 +// https://github.com/python/cpython/issues/119506 +while (self->pending_bytes != NULL) { +if (_textiowrapper_writeflush(self) < 0) { +Py_DECREF(b); +return NULL; +} } +} + +if (self->pending_bytes == NULL) { +assert(self->pending_bytes_count == 0); self->pending_bytes = b; } else if (!PyList_CheckExact(self->pending_bytes)) { @@ -1737,6 +1747,9 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) Py_DECREF(b); return NULL; } +// Since Python 3.12, allocating GC object won't trigger GC and release +// GIL. See https://github.com/python/cpython/issues/97922 +assert(!PyList_CheckExact(self->pending_bytes)); PyList_SET_ITEM(list, 0, self->pending_bytes); PyList_SET_ITEM(list, 1, b); self->pending_bytes = list; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (#119965)
https://github.com/python/cpython/commit/cde976d85c6632a908dde1ff8695ac5cafd879b6 commit: cde976d85c6632a908dde1ff8695ac5cafd879b6 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-19T10:23:29Z summary: [3.12] gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (#119965) gh-119506: fix `_io.TextIOWrapper.write()` write during flush (GH-119507) (cherry picked from commit 52586f930f62bd80374f0f240a4ecce0c0238174) Co-authored-by: Radislav Chugunov <[email protected]> Co-authored-by: Inada Naoki files: A Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst M Lib/test/test_io.py M Modules/_io/textio.c diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index daa40a6ba365b9..8b68653779e733 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -4066,6 +4066,28 @@ def write(self, data): t.write("x"*chunk_size) self.assertEqual([b"abcdef", b"ghi", b"x"*chunk_size], buf._write_stack) +def test_issue119506(self): +chunk_size = 8192 + +class MockIO(self.MockRawIO): +written = False +def write(self, data): +if not self.written: +self.written = True +t.write("middle") +return super().write(data) + +buf = MockIO() +t = self.TextIOWrapper(buf) +t.write("abc") +t.write("def") +# writing data which size >= chunk_size cause flushing buffer before write. +t.write("g" * chunk_size) +t.flush() + +self.assertEqual([b"abcdef", b"middle", b"g"*chunk_size], + buf._write_stack) + class PyTextIOWrapperTest(TextIOWrapperTest): io = pyio diff --git a/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst b/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst new file mode 100644 index 00..f9b764ae0c49b3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-05-24-14-32-24.gh-issue-119506.-nMNqq.rst @@ -0,0 +1 @@ +Fix :meth:`!io.TextIOWrapper.write` method breaks internal buffer when the method is called again during flushing internal buffer. diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index 14dd19d95c2498..4a1ba22d381a53 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -1723,16 +1723,26 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) bytes_len = PyBytes_GET_SIZE(b); } -if (self->pending_bytes == NULL) { -self->pending_bytes_count = 0; -self->pending_bytes = b; -} -else if (self->pending_bytes_count + bytes_len > self->chunk_size) { -// Prevent to concatenate more than chunk_size data. -if (_textiowrapper_writeflush(self) < 0) { -Py_DECREF(b); -return NULL; +// We should avoid concatinating huge data. +// Flush the buffer before adding b to the buffer if b is not small. +// https://github.com/python/cpython/issues/87426 +if (bytes_len >= self->chunk_size) { +// _textiowrapper_writeflush() calls buffer.write(). +// self->pending_bytes can be appended during buffer->write() +// or other thread. +// We need to loop until buffer becomes empty. +// https://github.com/python/cpython/issues/118138 +// https://github.com/python/cpython/issues/119506 +while (self->pending_bytes != NULL) { +if (_textiowrapper_writeflush(self) < 0) { +Py_DECREF(b); +return NULL; +} } +} + +if (self->pending_bytes == NULL) { +assert(self->pending_bytes_count == 0); self->pending_bytes = b; } else if (!PyList_CheckExact(self->pending_bytes)) { @@ -1741,6 +1751,9 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) Py_DECREF(b); return NULL; } +// Since Python 3.12, allocating GC object won't trigger GC and release +// GIL. See https://github.com/python/cpython/issues/97922 +assert(!PyList_CheckExact(self->pending_bytes)); PyList_SET_ITEM(list, 0, self->pending_bytes); PyList_SET_ITEM(list, 1, b); self->pending_bytes = list; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-119960: Add information about regex flags in re module functions (GH-119978) (#120730)
https://github.com/python/cpython/commit/c598e61a223edcd9a7eecf823f667e2e810aba53 commit: c598e61a223edcd9a7eecf823f667e2e810aba53 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-19T16:00:34+05:30 summary: [3.13] gh-119960: Add information about regex flags in re module functions (GH-119978) (#120730) gh-119960: Add information about regex flags in re module functions (GH-119978) (cherry picked from commit a86e6255c371e14cab8680dee979a7393b339ce5) Co-authored-by: Awbert <[email protected]> files: M Doc/library/re.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 39788de76b558b..bce36660afcada 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -911,6 +911,10 @@ Functions ``None`` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: match(pattern, string, flags=0) @@ -925,6 +929,10 @@ Functions If you want to locate a match anywhere in *string*, use :func:`search` instead (see also :ref:`search-vs-match`). + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: fullmatch(pattern, string, flags=0) @@ -932,6 +940,10 @@ Functions corresponding :class:`~re.Match`. Return ``None`` if the string does not match the pattern; note that this is different from a zero-length match. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionadded:: 3.4 @@ -974,6 +986,10 @@ Functions >>> re.split(r'(\W*)', '...words...') ['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', ''] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -1004,6 +1020,10 @@ Functions >>> re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10') [('width', '20'), ('height', '10')] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1015,6 +1035,10 @@ Functions is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1070,6 +1094,10 @@ Functions character ``'0'``. The backreference ``\g<0>`` substitutes in the entire substring matched by the RE. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -1102,6 +1130,10 @@ Functions Perform the same operation as :func:`sub`, but return a tuple ``(new_string, number_of_subs_made)``. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: escape(pattern) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] GH-120804: Remove SafeChildWatcher, FastChildWatcher and MultiLoopChildWatcher from asyncio (#120805)
https://github.com/python/cpython/commit/733dac01b0dc3047efc9027dba177d7116e47c50
commit: 733dac01b0dc3047efc9027dba177d7116e47c50
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-06-21T10:23:10+05:30
summary:
GH-120804: Remove SafeChildWatcher, FastChildWatcher and MultiLoopChildWatcher
from asyncio (#120805)
Remove SafeChildWatcher, FastChildWatcher and MultiLoopChildWatcher from
asyncio. These child watchers have been deprecated since Python 3.12. The tests
are also removed and some more tests will be added after the rewrite of child
watchers.
files:
M Lib/asyncio/unix_events.py
M Lib/test/test_asyncio/test_events.py
M Lib/test/test_asyncio/test_streams.py
M Lib/test/test_asyncio/test_subprocess.py
M Lib/test/test_asyncio/test_unix_events.py
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
index 41ccf1b78fb93b..9a2e300259ee8c 100644
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -28,9 +28,9 @@
__all__ = (
'SelectorEventLoop',
-'AbstractChildWatcher', 'SafeChildWatcher',
-'FastChildWatcher', 'PidfdChildWatcher',
-'MultiLoopChildWatcher', 'ThreadedChildWatcher',
+'AbstractChildWatcher',
+'PidfdChildWatcher',
+'ThreadedChildWatcher',
'DefaultEventLoopPolicy',
'EventLoop',
)
@@ -1062,325 +1062,6 @@ def _sig_chld(self):
})
-class SafeChildWatcher(BaseChildWatcher):
-"""'Safe' child watcher implementation.
-
-This implementation avoids disrupting other code spawning processes by
-polling explicitly each process in the SIGCHLD handler instead of calling
-os.waitpid(-1).
-
-This is a safe solution but it has a significant overhead when handling a
-big number of children (O(n) each time SIGCHLD is raised)
-"""
-
-def __init__(self):
-super().__init__()
-warnings._deprecated("SafeChildWatcher",
- "{name!r} is deprecated as of Python 3.12 and
will be "
- "removed in Python {remove}.",
- remove=(3, 14))
-
-def close(self):
-self._callbacks.clear()
-super().close()
-
-def __enter__(self):
-return self
-
-def __exit__(self, a, b, c):
-pass
-
-def add_child_handler(self, pid, callback, *args):
-self._callbacks[pid] = (callback, args)
-
-# Prevent a race condition in case the child is already terminated.
-self._do_waitpid(pid)
-
-def remove_child_handler(self, pid):
-try:
-del self._callbacks[pid]
-return True
-except KeyError:
-return False
-
-def _do_waitpid_all(self):
-
-for pid in list(self._callbacks):
-self._do_waitpid(pid)
-
-def _do_waitpid(self, expected_pid):
-assert expected_pid > 0
-
-try:
-pid, status = os.waitpid(expected_pid, os.WNOHANG)
-except ChildProcessError:
-# The child process is already reaped
-# (may happen if waitpid() is called elsewhere).
-pid = expected_pid
-returncode = 255
-logger.warning(
-"Unknown child process pid %d, will report returncode 255",
-pid)
-else:
-if pid == 0:
-# The child process is still alive.
-return
-
-returncode = waitstatus_to_exitcode(status)
-if self._loop.get_debug():
-logger.debug('process %s exited with returncode %s',
- expected_pid, returncode)
-
-try:
-callback, args = self._callbacks.pop(pid)
-except KeyError: # pragma: no cover
-# May happen if .remove_child_handler() is called
-# after os.waitpid() returns.
-if self._loop.get_debug():
-logger.warning("Child watcher got an unexpected pid: %r",
- pid, exc_info=True)
-else:
-callback(pid, returncode, *args)
-
-
-class FastChildWatcher(BaseChildWatcher):
-"""'Fast' child watcher implementation.
-
-This implementation reaps every terminated processes by calling
-os.waitpid(-1) directly, possibly breaking other code spawning processes
-and waiting for their termination.
-
-There is no noticeable overhead when handling a big number of children
-(O(1) each time a child terminates).
-"""
-def __init__(self):
-super().__init__()
-self._lock = threading.Lock()
-self._zombies = {}
-self._forks = 0
-warnings._deprecated("FastChildWatcher",
- "{name!r}
[Python-checkins] gh-120773: document introspective attributes of an async generator object in the inspect module (#120778)
https://github.com/python/cpython/commit/83d3d7aace32b8536f552f78dd29610344f13160 commit: 83d3d7aace32b8536f552f78dd29610344f13160 branch: main author: blhsing committer: kumaraditya303 date: 2024-06-21T10:55:36Z summary: gh-120773: document introspective attributes of an async generator object in the inspect module (#120778) files: M Doc/library/inspect.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 0ec7d7cecb89e0..7838eeed2843c4 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -42,220 +42,233 @@ attributes (see :ref:`import-mod-attrs` for module attributes): .. this function name is too big to fit in the ascii-art table below .. |coroutine-origin-link| replace:: :func:`sys.set_coroutine_origin_tracking_depth` -+---+---+---+ -| Type | Attribute | Description | -+===+===+===+ -| class | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | class was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __module__| name of module in which | -| | | this class was defined| -+---+---+---+ -| | __type_params__ | A tuple containing the| -| | | :ref:`type parameters | -| | | ` of | -| | | a generic class | -+---+---+---+ -| method| __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | method was defined| -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __func__ | function object | -| | | containing implementation | -| | | of method | -+---+---+---+ -| | __self__ | instance to which this| -| | | method is bound, or | -| | | ``None`` | -+---+---+---+ -| | __module__| name of module in which | -| | | this method was defined | -+---+---+---+ -| function | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | function was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __code__ | code object containing| -| | | compiled function | -| | | :term:`bytecode` | -+---+---+---+ -| | __defaults__ | tuple of any default | -| | | values for positional or | -| | | keyword parameters| -+---+---+---+ -| | __kwdefaults__| mapping of any default| -| | | values for keyword-only | -| | | parameters| -+---+---+---+ -| | __globals__ | global namespace in which | -| | | this function was defined | -+---+---+---+ -| | __builtins__ | builtins namespace| -+---+---+---+ -| | __annotations__ | mapping of parameters | -| | | names to annotations; | -| | | ``"return"`` key is | -| | | reserved for return | -| | | annotations. | -+---+---+---+ -| | __type_params__ | A tuple cont
[Python-checkins] [3.13] gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (#120827)
https://github.com/python/cpython/commit/abdbf337d4641cf27d39fae206c2a08d27f6fcd9 commit: abdbf337d4641cf27d39fae206c2a08d27f6fcd9 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-21T16:58:46+05:30 summary: [3.13] gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (#120827) gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (cherry picked from commit 83d3d7aace32b8536f552f78dd29610344f13160) Co-authored-by: blhsing files: M Doc/library/inspect.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 0ec7d7cecb89e0..7838eeed2843c4 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -42,220 +42,233 @@ attributes (see :ref:`import-mod-attrs` for module attributes): .. this function name is too big to fit in the ascii-art table below .. |coroutine-origin-link| replace:: :func:`sys.set_coroutine_origin_tracking_depth` -+---+---+---+ -| Type | Attribute | Description | -+===+===+===+ -| class | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | class was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __module__| name of module in which | -| | | this class was defined| -+---+---+---+ -| | __type_params__ | A tuple containing the| -| | | :ref:`type parameters | -| | | ` of | -| | | a generic class | -+---+---+---+ -| method| __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | method was defined| -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __func__ | function object | -| | | containing implementation | -| | | of method | -+---+---+---+ -| | __self__ | instance to which this| -| | | method is bound, or | -| | | ``None`` | -+---+---+---+ -| | __module__| name of module in which | -| | | this method was defined | -+---+---+---+ -| function | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | function was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __code__ | code object containing| -| | | compiled function | -| | | :term:`bytecode` | -+---+---+---+ -| | __defaults__ | tuple of any default | -| | | values for positional or | -| | | keyword parameters| -+---+---+---+ -| | __kwdefaults__| mapping of any default| -| | | values for keyword-only | -| | | parameters| -+---+---+---+ -| | __globals__ | global namespace in which | -| | | this function was defined | -+---+---+---+ -| | __builtins__ | builtins namespace| -+---+---+---+ -| | __annotations__ | mapping of parameters | -| | | names to annot
[Python-checkins] [3.12] gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (#120828)
https://github.com/python/cpython/commit/9bbb91c066d5aa3d9a1c5ac5ac72520b46e5e06f commit: 9bbb91c066d5aa3d9a1c5ac5ac72520b46e5e06f branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-21T16:58:58+05:30 summary: [3.12] gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (#120828) gh-120773: document introspective attributes of an async generator object in the inspect module (GH-120778) (cherry picked from commit 83d3d7aace32b8536f552f78dd29610344f13160) Co-authored-by: blhsing files: M Doc/library/inspect.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 7d1aab8e2999c7..f092d5171aa9b4 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -42,220 +42,233 @@ attributes (see :ref:`import-mod-attrs` for module attributes): .. this function name is too big to fit in the ascii-art table below .. |coroutine-origin-link| replace:: :func:`sys.set_coroutine_origin_tracking_depth` -+---+---+---+ -| Type | Attribute | Description | -+===+===+===+ -| class | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | class was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __module__| name of module in which | -| | | this class was defined| -+---+---+---+ -| | __type_params__ | A tuple containing the| -| | | :ref:`type parameters | -| | | ` of | -| | | a generic class | -+---+---+---+ -| method| __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | method was defined| -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __func__ | function object | -| | | containing implementation | -| | | of method | -+---+---+---+ -| | __self__ | instance to which this| -| | | method is bound, or | -| | | ``None`` | -+---+---+---+ -| | __module__| name of module in which | -| | | this method was defined | -+---+---+---+ -| function | __doc__ | documentation string | -+---+---+---+ -| | __name__ | name with which this | -| | | function was defined | -+---+---+---+ -| | __qualname__ | qualified name| -+---+---+---+ -| | __code__ | code object containing| -| | | compiled function | -| | | :term:`bytecode` | -+---+---+---+ -| | __defaults__ | tuple of any default | -| | | values for positional or | -| | | keyword parameters| -+---+---+---+ -| | __kwdefaults__| mapping of any default| -| | | values for keyword-only | -| | | parameters| -+---+---+---+ -| | __globals__ | global namespace in which | -| | | this function was defined | -+---+---+---+ -| | __builtins__ | builtins namespace| -+---+---+---+ -| | __annotations__ | mapping of parameters | -| | | names to annot
[Python-checkins] gh-120782: Update internal type cache when reloading datetime (#120829)
https://github.com/python/cpython/commit/a81d434c06335b0989ba83666ec7076b9d9d4e1e commit: a81d434c06335b0989ba83666ec7076b9d9d4e1e branch: main author: neonene <[email protected]> committer: kumaraditya303 date: 2024-06-21T22:39:33+05:30 summary: gh-120782: Update internal type cache when reloading datetime (#120829) files: A Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst M Lib/test/datetimetester.py M Modules/_datetimemodule.c diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index e55b738eb4a975..b8f69e774f7990 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -6870,6 +6870,23 @@ def pickle_fake_date(datetime_) -> Type[FakeDate]: """) script_helper.assert_python_ok('-c', script) +def test_update_type_cache(self): +# gh-120782 +script = textwrap.dedent(""" +import sys +for i in range(5): +import _datetime +_datetime.date.max > _datetime.date.min +_datetime.time.max > _datetime.time.min +_datetime.datetime.max > _datetime.datetime.min +_datetime.timedelta.max > _datetime.timedelta.min +isinstance(_datetime.timezone.min, _datetime.tzinfo) +isinstance(_datetime.timezone.utc, _datetime.tzinfo) +isinstance(_datetime.timezone.max, _datetime.tzinfo) +del sys.modules['_datetime'] +""") +script_helper.assert_python_ok('-c', script) + def load_tests(loader, standard_tests, pattern): standard_tests.addTest(ZoneInfoCompleteTest()) diff --git a/Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst b/Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst new file mode 100644 index 00..02acbd2873009b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst @@ -0,0 +1 @@ +Fix wrong references of the :mod:`datetime` types after reloading the module. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 31bf641152d803..85595dce0bad5c 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -7296,6 +7296,12 @@ _datetime_exec(PyObject *module) static_assert(DI100Y == 25 * DI4Y - 1, "DI100Y"); assert(DI100Y == days_before_year(100+1)); +if (reloading) { +for (size_t i = 0; i < Py_ARRAY_LENGTH(capi_types); i++) { +PyType_Modified(capi_types[i]); +} +} + if (set_current_module(interp, module) < 0) { goto error; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120811: Fix reference leak upon `_PyContext_Exit` failure (#120812)
https://github.com/python/cpython/commit/aed31beca9a54b85a1392631a48da80602210f18
commit: aed31beca9a54b85a1392631a48da80602210f18
branch: main
author: Peter
committer: kumaraditya303
date: 2024-06-21T17:28:19Z
summary:
gh-120811: Fix reference leak upon `_PyContext_Exit` failure (#120812)
Co-authored-by: Kumar Aditya
files:
A Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst
M Python/context.c
diff --git
a/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst
b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst
new file mode 100644
index 00..62cd7b5620474a
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst
@@ -0,0 +1 @@
+Fix possible memory leak in :meth:`contextvars.Context.run`.
diff --git a/Python/context.c b/Python/context.c
index 42000b13038834..a3830be17908fe 100644
--- a/Python/context.c
+++ b/Python/context.c
@@ -661,6 +661,7 @@ context_run(PyContext *self, PyObject *const *args,
ts, args[0], args + 1, nargs - 1, kwnames);
if (_PyContext_Exit(ts, (PyObject *)self)) {
+Py_XDECREF(call_result);
return NULL;
}
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (#120844)
https://github.com/python/cpython/commit/b1bccab588b978220f16eedf839af70cb8bb76ea commit: b1bccab588b978220f16eedf839af70cb8bb76ea branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-22T16:44:31+05:30 summary: [3.12] gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (#120844) gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (cherry picked from commit aed31beca9a54b85a1392631a48da80602210f18) Co-authored-by: Peter Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst M Python/context.c diff --git a/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst new file mode 100644 index 00..62cd7b5620474a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst @@ -0,0 +1 @@ +Fix possible memory leak in :meth:`contextvars.Context.run`. diff --git a/Python/context.c b/Python/context.c index 1ffae9871be7b3..7bccfad11a45b1 100644 --- a/Python/context.c +++ b/Python/context.c @@ -669,6 +669,7 @@ context_run(PyContext *self, PyObject *const *args, ts, args[0], args + 1, nargs - 1, kwnames); if (_PyContext_Exit(ts, (PyObject *)self)) { +Py_XDECREF(call_result); return NULL; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (#120843)
https://github.com/python/cpython/commit/a860b1d60b8e3d09c58daf6996a38606f407a009 commit: a860b1d60b8e3d09c58daf6996a38606f407a009 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-22T16:44:46+05:30 summary: [3.13] gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (#120843) gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (cherry picked from commit aed31beca9a54b85a1392631a48da80602210f18) Co-authored-by: Peter Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst M Python/context.c diff --git a/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst new file mode 100644 index 00..62cd7b5620474a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-21-14-32-56.gh-issue-120811.eBmVTV.rst @@ -0,0 +1 @@ +Fix possible memory leak in :meth:`contextvars.Context.run`. diff --git a/Python/context.c b/Python/context.c index 3937819b3c386c..63318d1e597439 100644 --- a/Python/context.c +++ b/Python/context.c @@ -661,6 +661,7 @@ context_run(PyContext *self, PyObject *const *args, ts, args[0], args + 1, nargs - 1, kwnames); if (_PyContext_Exit(ts, (PyObject *)self)) { +Py_XDECREF(call_result); return NULL; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] GH-120804: Remove `get_child_watcher` and `set_child_watcher` from asyncio (#120818)
https://github.com/python/cpython/commit/96ead91f0f0db59a942b8b34da9cc980c05588a2
commit: 96ead91f0f0db59a942b8b34da9cc980c05588a2
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-06-23T09:53:23+05:30
summary:
GH-120804: Remove `get_child_watcher` and `set_child_watcher` from asyncio
(#120818)
files:
M Lib/asyncio/events.py
M Lib/asyncio/unix_events.py
M Lib/test/test_asyncio/test_events.py
M Lib/test/test_asyncio/test_subprocess.py
M Lib/test/test_asyncio/test_unix_events.py
M Lib/test/test_asyncio/utils.py
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
index be495469a0558b..b63fe6aa79604b 100644
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -10,7 +10,6 @@
'Handle', 'TimerHandle',
'get_event_loop_policy', 'set_event_loop_policy',
'get_event_loop', 'set_event_loop', 'new_event_loop',
-'get_child_watcher', 'set_child_watcher',
'_set_running_loop', 'get_running_loop',
'_get_running_loop',
)
@@ -652,17 +651,6 @@ def new_event_loop(self):
the current context, set_event_loop must be called explicitly."""
raise NotImplementedError
-# Child processes handling (Unix only).
-
-def get_child_watcher(self):
-"Get the watcher for child processes."
-raise NotImplementedError
-
-def set_child_watcher(self, watcher):
-"""Set the watcher for child processes."""
-raise NotImplementedError
-
-
class BaseDefaultEventLoopPolicy(AbstractEventLoopPolicy):
"""Default policy implementation for accessing the event loop.
@@ -837,17 +825,6 @@ def new_event_loop():
return get_event_loop_policy().new_event_loop()
-def get_child_watcher():
-"""Equivalent to calling get_event_loop_policy().get_child_watcher()."""
-return get_event_loop_policy().get_child_watcher()
-
-
-def set_child_watcher(watcher):
-"""Equivalent to calling
-get_event_loop_policy().set_child_watcher(watcher)."""
-return get_event_loop_policy().set_child_watcher(watcher)
-
-
# Alias pure-Python implementations for testing purposes.
_py__get_running_loop = _get_running_loop
_py__set_running_loop = _set_running_loop
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
index 9a2e300259ee8c..ff2df653e41b89 100644
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -199,7 +199,7 @@ async def _make_subprocess_transport(self, protocol, args,
shell,
extra=None, **kwargs):
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
-watcher = events.get_child_watcher()
+watcher = events.get_event_loop_policy()._watcher
with watcher:
if not watcher.is_active():
@@ -1009,59 +1009,6 @@ def remove_child_handler(self, pid):
return True
-class BaseChildWatcher(AbstractChildWatcher):
-
-def __init__(self):
-self._loop = None
-self._callbacks = {}
-
-def close(self):
-self.attach_loop(None)
-
-def is_active(self):
-return self._loop is not None and self._loop.is_running()
-
-def _do_waitpid(self, expected_pid):
-raise NotImplementedError()
-
-def _do_waitpid_all(self):
-raise NotImplementedError()
-
-def attach_loop(self, loop):
-assert loop is None or isinstance(loop, events.AbstractEventLoop)
-
-if self._loop is not None and loop is None and self._callbacks:
-warnings.warn(
-'A loop is being detached '
-'from a child watcher with pending handlers',
-RuntimeWarning)
-
-if self._loop is not None:
-self._loop.remove_signal_handler(signal.SIGCHLD)
-
-self._loop = loop
-if loop is not None:
-loop.add_signal_handler(signal.SIGCHLD, self._sig_chld)
-
-# Prevent a race condition in case a child terminated
-# during the switch.
-self._do_waitpid_all()
-
-def _sig_chld(self):
-try:
-self._do_waitpid_all()
-except (SystemExit, KeyboardInterrupt):
-raise
-except BaseException as exc:
-# self._loop should always be available here
-# as '_sig_chld' is added as a signal handler
-# in 'attach_loop'
-self._loop.call_exception_handler({
-'message': 'Unknown exception in SIGCHLD handler',
-'exception': exc,
-})
-
-
class ThreadedChildWatcher(AbstractChildWatcher):
"""Threaded child watcher implementation.
@@ -1161,15 +1108
[Python-checkins] gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (#120898)
https://github.com/python/cpython/commit/b6fa8fe86a6f4d02c263682716a91285a94024fc commit: b6fa8fe86a6f4d02c263682716a91285a94024fc branch: main author: Nice Zombies committer: kumaraditya303 date: 2024-06-23T18:00:23+05:30 summary: gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (#120898) files: M Doc/library/urllib.parse.rst diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index cd402e87a8224b..27909b763e9e43 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -173,7 +173,7 @@ or on combining URL components into a URL string. Added IPv6 URL parsing capabilities. .. versionchanged:: 3.3 - The fragment is now parsed for all URL schemes (unless *allow_fragment* is + The fragment is now parsed for all URL schemes (unless *allow_fragments* is false), in accordance with :rfc:`3986`. Previously, an allowlist of schemes that support fragments existed. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-119003: Clarify slice assignments (GH-119935) (#120848)
https://github.com/python/cpython/commit/2b2530540f264926b854eb58c31ffe5dfb04895a commit: 2b2530540f264926b854eb58c31ffe5dfb04895a branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-23T18:04:25+05:30 summary: [3.12] gh-119003: Clarify slice assignments (GH-119935) (#120848) gh-119003: Clarify slice assignments (GH-119935) (cherry picked from commit 462832041e342f8aaf8c88ec44f7b14c70042575) Co-authored-by: Nice Zombies files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index d11bfb803f8c56..f1f413ef366dd1 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1220,7 +1220,7 @@ accepts integers that meet the value restriction ``0 <= x <= 255``). Notes: (1) - *t* must have the same length as the slice it is replacing. + If *k* is not equal to ``1``, *t* must have the same length as the slice it is replacing. (2) The optional argument *i* defaults to ``-1``, so that by default the last ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-119003: Clarify slice assignments (GH-119935) (#120847)
https://github.com/python/cpython/commit/bf7aac3d25dc396128d56b699ca269c89975e41f commit: bf7aac3d25dc396128d56b699ca269c89975e41f branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-23T18:04:48+05:30 summary: [3.13] gh-119003: Clarify slice assignments (GH-119935) (#120847) gh-119003: Clarify slice assignments (GH-119935) (cherry picked from commit 462832041e342f8aaf8c88ec44f7b14c70042575) Co-authored-by: Nice Zombies files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index ca4c949aafced0..79687b94f0a8e4 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1220,7 +1220,7 @@ accepts integers that meet the value restriction ``0 <= x <= 255``). Notes: (1) - *t* must have the same length as the slice it is replacing. + If *k* is not equal to ``1``, *t* must have the same length as the slice it is replacing. (2) The optional argument *i* defaults to ``-1``, so that by default the last ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (#120903)
https://github.com/python/cpython/commit/2d6d862706a630d5f043d2c01ab90e8ec9fbce41 commit: 2d6d862706a630d5f043d2c01ab90e8ec9fbce41 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-23T18:10:11+05:30 summary: [3.12] gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (#120903) gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (cherry picked from commit b6fa8fe86a6f4d02c263682716a91285a94024fc) Co-authored-by: Nice Zombies files: M Doc/library/urllib.parse.rst diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index cd402e87a8224b..27909b763e9e43 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -173,7 +173,7 @@ or on combining URL components into a URL string. Added IPv6 URL parsing capabilities. .. versionchanged:: 3.3 - The fragment is now parsed for all URL schemes (unless *allow_fragment* is + The fragment is now parsed for all URL schemes (unless *allow_fragments* is false), in accordance with :rfc:`3986`. Previously, an allowlist of schemes that support fragments existed. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (#120902)
https://github.com/python/cpython/commit/97acd295418493b19d78ed1a26d76e2dc8cc63dd commit: 97acd295418493b19d78ed1a26d76e2dc8cc63dd branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-23T18:28:06+05:30 summary: [3.13] gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (#120902) gh-120896: Fix typo in version changed note of `urllib.parse.urlparse()` (GH-120898) (cherry picked from commit b6fa8fe86a6f4d02c263682716a91285a94024fc) Co-authored-by: Nice Zombies files: M Doc/library/urllib.parse.rst diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index cd402e87a8224b..27909b763e9e43 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -173,7 +173,7 @@ or on combining URL components into a URL string. Added IPv6 URL parsing capabilities. .. versionchanged:: 3.3 - The fragment is now parsed for all URL schemes (unless *allow_fragment* is + The fragment is now parsed for all URL schemes (unless *allow_fragments* is false), in accordance with :rfc:`3986`. Previously, an allowlist of schemes that support fragments existed. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] GH-120804: Remove `PidfdChildWatcher`, `ThreadedChildWatcher` and `AbstractChildWatcher` from asyncio APIs (#120893)
https://github.com/python/cpython/commit/9d2e1ea3862e5950d48b45ac57995a206e33f38b
commit: 9d2e1ea3862e5950d48b45ac57995a206e33f38b
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-06-23T18:38:50+05:30
summary:
GH-120804: Remove `PidfdChildWatcher`, `ThreadedChildWatcher` and
`AbstractChildWatcher` from asyncio APIs (#120893)
files:
M Lib/asyncio/unix_events.py
M Lib/test/test_asyncio/test_events.py
M Lib/test/test_asyncio/test_subprocess.py
M Lib/test/test_asyncio/test_unix_events.py
M Lib/test/test_asyncio/utils.py
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
index ff2df653e41b89..c22d0777b0b3be 100644
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -28,9 +28,6 @@
__all__ = (
'SelectorEventLoop',
-'AbstractChildWatcher',
-'PidfdChildWatcher',
-'ThreadedChildWatcher',
'DefaultEventLoopPolicy',
'EventLoop',
)
@@ -65,6 +62,10 @@ def __init__(self, selector=None):
super().__init__(selector)
self._signal_handlers = {}
self._unix_server_sockets = {}
+if can_use_pidfd():
+self._watcher = _PidfdChildWatcher()
+else:
+self._watcher = _ThreadedChildWatcher()
def close(self):
super().close()
@@ -197,33 +198,22 @@ def _make_write_pipe_transport(self, pipe, protocol,
waiter=None,
async def _make_subprocess_transport(self, protocol, args, shell,
stdin, stdout, stderr, bufsize,
extra=None, **kwargs):
-with warnings.catch_warnings():
-warnings.simplefilter('ignore', DeprecationWarning)
-watcher = events.get_event_loop_policy()._watcher
-
-with watcher:
-if not watcher.is_active():
-# Check early.
-# Raising exception before process creation
-# prevents subprocess execution if the watcher
-# is not ready to handle it.
-raise RuntimeError("asyncio.get_child_watcher() is not
activated, "
-"subprocess support is not installed.")
-waiter = self.create_future()
-transp = _UnixSubprocessTransport(self, protocol, args, shell,
-stdin, stdout, stderr, bufsize,
-waiter=waiter, extra=extra,
-**kwargs)
-watcher.add_child_handler(transp.get_pid(),
-self._child_watcher_callback, transp)
-try:
-await waiter
-except (SystemExit, KeyboardInterrupt):
-raise
-except BaseException:
-transp.close()
-await transp._wait()
-raise
+watcher = self._watcher
+waiter = self.create_future()
+transp = _UnixSubprocessTransport(self, protocol, args, shell,
+stdin, stdout, stderr, bufsize,
+waiter=waiter, extra=extra,
+**kwargs)
+watcher.add_child_handler(transp.get_pid(),
+self._child_watcher_callback, transp)
+try:
+await waiter
+except (SystemExit, KeyboardInterrupt):
+raise
+except BaseException:
+transp.close()
+await transp._wait()
+raise
return transp
@@ -865,93 +855,7 @@ def _start(self, args, shell, stdin, stdout, stderr,
bufsize, **kwargs):
stdin_w.close()
-class AbstractChildWatcher:
-"""Abstract base class for monitoring child processes.
-
-Objects derived from this class monitor a collection of subprocesses and
-report their termination or interruption by a signal.
-
-New callbacks are registered with .add_child_handler(). Starting a new
-process must be done within a 'with' block to allow the watcher to suspend
-its activity until the new process if fully registered (this is needed to
-prevent a race condition in some implementations).
-
-Example:
-with watcher:
-proc = subprocess.Popen("sleep 1")
-watcher.add_child_handler(proc.pid, callback)
-
-Notes:
-Implementations of this class must be thread-safe.
-
-Since child watcher objects may catch the SIGCHLD signal and call
-waitpid(-1), there should be only one active object per process.
-"""
-
-def __init_subclass__(cls) -> None:
-if cls.__module__ != __name__:
-warnings._deprecated("AbstractChildWatcher",
- "{name!r} is deprecated as of Pytho
[Python-checkins] GH-120804: add docs for removal for asyncio child watchers (#120895)
https://github.com/python/cpython/commit/18b6ca9660370c7fb5fd50c4036be078c3267f4c commit: 18b6ca9660370c7fb5fd50c4036be078c3267f4c branch: main author: Kumar Aditya committer: kumaraditya303 date: 2024-06-23T13:14:12Z summary: GH-120804: add docs for removal for asyncio child watchers (#120895) Co-authored-by: Alex Waygood files: M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-policy.rst M Doc/library/asyncio-subprocess.rst M Doc/tools/.nitignore M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.14.rst M Doc/whatsnew/3.9.rst diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 19ec726c1be060..a2a3114ad6e4c5 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -77,11 +77,6 @@ Subprocess Support on Windows On Windows, the default event loop :class:`ProactorEventLoop` supports subprocesses, whereas :class:`SelectorEventLoop` does not. -The :meth:`policy.set_child_watcher() -` function is also -not supported, as :class:`ProactorEventLoop` has a different mechanism -to watch child processes. - macOS = diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 346b740a8f757a..837ccc6606786e 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -79,25 +79,6 @@ The abstract event loop policy base class is defined as follows: This method should never return ``None``. - .. method:: get_child_watcher() - - Get a child process watcher object. - - Return a watcher object implementing the - :class:`AbstractChildWatcher` interface. - - This function is Unix specific. - - .. deprecated:: 3.12 - - .. method:: set_child_watcher(watcher) - - Set the current child process watcher to *watcher*. - - This function is Unix specific. - - .. deprecated:: 3.12 - .. _asyncio-policy-builtin: @@ -139,172 +120,6 @@ asyncio ships with the following built-in policies: .. availability:: Windows. -.. _asyncio-watchers: - -Process Watchers - - -A process watcher allows customization of how an event loop monitors -child processes on Unix. Specifically, the event loop needs to know -when a child process has exited. - -In asyncio, child processes are created with -:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec` -functions. - -asyncio defines the :class:`AbstractChildWatcher` abstract base class, which child -watchers should implement, and has four different implementations: -:class:`ThreadedChildWatcher` (configured to be used by default), -:class:`MultiLoopChildWatcher`, :class:`SafeChildWatcher`, and -:class:`FastChildWatcher`. - -See also the :ref:`Subprocess and Threads ` -section. - -The following two functions can be used to customize the child process watcher -implementation used by the asyncio event loop: - -.. function:: get_child_watcher() - - Return the current child watcher for the current policy. - - .. deprecated:: 3.12 - -.. function:: set_child_watcher(watcher) - - Set the current child watcher to *watcher* for the current - policy. *watcher* must implement methods defined in the - :class:`AbstractChildWatcher` base class. - - .. deprecated:: 3.12 - -.. note:: - Third-party event loops implementations might not support - custom child watchers. For such event loops, using - :func:`set_child_watcher` might be prohibited or have no effect. - -.. class:: AbstractChildWatcher - - .. method:: add_child_handler(pid, callback, *args) - - Register a new child handler. - - Arrange for ``callback(pid, returncode, *args)`` to be called - when a process with PID equal to *pid* terminates. Specifying - another callback for the same process replaces the previous - handler. - - The *callback* callable must be thread-safe. - - .. method:: remove_child_handler(pid) - - Removes the handler for process with PID equal to *pid*. - - The function returns ``True`` if the handler was successfully - removed, ``False`` if there was nothing to remove. - - .. method:: attach_loop(loop) - - Attach the watcher to an event loop. - - If the watcher was previously attached to an event loop, then - it is first detached before attaching to the new loop. - - Note: loop may be ``None``. - - .. method:: is_active() - - Return ``True`` if the watcher is ready to use. - - Spawning a subprocess with *inactive* current child watcher raises - :exc:`RuntimeError`. - - .. versionadded:: 3.8 - - .. method:: close() - - Close the watcher. - - This method has to be called to ensure that underlying - resources are cleaned-up. - - .. deprecated:: 3.12 - - -.. class:: ThreadedChildWatcher - - This implementation starts a new waiting thread for every subprocess spawn. - - It works reliably even when the asyncio event loop is run in a non-main OS thread. - - There is no noticeable overhead when
[Python-checkins] [3.12] gh-119960: Add information about regex flags in re module functions (GH-119978) (#120908)
https://github.com/python/cpython/commit/32d43e5fa25c6d4a82bfc635e4704dda6cc34afc commit: 32d43e5fa25c6d4a82bfc635e4704dda6cc34afc branch: 3.12 author: Bénédikt Tran <[email protected]> committer: kumaraditya303 date: 2024-06-23T21:32:24+05:30 summary: [3.12] gh-119960: Add information about regex flags in re module functions (GH-119978) (#120908) gh-119960: Add information about regex flags in re module functions (#119978) (cherry picked from commit a86e6255c371e14cab8680dee979a7393b339ce5) Co-authored-by: Awbert <[email protected]> files: M Doc/library/re.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 00eafda664af89..813afcc483a28e 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -911,6 +911,10 @@ Functions ``None`` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: match(pattern, string, flags=0) @@ -925,6 +929,10 @@ Functions If you want to locate a match anywhere in *string*, use :func:`search` instead (see also :ref:`search-vs-match`). + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: fullmatch(pattern, string, flags=0) @@ -932,6 +940,10 @@ Functions corresponding :class:`~re.Match`. Return ``None`` if the string does not match the pattern; note that this is different from a zero-length match. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionadded:: 3.4 @@ -974,6 +986,10 @@ Functions >>> re.split(r'(\W*)', '...words...') ['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', ''] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -999,6 +1015,10 @@ Functions >>> re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10') [('width', '20'), ('height', '10')] + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1010,6 +1030,10 @@ Functions is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.7 Non-empty matches can now start just after a previous empty match. @@ -1065,6 +1089,10 @@ Functions character ``'0'``. The backreference ``\g<0>`` substitutes in the entire substring matched by the RE. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. versionchanged:: 3.1 Added the optional flags argument. @@ -1100,6 +1128,10 @@ Functions .. versionchanged:: 3.5 Unmatched groups are replaced with an empty string. + The expression's behaviour can be modified by specifying a *flags* value. + Values can be any of the `flags`_ variables, combined using bitwise OR + (the ``|`` operator). + .. function:: escape(pattern) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (#112191)
https://github.com/python/cpython/commit/fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a commit: fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a branch: main author: Alek Kowalczyk committer: kumaraditya303 date: 2024-06-24T08:35:02Z summary: gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (#112191) Co-authored-by: Guido van Rossum Co-authored-by: Kumar Aditya Co-authored-by: Carol Willing files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 374e789e91e790..1d79f78e8e1b67 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1155,6 +1155,14 @@ DNS Asynchronous version of :meth:`socket.getnameinfo`. +.. note:: + Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous + versions through the loop's default thread pool executor. + When this executor is saturated, these methods may experience delays, + which higher-level networking libraries may report as increased timeouts. + To mitigate this, consider using a custom executor for other user tasks, + or setting a default executor with a larger number of workers. + .. versionchanged:: 3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented to return a coroutine, but prior to Python 3.7 they were, in fact, ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120936)
https://github.com/python/cpython/commit/3afb856dda9fed82e3d2f75fc54d063d26a41160 commit: 3afb856dda9fed82e3d2f75fc54d063d26a41160 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-24T08:45:46Z summary: [3.12] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120936) gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (cherry picked from commit fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a) Co-authored-by: Alek Kowalczyk Co-authored-by: Guido van Rossum Co-authored-by: Kumar Aditya Co-authored-by: Carol Willing files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index ba0ee1b6c2c528..6081a052e6e9da 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1139,6 +1139,14 @@ DNS Asynchronous version of :meth:`socket.getnameinfo`. +.. note:: + Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous + versions through the loop's default thread pool executor. + When this executor is saturated, these methods may experience delays, + which higher-level networking libraries may report as increased timeouts. + To mitigate this, consider using a custom executor for other user tasks, + or setting a default executor with a larger number of workers. + .. versionchanged:: 3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented to return a coroutine, but prior to Python 3.7 they were, in fact, ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120935)
https://github.com/python/cpython/commit/206028dba986f982a940377ab1cb8b8276301b82 commit: 206028dba986f982a940377ab1cb8b8276301b82 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-24T08:55:28Z summary: [3.13] gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (#120935) gh-112169: Documented getaddrinfo/getnameinfo default loop executor usage and implications. (GH-112191) (cherry picked from commit fc297b4ba4c61febeb2d8f5d718f2955c6bbea0a) Co-authored-by: Alek Kowalczyk Co-authored-by: Guido van Rossum Co-authored-by: Kumar Aditya Co-authored-by: Carol Willing files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 374e789e91e790..1d79f78e8e1b67 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1155,6 +1155,14 @@ DNS Asynchronous version of :meth:`socket.getnameinfo`. +.. note:: + Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous + versions through the loop's default thread pool executor. + When this executor is saturated, these methods may experience delays, + which higher-level networking libraries may report as increased timeouts. + To mitigate this, consider using a custom executor for other user tasks, + or setting a default executor with a larger number of workers. + .. versionchanged:: 3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented to return a coroutine, but prior to Python 3.7 they were, in fact, ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-107803: fix thread safety issue in double linked list implementation (#121007)
https://github.com/python/cpython/commit/82235449b85165add62c1b200299456a50a1d097
commit: 82235449b85165add62c1b200299456a50a1d097
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-06-26T05:11:32Z
summary:
gh-107803: fix thread safety issue in double linked list implementation
(#121007)
files:
M Include/internal/pycore_object.h
M Modules/_asynciomodule.c
diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h
index 311f295685f361..ebb0f30b5f7377 100644
--- a/Include/internal/pycore_object.h
+++ b/Include/internal/pycore_object.h
@@ -126,8 +126,8 @@ static inline void _Py_RefcntAdd(PyObject* op, Py_ssize_t n)
}
#define _Py_RefcntAdd(op, n) _Py_RefcntAdd(_PyObject_CAST(op), n)
-extern void _Py_SetImmortal(PyObject *op);
-extern void _Py_SetImmortalUntracked(PyObject *op);
+PyAPI_FUNC(void) _Py_SetImmortal(PyObject *op);
+PyAPI_FUNC(void) _Py_SetImmortalUntracked(PyObject *op);
// Makes an immortal object mortal again with the specified refcnt. Should only
// be used during runtime finalization.
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 4672fc7f60efd6..87ad236cdbb39f 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -6,6 +6,7 @@
#include "pycore_dict.h" // _PyDict_GetItem_KnownHash()
#include "pycore_modsupport.h"// _PyArg_CheckPositional()
#include "pycore_moduleobject.h" // _PyModule_GetState()
+#include "pycore_object.h"// _Py_SetImmortalUntracked
#include "pycore_pyerrors.h" // _PyErr_ClearExcState()
#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing()
#include "pycore_pystate.h" // _PyThreadState_GET()
@@ -3706,6 +3707,7 @@ _asyncio_all_tasks_impl(PyObject *module, PyObject *loop)
}
Py_DECREF(eager_iter);
TaskObj *head = state->asyncio_tasks.head;
+Py_INCREF(head);
assert(head != NULL);
assert(head->prev == NULL);
TaskObj *tail = &state->asyncio_tasks.tail;
@@ -3714,10 +3716,11 @@ _asyncio_all_tasks_impl(PyObject *module, PyObject
*loop)
if (add_one_task(state, tasks, (PyObject *)head, loop) < 0) {
Py_DECREF(tasks);
Py_DECREF(loop);
+Py_DECREF(head);
return NULL;
}
-head = head->next;
-assert(head != NULL);
+Py_INCREF(head->next);
+Py_SETREF(head, head->next);
}
PyObject *scheduled_iter = PyObject_GetIter(state->non_asyncio_tasks);
if (scheduled_iter == NULL) {
@@ -3944,6 +3947,8 @@ static int
module_exec(PyObject *mod)
{
asyncio_state *state = get_asyncio_state(mod);
+Py_SET_TYPE(&state->asyncio_tasks.tail, state->TaskType);
+_Py_SetImmortalUntracked((PyObject *)&state->asyncio_tasks.tail);
state->asyncio_tasks.head = &state->asyncio_tasks.tail;
#define CREATE_TYPE(m, tp, spec, base) \
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.13] gh-121025: Improve partialmethod.__repr__ (GH-121033) (#121037)
https://github.com/python/cpython/commit/84634254fef19ab31439e88ec0213acb46bd7b1f commit: 84634254fef19ab31439e88ec0213acb46bd7b1f branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-26T09:49:42Z summary: [3.13] gh-121025: Improve partialmethod.__repr__ (GH-121033) (#121037) gh-121025: Improve partialmethod.__repr__ (GH-121033) It no longer contains redundant commas and spaces. (cherry picked from commit d2646e3f45e3e4e831ee2ae84d55b161a361d592) Co-authored-by: Bénédikt Tran <[email protected]> files: A Misc/NEWS.d/next/Library/2024-06-26-10-13-40.gh-issue-121025.M-XXlV.rst M Lib/functools.py M Lib/test/test_asyncio/test_events.py M Lib/test/test_functools.py diff --git a/Lib/functools.py b/Lib/functools.py index a80e1a6c6a56ac..3d0fd6671fb63e 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -373,15 +373,13 @@ def __init__(self, func, /, *args, **keywords): self.keywords = keywords def __repr__(self): -args = ", ".join(map(repr, self.args)) -keywords = ", ".join("{}={!r}".format(k, v) - for k, v in self.keywords.items()) -format_string = "{module}.{cls}({func}, {args}, {keywords})" -return format_string.format(module=self.__class__.__module__, -cls=self.__class__.__qualname__, -func=self.func, -args=args, -keywords=keywords) +cls = type(self) +module = cls.__module__ +qualname = cls.__qualname__ +args = [repr(self.func)] +args.extend(map(repr, self.args)) +args.extend(f"{k}={v!r}" for k, v in self.keywords.items()) +return f"{module}.{qualname}({', '.join(args)})" def _make_unbound_method(self): def _method(cls_or_self, /, *args, **keywords): diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index 88c85a36b5d448..c92bc81618c989 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -2364,7 +2364,7 @@ def test_handle_repr(self): h = asyncio.Handle(cb, (), self.loop) cb_regex = r'' -cb_regex = fr'functools.partialmethod\({cb_regex}, , \)\(\)' +cb_regex = fr'functools.partialmethod\({cb_regex}\)\(\)' regex = fr'^$' self.assertRegex(repr(h), regex) diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index 26701ea8b4daf9..559213fef1313d 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -569,6 +569,14 @@ class B: method = functools.partialmethod(func=capture, a=1) def test_repr(self): +self.assertEqual(repr(vars(self.A)['nothing']), + 'functools.partialmethod({})'.format(capture)) +self.assertEqual(repr(vars(self.A)['positional']), + 'functools.partialmethod({}, 1)'.format(capture)) +self.assertEqual(repr(vars(self.A)['keywords']), + 'functools.partialmethod({}, a=2)'.format(capture)) +self.assertEqual(repr(vars(self.A)['spec_keywords']), + 'functools.partialmethod({}, self=1, func=2)'.format(capture)) self.assertEqual(repr(vars(self.A)['both']), 'functools.partialmethod({}, 3, b=4)'.format(capture)) diff --git a/Misc/NEWS.d/next/Library/2024-06-26-10-13-40.gh-issue-121025.M-XXlV.rst b/Misc/NEWS.d/next/Library/2024-06-26-10-13-40.gh-issue-121025.M-XXlV.rst new file mode 100644 index 00..38cad610396787 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-26-10-13-40.gh-issue-121025.M-XXlV.rst @@ -0,0 +1,2 @@ +Improve the :meth:`~object.__repr__` of :class:`functools.partialmethod`. +Patch by Bénédikt Tran. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-107803: add whatsnew for asyncio double linked list implementation (#120995)
https://github.com/python/cpython/commit/58a3580836eca58c4a0c02cedc8a8d6080b8ab59 commit: 58a3580836eca58c4a0c02cedc8a8d6080b8ab59 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2024-06-28T14:33:31+05:30 summary: gh-107803: add whatsnew for asyncio double linked list implementation (#120995) files: M Doc/whatsnew/3.14.rst diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 9662044915b8ca..ee3001661b3143 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -125,6 +125,14 @@ symtable Optimizations = +asyncio +--- + +* :mod:`asyncio` now uses double linked list implementation for native tasks + which speeds up execution by 10% on standard pyperformance benchmarks and + reduces memory usage. + (Contributed by Kumar Aditya in :gh:`107803`.) + ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120804: remove `is_active` method from internal child watchers implementation in asyncio (#121124)
https://github.com/python/cpython/commit/ef3c400434eab53e358ebfa5da0411fab722cafb
commit: ef3c400434eab53e358ebfa5da0411fab722cafb
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-06-28T17:23:56+05:30
summary:
gh-120804: remove `is_active` method from internal child watchers
implementation in asyncio (#121124)
files:
M Lib/asyncio/unix_events.py
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
index c22d0777b0b3be..2796e397c0e84d 100644
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -867,9 +867,6 @@ class _PidfdChildWatcher:
recent (5.3+) kernels.
"""
-def is_active(self):
-return True
-
def add_child_handler(self, pid, callback, *args):
loop = events.get_running_loop()
pidfd = os.pidfd_open(pid)
@@ -911,9 +908,6 @@ def __init__(self):
self._pid_counter = itertools.count(0)
self._threads = {}
-def is_active(self):
-return True
-
def __del__(self, _warn=warnings.warn):
threads = [thread for thread in list(self._threads.values())
if thread.is_alive()]
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-121101: Document -Wall option (an alias for -Walways) (#121102)
https://github.com/python/cpython/commit/0a1e8ff9c15675fdc4d07fa6c59f83808bf00798 commit: 0a1e8ff9c15675fdc4d07fa6c59f83808bf00798 branch: main author: Wim Jeantine-Glenn committer: kumaraditya303 date: 2024-06-29T12:10:13+05:30 summary: gh-121101: Document -Wall option (an alias for -Walways) (#121102) files: M Doc/using/cmdline.rst M Misc/python.man diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 7b2019625eb27a..a575760c963327 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -447,6 +447,7 @@ Miscellaneous options -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn @@ -915,6 +916,7 @@ conflict. PYTHONWARNINGS=default # Warn once per call location PYTHONWARNINGS=error# Convert to exceptions PYTHONWARNINGS=always # Warn every time + PYTHONWARNINGS=all # Same as PYTHONWARNINGS=always PYTHONWARNINGS=module # Warn once per calling module PYTHONWARNINGS=once # Warn once per Python process PYTHONWARNINGS=ignore # Never warn diff --git a/Misc/python.man b/Misc/python.man index 4c90c0e2a998ba..4076b8d3d1ba30 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -251,6 +251,7 @@ emitted by a process (even those that are otherwise ignored by default): -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] doc: Mention the missing reflected special methods for all binary operations (GH-119931) (#120063)
https://github.com/python/cpython/commit/cbbd95328d9ca0fbb42e9dbb0060c4a98184e0fb commit: cbbd95328d9ca0fbb42e9dbb0060c4a98184e0fb branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-29T12:12:35+05:30 summary: [3.13] doc: Mention the missing reflected special methods for all binary operations (GH-119931) (#120063) doc: Mention the missing reflected special methods for all binary operations (GH-119931) (cherry picked from commit bf5e1065f4ec2077c6ca352fc1ad940a76d1f6c9) Co-authored-by: Paulo Freitas files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 00b57effd3e1c0..872773f4d28235 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1211,7 +1211,8 @@ Raising ``0.0`` to a negative power results in a :exc:`ZeroDivisionError`. Raising a negative number to a fractional power results in a :class:`complex` number. (In earlier versions it raised a :exc:`ValueError`.) -This operation can be customized using the special :meth:`~object.__pow__` method. +This operation can be customized using the special :meth:`~object.__pow__` and +:meth:`~object.__rpow__` methods. .. _unary: @@ -1299,6 +1300,9 @@ This operation can be customized using the special :meth:`~object.__mul__` and The ``@`` (at) operator is intended to be used for matrix multiplication. No builtin Python types implement this operator. +This operation can be customized using the special :meth:`~object.__matmul__` and +:meth:`~object.__rmatmul__` methods. + .. versionadded:: 3.5 .. index:: @@ -1314,8 +1318,10 @@ integer; the result is that of mathematical division with the 'floor' function applied to the result. Division by zero raises the :exc:`ZeroDivisionError` exception. -This operation can be customized using the special :meth:`~object.__truediv__` and -:meth:`~object.__floordiv__` methods. +The division operation can be customized using the special :meth:`~object.__truediv__` +and :meth:`~object.__rtruediv__` methods. +The floor division operation can be customized using the special +:meth:`~object.__floordiv__` and :meth:`~object.__rfloordiv__` methods. .. index:: single: modulo @@ -1340,7 +1346,8 @@ also overloaded by string objects to perform old-style string formatting (also known as interpolation). The syntax for string formatting is described in the Python Library Reference, section :ref:`old-string-formatting`. -The *modulo* operation can be customized using the special :meth:`~object.__mod__` method. +The *modulo* operation can be customized using the special :meth:`~object.__mod__` +and :meth:`~object.__rmod__` methods. The floor division operator, the modulo operator, and the :func:`divmod` function are not defined for complex numbers. Instead, convert to a floating @@ -1367,7 +1374,8 @@ This operation can be customized using the special :meth:`~object.__add__` and The ``-`` (subtraction) operator yields the difference of its arguments. The numeric arguments are first converted to a common type. -This operation can be customized using the special :meth:`~object.__sub__` method. +This operation can be customized using the special :meth:`~object.__sub__` and +:meth:`~object.__rsub__` methods. .. _shifting: @@ -1388,8 +1396,10 @@ The shifting operations have lower priority than the arithmetic operations: These operators accept integers as arguments. They shift the first argument to the left or right by the number of bits given by the second argument. -This operation can be customized using the special :meth:`~object.__lshift__` and -:meth:`~object.__rshift__` methods. +The left shift operation can be customized using the special :meth:`~object.__lshift__` +and :meth:`~object.__rlshift__` methods. +The right shift operation can be customized using the special :meth:`~object.__rshift__` +and :meth:`~object.__rrshift__` methods. .. index:: pair: exception; ValueError ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] doc: Mention the missing reflected special methods for all binary operations (GH-119931) (#120064)
https://github.com/python/cpython/commit/21a953613c17629321fc9e5a455a18af2a33862d commit: 21a953613c17629321fc9e5a455a18af2a33862d branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-29T12:12:46+05:30 summary: [3.12] doc: Mention the missing reflected special methods for all binary operations (GH-119931) (#120064) doc: Mention the missing reflected special methods for all binary operations (GH-119931) (cherry picked from commit bf5e1065f4ec2077c6ca352fc1ad940a76d1f6c9) Co-authored-by: Paulo Freitas files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 38f00ae2aa03c9..7c1dc6c0dc75ee 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1204,7 +1204,8 @@ Raising ``0.0`` to a negative power results in a :exc:`ZeroDivisionError`. Raising a negative number to a fractional power results in a :class:`complex` number. (In earlier versions it raised a :exc:`ValueError`.) -This operation can be customized using the special :meth:`~object.__pow__` method. +This operation can be customized using the special :meth:`~object.__pow__` and +:meth:`~object.__rpow__` methods. .. _unary: @@ -1292,6 +1293,9 @@ This operation can be customized using the special :meth:`~object.__mul__` and The ``@`` (at) operator is intended to be used for matrix multiplication. No builtin Python types implement this operator. +This operation can be customized using the special :meth:`~object.__matmul__` and +:meth:`~object.__rmatmul__` methods. + .. versionadded:: 3.5 .. index:: @@ -1307,8 +1311,10 @@ integer; the result is that of mathematical division with the 'floor' function applied to the result. Division by zero raises the :exc:`ZeroDivisionError` exception. -This operation can be customized using the special :meth:`~object.__truediv__` and -:meth:`~object.__floordiv__` methods. +The division operation can be customized using the special :meth:`~object.__truediv__` +and :meth:`~object.__rtruediv__` methods. +The floor division operation can be customized using the special +:meth:`~object.__floordiv__` and :meth:`~object.__rfloordiv__` methods. .. index:: single: modulo @@ -1333,7 +1339,8 @@ also overloaded by string objects to perform old-style string formatting (also known as interpolation). The syntax for string formatting is described in the Python Library Reference, section :ref:`old-string-formatting`. -The *modulo* operation can be customized using the special :meth:`~object.__mod__` method. +The *modulo* operation can be customized using the special :meth:`~object.__mod__` +and :meth:`~object.__rmod__` methods. The floor division operator, the modulo operator, and the :func:`divmod` function are not defined for complex numbers. Instead, convert to a floating @@ -1360,7 +1367,8 @@ This operation can be customized using the special :meth:`~object.__add__` and The ``-`` (subtraction) operator yields the difference of its arguments. The numeric arguments are first converted to a common type. -This operation can be customized using the special :meth:`~object.__sub__` method. +This operation can be customized using the special :meth:`~object.__sub__` and +:meth:`~object.__rsub__` methods. .. _shifting: @@ -1381,8 +1389,10 @@ The shifting operations have lower priority than the arithmetic operations: These operators accept integers as arguments. They shift the first argument to the left or right by the number of bits given by the second argument. -This operation can be customized using the special :meth:`~object.__lshift__` and -:meth:`~object.__rshift__` methods. +The left shift operation can be customized using the special :meth:`~object.__lshift__` +and :meth:`~object.__rlshift__` methods. +The right shift operation can be customized using the special :meth:`~object.__rshift__` +and :meth:`~object.__rrshift__` methods. .. index:: pair: exception; ValueError ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (#121146)
https://github.com/python/cpython/commit/d96a52e1366ad5e798839a63e68a0a1b9d728f9c commit: d96a52e1366ad5e798839a63e68a0a1b9d728f9c branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-29T13:57:11+05:30 summary: [3.13] gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (#121146) gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (cherry picked from commit 0a1e8ff9c15675fdc4d07fa6c59f83808bf00798) Co-authored-by: Wim Jeantine-Glenn files: M Doc/using/cmdline.rst M Misc/python.man diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 0620f7d7114ba5..29a96545dcf4cd 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -441,6 +441,7 @@ Miscellaneous options -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn @@ -909,6 +910,7 @@ conflict. PYTHONWARNINGS=default # Warn once per call location PYTHONWARNINGS=error# Convert to exceptions PYTHONWARNINGS=always # Warn every time + PYTHONWARNINGS=all # Same as PYTHONWARNINGS=always PYTHONWARNINGS=module # Warn once per calling module PYTHONWARNINGS=once # Warn once per Python process PYTHONWARNINGS=ignore # Never warn diff --git a/Misc/python.man b/Misc/python.man index 4c90c0e2a998ba..4076b8d3d1ba30 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -251,6 +251,7 @@ emitted by a process (even those that are otherwise ignored by default): -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (#121147)
https://github.com/python/cpython/commit/6a8cb77f49a82f359c5ee5f1f6bab8aba445c562 commit: 6a8cb77f49a82f359c5ee5f1f6bab8aba445c562 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-06-29T13:57:22+05:30 summary: [3.12] gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (#121147) gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (cherry picked from commit 0a1e8ff9c15675fdc4d07fa6c59f83808bf00798) Co-authored-by: Wim Jeantine-Glenn files: M Doc/using/cmdline.rst M Misc/python.man diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 82fbc82b5da8af..4c2798ab72b5f6 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -440,6 +440,7 @@ Miscellaneous options -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn @@ -842,6 +843,7 @@ conflict. PYTHONWARNINGS=default # Warn once per call location PYTHONWARNINGS=error# Convert to exceptions PYTHONWARNINGS=always # Warn every time + PYTHONWARNINGS=all # Same as PYTHONWARNINGS=always PYTHONWARNINGS=module # Warn once per calling module PYTHONWARNINGS=once # Warn once per Python process PYTHONWARNINGS=ignore # Never warn diff --git a/Misc/python.man b/Misc/python.man index 9f89c94adf5028..abb065cd6106ee 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -251,6 +251,7 @@ emitted by a process (even those that are otherwise ignored by default): -Wdefault # Warn once per call location -Werror# Convert to exceptions -Walways # Warn every time + -Wall # Same as -Walways -Wmodule # Warn once per calling module -Wonce # Warn once per Python process -Wignore # Never warn ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-87744: fix waitpid race while calling send_signal in asyncio (#121126)
https://github.com/python/cpython/commit/bd473aa598c5161521a7018896dc124728214a6c commit: bd473aa598c5161521a7018896dc124728214a6c branch: main author: Kumar Aditya committer: kumaraditya303 date: 2024-07-01T10:17:36+05:30 summary: gh-87744: fix waitpid race while calling send_signal in asyncio (#121126) asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether. files: A Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst M Lib/asyncio/base_subprocess.py M Lib/test/test_asyncio/test_subprocess.py diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py index 6dbde2b696ad1f..9c2ba679ce2bf1 100644 --- a/Lib/asyncio/base_subprocess.py +++ b/Lib/asyncio/base_subprocess.py @@ -1,6 +1,9 @@ import collections import subprocess import warnings +import os +import signal +import sys from . import protocols from . import transports @@ -142,17 +145,31 @@ def _check_proc(self): if self._proc is None: raise ProcessLookupError() -def send_signal(self, signal): -self._check_proc() -self._proc.send_signal(signal) +if sys.platform == 'win32': +def send_signal(self, signal): +self._check_proc() +self._proc.send_signal(signal) + +def terminate(self): +self._check_proc() +self._proc.terminate() + +def kill(self): +self._check_proc() +self._proc.kill() +else: +def send_signal(self, signal): +self._check_proc() +try: +os.kill(self._proc.pid, signal) +except ProcessLookupError: +pass -def terminate(self): -self._check_proc() -self._proc.terminate() +def terminate(self): +self.send_signal(signal.SIGTERM) -def kill(self): -self._check_proc() -self._proc.kill() +def kill(self): +self.send_signal(signal.SIGKILL) async def _connect_pipes(self, waiter): try: diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py index 23987c70ca7b63..54501300a29cf7 100644 --- a/Lib/test/test_asyncio/test_subprocess.py +++ b/Lib/test/test_asyncio/test_subprocess.py @@ -864,6 +864,21 @@ async def main(): self.loop.run_until_complete(main()) [email protected](sys.platform != 'linux', "Linux only") +def test_subprocess_send_signal_race(self): +# See https://github.com/python/cpython/issues/87744 +async def main(): +for _ in range(10): +proc = await asyncio.create_subprocess_exec('sleep', '0.1') +await asyncio.sleep(0.1) +try: +proc.send_signal(signal.SIGUSR1) +except ProcessLookupError: +pass +self.assertNotEqual(await proc.wait(), 255) + +self.loop.run_until_complete(main()) + if sys.platform != 'win32': # Unix diff --git a/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst b/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst new file mode 100644 index 00..c0b4f349fb6dac --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst @@ -0,0 +1 @@ +Fix waitpid race while calling :meth:`~asyncio.subprocess.Process.send_signal` in asyncio. Patch by Kumar Aditya. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126) (#121194)
https://github.com/python/cpython/commit/d481d4b7676f2e5d33465ca691346f69af655b24 commit: d481d4b7676f2e5d33465ca691346f69af655b24 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-01T05:11:47Z summary: [3.13] gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126) (#121194) gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126) asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether. (cherry picked from commit bd473aa598c5161521a7018896dc124728214a6c) Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst M Lib/asyncio/base_subprocess.py M Lib/test/test_asyncio/test_subprocess.py diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py index 6dbde2b696ad1f..9c2ba679ce2bf1 100644 --- a/Lib/asyncio/base_subprocess.py +++ b/Lib/asyncio/base_subprocess.py @@ -1,6 +1,9 @@ import collections import subprocess import warnings +import os +import signal +import sys from . import protocols from . import transports @@ -142,17 +145,31 @@ def _check_proc(self): if self._proc is None: raise ProcessLookupError() -def send_signal(self, signal): -self._check_proc() -self._proc.send_signal(signal) +if sys.platform == 'win32': +def send_signal(self, signal): +self._check_proc() +self._proc.send_signal(signal) + +def terminate(self): +self._check_proc() +self._proc.terminate() + +def kill(self): +self._check_proc() +self._proc.kill() +else: +def send_signal(self, signal): +self._check_proc() +try: +os.kill(self._proc.pid, signal) +except ProcessLookupError: +pass -def terminate(self): -self._check_proc() -self._proc.terminate() +def terminate(self): +self.send_signal(signal.SIGTERM) -def kill(self): -self._check_proc() -self._proc.kill() +def kill(self): +self.send_signal(signal.SIGKILL) async def _connect_pipes(self, waiter): try: diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py index cf1a1985338e40..c822d8045b0c0f 100644 --- a/Lib/test/test_asyncio/test_subprocess.py +++ b/Lib/test/test_asyncio/test_subprocess.py @@ -873,6 +873,21 @@ async def main(): self.loop.run_until_complete(main()) [email protected](sys.platform != 'linux', "Linux only") +def test_subprocess_send_signal_race(self): +# See https://github.com/python/cpython/issues/87744 +async def main(): +for _ in range(10): +proc = await asyncio.create_subprocess_exec('sleep', '0.1') +await asyncio.sleep(0.1) +try: +proc.send_signal(signal.SIGUSR1) +except ProcessLookupError: +pass +self.assertNotEqual(await proc.wait(), 255) + +self.loop.run_until_complete(main()) + if sys.platform != 'win32': # Unix diff --git a/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst b/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst new file mode 100644 index 00..c0b4f349fb6dac --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-06-29-05-08-59.gh-issue-87744.rpF6Jw.rst @@ -0,0 +1 @@ +Fix waitpid race while calling :meth:`~asyncio.subprocess.Process.send_signal` in asyncio. Patch by Kumar Aditya. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-114104: clarify asynchronous comprehension docs to match runtime behavior (#121175)
https://github.com/python/cpython/commit/91313afdb392d0d6105e9aaa57b5a50112b613e7 commit: 91313afdb392d0d6105e9aaa57b5a50112b613e7 branch: main author: Danny Yang committer: kumaraditya303 date: 2024-07-01T22:04:39+05:30 summary: gh-114104: clarify asynchronous comprehension docs to match runtime behavior (#121175) files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 872773f4d28235..95ece0e1608dcc 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -218,10 +218,12 @@ A comprehension in an :keyword:`!async def` function may consist of either a :keyword:`!for` or :keyword:`!async for` clause following the leading expression, may contain additional :keyword:`!for` or :keyword:`!async for` clauses, and may also use :keyword:`await` expressions. -If a comprehension contains either :keyword:`!async for` clauses or -:keyword:`!await` expressions or other asynchronous comprehensions it is called -an :dfn:`asynchronous comprehension`. An asynchronous comprehension may -suspend the execution of the coroutine function in which it appears. + +If a comprehension contains :keyword:`!async for` clauses, or if it contains +:keyword:`!await` expressions or other asynchronous comprehensions anywhere except +the iterable expression in the leftmost :keyword:`!for` clause, it is called an +:dfn:`asynchronous comprehension`. An asynchronous comprehension may suspend the +execution of the coroutine function in which it appears. See also :pep:`530`. .. versionadded:: 3.6 ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (#121235)
https://github.com/python/cpython/commit/b93386e1c4dbc871148e081ae147f7d1e8d21444 commit: b93386e1c4dbc871148e081ae147f7d1e8d21444 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-01T16:41:18Z summary: [3.12] gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (#121235) gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (cherry picked from commit 91313afdb392d0d6105e9aaa57b5a50112b613e7) Co-authored-by: Danny Yang files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 7c1dc6c0dc75ee..57a5bd13070940 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -218,10 +218,12 @@ A comprehension in an :keyword:`!async def` function may consist of either a :keyword:`!for` or :keyword:`!async for` clause following the leading expression, may contain additional :keyword:`!for` or :keyword:`!async for` clauses, and may also use :keyword:`await` expressions. -If a comprehension contains either :keyword:`!async for` clauses or -:keyword:`!await` expressions or other asynchronous comprehensions it is called -an :dfn:`asynchronous comprehension`. An asynchronous comprehension may -suspend the execution of the coroutine function in which it appears. + +If a comprehension contains :keyword:`!async for` clauses, or if it contains +:keyword:`!await` expressions or other asynchronous comprehensions anywhere except +the iterable expression in the leftmost :keyword:`!for` clause, it is called an +:dfn:`asynchronous comprehension`. An asynchronous comprehension may suspend the +execution of the coroutine function in which it appears. See also :pep:`530`. .. versionadded:: 3.6 ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (#121234)
https://github.com/python/cpython/commit/931c168219fc030d4b8a6986da4bb2a5a8ef91fa commit: 931c168219fc030d4b8a6986da4bb2a5a8ef91fa branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-01T16:51:43Z summary: [3.13] gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (#121234) gh-114104: clarify asynchronous comprehension docs to match runtime behavior (GH-121175) (cherry picked from commit 91313afdb392d0d6105e9aaa57b5a50112b613e7) Co-authored-by: Danny Yang files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 872773f4d28235..95ece0e1608dcc 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -218,10 +218,12 @@ A comprehension in an :keyword:`!async def` function may consist of either a :keyword:`!for` or :keyword:`!async for` clause following the leading expression, may contain additional :keyword:`!for` or :keyword:`!async for` clauses, and may also use :keyword:`await` expressions. -If a comprehension contains either :keyword:`!async for` clauses or -:keyword:`!await` expressions or other asynchronous comprehensions it is called -an :dfn:`asynchronous comprehension`. An asynchronous comprehension may -suspend the execution of the coroutine function in which it appears. + +If a comprehension contains :keyword:`!async for` clauses, or if it contains +:keyword:`!await` expressions or other asynchronous comprehensions anywhere except +the iterable expression in the leftmost :keyword:`!for` clause, it is called an +:dfn:`asynchronous comprehension`. An asynchronous comprehension may suspend the +execution of the coroutine function in which it appears. See also :pep:`530`. .. versionadded:: 3.6 ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-121210: handle nodes with missing attributes/fields in `ast.compare` (#121211)
https://github.com/python/cpython/commit/15232a0819a2f7e0f448f28f2e6081912d10e7cb commit: 15232a0819a2f7e0f448f28f2e6081912d10e7cb branch: main author: Bénédikt Tran <[email protected]> committer: kumaraditya303 date: 2024-07-02T16:23:17+05:30 summary: gh-121210: handle nodes with missing attributes/fields in `ast.compare` (#121211) files: A Misc/NEWS.d/next/Library/2024-07-01-11-23-18.gh-issue-121210.cD0zfn.rst M Lib/ast.py M Lib/test/test_ast.py diff --git a/Lib/ast.py b/Lib/ast.py index fb4d21b87d8bd0..a954d4a97d3c22 100644 --- a/Lib/ast.py +++ b/Lib/ast.py @@ -422,6 +422,8 @@ def compare( might differ in whitespace or similar details. """ +sentinel = object() # handle the possibility of a missing attribute/field + def _compare(a, b): # Compare two fields on an AST object, which may themselves be # AST objects, lists of AST objects, or primitive ASDL types @@ -449,8 +451,14 @@ def _compare_fields(a, b): if a._fields != b._fields: return False for field in a._fields: -a_field = getattr(a, field) -b_field = getattr(b, field) +a_field = getattr(a, field, sentinel) +b_field = getattr(b, field, sentinel) +if a_field is sentinel and b_field is sentinel: +# both nodes are missing a field at runtime +continue +if a_field is sentinel or b_field is sentinel: +# one of the node is missing a field +return False if not _compare(a_field, b_field): return False else: @@ -461,8 +469,11 @@ def _compare_attributes(a, b): return False # Attributes are always ints. for attr in a._attributes: -a_attr = getattr(a, attr) -b_attr = getattr(b, attr) +a_attr = getattr(a, attr, sentinel) +b_attr = getattr(b, attr, sentinel) +if a_attr is sentinel and b_attr is sentinel: +# both nodes are missing an attribute at runtime +continue if a_attr != b_attr: return False else: diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index 5d71d524516df2..fbd19620311159 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -948,6 +948,15 @@ def test_compare_fieldless(self): self.assertTrue(ast.compare(ast.Add(), ast.Add())) self.assertFalse(ast.compare(ast.Sub(), ast.Add())) +# test that missing runtime fields is handled in ast.compare() +a1, a2 = ast.Name('a'), ast.Name('a') +self.assertTrue(ast.compare(a1, a2)) +self.assertTrue(ast.compare(a1, a2)) +del a1.id +self.assertFalse(ast.compare(a1, a2)) +del a2.id +self.assertTrue(ast.compare(a1, a2)) + def test_compare_modes(self): for mode, sources in ( ("exec", exec_tests), @@ -970,6 +979,16 @@ def parse(a, b): self.assertTrue(ast.compare(a, b, compare_attributes=False)) self.assertFalse(ast.compare(a, b, compare_attributes=True)) +def test_compare_attributes_option_missing_attribute(self): +# test that missing runtime attributes is handled in ast.compare() +a1, a2 = ast.Name('a', lineno=1), ast.Name('a', lineno=1) +self.assertTrue(ast.compare(a1, a2)) +self.assertTrue(ast.compare(a1, a2, compare_attributes=True)) +del a1.lineno +self.assertFalse(ast.compare(a1, a2, compare_attributes=True)) +del a2.lineno +self.assertTrue(ast.compare(a1, a2, compare_attributes=True)) + def test_positional_only_feature_version(self): ast.parse('def foo(x, /): ...', feature_version=(3, 8)) ast.parse('def bar(x=1, /): ...', feature_version=(3, 8)) diff --git a/Misc/NEWS.d/next/Library/2024-07-01-11-23-18.gh-issue-121210.cD0zfn.rst b/Misc/NEWS.d/next/Library/2024-07-01-11-23-18.gh-issue-121210.cD0zfn.rst new file mode 100644 index 00..55d5b221bf0765 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-01-11-23-18.gh-issue-121210.cD0zfn.rst @@ -0,0 +1,2 @@ +Handle AST nodes with missing runtime fields or attributes in +:func:`ast.compare`. Patch by Bénédikt Tran. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-121165: protect macro expansion of `ADJUST_INDICES` with do-while(0) (#121166)
https://github.com/python/cpython/commit/6343486eb60ac5a9e15402a592298259c5afdee1 commit: 6343486eb60ac5a9e15402a592298259c5afdee1 branch: main author: Bénédikt Tran <[email protected]> committer: kumaraditya303 date: 2024-07-02T16:27:51+05:30 summary: gh-121165: protect macro expansion of `ADJUST_INDICES` with do-while(0) (#121166) files: M Objects/bytes_methods.c M Objects/unicodeobject.c diff --git a/Objects/bytes_methods.c b/Objects/bytes_methods.c index 55252406578774..c239ae18a593e3 100644 --- a/Objects/bytes_methods.c +++ b/Objects/bytes_methods.c @@ -432,19 +432,24 @@ parse_args_finds_byte(const char *function_name, PyObject **subobj, char *byte) } /* helper macro to fixup start/end slice values */ -#define ADJUST_INDICES(start, end, len) \ -if (end > len) \ -end = len; \ -else if (end < 0) { \ -end += len; \ -if (end < 0)\ -end = 0;\ -} \ -if (start < 0) {\ -start += len; \ -if (start < 0) \ -start = 0; \ -} +#define ADJUST_INDICES(start, end, len) \ +do {\ +if (end > len) {\ +end = len; \ +} \ +else if (end < 0) { \ +end += len; \ +if (end < 0) { \ +end = 0;\ +} \ +} \ +if (start < 0) {\ +start += len; \ +if (start < 0) {\ +start = 0; \ +} \ +} \ +} while (0) Py_LOCAL_INLINE(Py_ssize_t) find_internal(const char *str, Py_ssize_t len, diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 9738442ab962b0..394ea888fc9231 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -9315,19 +9315,24 @@ _PyUnicode_TransformDecimalAndSpaceToASCII(PyObject *unicode) /* --- Helpers */ /* helper macro to fixup start/end slice values */ -#define ADJUST_INDICES(start, end, len) \ -if (end > len) \ -end = len; \ -else if (end < 0) { \ -end += len; \ -if (end < 0)\ -end = 0;\ -} \ -if (start < 0) {\ -start += len; \ -if (start < 0) \ -start = 0; \ -} +#define ADJUST_INDICES(start, end, len) \ +do {\ +if (end > len) {\ +end = len; \ +} \ +else if (end < 0) { \ +end += len; \ +if (end < 0) { \ +end = 0;\ +} \ +} \ +if (start < 0) {\ +start += len; \ +if (start < 0) {\ +start = 0; \ +} \ +} \ +} while (0) static Py_ssize_t any_find_slice(PyObject* s1, PyObject* s2, ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (#121260)
https://github.com/python/cpython/commit/705a123898f1394b62076c00ab6008c18fd8e115 commit: 705a123898f1394b62076c00ab6008c18fd8e115 branch: main author: AN Long committer: kumaraditya303 date: 2024-07-03T13:05:05+05:30 summary: gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (#121260) files: M Modules/rotatingtree.c diff --git a/Modules/rotatingtree.c b/Modules/rotatingtree.c index 217e495b3d2a9d..5910e25bed6389 100644 --- a/Modules/rotatingtree.c +++ b/Modules/rotatingtree.c @@ -1,9 +1,4 @@ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif - #include "Python.h" -#include "pycore_lock.h" #include "rotatingtree.h" #define KEY_LOWER_THAN(key1, key2) ((char*)(key1) < (char*)(key2)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-111872: Document the max_children attribute for `socketserver.ForkingMixIn` (#118134)
https://github.com/python/cpython/commit/ff5751a208e05f9d054b6df44f7651b64d415908 commit: ff5751a208e05f9d054b6df44f7651b64d415908 branch: main author: AN Long committer: kumaraditya303 date: 2024-07-03T13:16:57+05:30 summary: gh-111872: Document the max_children attribute for `socketserver.ForkingMixIn` (#118134) files: M Doc/library/socketserver.rst diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst index f1f87ea975ca42..69f06e6cf4d923 100644 --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -126,6 +126,12 @@ server is the address family. waits until all non-daemon threads complete, except if :attr:`block_on_close` attribute is ``False``. + .. attribute:: max_children + + Specify how many child processes will exist to handle requests at a time + for :class:`ForkingMixIn`. If the limit is reached, + new requests will wait until one child process has finished. + .. attribute:: daemon_threads For :class:`ThreadingMixIn` use daemonic threads by setting ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] updated tp_flags initialization to use inplace or (#120625)
https://github.com/python/cpython/commit/f65d17bf471ae5932ebd8baacedca83bfc85bdb1 commit: f65d17bf471ae5932ebd8baacedca83bfc85bdb1 branch: main author: byundojin <[email protected]> committer: kumaraditya303 date: 2024-07-03T13:21:25+05:30 summary: updated tp_flags initialization to use inplace or (#120625) files: M Objects/typeobject.c diff --git a/Objects/typeobject.c b/Objects/typeobject.c index b042e64a188d9d..447e561c0d4440 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -8395,7 +8395,7 @@ type_ready(PyTypeObject *type, int initial) } /* All done -- set the ready flag */ -type->tp_flags = type->tp_flags | Py_TPFLAGS_READY; +type->tp_flags |= Py_TPFLAGS_READY; stop_readying(type); assert(_PyType_CheckConsistency(type)); ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] build(deps): bump hypothesis from 6.100.2 to 6.104.2 in /Tools (#121218)
https://github.com/python/cpython/commit/f49c83aa6683915fe6ba0a1177c3d4e873429703 commit: f49c83aa6683915fe6ba0a1177c3d4e873429703 branch: main author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> committer: kumaraditya303 date: 2024-07-03T13:22:59+05:30 summary: build(deps): bump hypothesis from 6.100.2 to 6.104.2 in /Tools (#121218) Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.100.2 to 6.104.2. - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.100.2...hypothesis-python-6.104.2) --- updated-dependencies: - dependency-name: hypothesis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> files: M Tools/requirements-hypothesis.txt diff --git a/Tools/requirements-hypothesis.txt b/Tools/requirements-hypothesis.txt index 9d5a18c881bf36..ab3f39ac6ee087 100644 --- a/Tools/requirements-hypothesis.txt +++ b/Tools/requirements-hypothesis.txt @@ -1,4 +1,4 @@ # Requirements file for hypothesis that # we use to run our property-based tests in CI. -hypothesis==6.100.2 +hypothesis==6.104.2 ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (GH-121260) (#121307)
https://github.com/python/cpython/commit/85971492b7afb0e29a1104dbf8adebe8d0336cf5 commit: 85971492b7afb0e29a1104dbf8adebe8d0336cf5 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-03T07:59:26Z summary: [3.13] gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (GH-121260) (#121307) gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (GH-121260) (cherry picked from commit 705a123898f1394b62076c00ab6008c18fd8e115) Co-authored-by: AN Long files: M Modules/rotatingtree.c diff --git a/Modules/rotatingtree.c b/Modules/rotatingtree.c index 217e495b3d2a9d..5910e25bed6389 100644 --- a/Modules/rotatingtree.c +++ b/Modules/rotatingtree.c @@ -1,9 +1,4 @@ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif - #include "Python.h" -#include "pycore_lock.h" #include "rotatingtree.h" #define KEY_LOWER_THAN(key1, key2) ((char*)(key1) < (char*)(key2)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (#112237)
https://github.com/python/cpython/commit/4232976b02cb999335c6bfdec3315520b21954f2 commit: 4232976b02cb999335c6bfdec3315520b21954f2 branch: main author: da-woods committer: kumaraditya303 date: 2024-07-03T13:35:02+05:30 summary: docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (#112237) files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index c9ef076c78c66a..0091e084308245 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1592,7 +1592,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) weak references to the type object itself. It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and - :c:member:`~PyTypeObject.tp_weaklist`. + :c:member:`~PyTypeObject.tp_weaklistoffset`. **Inheritance:** @@ -1604,7 +1604,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the - :c:member:`~PyTypeObject.tp_dict` field, then + :c:member:`~PyTypeObject.tp_flags` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] Docs: Add `os.splice` flags argument (#109847)
https://github.com/python/cpython/commit/9d3c9b822ce3c52cd747efe93b172f02c0d09289 commit: 9d3c9b822ce3c52cd747efe93b172f02c0d09289 branch: main author: Amin Alaee committer: kumaraditya303 date: 2024-07-03T13:40:57+05:30 summary: Docs: Add `os.splice` flags argument (#109847) Co-authored-by: Adam Turner <[email protected]> Co-authored-by: Blaise Pabon files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 8d95d01fe55ed9..2878d425310d75 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1724,10 +1724,27 @@ or `the MSDN <https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windo Added support for pipes on Windows. -.. function:: splice(src, dst, count, offset_src=None, offset_dst=None) +.. function:: splice(src, dst, count, offset_src=None, offset_dst=None, flags=0) Transfer *count* bytes from file descriptor *src*, starting from offset *offset_src*, to file descriptor *dst*, starting from offset *offset_dst*. + + The splicing behaviour can be modified by specifying a *flags* value. + Any of the following variables may used, combined using bitwise OR + (the ``|`` operator): + + * If :const:`SPLICE_F_MOVE` is specified, + the kernel is asked to move pages instead of copying, + but pages may still be copied if the kernel cannot move the pages from the pipe. + + * If :const:`SPLICE_F_NONBLOCK` is specified, + the kernel is asked to not block on I/O. + This makes the splice pipe operations nonblocking, + but splice may nevertheless block because the spliced file descriptors may block. + + * If :const:`SPLICE_F_MORE` is specified, + it hints to the kernel that more data will be coming in a subsequent splice. + At least one of the file descriptors must refer to a pipe. If *offset_src* is ``None``, then *src* is read from the current position; respectively for *offset_dst*. The offset associated to the file descriptor that refers to a @@ -1746,6 +1763,8 @@ or `the MSDN <https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windo make sense to block because there are no writers connected to the write end of the pipe. + .. seealso:: The :manpage:`splice(2)` man page. + .. availability:: Linux >= 2.6.17 with glibc >= 2.5 .. versionadded:: 3.10 ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (#121310)
https://github.com/python/cpython/commit/4ade7723e829af350f07c5b9fd20cfec2712a982 commit: 4ade7723e829af350f07c5b9fd20cfec2712a982 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-03T13:50:02+05:30 summary: [3.12] docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (#121310) docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (cherry picked from commit 4232976b02cb999335c6bfdec3315520b21954f2) Co-authored-by: da-woods files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index f6d865f2f52219..90896572046993 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1584,7 +1584,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) weak references to the type object itself. It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and - :c:member:`~PyTypeObject.tp_weaklist`. + :c:member:`~PyTypeObject.tp_weaklistoffset`. **Inheritance:** @@ -1596,7 +1596,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the - :c:member:`~PyTypeObject.tp_dict` field, then + :c:member:`~PyTypeObject.tp_flags` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-120782: Update internal type cache when reloading datetime (GH-120829) (#120855)
https://github.com/python/cpython/commit/2c3aa527fd37233527f1895bcb6eddbb7c1bab08 commit: 2c3aa527fd37233527f1895bcb6eddbb7c1bab08 branch: 3.13 author: neonene <[email protected]> committer: kumaraditya303 date: 2024-07-03T13:52:51+05:30 summary: [3.13] gh-120782: Update internal type cache when reloading datetime (GH-120829) (#120855) * [3.13] gh-120782: Update internal type cache when reloading datetime When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module. files: A Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst M Lib/test/datetimetester.py M Modules/_datetimemodule.c diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 3f67c6990b652e..00819fda45289f 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -52,25 +52,6 @@ pass # -# This is copied from test_import/__init__.py. -def no_rerun(reason): -"""Skip rerunning for a particular test. - -WARNING: Use this decorator with care; skipping rerunning makes it -impossible to find reference leaks. Provide a clear reason for skipping the -test using the 'reason' parameter. -""" -def deco(func): -_has_run = False -def wrapper(self): -nonlocal _has_run -if _has_run: -self.skipTest(reason) -func(self) -_has_run = True -return wrapper -return deco - pickle_loads = {pickle.loads, pickle._loads} pickle_choices = [(pickle, pickle, proto) @@ -6420,7 +6401,6 @@ class IranTest(ZoneInfoTest): @unittest.skipIf(_testcapi is None, 'need _testcapi module') -@no_rerun("the encapsulated datetime C API does not support reloading") class CapiTest(unittest.TestCase): def setUp(self): # Since the C API is not present in the _Pure tests, skip all tests @@ -6889,6 +6869,38 @@ def pickle_fake_date(datetime_) -> Type[FakeDate]: """) script_helper.assert_python_ok('-c', script) +def test_update_type_cache(self): +# gh-120782 +script = textwrap.dedent(""" +import sys +for i in range(5): +import _datetime +assert _datetime.date.max > _datetime.date.min +assert _datetime.time.max > _datetime.time.min +assert _datetime.datetime.max > _datetime.datetime.min +assert _datetime.timedelta.max > _datetime.timedelta.min +assert _datetime.date.__dict__["min"] is _datetime.date.min +assert _datetime.date.__dict__["max"] is _datetime.date.max +assert _datetime.date.__dict__["resolution"] is _datetime.date.resolution +assert _datetime.time.__dict__["min"] is _datetime.time.min +assert _datetime.time.__dict__["max"] is _datetime.time.max +assert _datetime.time.__dict__["resolution"] is _datetime.time.resolution +assert _datetime.datetime.__dict__["min"] is _datetime.datetime.min +assert _datetime.datetime.__dict__["max"] is _datetime.datetime.max +assert _datetime.datetime.__dict__["resolution"] is _datetime.datetime.resolution +assert _datetime.timedelta.__dict__["min"] is _datetime.timedelta.min +assert _datetime.timedelta.__dict__["max"] is _datetime.timedelta.max +assert _datetime.timedelta.__dict__["resolution"] is _datetime.timedelta.resolution +assert _datetime.timezone.__dict__["min"] is _datetime.timezone.min +assert _datetime.timezone.__dict__["max"] is _datetime.timezone.max +assert _datetime.timezone.__dict__["utc"] is _datetime.timezone.utc +assert isinstance(_datetime.timezone.min, _datetime.tzinfo) +assert isinstance(_datetime.timezone.max, _datetime.tzinfo) +assert isinstance(_datetime.timezone.utc, _datetime.tzinfo) +del sys.modules['_datetime'] +""") +script_helper.assert_python_ok('-c', script) + def load_tests(loader, standard_tests, pattern): standard_tests.addTest(ZoneInfoCompleteTest()) diff --git a/Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst b/Misc/NEWS.d/next/Library/2024-06-21-12-00-16.gh-issue-120782.LOE8tj.rst new file mode 100644 ind
[Python-checkins] [3.13] docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (#121309)
https://github.com/python/cpython/commit/ee1db25bc0ea9309f834b154bdf5bdd81830a8f3 commit: ee1db25bc0ea9309f834b154bdf5bdd81830a8f3 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-03T08:23:51Z summary: [3.13] docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (#121309) docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (GH-112237) (cherry picked from commit 4232976b02cb999335c6bfdec3315520b21954f2) Co-authored-by: da-woods files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index c9ef076c78c66a..0091e084308245 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1592,7 +1592,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) weak references to the type object itself. It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and - :c:member:`~PyTypeObject.tp_weaklist`. + :c:member:`~PyTypeObject.tp_weaklistoffset`. **Inheritance:** @@ -1604,7 +1604,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the - :c:member:`~PyTypeObject.tp_dict` field, then + :c:member:`~PyTypeObject.tp_flags` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-121300: Add `replace` to `copy.__all__` (#121302)
https://github.com/python/cpython/commit/7c66906802cd8534b05264bd47acf9eb9db6d09e
commit: 7c66906802cd8534b05264bd47acf9eb9db6d09e
branch: main
author: Max Muoto
committer: kumaraditya303
date: 2024-07-03T20:33:56+05:30
summary:
gh-121300: Add `replace` to `copy.__all__` (#121302)
files:
M Lib/copy.py
M Lib/test/test_copy.py
diff --git a/Lib/copy.py b/Lib/copy.py
index 7a1907d75494d7..a79976d3a658f0 100644
--- a/Lib/copy.py
+++ b/Lib/copy.py
@@ -4,8 +4,9 @@
import copy
-x = copy.copy(y)# make a shallow copy of y
-x = copy.deepcopy(y)# make a deep copy of y
+x = copy.copy(y)# make a shallow copy of y
+x = copy.deepcopy(y)# make a deep copy of y
+x = copy.replace(y, a=1, b=2) # new object with fields replaced, as
defined by `__replace__`
For module specific errors, copy.Error is raised.
@@ -56,7 +57,7 @@ class Error(Exception):
pass
error = Error # backward compatibility
-__all__ = ["Error", "copy", "deepcopy"]
+__all__ = ["Error", "copy", "deepcopy", "replace"]
def copy(x):
"""Shallow copy operation on arbitrary Python objects.
diff --git a/Lib/test/test_copy.py b/Lib/test/test_copy.py
index 89102373759ca0..3dec64cc9a2414 100644
--- a/Lib/test/test_copy.py
+++ b/Lib/test/test_copy.py
@@ -972,6 +972,10 @@ class C:
copy.replace(c, x=1, error=2)
+class MiscTestCase(unittest.TestCase):
+def test__all__(self):
+support.check__all__(self, copy, not_exported={"dispatch_table",
"error"})
+
def global_foo(x, y): return x+y
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-121333: Clarify what is the default executor for asyncio.run_in_executor (#121335)
https://github.com/python/cpython/commit/facf9862da0cf9331550747197800d682cd371fb commit: facf9862da0cf9331550747197800d682cd371fb branch: main author: AN Long committer: kumaraditya303 date: 2024-07-09T14:52:07+05:30 summary: gh-121333: Clarify what is the default executor for asyncio.run_in_executor (#121335) Co-authored-by: Kumar Aditya files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 1d79f78e8e1b67..70bdd154d6c406 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1262,6 +1262,9 @@ Executing code in thread or process pools The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. + The default executor can be set by :meth:`loop.set_default_executor`, + otherwise, a :class:`concurrent.futures.ThreadPoolExecutor` will be + lazy-initialized and used by :func:`run_in_executor` if needed. Example:: ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (#121525)
https://github.com/python/cpython/commit/f24f31656c320e6cefa093a2230620f6061dd806 commit: f24f31656c320e6cefa093a2230620f6061dd806 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-09T15:06:40+05:30 summary: [3.12] gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (#121525) gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (cherry picked from commit facf9862da0cf9331550747197800d682cd371fb) Co-authored-by: AN Long Co-authored-by: Kumar Aditya files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 6081a052e6e9da..2b6f7df10718a8 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1246,6 +1246,9 @@ Executing code in thread or process pools The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. + The default executor can be set by :meth:`loop.set_default_executor`, + otherwise, a :class:`concurrent.futures.ThreadPoolExecutor` will be + lazy-initialized and used by :func:`run_in_executor` if needed. Example:: ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (#121526)
https://github.com/python/cpython/commit/39f4add7a35cb3367f7d8d3d708c2aefcf0bebed commit: 39f4add7a35cb3367f7d8d3d708c2aefcf0bebed branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-09T15:12:45+05:30 summary: [3.13] gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (#121526) gh-121333: Clarify what is the default executor for asyncio.run_in_executor (GH-121335) (cherry picked from commit facf9862da0cf9331550747197800d682cd371fb) Co-authored-by: AN Long Co-authored-by: Kumar Aditya files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 1d79f78e8e1b67..70bdd154d6c406 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1262,6 +1262,9 @@ Executing code in thread or process pools The *executor* argument should be an :class:`concurrent.futures.Executor` instance. The default executor is used if *executor* is ``None``. + The default executor can be set by :meth:`loop.set_default_executor`, + otherwise, a :class:`concurrent.futures.ThreadPoolExecutor` will be + lazy-initialized and used by :func:`run_in_executor` if needed. Example:: ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-121621: Use PyMutex for writes to asyncio state (#121622)
https://github.com/python/cpython/commit/5d6861ad06b524358f52603f242e7c0d57532a58
commit: 5d6861ad06b524358f52603f242e7c0d57532a58
branch: main
author: Ken Jin
committer: kumaraditya303
date: 2024-07-14T12:22:56Z
summary:
gh-121621: Use PyMutex for writes to asyncio state (#121622)
Co-authored-by: Kumar Aditya
files:
M Modules/_asynciomodule.c
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 6938aad90dd412..d5d28f8ab0c016 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -77,10 +77,21 @@ typedef struct {
#define FI_FREELIST_MAXLEN 255
+#ifdef Py_GIL_DISABLED
+# define ASYNCIO_STATE_LOCK(state) PyMutex_Lock(&state->mutex)
+# define ASYNCIO_STATE_UNLOCK(state) PyMutex_Unlock(&state->mutex)
+#else
+# define ASYNCIO_STATE_LOCK(state) ((void)state)
+# define ASYNCIO_STATE_UNLOCK(state) ((void)state)
+#endif
+
typedef struct futureiterobject futureiterobject;
/* State of the _asyncio module */
typedef struct {
+#ifdef Py_GIL_DISABLED
+PyMutex mutex;
+#endif
PyTypeObject *FutureIterType;
PyTypeObject *TaskStepMethWrapper_Type;
PyTypeObject *FutureType;
@@ -341,6 +352,8 @@ get_running_loop(asyncio_state *state, PyObject **loop)
}
}
+// TODO GH-121621: This should be moved to PyThreadState
+// for easier and quicker access.
state->cached_running_loop = rl;
state->cached_running_loop_tsid = ts_id;
}
@@ -384,6 +397,9 @@ set_running_loop(asyncio_state *state, PyObject *loop)
return -1;
}
+
+// TODO GH-121621: This should be moved to PyThreadState
+// for easier and quicker access.
state->cached_running_loop = loop; // borrowed, kept alive by ts_dict
state->cached_running_loop_tsid = PyThreadState_GetID(tstate);
@@ -1664,6 +1680,7 @@ FutureIter_dealloc(futureiterobject *it)
state = get_asyncio_state(module);
}
+// TODO GH-121621: This should be moved to thread state as well.
if (state && state->fi_freelist_len < FI_FREELIST_MAXLEN) {
state->fi_freelist_len++;
it->future = (FutureObj*) state->fi_freelist;
@@ -2015,10 +2032,12 @@ static PyMethodDef TaskWakeupDef = {
static void
register_task(asyncio_state *state, TaskObj *task)
{
+ASYNCIO_STATE_LOCK(state);
assert(Task_Check(state, task));
assert(task != &state->asyncio_tasks.tail);
if (task->next != NULL) {
// already registered
+ASYNCIO_STATE_UNLOCK(state);
return;
}
assert(task->prev == NULL);
@@ -2027,6 +2046,7 @@ register_task(asyncio_state *state, TaskObj *task)
task->next = state->asyncio_tasks.head;
state->asyncio_tasks.head->prev = task;
state->asyncio_tasks.head = task;
+ASYNCIO_STATE_UNLOCK(state);
}
static int
@@ -2038,12 +2058,14 @@ register_eager_task(asyncio_state *state, PyObject
*task)
static void
unregister_task(asyncio_state *state, TaskObj *task)
{
+ASYNCIO_STATE_LOCK(state);
assert(Task_Check(state, task));
assert(task != &state->asyncio_tasks.tail);
if (task->next == NULL) {
// not registered
assert(task->prev == NULL);
assert(state->asyncio_tasks.head != task);
+ASYNCIO_STATE_UNLOCK(state);
return;
}
task->next->prev = task->prev;
@@ -2056,6 +2078,7 @@ unregister_task(asyncio_state *state, TaskObj *task)
task->next = NULL;
task->prev = NULL;
assert(state->asyncio_tasks.head != task);
+ASYNCIO_STATE_UNLOCK(state);
}
static int
@@ -2210,7 +2233,12 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject
*coro, PyObject *loop,
// optimization: defer task name formatting
// store the task counter as PyLong in the name
// for deferred formatting in get_name
-name = PyLong_FromUnsignedLongLong(++state->task_name_counter);
+#ifdef Py_GIL_DISABLED
+unsigned long long counter =
_Py_atomic_add_uint64(&state->task_name_counter, 1) + 1;
+#else
+unsigned long long counter = ++state->task_name_counter;
+#endif
+name = PyLong_FromUnsignedLongLong(counter);
} else if (!PyUnicode_CheckExact(name)) {
name = PyObject_Str(name);
} else {
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.13] gh-121621: Use PyMutex for writes to asyncio state (GH-121622) (#121774)
https://github.com/python/cpython/commit/a2a4f5ebc5d5351324c9d3c870238d2a2fafdc64
commit: a2a4f5ebc5d5351324c9d3c870238d2a2fafdc64
branch: 3.13
author: Sam Gross
committer: kumaraditya303
date: 2024-07-15T10:44:35+05:30
summary:
[3.13] gh-121621: Use PyMutex for writes to asyncio state (GH-121622) (#121774)
(cherry picked from commit 5d6861ad06b524358f52603f242e7c0d57532a58)
Co-authored-by: Ken Jin
Co-authored-by: Kumar Aditya
files:
M Modules/_asynciomodule.c
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 04b5fc505c074f..92d0e6de0dfb93 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -293,6 +293,8 @@ get_running_loop(asyncio_state *state, PyObject **loop)
}
}
+// TODO GH-121621: This should be moved to PyThreadState
+// for easier and quicker access.
state->cached_running_loop = rl;
state->cached_running_loop_tsid = ts_id;
}
@@ -336,6 +338,9 @@ set_running_loop(asyncio_state *state, PyObject *loop)
return -1;
}
+
+// TODO GH-121621: This should be moved to PyThreadState
+// for easier and quicker access.
state->cached_running_loop = loop; // borrowed, kept alive by ts_dict
state->cached_running_loop_tsid = PyThreadState_GetID(tstate);
@@ -1616,6 +1621,7 @@ FutureIter_dealloc(futureiterobject *it)
state = get_asyncio_state(module);
}
+// TODO GH-121621: This should be moved to thread state as well.
if (state && state->fi_freelist_len < FI_FREELIST_MAXLEN) {
state->fi_freelist_len++;
it->future = (FutureObj*) state->fi_freelist;
@@ -2146,7 +2152,12 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject
*coro, PyObject *loop,
// optimization: defer task name formatting
// store the task counter as PyLong in the name
// for deferred formatting in get_name
-name = PyLong_FromUnsignedLongLong(++state->task_name_counter);
+#ifdef Py_GIL_DISABLED
+unsigned long long counter =
_Py_atomic_add_uint64(&state->task_name_counter, 1) + 1;
+#else
+unsigned long long counter = ++state->task_name_counter;
+#endif
+name = PyLong_FromUnsignedLongLong(counter);
} else if (!PyUnicode_CheckExact(name)) {
name = PyObject_Str(name);
} else {
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] fix outdated comments in asyncio (#121783)
https://github.com/python/cpython/commit/48042c52a6b59089e7d7dda3c8fed79d646b6f8d
commit: 48042c52a6b59089e7d7dda3c8fed79d646b6f8d
branch: main
author: Kumar Aditya
committer: kumaraditya303
date: 2024-07-15T05:59:19Z
summary:
fix outdated comments in asyncio (#121783)
files:
M Lib/asyncio/futures.py
M Lib/asyncio/tasks.py
M Modules/_asynciomodule.c
diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py
index 9c1b5e49e1a70b..5f6fa2348726cf 100644
--- a/Lib/asyncio/futures.py
+++ b/Lib/asyncio/futures.py
@@ -43,7 +43,6 @@ class Future:
- This class is not compatible with the wait() and as_completed()
methods in the concurrent.futures package.
-(In Python 3.4 or later we may be able to unify the implementations.)
"""
# Class variables serving as defaults for instance variables.
@@ -61,7 +60,7 @@ class Future:
# the Future protocol (i.e. is intended to be duck-type compatible).
# The value must also be not-None, to enable a subclass to declare
# that it is not compatible by setting this to None.
-# - It is set by __iter__() below so that Task._step() can tell
+# - It is set by __iter__() below so that Task.__step() can tell
# the difference between
# `await Future()` or`yield from Future()` (correct) vs.
# `yield Future()` (incorrect).
diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py
index cd869931e01409..2f8f4f08a1e111 100644
--- a/Lib/asyncio/tasks.py
+++ b/Lib/asyncio/tasks.py
@@ -281,7 +281,7 @@ def __eager_start(self):
def __step(self, exc=None):
if self.done():
raise exceptions.InvalidStateError(
-f'_step(): already done: {self!r}, {exc!r}')
+f'__step(): already done: {self!r}, {exc!r}')
if self._must_cancel:
if not isinstance(exc, exceptions.CancelledError):
exc = self._make_cancelled_error()
@@ -379,7 +379,7 @@ def __wakeup(self, future):
else:
# Don't pass the value of `future.result()` explicitly,
# as `Future.__iter__` and `Future.__await__` don't need it.
-# If we call `_step(value, None)` instead of `_step()`,
+# If we call `__step(value, None)` instead of `__step()`,
# Python eval loop would use `.send(value)` method call,
# instead of `__next__()`, which is slower for futures
# that return non-generator iterators from their `__iter__`.
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index d5d28f8ab0c016..5824c470699d88 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -2892,7 +2892,7 @@ task_step_impl(asyncio_state *state, TaskObj *task,
PyObject *exc)
if (task->task_state != STATE_PENDING) {
PyErr_Format(state->asyncio_InvalidStateError,
- "_step(): already done: %R %R",
+ "__step(): already done: %R %R",
task,
exc ? exc : Py_None);
goto fail;
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (#122138)
https://github.com/python/cpython/commit/47847aa8ef66837f984fc4e30187d88f8d8ab201
commit: 47847aa8ef66837f984fc4e30187d88f8d8ab201
branch: main
author: Sam Gross
committer: kumaraditya303
date: 2024-07-23T14:17:14+05:30
summary:
gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (#122138)
Use `PyDict_SetDefaultRef` to set the current task in a single operation
under the dictionary's lock.
files:
M Modules/_asynciomodule.c
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 372f19794be0dd..dd64b233e90e60 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -2009,14 +2009,11 @@ static int
enter_task(asyncio_state *state, PyObject *loop, PyObject *task)
{
PyObject *item;
-Py_hash_t hash;
-hash = PyObject_Hash(loop);
-if (hash == -1) {
+int res = PyDict_SetDefaultRef(state->current_tasks, loop, task, &item);
+if (res < 0) {
return -1;
}
-item = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash);
-if (item != NULL) {
-Py_INCREF(item);
+else if (res == 1) {
PyErr_Format(
PyExc_RuntimeError,
"Cannot enter into task %R while another " \
@@ -2025,10 +2022,8 @@ enter_task(asyncio_state *state, PyObject *loop,
PyObject *task)
Py_DECREF(item);
return -1;
}
-if (PyErr_Occurred()) {
-return -1;
-}
-return _PyDict_SetItem_KnownHash(state->current_tasks, loop, task, hash);
+Py_DECREF(item);
+return 0;
}
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] Fix typos in comments and exception message (#122147)
https://github.com/python/cpython/commit/2a5d1eb7073179a13159bce937afdbe240432e7d
commit: 2a5d1eb7073179a13159bce937afdbe240432e7d
branch: main
author: Xie Yanbo
committer: kumaraditya303
date: 2024-07-23T14:34:14+05:30
summary:
Fix typos in comments and exception message (#122147)
files:
M Parser/action_helpers.c
M Parser/asdl_c.py
M Parser/pegen_errors.c
diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c
index 0307a0881ef882..db6f872c7224d1 100644
--- a/Parser/action_helpers.c
+++ b/Parser/action_helpers.c
@@ -1594,7 +1594,7 @@ _PyPegen_concatenate_strings(Parser *p, asdl_expr_seq
*strings,
for (i = 0; i < n_flattened_elements; i++) {
expr_ty elem = asdl_seq_GET(flattened, i);
-/* The concatenation of a FormattedValue and an empty Contant should
+/* The concatenation of a FormattedValue and an empty Constant should
lead to the FormattedValue itself. Thus, we will not take any empty
constants into account, just as in `_PyPegen_joined_str` */
if (f_string_found && elem->kind == Constant_kind &&
diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py
index e6867f138a5ccb..9fed69b12479d6 100755
--- a/Parser/asdl_c.py
+++ b/Parser/asdl_c.py
@@ -131,7 +131,7 @@ def emit(self, s, depth, reflow=True):
def metadata(self):
if self._metadata is None:
raise ValueError(
-"%s was expecting to be annnotated with metadata"
+"%s was expecting to be annotated with metadata"
% type(self).__name__
)
return self._metadata
diff --git a/Parser/pegen_errors.c b/Parser/pegen_errors.c
index e8f11a67e50fa0..e94a4923228d0f 100644
--- a/Parser/pegen_errors.c
+++ b/Parser/pegen_errors.c
@@ -155,7 +155,7 @@ _Pypegen_raise_decode_error(Parser *p)
static int
_PyPegen_tokenize_full_source_to_check_for_errors(Parser *p) {
// Tokenize the whole input to see if there are any tokenization
-// errors such as mistmatching parentheses. These will get priority
+// errors such as mismatching parentheses. These will get priority
// over generic syntax errors only if the line number of the error is
// before the one that we had for the generic error.
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.13] gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (GH-122138) (#122152)
https://github.com/python/cpython/commit/48154e705318f1ef800c48f39984ce5896a4ee47 commit: 48154e705318f1ef800c48f39984ce5896a4ee47 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-23T09:17:52Z summary: [3.13] gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (GH-122138) (#122152) gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (GH-122138) Use `PyDict_SetDefaultRef` to set the current task in a single operation under the dictionary's lock. (cherry picked from commit 47847aa8ef66837f984fc4e30187d88f8d8ab201) Co-authored-by: Sam Gross files: M Modules/_asynciomodule.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 23cbd48528e0f8..fffa6d25d022ac 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -1928,14 +1928,11 @@ static int enter_task(asyncio_state *state, PyObject *loop, PyObject *task) { PyObject *item; -Py_hash_t hash; -hash = PyObject_Hash(loop); -if (hash == -1) { +int res = PyDict_SetDefaultRef(state->current_tasks, loop, task, &item); +if (res < 0) { return -1; } -item = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash); -if (item != NULL) { -Py_INCREF(item); +else if (res == 1) { PyErr_Format( PyExc_RuntimeError, "Cannot enter into task %R while another " \ @@ -1944,10 +1941,8 @@ enter_task(asyncio_state *state, PyObject *loop, PyObject *task) Py_DECREF(item); return -1; } -if (PyErr_Occurred()) { -return -1; -} -return _PyDict_SetItem_KnownHash(state->current_tasks, loop, task, hash); +Py_DECREF(item); +return 0; } ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120974: Use common freelist code in asyncio (#122132)
https://github.com/python/cpython/commit/c908d1f87d287a4b3ec58c85b692a7eb617fa6ea
commit: c908d1f87d287a4b3ec58c85b692a7eb617fa6ea
branch: main
author: Sam Gross
committer: kumaraditya303
date: 2024-07-23T21:54:24+05:30
summary:
gh-120974: Use common freelist code in asyncio (#122132)
This refactors asyncio to use the common freelist helper functions and
macros. As a side effect, the freelist for _asyncio.Future is now
re-enabled in the free-threaded build.
files:
M Include/internal/pycore_freelist_state.h
M Modules/_asynciomodule.c
M Objects/object.c
diff --git a/Include/internal/pycore_freelist_state.h
b/Include/internal/pycore_freelist_state.h
index d4c3d600b61910..edf79dd7521c41 100644
--- a/Include/internal/pycore_freelist_state.h
+++ b/Include/internal/pycore_freelist_state.h
@@ -20,6 +20,7 @@ extern "C" {
# define Py_contexts_MAXFREELIST 255
# define Py_async_gens_MAXFREELIST 80
# define Py_async_gen_asends_MAXFREELIST 80
+# define Py_futureiters_MAXFREELIST 255
# define Py_object_stack_chunks_MAXFREELIST 4
#else
# define PyTuple_MAXSAVESIZE 0
@@ -47,6 +48,7 @@ struct _Py_freelists {
struct _Py_freelist contexts;
struct _Py_freelist async_gens;
struct _Py_freelist async_gen_asends;
+struct _Py_freelist futureiters;
struct _Py_freelist object_stack_chunks;
#else
char _unused; // Empty structs are not allowed.
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index dd64b233e90e60..c44e89d98256fe 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -4,6 +4,7 @@
#include "Python.h"
#include "pycore_dict.h" // _PyDict_GetItem_KnownHash()
+#include "pycore_freelist.h" // _Py_FREELIST_POP()
#include "pycore_modsupport.h"// _PyArg_CheckPositional()
#include "pycore_moduleobject.h" // _PyModule_GetState()
#include "pycore_object.h"// _Py_SetImmortalUntracked
@@ -75,8 +76,6 @@ typedef struct {
#define Future_Check(state, obj) PyObject_TypeCheck(obj, state->FutureType)
#define Task_Check(state, obj) PyObject_TypeCheck(obj, state->TaskType)
-#define FI_FREELIST_MAXLEN 255
-
#ifdef Py_GIL_DISABLED
# define ASYNCIO_STATE_LOCK(state) PyMutex_Lock(&state->mutex)
# define ASYNCIO_STATE_UNLOCK(state) PyMutex_Unlock(&state->mutex)
@@ -138,11 +137,6 @@ typedef struct {
/* Counter for autogenerated Task names */
uint64_t task_name_counter;
-#ifndef Py_GIL_DISABLED
-futureiterobject *fi_freelist;
-Py_ssize_t fi_freelist_len;
-#endif
-
/* Linked-list of all tasks which are instances of asyncio.Task or
subclasses
of it. Third party tasks implementations which don't inherit from
asyncio.Task are tracked separately using the 'non_asyncio_tasks'
WeakSet.
@@ -1584,24 +1578,7 @@ FutureIter_dealloc(futureiterobject *it)
PyObject_GC_UnTrack(it);
tp->tp_clear((PyObject *)it);
-#ifndef Py_GIL_DISABLED
-// GH-115874: We can't use PyType_GetModuleByDef here as the type might
have
-// already been cleared, which is also why we must check if ht_module !=
NULL.
-PyObject *module = ((PyHeapTypeObject*)tp)->ht_module;
-asyncio_state *state = NULL;
-if (module && _PyModule_GetDef(module) == &_asynciomodule) {
-state = get_asyncio_state(module);
-}
-
-// TODO GH-121621: This should be moved to thread state as well.
-if (state && state->fi_freelist_len < FI_FREELIST_MAXLEN) {
-state->fi_freelist_len++;
-it->future = (FutureObj*) state->fi_freelist;
-state->fi_freelist = it;
-}
-else
-#endif
-{
+if (!_Py_FREELIST_PUSH(futureiters, it, Py_futureiters_MAXFREELIST)) {
PyObject_GC_Del(it);
Py_DECREF(tp);
}
@@ -1805,17 +1782,8 @@ future_new_iter(PyObject *fut)
asyncio_state *state = get_asyncio_state_by_def((PyObject *)fut);
ENSURE_FUTURE_ALIVE(state, fut)
-#ifndef Py_GIL_DISABLED
-if (state->fi_freelist_len) {
-state->fi_freelist_len--;
-it = state->fi_freelist;
-state->fi_freelist = (futureiterobject*) it->future;
-it->future = NULL;
-_Py_NewReference((PyObject*) it);
-}
-else
-#endif
-{
+it = _Py_FREELIST_POP(futureiterobject, futureiters);
+if (it == NULL) {
it = PyObject_GC_New(futureiterobject, state->FutureIterType);
if (it == NULL) {
return NULL;
@@ -3678,27 +3646,6 @@ _asyncio_all_tasks_impl(PyObject *module, PyObject *loop)
return tasks;
}
-static void
-module_free_freelists(asyncio_state *state)
-{
-#ifndef Py_GIL_DISABLED
-PyObject *next;
-PyObject *current;
-
-next = (PyObject*) state->fi_freelist;
-while (next != NULL) {
-assert(state->fi_freelist_len > 0);
-state->fi_freelist_len--;
-
-current = next;
-
[Python-checkins] gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (#122139)
https://github.com/python/cpython/commit/a15feded71dd47202db169613effdafc468a8cf3
commit: a15feded71dd47202db169613effdafc468a8cf3
branch: main
author: Sam Gross
committer: kumaraditya303
date: 2024-07-23T17:06:03Z
summary:
gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (#122139)
* gh-120974: Make _asyncio._leave_task atomic in the free-threaded build
Update `_PyDict_DelItemIf` to allow for an argument to be passed to the
predicate.
files:
M Include/internal/pycore_dict.h
M Modules/_asynciomodule.c
M Modules/_weakref.c
M Objects/dictobject.c
diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h
index a4bdf0d7ad8283..fc304aca7fea10 100644
--- a/Include/internal/pycore_dict.h
+++ b/Include/internal/pycore_dict.h
@@ -14,8 +14,12 @@ extern "C" {
// Unsafe flavor of PyDict_GetItemWithError(): no error checking
extern PyObject* _PyDict_GetItemWithError(PyObject *dp, PyObject *key);
-extern int _PyDict_DelItemIf(PyObject *mp, PyObject *key,
- int (*predicate)(PyObject *value));
+// Delete an item from a dict if a predicate is true
+// Returns -1 on error, 1 if the item was deleted, 0 otherwise
+// Export for '_asyncio' shared extension
+PyAPI_FUNC(int) _PyDict_DelItemIf(PyObject *mp, PyObject *key,
+ int (*predicate)(PyObject *value, void *arg),
+ void *arg);
// "KnownHash" variants
// Export for '_asyncio' shared extension
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index c44e89d98256fe..1a223f9bd0cbae 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -1994,30 +1994,36 @@ enter_task(asyncio_state *state, PyObject *loop,
PyObject *task)
return 0;
}
+static int
+err_leave_task(PyObject *item, PyObject *task)
+{
+PyErr_Format(
+PyExc_RuntimeError,
+"Leaving task %R does not match the current task %R.",
+task, item);
+return -1;
+}
+
+static int
+leave_task_predicate(PyObject *item, void *task)
+{
+if (item != task) {
+return err_leave_task(item, (PyObject *)task);
+}
+return 1;
+}
static int
leave_task(asyncio_state *state, PyObject *loop, PyObject *task)
/*[clinic end generated code: output=0ebf6db4b858fb41 input=51296a46313d1ad8]*/
{
-PyObject *item;
-Py_hash_t hash;
-hash = PyObject_Hash(loop);
-if (hash == -1) {
-return -1;
-}
-item = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash);
-if (item != task) {
-if (item == NULL) {
-/* Not entered, replace with None */
-item = Py_None;
-}
-PyErr_Format(
-PyExc_RuntimeError,
-"Leaving task %R does not match the current task %R.",
-task, item, NULL);
-return -1;
+int res = _PyDict_DelItemIf(state->current_tasks, loop,
+leave_task_predicate, task);
+if (res == 0) {
+// task was not found
+return err_leave_task(Py_None, task);
}
-return _PyDict_DelItem_KnownHash(state->current_tasks, loop, hash);
+return res;
}
static PyObject *
diff --git a/Modules/_weakref.c b/Modules/_weakref.c
index a5c15c0f10b930..ecaa08ff60f203 100644
--- a/Modules/_weakref.c
+++ b/Modules/_weakref.c
@@ -31,7 +31,7 @@ _weakref_getweakrefcount_impl(PyObject *module, PyObject
*object)
static int
-is_dead_weakref(PyObject *value)
+is_dead_weakref(PyObject *value, void *unused)
{
if (!PyWeakref_Check(value)) {
PyErr_SetString(PyExc_TypeError, "not a weakref");
@@ -56,15 +56,8 @@ _weakref__remove_dead_weakref_impl(PyObject *module,
PyObject *dct,
PyObject *key)
/*[clinic end generated code: output=d9ff53061fcb875c input=19fc91f257f96a1d]*/
{
-if (_PyDict_DelItemIf(dct, key, is_dead_weakref) < 0) {
-if (PyErr_ExceptionMatches(PyExc_KeyError))
-/* This function is meant to allow safe weak-value dicts
- with GC in another thread (see issue #28427), so it's
- ok if the key doesn't exist anymore.
- */
-PyErr_Clear();
-else
-return NULL;
+if (_PyDict_DelItemIf(dct, key, is_dead_weakref, NULL) < 0) {
+return NULL;
}
Py_RETURN_NONE;
}
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index 7310c3c8e13b5b..ee88576cc77dec 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2508,7 +2508,7 @@ delete_index_from_values(PyDictValues *values, Py_ssize_t
ix)
values->size = size;
}
-static int
+static void
delitem_common(PyDictObject *mp, Py_hash_t hash, Py_ssize_t ix,
PyObject *old_value, uint64_t new_version)
{
@@ -2550,7 +2550,6 @@ delitem_common(PyDictObject *mp, Py_hash_t hash,
Py_ssize_t ix,
Py_DECREF(old_value);
[Python-checkins] [3.13] gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139) (#122186)
https://github.com/python/cpython/commit/77ab53a5f345c705eba9a9ad38d27548adbe670e commit: 77ab53a5f345c705eba9a9ad38d27548adbe670e branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-23T17:30:58Z summary: [3.13] gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139) (#122186) gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139) * gh-120974: Make _asyncio._leave_task atomic in the free-threaded build Update `_PyDict_DelItemIf` to allow for an argument to be passed to the predicate. (cherry picked from commit a15feded71dd47202db169613effdafc468a8cf3) Co-authored-by: Sam Gross files: M Include/internal/pycore_dict.h M Modules/_asynciomodule.c M Modules/_weakref.c M Objects/dictobject.c diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index 01f80e21ba2797..1c4a63c2271a93 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -16,8 +16,12 @@ extern "C" { // Unsafe flavor of PyDict_GetItemWithError(): no error checking extern PyObject* _PyDict_GetItemWithError(PyObject *dp, PyObject *key); -extern int _PyDict_DelItemIf(PyObject *mp, PyObject *key, - int (*predicate)(PyObject *value)); +// Delete an item from a dict if a predicate is true +// Returns -1 on error, 1 if the item was deleted, 0 otherwise +// Export for '_asyncio' shared extension +PyAPI_FUNC(int) _PyDict_DelItemIf(PyObject *mp, PyObject *key, + int (*predicate)(PyObject *value, void *arg), + void *arg); // "KnownHash" variants // Export for '_asyncio' shared extension diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index fffa6d25d022ac..429322c2bf177b 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -1945,30 +1945,36 @@ enter_task(asyncio_state *state, PyObject *loop, PyObject *task) return 0; } +static int +err_leave_task(PyObject *item, PyObject *task) +{ +PyErr_Format( +PyExc_RuntimeError, +"Leaving task %R does not match the current task %R.", +task, item); +return -1; +} + +static int +leave_task_predicate(PyObject *item, void *task) +{ +if (item != task) { +return err_leave_task(item, (PyObject *)task); +} +return 1; +} static int leave_task(asyncio_state *state, PyObject *loop, PyObject *task) /*[clinic end generated code: output=0ebf6db4b858fb41 input=51296a46313d1ad8]*/ { -PyObject *item; -Py_hash_t hash; -hash = PyObject_Hash(loop); -if (hash == -1) { -return -1; -} -item = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash); -if (item != task) { -if (item == NULL) { -/* Not entered, replace with None */ -item = Py_None; -} -PyErr_Format( -PyExc_RuntimeError, -"Leaving task %R does not match the current task %R.", -task, item, NULL); -return -1; +int res = _PyDict_DelItemIf(state->current_tasks, loop, +leave_task_predicate, task); +if (res == 0) { +// task was not found +return err_leave_task(Py_None, task); } -return _PyDict_DelItem_KnownHash(state->current_tasks, loop, hash); +return res; } static PyObject * diff --git a/Modules/_weakref.c b/Modules/_weakref.c index a5c15c0f10b930..ecaa08ff60f203 100644 --- a/Modules/_weakref.c +++ b/Modules/_weakref.c @@ -31,7 +31,7 @@ _weakref_getweakrefcount_impl(PyObject *module, PyObject *object) static int -is_dead_weakref(PyObject *value) +is_dead_weakref(PyObject *value, void *unused) { if (!PyWeakref_Check(value)) { PyErr_SetString(PyExc_TypeError, "not a weakref"); @@ -56,15 +56,8 @@ _weakref__remove_dead_weakref_impl(PyObject *module, PyObject *dct, PyObject *key) /*[clinic end generated code: output=d9ff53061fcb875c input=19fc91f257f96a1d]*/ { -if (_PyDict_DelItemIf(dct, key, is_dead_weakref) < 0) { -if (PyErr_ExceptionMatches(PyExc_KeyError)) -/* This function is meant to allow safe weak-value dicts - with GC in another thread (see issue #28427), so it's - ok if the key doesn't exist anymore. - */ -PyErr_Clear(); -else -return NULL; +if (_PyDict_DelItemIf(dct, key, is_dead_weakref, NULL) < 0) { +return NULL; } Py_RETURN_NONE; } diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 431c4965bdec09..e9861d47136931 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2567,7 +2567,7 @@ delete_index_from_values(PyDictValues *values, Py_ssize_t ix) values->si
[Python-checkins] gh-122187: Avoid TSan reported race in `run_udp_echo_server` (#122189)
https://github.com/python/cpython/commit/2f74b709b637cad7a9c18a2d90b0747823f2ff51 commit: 2f74b709b637cad7a9c18a2d90b0747823f2ff51 branch: main author: Sam Gross committer: kumaraditya303 date: 2024-07-25T13:46:53+05:30 summary: gh-122187: Avoid TSan reported race in `run_udp_echo_server` (#122189) TSan doesn't fully recognize the synchronization via I/O, so ensure that socket name is retrieved earlier and use a different socket for sending the "STOP" message. files: M Lib/test/test_asyncio/utils.py diff --git a/Lib/test/test_asyncio/utils.py b/Lib/test/test_asyncio/utils.py index dbb8d27c176950..35893ab3118e1e 100644 --- a/Lib/test/test_asyncio/utils.py +++ b/Lib/test/test_asyncio/utils.py @@ -301,12 +301,17 @@ def run_udp_echo_server(*, host='127.0.0.1', port=0): family, type, proto, _, sockaddr = addr_info[0] sock = socket.socket(family, type, proto) sock.bind((host, port)) +sockname = sock.getsockname() thread = threading.Thread(target=lambda: echo_datagrams(sock)) thread.start() try: -yield sock.getsockname() +yield sockname finally: -sock.sendto(b'STOP', sock.getsockname()) +# gh-122187: use a separate socket to send the stop message to avoid +# TSan reported race on the same socket. +sock2 = socket.socket(family, type, proto) +sock2.sendto(b'STOP', sockname) +sock2.close() thread.join() ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-122187: Avoid TSan reported race in `run_udp_echo_server` (GH-122189) (#122263)
https://github.com/python/cpython/commit/977c7992860a58f7a48bcdac53a99e67e5bb9b9c commit: 977c7992860a58f7a48bcdac53a99e67e5bb9b9c branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-25T08:42:38Z summary: [3.13] gh-122187: Avoid TSan reported race in `run_udp_echo_server` (GH-122189) (#122263) gh-122187: Avoid TSan reported race in `run_udp_echo_server` (GH-122189) TSan doesn't fully recognize the synchronization via I/O, so ensure that socket name is retrieved earlier and use a different socket for sending the "STOP" message. (cherry picked from commit 2f74b709b637cad7a9c18a2d90b0747823f2ff51) Co-authored-by: Sam Gross files: M Lib/test/test_asyncio/utils.py diff --git a/Lib/test/test_asyncio/utils.py b/Lib/test/test_asyncio/utils.py index 44943e1fa7bc4e..ce2408fc1aabfb 100644 --- a/Lib/test/test_asyncio/utils.py +++ b/Lib/test/test_asyncio/utils.py @@ -301,12 +301,17 @@ def run_udp_echo_server(*, host='127.0.0.1', port=0): family, type, proto, _, sockaddr = addr_info[0] sock = socket.socket(family, type, proto) sock.bind((host, port)) +sockname = sock.getsockname() thread = threading.Thread(target=lambda: echo_datagrams(sock)) thread.start() try: -yield sock.getsockname() +yield sockname finally: -sock.sendto(b'STOP', sock.getsockname()) +# gh-122187: use a separate socket to send the stop message to avoid +# TSan reported race on the same socket. +sock2 = socket.socket(family, type, proto) +sock2.sendto(b'STOP', sockname) +sock2.close() thread.join() ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (#122269)
https://github.com/python/cpython/commit/070f1e2e5b9b31ee3e7a1af2e30d7e3a66040b17
commit: 070f1e2e5b9b31ee3e7a1af2e30d7e3a66040b17
branch: main
author: AN Long
committer: kumaraditya303
date: 2024-07-25T17:26:04+05:30
summary:
gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events`
(#122269)
files:
M Lib/asyncio/base_events.py
diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index f0e690b61a73dd..e4a39f4d345c79 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -1028,8 +1028,7 @@ async def _connect_sock(self, exceptions, addr_info,
local_addr_infos=None):
except OSError as exc:
msg = (
f'error while attempting to bind on '
-f'address {laddr!r}: '
-f'{exc.strerror.lower()}'
+f'address {laddr!r}: {str(exc).lower()}'
)
exc = OSError(exc.errno, msg)
my_exceptions.append(exc)
@@ -1599,7 +1598,7 @@ async def create_server(
except OSError as err:
msg = ('error while attempting '
'to bind on address %r: %s'
- % (sa, err.strerror.lower()))
+ % (sa, str(err).lower()))
if err.errno == errno.EADDRNOTAVAIL:
# Assume the family is not enabled (bpo-30945)
sockets.pop()
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-121275: Fix test_logging and test_smtplib with Python build withoud IPv6 support (#121276)
https://github.com/python/cpython/commit/3998554bb05f5ce18e8a66492d23d094a2299442 commit: 3998554bb05f5ce18e8a66492d23d094a2299442 branch: main author: AN Long committer: kumaraditya303 date: 2024-07-25T17:33:39+05:30 summary: gh-121275: Fix test_logging and test_smtplib with Python build withoud IPv6 support (#121276) Fix test_logging and test_smtplib with Python build withoud IPv6 support files: M Lib/test/support/smtpd.py diff --git a/Lib/test/support/smtpd.py b/Lib/test/support/smtpd.py index c2e17cad422861..6537679db9ad24 100755 --- a/Lib/test/support/smtpd.py +++ b/Lib/test/support/smtpd.py @@ -633,7 +633,8 @@ def __init__(self, localaddr, remoteaddr, " be set to True at the same time") asyncore.dispatcher.__init__(self, map=map) try: -gai_results = socket.getaddrinfo(*localaddr, +family = 0 if socket.has_ipv6 else socket.AF_INET +gai_results = socket.getaddrinfo(*localaddr, family=family, type=socket.SOCK_STREAM) self.create_socket(gai_results[0][0], gai_results[0][1]) # try to re-use a server port if possible ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (#122279)
https://github.com/python/cpython/commit/0300e33b223a6cfd691bea186cd413424162d83a commit: 0300e33b223a6cfd691bea186cd413424162d83a branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-25T12:15:22Z summary: [3.12] gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (#122279) gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (cherry picked from commit 070f1e2e5b9b31ee3e7a1af2e30d7e3a66040b17) Co-authored-by: AN Long files: M Lib/asyncio/base_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 29eff0499cb1d6..cb037fd472c5aa 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -994,8 +994,7 @@ async def _connect_sock(self, exceptions, addr_info, local_addr_infos=None): except OSError as exc: msg = ( f'error while attempting to bind on ' -f'address {laddr!r}: ' -f'{exc.strerror.lower()}' +f'address {laddr!r}: {str(exc).lower()}' ) exc = OSError(exc.errno, msg) my_exceptions.append(exc) @@ -1561,7 +1560,7 @@ async def create_server( except OSError as err: msg = ('error while attempting ' 'to bind on address %r: %s' - % (sa, err.strerror.lower())) + % (sa, str(err).lower())) if err.errno == errno.EADDRNOTAVAIL: # Assume the family is not enabled (bpo-30945) sockets.pop() ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (#122278)
https://github.com/python/cpython/commit/d1af4f53720d252326ae9c91c93495b58e0b4922 commit: d1af4f53720d252326ae9c91c93495b58e0b4922 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-25T12:20:40Z summary: [3.13] gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (#122278) gh-121913: Use str(exc) instead of exc.strerror in `asyncio.base_events` (GH-122269) (cherry picked from commit 070f1e2e5b9b31ee3e7a1af2e30d7e3a66040b17) Co-authored-by: AN Long files: M Lib/asyncio/base_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index f0e690b61a73dd..e4a39f4d345c79 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -1028,8 +1028,7 @@ async def _connect_sock(self, exceptions, addr_info, local_addr_infos=None): except OSError as exc: msg = ( f'error while attempting to bind on ' -f'address {laddr!r}: ' -f'{exc.strerror.lower()}' +f'address {laddr!r}: {str(exc).lower()}' ) exc = OSError(exc.errno, msg) my_exceptions.append(exc) @@ -1599,7 +1598,7 @@ async def create_server( except OSError as err: msg = ('error while attempting ' 'to bind on address %r: %s' - % (sa, err.strerror.lower())) + % (sa, str(err).lower())) if err.errno == errno.EADDRNOTAVAIL: # Assume the family is not enabled (bpo-30945) sockets.pop() ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-116322: Fix typo in the #ifdef check (#122268)
https://github.com/python/cpython/commit/9bb2e4623f504c44655436eae181d802f544fff9 commit: 9bb2e4623f504c44655436eae181d802f544fff9 branch: main author: Serhiy Storchaka committer: kumaraditya303 date: 2024-07-25T20:01:57+05:30 summary: gh-116322: Fix typo in the #ifdef check (#122268) files: M Objects/moduleobject.c diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 73ad9711b6b0fc..efc74dafb5fc73 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -251,7 +251,7 @@ _PyModule_CreateInitialized(PyModuleDef* module, int module_api_version) } } m->md_def = module; -#ifdef Py_GIL_DISABLE +#ifdef Py_GIL_DISABLED m->md_gil = Py_MOD_GIL_USED; #endif return (PyObject*)m; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-116322: Fix typo in the GH-ifdef check (GH-122268) (#122284)
https://github.com/python/cpython/commit/88a96fc391d36f537a613c6efbe5980080156fea commit: 88a96fc391d36f537a613c6efbe5980080156fea branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-25T14:58:34Z summary: [3.13] gh-116322: Fix typo in the GH-ifdef check (GH-122268) (#122284) gh-116322: Fix typo in the GH-ifdef check (GH-122268) (cherry picked from commit 9bb2e4623f504c44655436eae181d802f544fff9) Co-authored-by: Serhiy Storchaka files: M Objects/moduleobject.c diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 46995b948a28e7..92554de055c4e5 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -250,7 +250,7 @@ _PyModule_CreateInitialized(PyModuleDef* module, int module_api_version) } } m->md_def = module; -#ifdef Py_GIL_DISABLE +#ifdef Py_GIL_DISABLED m->md_gil = Py_MOD_GIL_USED; #endif return (PyObject*)m; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (#122338)
https://github.com/python/cpython/commit/c08696286f52d286674f264eecf7b33a335a890b
commit: c08696286f52d286674f264eecf7b33a335a890b
branch: main
author: Peter Bierma
committer: kumaraditya303
date: 2024-07-27T11:57:48+05:30
summary:
gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (#122338)
files:
A Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst
M Lib/test/test_asyncio/test_eager_task_factory.py
M Modules/_asynciomodule.c
diff --git a/Lib/test/test_asyncio/test_eager_task_factory.py
b/Lib/test/test_asyncio/test_eager_task_factory.py
index 0f8212dbec47be..0777f39b572486 100644
--- a/Lib/test/test_asyncio/test_eager_task_factory.py
+++ b/Lib/test/test_asyncio/test_eager_task_factory.py
@@ -241,6 +241,18 @@ class DummyLoop:
_, out, err = assert_python_ok("-c", code)
self.assertFalse(err)
+def test_issue122332(self):
+ async def coro():
+ pass
+
+ async def run():
+ task = self.loop.create_task(coro())
+ await task
+ self.assertIsNone(task.get_coro())
+
+ self.run_coro(run())
+
+
class AsyncTaskCounter:
def __init__(self, loop, *, task_class, eager):
self.suspense_count = 0
diff --git
a/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst
b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst
new file mode 100644
index 00..55bb1dc44add1b
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst
@@ -0,0 +1,2 @@
+Fixed segfault with :meth:`asyncio.Task.get_coro` when using an eager task
+factory.
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 1a223f9bd0cbae..873c17cd78709d 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -2500,7 +2500,11 @@ static PyObject *
_asyncio_Task_get_coro_impl(TaskObj *self)
/*[clinic end generated code: output=bcac27c8cc6c8073 input=d2e8606c42a7b403]*/
{
-return Py_NewRef(self->task_coro);
+if (self->task_coro) {
+return Py_NewRef(self->task_coro);
+}
+
+Py_RETURN_NONE;
}
/*[clinic input]
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (#122345)
https://github.com/python/cpython/commit/102dc44a0afe3a0c6167c52a57f1cfcedb6fb3db commit: 102dc44a0afe3a0c6167c52a57f1cfcedb6fb3db branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-27T12:36:40+05:30 summary: [3.12] gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (#122345) gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (cherry picked from commit c08696286f52d286674f264eecf7b33a335a890b) Co-authored-by: Peter Bierma files: A Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst M Lib/test/test_asyncio/test_eager_task_factory.py M Modules/_asynciomodule.c diff --git a/Lib/test/test_asyncio/test_eager_task_factory.py b/Lib/test/test_asyncio/test_eager_task_factory.py index 346888735ff70b..58c06287bc3c5d 100644 --- a/Lib/test/test_asyncio/test_eager_task_factory.py +++ b/Lib/test/test_asyncio/test_eager_task_factory.py @@ -246,6 +246,18 @@ class DummyLoop: _, out, err = assert_python_ok("-c", code) self.assertFalse(err) +def test_issue122332(self): + async def coro(): + pass + + async def run(): + task = self.loop.create_task(coro()) + await task + self.assertIsNone(task.get_coro()) + + self.run_coro(run()) + + class AsyncTaskCounter: def __init__(self, loop, *, task_class, eager): self.suspense_count = 0 diff --git a/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst new file mode 100644 index 00..55bb1dc44add1b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst @@ -0,0 +1,2 @@ +Fixed segfault with :meth:`asyncio.Task.get_coro` when using an eager task +factory. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 05e79915ba7c28..6b969edca29804 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2509,7 +2509,11 @@ static PyObject * _asyncio_Task_get_coro_impl(TaskObj *self) /*[clinic end generated code: output=bcac27c8cc6c8073 input=d2e8606c42a7b403]*/ { -return Py_NewRef(self->task_coro); +if (self->task_coro) { +return Py_NewRef(self->task_coro); +} + +Py_RETURN_NONE; } /*[clinic input] ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (#122344)
https://github.com/python/cpython/commit/6b9a5af72f06b5dc2cc413e8ea2c6e8a914ab6d6 commit: 6b9a5af72f06b5dc2cc413e8ea2c6e8a914ab6d6 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-07-27T12:36:50+05:30 summary: [3.13] gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (#122344) gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (GH-122338) (cherry picked from commit c08696286f52d286674f264eecf7b33a335a890b) Co-authored-by: Peter Bierma files: A Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst M Lib/test/test_asyncio/test_eager_task_factory.py M Modules/_asynciomodule.c diff --git a/Lib/test/test_asyncio/test_eager_task_factory.py b/Lib/test/test_asyncio/test_eager_task_factory.py index 0f8212dbec47be..0777f39b572486 100644 --- a/Lib/test/test_asyncio/test_eager_task_factory.py +++ b/Lib/test/test_asyncio/test_eager_task_factory.py @@ -241,6 +241,18 @@ class DummyLoop: _, out, err = assert_python_ok("-c", code) self.assertFalse(err) +def test_issue122332(self): + async def coro(): + pass + + async def run(): + task = self.loop.create_task(coro()) + await task + self.assertIsNone(task.get_coro()) + + self.run_coro(run()) + + class AsyncTaskCounter: def __init__(self, loop, *, task_class, eager): self.suspense_count = 0 diff --git a/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst new file mode 100644 index 00..55bb1dc44add1b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-26-21-21-13.gh-issue-122332.fvw88r.rst @@ -0,0 +1,2 @@ +Fixed segfault with :meth:`asyncio.Task.get_coro` when using an eager task +factory. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 429322c2bf177b..28641b85451763 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2450,7 +2450,11 @@ static PyObject * _asyncio_Task_get_coro_impl(TaskObj *self) /*[clinic end generated code: output=bcac27c8cc6c8073 input=d2e8606c42a7b403]*/ { -return Py_NewRef(self->task_coro); +if (self->task_coro) { +return Py_NewRef(self->task_coro); +} + +Py_RETURN_NONE; } /*[clinic input] ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (#122317)
https://github.com/python/cpython/commit/b5e6fb39a246bf7ee470d58632cdf588bb9d0298
commit: b5e6fb39a246bf7ee470d58632cdf588bb9d0298
branch: main
author: Sam Gross
committer: kumaraditya303
date: 2024-08-02T19:02:08+05:30
summary:
gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
(#122317)
* gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
files:
M Include/internal/pycore_dict.h
M Modules/_asynciomodule.c
M Objects/dictobject.c
diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h
index fc304aca7fea10..a84246ee34efff 100644
--- a/Include/internal/pycore_dict.h
+++ b/Include/internal/pycore_dict.h
@@ -108,8 +108,13 @@ PyAPI_FUNC(PyObject *)_PyDict_LoadGlobal(PyDictObject *,
PyDictObject *, PyObjec
/* Consumes references to key and value */
PyAPI_FUNC(int) _PyDict_SetItem_Take2(PyDictObject *op, PyObject *key,
PyObject *value);
extern int _PyDict_SetItem_LockHeld(PyDictObject *dict, PyObject *name,
PyObject *value);
-extern int _PyDict_GetItemRef_Unicode_LockHeld(PyDictObject *op, PyObject
*key, PyObject **result);
+// Export for '_asyncio' shared extension
+PyAPI_FUNC(int) _PyDict_SetItem_KnownHash_LockHeld(PyDictObject *mp, PyObject
*key,
+ PyObject *value, Py_hash_t
hash);
+// Export for '_asyncio' shared extension
+PyAPI_FUNC(int) _PyDict_GetItemRef_KnownHash_LockHeld(PyDictObject *op,
PyObject *key, Py_hash_t hash, PyObject **result);
extern int _PyDict_GetItemRef_KnownHash(PyDictObject *op, PyObject *key,
Py_hash_t hash, PyObject **result);
+extern int _PyDict_GetItemRef_Unicode_LockHeld(PyDictObject *op, PyObject
*key, PyObject **result);
extern int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject *obj, PyObject
**dictptr, PyObject *name, PyObject *value);
extern int _PyDict_Pop_KnownHash(
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 873c17cd78709d..c6eb43f044fdbd 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -2026,6 +2026,24 @@ leave_task(asyncio_state *state, PyObject *loop,
PyObject *task)
return res;
}
+static PyObject *
+swap_current_task_lock_held(PyDictObject *current_tasks, PyObject *loop,
+Py_hash_t hash, PyObject *task)
+{
+PyObject *prev_task;
+if (_PyDict_GetItemRef_KnownHash_LockHeld(current_tasks, loop, hash,
&prev_task) < 0) {
+return NULL;
+}
+if (_PyDict_SetItem_KnownHash_LockHeld(current_tasks, loop, task, hash) <
0) {
+Py_XDECREF(prev_task);
+return NULL;
+}
+if (prev_task == NULL) {
+Py_RETURN_NONE;
+}
+return prev_task;
+}
+
static PyObject *
swap_current_task(asyncio_state *state, PyObject *loop, PyObject *task)
{
@@ -2041,24 +2059,15 @@ swap_current_task(asyncio_state *state, PyObject *loop,
PyObject *task)
return prev_task;
}
-Py_hash_t hash;
-hash = PyObject_Hash(loop);
+Py_hash_t hash = PyObject_Hash(loop);
if (hash == -1) {
return NULL;
}
-prev_task = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash);
-if (prev_task == NULL) {
-if (PyErr_Occurred()) {
-return NULL;
-}
-prev_task = Py_None;
-}
-Py_INCREF(prev_task);
-if (_PyDict_SetItem_KnownHash(state->current_tasks, loop, task, hash) ==
-1) {
-Py_DECREF(prev_task);
-return NULL;
-}
+PyDictObject *current_tasks = (PyDictObject *)state->current_tasks;
+Py_BEGIN_CRITICAL_SECTION(current_tasks);
+prev_task = swap_current_task_lock_held(current_tasks, loop, hash, task);
+Py_END_CRITICAL_SECTION();
return prev_task;
}
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index 6a16a04102a6c0..3e9f982ae070a3 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2216,6 +2216,29 @@ _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key,
Py_hash_t hash)
return value; // borrowed reference
}
+/* Gets an item and provides a new reference if the value is present.
+ * Returns 1 if the key is present, 0 if the key is missing, and -1 if an
+ * exception occurred.
+*/
+int
+_PyDict_GetItemRef_KnownHash_LockHeld(PyDictObject *op, PyObject *key,
+ Py_hash_t hash, PyObject **result)
+{
+PyObject *value;
+Py_ssize_t ix = _Py_dict_lookup(op, key, hash, &value);
+assert(ix >= 0 || value == NULL);
+if (ix == DKIX_ERROR) {
+*result = NULL;
+return -1;
+}
+if (value == NULL) {
+*result = NULL;
+return 0; // missing key
+}
+*result = Py_NewRef(value);
+return 1; // key is present
+}
+
/* Gets an item and provides a new reference if the value is present.
* Returns 1 if the key is present, 0 if the key is missing, and -1 if an
* exception occurred.
@@ -2460,11 +2483,21 @@ setitem_lock_held(PyDictObject
[Python-checkins] gh-122334: Fix crash when importing ssl after re-initialization (#122481)
https://github.com/python/cpython/commit/9fc1c992d6fcea0b7558c581846eef6bdd811f6c commit: 9fc1c992d6fcea0b7558c581846eef6bdd811f6c branch: main author: neonene <[email protected]> committer: kumaraditya303 date: 2024-08-02T19:06:20+05:30 summary: gh-122334: Fix crash when importing ssl after re-initialization (#122481) * Fix crash when importing ssl after re-initialization files: A Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst M Lib/test/test_embed.py M Python/getargs.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 9602f1a92c37c8..ab112d6be85b46 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -461,6 +461,25 @@ def add(cls, slot, own): self.assertEqual(result, {}) self.assertEqual(out, '') +def test_getargs_reset_static_parser(self): +# Test _PyArg_Parser initializations via _PyArg_UnpackKeywords() +# https://github.com/python/cpython/issues/122334 +code = textwrap.dedent(""" +import _ssl +_ssl.txt2obj(txt='1.3') +print('1') + +import _queue +_queue.SimpleQueue().put_nowait(item=None) +print('2') + +import _zoneinfo +_zoneinfo.ZoneInfo.clear_cache(only_keys=['Foo/Bar']) +print('3') +""") +out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) +self.assertEqual(out, '1\n2\n3\n' * INIT_LOOPS) + @unittest.skipIf(_testinternalcapi is None, "requires _testinternalcapi") class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst new file mode 100644 index 00..cef801c950faa6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst @@ -0,0 +1 @@ +Fix crash when importing :mod:`ssl` after the main interpreter restarts. diff --git a/Python/getargs.c b/Python/getargs.c index b96ce3a22dae7c..ec2eeb15c832c3 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -2030,6 +2030,19 @@ parser_clear(struct _PyArg_Parser *parser) if (parser->is_kwtuple_owned) { Py_CLEAR(parser->kwtuple); } + +if (parser->format) { +parser->fname = NULL; +} +else { +assert(parser->fname != NULL); +} +parser->custom_msg = NULL; +parser->pos = 0; +parser->min = 0; +parser->max = 0; +parser->is_kwtuple_owned = 0; +parser->once.v = 0; } static PyObject* ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) (#122495)
https://github.com/python/cpython/commit/6b8a9a10617817180f546e537d1b4407625390b9 commit: 6b8a9a10617817180f546e537d1b4407625390b9 branch: 3.12 author: neonene <[email protected]> committer: kumaraditya303 date: 2024-08-02T13:44:55Z summary: [3.12] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) (#122495) Fix crash when importing ssl after re-initialization The current METH_FASTCALL|METH_KEYWORDS functions in a non-builtin module can cause segfaults after restarting the main interpreter, invoking _PyArg_UnpackKeywords() with an insufficiently cleared _PyArg_Parser struct. This patch fixes the invalidation of the static argument parsers. files: A Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst M Lib/test/test_embed.py M Python/getargs.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index bb798ca8a9a22c..5ce23223a0f97f 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -434,6 +434,21 @@ def add(cls, slot, own): self.assertEqual(result, {}) self.assertEqual(out, '') +def test_getargs_reset_static_parser(self): +# Test _PyArg_Parser initializations via _PyArg_UnpackKeywords() +# https://github.com/python/cpython/issues/122334 +code = textwrap.dedent(""" +import _ssl +_ssl.txt2obj(txt='1.3') +print('1') + +import _queue +_queue.SimpleQueue().put_nowait(item=None) +print('2') +""") +out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) +self.assertEqual(out, '1\n2\n' * INIT_LOOPS) + class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): maxDiff = 4096 diff --git a/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst new file mode 100644 index 00..cef801c950faa6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst @@ -0,0 +1 @@ +Fix crash when importing :mod:`ssl` after the main interpreter restarts. diff --git a/Python/getargs.c b/Python/getargs.c index 02bddf0618e578..8ca865b53c8d54 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -2071,6 +2071,18 @@ parser_clear(struct _PyArg_Parser *parser) if (parser->initialized == 1) { Py_CLEAR(parser->kwtuple); } + +if (parser->format) { +parser->fname = NULL; +} +else { +assert(parser->fname != NULL); +} +parser->custom_msg = NULL; +parser->pos = 0; +parser->min = 0; +parser->max = 0; +parser->initialized = 0; } static PyObject* ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317) (#122612)
https://github.com/python/cpython/commit/c1efeb32345e93d423665e91a61ce34a482e0271 commit: c1efeb32345e93d423665e91a61ce34a482e0271 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-02T13:58:24Z summary: [3.13] gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317) (#122612) gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317) * gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (cherry picked from commit b5e6fb39a246bf7ee470d58632cdf588bb9d0298) Co-authored-by: Sam Gross files: M Include/internal/pycore_dict.h M Modules/_asynciomodule.c M Objects/dictobject.c diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index 1c4a63c2271a93..36da498db2c3e1 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -110,8 +110,13 @@ PyAPI_FUNC(PyObject *)_PyDict_LoadGlobal(PyDictObject *, PyDictObject *, PyObjec /* Consumes references to key and value */ PyAPI_FUNC(int) _PyDict_SetItem_Take2(PyDictObject *op, PyObject *key, PyObject *value); extern int _PyDict_SetItem_LockHeld(PyDictObject *dict, PyObject *name, PyObject *value); -extern int _PyDict_GetItemRef_Unicode_LockHeld(PyDictObject *op, PyObject *key, PyObject **result); +// Export for '_asyncio' shared extension +PyAPI_FUNC(int) _PyDict_SetItem_KnownHash_LockHeld(PyDictObject *mp, PyObject *key, + PyObject *value, Py_hash_t hash); +// Export for '_asyncio' shared extension +PyAPI_FUNC(int) _PyDict_GetItemRef_KnownHash_LockHeld(PyDictObject *op, PyObject *key, Py_hash_t hash, PyObject **result); extern int _PyDict_GetItemRef_KnownHash(PyDictObject *op, PyObject *key, Py_hash_t hash, PyObject **result); +extern int _PyDict_GetItemRef_Unicode_LockHeld(PyDictObject *op, PyObject *key, PyObject **result); extern int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject *obj, PyObject **dictptr, PyObject *name, PyObject *value); extern int _PyDict_Pop_KnownHash( diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 28641b85451763..6e87de5e954826 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -1977,6 +1977,24 @@ leave_task(asyncio_state *state, PyObject *loop, PyObject *task) return res; } +static PyObject * +swap_current_task_lock_held(PyDictObject *current_tasks, PyObject *loop, +Py_hash_t hash, PyObject *task) +{ +PyObject *prev_task; +if (_PyDict_GetItemRef_KnownHash_LockHeld(current_tasks, loop, hash, &prev_task) < 0) { +return NULL; +} +if (_PyDict_SetItem_KnownHash_LockHeld(current_tasks, loop, task, hash) < 0) { +Py_XDECREF(prev_task); +return NULL; +} +if (prev_task == NULL) { +Py_RETURN_NONE; +} +return prev_task; +} + static PyObject * swap_current_task(asyncio_state *state, PyObject *loop, PyObject *task) { @@ -1992,24 +2010,15 @@ swap_current_task(asyncio_state *state, PyObject *loop, PyObject *task) return prev_task; } -Py_hash_t hash; -hash = PyObject_Hash(loop); +Py_hash_t hash = PyObject_Hash(loop); if (hash == -1) { return NULL; } -prev_task = _PyDict_GetItem_KnownHash(state->current_tasks, loop, hash); -if (prev_task == NULL) { -if (PyErr_Occurred()) { -return NULL; -} -prev_task = Py_None; -} -Py_INCREF(prev_task); -if (_PyDict_SetItem_KnownHash(state->current_tasks, loop, task, hash) == -1) { -Py_DECREF(prev_task); -return NULL; -} +PyDictObject *current_tasks = (PyDictObject *)state->current_tasks; +Py_BEGIN_CRITICAL_SECTION(current_tasks); +prev_task = swap_current_task_lock_held(current_tasks, loop, hash, task); +Py_END_CRITICAL_SECTION(); return prev_task; } diff --git a/Objects/dictobject.c b/Objects/dictobject.c index dedcd232483575..05600772db7026 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2275,6 +2275,29 @@ _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash) return value; // borrowed reference } +/* Gets an item and provides a new reference if the value is present. + * Returns 1 if the key is present, 0 if the key is missing, and -1 if an + * exception occurred. +*/ +int +_PyDict_GetItemRef_KnownHash_LockHeld(PyDictObject *op, PyObject *key, + Py_hash_t hash, PyObject **result) +{ +PyObject *value; +Py_ssize_t ix = _Py_dict_lookup(op, key, hash, &value); +assert(ix >= 0 || value == NULL); +if (ix == DKIX_ERROR) { +*result = NULL; +return -1; +} +if (value == NULL) { +*result = NULL; +return 0; // missing key +} +*result = Py_NewRef(value); +return 1; //
[Python-checkins] [3.13] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) (#122614)
https://github.com/python/cpython/commit/b20893b5c2dc999bbad5723c20f0ff2aea83a428 commit: b20893b5c2dc999bbad5723c20f0ff2aea83a428 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-02T14:07:16Z summary: [3.13] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) (#122614) gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) * Fix crash when importing ssl after re-initialization (cherry picked from commit 9fc1c992d6fcea0b7558c581846eef6bdd811f6c) Co-authored-by: neonene <[email protected]> files: A Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst M Lib/test/test_embed.py M Python/getargs.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 9a63dbc3c6af37..19db79f8a161d5 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -460,6 +460,25 @@ def add(cls, slot, own): self.assertEqual(result, {}) self.assertEqual(out, '') +def test_getargs_reset_static_parser(self): +# Test _PyArg_Parser initializations via _PyArg_UnpackKeywords() +# https://github.com/python/cpython/issues/122334 +code = textwrap.dedent(""" +import _ssl +_ssl.txt2obj(txt='1.3') +print('1') + +import _queue +_queue.SimpleQueue().put_nowait(item=None) +print('2') + +import _zoneinfo +_zoneinfo.ZoneInfo.clear_cache(only_keys=['Foo/Bar']) +print('3') +""") +out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) +self.assertEqual(out, '1\n2\n3\n' * INIT_LOOPS) + @unittest.skipIf(_testinternalcapi is None, "requires _testinternalcapi") class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst new file mode 100644 index 00..cef801c950faa6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-07-30-21-29-30.gh-issue-122334.LeoE1x.rst @@ -0,0 +1 @@ +Fix crash when importing :mod:`ssl` after the main interpreter restarts. diff --git a/Python/getargs.c b/Python/getargs.c index 0b272374e08b58..f40972f23e67af 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -2004,6 +2004,19 @@ parser_clear(struct _PyArg_Parser *parser) if (parser->is_kwtuple_owned) { Py_CLEAR(parser->kwtuple); } + +if (parser->format) { +parser->fname = NULL; +} +else { +assert(parser->fname != NULL); +} +parser->custom_msg = NULL; +parser->pos = 0; +parser->min = 0; +parser->max = 0; +parser->is_kwtuple_owned = 0; +parser->once.v = 0; } static PyObject* ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-122581: Avoid data races when collecting parser statistics (GH-122694) (#122733)
https://github.com/python/cpython/commit/9fb6bfbef0273622421bcc109e00a51923f7b4a7 commit: 9fb6bfbef0273622421bcc109e00a51923f7b4a7 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: kumaraditya303 date: 2024-08-07T18:29:03+05:30 summary: [3.13] gh-122581: Avoid data races when collecting parser statistics (GH-122694) (#122733) gh-122581: Avoid data races when collecting parser statistics (GH-122694) (cherry picked from commit ce0d66c8d238c9676c6ecd3f04294a3299e07f74) Co-authored-by: Lysandros Nikolaou files: M Include/internal/pycore_parser.h M Parser/pegen.c diff --git a/Include/internal/pycore_parser.h b/Include/internal/pycore_parser.h index 067b34c12c4e7f..b16084aaa15515 100644 --- a/Include/internal/pycore_parser.h +++ b/Include/internal/pycore_parser.h @@ -21,6 +21,9 @@ extern "C" { struct _parser_runtime_state { #ifdef Py_DEBUG long memo_statistics[_PYPEGEN_NSTATISTICS]; +#ifdef Py_GIL_DISABLED +PyMutex mutex; +#endif #else int _not_used; #endif @@ -28,8 +31,10 @@ struct _parser_runtime_state { }; _Py_DECLARE_STR(empty, "") +#if defined(Py_DEBUG) && defined(Py_GIL_DISABLED) #define _parser_runtime_state_INIT \ { \ +.mutex = {0}, \ .dummy_name = { \ .kind = Name_kind, \ .v.Name.id = &_Py_STR(empty), \ @@ -40,6 +45,20 @@ _Py_DECLARE_STR(empty, "") .end_col_offset = 0, \ }, \ } +#else +#define _parser_runtime_state_INIT \ +{ \ +.dummy_name = { \ +.kind = Name_kind, \ +.v.Name.id = &_Py_STR(empty), \ +.v.Name.ctx = Load, \ +.lineno = 1, \ +.col_offset = 0, \ +.end_lineno = 1, \ +.end_col_offset = 0, \ +}, \ +} +#endif extern struct _mod* _PyParser_ASTFromString( const char *str, diff --git a/Parser/pegen.c b/Parser/pegen.c index ac428be0958bdf..0c3c4689dd7ce6 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -296,12 +296,22 @@ _PyPegen_fill_token(Parser *p) #define NSTATISTICS _PYPEGEN_NSTATISTICS #define memo_statistics _PyRuntime.parser.memo_statistics +#ifdef Py_GIL_DISABLED +#define MUTEX_LOCK() PyMutex_Lock(&_PyRuntime.parser.mutex) +#define MUTEX_UNLOCK() PyMutex_Unlock(&_PyRuntime.parser.mutex) +#else +#define MUTEX_LOCK() +#define MUTEX_UNLOCK() +#endif + void _PyPegen_clear_memo_statistics(void) { +MUTEX_LOCK(); for (int i = 0; i < NSTATISTICS; i++) { memo_statistics[i] = 0; } +MUTEX_UNLOCK(); } PyObject * @@ -311,18 +321,23 @@ _PyPegen_get_memo_statistics(void) if (ret == NULL) { return NULL; } + +MUTEX_LOCK(); for (int i = 0; i < NSTATISTICS; i++) { PyObject *value = PyLong_FromLong(memo_statistics[i]); if (value == NULL) { +MUTEX_UNLOCK(); Py_DECREF(ret); return NULL; } // PyList_SetItem borrows a reference to value. if (PyList_SetItem(ret, i, value) < 0) { +MUTEX_UNLOCK(); Py_DECREF(ret); return NULL; } } +MUTEX_UNLOCK(); return ret; } #endif @@ -348,7 +363,9 @@ _PyPegen_is_memoized(Parser *p, int type, void *pres) if (count <= 0) { count = 1; } +MUTEX_LOCK(); memo_statistics[type] += count; +MUTEX_UNLOCK(); } #endif p->mark = m->mark; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-117139: Fix an incorrect borrow in bytecodes.c (#122318)
https://github.com/python/cpython/commit/674a50ef2f8909c1c5d812e166bcc12ae6377908
commit: 674a50ef2f8909c1c5d812e166bcc12ae6377908
branch: main
author: Sam Gross
committer: kumaraditya303
date: 2024-08-07T19:06:19+05:30
summary:
gh-117139: Fix an incorrect borrow in bytecodes.c (#122318)
`_PyDict_SetItem_Take2` steals both the key (i.e., `sub`) and the value.
files:
M Python/bytecodes.c
M Python/executor_cases.c.h
M Python/generated_cases.c.h
diff --git a/Python/bytecodes.c b/Python/bytecodes.c
index 9a1af0e920188b..e4c97dee1f80ea 100644
--- a/Python/bytecodes.c
+++ b/Python/bytecodes.c
@@ -862,13 +862,14 @@ dummy_func(
PyStackRef_CLOSE(list_st);
}
-inst(STORE_SUBSCR_DICT, (unused/1, value, dict_st, sub_st -- )) {
-PyObject *sub = PyStackRef_AsPyObjectBorrow(sub_st);
+inst(STORE_SUBSCR_DICT, (unused/1, value, dict_st, sub -- )) {
PyObject *dict = PyStackRef_AsPyObjectBorrow(dict_st);
DEOPT_IF(!PyDict_CheckExact(dict));
STAT_INC(STORE_SUBSCR, hit);
-int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub,
PyStackRef_AsPyObjectSteal(value));
+int err = _PyDict_SetItem_Take2((PyDictObject *)dict,
+PyStackRef_AsPyObjectSteal(sub),
+PyStackRef_AsPyObjectSteal(value));
PyStackRef_CLOSE(dict_st);
ERROR_IF(err, error);
}
diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h
index afc7786c9e434d..4def11c515fd3c 100644
--- a/Python/executor_cases.c.h
+++ b/Python/executor_cases.c.h
@@ -1105,20 +1105,21 @@
}
case _STORE_SUBSCR_DICT: {
-_PyStackRef sub_st;
+_PyStackRef sub;
_PyStackRef dict_st;
_PyStackRef value;
-sub_st = stack_pointer[-1];
+sub = stack_pointer[-1];
dict_st = stack_pointer[-2];
value = stack_pointer[-3];
-PyObject *sub = PyStackRef_AsPyObjectBorrow(sub_st);
PyObject *dict = PyStackRef_AsPyObjectBorrow(dict_st);
if (!PyDict_CheckExact(dict)) {
UOP_STAT_INC(uopcode, miss);
JUMP_TO_JUMP_TARGET();
}
STAT_INC(STORE_SUBSCR, hit);
-int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub,
PyStackRef_AsPyObjectSteal(value));
+int err = _PyDict_SetItem_Take2((PyDictObject *)dict,
+PyStackRef_AsPyObjectSteal(sub),
+PyStackRef_AsPyObjectSteal(value));
PyStackRef_CLOSE(dict_st);
if (err) JUMP_TO_ERROR();
stack_pointer += -3;
diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h
index f670353cdbde56..e560372a5f983e 100644
--- a/Python/generated_cases.c.h
+++ b/Python/generated_cases.c.h
@@ -6625,16 +6625,17 @@
static_assert(INLINE_CACHE_ENTRIES_STORE_SUBSCR == 1, "incorrect
cache size");
_PyStackRef value;
_PyStackRef dict_st;
-_PyStackRef sub_st;
+_PyStackRef sub;
/* Skip 1 cache entry */
-sub_st = stack_pointer[-1];
+sub = stack_pointer[-1];
dict_st = stack_pointer[-2];
value = stack_pointer[-3];
-PyObject *sub = PyStackRef_AsPyObjectBorrow(sub_st);
PyObject *dict = PyStackRef_AsPyObjectBorrow(dict_st);
DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR);
STAT_INC(STORE_SUBSCR, hit);
-int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub,
PyStackRef_AsPyObjectSteal(value));
+int err = _PyDict_SetItem_Take2((PyDictObject *)dict,
+PyStackRef_AsPyObjectSteal(sub),
+PyStackRef_AsPyObjectSteal(value));
PyStackRef_CLOSE(dict_st);
if (err) goto pop_3_error;
stack_pointer += -3;
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
