[Python-checkins] GH-113858: GitHub Actions config: Only save ccache on pushes (GH-113859)
https://github.com/python/cpython/commit/5d384b0468b35b393f8ae2d3149d13ff607c9501 commit: 5d384b0468b35b393f8ae2d3149d13ff607c9501 branch: main author: Petr Viktorin committer: encukou date: 2024-01-10T09:49:18+01:00 summary: GH-113858: GitHub Actions config: Only save ccache on pushes (GH-113859) files: M .github/workflows/build.yml M .github/workflows/reusable-ubuntu.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 2168ec101cf3d9..a3b1d7786ee914 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -142,6 +142,8 @@ jobs: run: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - name: Configure ccache action uses: hendrikmuhs/[email protected] +with: + save: ${{ github.event_name == 'push' }} - name: Check Autoconf and aclocal versions run: | grep "Generated by GNU Autoconf 2.71" configure @@ -284,6 +286,8 @@ jobs: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - name: Configure ccache action uses: hendrikmuhs/[email protected] + with: +save: ${{ github.event_name == 'push' }} - name: Configure CPython run: ./configure --config-cache --with-pydebug --with-openssl=$OPENSSL_DIR - name: Build CPython @@ -327,6 +331,8 @@ jobs: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - name: Configure ccache action uses: hendrikmuhs/[email protected] + with: +save: ${{ github.event_name == 'push' }} - name: Setup directory envs for out-of-tree builds run: | echo "CPYTHON_RO_SRCDIR=$(realpath -m ${GITHUB_WORKSPACE}/../cpython-ro-srcdir)" >> $GITHUB_ENV @@ -446,6 +452,8 @@ jobs: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - name: Configure ccache action uses: hendrikmuhs/[email protected] + with: +save: ${{ github.event_name == 'push' }} - name: Configure CPython run: ./configure --config-cache --with-address-sanitizer --without-pymalloc - name: Build CPython diff --git a/.github/workflows/reusable-ubuntu.yml b/.github/workflows/reusable-ubuntu.yml index 819b45bda7f980..f208064767d42f 100644 --- a/.github/workflows/reusable-ubuntu.yml +++ b/.github/workflows/reusable-ubuntu.yml @@ -41,6 +41,8 @@ jobs: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - name: Configure ccache action uses: hendrikmuhs/[email protected] + with: +save: ${{ github.event_name == 'push' }} - name: Setup directory envs for out-of-tree builds run: | echo "CPYTHON_RO_SRCDIR=$(realpath -m ${GITHUB_WORKSPACE}/../cpython-ro-srcdir)" >> $GITHUB_ENV ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.9] Fix documentation build by pinning Alabaster version to 0.7.13 (#113815)
https://github.com/python/cpython/commit/f86e20e4a86de4a06c04200c55e6f639f6795257 commit: f86e20e4a86de4a06c04200c55e6f639f6795257 branch: 3.9 author: Maciej Olko committer: ambv date: 2024-01-10T10:35:38+01:00 summary: [3.9] Fix documentation build by pinning Alabaster version to 0.7.13 (#113815) Alabaster is Sphinx's dependency. Alabaster 0.7.14 released on 2024-01-08 dropped support for Sphinx 3.3 and earlier. https://alabaster.readthedocs.io/en/latest/changelog.html files: M Doc/requirements.txt diff --git a/Doc/requirements.txt b/Doc/requirements.txt index cf659a0fbac2ec..d674857a353d2b 100644 --- a/Doc/requirements.txt +++ b/Doc/requirements.txt @@ -10,6 +10,8 @@ sphinx==2.4.4 docutils==0.17.1 # Jinja version is pinned to a version compatible with Sphinx version 2.4.4. jinja2==3.0.3 +# Alabaster version is pinned to a version compatible with Sphinx version 2.4.4. +alabaster==0.7.13 blurb ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.8] Fix documentation build by pinning Alabaster version to 0.7.13 (GH-113815) (#113899)
https://github.com/python/cpython/commit/fb57c39c2dc633b3cfe185837815196076a89973 commit: fb57c39c2dc633b3cfe185837815196076a89973 branch: 3.8 author: Miss Islington (bot) <[email protected]> committer: ambv date: 2024-01-10T10:45:31+01:00 summary: [3.8] Fix documentation build by pinning Alabaster version to 0.7.13 (GH-113815) (#113899) Alabaster is Sphinx's dependency. Alabaster 0.7.14 released on 2024-01-08 dropped support for Sphinx 3.3 and earlier. (cherry picked from commit f86e20e4a86de4a06c04200c55e6f639f6795257) https://alabaster.readthedocs.io/en/latest/changelog.html Co-authored-by: Maciej Olko files: M Doc/requirements.txt diff --git a/Doc/requirements.txt b/Doc/requirements.txt index aa615e6bf3141f..b3d48f60893778 100644 --- a/Doc/requirements.txt +++ b/Doc/requirements.txt @@ -10,6 +10,8 @@ sphinx==2.4.4 docutils==0.17.1 # Jinja version is pinned to a version compatible with Sphinx version 2.4.4. jinja2==3.0.3 +# Alabaster version is pinned to a version compatible with Sphinx version 2.4.4. +alabaster==0.7.13 blurb ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900)
https://github.com/python/cpython/commit/1b7e0024a16c1820f61c04a8a100498568410afd
commit: 1b7e0024a16c1820f61c04a8a100498568410afd
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T12:36:03+02:00
summary:
gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900)
winfo_id() converts the result of "winfo id" command to integer, but
"winfo pathname" command requires an argument to be a hexadecimal number
on Win64.
files:
A Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst
M Lib/test/test_tkinter/test_misc.py
M Lib/tkinter/__init__.py
diff --git a/Lib/test/test_tkinter/test_misc.py
b/Lib/test/test_tkinter/test_misc.py
index 6639eaaa59936a..dc8a810235fc9b 100644
--- a/Lib/test/test_tkinter/test_misc.py
+++ b/Lib/test/test_tkinter/test_misc.py
@@ -281,6 +281,18 @@ def assertApprox(col1, col2):
with self.assertRaises(tkinter.TclError):
rgb((111, 78, 55))
+def test_winfo_pathname(self):
+t = tkinter.Toplevel(self.root)
+w = tkinter.Button(t)
+wid = w.winfo_id()
+self.assertIsInstance(wid, int)
+self.assertEqual(self.root.winfo_pathname(hex(wid)), str(w))
+self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=None),
str(w))
+self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=t),
str(w))
+self.assertEqual(self.root.winfo_pathname(wid), str(w))
+self.assertEqual(self.root.winfo_pathname(wid, displayof=None), str(w))
+self.assertEqual(self.root.winfo_pathname(wid, displayof=t), str(w))
+
def test_event_repr_defaults(self):
e = tkinter.Event()
e.serial = 12345
diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py
index 124882420c255c..2590acdc87e695 100644
--- a/Lib/tkinter/__init__.py
+++ b/Lib/tkinter/__init__.py
@@ -1260,6 +1260,8 @@ def winfo_parent(self):
def winfo_pathname(self, id, displayof=0):
"""Return the pathname of the widget given by ID."""
+if isinstance(id, int):
+id = hex(id)
args = ('winfo', 'pathname') \
+ self._displayof(displayof) + (id,)
return self.tk.call(args)
diff --git
a/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst
b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst
new file mode 100644
index 00..173e185fe6c632
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst
@@ -0,0 +1 @@
+Fix :mod:`tkinter` method ``winfo_pathname()`` on 64-bit Windows.
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-113879: Fix ResourceWarning in test_asyncio.test_server (GH-113881)
https://github.com/python/cpython/commit/ab0ad62038317a3d15099c23d2b0f03bee9f8fa7 commit: ab0ad62038317a3d15099c23d2b0f03bee9f8fa7 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2024-01-10T12:38:36+02:00 summary: gh-113879: Fix ResourceWarning in test_asyncio.test_server (GH-113881) files: M Lib/test/test_asyncio/test_server.py diff --git a/Lib/test/test_asyncio/test_server.py b/Lib/test/test_asyncio/test_server.py index f22cf3026e244b..918faac909b9bf 100644 --- a/Lib/test/test_asyncio/test_server.py +++ b/Lib/test/test_asyncio/test_server.py @@ -200,13 +200,13 @@ async def test_unix_server_sock_cleanup(self): async def serve(*args): pass -sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) -sock.bind(addr) +with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: +sock.bind(addr) -srv = await asyncio.start_unix_server(serve, sock=sock) +srv = await asyncio.start_unix_server(serve, sock=sock) -srv.close() -self.assertFalse(os.path.exists(addr)) +srv.close() +self.assertFalse(os.path.exists(addr)) @socket_helper.skip_unless_bind_unix_socket async def test_unix_server_cleanup_gone(self): @@ -215,14 +215,14 @@ async def test_unix_server_cleanup_gone(self): async def serve(*args): pass -sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) -sock.bind(addr) +with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: +sock.bind(addr) -srv = await asyncio.start_unix_server(serve, sock=sock) +srv = await asyncio.start_unix_server(serve, sock=sock) -os.unlink(addr) +os.unlink(addr) -srv.close() +srv.close() @socket_helper.skip_unless_bind_unix_socket async def test_unix_server_cleanup_replaced(self): @@ -234,11 +234,11 @@ async def serve(*args): srv = await asyncio.start_unix_server(serve, addr) os.unlink(addr) -sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) -sock.bind(addr) +with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: +sock.bind(addr) -srv.close() -self.assertTrue(os.path.exists(addr)) +srv.close() +self.assertTrue(os.path.exists(addr)) @socket_helper.skip_unless_bind_unix_socket async def test_unix_server_cleanup_prevented(self): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-96037: Always insert TimeoutError when exit an expired asyncio.timeout() block (GH-113819)
https://github.com/python/cpython/commit/aef4a1203c06efde8505aefc9cf994e9a23f398d commit: aef4a1203c06efde8505aefc9cf994e9a23f398d branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2024-01-10T12:50:31+02:00 summary: gh-96037: Always insert TimeoutError when exit an expired asyncio.timeout() block (GH-113819) If other exception was raised during exiting an expired asyncio.timeout() block, insert TimeoutError in the exception context just above the CancelledError. files: A Misc/NEWS.d/next/Library/2024-01-08-19-38-42.gh-issue-96037.Yr2Y1C.rst M Lib/asyncio/timeouts.py M Lib/test/test_asyncio/test_timeouts.py diff --git a/Lib/asyncio/timeouts.py b/Lib/asyncio/timeouts.py index 2c5dd295ff5ade..e6f5100691d362 100644 --- a/Lib/asyncio/timeouts.py +++ b/Lib/asyncio/timeouts.py @@ -110,10 +110,15 @@ async def __aexit__( self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is not None: +# Since there are no new cancel requests, we're +# handling this. if issubclass(exc_type, exceptions.CancelledError): -# Since there are no new cancel requests, we're -# handling this. raise TimeoutError from exc_val +elif exc_val is not None: +self._insert_timeout_error(exc_val) +if isinstance(exc_val, ExceptionGroup): +for exc in exc_val.exceptions: +self._insert_timeout_error(exc) elif self._state is _State.ENTERED: self._state = _State.EXITED @@ -126,6 +131,16 @@ def _on_timeout(self) -> None: # drop the reference early self._timeout_handler = None +@staticmethod +def _insert_timeout_error(exc_val: BaseException) -> None: +while exc_val.__context__ is not None: +if isinstance(exc_val.__context__, exceptions.CancelledError): +te = TimeoutError() +te.__context__ = te.__cause__ = exc_val.__context__ +exc_val.__context__ = te +break +exc_val = exc_val.__context__ + def timeout(delay: Optional[float]) -> Timeout: """Timeout async context manager. diff --git a/Lib/test/test_asyncio/test_timeouts.py b/Lib/test/test_asyncio/test_timeouts.py index f54e79e4d8e600..1f7f9ee696a525 100644 --- a/Lib/test/test_asyncio/test_timeouts.py +++ b/Lib/test/test_asyncio/test_timeouts.py @@ -116,15 +116,68 @@ async def test_foreign_exception_passed(self): raise KeyError self.assertFalse(cm.expired()) +async def test_timeout_exception_context(self): +with self.assertRaises(TimeoutError) as cm: +async with asyncio.timeout(0.01): +try: +1/0 +finally: +await asyncio.sleep(1) +e = cm.exception +# Expect TimeoutError caused by CancelledError raised during handling +# of ZeroDivisionError. +e2 = e.__cause__ +self.assertIsInstance(e2, asyncio.CancelledError) +self.assertIs(e.__context__, e2) +self.assertIsNone(e2.__cause__) +self.assertIsInstance(e2.__context__, ZeroDivisionError) + async def test_foreign_exception_on_timeout(self): async def crash(): try: await asyncio.sleep(1) finally: 1/0 -with self.assertRaises(ZeroDivisionError): +with self.assertRaises(ZeroDivisionError) as cm: async with asyncio.timeout(0.01): await crash() +e = cm.exception +# Expect ZeroDivisionError raised during handling of TimeoutError +# caused by CancelledError. +self.assertIsNone(e.__cause__) +e2 = e.__context__ +self.assertIsInstance(e2, TimeoutError) +e3 = e2.__cause__ +self.assertIsInstance(e3, asyncio.CancelledError) +self.assertIs(e2.__context__, e3) + +async def test_foreign_exception_on_timeout_2(self): +with self.assertRaises(ZeroDivisionError) as cm: +async with asyncio.timeout(0.01): +try: +try: +raise ValueError +finally: +await asyncio.sleep(1) +finally: +try: +raise KeyError +finally: +1/0 +e = cm.exception +# Expect ZeroDivisionError raised during handling of KeyError +# raised during handling of TimeoutError caused by CancelledError. +self.assertIsNone(e.__cause__) +e2 = e.__context__ +self.assertIsInstance(e2, KeyError) +self.assertIsNone(e2.__cause__) +e3 = e2.__context__ +self.assertIsInstance(e3, TimeoutError) +e4 = e3.__ca
[Python-checkins] [3.11] gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900) (GH-113902)
https://github.com/python/cpython/commit/b4a6bbdd7de8bdc43ea62ed4bbb610bf1cd64278 commit: b4a6bbdd7de8bdc43ea62ed4bbb610bf1cd64278 branch: 3.11 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T10:50:51Z summary: [3.11] gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900) (GH-113902) winfo_id() converts the result of "winfo id" command to integer, but "winfo pathname" command requires an argument to be a hexadecimal number on Win64. (cherry picked from commit 1b7e0024a16c1820f61c04a8a100498568410afd) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst M Lib/tkinter/__init__.py M Lib/tkinter/test/test_tkinter/test_misc.py diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 05558e97733cd9..012737c0d7bae3 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -1181,6 +1181,8 @@ def winfo_parent(self): def winfo_pathname(self, id, displayof=0): """Return the pathname of the widget given by ID.""" +if isinstance(id, int): +id = hex(id) args = ('winfo', 'pathname') \ + self._displayof(displayof) + (id,) return self.tk.call(args) diff --git a/Lib/tkinter/test/test_tkinter/test_misc.py b/Lib/tkinter/test/test_tkinter/test_misc.py index aa11e65ea948ce..11f5b32e4b8995 100644 --- a/Lib/tkinter/test/test_tkinter/test_misc.py +++ b/Lib/tkinter/test/test_tkinter/test_misc.py @@ -227,6 +227,18 @@ def assertApprox(col1, col2): with self.assertRaises(tkinter.TclError): rgb((111, 78, 55)) +def test_winfo_pathname(self): +t = tkinter.Toplevel(self.root) +w = tkinter.Button(t) +wid = w.winfo_id() +self.assertIsInstance(wid, int) +self.assertEqual(self.root.winfo_pathname(hex(wid)), str(w)) +self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=None), str(w)) +self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=t), str(w)) +self.assertEqual(self.root.winfo_pathname(wid), str(w)) +self.assertEqual(self.root.winfo_pathname(wid, displayof=None), str(w)) +self.assertEqual(self.root.winfo_pathname(wid, displayof=t), str(w)) + def test_event_repr_defaults(self): e = tkinter.Event() e.serial = 12345 diff --git a/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst new file mode 100644 index 00..173e185fe6c632 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst @@ -0,0 +1 @@ +Fix :mod:`tkinter` method ``winfo_pathname()`` on 64-bit Windows. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900) (GH-113901)
https://github.com/python/cpython/commit/7530c612d10030cf9d53e70b1acd333c31a97d7f commit: 7530c612d10030cf9d53e70b1acd333c31a97d7f branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T10:53:27Z summary: [3.12] gh-113877: Fix Tkinter method winfo_pathname() on 64-bit Windows (GH-113900) (GH-113901) winfo_id() converts the result of "winfo id" command to integer, but "winfo pathname" command requires an argument to be a hexadecimal number on Win64. (cherry picked from commit 1b7e0024a16c1820f61c04a8a100498568410afd) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst M Lib/test/test_tkinter/test_misc.py M Lib/tkinter/__init__.py diff --git a/Lib/test/test_tkinter/test_misc.py b/Lib/test/test_tkinter/test_misc.py index a84bb02b88c920..f4d51c438fedb3 100644 --- a/Lib/test/test_tkinter/test_misc.py +++ b/Lib/test/test_tkinter/test_misc.py @@ -227,6 +227,18 @@ def assertApprox(col1, col2): with self.assertRaises(tkinter.TclError): rgb((111, 78, 55)) +def test_winfo_pathname(self): +t = tkinter.Toplevel(self.root) +w = tkinter.Button(t) +wid = w.winfo_id() +self.assertIsInstance(wid, int) +self.assertEqual(self.root.winfo_pathname(hex(wid)), str(w)) +self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=None), str(w)) +self.assertEqual(self.root.winfo_pathname(hex(wid), displayof=t), str(w)) +self.assertEqual(self.root.winfo_pathname(wid), str(w)) +self.assertEqual(self.root.winfo_pathname(wid, displayof=None), str(w)) +self.assertEqual(self.root.winfo_pathname(wid, displayof=t), str(w)) + def test_event_repr_defaults(self): e = tkinter.Event() e.serial = 12345 diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index c47c8c27d46e32..c5519060dc6f61 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -1181,6 +1181,8 @@ def winfo_parent(self): def winfo_pathname(self, id, displayof=0): """Return the pathname of the widget given by ID.""" +if isinstance(id, int): +id = hex(id) args = ('winfo', 'pathname') \ + self._displayof(displayof) + (id,) return self.tk.call(args) diff --git a/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst new file mode 100644 index 00..173e185fe6c632 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-10-12-03-38.gh-issue-113877.RxKlrQ.rst @@ -0,0 +1 @@ +Fix :mod:`tkinter` method ``winfo_pathname()`` on 64-bit Windows. ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-70835: Clarify error message for CSV file opened with wrong newline (GH-113786)
https://github.com/python/cpython/commit/568d220993fa9b4b812ff1b425edd80dbe17dda9
commit: 568d220993fa9b4b812ff1b425edd80dbe17dda9
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T14:52:29+02:00
summary:
gh-70835: Clarify error message for CSV file opened with wrong newline
(GH-113786)
Based on patch by SilentGhost.
files:
M Lib/test/test_csv.py
M Modules/_csv.c
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
index 97b9bba24bcbca..282cff4344b77f 100644
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -297,13 +297,18 @@ def test_read_oddinputs(self):
[b'abc'], None)
def test_read_eol(self):
-self._read_test(['a,b'], [['a','b']])
-self._read_test(['a,b\n'], [['a','b']])
-self._read_test(['a,b\r\n'], [['a','b']])
-self._read_test(['a,b\r'], [['a','b']])
-self.assertRaises(csv.Error, self._read_test, ['a,b\rc,d'], [])
-self.assertRaises(csv.Error, self._read_test, ['a,b\nc,d'], [])
-self.assertRaises(csv.Error, self._read_test, ['a,b\r\nc,d'], [])
+self._read_test(['a,b', 'c,d'], [['a','b'], ['c','d']])
+self._read_test(['a,b\n', 'c,d\n'], [['a','b'], ['c','d']])
+self._read_test(['a,b\r\n', 'c,d\r\n'], [['a','b'], ['c','d']])
+self._read_test(['a,b\r', 'c,d\r'], [['a','b'], ['c','d']])
+
+errmsg = "with newline=''"
+with self.assertRaisesRegex(csv.Error, errmsg):
+next(csv.reader(['a,b\rc,d']))
+with self.assertRaisesRegex(csv.Error, errmsg):
+next(csv.reader(['a,b\nc,d']))
+with self.assertRaisesRegex(csv.Error, errmsg):
+next(csv.reader(['a,b\r\nc,d']))
def test_read_eof(self):
self._read_test(['a,"'], [['a', '']])
diff --git a/Modules/_csv.c b/Modules/_csv.c
index ae6b6457ffad9a..d45a15aa8c255a 100644
--- a/Modules/_csv.c
+++ b/Modules/_csv.c
@@ -837,7 +837,8 @@ parse_process_char(ReaderObj *self, _csvstate
*module_state, Py_UCS4 c)
self->state = START_RECORD;
else {
PyErr_Format(module_state->error_obj,
- "new-line character seen in unquoted field - do you
need to open the file in universal-newline mode?");
+ "new-line character seen in unquoted field - "
+ "do you need to open the file with newline=''?");
return -1;
}
break;
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730)
https://github.com/python/cpython/commit/e9d5b6ea2d68564f176fdf70c2d7028e060c62b5
commit: e9d5b6ea2d68564f176fdf70c2d7028e060c62b5
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T14:54:36+02:00
summary:
gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730)
It occurred when try to re-encode an unknown-8bit part combined with
non-unknown-8bit part.
files:
A Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst
M Lib/email/_header_value_parser.py
M Lib/test/test_email/test__header_value_parser.py
diff --git a/Lib/email/_header_value_parser.py
b/Lib/email/_header_value_parser.py
index 0d6bd812475eea..5b653f66c18554 100644
--- a/Lib/email/_header_value_parser.py
+++ b/Lib/email/_header_value_parser.py
@@ -2766,6 +2766,7 @@ def _refold_parse_tree(parse_tree, *, policy):
encoding = 'utf-8' if policy.utf8 else 'us-ascii'
lines = ['']
last_ew = None
+last_charset = None
wrap_as_ew_blocked = 0
want_encoding = False
end_ew_not_allowed = Terminal('', 'wrap_as_ew_blocked')
@@ -2820,8 +2821,14 @@ def _refold_parse_tree(parse_tree, *, policy):
else:
# It's a terminal, wrap it as an encoded word, possibly
# combining it with previously encoded words if allowed.
+if (last_ew is not None and
+charset != last_charset and
+(last_charset == 'unknown-8bit' or
+ last_charset == 'utf-8' and charset != 'us-ascii')):
+last_ew = None
last_ew = _fold_as_ew(tstr, lines, maxlen, last_ew,
part.ew_combine_allowed, charset)
+last_charset = charset
want_encoding = False
continue
if len(tstr) <= maxlen - len(lines[-1]):
diff --git a/Lib/test/test_email/test__header_value_parser.py
b/Lib/test/test_email/test__header_value_parser.py
index 854f2ff009c618..bdb0e55f21069f 100644
--- a/Lib/test/test_email/test__header_value_parser.py
+++ b/Lib/test/test_email/test__header_value_parser.py
@@ -2915,6 +2915,45 @@ def test_ews_combined_before_wrap(self):
"mich. And that's\n"
" all I'm sayin.\n")
+def test_unicode_after_unknown_not_combined(self):
+self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=\xa4"),
+ "=?unknown-8bit?q?=A4?==?utf-8?q?=C2=A4?=\n")
+prefix = "0123456789 "*5
+self._test(parser.get_unstructured(prefix +
"=?unknown-8bit?q?=A4?=\xa4"),
+ prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?=C2=A4?=\n")
+
+def test_ascii_after_unknown_not_combined(self):
+self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=abc"),
+ "=?unknown-8bit?q?=A4?=abc\n")
+prefix = "0123456789 "*5
+self._test(parser.get_unstructured(prefix +
"=?unknown-8bit?q?=A4?=abc"),
+ prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?abc?=\n")
+
+def test_unknown_after_unicode_not_combined(self):
+self._test(parser.get_unstructured("\xa4"
+ "=?unknown-8bit?q?=A4?="),
+ "=?utf-8?q?=C2=A4?==?unknown-8bit?q?=A4?=\n")
+prefix = "0123456789 "*5
+self._test(parser.get_unstructured(prefix +
"\xa4=?unknown-8bit?q?=A4?="),
+ prefix + "=?utf-8?q?=C2=A4?=\n =?unknown-8bit?q?=A4?=\n")
+
+def test_unknown_after_ascii_not_combined(self):
+self._test(parser.get_unstructured("abc"
+ "=?unknown-8bit?q?=A4?="),
+ "abc=?unknown-8bit?q?=A4?=\n")
+prefix = "0123456789 "*5
+self._test(parser.get_unstructured(prefix +
"abcd=?unknown-8bit?q?=A4?="),
+ prefix + "abcd\n =?unknown-8bit?q?=A4?=\n")
+
+def test_unknown_after_unknown(self):
+self._test(parser.get_unstructured("=?unknown-8bit?q?=C2?="
+ "=?unknown-8bit?q?=A4?="),
+ "=?unknown-8bit?q?=C2=A4?=\n")
+prefix = "0123456789 "*5
+self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=C2?="
+ "=?unknown-8bit?q?=A4?="),
+ prefix + "=?unknown-8bit?q?=C2?=\n
=?unknown-8bit?q?=A4?=\n")
+
# XXX Need test of an encoded word so long that it needs to be wrapped
def test_simple_address(self):
diff --git
a/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst
b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst
new file mode 100644
index 00..c71bc9c20e4596
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst
@@ -0,0 +1,2 @@
+Fix :exc:`UnicodeEncodeError` in :mod:`email` when re-fold lines that
+contain unknown-8bit encoded part followed by non-unknown-8bit encoded
[Python-checkins] gh-113664: Improve style of Big O notation (GH-113695)
https://github.com/python/cpython/commit/a8629816c6c0e6770248a60529fd7c9ba08aad55 commit: a8629816c6c0e6770248a60529fd7c9ba08aad55 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2024-01-10T15:01:18+02:00 summary: gh-113664: Improve style of Big O notation (GH-113695) Use cursive to make it looking like mathematic formulas. files: M Doc/faq/design.rst M Doc/glossary.rst M Doc/library/asyncio-policy.rst M Doc/library/bisect.rst M Doc/library/collections.rst M Doc/library/contextvars.rst M Doc/library/heapq.rst M Doc/library/select.rst M Doc/reference/datamodel.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.3.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a7.rst M Misc/NEWS.d/3.5.0a1.rst diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index ae02c443e5938b..300e1b6cc40a58 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -451,7 +451,7 @@ on the key and a per-process seed; for example, ``'Python'`` could hash to to ``1142331976``. The hash code is then used to calculate a location in an internal array where the value will be stored. Assuming that you're storing keys that all have different hash values, this means that dictionaries take -constant time -- O(1), in Big-O notation -- to retrieve a key. +constant time -- *O*\ (1), in Big-O notation -- to retrieve a key. Why must dictionary keys be immutable? diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 601443d5aade94..098bfffb104ef6 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -742,7 +742,7 @@ Glossary list A built-in Python :term:`sequence`. Despite its name it is more akin to an array in other languages than to a linked list since access to - elements is O(1). + elements is *O*\ (1). list comprehension A compact way to process all or part of the elements in a sequence and diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 0d7821e608ec98..346b740a8f757a 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -237,7 +237,7 @@ implementation used by the asyncio event loop: It works reliably even when the asyncio event loop is run in a non-main OS thread. - There is no noticeable overhead when handling a big number of children (*O(1)* each + There is no noticeable overhead when handling a big number of children (*O*\ (1) each time a child terminates), but starting a thread per process requires extra memory. This watcher is used by default. @@ -257,7 +257,7 @@ implementation used by the asyncio event loop: watcher is installed. The solution is safe but it has a significant overhead when - handling a big number of processes (*O(n)* each time a + handling a big number of processes (*O*\ (*n*) each time a :py:data:`SIGCHLD` is received). .. versionadded:: 3.8 @@ -273,7 +273,7 @@ implementation used by the asyncio event loop: The watcher avoids disrupting other code spawning processes by polling every process explicitly on a :py:data:`SIGCHLD` signal. - This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O(N)* + This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*) complexity but requires a running event loop in the main thread to work. .. deprecated:: 3.12 @@ -285,7 +285,7 @@ implementation used by the asyncio event loop: processes and waiting for their termination. There is no noticeable overhead when handling a big number of - children (*O(1)* each time a child terminates). + children (*O*\ (1) each time a child terminates). This solution requires a running event loop in the main thread to work, as :class:`SafeChildWatcher`. diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index c0923093c1cb06..31c79b91061591 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -79,7 +79,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -99,7 +99,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -115,7 +115,7 @@ thoughts in mind: * Bisection is effective for searching ranges of values. For locating specific values, dictionaries are more performant. -* The *insort()* functions
[Python-checkins] gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
https://github.com/python/cpython/commit/b3d2427f2280fa8dae3515036c518d74ba43ebd1
commit: b3d2427f2280fa8dae3515036c518d74ba43ebd1
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:07:19+02:00
summary:
gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
Open and close files manually. It prevents from leaking files,
preliminary creation of output files, and accidental closing of stdin
and stdout.
files:
M Lib/ast.py
M Lib/dis.py
M Lib/json/tool.py
M Tools/importbench/importbench.py
M Tools/peg_generator/pegen/keywordgen.py
M Tools/scripts/summarize_stats.py
M Tools/ssl/make_ssl_data.py
diff --git a/Lib/ast.py b/Lib/ast.py
index f7888d18859ae4..7d3cd489942393 100644
--- a/Lib/ast.py
+++ b/Lib/ast.py
@@ -1812,8 +1812,7 @@ def main():
import argparse
parser = argparse.ArgumentParser(prog='python -m ast')
-parser.add_argument('infile', type=argparse.FileType(mode='rb'), nargs='?',
-default='-',
+parser.add_argument('infile', nargs='?', default='-',
help='the file to parse; defaults to stdin')
parser.add_argument('-m', '--mode', default='exec',
choices=('exec', 'single', 'eval', 'func_type'),
@@ -1827,9 +1826,14 @@ def main():
help='indentation of nodes (number of spaces)')
args = parser.parse_args()
-with args.infile as infile:
-source = infile.read()
-tree = parse(source, args.infile.name, args.mode,
type_comments=args.no_type_comments)
+if args.infile == '-':
+name = ''
+source = sys.stdin.buffer.read()
+else:
+name = args.infile
+with open(args.infile, 'rb') as infile:
+source = infile.read()
+tree = parse(source, name, args.mode, type_comments=args.no_type_comments)
print(dump(tree, include_attributes=args.include_attributes,
indent=args.indent))
if __name__ == '__main__':
diff --git a/Lib/dis.py b/Lib/dis.py
index 1a2f1032d500af..f05ea1a24f45a7 100644
--- a/Lib/dis.py
+++ b/Lib/dis.py
@@ -1032,11 +1032,16 @@ def main():
help='show inline caches')
parser.add_argument('-O', '--show-offsets', action='store_true',
help='show instruction offsets')
-parser.add_argument('infile', type=argparse.FileType('rb'), nargs='?',
default='-')
+parser.add_argument('infile', nargs='?', default='-')
args = parser.parse_args()
-with args.infile as infile:
-source = infile.read()
-code = compile(source, args.infile.name, "exec")
+if args.infile == '-':
+name = ''
+source = sys.stdin.buffer.read()
+else:
+name = args.infile
+with open(args.infile, 'rb') as infile:
+source = infile.read()
+code = compile(source, name, "exec")
dis(code, show_caches=args.show_caches, show_offsets=args.show_offsets)
if __name__ == "__main__":
diff --git a/Lib/json/tool.py b/Lib/json/tool.py
index 0490b8c0be11df..fdfc3372bcca02 100644
--- a/Lib/json/tool.py
+++ b/Lib/json/tool.py
@@ -13,7 +13,6 @@
import argparse
import json
import sys
-from pathlib import Path
def main():
@@ -22,11 +21,9 @@ def main():
'to validate and pretty-print JSON objects.')
parser = argparse.ArgumentParser(prog=prog, description=description)
parser.add_argument('infile', nargs='?',
-type=argparse.FileType(encoding="utf-8"),
help='a JSON file to be validated or pretty-printed',
-default=sys.stdin)
+default='-')
parser.add_argument('outfile', nargs='?',
-type=Path,
help='write the output of infile to outfile',
default=None)
parser.add_argument('--sort-keys', action='store_true', default=False,
@@ -59,23 +56,30 @@ def main():
dump_args['indent'] = None
dump_args['separators'] = ',', ':'
-with options.infile as infile:
+try:
+if options.infile == '-':
+infile = sys.stdin
+else:
+infile = open(options.infile, encoding='utf-8')
try:
if options.json_lines:
objs = (json.loads(line) for line in infile)
else:
objs = (json.load(infile),)
+finally:
+if infile is not sys.stdin:
+infile.close()
-if options.outfile is None:
-out = sys.stdout
-else:
-out = options.outfile.open('w', encoding='utf-8')
-with out as outfile:
-for obj in objs:
-json.dump(obj, outfile, **dump_args)
-outfile.write('\n')
-except ValueError as e:
-raise SystemExit(e)
+if options.outfile is None:
+outfile = sys.stdout
+else:
+out
[Python-checkins] [3.12] gh-113664: Improve style of Big O notation (GH-113695) (GH-113909)
https://github.com/python/cpython/commit/082998c3af5bfd163da5c0198cd10acf588a44ba commit: 082998c3af5bfd163da5c0198cd10acf588a44ba branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T13:08:08Z summary: [3.12] gh-113664: Improve style of Big O notation (GH-113695) (GH-113909) Use cursive to make it looking like mathematic formulas. (cherry picked from commit a8629816c6c0e6770248a60529fd7c9ba08aad55) Co-authored-by: Serhiy Storchaka files: M Doc/faq/design.rst M Doc/glossary.rst M Doc/library/asyncio-policy.rst M Doc/library/bisect.rst M Doc/library/collections.rst M Doc/library/contextvars.rst M Doc/library/heapq.rst M Doc/library/select.rst M Doc/reference/datamodel.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.3.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a7.rst M Misc/NEWS.d/3.5.0a1.rst diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index ae02c443e5938b..300e1b6cc40a58 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -451,7 +451,7 @@ on the key and a per-process seed; for example, ``'Python'`` could hash to to ``1142331976``. The hash code is then used to calculate a location in an internal array where the value will be stored. Assuming that you're storing keys that all have different hash values, this means that dictionaries take -constant time -- O(1), in Big-O notation -- to retrieve a key. +constant time -- *O*\ (1), in Big-O notation -- to retrieve a key. Why must dictionary keys be immutable? diff --git a/Doc/glossary.rst b/Doc/glossary.rst index a687086c8d2138..a5f01c4c466f0a 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -741,7 +741,7 @@ Glossary list A built-in Python :term:`sequence`. Despite its name it is more akin to an array in other languages than to a linked list since access to - elements is O(1). + elements is *O*\ (1). list comprehension A compact way to process all or part of the elements in a sequence and diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 0d7821e608ec98..346b740a8f757a 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -237,7 +237,7 @@ implementation used by the asyncio event loop: It works reliably even when the asyncio event loop is run in a non-main OS thread. - There is no noticeable overhead when handling a big number of children (*O(1)* each + There is no noticeable overhead when handling a big number of children (*O*\ (1) each time a child terminates), but starting a thread per process requires extra memory. This watcher is used by default. @@ -257,7 +257,7 @@ implementation used by the asyncio event loop: watcher is installed. The solution is safe but it has a significant overhead when - handling a big number of processes (*O(n)* each time a + handling a big number of processes (*O*\ (*n*) each time a :py:data:`SIGCHLD` is received). .. versionadded:: 3.8 @@ -273,7 +273,7 @@ implementation used by the asyncio event loop: The watcher avoids disrupting other code spawning processes by polling every process explicitly on a :py:data:`SIGCHLD` signal. - This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O(N)* + This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*) complexity but requires a running event loop in the main thread to work. .. deprecated:: 3.12 @@ -285,7 +285,7 @@ implementation used by the asyncio event loop: processes and waiting for their termination. There is no noticeable overhead when handling a big number of - children (*O(1)* each time a child terminates). + children (*O*\ (1) each time a child terminates). This solution requires a running event loop in the main thread to work, as :class:`SafeChildWatcher`. diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index c0923093c1cb06..31c79b91061591 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -79,7 +79,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -99,7 +99,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -115,7 +115,7 @@
[Python-checkins] [3.11] gh-113664: Improve style of Big O notation (GH-113695) (GH-113910)
https://github.com/python/cpython/commit/c92a473a71a0c395df57d31cd49900057da3c25b commit: c92a473a71a0c395df57d31cd49900057da3c25b branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2024-01-10T13:13:27Z summary: [3.11] gh-113664: Improve style of Big O notation (GH-113695) (GH-113910) Use cursive to make it looking like mathematic formulas. (cherry picked from commit a8629816c6c0e6770248a60529fd7c9ba08aad55) files: M Doc/faq/design.rst M Doc/glossary.rst M Doc/library/asyncio-policy.rst M Doc/library/bisect.rst M Doc/library/collections.rst M Doc/library/contextvars.rst M Doc/library/heapq.rst M Doc/library/select.rst M Doc/reference/datamodel.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.3.rst M Misc/NEWS.d/3.5.0a1.rst diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index 83c0152c85e84a..d0c136deabad76 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -449,7 +449,7 @@ on the key and a per-process seed; for example, "Python" could hash to to 1142331976. The hash code is then used to calculate a location in an internal array where the value will be stored. Assuming that you're storing keys that all have different hash values, this means that dictionaries take -constant time -- O(1), in Big-O notation -- to retrieve a key. +constant time -- *O*\ (1), in Big-O notation -- to retrieve a key. Why must dictionary keys be immutable? diff --git a/Doc/glossary.rst b/Doc/glossary.rst index ec6cec3acc7939..7dd178d5ad34f9 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -741,7 +741,7 @@ Glossary list A built-in Python :term:`sequence`. Despite its name it is more akin to an array in other languages than to a linked list since access to - elements is O(1). + elements is *O*\ (1). list comprehension A compact way to process all or part of the elements in a sequence and diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index f846f76ca095f4..3cd0f1e9d23cf3 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -226,7 +226,7 @@ implementation used by the asyncio event loop: It works reliably even when the asyncio event loop is run in a non-main OS thread. - There is no noticeable overhead when handling a big number of children (*O(1)* each + There is no noticeable overhead when handling a big number of children (*O*\ (1) each time a child terminates), but starting a thread per process requires extra memory. This watcher is used by default. @@ -246,7 +246,7 @@ implementation used by the asyncio event loop: watcher is installed. The solution is safe but it has a significant overhead when - handling a big number of processes (*O(n)* each time a + handling a big number of processes (*O*\ (*n*) each time a :py:data:`SIGCHLD` is received). .. versionadded:: 3.8 @@ -260,7 +260,7 @@ implementation used by the asyncio event loop: The watcher avoids disrupting other code spawning processes by polling every process explicitly on a :py:data:`SIGCHLD` signal. - This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O(N)* + This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*) complexity but requires a running event loop in the main thread to work. .. class:: FastChildWatcher @@ -270,7 +270,7 @@ implementation used by the asyncio event loop: processes and waiting for their termination. There is no noticeable overhead when handling a big number of - children (*O(1)* each time a child terminates). + children (*O*\ (1) each time a child terminates). This solution requires a running event loop in the main thread to work, as :class:`SafeChildWatcher`. diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 75d16dc3e1021c..92372f12ca4425 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -79,7 +79,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -99,7 +99,7 @@ The following functions are provided: To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step. - Keep in mind that the ``O(log n)`` search is dominated by the slow O(n) + Keep in mind that the *O*\ (log *n*) search is dominated by the slow *O*\ (*n*) insertion step. .. versionchanged:: 3.10 @@ -115,7 +115,7 @@ thoughts in mind: * Bisection is effective for searching ranges of values. For locating specific values, dictionaries are more performant. -*
[Python-checkins] [3.12] gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730) (GH-113907)
https://github.com/python/cpython/commit/84677ff19c0ffdc328203cdb25d16053e9c5a910 commit: 84677ff19c0ffdc328203cdb25d16053e9c5a910 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T13:21:50Z summary: [3.12] gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730) (GH-113907) It occurred when try to re-encode an unknown-8bit part combined with non-unknown-8bit part. (cherry picked from commit e9d5b6ea2d68564f176fdf70c2d7028e060c62b5) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index 0d6bd812475eea..5b653f66c18554 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -2766,6 +2766,7 @@ def _refold_parse_tree(parse_tree, *, policy): encoding = 'utf-8' if policy.utf8 else 'us-ascii' lines = [''] last_ew = None +last_charset = None wrap_as_ew_blocked = 0 want_encoding = False end_ew_not_allowed = Terminal('', 'wrap_as_ew_blocked') @@ -2820,8 +2821,14 @@ def _refold_parse_tree(parse_tree, *, policy): else: # It's a terminal, wrap it as an encoded word, possibly # combining it with previously encoded words if allowed. +if (last_ew is not None and +charset != last_charset and +(last_charset == 'unknown-8bit' or + last_charset == 'utf-8' and charset != 'us-ascii')): +last_ew = None last_ew = _fold_as_ew(tstr, lines, maxlen, last_ew, part.ew_combine_allowed, charset) +last_charset = charset want_encoding = False continue if len(tstr) <= maxlen - len(lines[-1]): diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 854f2ff009c618..bdb0e55f21069f 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -2915,6 +2915,45 @@ def test_ews_combined_before_wrap(self): "mich. And that's\n" " all I'm sayin.\n") +def test_unicode_after_unknown_not_combined(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=\xa4"), + "=?unknown-8bit?q?=A4?==?utf-8?q?=C2=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=A4?=\xa4"), + prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?=C2=A4?=\n") + +def test_ascii_after_unknown_not_combined(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=abc"), + "=?unknown-8bit?q?=A4?=abc\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=A4?=abc"), + prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?abc?=\n") + +def test_unknown_after_unicode_not_combined(self): +self._test(parser.get_unstructured("\xa4" + "=?unknown-8bit?q?=A4?="), + "=?utf-8?q?=C2=A4?==?unknown-8bit?q?=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "\xa4=?unknown-8bit?q?=A4?="), + prefix + "=?utf-8?q?=C2=A4?=\n =?unknown-8bit?q?=A4?=\n") + +def test_unknown_after_ascii_not_combined(self): +self._test(parser.get_unstructured("abc" + "=?unknown-8bit?q?=A4?="), + "abc=?unknown-8bit?q?=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "abcd=?unknown-8bit?q?=A4?="), + prefix + "abcd\n =?unknown-8bit?q?=A4?=\n") + +def test_unknown_after_unknown(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=C2?=" + "=?unknown-8bit?q?=A4?="), + "=?unknown-8bit?q?=C2=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=C2?=" + "=?unknown-8bit?q?=A4?="), + prefix + "=?unknown-8bit?q?=C2?=\n =?unknown-8bit?q?=A4?=\n") + # XXX Need test of an encoded word so long that it needs to be wrapped def test_simple_address(self): diff --git a/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst new file mode 100644 index 00..c71bc9c20e4596 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113
[Python-checkins] [3.11] gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730) (GH-113908)
https://github.com/python/cpython/commit/435e891b32318b4a0fed2843d0cb37ee21c07e4b commit: 435e891b32318b4a0fed2843d0cb37ee21c07e4b branch: 3.11 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T13:24:17Z summary: [3.11] gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730) (GH-113908) It occurred when try to re-encode an unknown-8bit part combined with non-unknown-8bit part. (cherry picked from commit e9d5b6ea2d68564f176fdf70c2d7028e060c62b5) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index e637e6df06612d..f4334f1fe69cbe 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -2768,6 +2768,7 @@ def _refold_parse_tree(parse_tree, *, policy): encoding = 'utf-8' if policy.utf8 else 'us-ascii' lines = [''] last_ew = None +last_charset = None wrap_as_ew_blocked = 0 want_encoding = False end_ew_not_allowed = Terminal('', 'wrap_as_ew_blocked') @@ -2822,8 +2823,14 @@ def _refold_parse_tree(parse_tree, *, policy): else: # It's a terminal, wrap it as an encoded word, possibly # combining it with previously encoded words if allowed. +if (last_ew is not None and +charset != last_charset and +(last_charset == 'unknown-8bit' or + last_charset == 'utf-8' and charset != 'us-ascii')): +last_ew = None last_ew = _fold_as_ew(tstr, lines, maxlen, last_ew, part.ew_combine_allowed, charset) +last_charset = charset want_encoding = False continue if len(tstr) <= maxlen - len(lines[-1]): diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 854f2ff009c618..bdb0e55f21069f 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -2915,6 +2915,45 @@ def test_ews_combined_before_wrap(self): "mich. And that's\n" " all I'm sayin.\n") +def test_unicode_after_unknown_not_combined(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=\xa4"), + "=?unknown-8bit?q?=A4?==?utf-8?q?=C2=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=A4?=\xa4"), + prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?=C2=A4?=\n") + +def test_ascii_after_unknown_not_combined(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=A4?=abc"), + "=?unknown-8bit?q?=A4?=abc\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=A4?=abc"), + prefix + "=?unknown-8bit?q?=A4?=\n =?utf-8?q?abc?=\n") + +def test_unknown_after_unicode_not_combined(self): +self._test(parser.get_unstructured("\xa4" + "=?unknown-8bit?q?=A4?="), + "=?utf-8?q?=C2=A4?==?unknown-8bit?q?=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "\xa4=?unknown-8bit?q?=A4?="), + prefix + "=?utf-8?q?=C2=A4?=\n =?unknown-8bit?q?=A4?=\n") + +def test_unknown_after_ascii_not_combined(self): +self._test(parser.get_unstructured("abc" + "=?unknown-8bit?q?=A4?="), + "abc=?unknown-8bit?q?=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "abcd=?unknown-8bit?q?=A4?="), + prefix + "abcd\n =?unknown-8bit?q?=A4?=\n") + +def test_unknown_after_unknown(self): +self._test(parser.get_unstructured("=?unknown-8bit?q?=C2?=" + "=?unknown-8bit?q?=A4?="), + "=?unknown-8bit?q?=C2=A4?=\n") +prefix = "0123456789 "*5 +self._test(parser.get_unstructured(prefix + "=?unknown-8bit?q?=C2?=" + "=?unknown-8bit?q?=A4?="), + prefix + "=?unknown-8bit?q?=C2?=\n =?unknown-8bit?q?=A4?=\n") + # XXX Need test of an encoded word so long that it needs to be wrapped def test_simple_address(self): diff --git a/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113594.4t8HiR.rst new file mode 100644 index 00..c71bc9c20e4596 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-05-12-42-07.gh-issue-113
[Python-checkins] [3.12] gh-70835: Clarify error message for CSV file opened with wrong newline (GH-113786) (GH-113905)
https://github.com/python/cpython/commit/59a7b9495b64f0be8992564d30217137f0f0a662 commit: 59a7b9495b64f0be8992564d30217137f0f0a662 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka date: 2024-01-10T13:26:04Z summary: [3.12] gh-70835: Clarify error message for CSV file opened with wrong newline (GH-113786) (GH-113905) Based on patch by SilentGhost. (cherry picked from commit 568d220993fa9b4b812ff1b425edd80dbe17dda9) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_csv.py M Modules/_csv.c diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py index bc9961e0831f0e..30383698d04184 100644 --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -309,13 +309,18 @@ def test_read_oddinputs(self): [b'abc'], None) def test_read_eol(self): -self._read_test(['a,b'], [['a','b']]) -self._read_test(['a,b\n'], [['a','b']]) -self._read_test(['a,b\r\n'], [['a','b']]) -self._read_test(['a,b\r'], [['a','b']]) -self.assertRaises(csv.Error, self._read_test, ['a,b\rc,d'], []) -self.assertRaises(csv.Error, self._read_test, ['a,b\nc,d'], []) -self.assertRaises(csv.Error, self._read_test, ['a,b\r\nc,d'], []) +self._read_test(['a,b', 'c,d'], [['a','b'], ['c','d']]) +self._read_test(['a,b\n', 'c,d\n'], [['a','b'], ['c','d']]) +self._read_test(['a,b\r\n', 'c,d\r\n'], [['a','b'], ['c','d']]) +self._read_test(['a,b\r', 'c,d\r'], [['a','b'], ['c','d']]) + +errmsg = "with newline=''" +with self.assertRaisesRegex(csv.Error, errmsg): +next(csv.reader(['a,b\rc,d'])) +with self.assertRaisesRegex(csv.Error, errmsg): +next(csv.reader(['a,b\nc,d'])) +with self.assertRaisesRegex(csv.Error, errmsg): +next(csv.reader(['a,b\r\nc,d'])) def test_read_eof(self): self._read_test(['a,"'], [['a', '']]) diff --git a/Modules/_csv.c b/Modules/_csv.c index 9ab2ad266c2739..91cb63628a1f7b 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -841,7 +841,8 @@ parse_process_char(ReaderObj *self, _csvstate *module_state, Py_UCS4 c) self->state = START_RECORD; else { PyErr_Format(module_state->error_obj, - "new-line character seen in unquoted field - do you need to open the file in universal-newline mode?"); + "new-line character seen in unquoted field - " + "do you need to open the file with newline=''?"); return -1; } break; ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-89850: Add default C implementations of persistent_id() and persistent_load() (GH-113579)
https://github.com/python/cpython/commit/89cee94b315c88d3cd4c9ffc051e7abd6a5f2196
commit: 89cee94b315c88d3cd4c9ffc051e7abd6a5f2196
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:30:37+02:00
summary:
gh-89850: Add default C implementations of persistent_id() and
persistent_load() (GH-113579)
Previously the C implementation of pickle.Pickler and pickle.Unpickler
classes did not have such methods and they could only be used if
they were overloaded in subclasses or set as instance attributes.
Fixed calling super().persistent_id() and super().persistent_load() in
subclasses of the C implementation of pickle.Pickler and pickle.Unpickler
classes. It no longer causes an infinite recursion.
files:
A Misc/NEWS.d/next/Library/2023-12-29-22-29-34.gh-issue-89850.KnxiZA.rst
M Doc/library/pickle.rst
M Lib/test/test_pickle.py
M Modules/_pickle.c
M Modules/clinic/_pickle.c.h
diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst
index 93387fb0b45038..cfb251fca5c7cd 100644
--- a/Doc/library/pickle.rst
+++ b/Doc/library/pickle.rst
@@ -345,6 +345,10 @@ The :mod:`pickle` module exports three classes,
:class:`Pickler`,
See :ref:`pickle-persistent` for details and examples of uses.
+ .. versionchanged:: 3.13
+ Add the default implementation of this method in the C implementation
+ of :class:`!Pickler`.
+
.. attribute:: dispatch_table
A pickler object's dispatch table is a registry of *reduction
@@ -446,6 +450,10 @@ The :mod:`pickle` module exports three classes,
:class:`Pickler`,
See :ref:`pickle-persistent` for details and examples of uses.
+ .. versionchanged:: 3.13
+ Add the default implementation of this method in the C implementation
+ of :class:`!Unpickler`.
+
.. method:: find_class(module, name)
Import *module* if necessary and return the object called *name* from it,
diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py
index 1a55da39bdc58d..f6405d6dd44ef6 100644
--- a/Lib/test/test_pickle.py
+++ b/Lib/test/test_pickle.py
@@ -122,6 +122,7 @@ class
PyIdPersPicklerTests(AbstractIdentityPersistentPicklerTests,
pickler = pickle._Pickler
unpickler = pickle._Unpickler
+persistent_load_error = pickle.UnpicklingError
@support.cpython_only
def test_pickler_reference_cycle(self):
@@ -176,7 +177,6 @@ class DispatchTable:
support.gc_collect()
self.assertIsNone(table_ref())
-
@support.cpython_only
def test_unpickler_reference_cycle(self):
def check(Unpickler):
@@ -206,6 +206,28 @@ def persistent_load(pid):
return pid
check(PersUnpickler)
+def test_pickler_super(self):
+class PersPickler(self.pickler):
+def persistent_id(subself, obj):
+self.assertIsNone(super().persistent_id(obj))
+return obj
+
+for proto in range(pickle.HIGHEST_PROTOCOL + 1):
+f = io.BytesIO()
+pickler = PersPickler(f, proto)
+pickler.dump('abc')
+self.assertEqual(self.loads(f.getvalue()), 'abc')
+
+def test_unpickler_super(self):
+class PersUnpickler(self.unpickler):
+def persistent_load(subself, pid):
+with self.assertRaises(self.persistent_load_error):
+super().persistent_load(pid)
+return pid
+
+for proto in range(pickle.HIGHEST_PROTOCOL + 1):
+unpickler = PersUnpickler(io.BytesIO(self.dumps('abc', proto)))
+self.assertEqual(unpickler.load(), 'abc')
class PyPicklerUnpicklerObjectTests(AbstractPicklerUnpicklerObjectTests,
unittest.TestCase):
@@ -256,6 +278,7 @@ class CPersPicklerTests(PyPersPicklerTests):
class CIdPersPicklerTests(PyIdPersPicklerTests):
pickler = _pickle.Pickler
unpickler = _pickle.Unpickler
+persistent_load_error = _pickle.UnpicklingError
class CDumpPickle_LoadPickle(PyPicklerTests):
pickler = _pickle.Pickler
@@ -326,7 +349,7 @@ class SizeofTests(unittest.TestCase):
check_sizeof = support.check_sizeof
def test_pickler(self):
-basesize = support.calcobjsize('7P2n3i2n3i2P')
+basesize = support.calcobjsize('6P2n3i2n3i2P')
p = _pickle.Pickler(io.BytesIO())
self.assertEqual(object.__sizeof__(p), basesize)
MT_size = struct.calcsize('3nP0n')
@@ -343,7 +366,7 @@ def test_pickler(self):
0) # Write buffer is cleared after every dump().
def test_unpickler(self):
-basesize = support.calcobjsize('2P2n2P 2P2n2i5P 2P3n8P2n2i')
+basesize = support.calcobjsize('2P2nP 2P2n2i5P 2P3n8P2n2i')
unpickler = _pickle.Unpickler
P = struct.calcsize('P') # Size of memo table entry.
n = struct.calcsize('n') # Size of mark table entry.
diff --git
a/Misc/NEWS.d/next/Library/2023-12-29-22-2
[Python-checkins] gh-66515: Fix locking of an MH mailbox without ".mh_sequences" file (GH-113482)
https://github.com/python/cpython/commit/be5e65fdf67b1817e173e73443564c7c146b09a4
commit: be5e65fdf67b1817e173e73443564c7c146b09a4
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:31:55+02:00
summary:
gh-66515: Fix locking of an MH mailbox without ".mh_sequences" file (GH-113482)
Guarantee that it either open an existing ".mh_sequences" file or create
a new ".mh_sequences" file, but do not replace existing ".mh_sequences"
file.
files:
M Lib/mailbox.py
M Lib/test/test_mailbox.py
diff --git a/Lib/mailbox.py b/Lib/mailbox.py
index 0e1d49b399d077..81ea210cf815a4 100644
--- a/Lib/mailbox.py
+++ b/Lib/mailbox.py
@@ -1141,10 +1141,24 @@ def __len__(self):
"""Return a count of messages in the mailbox."""
return len(list(self.iterkeys()))
+def _open_mh_sequences_file(self, text):
+mode = '' if text else 'b'
+kwargs = {'encoding': 'ASCII'} if text else {}
+path = os.path.join(self._path, '.mh_sequences')
+while True:
+try:
+return open(path, 'r+' + mode, **kwargs)
+except FileNotFoundError:
+pass
+try:
+return open(path, 'x+' + mode, **kwargs)
+except FileExistsError:
+pass
+
def lock(self):
"""Lock the mailbox."""
if not self._locked:
-self._file = open(os.path.join(self._path, '.mh_sequences'), 'rb+')
+self._file = self._open_mh_sequences_file(text=False)
_lock_file(self._file)
self._locked = True
@@ -1225,8 +1239,9 @@ def get_sequences(self):
def set_sequences(self, sequences):
"""Set sequences using the given name-to-key-list dictionary."""
-f = open(os.path.join(self._path, '.mh_sequences'), 'w',
encoding='ASCII')
+f = self._open_mh_sequences_file(text=True)
try:
+os.close(os.open(f.name, os.O_WRONLY | os.O_TRUNC))
for name, keys in sequences.items():
if len(keys) == 0:
continue
diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
index 8c350eb02ccc17..d84faad0eb3406 100644
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -1360,6 +1360,15 @@ def test_no_dot_mh_sequences_file(self):
box.set_sequences({})
self.assertEqual(os.listdir(path), ['.mh_sequences'])
+def test_lock_unlock_no_dot_mh_sequences_file(self):
+path = os.path.join(self._path, 'foo.bar')
+os.mkdir(path)
+box = self._factory(path)
+self.assertEqual(os.listdir(path), [])
+box.lock()
+box.unlock()
+self.assertEqual(os.listdir(path), ['.mh_sequences'])
+
def test_issue2625(self):
msg0 = mailbox.MHMessage(self._template % 0)
msg0.add_sequence('foo')
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-111789: Use PyDict_GetItemRef() in Modules/_zoneinfo.c (GH-112078)
https://github.com/python/cpython/commit/183b97bb9db075197153ad82b8ffdfce8e913250
commit: 183b97bb9db075197153ad82b8ffdfce8e913250
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:35:10+02:00
summary:
gh-111789: Use PyDict_GetItemRef() in Modules/_zoneinfo.c (GH-112078)
files:
M Modules/_zoneinfo.c
diff --git a/Modules/_zoneinfo.c b/Modules/_zoneinfo.c
index 77644c3155bc33..fcd4af64df0be9 100644
--- a/Modules/_zoneinfo.c
+++ b/Modules/_zoneinfo.c
@@ -853,28 +853,19 @@ load_timedelta(zoneinfo_state *state, long seconds)
if (pyoffset == NULL) {
return NULL;
}
-rv = PyDict_GetItemWithError(state->TIMEDELTA_CACHE, pyoffset);
-if (rv == NULL) {
-if (PyErr_Occurred()) {
-goto error;
-}
+if (PyDict_GetItemRef(state->TIMEDELTA_CACHE, pyoffset, &rv) == 0) {
PyObject *tmp = PyDateTimeAPI->Delta_FromDelta(
0, seconds, 0, 1, PyDateTimeAPI->DeltaType);
-if (tmp == NULL) {
-goto error;
+if (tmp != NULL) {
+rv = PyDict_SetDefault(state->TIMEDELTA_CACHE, pyoffset, tmp);
+Py_XINCREF(rv);
+Py_DECREF(tmp);
}
-
-rv = PyDict_SetDefault(state->TIMEDELTA_CACHE, pyoffset, tmp);
-Py_DECREF(tmp);
}
-Py_XINCREF(rv);
Py_DECREF(pyoffset);
return rv;
-error:
-Py_DECREF(pyoffset);
-return NULL;
}
/* Constructor for _ttinfo object - this starts by initializing the _ttinfo
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016)
https://github.com/python/cpython/commit/66363b9a7b9fe7c99eba3a185b74c5fdbf842eba
commit: 66363b9a7b9fe7c99eba3a185b74c5fdbf842eba
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:55:36+02:00
summary:
gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016)
Raise BadZipFile when try to read an entry that overlaps with other entry or
central directory.
files:
A Misc/NEWS.d/next/Library/2023-09-28-13-15-51.gh-issue-109858.43e2dg.rst
M Lib/test/test_zipfile/test_core.py
M Lib/zipfile/__init__.py
diff --git a/Lib/test/test_zipfile/test_core.py
b/Lib/test/test_zipfile/test_core.py
index a51764b9297363..41ce81a9d08c4b 100644
--- a/Lib/test/test_zipfile/test_core.py
+++ b/Lib/test/test_zipfile/test_core.py
@@ -2272,6 +2272,64 @@ def test_decompress_without_3rd_party_library(self):
with zipfile.ZipFile(zip_file) as zf:
self.assertRaises(RuntimeError, zf.extract, 'a.txt')
+def test_full_overlap(self):
+data = (
+b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2\x1e'
+b'8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00a\xed'
+b'\xc0\x81\x08\x00\x00\x00\xc00\xd6\xfbK\\d\x0b`P'
+b'K\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2'
+b'\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00\x00'
+b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00aPK'
+b'\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2\x1e'
+b'8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00\x00\x00'
+b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00bPK\x05'
+b'\x06\x00\x00\x00\x00\x02\x00\x02\x00^\x00\x00\x00/\x00\x00'
+b'\x00\x00\x00'
+)
+with zipfile.ZipFile(io.BytesIO(data), 'r') as zipf:
+self.assertEqual(zipf.namelist(), ['a', 'b'])
+zi = zipf.getinfo('a')
+self.assertEqual(zi.header_offset, 0)
+self.assertEqual(zi.compress_size, 16)
+self.assertEqual(zi.file_size, 1033)
+zi = zipf.getinfo('b')
+self.assertEqual(zi.header_offset, 0)
+self.assertEqual(zi.compress_size, 16)
+self.assertEqual(zi.file_size, 1033)
+self.assertEqual(len(zipf.read('a')), 1033)
+with self.assertRaisesRegex(zipfile.BadZipFile, 'File
name.*differ'):
+zipf.read('b')
+
+def test_quoted_overlap(self):
+data = (
+b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05Y\xfc'
+b'8\x044\x00\x00\x00(\x04\x00\x00\x01\x00\x00\x00a\x00'
+b'\x1f\x00\xe0\xffPK\x03\x04\x14\x00\x00\x00\x08\x00\xa0l'
+b'H\x05\xe2\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00'
+b'\x00\x00b\xed\xc0\x81\x08\x00\x00\x00\xc00\xd6\xfbK\\'
+b'd\x0b`PK\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0'
+b'lH\x05Y\xfc8\x044\x00\x00\x00(\x04\x00\x00\x01'
+b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+b'\x00aPK\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0l'
+b'H\x05\xe2\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00'
+b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00$\x00\x00\x00'
+b'bPK\x05\x06\x00\x00\x00\x00\x02\x00\x02\x00^\x00\x00'
+b'\x00S\x00\x00\x00\x00\x00'
+)
+with zipfile.ZipFile(io.BytesIO(data), 'r') as zipf:
+self.assertEqual(zipf.namelist(), ['a', 'b'])
+zi = zipf.getinfo('a')
+self.assertEqual(zi.header_offset, 0)
+self.assertEqual(zi.compress_size, 52)
+self.assertEqual(zi.file_size, 1064)
+zi = zipf.getinfo('b')
+self.assertEqual(zi.header_offset, 36)
+self.assertEqual(zi.compress_size, 16)
+self.assertEqual(zi.file_size, 1033)
+with self.assertRaisesRegex(zipfile.BadZipFile, 'Overlapped
entries'):
+zipf.read('a')
+self.assertEqual(len(zipf.read('b')), 1033)
+
def tearDown(self):
unlink(TESTFN)
unlink(TESTFN2)
diff --git a/Lib/zipfile/__init__.py b/Lib/zipfile/__init__.py
index 1c415a2eb7bc09..1d8a607fc728c8 100644
--- a/Lib/zipfile/__init__.py
+++ b/Lib/zipfile/__init__.py
@@ -395,6 +395,7 @@ class ZipInfo (object):
'compress_size',
'file_size',
'_raw_time',
+'_end_offset',
)
def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)):
@@ -429,6 +430,7 @@ def __init__(self, filename="NoName",
date_time=(1980,1,1,0,0,0)):
self.external_attr = 0 # External file attributes
self.compress_size = 0 # Size of the compressed file
self.file_size = 0 # Size of the uncompressed file
+self._end_offset = None # Start of the next local header or
central directory
# Other attributes are set by class ZipFile:
[Python-checkins] gh-111139: Optimize math.gcd(int, int) (#113887)
https://github.com/python/cpython/commit/93930eaf0acd64dc0d08d58321d2682cb019bc1a
commit: 93930eaf0acd64dc0d08d58321d2682cb019bc1a
branch: main
author: Victor Stinner
committer: vstinner
date: 2024-01-10T16:38:56+01:00
summary:
gh-39: Optimize math.gcd(int, int) (#113887)
Add a fast-path for the common case.
Benchmark:
python -m pyperf timeit \
-s 'import math; gcd=math.gcd; x=2*3; y=3*5' \
'gcd(x,y)'
Result: 1.07x faster (-3.4 ns)
Mean +- std dev: 52.6 ns +- 4.0 ns -> 49.2 ns +- 0.4 ns: 1.07x faster
files:
M Modules/mathmodule.c
diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c
index 6cd61e9ab75424..2a796c1c55d2f0 100644
--- a/Modules/mathmodule.c
+++ b/Modules/mathmodule.c
@@ -759,13 +759,17 @@ m_log10(double x)
static PyObject *
math_gcd(PyObject *module, PyObject * const *args, Py_ssize_t nargs)
{
-PyObject *res, *x;
-Py_ssize_t i;
+// Fast-path for the common case: gcd(int, int)
+if (nargs == 2 && PyLong_CheckExact(args[0]) && PyLong_CheckExact(args[1]))
+{
+return _PyLong_GCD(args[0], args[1]);
+}
if (nargs == 0) {
return PyLong_FromLong(0);
}
-res = PyNumber_Index(args[0]);
+
+PyObject *res = PyNumber_Index(args[0]);
if (res == NULL) {
return NULL;
}
@@ -775,8 +779,8 @@ math_gcd(PyObject *module, PyObject * const *args,
Py_ssize_t nargs)
}
PyObject *one = _PyLong_GetOne(); // borrowed ref
-for (i = 1; i < nargs; i++) {
-x = _PyNumber_Index(args[i]);
+for (Py_ssize_t i = 1; i < nargs; i++) {
+PyObject *x = _PyNumber_Index(args[i]);
if (x == NULL) {
Py_DECREF(res);
return NULL;
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] GH-113860: All executors are now defined in terms of micro ops. Convert counter executor to use uops. (GH-113864)
https://github.com/python/cpython/commit/a0c9cf9456c2ee7a89d9bd859c07afac8cf5e893
commit: a0c9cf9456c2ee7a89d9bd859c07afac8cf5e893
branch: main
author: Mark Shannon
committer: markshannon
date: 2024-01-10T15:44:34Z
summary:
GH-113860: All executors are now defined in terms of micro ops. Convert counter
executor to use uops. (GH-113864)
files:
M Include/cpython/optimizer.h
M Include/internal/pycore_opcode_metadata.h
M Include/internal/pycore_uop_ids.h
M Include/internal/pycore_uop_metadata.h
M Include/internal/pycore_uops.h
M Python/bytecodes.c
M Python/executor_cases.c.h
M Python/generated_cases.c.h
M Python/optimizer.c
diff --git a/Include/cpython/optimizer.h b/Include/cpython/optimizer.h
index d521eac79d1b97..f077da7ee88456 100644
--- a/Include/cpython/optimizer.h
+++ b/Include/cpython/optimizer.h
@@ -31,8 +31,6 @@ typedef struct {
typedef struct _PyExecutorObject {
PyObject_VAR_HEAD
-/* WARNING: execute consumes a reference to self. This is necessary to
allow executors to tail call into each other. */
-_Py_CODEUNIT *(*execute)(struct _PyExecutorObject *self, struct
_PyInterpreterFrame *frame, PyObject **stack_pointer);
_PyVMData vm_data; /* Used by the VM, but opaque to the optimizer */
/* Data needed by the executor goes here, but is opaque to the VM */
} _PyExecutorObject;
@@ -52,6 +50,12 @@ typedef struct _PyOptimizerObject {
/* Data needed by the optimizer goes here, but is opaque to the VM */
} _PyOptimizerObject;
+/** Test support **/
+typedef struct {
+_PyOptimizerObject base;
+int64_t count;
+} _PyCounterOptimizerObject;
+
PyAPI_FUNC(int) PyUnstable_Replace_Executor(PyCodeObject *code, _Py_CODEUNIT
*instr, _PyExecutorObject *executor);
PyAPI_FUNC(void) PyUnstable_SetOptimizer(_PyOptimizerObject* optimizer);
diff --git a/Include/internal/pycore_opcode_metadata.h
b/Include/internal/pycore_opcode_metadata.h
index 7d39e4bc03099c..a9d698da25a1db 100644
--- a/Include/internal/pycore_opcode_metadata.h
+++ b/Include/internal/pycore_opcode_metadata.h
@@ -1009,7 +1009,7 @@ const struct opcode_metadata
_PyOpcode_opcode_metadata[268] = {
[END_ASYNC_FOR] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ESCAPES_FLAG
},
[END_FOR] = { true, INSTR_FMT_IX, 0 },
[END_SEND] = { true, INSTR_FMT_IX, 0 },
-[ENTER_EXECUTOR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG |
HAS_EVAL_BREAK_FLAG | HAS_ERROR_FLAG | HAS_ESCAPES_FLAG },
+[ENTER_EXECUTOR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG |
HAS_EVAL_BREAK_FLAG },
[EXIT_INIT_CHECK] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG |
HAS_ESCAPES_FLAG },
[EXTENDED_ARG] = { true, INSTR_FMT_IB, HAS_ARG_FLAG },
[FORMAT_SIMPLE] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ESCAPES_FLAG
},
diff --git a/Include/internal/pycore_uop_ids.h
b/Include/internal/pycore_uop_ids.h
index 4a9a00ba352d33..b3b36327c480cc 100644
--- a/Include/internal/pycore_uop_ids.h
+++ b/Include/internal/pycore_uop_ids.h
@@ -231,7 +231,9 @@ extern "C" {
#define _SAVE_RETURN_OFFSET 378
#define _INSERT 379
#define _CHECK_VALIDITY 380
-#define MAX_UOP_ID 380
+#define _LOAD_CONST_INLINE_BORROW 381
+#define _INTERNAL_INCREMENT_OPT_COUNTER 382
+#define MAX_UOP_ID 382
#ifdef __cplusplus
}
diff --git a/Include/internal/pycore_uop_metadata.h
b/Include/internal/pycore_uop_metadata.h
index 300bd3baa7b377..ab498e9cefde22 100644
--- a/Include/internal/pycore_uop_metadata.h
+++ b/Include/internal/pycore_uop_metadata.h
@@ -203,6 +203,8 @@ const uint16_t _PyUop_Flags[MAX_UOP_ID+1] = {
[_EXIT_TRACE] = HAS_DEOPT_FLAG,
[_INSERT] = HAS_ARG_FLAG,
[_CHECK_VALIDITY] = HAS_DEOPT_FLAG,
+[_LOAD_CONST_INLINE_BORROW] = 0,
+[_INTERNAL_INCREMENT_OPT_COUNTER] = 0,
};
const char *const _PyOpcode_uop_name[MAX_UOP_ID+1] = {
@@ -303,6 +305,7 @@ const char *const _PyOpcode_uop_name[MAX_UOP_ID+1] = {
[_INIT_CALL_BOUND_METHOD_EXACT_ARGS] =
"_INIT_CALL_BOUND_METHOD_EXACT_ARGS",
[_INIT_CALL_PY_EXACT_ARGS] = "_INIT_CALL_PY_EXACT_ARGS",
[_INSERT] = "_INSERT",
+[_INTERNAL_INCREMENT_OPT_COUNTER] = "_INTERNAL_INCREMENT_OPT_COUNTER",
[_IS_NONE] = "_IS_NONE",
[_IS_OP] = "_IS_OP",
[_ITER_CHECK_LIST] = "_ITER_CHECK_LIST",
@@ -328,6 +331,7 @@ const char *const _PyOpcode_uop_name[MAX_UOP_ID+1] = {
[_LOAD_ATTR_WITH_HINT] = "_LOAD_ATTR_WITH_HINT",
[_LOAD_BUILD_CLASS] = "_LOAD_BUILD_CLASS",
[_LOAD_CONST] = "_LOAD_CONST",
+[_LOAD_CONST_INLINE_BORROW] = "_LOAD_CONST_INLINE_BORROW",
[_LOAD_DEREF] = "_LOAD_DEREF",
[_LOAD_FAST] = "_LOAD_FAST",
[_LOAD_FAST_AND_CLEAR] = "_LOAD_FAST_AND_CLEAR",
diff --git a/Include/internal/pycore_uops.h b/Include/internal/pycore_uops.h
index 153884f4bd2902..eb10002d34ce51 100644
--- a/Include/internal/pycore_uops.h
+++ b/Include/internal/pycore_uops.h
@@ -24,11 +24,6 @@ typedef struct {
_PyUOpInstruction trace[1];
} _PyUOpExecutorObject;
-_Py_CODEUNIT *_PyUOpExecute(
-_PyExecutorObject *executor,
-_PyInterpreterFrame *frame,
-PyObject **sta
[Python-checkins] gh-111968: Use per-thread freelists for float in free-threading (gh-113886)
https://github.com/python/cpython/commit/f728f7242c6008a16daaa5dde8e1db786857c50e
commit: f728f7242c6008a16daaa5dde8e1db786857c50e
branch: main
author: Donghee Na
committer: corona10
date: 2024-01-10T15:47:13Z
summary:
gh-111968: Use per-thread freelists for float in free-threading (gh-113886)
files:
M Include/internal/pycore_floatobject.h
M Include/internal/pycore_freelist.h
M Include/internal/pycore_gc.h
M Include/internal/pycore_interp.h
M Objects/floatobject.c
M Python/gc_free_threading.c
M Python/gc_gil.c
M Python/pylifecycle.c
M Python/pystate.c
diff --git a/Include/internal/pycore_floatobject.h
b/Include/internal/pycore_floatobject.h
index 4e5474841bc25d..038578e1f9680a 100644
--- a/Include/internal/pycore_floatobject.h
+++ b/Include/internal/pycore_floatobject.h
@@ -8,14 +8,14 @@ extern "C" {
# error "this header requires Py_BUILD_CORE define"
#endif
-
+#include "pycore_freelist.h" // _PyFreeListState
#include "pycore_unicodeobject.h" // _PyUnicodeWriter
/* runtime lifecycle */
extern void _PyFloat_InitState(PyInterpreterState *);
extern PyStatus _PyFloat_InitTypes(PyInterpreterState *);
-extern void _PyFloat_Fini(PyInterpreterState *);
+extern void _PyFloat_Fini(_PyFreeListState *);
extern void _PyFloat_FiniType(PyInterpreterState *);
@@ -33,24 +33,7 @@ struct _Py_float_runtime_state {
};
-#ifndef WITH_FREELISTS
-// without freelists
-# define PyFloat_MAXFREELIST 0
-#endif
-
-#ifndef PyFloat_MAXFREELIST
-# define PyFloat_MAXFREELIST 100
-#endif
-struct _Py_float_state {
-#if PyFloat_MAXFREELIST > 0
-/* Special free list
- free_list is a singly-linked list of available PyFloatObjects,
- linked via abuse of their ob_type members. */
-int numfree;
-PyFloatObject *free_list;
-#endif
-};
void _PyFloat_ExactDealloc(PyObject *op);
diff --git a/Include/internal/pycore_freelist.h
b/Include/internal/pycore_freelist.h
index b725986528d864..d9619218b8dada 100644
--- a/Include/internal/pycore_freelist.h
+++ b/Include/internal/pycore_freelist.h
@@ -8,24 +8,34 @@ extern "C" {
# error "this header requires Py_BUILD_CORE define"
#endif
-#ifndef WITH_FREELISTS
-// without freelists
-# define PyList_MAXFREELIST 0
-#endif
-
-/* Empty list reuse scheme to save calls to malloc and free */
-#ifndef PyList_MAXFREELIST
+#ifdef WITH_FREELISTS
+// with freelists
# define PyList_MAXFREELIST 80
+# define PyFloat_MAXFREELIST 100
+#else
+# define PyList_MAXFREELIST 0
+# define PyFloat_MAXFREELIST 0
#endif
struct _Py_list_state {
-#if PyList_MAXFREELIST > 0
+#ifdef WITH_FREELISTS
PyListObject *free_list[PyList_MAXFREELIST];
int numfree;
#endif
};
+struct _Py_float_state {
+#ifdef WITH_FREELISTS
+/* Special free list
+ free_list is a singly-linked list of available PyFloatObjects,
+ linked via abuse of their ob_type members. */
+int numfree;
+PyFloatObject *free_list;
+#endif
+};
+
typedef struct _Py_freelist_state {
+struct _Py_float_state float_state;
struct _Py_list_state list;
} _PyFreeListState;
diff --git a/Include/internal/pycore_gc.h b/Include/internal/pycore_gc.h
index 5d90d3a7f865da..2a0730eebb8db7 100644
--- a/Include/internal/pycore_gc.h
+++ b/Include/internal/pycore_gc.h
@@ -243,7 +243,7 @@ extern PyObject *_PyGC_GetReferrers(PyInterpreterState
*interp, PyObject *objs);
extern void _PyGC_ClearAllFreeLists(PyInterpreterState *interp);
extern void _Py_ClearFreeLists(_PyFreeListState *state, int is_finalization);
extern void _PyTuple_ClearFreeList(PyInterpreterState *interp);
-extern void _PyFloat_ClearFreeList(PyInterpreterState *interp);
+extern void _PyFloat_ClearFreeList(_PyFreeListState *state, int
is_finalization);
extern void _PyList_ClearFreeList(_PyFreeListState *state, int
is_finalization);
extern void _PyDict_ClearFreeList(PyInterpreterState *interp);
extern void _PyAsyncGen_ClearFreeLists(PyInterpreterState *interp);
diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h
index 4d49fa2a51b88c..dadc8e3b91a75d 100644
--- a/Include/internal/pycore_interp.h
+++ b/Include/internal/pycore_interp.h
@@ -184,7 +184,6 @@ struct _is {
#endif
struct _py_object_state object_state;
struct _Py_unicode_state unicode;
-struct _Py_float_state float_state;
struct _Py_long_state long_state;
struct _dtoa_state dtoa;
struct _py_func_state func_state;
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
index 364cf1553bb5d4..f1a09c0a94f4a6 100644
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -26,17 +26,13 @@ class float "PyObject *" "&PyFloat_Type"
#include "clinic/floatobject.c.h"
-#ifndef PyFloat_MAXFREELIST
-# define PyFloat_MAXFREELIST 100
-#endif
-
-
-#if PyFloat_MAXFREELIST > 0
+#ifdef WITH_FREELISTS
static struct _Py_float_state *
get_float_state(void)
{
-PyInterpreterState *interp = _PyInterpreterState_GET();
-return &interp->float_state;
+_PyFreeListState *state = _PyFreeListState_GET
[Python-checkins] Add @requires_zlib() decorator for gh-109858 tests (GH-113918)
https://github.com/python/cpython/commit/70497218351ba44bffc8b571201ecb5652d84675
commit: 70497218351ba44bffc8b571201ecb5652d84675
branch: main
author: Serhiy Storchaka
committer: serhiy-storchaka
date: 2024-01-10T15:56:40Z
summary:
Add @requires_zlib() decorator for gh-109858 tests (GH-113918)
files:
M Lib/test/test_zipfile/test_core.py
diff --git a/Lib/test/test_zipfile/test_core.py
b/Lib/test/test_zipfile/test_core.py
index 41ce81a9d08c4b..f7b6db465b4bc7 100644
--- a/Lib/test/test_zipfile/test_core.py
+++ b/Lib/test/test_zipfile/test_core.py
@@ -2272,6 +2272,7 @@ def test_decompress_without_3rd_party_library(self):
with zipfile.ZipFile(zip_file) as zf:
self.assertRaises(RuntimeError, zf.extract, 'a.txt')
+@requires_zlib()
def test_full_overlap(self):
data = (
b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2\x1e'
@@ -2300,6 +2301,7 @@ def test_full_overlap(self):
with self.assertRaisesRegex(zipfile.BadZipFile, 'File
name.*differ'):
zipf.read('b')
+@requires_zlib()
def test_quoted_overlap(self):
data = (
b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05Y\xfc'
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-113625: Align object addresses in the Descriptor HowTo Guide (#113894)
https://github.com/python/cpython/commit/901a971e161e060bd95f3cf3aeebe8b48d6e6dac commit: 901a971e161e060bd95f3cf3aeebe8b48d6e6dac branch: main author: Raymond Hettinger committer: ambv date: 2024-01-10T17:23:40+01:00 summary: gh-113625: Align object addresses in the Descriptor HowTo Guide (#113894) files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 87274a5133d1cf..75346f2c7618c2 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -1250,7 +1250,7 @@ instance:: >>> d.f.__self__ -<__main__.D object at 0x1012e1f98> +<__main__.D object at 0x00B18C90> If you have ever wondered where *self* comes from in regular methods or where *cls* comes from in class methods, this is it! ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.11] gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (#113923)
https://github.com/python/cpython/commit/6e21c590f6c68dace297c82c62d0a1286a2ab066 commit: 6e21c590f6c68dace297c82c62d0a1286a2ab066 branch: 3.11 author: Miss Islington (bot) <[email protected]> committer: ambv date: 2024-01-10T18:32:52+01:00 summary: [3.11] gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (#113923) gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (cherry picked from commit 901a971e161e060bd95f3cf3aeebe8b48d6e6dac) Co-authored-by: Raymond Hettinger files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 2469911569224a..308aa84724e2ed 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -1203,7 +1203,7 @@ instance:: >>> d.f.__self__ -<__main__.D object at 0x1012e1f98> +<__main__.D object at 0x00B18C90> If you have ever wondered where *self* comes from in regular methods or where *cls* comes from in class methods, this is it! ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (#113922)
https://github.com/python/cpython/commit/3fc45e6c4e31fe817e91ebeeb5f7dc8afc6c6c13 commit: 3fc45e6c4e31fe817e91ebeeb5f7dc8afc6c6c13 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: ambv date: 2024-01-10T18:32:54+01:00 summary: [3.12] gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (#113922) gh-113625: Align object addresses in the Descriptor HowTo Guide (GH-113894) (cherry picked from commit 901a971e161e060bd95f3cf3aeebe8b48d6e6dac) Co-authored-by: Raymond Hettinger files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 41ace7c208d2c9..1346b0d13514c2 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -1240,7 +1240,7 @@ instance:: >>> d.f.__self__ -<__main__.D object at 0x1012e1f98> +<__main__.D object at 0x00B18C90> If you have ever wondered where *self* comes from in regular methods or where *cls* comes from in class methods, this is it! ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113753: Clear finalized bit when putting PyAsyncGenASend back into free list (#113754)
https://github.com/python/cpython/commit/73ae2023a76f199ff854f8da14bd9ff8e93ee7fd
commit: 73ae2023a76f199ff854f8da14bd9ff8e93ee7fd
branch: main
author: Sam Gross
committer: gvanrossum
date: 2024-01-10T10:18:38-08:00
summary:
gh-113753: Clear finalized bit when putting PyAsyncGenASend back into free list
(#113754)
files:
A Misc/NEWS.d/next/Core and
Builtins/2024-01-05-21-28-48.gh-issue-113753.2HNiuq.rst
M Include/internal/pycore_gc.h
M Lib/test/test_asyncgen.py
M Objects/genobject.c
diff --git a/Include/internal/pycore_gc.h b/Include/internal/pycore_gc.h
index 2a0730eebb8db7..753763a5a50220 100644
--- a/Include/internal/pycore_gc.h
+++ b/Include/internal/pycore_gc.h
@@ -122,6 +122,10 @@ static inline void _PyGC_SET_FINALIZED(PyObject *op) {
PyGC_Head *gc = _Py_AS_GC(op);
_PyGCHead_SET_FINALIZED(gc);
}
+static inline void _PyGC_CLEAR_FINALIZED(PyObject *op) {
+PyGC_Head *gc = _Py_AS_GC(op);
+gc->_gc_prev &= ~_PyGC_PREV_MASK_FINALIZED;
+}
/* GC runtime state */
diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py
index a49630112af510..7fa0a85100a581 100644
--- a/Lib/test/test_asyncgen.py
+++ b/Lib/test/test_asyncgen.py
@@ -1701,6 +1701,14 @@ def test_asend(self):
async def gen():
yield 1
+# gh-113753: asend objects allocated from a free-list should warn.
+# Ensure there is a finalized 'asend' object ready to be reused.
+try:
+g = gen()
+g.asend(None).send(None)
+except StopIteration:
+pass
+
msg = f"coroutine method 'asend' of '{gen.__qualname__}' was never
awaited"
with self.assertWarnsRegex(RuntimeWarning, msg):
g = gen()
diff --git a/Misc/NEWS.d/next/Core and
Builtins/2024-01-05-21-28-48.gh-issue-113753.2HNiuq.rst b/Misc/NEWS.d/next/Core
and Builtins/2024-01-05-21-28-48.gh-issue-113753.2HNiuq.rst
new file mode 100644
index 00..32cf2cb2a4ae56
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and
Builtins/2024-01-05-21-28-48.gh-issue-113753.2HNiuq.rst
@@ -0,0 +1,2 @@
+Fix an issue where the finalizer of ``PyAsyncGenASend`` objects might not be
+called if they were allocated from a free list.
diff --git a/Objects/genobject.c b/Objects/genobject.c
index 9614713883741c..f03919c75d70a5 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -6,6 +6,7 @@
#include "pycore_call.h" // _PyObject_CallNoArgs()
#include "pycore_ceval.h" // _PyEval_EvalFrame()
#include "pycore_frame.h" // _PyInterpreterFrame
+#include "pycore_gc.h"// _PyGC_CLEAR_FINALIZED()
#include "pycore_genobject.h" // struct _Py_async_gen_state
#include "pycore_modsupport.h"// _PyArg_CheckPositional()
#include "pycore_object.h"// _PyObject_GC_UNTRACK()
@@ -1739,6 +1740,7 @@ async_gen_asend_dealloc(PyAsyncGenASend *o)
#endif
if (state->asend_numfree < _PyAsyncGen_MAXFREELIST) {
assert(PyAsyncGenASend_CheckExact(o));
+_PyGC_CLEAR_FINALIZED((PyObject *)o);
state->asend_freelist[state->asend_numfree++] = o;
}
else
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-112302: Point core developers to SBOM devguide on errors (#113490)
https://github.com/python/cpython/commit/e82b096335dcf31aedd2d265cbcc2f87ad971bfe commit: e82b096335dcf31aedd2d265cbcc2f87ad971bfe branch: main author: Seth Michael Larson committer: hugovk <[email protected]> date: 2024-01-10T19:21:04Z summary: gh-112302: Point core developers to SBOM devguide on errors (#113490) Co-authored-by: Hugo van Kemenade <[email protected]> files: M Tools/build/generate_sbom.py diff --git a/Tools/build/generate_sbom.py b/Tools/build/generate_sbom.py index 93d0d8a3762df3..282ee20cc402b0 100644 --- a/Tools/build/generate_sbom.py +++ b/Tools/build/generate_sbom.py @@ -82,6 +82,14 @@ def spdx_id(value: str) -> str: return re.sub(r"[^a-zA-Z0-9.\-]+", "-", value) +def error_if(value: bool, error_message: str) -> None: +"""Prints an error if a comparison fails along with a link to the devguide""" +if value: +print(error_message) +print("See 'https://devguide.python.org/developer-workflow/sbom' for more information.") +sys.exit(1) + + def filter_gitignored_paths(paths: list[str]) -> list[str]: """ Filter out paths excluded by the gitignore file. @@ -206,22 +214,47 @@ def main() -> None: discover_pip_sbom_package(sbom_data) # Ensure all packages in this tool are represented also in the SBOM file. -assert {package["name"] for package in sbom_data["packages"]} == set(PACKAGE_TO_FILES) +error_if( +{package["name"] for package in sbom_data["packages"]} != set(PACKAGE_TO_FILES), +"Packages defined in SBOM tool don't match those defined in SBOM file.", +) # Make a bunch of assertions about the SBOM data to ensure it's consistent. for package in sbom_data["packages"]: - # Properties and ID must be properly formed. -assert set(package.keys()) == REQUIRED_PROPERTIES_PACKAGE -assert package["SPDXID"] == spdx_id(f"SPDXRef-PACKAGE-{package['name']}") +error_if( +"name" not in package, +"Package is missing the 'name' field" +) +error_if( +set(package.keys()) != REQUIRED_PROPERTIES_PACKAGE, +f"Package '{package['name']}' is missing required fields", +) +error_if( +package["SPDXID"] != spdx_id(f"SPDXRef-PACKAGE-{package['name']}"), +f"Package '{package['name']}' has a malformed SPDXID", +) # Version must be in the download and external references. version = package["versionInfo"] -assert version in package["downloadLocation"] -assert all(version in ref["referenceLocator"] for ref in package["externalRefs"]) +error_if( +version not in package["downloadLocation"], +f"Version '{version}' for package '{package['name']} not in 'downloadLocation' field", +) +error_if( +any(version not in ref["referenceLocator"] for ref in package["externalRefs"]), +( +f"Version '{version}' for package '{package['name']} not in " +f"all 'externalRefs[].referenceLocator' fields" +), +) # License must be on the approved list for SPDX. -assert package["licenseConcluded"] in ALLOWED_LICENSE_EXPRESSIONS, package["licenseConcluded"] +license_concluded = package["licenseConcluded"] +error_if( +license_concluded not in ALLOWED_LICENSE_EXPRESSIONS, +f"License identifier '{license_concluded}' not in SBOM tool allowlist" +) # Regenerate file information from current data. sbom_files = [] @@ -232,11 +265,13 @@ def main() -> None: package_spdx_id = spdx_id(f"SPDXRef-PACKAGE-{name}") exclude = files.exclude or () for include in sorted(files.include): - # Find all the paths and then filter them through .gitignore. paths = glob.glob(include, root_dir=CPYTHON_ROOT_DIR, recursive=True) paths = filter_gitignored_paths(paths) -assert paths, include # Make sure that every value returns something! +error_if( +len(paths) == 0, +f"No valid paths found at path '{include}' for package '{name}", +) for path in paths: # Skip directories and excluded files ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-77046: os.pipe() sets _O_NOINHERIT flag on fds (#113817)
https://github.com/python/cpython/commit/1d75fa43a25e5f3c33f2aaaec28fab9430834792 commit: 1d75fa43a25e5f3c33f2aaaec28fab9430834792 branch: main author: Victor Stinner committer: vstinner date: 2024-01-10T23:02:17+01:00 summary: gh-77046: os.pipe() sets _O_NOINHERIT flag on fds (#113817) On Windows, set _O_NOINHERIT flag on file descriptors created by os.pipe() and io.WindowsConsoleIO. Add test_pipe_spawnl() to test_os. Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Core and Builtins/2024-01-08-14-34-02.gh-issue-77046.sDUh2d.rst M Doc/library/msvcrt.rst M Lib/test/test_os.py M Modules/_io/winconsoleio.c M Modules/posixmodule.c diff --git a/Doc/library/msvcrt.rst b/Doc/library/msvcrt.rst index 0b059e746c61af..2a6d980ab78a60 100644 --- a/Doc/library/msvcrt.rst +++ b/Doc/library/msvcrt.rst @@ -75,10 +75,14 @@ File Operations .. function:: open_osfhandle(handle, flags) Create a C runtime file descriptor from the file handle *handle*. The *flags* - parameter should be a bitwise OR of :const:`os.O_APPEND`, :const:`os.O_RDONLY`, - and :const:`os.O_TEXT`. The returned file descriptor may be used as a parameter + parameter should be a bitwise OR of :const:`os.O_APPEND`, + :const:`os.O_RDONLY`, :const:`os.O_TEXT` and :const:`os.O_NOINHERIT`. + The returned file descriptor may be used as a parameter to :func:`os.fdopen` to create a file object. + The file descriptor is inheritable by default. Pass :const:`os.O_NOINHERIT` + flag to make it non inheritable. + .. audit-event:: msvcrt.open_osfhandle handle,flags msvcrt.open_osfhandle diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index c66c5797471413..bff6e604cccdd6 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -4485,6 +4485,61 @@ def test_openpty(self): self.assertEqual(os.get_inheritable(master_fd), False) self.assertEqual(os.get_inheritable(slave_fd), False) [email protected](hasattr(os, 'spawnl'), "need os.openpty()") +def test_pipe_spawnl(self): +# gh-77046: On Windows, os.pipe() file descriptors must be created with +# _O_NOINHERIT to make them non-inheritable. UCRT has no public API to +# get (_osfile(fd) & _O_NOINHERIT), so use a functional test. +# +# Make sure that fd is not inherited by a child process created by +# os.spawnl(): get_osfhandle() and dup() must fail with EBADF. + +fd, fd2 = os.pipe() +self.addCleanup(os.close, fd) +self.addCleanup(os.close, fd2) + +code = textwrap.dedent(f""" +import errno +import os +import test.support +try: +import msvcrt +except ImportError: +msvcrt = None + +fd = {fd} + +with test.support.SuppressCrashReport(): +if msvcrt is not None: +try: +handle = msvcrt.get_osfhandle(fd) +except OSError as exc: +if exc.errno != errno.EBADF: +raise +# get_osfhandle(fd) failed with EBADF as expected +else: +raise Exception("get_osfhandle() must fail") + +try: +fd3 = os.dup(fd) +except OSError as exc: +if exc.errno != errno.EBADF: +raise +# os.dup(fd) failed with EBADF as expected +else: +os.close(fd3) +raise Exception("dup must fail") +""") + +filename = os_helper.TESTFN +self.addCleanup(os_helper.unlink, os_helper.TESTFN) +with open(filename, "w") as fp: +print(code, file=fp, end="") + +cmd = [sys.executable, filename] +exitcode = os.spawnl(os.P_WAIT, cmd[0], *cmd) +self.assertEqual(exitcode, 0) + class PathTConverterTests(unittest.TestCase): # tuples of (function name, allows fd arguments, additional arguments to diff --git a/Misc/NEWS.d/next/Core and Builtins/2024-01-08-14-34-02.gh-issue-77046.sDUh2d.rst b/Misc/NEWS.d/next/Core and Builtins/2024-01-08-14-34-02.gh-issue-77046.sDUh2d.rst new file mode 100644 index 00..9f0f144451df6c --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2024-01-08-14-34-02.gh-issue-77046.sDUh2d.rst @@ -0,0 +1,3 @@ +On Windows, file descriptors wrapping Windows handles are now created non +inheritable by default (:pep:`446`). Patch by Zackery Spytz and Victor +Stinner. diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index fecb3389570780..54e1417287 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -391,9 +391,9 @@ _io__WindowsConsoleIO___init___impl(winconsoleio *self, PyObject *nameobj, } if (self->writable) -self->fd = _Py_open_osfhandle_noraise(handle, _O_WRONLY
[Python-checkins] gh-87868: Skip `test_one_environment_variable` in `test_subprocess` when the platform or build cannot do that (#113867)
https://github.com/python/cpython/commit/fafb3275f25e116e51ff0b867aec597cb3de840f commit: fafb3275f25e116e51ff0b867aec597cb3de840f branch: main author: AN Long committer: gpshead date: 2024-01-10T15:17:05-08:00 summary: gh-87868: Skip `test_one_environment_variable` in `test_subprocess` when the platform or build cannot do that (#113867) * improve the assert for test_one_environment_variable * skip some test in test_subprocess when python is configured with shared * also skip the test if AddressSanitizer is enabled - Co-authored-by: Steve Dower files: M Lib/test/test_subprocess.py diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 102e697ba7a90d..944a7de4210bc9 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -835,6 +835,11 @@ def is_env_var_to_ignore(n): if not is_env_var_to_ignore(k)] self.assertEqual(child_env_names, []) [email protected](sysconfig.get_config_var('Py_ENABLE_SHARED') == 1, + 'The Python shared library cannot be loaded ' + 'without some system environments.') [email protected](check_sanitizer(address=True), + 'AddressSanitizer adds to the environment.') def test_one_environment_variable(self): newenv = {'fruit': 'orange'} cmd = [sys.executable, '-c', @@ -842,9 +847,13 @@ def test_one_environment_variable(self): 'sys.stdout.write("fruit="+os.getenv("fruit"))'] if sys.platform == "win32": cmd = ["CMD", "/c", "SET", "fruit"] -with subprocess.Popen(cmd, stdout=subprocess.PIPE, env=newenv) as p: -stdout, _ = p.communicate() -self.assertTrue(stdout.startswith(b"fruit=orange")) +with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=newenv) as p: +stdout, stderr = p.communicate() +if p.returncode and support.verbose: +print("STDOUT:", stdout.decode("ascii", "replace")) +print("STDERR:", stderr.decode("ascii", "replace")) +self.assertEqual(p.returncode, 0) +self.assertEqual(stdout.strip(), b"fruit=orange") def test_invalid_cmd(self): # null character in the command name ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113896: Fix test_builtin.BuiltinTest.test___ne__() (#113897)
https://github.com/python/cpython/commit/9d33c23857cfd952bf3e1e7f34c77b7c9a5accc3
commit: 9d33c23857cfd952bf3e1e7f34c77b7c9a5accc3
branch: main
author: Kirill Podoprigora
committer: vstinner
date: 2024-01-11T00:39:48+01:00
summary:
gh-113896: Fix test_builtin.BuiltinTest.test___ne__() (#113897)
Fix DeprecationWarning in test___ne__().
Co-authored-by: Nikita Sobolev
files:
M Lib/test/test_builtin.py
diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
index e15492783aeec1..fcddd147bac63e 100644
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -629,8 +629,8 @@ def __dir__(self):
def test___ne__(self):
self.assertFalse(None.__ne__(None))
-self.assertTrue(None.__ne__(0))
-self.assertTrue(None.__ne__("abc"))
+self.assertIs(None.__ne__(0), NotImplemented)
+self.assertIs(None.__ne__("abc"), NotImplemented)
def test_divmod(self):
self.assertEqual(divmod(12, 7), (1, 5))
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-87868: Sort and remove duplicates in getenvironment() (GH-102731)
https://github.com/python/cpython/commit/4050a150e9ce08bacb1ce0140ba16d1689dbbcdc commit: 4050a150e9ce08bacb1ce0140ba16d1689dbbcdc branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: zooba date: 2024-01-10T23:43:39Z summary: gh-87868: Sort and remove duplicates in getenvironment() (GH-102731) (cherry picked from commit c31be58da8577ef140e83d4e46502c7bb1eb9abf) Co-authored-by: AN Long Co-authored-by: Alex Waygood Co-authored-by: Pieter Eendebak Co-authored-by: Erlend E. Aasland files: A Misc/NEWS.d/next/Windows/2023-03-15-23-53-45.gh-issue-87868.4C36oQ.rst M Lib/test/test_subprocess.py M Modules/_winapi.c diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 79497f9301557e..9ade90f741bc6f 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -791,6 +791,19 @@ def test_env(self): stdout, stderr = p.communicate() self.assertEqual(stdout, b"orange") [email protected](sys.platform == "win32", "Windows only issue") +def test_win32_duplicate_envs(self): +newenv = os.environ.copy() +newenv["fRUit"] = "cherry" +newenv["fruit"] = "lemon" +newenv["FRUIT"] = "orange" +newenv["frUit"] = "banana" +with subprocess.Popen(["CMD", "/c", "SET", "fruit"], + stdout=subprocess.PIPE, + env=newenv) as p: +stdout, _ = p.communicate() +self.assertEqual(stdout.strip(), b"frUit=banana") + # Windows requires at least the SYSTEMROOT environment variable to start # Python @unittest.skipIf(sys.platform == 'win32', @@ -822,6 +835,26 @@ def is_env_var_to_ignore(n): if not is_env_var_to_ignore(k)] self.assertEqual(child_env_names, []) [email protected](sysconfig.get_config_var('Py_ENABLE_SHARED') == 1, + 'The Python shared library cannot be loaded ' + 'without some system environments.') [email protected](check_sanitizer(address=True), + 'AddressSanitizer adds to the environment.') +def test_one_environment_variable(self): +newenv = {'fruit': 'orange'} +cmd = [sys.executable, '-c', + 'import sys,os;' + 'sys.stdout.write("fruit="+os.getenv("fruit"))'] +if sys.platform == "win32": +cmd = ["CMD", "/c", "SET", "fruit"] +with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=newenv) as p: +stdout, stderr = p.communicate() +if p.returncode and support.verbose: +print("STDOUT:", stdout.decode("ascii", "replace")) +print("STDERR:", stderr.decode("ascii", "replace")) +self.assertEqual(p.returncode, 0) +self.assertEqual(stdout.strip(), b"fruit=orange") + def test_invalid_cmd(self): # null character in the command name cmd = sys.executable + '\0' @@ -862,6 +895,19 @@ def test_invalid_env(self): stdout, stderr = p.communicate() self.assertEqual(stdout, b"orange=lemon") [email protected](sys.platform == "win32", "Windows only issue") +def test_win32_invalid_env(self): +# '=' in the environment variable name +newenv = os.environ.copy() +newenv["FRUIT=VEGETABLE"] = "cabbage" +with self.assertRaises(ValueError): +subprocess.Popen(ZERO_RETURN_CMD, env=newenv) + +newenv = os.environ.copy() +newenv["==FRUIT"] = "cabbage" +with self.assertRaises(ValueError): +subprocess.Popen(ZERO_RETURN_CMD, env=newenv) + def test_communicate_stdin(self): p = subprocess.Popen([sys.executable, "-c", 'import sys;' diff --git a/Misc/NEWS.d/next/Windows/2023-03-15-23-53-45.gh-issue-87868.4C36oQ.rst b/Misc/NEWS.d/next/Windows/2023-03-15-23-53-45.gh-issue-87868.4C36oQ.rst new file mode 100644 index 00..37e8103c9ec34b --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-03-15-23-53-45.gh-issue-87868.4C36oQ.rst @@ -0,0 +1,2 @@ +Correctly sort and remove duplicate environment variables in +:py:func:`!_winapi.CreateProcess`. diff --git a/Modules/_winapi.c b/Modules/_winapi.c index 77275408aed868..fdbe3f626fd2a7 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -772,12 +772,157 @@ gethandle(PyObject* obj, const char* name) return ret; } +static PyObject * +sortenvironmentkey(PyObject *module, PyObject *item) +{ +return _winapi_LCMapStringEx_impl(NULL, LOCALE_NAME_INVARIANT, + LCMAP_UPPERCASE, item); +} + +static PyMethodDef sortenvironmentkey_def = { +"sortenvironmentkey", _PyCFunction_CAST(sortenvironmentkey), METH_O, "", +}; + +static int +sort_environment_keys(PyObject *keys) +{ +PyObject *keyfunc = PyCFunction_New
[Python-checkins] gh-111968: Unify naming scheme for freelist (gh-113919)
https://github.com/python/cpython/commit/c65ae26f2b46ca616a7ca000bbfcdf63b9bdd779
commit: c65ae26f2b46ca616a7ca000bbfcdf63b9bdd779
branch: main
author: Donghee Na
committer: corona10
date: 2024-01-11T08:51:51+09:00
summary:
gh-111968: Unify naming scheme for freelist (gh-113919)
files:
M Include/internal/pycore_freelist.h
M Objects/listobject.c
diff --git a/Include/internal/pycore_freelist.h
b/Include/internal/pycore_freelist.h
index d9619218b8dada..d41153d26a2546 100644
--- a/Include/internal/pycore_freelist.h
+++ b/Include/internal/pycore_freelist.h
@@ -36,7 +36,7 @@ struct _Py_float_state {
typedef struct _Py_freelist_state {
struct _Py_float_state float_state;
-struct _Py_list_state list;
+struct _Py_list_state list_state;
} _PyFreeListState;
#ifdef __cplusplus
diff --git a/Objects/listobject.c b/Objects/listobject.c
index 2fc57e13f632f8..c05c4fdff83883 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -26,7 +26,7 @@ get_list_state(void)
{
_PyFreeListState *state = _PyFreeListState_GET();
assert(state != NULL);
-return &state->list;
+return &state->list_state;
}
#endif
@@ -124,7 +124,7 @@ void
_PyList_ClearFreeList(_PyFreeListState *freelist_state, int is_finalization)
{
#if PyList_MAXFREELIST > 0
-struct _Py_list_state *state = &freelist_state->list;
+struct _Py_list_state *state = &freelist_state->list_state;
while (state->numfree > 0) {
PyListObject *op = state->free_list[--state->numfree];
assert(PyList_CheckExact(op));
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] [3.12] gh-113896: Fix test_builtin.BuiltinTest.test___ne__() (GH-113897) (#113928)
https://github.com/python/cpython/commit/3f607a03242e98ec6ff141fa9bd27ebfb46c7519 commit: 3f607a03242e98ec6ff141fa9bd27ebfb46c7519 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: vstinner date: 2024-01-10T23:56:57Z summary: [3.12] gh-113896: Fix test_builtin.BuiltinTest.test___ne__() (GH-113897) (#113928) gh-113896: Fix test_builtin.BuiltinTest.test___ne__() (GH-113897) Fix DeprecationWarning in test___ne__(). (cherry picked from commit 9d33c23857cfd952bf3e1e7f34c77b7c9a5accc3) Co-authored-by: Kirill Podoprigora Co-authored-by: Nikita Sobolev files: M Lib/test/test_builtin.py diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index a735def65f517e..4d03c46382e393 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -604,8 +604,8 @@ def __dir__(self): def test___ne__(self): self.assertFalse(None.__ne__(None)) -self.assertTrue(None.__ne__(0)) -self.assertTrue(None.__ne__("abc")) +self.assertIs(None.__ne__(0), NotImplemented) +self.assertIs(None.__ne__("abc"), NotImplemented) def test_divmod(self): self.assertEqual(divmod(12, 7), (1, 5)) ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-89811: Check for valid tp_version_tag in specializer (GH-113558)
https://github.com/python/cpython/commit/f653caa5a88d3b5027a8f286ff3a3ccd9e6fe4ed
commit: f653caa5a88d3b5027a8f286ff3a3ccd9e6fe4ed
branch: main
author: Peter Lazorchak
committer: Fidget-Spinner
date: 2024-01-11T13:33:05+08:00
summary:
gh-89811: Check for valid tp_version_tag in specializer (GH-113558)
files:
A Misc/NEWS.d/next/Core and
Builtins/2024-01-03-12-19-37.gh-issue-89811.cZOj6d.rst
M Lib/test/test_type_cache.py
M Modules/_testcapimodule.c
M Python/specialize.c
diff --git a/Lib/test/test_type_cache.py b/Lib/test/test_type_cache.py
index 72587ecc11b6f3..95b55009c7187d 100644
--- a/Lib/test/test_type_cache.py
+++ b/Lib/test/test_type_cache.py
@@ -1,5 +1,6 @@
""" Tests for the internal type cache in CPython. """
import unittest
+import dis
from test import support
from test.support import import_helper
try:
@@ -8,8 +9,11 @@
_clear_type_cache = None
# Skip this test if the _testcapi module isn't available.
-type_get_version = import_helper.import_module('_testcapi').type_get_version
-type_assign_version =
import_helper.import_module('_testcapi').type_assign_version
+_testcapi = import_helper.import_module("_testcapi")
+type_get_version = _testcapi.type_get_version
+type_assign_specific_version_unsafe =
_testcapi.type_assign_specific_version_unsafe
+type_assign_version = _testcapi.type_assign_version
+type_modified = _testcapi.type_modified
@support.cpython_only
@@ -56,6 +60,183 @@ class C:
self.assertNotEqual(type_get_version(C), 0)
self.assertNotEqual(type_get_version(C), c_ver)
+def test_type_assign_specific_version(self):
+"""meta-test for type_assign_specific_version_unsafe"""
+class C:
+pass
+
+type_assign_version(C)
+orig_version = type_get_version(C)
+self.assertNotEqual(orig_version, 0)
+
+type_modified(C)
+type_assign_specific_version_unsafe(C, orig_version + 5)
+type_assign_version(C) # this should do nothing
+
+new_version = type_get_version(C)
+self.assertEqual(new_version, orig_version + 5)
+
+_clear_type_cache()
+
+
[email protected]_only
+class TypeCacheWithSpecializationTests(unittest.TestCase):
+def tearDown(self):
+_clear_type_cache()
+
+def _assign_and_check_valid_version(self, user_type):
+type_modified(user_type)
+type_assign_version(user_type)
+self.assertNotEqual(type_get_version(user_type), 0)
+
+def _assign_and_check_version_0(self, user_type):
+type_modified(user_type)
+type_assign_specific_version_unsafe(user_type, 0)
+self.assertEqual(type_get_version(user_type), 0)
+
+def _all_opnames(self, func):
+return set(instr.opname for instr in dis.Bytecode(func, adaptive=True))
+
+def _check_specialization(self, func, arg, opname, *, should_specialize):
+self.assertIn(opname, self._all_opnames(func))
+
+for _ in range(100):
+func(arg)
+
+if should_specialize:
+self.assertNotIn(opname, self._all_opnames(func))
+else:
+self.assertIn(opname, self._all_opnames(func))
+
+def test_class_load_attr_specialization_user_type(self):
+class A:
+def foo(self):
+pass
+
+self._assign_and_check_valid_version(A)
+
+def load_foo_1(type_):
+type_.foo
+
+self._check_specialization(load_foo_1, A, "LOAD_ATTR",
should_specialize=True)
+del load_foo_1
+
+self._assign_and_check_version_0(A)
+
+def load_foo_2(type_):
+return type_.foo
+
+self._check_specialization(load_foo_2, A, "LOAD_ATTR",
should_specialize=False)
+
+def test_class_load_attr_specialization_static_type(self):
+self._assign_and_check_valid_version(str)
+self._assign_and_check_valid_version(bytes)
+
+def get_capitalize_1(type_):
+return type_.capitalize
+
+self._check_specialization(get_capitalize_1, str, "LOAD_ATTR",
should_specialize=True)
+self.assertEqual(get_capitalize_1(str)('hello'), 'Hello')
+self.assertEqual(get_capitalize_1(bytes)(b'hello'), b'Hello')
+del get_capitalize_1
+
+# Permanently overflow the static type version counter, and force str
and bytes
+# to have tp_version_tag == 0
+for _ in range(2**16):
+type_modified(str)
+type_assign_version(str)
+type_modified(bytes)
+type_assign_version(bytes)
+
+self.assertEqual(type_get_version(str), 0)
+self.assertEqual(type_get_version(bytes), 0)
+
+def get_capitalize_2(type_):
+return type_.capitalize
+
+self._check_specialization(get_capitalize_2, str, "LOAD_ATTR",
should_specialize=False)
+self.assertEqual(get_capitalize_2(str)('hello'), 'Hello')
+self.assertEqual(get_capitalize_2(bytes)(b'hello'), b'Hello')
+
+def test_property_load_attr_specialization_user_ty
