[Python-checkins] gh-88531 Fix dataclass __post_init__/__init__ interplay documentation (gh-107404)
https://github.com/python/cpython/commit/05008c27b73da640b63c0d335c65ade517c0eb84 commit: 05008c27b73da640b63c0d335c65ade517c0eb84 branch: main author: Steffen Zeile <[email protected]> committer: ericvsmith date: 2024-01-16T20:17:34-05:00 summary: gh-88531 Fix dataclass __post_init__/__init__ interplay documentation (gh-107404) * Simplify __post_init__ example usage. It applies to all base classes, not just dataclasses. files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index cb00d8fef8..cde147d1cbb501 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -536,10 +536,10 @@ class :meth:`~object.__init__` methods. If the base class has an :meth:`~object. that has to be called, it is common to call this method in a :meth:`__post_init__` method:: -@dataclass class Rectangle: -height: float -width: float +def __init__(self, height, width): + self.height = height + self.width = width @dataclass class Square(Rectangle): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-114198: Rename dataclass __replace__ argument to 'self' (gh-114251)
https://github.com/python/cpython/commit/339fc3c22446a148d27d9ec061594ac8d0abd33d
commit: 339fc3c22446a148d27d9ec061594ac8d0abd33d
branch: main
author: Phillip Schanely
committer: ericvsmith
date: 2024-01-18T11:01:17-05:00
summary:
gh-114198: Rename dataclass __replace__ argument to 'self' (gh-114251)
This change renames the dataclass __replace__ method's first argument
name from 'obj' to 'self'.
files:
A Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst
M Lib/dataclasses.py
diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py
index 2fba32b5ffbc1e..d0827e96097c14 100644
--- a/Lib/dataclasses.py
+++ b/Lib/dataclasses.py
@@ -1558,14 +1558,14 @@ class C:
return _replace(obj, **changes)
-def _replace(obj, /, **changes):
+def _replace(self, /, **changes):
# We're going to mutate 'changes', but that's okay because it's a
-# new dict, even if called with 'replace(obj, **my_changes)'.
+# new dict, even if called with 'replace(self, **my_changes)'.
# It's an error to have init=False fields in 'changes'.
-# If a field is not in 'changes', read its value from the provided obj.
+# If a field is not in 'changes', read its value from the provided 'self'.
-for f in getattr(obj, _FIELDS).values():
+for f in getattr(self, _FIELDS).values():
# Only consider normal fields or InitVars.
if f._field_type is _FIELD_CLASSVAR:
continue
@@ -1582,11 +1582,11 @@ def _replace(obj, /, **changes):
if f._field_type is _FIELD_INITVAR and f.default is MISSING:
raise TypeError(f"InitVar {f.name!r} "
f'must be specified with replace()')
-changes[f.name] = getattr(obj, f.name)
+changes[f.name] = getattr(self, f.name)
# Create the new object, which calls __init__() and
# __post_init__() (if defined), using all of the init fields we've
# added and/or left in 'changes'. If there are values supplied in
# changes that aren't fields, this will correctly raise a
# TypeError.
-return obj.__class__(**changes)
+return self.__class__(**changes)
diff --git
a/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst
b/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst
new file mode 100644
index 00..fa047e288f807e
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst
@@ -0,0 +1,2 @@
+The signature for the ``__replace__`` method on :mod:`dataclasses` now has
+the first argument named ``self``, rather than ``obj``.
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] Fix indentation in `__post_init__` documentation. (gh-114666)
https://github.com/python/cpython/commit/d00fbed68ffcd5823acbb32a0e47e2e5f9732ff7 commit: d00fbed68ffcd5823acbb32a0e47e2e5f9732ff7 branch: main author: Bhushan Mohanraj <[email protected]> committer: ericvsmith date: 2024-01-28T15:10:32-05:00 summary: Fix indentation in `__post_init__` documentation. (gh-114666) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 88f2e0251b1e51..4ada69d63abada 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -538,8 +538,8 @@ that has to be called, it is common to call this method in a class Rectangle: def __init__(self, height, width): - self.height = height - self.width = width +self.height = height +self.width = width @dataclass class Square(Rectangle): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-109653: Just import `recursive_repr` in `dataclasses` (gh-109822)
https://github.com/python/cpython/commit/edc9d85c68af600431556f3e8edae9b0fbfdfd34 commit: edc9d85c68af600431556f3e8edae9b0fbfdfd34 branch: main author: Nikita Sobolev committer: ericvsmith date: 2024-03-05T13:12:00-05:00 summary: gh-109653: Just import `recursive_repr` in `dataclasses` (gh-109822) files: M Lib/dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 3335821f3f32e9..45ce5a98b51ae0 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -4,10 +4,9 @@ import types import inspect import keyword -import functools import itertools import abc -import _thread +from reprlib import recursive_repr from types import FunctionType, GenericAlias @@ -245,25 +244,6 @@ def __repr__(self): property, }) -# This function's logic is copied from "recursive_repr" function in -# reprlib module to avoid dependency. -def _recursive_repr(user_function): -# Decorator to make a repr function return "..." for a recursive -# call. -repr_running = set() - [email protected](user_function) -def wrapper(self): -key = id(self), _thread.get_ident() -if key in repr_running: -return '...' -repr_running.add(key) -try: -result = user_function(self) -finally: -repr_running.discard(key) -return result -return wrapper class InitVar: __slots__ = ('type', ) @@ -322,7 +302,7 @@ def __init__(self, default, default_factory, init, repr, hash, compare, self.kw_only = kw_only self._field_type = None -@_recursive_repr +@recursive_repr() def __repr__(self): return ('Field(' f'name={self.name!r},' @@ -632,7 +612,7 @@ def _repr_fn(fields, globals): for f in fields]) + ')"'], globals=globals) -return _recursive_repr(fn) +return recursive_repr()(fn) def _frozen_get_del_attr(cls, fields, globals): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-116535: Fix distracting "TypeError" in example code (gh-116538)
https://github.com/python/cpython/commit/db8f423f58e336eb6180a70d9886b443d7203c2c commit: db8f423f58e336eb6180a70d9886b443d7203c2c branch: main author: Declan <[email protected]> committer: ericvsmith date: 2024-03-09T16:52:57-05:00 summary: gh-116535: Fix distracting "TypeError" in example code (gh-116538) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 4ada69d63abada..c612c138fc6ea8 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -719,7 +719,7 @@ Using dataclasses, *if* this code was valid:: class D: x: list = [] # This code raises ValueError def add(self, element): - self.x += element + self.x.append(element) it would generate code similar to:: @@ -728,7 +728,7 @@ it would generate code similar to:: def __init__(self, x=x): self.x = x def add(self, element): - self.x += element + self.x.append(element) assert D().x is D().x ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.12] gh-116535: Fix distracting "TypeError" in example code (gh-116538) (gh-116551)
https://github.com/python/cpython/commit/0d5455fc7fa921b56d1f4cbdc89e1ba9036c2cd8 commit: 0d5455fc7fa921b56d1f4cbdc89e1ba9036c2cd8 branch: 3.12 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2024-03-09T17:08:12-05:00 summary: [3.12] gh-116535: Fix distracting "TypeError" in example code (gh-116538) (gh-116551) gh-116535: Fix distracting "TypeError" in example code (gh-116538) (cherry picked from commit db8f423f58e336eb6180a70d9886b443d7203c2c) Co-authored-by: Declan <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 020818a0812f0c..19272376e68c71 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -717,7 +717,7 @@ Using dataclasses, *if* this code was valid:: class D: x: list = [] # This code raises ValueError def add(self, element): - self.x += element + self.x.append(element) it would generate code similar to:: @@ -726,7 +726,7 @@ it would generate code similar to:: def __init__(self, x=x): self.x = x def add(self, element): - self.x += element + self.x.append(element) assert D().x is D().x ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.11] gh-116535: Fix distracting "TypeError" in example code (gh-116538) (gh-116552)
https://github.com/python/cpython/commit/982f457f6f86700a62ce00f1914101dcbde7de15 commit: 982f457f6f86700a62ce00f1914101dcbde7de15 branch: 3.11 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2024-03-09T17:08:23-05:00 summary: [3.11] gh-116535: Fix distracting "TypeError" in example code (gh-116538) (gh-116552) gh-116535: Fix distracting "TypeError" in example code (gh-116538) (cherry picked from commit db8f423f58e336eb6180a70d9886b443d7203c2c) Co-authored-by: Declan <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index a1066c748a7762..ee34cd76b2cc41 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -712,7 +712,7 @@ Using dataclasses, *if* this code was valid:: class D: x: list = [] # This code raises ValueError def add(self, element): - self.x += element + self.x.append(element) it would generate code similar to:: @@ -721,7 +721,7 @@ it would generate code similar to:: def __init__(self, x=x): self.x = x def add(self, element): - self.x += element + self.x.append(element) assert D().x is D().x ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-109870: Dataclasses: batch up exec calls (gh-110851)
https://github.com/python/cpython/commit/8945b7ff55b87d11c747af2dad0e3e4d631e62d6
commit: 8945b7ff55b87d11c747af2dad0e3e4d631e62d6
branch: main
author: Eric V. Smith
committer: ericvsmith
date: 2024-03-25T19:59:14-04:00
summary:
gh-109870: Dataclasses: batch up exec calls (gh-110851)
Instead of calling `exec()` once for each function added to a dataclass, only
call `exec()` once per dataclass. This can lead to speed improvements of up to
20%.
files:
A Misc/NEWS.d/next/Core and
Builtins/2023-10-14-00-05-17.gh-issue-109870.oKpJ3P.rst
M Lib/dataclasses.py
diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py
index 7db8a4233df883..3acd03cd865234 100644
--- a/Lib/dataclasses.py
+++ b/Lib/dataclasses.py
@@ -426,32 +426,95 @@ def _tuple_str(obj_name, fields):
return f'({",".join([f"{obj_name}.{f.name}" for f in fields])},)'
-def _create_fn(name, args, body, *, globals=None, locals=None,
- return_type=MISSING):
-# Note that we may mutate locals. Callers beware!
-# The only callers are internal to this module, so no
-# worries about external callers.
-if locals is None:
-locals = {}
-return_annotation = ''
-if return_type is not MISSING:
-locals['__dataclass_return_type__'] = return_type
-return_annotation = '->__dataclass_return_type__'
-args = ','.join(args)
-body = '\n'.join(f' {b}' for b in body)
-
-# Compute the text of the entire function.
-txt = f' def {name}({args}){return_annotation}:\n{body}'
-
-# Free variables in exec are resolved in the global namespace.
-# The global namespace we have is user-provided, so we can't modify it for
-# our purposes. So we put the things we need into locals and introduce a
-# scope to allow the function we're creating to close over them.
-local_vars = ', '.join(locals.keys())
-txt = f"def __create_fn__({local_vars}):\n{txt}\n return {name}"
-ns = {}
-exec(txt, globals, ns)
-return ns['__create_fn__'](**locals)
+class _FuncBuilder:
+def __init__(self, globals):
+self.names = []
+self.src = []
+self.globals = globals
+self.locals = {}
+self.overwrite_errors = {}
+self.unconditional_adds = {}
+
+def add_fn(self, name, args, body, *, locals=None, return_type=MISSING,
+ overwrite_error=False, unconditional_add=False, decorator=None):
+if locals is not None:
+self.locals.update(locals)
+
+# Keep track if this method is allowed to be overwritten if it already
+# exists in the class. The error is method-specific, so keep it with
+# the name. We'll use this when we generate all of the functions in
+# the add_fns_to_class call. overwrite_error is either True, in which
+# case we'll raise an error, or it's a string, in which case we'll
+# raise an error and append this string.
+if overwrite_error:
+self.overwrite_errors[name] = overwrite_error
+
+# Should this function always overwrite anything that's already in the
+# class? The default is to not overwrite a function that already
+# exists.
+if unconditional_add:
+self.unconditional_adds[name] = True
+
+self.names.append(name)
+
+if return_type is not MISSING:
+self.locals[f'__dataclass_{name}_return_type__'] = return_type
+return_annotation = f'->__dataclass_{name}_return_type__'
+else:
+return_annotation = ''
+args = ','.join(args)
+body = '\n'.join(body)
+
+# Compute the text of the entire function, add it to the text we're
generating.
+self.src.append(f'{f' {decorator}\n' if decorator else ''} def
{name}({args}){return_annotation}:\n{body}')
+
+def add_fns_to_class(self, cls):
+# The source to all of the functions we're generating.
+fns_src = '\n'.join(self.src)
+
+# The locals they use.
+local_vars = ','.join(self.locals.keys())
+
+# The names of all of the functions, used for the return value of the
+# outer function. Need to handle the 0-tuple specially.
+if len(self.names) == 0:
+return_names = '()'
+else:
+return_names =f'({",".join(self.names)},)'
+
+# txt is the entire function we're going to execute, including the
+# bodies of the functions we're defining. Here's a greatly simplified
+# version:
+# def __create_fn__():
+# def __init__(self, x, y):
+# self.x = x
+# self.y = y
+# @recursive_repr
+# def
[Python-checkins] Tweak wording for dataclasses.replace (gh-117758)
https://github.com/python/cpython/commit/e7cce2a9c696250aff64a57b85182356367be041 commit: e7cce2a9c696250aff64a57b85182356367be041 branch: main author: Gouvernathor <[email protected]> committer: ericvsmith date: 2024-04-13T20:03:09-04:00 summary: Tweak wording for dataclasses.replace (gh-117758) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 61b2263339da71..0b479f0d569981 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -430,8 +430,8 @@ Module contents Creates a new object of the same type as ``obj``, replacing fields with values from ``changes``. If ``obj`` is not a Data - Class, raises :exc:`TypeError`. If values in ``changes`` do not - specify fields, raises :exc:`TypeError`. + Class, raises :exc:`TypeError`. If keys in ``changes`` are not + field names of the given dataclass, raises :exc:`TypeError`. The newly returned object is created by calling the :meth:`~object.__init__` method of the dataclass. This ensures that ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-90562: Support zero argument super with dataclasses when slots=True (gh-124455)
https://github.com/python/cpython/commit/5c6e3b715082bfccd0b4cf2bb1c18e8b1afcad3e
commit: 5c6e3b715082bfccd0b4cf2bb1c18e8b1afcad3e
branch: main
author: Eric V. Smith
committer: ericvsmith
date: 2024-09-24T21:26:26-04:00
summary:
gh-90562: Support zero argument super with dataclasses when slots=True
(gh-124455)
Co-authored-by: @wookie184
Co-authored-by: Carl Meyer
files:
A Misc/NEWS.d/next/Library/2024-09-23-18-26-17.gh-issue-90562.Yj566G.rst
M Doc/library/dataclasses.rst
M Lib/dataclasses.py
M Lib/test/test_dataclasses/__init__.py
diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst
index cfca11afbd2e41..1457392ce6e86c 100644
--- a/Doc/library/dataclasses.rst
+++ b/Doc/library/dataclasses.rst
@@ -187,13 +187,6 @@ Module contents
If :attr:`!__slots__` is already defined in the class, then
:exc:`TypeError`
is raised.
-.. warning::
-Calling no-arg :func:`super` in dataclasses using ``slots=True``
-will result in the following exception being raised:
-``TypeError: super(type, obj): obj must be an instance or subtype of
type``.
-The two-arg :func:`super` is a valid workaround.
-See :gh:`90562` for full details.
-
.. warning::
Passing parameters to a base class :meth:`~object.__init_subclass__`
when using ``slots=True`` will result in a :exc:`TypeError`.
diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py
index 6255d8980974e0..f5cb97edaf72cd 100644
--- a/Lib/dataclasses.py
+++ b/Lib/dataclasses.py
@@ -1218,9 +1218,31 @@ def _get_slots(cls):
raise TypeError(f"Slots of '{cls.__name__}' cannot be determined")
+def _update_func_cell_for__class__(f, oldcls, newcls):
+# Returns True if we update a cell, else False.
+if f is None:
+# f will be None in the case of a property where not all of
+# fget, fset, and fdel are used. Nothing to do in that case.
+return False
+try:
+idx = f.__code__.co_freevars.index("__class__")
+except ValueError:
+# This function doesn't reference __class__, so nothing to do.
+return False
+# Fix the cell to point to the new class, if it's already pointing
+# at the old class. I'm not convinced that the "is oldcls" test
+# is needed, but other than performance can't hurt.
+closure = f.__closure__[idx]
+if closure.cell_contents is oldcls:
+closure.cell_contents = newcls
+return True
+return False
+
+
def _add_slots(cls, is_frozen, weakref_slot):
-# Need to create a new class, since we can't set __slots__
-# after a class has been created.
+# Need to create a new class, since we can't set __slots__ after a
+# class has been created, and the @dataclass decorator is called
+# after the class is created.
# Make sure __slots__ isn't already set.
if '__slots__' in cls.__dict__:
@@ -1259,18 +1281,37 @@ def _add_slots(cls, is_frozen, weakref_slot):
# And finally create the class.
qualname = getattr(cls, '__qualname__', None)
-cls = type(cls)(cls.__name__, cls.__bases__, cls_dict)
+newcls = type(cls)(cls.__name__, cls.__bases__, cls_dict)
if qualname is not None:
-cls.__qualname__ = qualname
+newcls.__qualname__ = qualname
if is_frozen:
# Need this for pickling frozen classes with slots.
if '__getstate__' not in cls_dict:
-cls.__getstate__ = _dataclass_getstate
+newcls.__getstate__ = _dataclass_getstate
if '__setstate__' not in cls_dict:
-cls.__setstate__ = _dataclass_setstate
-
-return cls
+newcls.__setstate__ = _dataclass_setstate
+
+# Fix up any closures which reference __class__. This is used to
+# fix zero argument super so that it points to the correct class
+# (the newly created one, which we're returning) and not the
+# original class. We can break out of this loop as soon as we
+# make an update, since all closures for a class will share a
+# given cell.
+for member in newcls.__dict__.values():
+# If this is a wrapped function, unwrap it.
+member = inspect.unwrap(member)
+
+if isinstance(member, types.FunctionType):
+if _update_func_cell_for__class__(member, cls, newcls):
+break
+elif isinstance(member, property):
+if (_update_func_cell_for__class__(member.fget, cls, newcls)
+or _update_func_cell_for__class__(member.fset, cls, newcls)
+or _update_func_cell_for__class__(member.fdel, cls, newcls)):
+break
+
+return newcls
def dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False,
diff --git a/Lib/test/test_dataclasses/__init__.py
b/Lib/test/test_dataclasses/__init__.py
index 6934e88d9d338c..69e86162e0c11a 100644
--- a
[Python-checkins] gh-89683: add tests for `deepcopy` on frozen dataclasses (gh-123098)
https://github.com/python/cpython/commit/5e7eba09bcdafe361b491b3f8cf30d7dd2df0a78 commit: 5e7eba09bcdafe361b491b3f8cf30d7dd2df0a78 branch: main author: Bénédikt Tran <[email protected]> committer: ericvsmith date: 2024-09-26T21:15:28Z summary: gh-89683: add tests for `deepcopy` on frozen dataclasses (gh-123098) Co-authored-by: Eric V. Smith files: M Lib/test/test_dataclasses/__init__.py diff --git a/Lib/test/test_dataclasses/__init__.py b/Lib/test/test_dataclasses/__init__.py index 69e86162e0c11a..bd2f87819a8eb0 100644 --- a/Lib/test/test_dataclasses/__init__.py +++ b/Lib/test/test_dataclasses/__init__.py @@ -17,6 +17,7 @@ from typing import ClassVar, Any, List, Union, Tuple, Dict, Generic, TypeVar, Optional, Protocol, DefaultDict from typing import get_type_hints from collections import deque, OrderedDict, namedtuple, defaultdict +from copy import deepcopy from functools import total_ordering, wraps import typing # Needed for the string "typing.ClassVar[int]" to work as an annotation. @@ -3175,6 +3176,48 @@ class C: with self.assertRaisesRegex(TypeError, 'unhashable type'): hash(C({})) +def test_frozen_deepcopy_without_slots(self): +# see: https://github.com/python/cpython/issues/89683 +@dataclass(frozen=True, slots=False) +class C: +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + +def test_frozen_deepcopy_with_slots(self): +# see: https://github.com/python/cpython/issues/89683 +with self.subTest('generated __slots__'): +@dataclass(frozen=True, slots=True) +class C: +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + +with self.subTest('user-defined __slots__ and no __{get,set}state__'): +@dataclass(frozen=True, slots=False) +class C: +__slots__ = ('s',) +s: str + +# with user-defined slots, __getstate__ and __setstate__ are not +# automatically added, hence the error +err = r"^cannot\ assign\ to\ field\ 's'$" +self.assertRaisesRegex(FrozenInstanceError, err, deepcopy, C('')) + +with self.subTest('user-defined __slots__ and __{get,set}state__'): +@dataclass(frozen=True, slots=False) +class C: +__slots__ = ('s',) +__getstate__ = dataclasses._dataclass_getstate +__setstate__ = dataclasses._dataclass_setstate + +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + class TestSlots(unittest.TestCase): def test_simple(self): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-113878: Add `doc` parameter to `dataclasses.field` (gh-114051)
https://github.com/python/cpython/commit/9c7657f09914254724683d91177aed7947637be5
commit: 9c7657f09914254724683d91177aed7947637be5
branch: main
author: sobolevn
committer: ericvsmith
date: 2024-09-27T12:20:49-04:00
summary:
gh-113878: Add `doc` parameter to `dataclasses.field` (gh-114051)
If using `slots=True`, the `doc` parameter ends up in the `__slots__` dict. The
`doc` parameter is also in the corresponding `Field` object.
files:
A Misc/NEWS.d/next/Library/2024-01-14-11-43-31.gh-issue-113878.dmEIN3.rst
M Doc/library/dataclasses.rst
M Lib/dataclasses.py
M Lib/test/test_dataclasses/__init__.py
M Lib/test/test_pydoc/test_pydoc.py
diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst
index 1457392ce6e86c..51c1a427b63787 100644
--- a/Doc/library/dataclasses.rst
+++ b/Doc/library/dataclasses.rst
@@ -231,7 +231,7 @@ Module contents
follows a field with a default value. This is true whether this
occurs in a single class, or as a result of class inheritance.
-.. function:: field(*, default=MISSING, default_factory=MISSING, init=True,
repr=True, hash=None, compare=True, metadata=None, kw_only=MISSING)
+.. function:: field(*, default=MISSING, default_factory=MISSING, init=True,
repr=True, hash=None, compare=True, metadata=None, kw_only=MISSING, doc=None)
For common and simple use cases, no other functionality is
required. There are, however, some dataclass features that
@@ -300,6 +300,10 @@ Module contents
.. versionadded:: 3.10
+ - ``doc``: optional docstring for this field.
+
+.. versionadded:: 3.13
+
If the default value of a field is specified by a call to
:func:`!field`, then the class attribute for this field will be
replaced by the specified *default* value. If *default* is not
diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py
index f5cb97edaf72cd..bdda7cc6c00f5d 100644
--- a/Lib/dataclasses.py
+++ b/Lib/dataclasses.py
@@ -283,11 +283,12 @@ class Field:
'compare',
'metadata',
'kw_only',
+ 'doc',
'_field_type', # Private: not to be used by user code.
)
def __init__(self, default, default_factory, init, repr, hash, compare,
- metadata, kw_only):
+ metadata, kw_only, doc):
self.name = None
self.type = None
self.default = default
@@ -300,6 +301,7 @@ def __init__(self, default, default_factory, init, repr,
hash, compare,
if metadata is None else
types.MappingProxyType(metadata))
self.kw_only = kw_only
+self.doc = doc
self._field_type = None
@recursive_repr()
@@ -315,6 +317,7 @@ def __repr__(self):
f'compare={self.compare!r},'
f'metadata={self.metadata!r},'
f'kw_only={self.kw_only!r},'
+f'doc={self.doc!r},'
f'_field_type={self._field_type}'
')')
@@ -382,7 +385,7 @@ def __repr__(self):
# so that a type checker can be told (via overloads) that this is a
# function whose type depends on its parameters.
def field(*, default=MISSING, default_factory=MISSING, init=True, repr=True,
- hash=None, compare=True, metadata=None, kw_only=MISSING):
+ hash=None, compare=True, metadata=None, kw_only=MISSING, doc=None):
"""Return an object to identify dataclass fields.
default is the default value of the field. default_factory is a
@@ -394,7 +397,7 @@ def field(*, default=MISSING, default_factory=MISSING,
init=True, repr=True,
comparison functions. metadata, if specified, must be a mapping
which is stored but not otherwise examined by dataclass. If kw_only
is true, the field will become a keyword-only parameter to
-__init__().
+__init__(). doc is an optional docstring for this field.
It is an error to specify both default and default_factory.
"""
@@ -402,7 +405,7 @@ def field(*, default=MISSING, default_factory=MISSING,
init=True, repr=True,
if default is not MISSING and default_factory is not MISSING:
raise ValueError('cannot specify both default and default_factory')
return Field(default, default_factory, init, repr, hash, compare,
- metadata, kw_only)
+ metadata, kw_only, doc)
def _fields_in_init_order(fields):
@@ -1174,7 +1177,7 @@ def _process_class(cls, init, repr, eq, order,
unsafe_hash, frozen,
if weakref_slot and not slots:
raise TypeError('weakref_slot is True but slots is False')
if slots:
-cls = _add_slots(cls, frozen, weakref_slot)
+cls = _add_slots(cls, frozen, weakref_slot, fields)
abc.update_abstractmethods(cls)
@@ -1239,7 +1242,32 @@ def _update_fu
[Python-checkins] [3.12] gh-89683: add tests for `deepcopy` on frozen dataclasses (GH-123098) (gh-124679)
https://github.com/python/cpython/commit/373036a82c2127da8c54993ec95a966a135ec4c1 commit: 373036a82c2127da8c54993ec95a966a135ec4c1 branch: 3.12 author: Bénédikt Tran <[email protected]> committer: ericvsmith date: 2024-09-27T16:57:30Z summary: [3.12] gh-89683: add tests for `deepcopy` on frozen dataclasses (GH-123098) (gh-124679) gh-89683: add tests for `deepcopy` on frozen dataclasses (gh-123098) Co-authored-by: Eric V. Smith files: M Lib/test/test_dataclasses/__init__.py diff --git a/Lib/test/test_dataclasses/__init__.py b/Lib/test/test_dataclasses/__init__.py index 94183b002dabb5..8421cf974cf5ab 100644 --- a/Lib/test/test_dataclasses/__init__.py +++ b/Lib/test/test_dataclasses/__init__.py @@ -17,6 +17,7 @@ from typing import ClassVar, Any, List, Union, Tuple, Dict, Generic, TypeVar, Optional, Protocol, DefaultDict from typing import get_type_hints from collections import deque, OrderedDict, namedtuple, defaultdict +from copy import deepcopy from functools import total_ordering import typing # Needed for the string "typing.ClassVar[int]" to work as an annotation. @@ -3071,6 +3072,48 @@ class C: with self.assertRaisesRegex(TypeError, 'unhashable type'): hash(C({})) +def test_frozen_deepcopy_without_slots(self): +# see: https://github.com/python/cpython/issues/89683 +@dataclass(frozen=True, slots=False) +class C: +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + +def test_frozen_deepcopy_with_slots(self): +# see: https://github.com/python/cpython/issues/89683 +with self.subTest('generated __slots__'): +@dataclass(frozen=True, slots=True) +class C: +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + +with self.subTest('user-defined __slots__ and no __{get,set}state__'): +@dataclass(frozen=True, slots=False) +class C: +__slots__ = ('s',) +s: str + +# with user-defined slots, __getstate__ and __setstate__ are not +# automatically added, hence the error +err = r"^cannot\ assign\ to\ field\ 's'$" +self.assertRaisesRegex(FrozenInstanceError, err, deepcopy, C('')) + +with self.subTest('user-defined __slots__ and __{get,set}state__'): +@dataclass(frozen=True, slots=False) +class C: +__slots__ = ('s',) +__getstate__ = dataclasses._dataclass_getstate +__setstate__ = dataclasses._dataclass_setstate + +s: str + +c = C('hello') +self.assertEqual(deepcopy(c), c) + class TestSlots(unittest.TestCase): def test_simple(self): ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-118974: Add `decorator` argument to `make_dataclass` (gh-122723)
https://github.com/python/cpython/commit/3e3a4d231518f91ff2f3c5a085b3849e32f1d548 commit: 3e3a4d231518f91ff2f3c5a085b3849e32f1d548 branch: main author: Victorien <[email protected]> committer: ericvsmith date: 2024-10-01T09:51:51-04:00 summary: gh-118974: Add `decorator` argument to `make_dataclass` (gh-122723) This is to allow the `dataclasses.make_dataclass` infrastructure to be used with another decorator that's compliant with `typing.dataclass_transform`. The new `decorator` argument to `dataclasses.make_dataclass` is `dataclasses.dataclass`, which used to be hard coded. files: A Misc/NEWS.d/next/Library/2024-08-06-07-24-00.gh-issue-118974.qamsCQ.rst M Doc/library/dataclasses.rst M Lib/dataclasses.py M Lib/test/test_dataclasses/__init__.py diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 51c1a427b63787..e34b2db0210960 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -399,7 +399,7 @@ Module contents :func:`!astuple` raises :exc:`TypeError` if *obj* is not a dataclass instance. -.. function:: make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, weakref_slot=False, module=None) +.. function:: make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, weakref_slot=False, module=None, decorator=dataclass) Creates a new dataclass with name *cls_name*, fields as defined in *fields*, base classes as given in *bases*, and initialized @@ -415,6 +415,11 @@ Module contents of the dataclass is set to that value. By default, it is set to the module name of the caller. + The *decorator* parameter is a callable that will be used to create the dataclass. + It should take the class object as a first argument and the same keyword arguments + as :func:`@dataclass `. By default, the :func:`@dataclass ` + function is used. + This function is not strictly required, because any Python mechanism for creating a new class with :attr:`!__annotations__` can then apply the :func:`@dataclass ` function to convert that class to @@ -438,6 +443,9 @@ Module contents def add_one(self): return self.x + 1 + .. versionadded:: 3.14 + Added the *decorator* parameter. + .. function:: replace(obj, /, **changes) Creates a new object of the same type as *obj*, replacing diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index bdda7cc6c00f5d..7a24f8a9e5ccee 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1550,7 +1550,7 @@ def _astuple_inner(obj, tuple_factory): def make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, - weakref_slot=False, module=None): + weakref_slot=False, module=None, decorator=dataclass): """Return a new dynamically created dataclass. The dataclass name will be 'cls_name'. 'fields' is an iterable @@ -1630,8 +1630,8 @@ def exec_body_callback(ns): if module is not None: cls.__module__ = module -# Apply the normal decorator. -return dataclass(cls, init=init, repr=repr, eq=eq, order=order, +# Apply the normal provided decorator. +return decorator(cls, init=init, repr=repr, eq=eq, order=order, unsafe_hash=unsafe_hash, frozen=frozen, match_args=match_args, kw_only=kw_only, slots=slots, weakref_slot=weakref_slot) diff --git a/Lib/test/test_dataclasses/__init__.py b/Lib/test/test_dataclasses/__init__.py index 2984f4261bd2c4..2e6c49e29ce828 100644 --- a/Lib/test/test_dataclasses/__init__.py +++ b/Lib/test/test_dataclasses/__init__.py @@ -4317,6 +4317,23 @@ def test_funny_class_names_names(self): C = make_dataclass(classname, ['a', 'b']) self.assertEqual(C.__name__, classname) +def test_dataclass_decorator_default(self): +C = make_dataclass('C', [('x', int)], decorator=dataclass) +c = C(10) +self.assertEqual(c.x, 10) + +def test_dataclass_custom_decorator(self): +def custom_dataclass(cls, *args, **kwargs): +dc = dataclass(cls, *args, **kwargs) +dc.__custom__ = True +return dc + +C = make_dataclass('C', [('x', int)], decorator=custom_dataclass) +c = C(10) +self.assertEqual(c.x, 10) +self.assertEqual(c.__custom__, True) + + class TestReplace(unittest.TestCase): def test(self): @dataclass(frozen=True) diff --git a/M
[Python-checkins] gh-132111: Document dataclasses.InitVar (#132446)
https://github.com/python/cpython/commit/281fc338fdf57ef119e213bf1b2c772261c359c1 commit: 281fc338fdf57ef119e213bf1b2c772261c359c1 branch: main author: Tapeline committer: ericvsmith date: 2025-04-13T12:47:44-04:00 summary: gh-132111: Document dataclasses.InitVar (#132446) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 0bc171da4eefc7..72612211b43d5e 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -344,6 +344,15 @@ Module contents Other attributes may exist, but they are private and must not be inspected or relied on. +.. class:: InitVar + + ``InitVar[T]`` type annotations describe variables that are :ref:`init-only + `. Fields annotated with :class:`!InitVar` + are considered pseudo-fields, and thus are neither returned by the + :func:`fields` function nor used in any way except adding them as + parameters to :meth:`~object.__init__` and an optional + :meth:`__post_init__`. + .. function:: fields(class_or_instance) Returns a tuple of :class:`Field` objects that define the fields for this @@ -600,8 +609,8 @@ Init-only variables Another place where :func:`@dataclass ` inspects a type annotation is to determine if a field is an init-only variable. It does this by seeing -if the type of a field is of type ``dataclasses.InitVar``. If a field -is an ``InitVar``, it is considered a pseudo-field called an init-only +if the type of a field is of type :class:`InitVar`. If a field +is an :class:`InitVar`, it is considered a pseudo-field called an init-only field. As it is not a true field, it is not returned by the module-level :func:`fields` function. Init-only fields are added as parameters to the generated :meth:`~object.__init__` method, and are passed to ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.13] gh-132111: Document dataclasses.InitVar (GH-132446) (#132483)
https://github.com/python/cpython/commit/d021b719ed87d4ce6e03913361564c1c79215123 commit: d021b719ed87d4ce6e03913361564c1c79215123 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-04-13T16:53:40Z summary: [3.13] gh-132111: Document dataclasses.InitVar (GH-132446) (#132483) gh-132111: Document dataclasses.InitVar (GH-132446) (cherry picked from commit 281fc338fdf57ef119e213bf1b2c772261c359c1) Co-authored-by: Tapeline files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 134ba0fa9b4b9f..47b2d559989696 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -347,6 +347,15 @@ Module contents Other attributes may exist, but they are private and must not be inspected or relied on. +.. class:: InitVar + + ``InitVar[T]`` type annotations describe variables that are :ref:`init-only + `. Fields annotated with :class:`!InitVar` + are considered pseudo-fields, and thus are neither returned by the + :func:`fields` function nor used in any way except adding them as + parameters to :meth:`~object.__init__` and an optional + :meth:`__post_init__`. + .. function:: fields(class_or_instance) Returns a tuple of :class:`Field` objects that define the fields for this @@ -595,8 +604,8 @@ Init-only variables Another place where :func:`@dataclass ` inspects a type annotation is to determine if a field is an init-only variable. It does this by seeing -if the type of a field is of type ``dataclasses.InitVar``. If a field -is an ``InitVar``, it is considered a pseudo-field called an init-only +if the type of a field is of type :class:`InitVar`. If a field +is an :class:`InitVar`, it is considered a pseudo-field called an init-only field. As it is not a true field, it is not returned by the module-level :func:`fields` function. Init-only fields are added as parameters to the generated :meth:`~object.__init__` method, and are passed to ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] gh-130637: Add validation for numeric response data in `stat()` method (#130646)
https://github.com/python/cpython/commit/a42168d316f0c9a4fc5658dab87682dc19054efb
commit: a42168d316f0c9a4fc5658dab87682dc19054efb
branch: main
author: Kanishk Pachauri
committer: ericvsmith
date: 2025-03-02T08:05:40-05:00
summary:
gh-130637: Add validation for numeric response data in `stat()` method (#130646)
Co-authored-by: Eric V. Smith
files:
A Misc/NEWS.d/next/Library/2025-03-01-02-19-28.gh-issue-130637.swet54w4rs.rst
M Lib/poplib.py
M Lib/test/test_poplib.py
diff --git a/Lib/poplib.py b/Lib/poplib.py
index beb93a0d57cf93..4469bff44b4c45 100644
--- a/Lib/poplib.py
+++ b/Lib/poplib.py
@@ -226,8 +226,19 @@ def stat(self):
retval = self._shortcmd('STAT')
rets = retval.split()
if self._debugging: print('*stat*', repr(rets))
-numMessages = int(rets[1])
-sizeMessages = int(rets[2])
+
+# Check if the response has enough elements
+# RFC 1939 requires at least 3 elements (+OK, message count, mailbox
size)
+# but allows additional data after the required fields
+if len(rets) < 3:
+raise error_proto("Invalid STAT response format")
+
+try:
+numMessages = int(rets[1])
+sizeMessages = int(rets[2])
+except ValueError:
+raise error_proto("Invalid STAT response data: non-numeric values")
+
return (numMessages, sizeMessages)
diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py
index 869f9431b928bb..f1ebbeafe0cfb4 100644
--- a/Lib/test/test_poplib.py
+++ b/Lib/test/test_poplib.py
@@ -289,6 +289,37 @@ def test_pass_(self):
def test_stat(self):
self.assertEqual(self.client.stat(), (10, 100))
+original_shortcmd = self.client._shortcmd
+def mock_shortcmd_invalid_format(cmd):
+if cmd == 'STAT':
+return b'+OK'
+return original_shortcmd(cmd)
+
+self.client._shortcmd = mock_shortcmd_invalid_format
+with self.assertRaises(poplib.error_proto):
+self.client.stat()
+
+def mock_shortcmd_invalid_data(cmd):
+if cmd == 'STAT':
+return b'+OK abc def'
+return original_shortcmd(cmd)
+
+self.client._shortcmd = mock_shortcmd_invalid_data
+with self.assertRaises(poplib.error_proto):
+self.client.stat()
+
+def mock_shortcmd_extra_fields(cmd):
+if cmd == 'STAT':
+return b'+OK 1 2 3 4 5'
+return original_shortcmd(cmd)
+
+self.client._shortcmd = mock_shortcmd_extra_fields
+
+result = self.client.stat()
+self.assertEqual(result, (1, 2))
+
+self.client._shortcmd = original_shortcmd
+
def test_list(self):
self.assertEqual(self.client.list()[1:],
([b'1 1', b'2 2', b'3 3', b'4 4', b'5 5'],
diff --git
a/Misc/NEWS.d/next/Library/2025-03-01-02-19-28.gh-issue-130637.swet54w4rs.rst
b/Misc/NEWS.d/next/Library/2025-03-01-02-19-28.gh-issue-130637.swet54w4rs.rst
new file mode 100644
index 00..83cd6c63c35215
--- /dev/null
+++
b/Misc/NEWS.d/next/Library/2025-03-01-02-19-28.gh-issue-130637.swet54w4rs.rst
@@ -0,0 +1 @@
+Add validation for numeric response data in poplib.POP3.stat() method
___
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: [email protected]
[Python-checkins] gh-130130: Clarify `hash=False` docs in `dataclasses.field` (#130324)
https://github.com/python/cpython/commit/47ace539950fb675d5968736348f0d724ba199f0 commit: 47ace539950fb675d5968736348f0d724ba199f0 branch: main author: Sabfo committer: ericvsmith date: 2025-02-20T02:43:27-05:00 summary: gh-130130: Clarify `hash=False` docs in `dataclasses.field` (#130324) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index e34b2db0210960..f63a01e9570791 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -270,10 +270,11 @@ Module contents string returned by the generated :meth:`~object.__repr__` method. - *hash*: This can be a bool or ``None``. If true, this field is - included in the generated :meth:`~object.__hash__` method. If ``None`` (the - default), use the value of *compare*: this would normally be - the expected behavior. A field should be considered in the hash - if it's used for comparisons. Setting this value to anything + included in the generated :meth:`~object.__hash__` method. If false, + this field is excluded from the generated :meth:`~object.__hash__`. + If ``None`` (the default), use the value of *compare*: this would + normally be the expected behavior, since a field should be included + in the hash if it's used for comparisons. Setting this value to anything other than ``None`` is discouraged. One possible reason to set ``hash=False`` but ``compare=True`` ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-checkins.python.org/ Member address: [email protected]
[Python-checkins] [3.14] gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes. (GH-134766) (#134782)
https://github.com/python/cpython/commit/452d098c0b024266702c64d6a76d908d721c5067 commit: 452d098c0b024266702c64d6a76d908d721c5067 branch: 3.14 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-05-27T09:16:23Z summary: [3.14] gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes. (GH-134766) (#134782) gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes. (GH-134766) (cherry picked from commit 579686d9fb1bccc74c694d569f0a8bf28d9ca85a) Co-authored-by: Eric V. Smith files: M Lib/test/test_tokenize.py diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py index d4b51841891b28..865e0c5b40ddd3 100644 --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -3241,39 +3241,40 @@ def test_exact_flag(self): class StringPrefixTest(unittest.TestCase): -def test_prefixes(self): -# Get the list of defined string prefixes. I don't see an -# obvious documented way of doing this, but probably the best -# thing is to split apart tokenize.StringPrefix. - -# Make sure StringPrefix begins and ends in parens. -self.assertEqual(tokenize.StringPrefix[0], '(') -self.assertEqual(tokenize.StringPrefix[-1], ')') - -# Then split apart everything else by '|'. -defined_prefixes = set(tokenize.StringPrefix[1:-1].split('|')) - -# Now compute the actual string prefixes, by exec-ing all -# valid prefix combinations, followed by an empty string. - -# Try all prefix lengths until we find a length that has zero -# valid prefixes. This will miss the case where for example -# there are no valid 3 character prefixes, but there are valid -# 4 character prefixes. That seems extremely unlikely. - -# Note that the empty prefix is being included, because length -# starts at 0. That's expected, since StringPrefix includes -# the empty prefix. +@staticmethod +def determine_valid_prefixes(): +# Try all lengths until we find a length that has zero valid +# prefixes. This will miss the case where for example there +# are no valid 3 character prefixes, but there are valid 4 +# character prefixes. That seems unlikely. + +single_char_valid_prefixes = set() + +# Find all of the single character string prefixes. Just get +# the lowercase version, we'll deal with combinations of upper +# and lower case later. I'm using this logic just in case +# some uppercase-only prefix is added. +for letter in itertools.chain(string.ascii_lowercase, string.ascii_uppercase): +try: +eval(f'{letter}""') +single_char_valid_prefixes.add(letter.lower()) +except SyntaxError: +pass +# This logic assumes that all combinations of valid prefixes only use +# the characters that are valid single character prefixes. That seems +# like a valid assumption, but if it ever changes this will need +# adjusting. valid_prefixes = set() for length in itertools.count(): num_at_this_length = 0 for prefix in ( -"".join(l) for l in list(itertools.combinations(string.ascii_lowercase, length)) +"".join(l) +for l in itertools.combinations(single_char_valid_prefixes, length) ): for t in itertools.permutations(prefix): for u in itertools.product(*[(c, c.upper()) for c in t]): -p = ''.join(u) +p = "".join(u) if p == "not": # 'not' can never be a string prefix, # because it's a valid expression: not "" @@ -3289,9 +3290,26 @@ def test_prefixes(self): except SyntaxError: pass if num_at_this_length == 0: -break +return valid_prefixes + + +def test_prefixes(self): +# Get the list of defined string prefixes. I don't see an +# obvious documented way of doing this, but probably the best +# thing is to split apart tokenize.StringPrefix. + +# Make sure StringPrefix begins and ends in parens. We're +# assuming it's of the form "(a|b|ab)", if a, b, and cd are +# valid string prefixes. +self.assertEqual(tokenize.StringPrefix[0], '(') +self.assertEqual(tokenize.StringPrefix[-1], ')') + +# Then split apart everything else by '|'. +defined_prefixes = set(tokenize
[Python-checkins] gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes. (#134766)
https://github.com/python/cpython/commit/579686d9fb1bccc74c694d569f0a8bf28d9ca85a
commit: 579686d9fb1bccc74c694d569f0a8bf28d9ca85a
branch: main
author: Eric V. Smith
committer: ericvsmith
date: 2025-05-27T04:49:28-04:00
summary:
gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes.
(#134766)
files:
M Lib/test/test_tokenize.py
diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py
index d4b51841891b28..865e0c5b40ddd3 100644
--- a/Lib/test/test_tokenize.py
+++ b/Lib/test/test_tokenize.py
@@ -3241,39 +3241,40 @@ def test_exact_flag(self):
class StringPrefixTest(unittest.TestCase):
-def test_prefixes(self):
-# Get the list of defined string prefixes. I don't see an
-# obvious documented way of doing this, but probably the best
-# thing is to split apart tokenize.StringPrefix.
-
-# Make sure StringPrefix begins and ends in parens.
-self.assertEqual(tokenize.StringPrefix[0], '(')
-self.assertEqual(tokenize.StringPrefix[-1], ')')
-
-# Then split apart everything else by '|'.
-defined_prefixes = set(tokenize.StringPrefix[1:-1].split('|'))
-
-# Now compute the actual string prefixes, by exec-ing all
-# valid prefix combinations, followed by an empty string.
-
-# Try all prefix lengths until we find a length that has zero
-# valid prefixes. This will miss the case where for example
-# there are no valid 3 character prefixes, but there are valid
-# 4 character prefixes. That seems extremely unlikely.
-
-# Note that the empty prefix is being included, because length
-# starts at 0. That's expected, since StringPrefix includes
-# the empty prefix.
+@staticmethod
+def determine_valid_prefixes():
+# Try all lengths until we find a length that has zero valid
+# prefixes. This will miss the case where for example there
+# are no valid 3 character prefixes, but there are valid 4
+# character prefixes. That seems unlikely.
+
+single_char_valid_prefixes = set()
+
+# Find all of the single character string prefixes. Just get
+# the lowercase version, we'll deal with combinations of upper
+# and lower case later. I'm using this logic just in case
+# some uppercase-only prefix is added.
+for letter in itertools.chain(string.ascii_lowercase,
string.ascii_uppercase):
+try:
+eval(f'{letter}""')
+single_char_valid_prefixes.add(letter.lower())
+except SyntaxError:
+pass
+# This logic assumes that all combinations of valid prefixes only use
+# the characters that are valid single character prefixes. That seems
+# like a valid assumption, but if it ever changes this will need
+# adjusting.
valid_prefixes = set()
for length in itertools.count():
num_at_this_length = 0
for prefix in (
-"".join(l) for l in
list(itertools.combinations(string.ascii_lowercase, length))
+"".join(l)
+for l in itertools.combinations(single_char_valid_prefixes,
length)
):
for t in itertools.permutations(prefix):
for u in itertools.product(*[(c, c.upper()) for c in t]):
-p = ''.join(u)
+p = "".join(u)
if p == "not":
# 'not' can never be a string prefix,
# because it's a valid expression: not ""
@@ -3289,9 +3290,26 @@ def test_prefixes(self):
except SyntaxError:
pass
if num_at_this_length == 0:
-break
+return valid_prefixes
+
+
+def test_prefixes(self):
+# Get the list of defined string prefixes. I don't see an
+# obvious documented way of doing this, but probably the best
+# thing is to split apart tokenize.StringPrefix.
+
+# Make sure StringPrefix begins and ends in parens. We're
+# assuming it's of the form "(a|b|ab)", if a, b, and cd are
+# valid string prefixes.
+self.assertEqual(tokenize.StringPrefix[0], '(')
+self.assertEqual(tokenize.StringPrefix[-1], ')')
+
+# Then split apart everything else by '|'.
+defined_prefixes = set(tokenize.StringPrefix[1:-1].split('|'))
-self.assertEqual(defined_prefixes, valid_prefixes)
+# Now compute the actual allowed string prefixes and compare
+# to what is defined in the tokenize module.
+self.assertEqual(defined_prefixes, self.determine_va
[Python-checkins] gh-134675: Add t-string prefixes to tokenizer module, lexical analysis doc, and add a test to make sure we catch this error in the future. (#134734)
https://github.com/python/cpython/commit/08c78e02fab4a1c9c075637422d621f9c740959a
commit: 08c78e02fab4a1c9c075637422d621f9c740959a
branch: main
author: Eric V. Smith
committer: ericvsmith
date: 2025-05-26T13:49:39-04:00
summary:
gh-134675: Add t-string prefixes to tokenizer module, lexical analysis doc, and
add a test to make sure we catch this error in the future. (#134734)
* Add t-string prefixes to _all_string_prefixes, and add a test to make sure we
catch this error in the future.
* Update lexical analysis docs for t-string prefixes.
files:
M Doc/reference/lexical_analysis.rst
M Lib/test/test_tokenize.py
M Lib/tokenize.py
diff --git a/Doc/reference/lexical_analysis.rst
b/Doc/reference/lexical_analysis.rst
index 6c4a4ea81afe29..b22eb4db7945d1 100644
--- a/Doc/reference/lexical_analysis.rst
+++ b/Doc/reference/lexical_analysis.rst
@@ -489,8 +489,9 @@ String literals are described by the following lexical
definitions:
.. productionlist:: python-grammar
stringliteral: [`stringprefix`](`shortstring` | `longstring`)
- stringprefix: "r" | "u" | "R" | "U" | "f" | "F"
+ stringprefix: "r" | "u" | "R" | "U" | "f" | "F" | "t" | "T"
: | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF"
+ : | "tr" | "Tr" | "tR" | "TR" | "rt" | "rT" | "Rt" | "RT"
shortstring: "'" `shortstringitem`* "'" | '"' `shortstringitem`* '"'
longstring: "'''" `longstringitem`* "'''" | '"""' `longstringitem`* '"""'
shortstringitem: `shortstringchar` | `stringescapeseq`
diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py
index e6b19fe1812d44..d4b51841891b28 100644
--- a/Lib/test/test_tokenize.py
+++ b/Lib/test/test_tokenize.py
@@ -1,6 +1,8 @@
import contextlib
+import itertools
import os
import re
+import string
import tempfile
import token
import tokenize
@@ -3238,5 +3240,59 @@ def test_exact_flag(self):
self.check_output(source, expect, flag)
+class StringPrefixTest(unittest.TestCase):
+def test_prefixes(self):
+# Get the list of defined string prefixes. I don't see an
+# obvious documented way of doing this, but probably the best
+# thing is to split apart tokenize.StringPrefix.
+
+# Make sure StringPrefix begins and ends in parens.
+self.assertEqual(tokenize.StringPrefix[0], '(')
+self.assertEqual(tokenize.StringPrefix[-1], ')')
+
+# Then split apart everything else by '|'.
+defined_prefixes = set(tokenize.StringPrefix[1:-1].split('|'))
+
+# Now compute the actual string prefixes, by exec-ing all
+# valid prefix combinations, followed by an empty string.
+
+# Try all prefix lengths until we find a length that has zero
+# valid prefixes. This will miss the case where for example
+# there are no valid 3 character prefixes, but there are valid
+# 4 character prefixes. That seems extremely unlikely.
+
+# Note that the empty prefix is being included, because length
+# starts at 0. That's expected, since StringPrefix includes
+# the empty prefix.
+
+valid_prefixes = set()
+for length in itertools.count():
+num_at_this_length = 0
+for prefix in (
+"".join(l) for l in
list(itertools.combinations(string.ascii_lowercase, length))
+):
+for t in itertools.permutations(prefix):
+for u in itertools.product(*[(c, c.upper()) for c in t]):
+p = ''.join(u)
+if p == "not":
+# 'not' can never be a string prefix,
+# because it's a valid expression: not ""
+continue
+try:
+eval(f'{p}""')
+
+# No syntax error, so p is a valid string
+# prefix.
+
+valid_prefixes.add(p)
+num_at_this_length += 1
+except SyntaxError:
+pass
+if num_at_this_length == 0:
+break
+
+self.assertEqual(defined_prefixes, valid_prefixes)
+
+
if __name__ == "__main__":
unittest.main()
diff --git a/Lib/tokenize.py b/Lib/tokenize.py
index 559a7aecbde2d1..7e71755068e1df 100644
[Python-checkins] [3.14] gh-134675: Add t-string prefixes to tokenizer module, lexical analysis doc, and add a test to make sure we catch this error in the future. (GH-134734) (#134739)
https://github.com/python/cpython/commit/74f5667bd9293337bc7c1b2fb89d9a8cb0598bc8 commit: 74f5667bd9293337bc7c1b2fb89d9a8cb0598bc8 branch: 3.14 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-05-26T15:24:24-04:00 summary: [3.14] gh-134675: Add t-string prefixes to tokenizer module, lexical analysis doc, and add a test to make sure we catch this error in the future. (GH-134734) (#134739) gh-134675: Add t-string prefixes to tokenizer module, lexical analysis doc, and add a test to make sure we catch this error in the future. (GH-134734) * Add t-string prefixes to _all_string_prefixes, and add a test to make sure we catch this error in the future. * Update lexical analysis docs for t-string prefixes. (cherry picked from commit 08c78e02fab4a1c9c075637422d621f9c740959a) Co-authored-by: Eric V. Smith files: M Doc/reference/lexical_analysis.rst M Lib/test/test_tokenize.py M Lib/tokenize.py diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 6c4a4ea81afe29..b22eb4db7945d1 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -489,8 +489,9 @@ String literals are described by the following lexical definitions: .. productionlist:: python-grammar stringliteral: [`stringprefix`](`shortstring` | `longstring`) - stringprefix: "r" | "u" | "R" | "U" | "f" | "F" + stringprefix: "r" | "u" | "R" | "U" | "f" | "F" | "t" | "T" : | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF" + : | "tr" | "Tr" | "tR" | "TR" | "rt" | "rT" | "Rt" | "RT" shortstring: "'" `shortstringitem`* "'" | '"' `shortstringitem`* '"' longstring: "'''" `longstringitem`* "'''" | '"""' `longstringitem`* '"""' shortstringitem: `shortstringchar` | `stringescapeseq` diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py index e6b19fe1812d44..d4b51841891b28 100644 --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -1,6 +1,8 @@ import contextlib +import itertools import os import re +import string import tempfile import token import tokenize @@ -3238,5 +3240,59 @@ def test_exact_flag(self): self.check_output(source, expect, flag) +class StringPrefixTest(unittest.TestCase): +def test_prefixes(self): +# Get the list of defined string prefixes. I don't see an +# obvious documented way of doing this, but probably the best +# thing is to split apart tokenize.StringPrefix. + +# Make sure StringPrefix begins and ends in parens. +self.assertEqual(tokenize.StringPrefix[0], '(') +self.assertEqual(tokenize.StringPrefix[-1], ')') + +# Then split apart everything else by '|'. +defined_prefixes = set(tokenize.StringPrefix[1:-1].split('|')) + +# Now compute the actual string prefixes, by exec-ing all +# valid prefix combinations, followed by an empty string. + +# Try all prefix lengths until we find a length that has zero +# valid prefixes. This will miss the case where for example +# there are no valid 3 character prefixes, but there are valid +# 4 character prefixes. That seems extremely unlikely. + +# Note that the empty prefix is being included, because length +# starts at 0. That's expected, since StringPrefix includes +# the empty prefix. + +valid_prefixes = set() +for length in itertools.count(): +num_at_this_length = 0 +for prefix in ( +"".join(l) for l in list(itertools.combinations(string.ascii_lowercase, length)) +): +for t in itertools.permutations(prefix): +for u in itertools.product(*[(c, c.upper()) for c in t]): +p = ''.join(u) +if p == "not": +# 'not' can never be a string prefix, +# because it's a valid expression: not "" +continue +try: +eval(f'{p}""') + +# No syntax error, so p is a valid string +# prefix. + +valid_prefixes.add(p) +num_at_this_length += 1 +
[Python-checkins] Fix presentation of dataclasses' `unsafe_hash` default value (#116532)
https://github.com/python/cpython/commit/71f5fafdfb2e509f59cd584d45949c6496f88d41 commit: 71f5fafdfb2e509f59cd584d45949c6496f88d41 branch: main author: Victorien <[email protected]> committer: ericvsmith date: 2025-06-11T21:30:33-04:00 summary: Fix presentation of dataclasses' `unsafe_hash` default value (#116532) Co-authored-by: Adam Turner <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index f18c7cc9c02da6..299c8aa399c25c 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -121,8 +121,11 @@ Module contents :meth:`!__le__`, :meth:`!__gt__`, or :meth:`!__ge__`, then :exc:`TypeError` is raised. - - *unsafe_hash*: If ``False`` (the default), a :meth:`~object.__hash__` method - is generated according to how *eq* and *frozen* are set. + - *unsafe_hash*: If true, force ``dataclasses`` to create a + :meth:`~object.__hash__` method, even though it may not be safe to do so. + Otherwise, generate a :meth:`~object.__hash__` method according to how + *eq* and *frozen* are set. + The default value is ``False``. :meth:`!__hash__` is used by built-in :meth:`hash`, and when objects are added to hashed collections such as dictionaries and sets. Having a ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
[Python-checkins] [3.13] Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (#135417)
https://github.com/python/cpython/commit/9a10b734f164ca5a253ae3a05f4960e3fcbeef2b commit: 9a10b734f164ca5a253ae3a05f4960e3fcbeef2b branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-06-12T01:37:30Z summary: [3.13] Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (#135417) Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (cherry picked from commit 71f5fafdfb2e509f59cd584d45949c6496f88d41) Co-authored-by: Victorien <[email protected]> Co-authored-by: Adam Turner <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 47b2d559989696..432c2b0d7bc000 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -121,8 +121,11 @@ Module contents :meth:`!__le__`, :meth:`!__gt__`, or :meth:`!__ge__`, then :exc:`TypeError` is raised. - - *unsafe_hash*: If ``False`` (the default), a :meth:`~object.__hash__` method - is generated according to how *eq* and *frozen* are set. + - *unsafe_hash*: If true, force ``dataclasses`` to create a + :meth:`~object.__hash__` method, even though it may not be safe to do so. + Otherwise, generate a :meth:`~object.__hash__` method according to how + *eq* and *frozen* are set. + The default value is ``False``. :meth:`!__hash__` is used by built-in :meth:`hash`, and when objects are added to hashed collections such as dictionaries and sets. Having a ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
[Python-checkins] [3.14] Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (#135416)
https://github.com/python/cpython/commit/4a30154fd27f055a97d6543a2cbf71f578b7eed5 commit: 4a30154fd27f055a97d6543a2cbf71f578b7eed5 branch: 3.14 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-06-12T01:36:17Z summary: [3.14] Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (#135416) Fix presentation of dataclasses' `unsafe_hash` default value (GH-116532) (cherry picked from commit 71f5fafdfb2e509f59cd584d45949c6496f88d41) Co-authored-by: Victorien <[email protected]> Co-authored-by: Adam Turner <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index f18c7cc9c02da6..299c8aa399c25c 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -121,8 +121,11 @@ Module contents :meth:`!__le__`, :meth:`!__gt__`, or :meth:`!__ge__`, then :exc:`TypeError` is raised. - - *unsafe_hash*: If ``False`` (the default), a :meth:`~object.__hash__` method - is generated according to how *eq* and *frozen* are set. + - *unsafe_hash*: If true, force ``dataclasses`` to create a + :meth:`~object.__hash__` method, even though it may not be safe to do so. + Otherwise, generate a :meth:`~object.__hash__` method according to how + *eq* and *frozen* are set. + The default value is ``False``. :meth:`!__hash__` is used by built-in :meth:`hash`, and when objects are added to hashed collections such as dictionaries and sets. Having a ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
[Python-checkins] gh-137900: Improve dataclasses frozen parameter documentation (#137937)
https://github.com/python/cpython/commit/7685b8ada84822a7ee9ce51e8ee0e2e35fd60b4e commit: 7685b8ada84822a7ee9ce51e8ee0e2e35fd60b4e branch: main author: Tangyuan <[email protected]> committer: ericvsmith date: 2025-08-20T09:08:45-04:00 summary: gh-137900: Improve dataclasses frozen parameter documentation (#137937) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 299c8aa399c25c..2e4520c823bf3e 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -161,9 +161,11 @@ Module contents :class:`object`, this means it will fall back to id-based hashing). - *frozen*: If true (the default is ``False``), assigning to fields will - generate an exception. This emulates read-only frozen instances. If - :meth:`~object.__setattr__` or :meth:`~object.__delattr__` is defined in the class, then - :exc:`TypeError` is raised. See the discussion below. + generate an exception. This emulates read-only frozen instances. + See the :ref:`discussion ` below. + + If :meth:`~object.__setattr__` or :meth:`~object.__delattr__` is defined in the class + and *frozen* is true, then :exc:`TypeError` is raised. - *match_args*: If true (the default is ``True``), the :attr:`~object.__match_args__` tuple will be created from the list of ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
[Python-checkins] [3.13] gh-137900: Improve dataclasses frozen parameter documentation (GH-137937) (#137991)
https://github.com/python/cpython/commit/47b1c5d74efffbfc72799a83ad7f7d97635340f6 commit: 47b1c5d74efffbfc72799a83ad7f7d97635340f6 branch: 3.13 author: Miss Islington (bot) <[email protected]> committer: ericvsmith date: 2025-08-20T13:15:39Z summary: [3.13] gh-137900: Improve dataclasses frozen parameter documentation (GH-137937) (#137991) gh-137900: Improve dataclasses frozen parameter documentation (GH-137937) (cherry picked from commit 7685b8ada84822a7ee9ce51e8ee0e2e35fd60b4e) Co-authored-by: Tangyuan <[email protected]> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 432c2b0d7bc000..ae474a1f46c29e 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -161,9 +161,11 @@ Module contents :class:`object`, this means it will fall back to id-based hashing). - *frozen*: If true (the default is ``False``), assigning to fields will - generate an exception. This emulates read-only frozen instances. If - :meth:`~object.__setattr__` or :meth:`~object.__delattr__` is defined in the class, then - :exc:`TypeError` is raised. See the discussion below. + generate an exception. This emulates read-only frozen instances. + See the :ref:`discussion ` below. + + If :meth:`~object.__setattr__` or :meth:`~object.__delattr__` is defined in the class + and *frozen* is true, then :exc:`TypeError` is raised. - *match_args*: If true (the default is ``True``), the :attr:`~object.__match_args__` tuple will be created from the list of ___ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
