[issue46771] Add some form of cancel scopes
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue46771> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45670] New .mapping attribute is broken for some existing uses of dict views
Joshua Bronson added the comment: Thank you for confirming that ChainMap.__iter__() would be in the same boat as bidict if a similar .mapping attribute were ever added to dict_keyiterators. The specifics of this issue are interesting, but even more interesting to me is whatever learnings we can generalize from this. After testing that the performance impact would be significant, I created the feature request you suggested in https://bugs.python.org/issue46713. Thanks for suggesting that. In the meantime, I've updated the relevant docstrings: >>> help(b.keys) keys() -> KeysView[~KT] method of bidict.bidict instance A set-like object providing a view on the contained keys. When *b._fwdm* is a :class:`dict`, *b.keys()* returns a *dict_keys* object that behaves exactly the same as *collections.abc.KeysView(b)*, except for - offering better performance - being reversible on Python 3.8+ - having a .mapping attribute in Python 3.10+ that exposes a mappingproxy to *b._fwdm*. >>> help(b.values) values() -> bidict.BidictKeysView[~VT] method of bidict.bidict instance A set-like object providing a view on the contained values. Since the values of a bidict are equivalent to the keys of its inverse, this method returns a set-like object for this bidict's values rather than just a collections.abc.ValuesView. This object supports set operations like union and difference, and constant- rather than linear-time containment checks, and is no more expensive to provide than the less capable collections.abc.ValuesView would be. See :meth:`keys` for more information. etc. Regarding: > The values() call unexpectedly returns an instance of dict_keys(). At first, > I was surprised that this got past the type checker -- you can do set > operations with KeysView but not with a ValuesView. Check out https://github.com/jab/bidict/blob/82f931/bidict/_base.py#L38-L45: ``` class BidictKeysView(t.KeysView[KT], t.ValuesView[KT]): """Since the keys of a bidict are the values of its inverse (and vice versa), the ValuesView result of calling *bi.values()* is also a KeysView of *bi.inverse*. """ dict_keys: t.Type[t.KeysView[t.Any]] = type({}.keys()) BidictKeysView.register(dict_keys) ``` See also https://github.com/python/typeshed/issues/4435 for a request that typeshed use a Protocol (structural typing) here. Thanks again for taking the time to look at my code and discuss so generously. -- ___ Python tracker <https://bugs.python.org/issue45670> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46713] Provide a C implementation of collections.abc.KeysView and friends
New submission from Joshua Bronson : As suggested by @rhettinger in https://bugs.python.org/msg409443, I'm creating a feature request for C implementations of collections.abc.KeysView, ValuesView, and ItemsView. Because these do not currently benefit from C speedups, they're a lot slower than their dict_keys, dict_values, and dict_items counterparts. As a result, libraries that implement custom Mapping types that are backed by dicts are incentivized to override the implementations of keys(), values(), and items() they inherit from collections.abc.Mapping to instead return their backing dicts' mapping views, causing a potential abstraction leak. An example can be found in https://github.com/jab/bidict, which implements bidirectional mapping types that wrap a forward and an inverse dict which are kept in sync with one another. >>> from bidict import * >>> bi = bidict({1: 'one', 2: 'two'}) >>> bi.items() # Overridden for performance: dict_items([(1, 'one'), (2, 'two')]) Ditto for OrderedBidict: >>> OrderedBidict(bi).keys() _OrderedBidictItemsView(OrderedBidict([(1, 'one'), (2, 'two')])) (The _OrderedBidictItemsView is a custom view whose __iter__ uses the implementation inherited by its collections.abc.ItemsView base class so that the correct order is respected, but proxies other method calls through to the backing dict's dict_items object: https://github.com/jab/bidict/blob/2ab42a/bidict/_orderedbidict.py#L90-L150) Here is a microbenchmark of calling __eq__ on an _OrderedBidictItemsView vs. a collections.abc.ItemsView, to estimate the performance impact (using Python 3.10): ❯ set setup ' from collections.abc import ItemsView from bidict import OrderedBidict d = dict(zip(range(), range())) ob = OrderedBidict(d)' ❯ python -m pyperf timeit -s $setup 'ob.items() == d.items()' -o 1.json ❯ python -m pyperf timeit -s $setup 'ItemsView(ob) == d.items()' -o 2.json ❯ pyperf compare_to 2.json 1.json Mean +- std dev: [2] 4.21 ms +- 1.10 ms -> [1] 168 us +- 6 us: 25.13x faster This demonstrates a potentially significant speedup. Similar microbenchmarks for ItemsView vs. dict_items, as well as KeysView vs. both dict_keys and _OrderedBidictKeysView, also indicate similarly significant potential. Note that the performance benefits of this may propagate to other code as well. For example, bidicts' __eq__ methods are implemented in terms of their itemsviews (see https://github.com/jab/bidict/blob/2ab42a/bidict/_base.py#L285-L286), so speeding up bidict.items().__eq__ speeds up bidict.__eq__ commensurately. -- messages: 413020 nosy: jab priority: normal severity: normal status: open title: Provide a C implementation of collections.abc.KeysView and friends ___ Python tracker <https://bugs.python.org/issue46713> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46684] Expose frozenset._hash classmethod
Joshua Bronson added the comment: Thanks for the explanation, Raymond. Regarding: > Lastly, pure python hashable sets based on the ABC are not common This would have implications for other use cases though, that are perhaps more common. See, for example, the following code: https://github.com/jab/bidict/blob/ae9d06/bidict/_frozenbidict.py#L36-L38 This example demonstrates an implementation of a hashable, immutable Mapping type, whose __hash__ implementation returns collections.abc.ItemsView(self)._hash(). Since there are several other libraries I know of that implement hashable/immutable mapping types as well, I thought this might be beneficial enough to users to warrant consideration. -- ___ Python tracker <https://bugs.python.org/issue46684> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46684] Expose frozenset._hash classmethod
New submission from Joshua Bronson : collections.abc.Set provides a _hash() method that includes the following in its docstring: """ Note that we don't define __hash__: not all sets are hashable. But if you define a hashable set type, its __hash__ should call this function. ... We match the algorithm used by the built-in frozenset type. """ Because Set._hash() is currently implemented in pure Python, users face having to make a potentially challenging decision between whether to trade off runtime efficiency vs. space efficiency: >>> hash(frozenset(x)) # Should I use this? >>> Set._hash(x)# Or this? The former requires O(n) memory to create the frozenset, merely to throw it immediately away, but on the other hand gets to use frozenset's __hash__ implementation, which is implemented in C. The latter requires only O(1) memory, but does not get the performance benefit of using the C implementation of this algorithm. Why not expose the C implementation via a frozenset._hash() classmethod, and change Set._hash() to merely call that? Then it would be much clearer that using Set._hash() is always the right answer. -- messages: 412856 nosy: jab priority: normal severity: normal status: open title: Expose frozenset._hash classmethod ___ Python tracker <https://bugs.python.org/issue46684> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45670] New .mapping attribute is broken for some existing uses of dict views
Joshua Bronson added the comment: Dear Raymond, Thanks so much for the detailed response! I wonder if you could explain how this case is different from the following: >>> c = collections.ChainMap({1: 1}, {1: 2}) >>> iter(c) >>> isinstance(c, dict) # it's fine, we <3 duck typing! False Now suppose a new .mapping attribute were added to dict_keyiterator objects in a future version of Python, like the one that was added to dict view objects in Python 3.10. Then, wouldn't ChainMap find itself in a similar predicament as this issue is raising? >>> iter(c).mapping mappingproxy({1: None}) As in my example above, the {1: None} mapping here is an implementation detail of ChainMap.__iter__() that was never intended to be leaked to users. (Ref: <https://github.com/python/cpython/blob/e18d815/Lib/collections/__init__.py#L998-L1002>) Note also that ChainMap.__iter__() returns a dict_keyiterator object even though issubclass(ChainMap, dict) is False. We consider this just fine though, because the dict_keyiterator object currently behaves exactly like the generator object we would get if the code had done a `yield from d` rather than a `return iter(d)`. Except for being faster. This parallels the implementations of bidict.keys()/values()/items(), which currently return dict_keys/dict_values/dict_items objects generated from internal data, that behave exactly like KeysViews(b)/ValuesView(b)/ItemsView(b) would*, except for being faster. And, as this issue is raising, except for this new .mapping attribute in Python 3.10+ now leaking internal state. * I even have the passing property-based tests to prove it: <https://github.com/jab/bidict/pull/217/files#diff-995af13b14fe897c5d200fa97ed88fad03e401b2fc0cc167624d482ea512ba96R431-R459> :) (I also have counterpoints for your other feedback, but wanted to post this part first. And sorry for my delay in responding – hope it's not too late! And finally thanks so much again for considering this and for the time you took to give feedback on bidict – there's literally no better-qualified person in the world. I so appreciate it!) -- ___ Python tracker <https://bugs.python.org/issue45670> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45670] New .mapping attribute is broken for some existing uses of dict views
New submission from Joshua Bronson : As of bpo-40890 (released in Python 3.10), dict views now provide a public .mapping attribute, intended to allow users to recover a mappingproxy pointing to the original mapping. However, this new attribute can actually point to the wrong mapping for some existing uses of dict views. And since the .mapping attribute is read-only, these existing uses have no way to set it to the correct value. My bidict library (see https://github.com/jab/bidict) provides an example of this. A bidict implements a bidirectional mapping by building on top of two dicts (i.e. regular old one-directional mappings) internally -- one for the forward direction, and one for the inverse. When you call e.g. keys() or values() on a bidict, you get back a dict_keys view from one of the backing dicts, because this is a much more optimized implementation of these views than collections.abc.KeysView would be: >>> import bidict >>> b = bidict.bidict(one=1, two=2) >>> b bidict({'one': 1, 'two': 2}) >>> b.keys() dict_keys(['one', 'two']) >>> b.values() dict_keys([1, 2]) However, now that these built-in dict_keys objects provide a .mapping attribute in Python 3.10, it points to one of the internal, backing dicts in this case, which is an implementation detail, rather than to the bidict instance: >>> b.keys().mapping # wrong mappingproxy({'one': 1, 'two': 2}) >>> b.values().mapping # wrong mappingproxy({1: 'one', 2: 'two'}) Instead of the above, you should get: >>> b.keys().mapping # corrected: mappingproxy(bidict({'one': 1, 'two': 2})) >>> b.values().mapping # corrected: mappingproxy(bidict({'one': 1, 'two': 2})) Since the .mapping attribute is read-only, there's no way for bidict to both keep exposing the optimized dict_keys implementations, which up till now have been perfectly correct, while now exposing a correct .mapping attribute for users of Python 3.10+. (Other bidict types demonstrate this problem more by exposing even more obviously-unintended implementation details via this new .mapping attribute: >>> f = bidict.FrozenOrderedBidict(b) >>> f FrozenOrderedBidict([('one', 1), ('two', 2)]) >>> f.keys().mapping # ouch mappingproxy({'one': _Node(prv=..., self=..., nxt=...), 'two': _Node(prv=..., self=..., nxt=...)}) Those internal _Node objects were never meant to be exposed to consumers, they're an implementation detail.) It looks like cases like this were not considered when discussing bpo-40890, and if they had been, I wonder if the implementation would have been accepted as-is. Here are some potential directions for how to improve things for the future: 1. Expose a way for dict view users like bidict to continue to use optimized dict view implementations while opting out of the new .mapping attribute 2. Make the .mapping attribute no longer read-only, so libraries like bidict can set it correctly before exposing it to users 3. Merely update the documentation in https://docs.python.org/3/library/stdtypes.html#:~:text=dictview.mapping,in%20version%203.10. to mention that, because the .mapping attribute is read-only, it may not point to the original, intended mapping, but rather some internal mapping that the user was not intended to be exposed to. Looking forward to hearing your thoughts, and thanks for your consideration. -- components: Interpreter Core messages: 405317 nosy: jab, rhettinger priority: normal severity: normal status: open title: New .mapping attribute is broken for some existing uses of dict views type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue45670> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40890] Dict views should be introspectable
Change by Joshua Bronson : -- nosy: +jab nosy_count: 6.0 -> 7.0 pull_requests: +27187 pull_request: https://github.com/python/cpython/pull/28892 ___ Python tracker <https://bugs.python.org/issue40890> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44963] anext_awaitable is not a collections.abc.Generator
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue44963> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43468] functools.cached_property incorrectly locks the entire descriptor on class instead of per-instance locking
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue43468> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43751] await anext() returns None when default is given
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue43751> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27815] Make SSL suppress_ragged_eofs default more secure
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue27815> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43246] Dict copy optimization violates subclass invariant
New submission from Joshua Bronson : If I understand correctly, it should be an invariant that in the code below, for all "Parent" classes, for all "method"s, Child1.method should return the same result as Child2.method: ``` class Parent: def __init__(self, value): self._value = value def method(self): return self._value class Child1(Parent): pass c1 = Child1(42) result = c1.method() assert result == 42, result class Child2(Parent): def method(self): return super().method() c2 = Child2(42) result = c2.method() assert result == 42, result ``` But when "Parent" is "dict" and method is "__iter__", that is not the case: ``` SHOW_BUG = True class ChildDict1(dict): """Simplification of werkzeug.datastructures.MultiDict.""" def __init__(self): pass if not SHOW_BUG: def __iter__(self): return super().__iter__() def add(self, key, value): self.setdefault(key, []).append(value) def __setitem__(self, key, value): """Like add, but removes any existing key first.""" super().__setitem__(key, [value]) def getall(self, key) -> list: return super().__getitem__(key) def __getitem__(self, key): """Return the first value for this key.""" return self.getall(key)[0] def items(self, multi=False): for (key, values) in super().items(): if multi: yield from ((key, value) for value in values) else: yield key, values[0] def values(self): return (values[0] for values in super().values()) # Remaining overridden implementations of methods # inherited from dict are elided for brevity. cd1 = ChildDict1() assert dict(cd1) == {} cd1[1] = "one" cd1.add(1, "uno") assert cd1.getall(1) == ["one", "uno"] assert list(cd1.items()) == [(1, "one")] assert list(cd1.values()) == [ "one"] assert dict(cd1) == {1: "one"}, cd1 # XXX ``` If you run the above as-is, the line marked "XXX" will trigger an AssertionError demonstrating the unexpected behavior. If you change SHOW_BUG to False, it won’t. Is it intended that toggling the value of SHOW_BUG in this code causes different results? You can visit https://repl.it/@jab/dict-subclass-copy-surprise to run the examples above directly in your browser. -- messages: 387191 nosy: jab priority: normal severity: normal status: open title: Dict copy optimization violates subclass invariant type: behavior ___ Python tracker <https://bugs.python.org/issue43246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14757] INCA: Inline Caching meets Quickening in Python 3.3
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue14757> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions
Joshua Bronson added the comment: Please see https://github.com/python/cpython/pull/23847 for the C implementation of aiter and anext added to builtins, as requested. -- title: add aiter() and anext() functions to operator module -> add aiter() and anext() functions ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Change by Joshua Bronson : -- pull_requests: +22708 pull_request: https://github.com/python/cpython/pull/23847 ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Joshua Bronson added the comment: Nice to see there is still interest in this from someone else! Thanks, John. Are any core developers still interested in this? If I can get a new PR together that adds C implementations of `aiter` and `anext` to builtins, would a committer still be interested in reviewing the patch? A week from Friday, I'll have a rare and precious opportunity to spend the day contributing to open source. I have a bunch of different things I could work on, but would work on this if there is still interest. Thanks and hope this finds you all well. -- ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7946] Convoy effect with I/O bound threads and New GIL
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue7946> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41303] perf_counter result does not count system sleep time in Mac OS
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue41303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41451] Cannot subclass typing.Generic with __weakref__ slot in Python 3.6
Joshua Bronson added the comment: Thanks so much, @oremanj! Indeed, merely subscripting the class triggers the bug, and your 'del slots' workaround does the trick! For completeness, there is an updated (yet more minimal) repro below/attached. """Repro for Python 3.6 slots + weakref + typing.Generic subclass bug.""" from typing import Generic, TypeVar from weakref import ref T = TypeVar("T") class MyGeneric(Generic[T]): """MyGeneric works as expected. >>> example = MyGeneric() >>> example <__main__.MyGeneric object at ...> >>> example._other <__main__.MyGeneric object at ...> >>> example._other._other >>> from pickle import dumps, loads >>> pickled = dumps(example) >>> roundtripped = loads(pickled) >>> roundtripped <__main__.MyGeneric object at ...> """ __slots__ = ("_other", "__weakref__") def __init__(self) -> None: self._init_other() def _init_other(self) -> None: other = self.__class__.__new__(self.__class__) other._other = ref(self) self._other = other def __getstate__(self) -> dict: """Needed to enable pickling due to use of __slots__ and weakrefs.""" return {} def __setstate__(self, _) -> None: """Needed because use of __slots__ would prevent unpickling otherwise.""" self._init_other() # Merely the following is enough to trigger the bug on Python 3.6: MyGeneric[T] # This works around the issue if run first (thanks @oremanj): del MyGeneric.__slots__ # does not actually 'unslot' the class if __name__ == "__main__": import doctest doctest.testmod(optionflags=doctest.ELLIPSIS) -- Added file: https://bugs.python.org/file49357/bpo41451-repro-min+workaround.py ___ Python tracker <https://bugs.python.org/issue41451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41451] Cannot subclass typing.Generic with __weakref__ slot in Python 3.6
Joshua Bronson added the comment: Whittled this down to an even more minimal repro: """Repro for Python 3.6 slots + weakref + typing.Generic subclass bug.""" from typing import Generic, TypeVar from weakref import ref T = TypeVar("T") class MyGeneric(Generic[T]): """MyGeneric works as expected. >>> example = MyGeneric() >>> from pickle import dumps, loads >>> pickled = dumps(example) >>> roundtripped = loads(pickled) >>> roundtripped <__main__.MyGeneric object at ...> """ __slots__ = ("_other", "__weakref__") def __init__(self) -> None: self._init_other() def _init_other(self) -> None: other = self.__class__.__new__(self.__class__) other._other = self self._other = ref(other) def __getstate__(self) -> dict: """Needed to enable pickling due to use of __slots__ and weakrefs.""" return {} def __setstate__(self, _) -> None: """Needed because use of __slots__ would prevent unpickling otherwise.""" self._init_other() # So far so good, but now let's make a subclass. # The following class definition works on Python > 3.6, but fails on 3.6 with # TypeError: __weakref__ slot disallowed: either we already got one, or __itemsize__ != 0 class FailsInPy36(MyGeneric[T]): """Minimal repro. >>> repro = FailsInPy36() >>> repro <__main__.FailsInPy36 object at ...> """ if __name__ == "__main__": import doctest doctest.testmod(optionflags=doctest.ELLIPSIS) -- title: Class with __weakref__ slot cannot inherit from typing.Generic subclass -> Cannot subclass typing.Generic with __weakref__ slot in Python 3.6 Added file: https://bugs.python.org/file49355/bpo41451-repro-min.py ___ Python tracker <https://bugs.python.org/issue41451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41451] Class with __weakref__ slot cannot inherit from typing.Generic subclass
Joshua Bronson added the comment: It turns out that this bug reproduces with any subclass of the generic type with a weakref slot, even without any multiple inheritance going on. For example: class Invertible2(Invertible[KT1, KT2]): pass is enough to trigger this bug along with the Invertible class in my previous example. Attaching the more minimal repro with this comment, and renaming the issue to remove the reference to multiple inheritance. -- title: Class with __weakref__ slot cannot inherit from multiple typing.Generic classes -> Class with __weakref__ slot cannot inherit from typing.Generic subclass Added file: https://bugs.python.org/file49353/bpo41451-repro.py ___ Python tracker <https://bugs.python.org/issue41451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41451] Class with __weakref__ slot cannot inherit from multiple typing.Generic classes
New submission from Joshua Bronson : This appears to be a bug in Python 3.6 that I hit while trying to add type hints to my bidirectional mapping library (https://bidict.rtfd.io). Pasting a working, minimal repro below for easier inline viewing, and also attaching for easier downloading and running. Please let me know if there is a workaround that would allow me to continue to support Python 3.6 after adding type hints without having to remove the use of slots and weak references. I spent a while trying to find one first but was then encouraged to report this by @ethanhs. Thanks in advance for any pointers you may have. #!/usr/bin/env python3 """Repro for Python 3.6 slots + weakref + typing.Generic bug.""" from typing import Iterator, Mapping, MutableMapping, TypeVar from weakref import ref KT1 = TypeVar("KT1") KT2 = TypeVar("KT2") class Invertible(Mapping[KT1, KT2]): """A one-element mapping that is generic in two key types with a reference to its inverse. ...which in turn holds a (weak) reference back to it. >>> element = Invertible("H", 1) >>> element >>> element.inverse >>> element.inverse.inverse >>> element.inverse.inverse is element True >>> dict(element.items()) {'H': 1} >>> dict(element.inverse.items()) {1: 'H'} >>> list(element) ['H'] Uses the __slots__ optimization, and uses weakrefs for references in one direction to avoid strong reference cycles. And still manages to support pickling to boot! >>> from pickle import dumps, loads >>> pickled = dumps(element) >>> roundtripped = loads(pickled) >>> roundtripped """ # Each instance has (either a strong or a weak) reference to its # inverse instance, which has a (weak or strong) reference back. __slots__ = ("_inverse_strong", "_inverse_weak", "__weakref__", "key1", "key2") def __init__(self, key1: KT1, key2: KT2) -> None: self._inverse_weak = None self._inverse_strong = inverse = self.__class__.__new__(self.__class__) self.key1 = inverse.key2 = key1 self.key2 = inverse.key1 = key2 inverse._inverse_strong = None inverse._inverse_weak = ref(self) def __len__(self) -> int: return 1 def __iter__(self) -> Iterator[KT1]: yield self.key1 def __getitem__(self, key: KT1) -> KT2: if key == self.key1: return self.key2 raise KeyError(key) def __repr__(self) -> str: return f"<{self.__class__.__name__} key1={self.key1!r} key2={self.key2!r}>" @property def inverse(self) -> "Invertible[KT2, KT1]": """The inverse instance.""" if self._inverse_strong is not None: return self._inverse_strong inverse = self._inverse_weak() if inverse is not None: return inverse # Refcount of referent must have dropped to zero, # as in `Invertible().inverse.inverse`, so init a new one. self._inverse_weak = None self._inverse_strong = inverse = self.__class__.__new__(self.__class__) inverse.key2 = self.key1 inverse.key1 = self.key2 inverse._inverse_strong = None inverse._inverse_weak = ref(self) return inverse def __getstate__(self) -> dict: """Needed to enable pickling due to use of __slots__ and weakrefs.""" state = {} for cls in self.__class__.__mro__: slots = getattr(cls, '__slots__', ()) for slot in slots: if hasattr(self, slot): state[slot] = getattr(self, slot) # weakrefs can't be pickled. state.pop('_inverse_weak', None) # Added back in __setstate__ below. state.pop('__weakref__', None) # Not added back in __setstate__. Python manages this one. return state def __setstate__(self, state) -> None: """Needed because use of __slots__ would prevent unpickling otherwise.""" for slot, value in state.items(): setattr(self, slot, value) self._inverse_weak = None self._inverse_strong = inverse = self.__class__.__new__(self.__class__) inverse.key2 = self.key1 inverse.key1 = self.key2 inverse._inverse_strong = None inverse._inverse_weak = ref(self) # So far so good, but now let's make a mutable version. # # The following class definition works on Python &g
[issue40816] Add missed AsyncContextDecorator to contextlib
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue40816> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22239] asyncio: nested event loop
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue22239> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13322] The io module doesn't support non-blocking files
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue13322> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32561] Add API to io objects for cache-only reads/writes
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue32561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32884] Adding the ability for getpass to print asterisks when password is typed
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue32884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue31122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36054] Way to detect CPU count inside docker container
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue36054> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Joshua Bronson added the comment: If the deciders prefer to have these in the operator module for 3.8 (as Yury and Jelle requested above), my PR from a few months ago which implements this (https://github.com/python/cpython/pull/8895) can still be merged with no conflicts. I don't think any other changes to that patch are requested before it can be merged (i.e. it's only stalled on the decision whether to add these to builtins or operator), but I can still make time to address any new requested code changes if these are to go in operator. If these are to go in builtins instead, @nanjekyejoannah has volunteered to pick that up. So it seems like this can move forward one way or the other once we have a decision on operator vs. builtins. -- ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32956] python 3 round bug
Joshua Bronson added the comment: I spent a few minutes with git blame/checkout/show and so far have found https://bugs.python.org/issue1869 (via https://github.com/python/cpython/commit/e6a076d). Still reading -- looks like there were a number of different changes made to round() at the same time for various reasons -- so maybe changing from round_half_up to round_half_even was necessary for the other improvements, and it couldn't have been exposed as a separate function? Or maybe that was just never proposed? -- ___ Python tracker <https://bugs.python.org/issue32956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32956] python 3 round bug
Joshua Bronson added the comment: Thanks, Mark. Yes, I saw where Tim said round-half-even should be the default, but I didn't see any proposal to expose it as e.g. math.round_half_even() instead, nor a more complete look at what other languages do. That, along with the subject being 2.6 and not 3, made me think this change in Python 3 must have been discussed more fully elsewhere. Was it not? And I agree -- nowhere have I been proposing changing "round" again. My proposals have been: 1. Update the round() docs to make the documentation of this behavior less buried, 2. include a (brief) justification (possibly even just a link to http://wiki.c2.com/?BankersRounding or some more-authoritative document), and 3. link to where else this change in Python 3 was discussed more, if anywhere, or else confirm this change was made based on no additional analysis that we can find written down. It'd also be interesting to hear if this is something we wish we'd done differently now, but that shouldn't distract from 1, 2, and 3. -- ___ Python tracker <https://bugs.python.org/issue32956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32956] python 3 round bug
Joshua Bronson added the comment: Thanks Serhiy, I read the Python-Dev thread you linked to, but that doesn't resolve the issues: - Its topic is Python 2.6 (where this behavior does not occur) rather than Python 3 (where it does). - A few messages into the thread Guido does address Python 3, but in fact says "I think the consensus is against round-to-even in 3.0" (see https://mail.python.org/pipermail/python-dev/2008-January/075897.html). - There is no discussion of the fact that this behavior differs from the function named "round" in all the other programming languages I mentioned, and whether it would therefore be better exposed as an additional function (e.g. "round_to_even" or "round_unbiased", and in the math or statistics package rather than builtins). Surprisingly, Excel is the only other programming environment I saw discussed in the thread. (And round(2.5) == 3 there.) So that all suggests there must be some other thread or issue where this change for Python 3 have been discussed, but I looked again and could not find it. The C "rint" example you gave just seems to prove the point that this behavior should have a distinct name from "round". Regarding: > It is a common knowledge that rounding half-to-even is what users want in > most cases I don't think that's common knowledge; seems like citation needed? Based on all the other languages where this differs (not to mention Python 2), it's not clear users would want Python 3 to be the only different one. And this is definitely a surprise for the majority of programmers, whose experience with "round" is how it works everywhere else. (This is making it into pywat after all: https://github.com/cosmologicon/pywat/pull/40) I can submit a PR for at least updating the docs about this (as per my previous comment) if that would be welcomed. -- ___ Python tracker <https://bugs.python.org/issue32956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32956] python 3 round bug
Joshua Bronson added the comment: This was so surprising to me that I had to check some other languages that I had handy. It turns out that not one of JavaScript, Ruby, Perl, C++, Java, Go, or Rust agrees with Python. In fact they all agreed with one another that 2.5 should round to 3. Examples below. I understand from https://github.com/cosmologicon/pywat/pull/40#discussion_r219962259 that "to always round up... can theoretically skew the data" but it's not clear why that's a good enough reason to differ from the "round" function in all these other languages (as opposed to e.g. offering this alternative behavior in some additional "round_unskewed" function). I assume the rationale for having Python 3's "round" differ from that of so many other languages was written down when this decision was made, but I searched and couldn't find it. Could anyone link to it in a comment here? And would it be worth including rationale and a larger callout in the https://docs.python.org/3/library/functions.html#round docs? The documentation of this behavior is a bit buried among other things, and the rationale for it is missing entirely. $ node -e 'console.log(Math.round(2.5))' 3 $ ruby -e 'puts (2.5).round()' 3 $ perl -e 'use Math::Round; print round(2.5)' 3 $ cat test_round.cpp #include #include int main(void) { printf("%f\n", round(2.5)); } $ g++ test_round.cpp && ./a.out 3.00 $ cat TestRound.java class TestRound { public static void main(String[] args) { System.out.println(Math.round(2.5)); } } $ javac TestRound.java && java TestRound 3 $ cat test_round.go package main import "fmt" import "math" func main() { fmt.Println(math.Round(2.5)) } $ go build test_round.go && ./test_round 3 $ cat test_round.rs fn main() { println!("{}", (2.5_f64).round()); } $ rustc test_round.rs && ./test_round 3 -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue32956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Joshua Bronson added the comment: Interesting, thanks Yury! I only saw the discussion here which implied these wouldn't go directly into builtins for 3.8 (and I searched here and in GitHub, and couldn't find the PR you mentioned either), so I'm curious if that was tracked somewhere. It'd be unfortunate if the work I did on that PR couldn't be used, but I'd be even happier to have these as builtins. Thanks again for reviewing when you're back, and have a wonderful vacation in the meantime! -- ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Joshua Bronson added the comment: Updating the issue title to reflect the decision to add these to the operator module rather than to built-ins. Submitted a PR here: https://github.com/python/cpython/pull/8895 -- title: aiter() and anext() built-in functions -> add aiter() and anext() functions to operator module ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] aiter() and anext() built-in functions
Change by Joshua Bronson : -- keywords: +patch pull_requests: +8368 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20849] add exist_ok to shutil.copytree
Joshua Bronson added the comment: I submitted a new PR in https://github.com/python/cpython/pull/8792 that addresses the outstanding concerns in @ofekmeister's original PR. It includes passing tests and passes all the GitHub PR status checks. Does it look ok to merge? Thanks. -- ___ Python tracker <https://bugs.python.org/issue20849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20849] add exist_ok to shutil.copytree
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue20849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20849] add exist_ok to shutil.copytree
Change by Joshua Bronson : -- pull_requests: +8266 ___ Python tracker <https://bugs.python.org/issue20849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] aiter() and anext() built-in functions
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33018] Improve issubclass() error checking and message
Joshua Bronson added the comment: I'll share the use case that prompted me to submit this PR in the first place. I am the author of bidict (https://pypi.python.org/pypi/bidict), which provides a bidirectional dict class. A bidict is just like a dict, except it automatically maintains its inverse bidict, which is accessible via its .inv attribute. To prevent a bidict and its inverse from creating a strong reference cycle, a weak ref is used to store the reference one direction. bidicts implement my BidirectionalMapping ABC, which extends collections.abc.Mapping to include the .inv abstractproperty. BidirectionalMapping overrides __subclasshook__ so that outside implementations that don't subclass it explicitly may still be considered subclasses. Recently, I tried something like `issublass('foo', BidirectionalMapping)`, and got the "cannot create weak reference to 'str' object" error. Because this error message differs from the (much more helpful) "arg 1 must be a class" error message that you get when you do e.g. `issubclass('foo', Mapping)`, I thought there might be a bug somewhere in my code. Then I looked deeper and found where this is really coming from. I experimented more and noticed that `issubclass('foo', Reversible)` raises AttributeError, which isn't even the same type of error. The exceptions that are raised in these cases seem like an abstraction leak. The error messages do not help users immediately realize what they did wrong and how they can fix it; more knowledge of internals is required to make sense of what's going on than should be needed. The inconsistency in these errors is a further problem. The same mistake should not cause three different errors unless there is some really good reason. This seems unintentional. Can any of the original authors say whether this is working as intended or if this is in fact an oversight? The current behavior causes confusion for both less experienced and more experienced Python users alike. (Would anyone else here have correctly predicted all of the different errors that the examples above cause? How many other Python experts could have?) For less experienced users, Python's general consistency and predictability, lack of gotchas, and good errors are some of its best features. This is such an exception that it seems like a bug. I'm happy for some other patch than the one I submitted in https://github.com/python/cpython/pull/5944 to land if necessary, as long as something fixes this. And fwiw, +1 for 3.7, unless anyone can demonstrate any credible risk. Thanks for your consideration :) -- ___ Python tracker <https://bugs.python.org/issue33018> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33018] Improve issubclass() error checking and message
New submission from Joshua Bronson : Creating this issue by request of INADA Naoki to discuss my proposed patch in https://github.com/python/cpython/pull/5944. Copy/pasting from that PR: If you try something like issubclass('not a class', str), you get a helpful error message that immediately clues you in on what you did wrong: >>> issubclass('not a class', str) TypeError: issubclass() arg 1 must be a class ("AHA! I meant isinstance there. Thanks, friendly error message!") But if you try this with some ABC, the error message is much less friendly! >>> from some_library import SomeAbc >>> issubclass('not a class', SomeAbc) Traceback (most recent call last): File "", line 1, in File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 230, in __subclasscheck__ cls._abc_negative_cache.add(subclass) File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_weakrefset.py", line 84, in add self.data.add(ref(item, self._remove)) TypeError: cannot create weak reference to 'str' object ("WTF just went wrong?" Several more minutes of head-scratching ensues. Maybe a less experienced Python programmer who hits this hasn't seen weakrefs before and gets overwhelmed, maybe needlessly proceeding down a deep rabbit hole before realizing no knowledge of weakrefs was required to understand what they did wrong.) Or how about this example: >>> from collections import Reversible >>> issubclass([1, 2, 3], Reversible) Traceback (most recent call last): File "", line 1, in File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 207, in __subclasscheck__ ok = cls.__subclasshook__(subclass) File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_collections_abc.py", line 305, in __subclasshook__ return _check_methods(C, "__reversed__", "__iter__") File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_collections_abc.py", line 73, in _check_methods mro = C.__mro__ AttributeError: 'list' object has no attribute '__mro__' Here you don't even get the same type of error (AttributeError rather than TypeError), which seems unintentionally inconsistent. This trivial patch fixes this, and will hopefully save untold numbers of future Python programmers some time and headache. Let me know if any further changes are required, and thanks in advance for reviewing. -- messages: 313376 nosy: inada.naoki, izbyshev, jab, serhiy.storchaka priority: normal pull_requests: 5781 severity: normal status: open title: Improve issubclass() error checking and message type: enhancement versions: Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue33018> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32999] issubclass(obj, abc.ABC) causes a segfault
Change by Joshua Bronson : -- nosy: +jab ___ Python tracker <https://bugs.python.org/issue32999> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32999] issubclass(obj, abc.ABC) causes a segfault
Change by Joshua Bronson : -- pull_requests: +5780 ___ Python tracker <https://bugs.python.org/issue32999> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
Joshua Bronson added the comment: For the record, it looks like Victor Stinner suggested doing this in https://mail.python.org/pipermail/python-dev/2016-September/146349.html Brett Cannon replied in https://mail.python.org/pipermail/python-dev/2016-September/146350.html to suggest adding "ordered mapping" to the glossary instead. I took a quick look at https://docs.python.org/3.6/glossary.html and don't see it there. -- nosy: +brett.cannon, haypo ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
Joshua Bronson added the comment: Sorry to hear but thanks for the consideration. To follow up on your comments: > nice to see Guido's reasons for giving a -0 on the proposal. (Guido was giving his -0 on a proposal for collections.abc.Ordered, whereas the main proposal here is collections.abc.OrderedMapping, sorry if that was confusing.) > I would add that I haven't see any code in the wild that would benefit from > testing isinstance(m, OrderedMapping) and taking some different action > depending on the result. Just to give you another data point, bidict.orderedbidict has been released into the wild for a little over 10 months, and people are using it. OrderedMapping would give users the following benefits: >>> noble = orderedbidict([(2, 'helium'), (10, 'neon'), (18, 'argon')]) >>> b = bidict(noble) # Discards the order. >>> b # Calls BidictBase.__repr__. bidict({10: 'neon', 18: 'argon', 2: 'helium'}) >>> noble # Also calls BidictBase.__repr__. orderedbidict([('Cairo', 'Egypt'), ('Lima', 'Peru'), ('Tokyo', 'Japan')]) >>> # i.e. BidictBase.__repr__ interrogates isinstance(self, OrderedMapping) >>> # to reflect any ordering correctly in the string representation. >>> # OrderedMapping.__eq__ also checks isinstance(other, OrderedMapping) >>> # to conditionally turn on order-sensitivity: >>> noble == collections.OrderedDict(noble.items()) True >>> noble == collections.OrderedDict(reversed(noble.items())) False >>> noble == dict(reversed(noble.items())) # order-insensitive True (collections.OrderedDict and sortedcontainers.SortedDict could reap these same benefits from an OrderedMapping class.) > Grant's sortedcollection is more sequence-like than mapping-like and the > bidict has its own interesting properties and neither are substitutable for > an OrderedDict. Hm, I don't quite follow this. OrderedDict, SortedDict, and orderedbidict are all OrderedMappings, and should therefore be passable to any function that expects an OrderedMapping. This part I do follow: > In other words, the only properties these cluster on is the ordered equality > feature. IMO, that particular feature hasn't established itself as being > valuable It is an interesting idea but I believe it that it would clutter > the module, slightly drowning-out the ABCs that have more broad and > established utility. But based on this rationale, I would have thought Reversible wouldn't have been added. It was only after I found https://bugs.python.org/issue25987 the other day and saw that Reversible will be in 3.6 that I thought OrderedMapping had a case of similar strength, and so it seemed worth taking the time to try to contribute it for 3.7. In any case, thanks again for your consideration, and if you have any further thoughts to share, it'd be much appreciated. -- nosy: +abarnert ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
Joshua Bronson added the comment: I only just found the "[Python-ideas] Adding collections.abc.Ordered" thread at https://mail.python.org/pipermail/python-ideas/2015-November/037146.html - sorry for not seeing it sooner. Looking forward to catching up on what I missed. -- ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
Joshua Bronson added the comment: Come to think of it, to be exact, rather than extending Reversible, OrderedMapping could extend a narrower interface, something like collections.abc.Ordered, along with extending Mapping. (Reversible implies Ordered, but Ordered does not imply Reversible: a singly-linked list is Ordered but not Reversible, for example.) I guess we don't currently have Ordered because it wouldn't necessarily add any abstractmethods beyond what Iterable already provides. But it could provide a generic, concrete __eq__ implementation, something like: class Ordered(Iterable): def __eq__(self, other): if not isinstance(other, Ordered): return NotImplemented return all(i == j for (i, j) in zip(self, other)) which is not something that any existing collections.abc Iterable currently provides. Even if an Ordered interface that's totally empty were available in the standard library, then Iterables across diparate codebases could opt into extending it as a standard signal that they iterate over their elements in a well-defined order, and could participate in other polymorphic code. Currently we have no such standard signal, do we? -- ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
Joshua Bronson added the comment: This patch improves the OrderedMapping.__eq__ implementation to be more generic in the case that ``other`` is an unordered Mapping of the same length as ``self``. -- Added file: http://bugs.python.org/file45805/jab-orderedmapping-2.patch ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28912] collections.abc.OrderedMapping
New submission from Joshua Bronson: Since code can be clearer than prose, I just sketched this idea out in the attached patch. Please take a look at it as a minimum demonstration of the concept. Rationale: The Python standard library provides collections.OrderedDict, along with several ABCs which OrderedDict extends, such as collections.abc.Mapping and (as of 3.6) collections.abc.Reversible, to enable better composability and interoperability. Currently there is no collections.abc.OrderedMapping ABC, but I wonder if there should be. OrderedMapping could provide a concrete, generic implementation of __eq__, that performed an order-sensitive comparison when passed another OrderedMapping, while performing an order-insensitive comparison when passed an unordered Mapping. Then OrderedDict could derive from OrderedMapping, inheriting its generic __eq__ implementation, rather than implementing its own __eq__ method. Currently, OrderedDict's own __eq__ implementation does ``isinstance(other, OrderedDict)`` to decide whether to perform an order-sensitive comparison, which thwarts interoperability. OrderedMapping could also derive from Reversible, signaling that classes that extend it implement __reversed__. The interoperability gain here is not just theoretical. Several packages are published on PyPI which would be able to interoperate better if they could all extend a common OrderedMapping interface. Grant Jenks' sortedcontainers.SortedDict[1] and my own bidict.OrderedBidict[2] are two particularly popular examples. Thanks for your consideration, and look forward to your feedback. [1] http://www.grantjenks.com/docs/sortedcontainers/sorteddict.html [2] https://bidict.readthedocs.io/en/dev/other-bidict-types.html#orderedbidict -- components: Library (Lib) files: jab-orderedmapping-1.patch keywords: patch messages: 282742 nosy: gvanrossum, jab, rhettinger priority: normal severity: normal status: open title: collections.abc.OrderedMapping type: enhancement Added file: http://bugs.python.org/file45804/jab-orderedmapping-1.patch ___ Python tracker <http://bugs.python.org/issue28912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27350] Compact and ordered dict
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue27350> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
Joshua Bronson added the comment: Though come to think of it, the issue you raised with using "from" in the method name wouldn't apply here, since the static method is on the bool class, and the method does return bool. -- ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
Joshua Bronson added the comment: Actually, looks like the version of the patch I attached did use the name `bool.from_config_str`, sorry about that -- I'll attach a new patch renaming this to `bool.parse_config_str` if there is interest in further consideration. -- ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
Joshua Bronson added the comment: Hi Raymond, I'm a bit confused by your comment. The patch I attached to my previous message adds the static method `bool.parse_config_str`. So there's no need to `import configparser` to use this, and "from" is no longer included in the proposed method name. Could you please take a look at my previous message, if you missed it, and reopen this if appropriate? -- ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
Joshua Bronson added the comment: My friend @novalis_dt and I worked up a patch for this including tests (attached). First time working with the CPython codebase but hope it's a reasonable start. Here's a preview: ~> ./python.exe Python 3.6.0a0 (default:9f8b5053e30d+, Nov 1 2015, 18:38:37) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> trues = ['true', 'yes', 'on', '1'] >>> [bool.from_config_str(i) for i in trues] [True, True, True, True] >>> falses = ['false', 'no', 'off', '0', ''] >>> [bool.from_config_str(i) for i in falses] [False, False, False, False, False] I would be happy to try to make any additional changes necessary (including changing ConfigParser.getboolean to use the new method) if there is interest. Thanks! -- keywords: +patch nosy: +novalis_dt Added file: http://bugs.python.org/file40923/bool_from_config_str.patch ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17006] Add advice on best practices for hashing secrets
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue17006> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
Joshua Bronson added the comment: One way this could be offered is as a new static method on bool (something like bool.parse_str?), but I of course defer to the better judgment of the Python core developers. I'd be happy to take a crack at a patch adding it wherever you like, if you like. -- ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25243] decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function
New submission from Joshua Bronson: ConfigParser.getboolean[1] has logic to convert strings like '0' and 'False' to False. This logic is generally useful in other contexts and need not be coupled to ConfigParser. Would you consider accepting a patch that factored this string-to-boolean logic out of ConfigParser into a standalone function, and changed ConfigParser.getboolean internally to call that? Thanks for your consideration. [1] https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.getboolean -- messages: 251664 nosy: jab priority: normal severity: normal status: open title: decouple string-to-boolean logic from ConfigParser.getboolean and offer as separate function type: enhancement ___ Python tracker <http://bugs.python.org/issue25243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24286] Should OrderedDict.viewitems compare equal to dict.viewitems when the items are equal?
New submission from Joshua Bronson: Is it intentional that the second assertion in the following code fails? ``` from collections import OrderedDict d = dict(C='carbon') o = OrderedDict(d) assert d == o assert d.viewitems() == o.viewitems() ``` Since d == o, I'm surprised that d.viewitems() != o.viewitems(). If that's intentional, I'd love to understand the rationale. Note: I hit this while testing a library I authored, https://pypi.python.org/pypi/bidict, which provides a https://en.wikipedia.org/wiki/Bidirectional_map implementation for Python, so I'm especially keen to understand all the subtleties in this area. Thanks in advance. -- messages: 244066 nosy: jab priority: normal severity: normal status: open title: Should OrderedDict.viewitems compare equal to dict.viewitems when the items are equal? versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue24286> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23894] lib2to3 doesn't recognize rb'...' as a raw byte string in Python 3
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue23894> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23548] TypeError in event loop finalizer, new in Python 3.4.3
Joshua Bronson added the comment: Not sure if it's related / helpful but just in case, since upgrading from 3.4.2 to 3.4.3, I'm now seeing this printed to stderr sometimes when my program exits: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: -- ___ Python tracker <http://bugs.python.org/issue23548> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23548] TypeError in event loop finalizer, new in Python 3.4.3
Joshua Bronson added the comment: Quoting Victor Stinner: > I may workaround the bug during Python finalization if more users report > this issue. Read the above so reporting I'm hitting this too fwiw. Thanks for the great work on asyncio. -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue23548> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18652] Add itertools.first_true (return first true item in iterable)
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue18652> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13035] "maintainer" value clear the "author" value when registering
New submission from Joshua Bronson : This issue was originally opened in the PyPI tracker but was dismissed on the theory that it's a bug in Python: https://sourceforge.net/tracker/index.php?func=detail&aid=3396924&group_id=66150&atid=513503 """ If in one package's setup.py I provide maintainer (with email) and author (whith email) after the "python setup.py register ... upload" I create a new package where I see the maintainer as a creator. If I manually fix it through the pypi user interface it works, so seems that this is only a bug in the register procedure. """ -- messages: 14 nosy: jab priority: normal severity: normal status: open title: "maintainer" value clear the "author" value when registering ___ Python tracker <http://bugs.python.org/issue13035> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7734] discuss mark-as-invalid trick in heapq docs
Joshua Bronson added the comment: Thanks for the great additions. -- ___ Python tracker <http://bugs.python.org/issue7734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7734] discuss mark-as-invalid trick in heapq docs
New submission from Joshua Bronson : Though the heapq module does not support changing the priority of a particular element of the heap (a necessary operation for the A* search family of algorithms), such an element can be marked as invalid and a new element can be added with different priority. Any element marked as invalid that makes it to the top of the heap can simply be popped off and ignored. Users who haven't seen this trick before might mistakenly think the heapq module does not provide sufficient operations to implement A* search. Please see the recent thread on comp.lang.python for more background: http://groups.google.com/group/comp.lang.python/browse_frm/thread/8adc3ce8d2219647 -- assignee: georg.brandl components: Documentation messages: 98034 nosy: georg.brandl, jab, rhettinger severity: normal status: open title: discuss mark-as-invalid trick in heapq docs versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue7734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6763] Crash on mac os x leopard in mimetypes.guess_type (or PyObject_Malloc)
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue6763> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6626] show Python mimetypes module some love
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue6626> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6614] heapq.nsmallest and nlargest should be smarter/more usable/more consistent
Joshua Bronson added the comment: > That is in the pure python version of nsmallest() and that > code is not used (it is overriden by the C version). So just because it isn't used by CPython it should remain in there even though as you said yourself it's completely without basis? What about non CPython? I'm not trying to "kick the ball around" here, we're both on the same team just trying to make Python better. -- ___ Python tracker <http://bugs.python.org/issue6614> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6614] heapq.nsmallest and nlargest should be smarter/more usable/more consistent
Joshua Bronson added the comment: One more thing: > I prefer the docs the way they are. They help the reader understand > the relationship between min, max, nsmallest, nlargest, and sorted. The docs still use the unspecific language "for smaller values of n" and "for larger values". I think careful readers would appreciate an addition along the lines of what you wrote earlier -- the optimal switchover point depends on the cost of the comparison function, the ordering of the input data, etc. -- ___ Python tracker <http://bugs.python.org/issue6614> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6614] heapq.nsmallest and nlargest should be smarter/more usable/more consistent
Joshua Bronson added the comment: > I prefer the docs the way they are. They help the reader understand > the relationship between min, max, nsmallest, nlargest, and sorted. Except that it's no longer true that "when n==1, it is more efficient to use the builtin min() and max() functions." Shouldn't this be updated to say "when n==1, it is equivalent to using the builtin min() and max() functions"? > I'm not sure where you got the n * 10 <= len(iterable) switch-over > point. It's right in the file you linked to. Search for "n * 10" in http://svn.python.org/view/python/branches/release31-maint/Lib/heapq.py? revision=73579&view=markup. > That is arbitrary. The correct switchover point depends on the > cost of the comparison function, whether the length of the input is > known, whether the input data is partially ordered, memory constraints, > whether a key function is used, and on other factors. So should that be removed, then? > FWIW, I also wrote the logic for random.sample(). The switchover logic > was straight-forward because performance depended on factors that were > fully known (length of input, sample size, memory utilization, and > average number of probes for each strategy). > One other thought: When memory is tight, the programmer needs to be > able to select the heap algorithm in favor of sorted() even for > relatively large values of n. I do not want an automatic switchover > point that takes away a programmer's choice between speed and space > efficiency. Agreed, and thanks for the additional info. -- ___ Python tracker <http://bugs.python.org/issue6614> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6614] heapq.nsmallest and nlargest should be smarter/more usable/more consistent
Joshua Bronson added the comment: Oh, that's great! (I also noticed that the previously inutile line "_heappushpop = heappushpop" is now doing something in the heapq.py you linked to, nice.) It looks like the docs haven't been updated yet though. For instance, http://docs.python.org/3.1/library/heapq.html still says "The latter two functions perform best for smaller values of n. For larger values, it is more efficient to use the sorted() function. Also, when n==1, it is more efficient to use the builtin min() and max() functions." Also, I notice the pure Python implementation of nsmallest still does that check to see if n * 10 <= len(iterable), and if so uses an insort-based algorithm. It looks like this is still undocumented, inconsistent with the C implementation, and asymmetrical to the implementations of nlargest. If this branch is remaining there on purpose, I'd love see its existence mentioned and its theoretical basis explained in the docs, along with any similar branches implemented elsewhere in the code, if they should be. -- ___ Python tracker <http://bugs.python.org/issue6614> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6614] heapq.nsmallest and nlargest should be smarter/more usable/more consistent
New submission from Joshua Bronson : >From http://docs.python.org/library/heapq.html: > The latter two functions (nlargest and nsmallest) perform best for > smaller values of n. For larger values, it is more efficient to use > the sorted() function. Also, when n==1, it is more efficient to use > the builtin min() and max() functions. A quick usability win for the heapq module: Be more specific than "smaller" and "larger", e.g. "when n is O(...(len(iterable)))". >From http://groups.google.com/group/comp.lang.python/msg/9556f56ae25ecf3b: > I find it interesting that the heapq functions tell you in the > documentation that they aren't suitable for use where n==1 or where > n is near the total size of the sequence whereas random.sample() > chooses what it thinks is the best algorithm based on the input. At > the very least I would have thought the heapq functions could check > for n==1 and call min/max when it is. +1. It looks like the pure Python implementation of nsmallest is actually already choosing either an insort algorithm or a minheap algorithm based on whether n * 10 <= len(iterable), but the C implementation of nsmallest unconditionally uses a minheap algorithm. Neither the pure Python nor the C implementation of nlargest chooses its algorithm conditionally. For the sake of consistency and usability, all implementations of nsmallest and nlargest should choose the most efficient algorithm from among min/max, insort, and minheap, and the docs should be updated to describe this behavior. Also, in looking through the heapq.py that came with my Python 2.6.2 distribution, I noticed that line 196 seems to have no reason to be there: _heappushpop = heappushpop This is the only occurrence of "_heappushpop" in the file. I made a patch for heapq.py, but since I'm not a CPython hacker, I thought it might be better to leave the changes up to someone who could do both the pure Python and the C implementations in tandem. I'd be happy to contribute documentation when the time comes if that would help. -- messages: 91134 nosy: jab severity: normal status: open title: heapq.nsmallest and nlargest should be smarter/more usable/more consistent type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker <http://bugs.python.org/issue6614> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4753] Faster opcode dispatch on gcc
Changes by Joshua Bronson : -- nosy: +jab ___ Python tracker <http://bugs.python.org/issue4753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5220] os.makedirs' mode argument has bad default value
Joshua Bronson added the comment: > My suspicion is that people setting explicit file permissions > typically know what they are doing, and that you will find that > your tickets get closed as invalid, explaining to you that this > mode usage is fully intentional. For what it's worth, I asked Mike Bayer (author of Beaker and Mako, the packages in question) what the intention was, and the 0750 seems to have been an error: Apparently HTML::Mason (what Myghty was based on) may have originally used 0750 but has since changed it to 0775 (search for "0775" in http://svn.urth.org/filedetails.php? repname=Mason&path=%2Ftrunk%2Flib%2FHTML%2FMason%2FInterp.pm&rev=0&sc=0) . The 0750 propagated from Myghty into Mako and Beaker, but has no apparent reason to be there. For what else it's worth, I've run this by a few other programmers I consider quite smart and experienced (including Ian Bicking), and none of them realized the meaning of the mode parameter either. (If they had I never would have filed this ticket, as I checked with them first!) I agree that knowledge of relative paths should be assumed, but I'm convinced that the mode behavior is widely misunderstood enough to warrant at least one-line "gotcha" in the docstring. ___ Python tracker <http://bugs.python.org/issue5220> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5220] os.makedirs' mode argument has bad default value
Joshua Bronson added the comment: Ah, I was misunderstanding the behavior of mkdir, thank you for the explanation. My misunderstanding stemmed from recently coming across two widely-used packages which both pass mode=0750 to os.makedirs. I have no idea why they would be doing this (as it effectively throws away part of the umask), unless they too are misunderstanding the mode parameter. My suspicion is that the mode parameter is widely misunderstood to mean the desired permissions of the created directory. I have filed tickets in the packages in which I came across this to make sure they're doing what they intend. Could the __doc__ for os.mkdir and os.makedirs be expanded to make this clearer? ___ Python tracker <http://bugs.python.org/issue5220> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5220] os.makedirs' mode argument has bad default value
New submission from Joshua Bronson : os.makedirs' mode argument defaults to a hardcoded value of 511 (0777 in octal). This forces the caller to either pass in a different hardcoded value (commonly 0750), or to implement a workaround that calculates the expected mode based on the process owner's umask: umask, _ = subprocess.Popen(['sh', '-c', 'umask'], stdout=subprocess.PIPE).communicate() umask = int(umask, 8) mode = 0777 ^ umask os.makedirs(dir, mode) Preferred behavior would be to have the mode default to the value which takes the umask into account rather than the hardcoded value 0777, so that directories would be created with the same permissions as e.g. files created via open(..). N.B. I'm guessing the above workaround won't work on Windows (please excuse my poor Windows knowledge). All the more reason to have os.makedirs calculate the mode properly if none is given. -- messages: 81666 nosy: jab severity: normal status: open title: os.makedirs' mode argument has bad default value versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker <http://bugs.python.org/issue5220> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com