I've implemented the class as a stand-alone module here: https://github.com/swfarnsworth/dynamicdict
It could in theory be made significantly more concise if `defdict_type` were the base for this class instead of `PyDict_Type`. On Tue, Apr 14, 2020 at 1:32 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote: > On Apr 13, 2020, at 18:44, Caleb Donovick <donov...@cs.stanford.edu> > wrote: > > > I have built this data structure countless times. So I am in favor. > > > Maybe you can give a concrete example of what you need it for, then? I > think that would really help the proposal. Especially if your example needs > a per-instance rather than per-class factory function. > > > Why can’t you just subclass dict and override that? > > Because TypeError: multiple bases have instance lay-out conflict is one > of my least favorite errors. > > > But defaultdict, being a subclass or dict, has the same problem in the > same situations, and (although I haven’t checked) I assume the same is true > for the OP’s dynamicdict. > > Perhaps `__missing__` could be a first class part of the getitem of > protocol, instead of a `dict` specific feature. So that > > ``` > r = x[key] > ``` > means: > ``` > try: > r = x.__getitem__(key) > except KeyError as e: # should we also catch IndexError? > try: > missing = x.__missing__ > except AttributeError: > raise e from None > r = missing(key) > ``` > > Obviously this would come at some performance cost for non dict mappings > so I don't know if this would fly. > > > Besides performance, I don’t think it fits with Guido’s conception of the > protocols as being more minimal than the builtin types—e.g., set has not > just a & operator, but also an intersection method that takes 0 or more > arbitrary iterables; the set protocol has no such method, so > collections.abc.Set neither specifies nor provides an intersection method). > It’s a bit muddy of a conception at the edges, but I think this goes over > the line, and maybe have been explicitly thought about and rejected for the > same reason as Set.intersection. > > On the other hand, none of that is an argument or any kind against your > method decorator: > > So instead maybe there could have standard decorator to get the same > behavior? > ``` > def usemissing(getitem): > @wraps(getitem) > def wrapped(self, key): > try: > return getitem(self, key) > except KeyError as e: > try: > missing = self.__missing__ > except AttributeError: > raise e from None > return missing(key) > return wrapped > ``` > > > This seems like a great idea, although maybe it would be easier to use as > a class decorator rather than a method decorator. Either this: > > def usemissing(cls): > missing = cls.__missing__ > getitem = cls.__getitem__ > def __getitem__(self, key): > try: > return getitem(self, key) > except KeyError: > return missing(self, key) > cls.__getitem__ = __getitem__ > return cls > > Or this: > > def usemissing(cls): > getitem = cls.__getitem__ > def __getitem__(self, key): > try: > return getitem(self, key) > except KeyError: > return type(self).__missing__(self, key) > cls.__getitem__ = __getitem__ > return cls > > This also preserves the usual class-based rather than instance-based > lookup for most special methods (including __missing__ on dict subclasses). > > The first one has the advantage of failing at class decoration time rather > than at first missing lookup time if you forget to include a __missing__, > but it has the cost that (unlike a dict subclass) you can’t monkeypatch > __missing__ after construction time. So I think I’d almost always prefer > the first, but the second might be a better fit for the stdlib anyway? > > I think either the method decorator or the class decorator makes sense for > the stdlib. The only question is where to put it. Either fits in nicely > with things like cached_property and total_ordering in functools. I’m not > sure people will think to look for it there, as opposed to in collections > or something else in the Data Types chapter in the docs, but that’s true of > most of functools, and at least once people discover it (from a Python-list > or StackOverflow question or whatever) they’ll learn where it is and be > able to use it easily, just like the rest of that module. > > It’s simple, but something many Python programmers couldn’t write for > themselves, or would get wrong and have a hard time debugging, and it seems > like the most flexible and least obtrusive way to do it. (It does still > need actual motivating examples, though. Historically, the bar seems to be > lower for new decorators in functools than new classes in collections, but > it’s still not no bar…) > > Additionally—I’m a lot less sure if this one belongs in the stdlib like > @usemissing, but if you were going to put this on PyPI as a mappingtools or > collections2 or more-functools or whatever—you could have an > @addmissing(missingfunc) decorator, to handle cases where you want to adapt > some third-party mapping type without modifying the code or subclassing: > > from sometreelib import SortedDict > def missing(self, key): > # ... whatever ... > SortedDict = addmissing(missing)(SortedDict) > > And if the common use cases are the same kinds of trivial functions as > defaultdict, you could also do this: > > @addmissing(lambda self, key: key) > class MyDict… > > Alternatively, it could be implemented as part of one of the ABCs maybe > something like: > ``` > class MissingMapping(Mapping): > # Could also give MissingMapping its own metaclass > # and do the modification of __getitem__ there. > def __init_subclass__(cls, **kwargs): > super().__init_subclass__(**kwargs) > cls.__getitem__ = usemissing(cls.__getitem__) > > @abstractmethod > def __missing__(self, key): pass > ``` > > > Presumably you’d also want to precompose a MutableMissingMapping ABC. Most > user mappings are mutable, and I suspect that’s even more true for those > that need __missing__, given that most uses of defaultdict are things like > building up a multidict without knowing all the keys in advance. > > As for the implementation, I think __init_subclass__ makes more sense than > a metaclass (presumably a subclass or ABCMeta). Since mixins are all about > composing, often with multiple inheritance, it’s hard to add metaclasses > without interfering with user subclasses. (Or even with your own > future—imagine if someone realizes MutableMapping needs its own metaclass, > and MissingMapping already has one; now there’s no way to write > MutableMissingMapping.) Composing ABCs with non-ABC mixins is already more > of a pain than would be ideal, and I think a new submetaclass would make it > worse. > > But at any rate, I’m not sure this is a good idea. None of the other ABCs > hide methods like this. For example, Mapping will give you a __contains__ > if you don’t have one, but if you do write one that does things > differently, yours overrides the default; here, there’d be no way to > override the default __getitem__ to do things differently, because that > would just get wrapped and replaced. That isn’t unreasonable behavior for a > mixin in general, but I think it is confusing for an ABC/mixin hybrid in > collections.abc. > > Also, a protocol or ABC is something you can check for compliance (at > runtime, or statically in mypy, or just in theory even if not in the actual > code); is there ever any point in asking whether an object complies with > MissingMapping? It’s something you can use an object as, but is there any > way you can use a MissingMapping differently from a Mapping? So I think > it’s not an ABC because it’s not a protocol. > > Of course you could just make it a pure mixin that isn’t an ABC (and maybe > isn’t in collections.abc), which also solves all of the problems above > (except the optional one with the metaclass, but you already avoided that). > But at that point, are there any advantages over the method or class > decorator? > > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/N6K2PLYJ7ZWEAN6FZWUGNJH23JBQQM33/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/D2AMDKMQNKL7TQ4HXL6CKDA5U3W5XMNP/ Code of Conduct: http://python.org/psf/codeofconduct/
