Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Gregory Ewing]

Erik wrote:
Is there not a class that is somewhere between "dict" and "OrderedDict" 
that provides what I need?


Such a class could exist, but the stdlib doesn't happen to provide
one as far as I know.

Note, though, that you're relying on implementation details of
OrderedDict when you use it to "fix" your problem in this way.
It happens to work in some versions of Python, but it might not
work in the next one.

> OK, so technically it is a subclass, I didn't notice that, but where
> does the documentation explain all of the differences in behaviour?

It doesn't, but the differences you're talking about are differences
between undocumented behavour of dict and undocumented behaviour
of OrderedDict. The docs are under no obligation to tell you about any
of that.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[breamoreboy]

On Sunday, April 30, 2017 at 2:30:25 AM UTC+1, Erik wrote:
> On 30/04/17 01:17, breamoreboy wrote:
> > On Sunday, April 30, 2017 at 12:23:19 AM UTC+1, Erik wrote:
> >> The other is that the documentation of collections.OrderedDict seems to
> >> be lacking (it is talking in terms of being a "dict" subclass, but it
> >> actually isn't one).
> >>
> >> E.
> >
> > Could have fooled me.
> >
> > C:\python
> > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit 
> > (AMD64)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
>  from collections import OrderedDict
>  o = OrderedDict()
>  isinstance(o, dict)
> > True
> 
> OK, so technically it is a subclass, I didn't notice that, but where 
> does the documentation explain all of the differences in behaviour? It 
> doesn't. It states "it's a subclass of dict that does  and " 
> (by which I mean "order the keys") and it's not unreasonable to 
> therefore believe that's _all_ it does differently. If if it actually 
> does ", , , , " (by which I mean 
> "order the keys and implement very different behavior for other methods 
> such as __init__ and __update__") then the documentation is lacking 
> (because it doesn't mention it). Isn't that what I said?
> 
> E.

So provide a documentation patch on the bug tracker.

Once you've done that how about subclassing UserDict 
https://docs.python.org/3.5/library/collections.html#collections.UserDict, 
pinching any code from OrderedDict as needed?

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Erik]

On 30/04/17 01:17, breamore...@gmail.com wrote:

On Sunday, April 30, 2017 at 12:23:19 AM UTC+1, Erik wrote:

The other is that the documentation of collections.OrderedDict seems to
be lacking (it is talking in terms of being a "dict" subclass, but it
actually isn't one).

E.


Could have fooled me.

C:\python
Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from collections import OrderedDict
o = OrderedDict()
isinstance(o, dict)

True


OK, so technically it is a subclass, I didn't notice that, but where 
does the documentation explain all of the differences in behaviour? It 
doesn't. It states "it's a subclass of dict that does  and " 
(by which I mean "order the keys") and it's not unreasonable to 
therefore believe that's _all_ it does differently. If if it actually 
does ", , , , " (by which I mean 
"order the keys and implement very different behavior for other methods 
such as __init__ and __update__") then the documentation is lacking 
(because it doesn't mention it). Isn't that what I said?


E.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Erik]

On 30/04/17 01:31, Ben Finney wrote:

Erik  writes:


On 29/04/17 23:40, Ned Batchelder wrote:

For creating your own class that acts like a dict, you should derive
from collections.abc.MutableMapping, which only requires
implementing __getitem__, __setitem__, __delitem__, __iter__, and
__len__.


Or, I could derive from collections.OrderedDict and just implement the
two methods that I actually want to change the behavior of (did you
read the rest of my post?) ;)


Did you read Ned's? :-)


Yes, I did. Which part of Ned's do you think I did not read properly?


You say that there are only two methods you want to change the behaviour
of;


Yes, __init__ (so that I can store the valid keys) and __setitem__ (so 
that I can check the key being set).



but as you have found, those two methods are not the only ones you
need to implement, in order to get the changed behaviour.


I didn't find that. But I found that I could change exactly those two 
methods if I just derived from a different class that also provides 
another behavior that I don't need (ordered keys).



you need to implement the MutableMapping
protocol if you want a custom class


Why do I "need" to do that when I could subclass from OrderedDict instead?

If I'm not being clear:

Is there not a class that is somewhere between "dict" and "OrderedDict" 
that provides what I need? collections.abc.MutableMapping is not that 
class, as it requires me to write all sorts of methods that are 
irrelevant to the behavior changes I want to make (__len__ etc).


E.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[breamoreboy]

On Sunday, April 30, 2017 at 12:23:19 AM UTC+1, Erik wrote:
> On 29/04/17 23:40, Ned Batchelder wrote:
> > For creating your own class that acts like
> > a dict, you should derive from collections.abc.MutableMapping, which
> > only requires implementing __getitem__, __setitem__, __delitem__,
> > __iter__, and __len__.
> 
> Or, I could derive from collections.OrderedDict and just implement the 
> two methods that I actually want to change the behavior of (did you read 
> the rest of my post?) ;)
> 
> That's one of the points I'm trying to make - why is it harder than it 
> needs to be to do something this simple?
> 
> The other is that the documentation of collections.OrderedDict seems to 
> be lacking (it is talking in terms of being a "dict" subclass, but it 
> actually isn't one).
> 
> E.

Could have fooled me.

C:\python  
Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import OrderedDict
>>> o = OrderedDict()
>>> isinstance(o, dict)
True

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Ben Finney]

Erik  writes:

> On 29/04/17 23:40, Ned Batchelder wrote:
> > For creating your own class that acts like a dict, you should derive
> > from collections.abc.MutableMapping, which only requires
> > implementing __getitem__, __setitem__, __delitem__, __iter__, and
> > __len__.
>
> Or, I could derive from collections.OrderedDict and just implement the
> two methods that I actually want to change the behavior of (did you
> read the rest of my post?) ;)

Did you read Ned's? :-)

You say that there are only two methods you want to change the behaviour
of; but as you have found, those two methods are not the only ones you
need to implement, in order to get the changed behaviour.

> That's one of the points I'm trying to make - why is it harder than it
> needs to be to do something this simple?

The “needs” have already been expressed, by yourself and Ned: the type
needs to provide fast performance. One cost of that, with the current
implementation, is that you need to implement the MutableMapping
protocol if you want a custom class.

-- 
 \“Perchance you who pronounce my sentence are in greater fear |
  `\   than I who receive it.” —Giordano Bruno, burned at the stake by |
_o__)  the Catholic church for the heresy of heliocentrism, 1600-02-16 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Gregory Ewing]

Erik wrote:
That's one of the points I'm trying to make - why is it harder than it 
needs to be to do something this simple?


The built-in dict class is used internally to implement
various namespaces (module, class, instance, etc.), so it
needs to be extremely efficient. Funnelling all updates
through a single method would be unacceptable.

This does make it harder to override behaviour in a
subclass, but that's always a tricky proposition that
relies on knowing a lot about implementation details of
the class. If a particular method isn't documented as
being a hook intended for overriding, you can't make
any assumptions.

The other is that the documentation of collections.OrderedDict seems to 
be lacking (it is talking in terms of being a "dict" subclass, but it 
actually isn't one).


The docs probably mean to say that it has the same interface
as a dict, not that it's necessarily implemented as a
subclass of dict. Nevertheless, it could actually be a
dict subclass that overrides all the necessary methods.
Either way, its behaviour under subclassing is not spelled
out.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Erik]

On 29/04/17 23:40, Ned Batchelder wrote:

For creating your own class that acts like
a dict, you should derive from collections.abc.MutableMapping, which
only requires implementing __getitem__, __setitem__, __delitem__,
__iter__, and __len__.


Or, I could derive from collections.OrderedDict and just implement the 
two methods that I actually want to change the behavior of (did you read 
the rest of my post?) ;)


That's one of the points I'm trying to make - why is it harder than it 
needs to be to do something this simple?


The other is that the documentation of collections.OrderedDict seems to 
be lacking (it is talking in terms of being a "dict" subclass, but it 
actually isn't one).


E.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Inconsistency between dict() and collections.OrderedDict() methods.

[Ned Batchelder]

On Saturday, April 29, 2017 at 4:20:06 PM UTC-4, Erik wrote: 
> It seems a little onerous that I have to put the key checks in several 
> places and implement each of those APIs manually again (and keep on top 
> of that if dict() grows some new methods that involve setting items). Is 
> there a compelling reason why the dict module doesn't call a custom 
> __setitem__ each time an item needs to be set? Performance is 
> undoubtably a concern with something a fundamental as dicts...

Performance is the reason.  For creating your own class that acts like
a dict, you should derive from collections.abc.MutableMapping, which
only requires implementing __getitem__, __setitem__, __delitem__,
__iter__, and __len__.

--Ned.
-- 
https://mail.python.org/mailman/listinfo/python-list

Inconsistency between dict() and collections.OrderedDict() methods.

[Erik]

I have a subclass of dict that enforces which keys are allowed to be set 
and only allows each key to be set at most once:


class StrictDict(dict):
  def __init__(self, validkeys, *args, **kwargs):
self.validkeys = validkeys
super(StrictDict, self).__init__(*args, **kwargs)

  def __setitem__(self, key, value):
if key not in self.validkeys:
  raise KeyError("'%s' is not a valid key" % key)

if key in self.keys():
  raise KeyError("'%s' is already set" % key)

super(StrictDict, self).__setitem__(key, value)

This works fine in the general case:

/tmp$ ~/sw/Python-3.6.1/python
Python 3.6.1 (heads/master:7bcbcb9, Apr 21 2017, 01:52:28)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from strictdict import *
>>> d = StrictDict(('foo', 'bar'))
>>> d['foo'] = 1
>>> d['foo'] = 1
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/strictdict.py", line 11, in __setitem__
raise KeyError("'%s' is already set" % key)
KeyError: "'foo' is already set"
>>> d['baz'] = 1
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/strictdict.py", line 8, in __setitem__
raise KeyError("'%s' is not a valid key" % key)
KeyError: "'baz' is not a valid key"
>>> d
{'foo': 1}

However, when I call other methods which set items in the dictionary, I 
do not get errors for invalid or duplicate keys (__setitem__ is not called):


/tmp$ ~/sw/Python-3.6.1/python
Python 3.6.1 (heads/master:7bcbcb9, Apr 21 2017, 01:52:28)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from strictdict import *
>>> d = StrictDict(('foo', 'bar'), baz=1)
>>> d
{'baz': 1}
>>> d.update({'spam': 1})
>>> d
{'baz': 1, 'spam': 1}

It seems a little onerous that I have to put the key checks in several 
places and implement each of those APIs manually again (and keep on top 
of that if dict() grows some new methods that involve setting items). Is 
there a compelling reason why the dict module doesn't call a custom 
__setitem__ each time an item needs to be set? Performance is 
undoubtably a concern with something a fundamental as dicts, but what 
I'm suggesting could be done fairly efficiently (with a single "is the 
self object a subclass which overrides __setitem__" test followed by 
running the existing code or a slower version which calls __setitem__).


Anyway, related to but separate from that, as the subject of this 
message states it turns out that collections.OrderedDict() *does* do 
what I would expect:


from collections import OrderedDict
class OrderedStrictDict(OrderedDict):
  def __init__(self, validkeys, *args, **kwargs):
self.validkeys = validkeys
super(OrderedStrictDict, self).__init__(*args, **kwargs)

  def __setitem__(self, key, value):
if key not in self.validkeys:
  raise KeyError("'%s' is not a valid key" % key)

if key in self.keys():
  raise KeyError("'%s' is already set" % key)

super(OrderedStrictDict, self).__setitem__(key, value)

/tmp$ ~/sw/Python-3.6.1/python
Python 3.6.1 (heads/master:7bcbcb9, Apr 21 2017, 01:52:28)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from strictdict import *
>>> d = OrderedStrictDict(('foo', 'bar'), baz=1)
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/strictdict.py", line 19, in __init__
super(OrderedStrictDict, self).__init__(*args, **kwargs)
  File "/tmp/strictdict.py", line 23, in __setitem__
raise KeyError("'%s' is not a valid key" % key)
KeyError: "'baz' is not a valid key"
>>> d = OrderedStrictDict(('foo', 'bar'))
>>> d.update({'baz': 1})
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/strictdict.py", line 23, in __setitem__
raise KeyError("'%s' is not a valid key" % key)
KeyError: "'baz' is not a valid key"


The documentation for OrderedDict says: "Return an instance of a dict 
subclass, supporting the usual dict methods. An OrderedDict is a dict 
that remembers the order that keys were first inserted. If a new entry 
overwrites an existing entry, the original insertion position is left 
unchanged. Deleting an entry and reinserting it will move it to the end."


The strong implication there is that the class behaves exactly like a 
dict apart from the ordering, but that's not true.


E.
--
https://mail.python.org/mailman/listinfo/python-list