[issue24931] _asdict breaks when inheriting from a namedtuple

[Jens Troeger]

Jens Troeger added the comment:

With my update from Python 3.4.3 to Python 3.4.4 (default, Dec 25 2015, 
06:14:41) I started experiencing crashes of my applications and I suspect this 
change is the culprit.

I have a class that inherits from namedtuple, and code calls vars() (i.e. 
retrieve __dict__) to iterate over an instance's attributes. Much like Raymond 
points out in http://bugs.python.org/msg249100

For example with 3.4.3:

>>> from collections import namedtuple
>>> Point = namedtuple('Point', ['x', 'y'])
>>> p = Point(1,2)
>>> p
Point(x=1, y=2)
>>> p.__dict__
OrderedDict([('x', 1), ('y', 2)])
>>> vars(p)
OrderedDict([('x', 1), ('y', 2)])

After the most recent update this breaks with 3.4.4:

>>> from collections import namedtuple
>>> Point = namedtuple('Point', ['x', 'y'])
>>> p = Point(1,2)
>>> p
Point(x=1, y=2)
>>> p.__dict__
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'Point' object has no attribute '__dict__'
>>> vars(p)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: vars() argument must have __dict__ attribute

I am not sure about the fix on my side. Should I use _asdict() instead of 
vars() although I would argue that vars() should remain functional across this 
change.  Calling _asdict() seems messy to me, but it works:

>>> p._asdict()
OrderedDict([('x', 1), ('y', 2)])

Why not keep the __dict__ property in tact?

  @property
  def __dict__(self):
  return self._asdict()

Thanks!

--
nosy: +_savage

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Vedran Čačić]

Vedran Čačić added the comment:

I agree that namedtuples having __dict__ is probably more trouble than benefit. 
But in my view, that's no reason for _asdict to not work correctly. The whole 
point of separate function (even going through the pain of underscore-starting 
public API, since everything else is even bigger pain) is that we sidestep the 
question of vars() and other standard Python hooks, and provide our way of 
extracting a namedtuple's namespace, for ones who need it.

Of course, the fix/workaround is trivial, as Raymond says: just

def _asdict(self):
return collections.OrderedDict(zip(self._fields, self))

That way, it doesn't matter whether self has a `__dict__` or not.

If you think it shouldn't be done, the documentation really needs to be 
changed. Current wording is very misleading, showing that `_asdict` works on a 
class having `__slots__ = ()`, but containing no explanation of that attribute 
being _necessary_ for `_asdict` to work. (It just says "This helps keep memory 
requirements low by preventing the creation of instance dictionaries.")

--
nosy: +Vedran.Čačić

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Nicholas Chammas]

Nicholas Chammas added the comment:

I know. I came across this issue after upgrading to the 3.5.1 release and 
seeing that vars(namedtuple) didn't work anymore.

I looked through the changelog [0] for an explanation of why that might be and 
couldn't find one, so I posted that question on Stack Overflow.

I'm guessing others will go through the same flow after they upgrade to 3.5.1 
and wonder why their vars(namedtuple) code broke, so I posted here asking if we 
should amend the changelog to call this change out.

But I gather from your comment that the changelog cannot be updated after the 
release, so I guess there is nothing to do here. (Sorry about the distraction. 
I'm new to the Python dev community.)

[0] https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-1-final

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Larry Hastings]

Larry Hastings added the comment:

You're a little late; 3.5.1 was released two days ago.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Nicholas Chammas]

Nicholas Chammas added the comment:

Should this change be called out in the 3.5.1 release docs? It makes some code 
that works on 3.5.0 break in 3.5.1.

See: http://stackoverflow.com/q/34166469/877069

--
nosy: +Nicholas Chammas

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Raymond Hettinger]

Raymond Hettinger added the comment:

> it seems odd that it has disappeared.

It disappeared because it was fundamentally broken in Python 3, so it had to be 
removed.  Providing __dict__ broke subclassing and produced odd behaviors.

--
keywords:  -3.5regression
priority: high -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Larry Hastings]

Larry Hastings added the comment:

Why is this marked as a release blocker?  It doesn't strike me as all that 
major.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Matthias Klose]

Matthias Klose added the comment:

now marked as regression, and lowered the priority.

but how should regressions on release branches be marked else?

--
keywords: +3.4regression, 3.5regression -patch
priority: release blocker -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Eric Snow]

Eric Snow added the comment:

Doesn't the fix mean that `vars(MyNamedtuple)` will no longer work?  While I 
personally don't mind (though I prefer that spelling) and appreciate the 
benefit of simpler code, isn't there a backward-compatibility issue here?

I do concede that fixing this bug without a compatibility break is complicated 
and probably not worth it.  However I want to make sure that the point is at 
least discussed briefly.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Roundup Robot]

Roundup Robot added the comment:

New changeset fa3ac31cfa44 by Raymond Hettinger in branch '3.4':
Issue #24931:  Resolve __dict__ conflict in namedtuple subclasses.
https://hg.python.org/cpython/rev/fa3ac31cfa44

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
assignee:  - rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Yury Selivanov]

Changes by Yury Selivanov yseliva...@gmail.com:


--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Samuel Isaacson]

New submission from Samuel Isaacson:

When inheriting from namedtuples, _asdict and __dict__ return empty 
dictionaries:

from collections import namedtuple

class Point(namedtuple('_Point', ['x', 'y'])):
pass

a = Point(3, 4)
print(a._asdict() == {})

gives False; it is True on Python 2.7.6

--
components: Library (Lib)
messages: 249082
nosy: Samuel Isaacson
priority: normal
severity: normal
status: open
title: _asdict breaks when inheriting from a namedtuple
type: behavior
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Samuel Isaacson]

Samuel Isaacson added the comment:

Sorry; it returns True on Python 3.4, False on Python 2.7.6.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
keywords: +patch
Added file: http://bugs.python.org/file40251/nt_fix1.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24931] _asdict breaks when inheriting from a namedtuple

[Raymond Hettinger]

Raymond Hettinger added the comment:

For __dict__, I'm not sure what the right behavior should by for subclasses 
that don't define __slots__.  In Python 3, the __dict__ is returning the dict 
for the subclass.  This might be the correct and most desirable behavior:

 class Point(namedtuple('_Point', ['x', 'y'])):
pass
 a = Point(3, 4)
 a.w = 5
 a.__dict__
{'w': 5}

If we leave the __dict__ behavior as is in Py3, then we still need to get 
_asdict() back to its documented behavior.  For that, we would need to 
disconnect it from __dict__ by restoring the Py2.7 code for _asdict():

def _asdict(self):
'Return a new OrderedDict which maps field names to their values'
return OrderedDict(zip(self._fields, self))

All this needs to be thought-out carefully.  Putting in __dict__ support 
originally looked like a bugfix to get vars() working correctly, but it caused 
problems with pickling which then led to the addition of __getnewargs__.  It 
seems that defining __dict__ leads to problems no matter how you do it.

My inclination is to remove __dict__ and __getewargs__ from the namedtuple 
definition entirely and return to a simpler state of affairs that is easier to 
reason about and less likely to lead to unexpected behaviors like the one in 
this bug report.

One note:  using the Py2.7 namedtuple code in Python3 still doesn't restore the 
old behavior.  Something else in the language appears to have changed (causing 
the subclasses' __dict__ to take precedence over the inherited __dict__ 
property).

--
priority: normal - high
versions: +Python 3.5, Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24931
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com