[issue23640] int.from_bytes() is broken for subclasses

[R. David Murray]

R. David Murray added the comment:

@eltoder: Ah, you are right, I was looking at the python version.  There's an 
open issue about this, #20371, which indicates that the design decision was to 
support subclasses.  Clearly I was remembering the conversation backward based 
on looking at the wrong code :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Discussion on Python-Dev: 
http://comments.gmane.org/gmane.comp.python.devel/157649 .

The conclusion is that alternate constructors should return an instance of the 
subclass, and either call constructor or use pickling machinery for creating an 
object in correct state.

Pushed the fix for int.from_bytes() and tests. I'll open separate issues for 
other alternate constructors (for example issue27006 -- for 
Decimal.from_float()).

Yes, there is similar bug in date.replace(). C implementation doesn't match 
Python implementation and bypasses the constructor. Please open separate issue 
for this Eugene.

--
assignee:  -> serhiy.storchaka
components: +Interpreter Core -Library (Lib)
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0af15b8ef3b2 by Serhiy Storchaka in branch '3.5':
Issue #23640: int.from_bytes() no longer bypasses constructors for subclasses.
https://hg.python.org/cpython/rev/0af15b8ef3b2

New changeset df14ea17a517 by Serhiy Storchaka in branch 'default':
Issue #23640: int.from_bytes() no longer bypasses constructors for subclasses.
https://hg.python.org/cpython/rev/df14ea17a517

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Eugene Toder]

Eugene Toder added the comment:

They sure do. Check the example in my comment, or if you prefer the source 
code, it's pretty clear as well: 
https://hg.python.org/cpython/file/fff0c783d3db/Modules/_datetimemodule.c#l2770

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[R. David Murray]

R. David Murray added the comment:

No, they do not return instances of the derived type, they return instances of 
the base type.  This is an intentional design decision that has been discussed 
elsewhere (including, I think, somewhere in this tracker).

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Eugene Toder]

Eugene Toder added the comment:

There's a similar issue with replace() methods on date/time/datetime classes. 
They create instances of derived types without calling derived 
__new__/__init__, thus potentially leaving those uninitialized.

>>> from datetime import date
>>> class D(date):
...   def __init__(self, y, m, d):
... self.y = y
>>> D(2016,1,1).y
2016
>>> D(2016,1,1).replace(2015).y
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'D' object has no attribute 'y'

--
nosy: +eltoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Ethan Furman added the comment:

With the patch:

  --> import enum
  --> class Huh(enum.IntEnum):
  ...   blah = 2
  ...
  --> Huh.blah.from_bytes(b'\04', 'big')
  Traceback (most recent call last):
File "", line 1, in 
File "/home/ethan/source/python/issue23640/Lib/enum.py", line 222, in 
__call__
  return cls.__new__(cls, value)
File "/home/ethan/source/python/issue23640/Lib/enum.py", line 457, in 
__new__
  raise ValueError("%r is not a valid %s" % (value, cls.__name__))
  ValueError: 4 is not a valid Huh

This is not the correct behavior.  An IntEnum should act like an int, and in 
cases where it can't and still be an IntEnum, it becomes an int.  But this 
behavior is Enum specific, and I would not expect other int subclasses to need 
or want that behavior.

Also, in cases where class methods are alternate constructors there is no 
requirement that they go through the main __new__/__init__ constructors to do 
their job.

In other words, if IntEnum.from_bytes (which is inherited) is not behaving 
correctly, it is up to IntEnum to fix it -- it is not the job of int, and this 
is not a bug in int.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Ethan Furman added the comment:

I think the classmethod-as-constructor behavior is correct, so it's up to 
IntEnum (or EnumMeta, or foo, or ...), to work around the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Ethan Furman added the comment:

'from_bytes' is a classmethod.  As such, it should return the same type as the 
class it is called on.  If that wasn't the intent it would be a staticmethod 
instead.

It is the responsibility of the subclass to override base class behavior, not 
the other way around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Changes by Ethan Furman :


--
Removed message: http://bugs.python.org/msg237891

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Changes by Ethan Furman :


--
Removed message: http://bugs.python.org/msg240053

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Changes by Ethan Furman :


--
Removed message: http://bugs.python.org/msg239619

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Ethan Furman added the comment:

Not sure what I was thinking at the time, but several of my comments were 
supportive of `classmethod`s not calling subclass' __new__; I actually do not 
think that, and am +1 on the patch.

--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[ashutosh]

ashutosh added the comment:

hi

i am ashutosh. I am running kubuntu 14.04.
before applying the patch. It failed in 2 test i.e

test___all__ 
test_re

After applying patch. It again gave me 2 error
test___all__ 
test_re

So,it worked for me.

--
nosy: +singh_jug

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Yogesh Joshi]

Yogesh Joshi added the comment:

I applied patch to module and ran all the tested on Mac Os El Captain, only 4 
tests were failing with or without applying patch, following are the tests 
failing:
test___all__ test_calendar test_re test_socket

--
nosy: +iyogeshjoshi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[STINNER Victor]

STINNER Victor added the comment:

0002-int.from_bytes-calls-constructor-for-subclasses.patch looks good to me, 
but see my review on Rietveld for 2 minor comments.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Stefan Krah]

Stefan Krah added the comment:

> There are similar issues with Decimal.from_float() (C implementation only), 
> chain.from_iterable(), epoll.fromfd() and kqueue.fromfd(). All these 
> alternative constructors don't call __new__ or __init__.

Could you create new issues? I need a summary. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are similar issues with Decimal.from_float() (C implementation only), 
chain.from_iterable(), epoll.fromfd() and kqueue.fromfd(). All these 
alternative constructors don't call __new__ or __init__.

But float.fromhex() calls the constructor.

>>> import enum
>>> class M(float, enum.Enum):
... PI = 3.14
... 
>>> M.PI

>>> M.fromhex((3.14).hex())

>>> M.fromhex((2.72).hex())
Traceback (most recent call last):
File "", line 1, in 
File "/home/serhiy/py/cpython/Lib/enum.py", line 241, in __call__
return cls.__new__(cls, value)
File "/home/serhiy/py/cpython/Lib/enum.py", line 476, in __new__
raise ValueError("%r is not a valid %s" % (value, cls.__name__))
ValueError: 2.72 is not a valid M

And this behavior looks correct to me.

--
nosy: +facundobatista, mark.dickinson, rhettinger, skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is simplified Bruno's patch with additional tests for float, int, bool and 
enum. But I'm not sure that correct solution is so simple.

--
Added file: 
http://bugs.python.org/file40606/0002-int.from_bytes-calls-constructor-for-subclasses.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23640] int.from_bytes() is broken for subclasses

[Ethan Furman]

Changes by Ethan Furman et...@stoneleaf.us:


--
assignee: ethan.furman - 
nosy:  -ethan.furman
title: Enum.from_bytes() is broken - int.from_bytes() is broken for subclasses

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23640
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com