[issue27934] json float encoding incorrect for dbus.Double

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Agreed. It is too late.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson  added the comment:

[Serhiy]

> I have doubts about breaking it in a bugfix release of 2.7.

Yes, possibly we shouldn't have changed this (though it's a fairly horrible 
trick, IMO). I don't think it would be a good idea to revert the change at this 
point, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This change broke a trick used for serializing Decimal to JSON without loss 
(see msg176158). It was good to break it in a new release, but I have doubts 
about breaking it in a bugfix release of 2.7.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Changes by Mark Dickinson :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Roundup Robot]

Roundup Robot added the comment:

New changeset 86d66a627b77 by Mark Dickinson in branch '2.7':
Issue #27934: Use float.__repr__ instead of plain repr when JSON-encoding an 
instance of a float subclass. Thanks Eddie James.
https://hg.python.org/cpython/rev/86d66a627b77

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

Note that this will also change the results for JSON output of NumPy float64 
values, so at the very least I'd expect this change to break (admittedly poorly 
written) unit tests / doctests.

Python 2.7.12 (default, Jul 10 2016, 18:28:23) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import json
>>> x = np.float64(3.3)
>>> json.dumps(x)
'3.2998'
>>> repr(x)
'3.2998'
>>> float.__repr__(x)
'3.3'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

Yes. I'm fairly convinced about the bug part; it's the gratuitous breakage part 
that worries me. It wouldn't surprise me at all to find that there's someone 
out there who wants all their numbers to be written out to JSON with exactly 6 
places after the point, and subclasses float just for that purpose. (I'm not 
for a moment suggesting that this is a good idea, but there's a big difference 
between the set of Python code that *should* have been written and the set of 
Python code that *has* been written. :-)

>>> class MyFloat(float):
... def __repr__(self):
... return '{:.6f}'.format(self)
... 
>>> import math, json
>>> json.dumps(MyFloat(math.pi))
'3.141593'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Changes by Eddie James :


Added file: http://bugs.python.org/file44349/json-float-repr-2.7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Eddie James added the comment:

Python 2.7 also already behaves correctly for other dbus types:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>> import json
>>> x = dbus.Int32(3)
>>> x
dbus.Int32(3)
>>> json.dumps(x)
'3'
>>> repr(x)
'dbus.Int32(3)'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

> That's still using PyObject_Repr() which will call tp_repr for dbus.Double... 
> Any suggestions?

Right, you'll want to replace that with a call to `PyFloat_Type.tp_repr(obj)`, 
as in the Python 3 code.

We're in a bit of a grey area here: making this change to the 2.7 branch does 
require making a case that it's a bugfix rather than a new feature (which isn't 
permitted in 2.7), *and* that it's not going to cause gratuitous breakage in 
existing json-using applications. I think there *is* a reasonable case there, 
but others may disagree. One point in its favour is that we're *already* 
behaving like a regular float for special values:

Python 2.7.12 (default, Jun 29 2016, 12:46:54) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus, json
>>> x = dbus.Double(float('inf'))
>>> json.dumps(x)  # not using repr(x) here
'Infinity'
>>> repr(x)
'dbus.Double(inf)'
>>> x = dbus.Double(2.3)
>>> json.dumps(x)  # using repr(x) here
'dbus.Double(2.3)'
>>> repr(x)
'dbus.Double(2.3)'

--
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[R. David Murray]

Changes by R. David Murray :


--
resolution: third party -> 
stage: resolved -> needs patch
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Eddie James added the comment:

Wait what about the json C code for 2.7? That's still using PyObject_Repr() 
which will call tp_repr for dbus.Double... Any suggestions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Eddie James added the comment:

Thanks Mark, yes you installed the right package.

OK I didn't dig deep enough in the sub class. And yea, there shouldn't be any 
difference between float.__repr__ and float.__str__. Obviously repr calls the 
object's tp_repr method, while float.__repr__ calls the base float method, 
which is the one we want.

I didn't do any testing for python 3.x as I couldn't figure out how to import 
dbus into a local build of 3.x. My mistake here.

> For 2.7, we may want to consider processing float instances using 
> `float.__repr__` instead of plain old `repr`

That should work! I'll upload a new patch for 2.7

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

> Nothing that can/should be done on the stdlib side, then.

I think there's nothing to do for 3.x: as far as I can tell, everything should 
be working exactly as desired there.

For 2.7, we may want to consider processing float instances using 
`float.__repr__` instead of plain old `repr`. I believe that would fix the OP's 
problem, and also bring the 2.7 behaviour more in line with the 3.x behaviour. 
It's a backwards incompatible behaviour change, but probably not a particularly 
disruptive one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

> Dbus.Double is not a subclass of float unfortunately.

Okay, now I'm *really* confused. :-)

If it's not a subclass of `float`, then how do you end up in the `floatstr` 
code? Every path to that code that I can see in the source is via an 
`isinstance(, float`) check.

I don't know dbus at all, but I just tried installing it under Macports (on OS 
X 10.9). Here's the package description, so you can tell me whether I'm 
actually installing the right thing, or something totally unrelated:

dbus-python35 @1.2.0_2 (devel, python)
Python bindings for the dbus message bus system.

Once I've done that, I get the following behaviour in Python:

Python 3.5.2 (default, Aug 16 2016, 08:43:53) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>> dbus.Double

>>> dbus.Double.__mro__
(, , , 
)

So it looks as though at least for this version of dbus, we do have a subclass 
of `float`. Looking at an instance, I see the following:

>>> x = dbus.Double(4.3)
>>> isinstance(x, float)
True
>>> repr(x)
'dbus.Double(4.3)'
>>> str(x)
'4.3'
>>> float.__repr__(x)
'4.3'
>>> float.__str__(x)
'4.3'
>>> import json
>>> json.dumps(x)
'4.3'

So I'm still struggling to see what difference swapping out float.__repr__ for 
float.__str__ would make.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[R. David Murray]

R. David Murray added the comment:

Nothing that can/should be done on the stdlib side, then.

--
nosy: +r.david.murray
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Eddie James added the comment:

Understood on 2.7, I wasn't aware it would cause any issues.

Dbus.Double is not a subclass of float unfortunately. Problem is that all Dbus 
types seem to have a custom tp_repr method that returns that strange formatting 
I mentioned. So repr won't be the same as str, in any version of python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eric V. Smith]

Changes by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Mark Dickinson]

Mark Dickinson added the comment:

> I propose a change to use str() for float objects as well. This could be 
> ported back to 2.7 as well

That would be a behaviour change for 2.7, and an undesirable one. `str` loses 
precision in Python 2, so e.g., `json.loads(json.dumps(pi)) == pi` would no 
longer be true. (Indeed, the json library was changed to use `repr` rather than 
`str` for floats at some point in the past, for exactly this reason.)

I assume that dbus.Double is a subclass of float. Is that correct? If that's 
the case, I'm a bit confused: `float.__repr__` behaves identically to 
`float.__str__` in Python versions >= 3.2, so I don't see what your suggested 
change would achieve.

>>> class MyFloat(float):
... def __repr__(self): return "MyFloat(something_or_other)"
... def __str__(self): return "1729.0"
... 
>>> x = MyFloat(2.3)
>>> import json
>>> json.dumps(x)
'2.3'

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

Changes by Eddie James :


Added file: http://bugs.python.org/file44334/json-float-str-2.7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27934] json float encoding incorrect for dbus.Double

[Eddie James]

New submission from Eddie James:

JSON does not correctly encode dbus.Double types, even though all other dbus 
types are handled fine. I end up with output like this (0.25 is the floating 
point value): dbus.Double(0.25, variant_level=1)

Found that the encoding uses repr() for float objects but uses str() for 
integer objects. I propose a change to use str() for float objects as well. 
This could be ported back to 2.7 as well

--
components: Library (Lib)
files: json-float-str-default.patch
keywords: patch
messages: 274179
nosy: eajames
priority: normal
severity: normal
status: open
title: json float encoding incorrect for dbus.Double
type: behavior
versions: Python 2.7, Python 3.6
Added file: http://bugs.python.org/file44333/json-float-str-default.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com