[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Daniël van Noord]

Daniël van Noord  added the comment:

@Bayard Randel, do you want to make this patch into a GitHub PR? If not, I 
could help by doing so and try to get this landed.

--
nosy: +danielnoord

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Antti Laine]

New submission from Antti Laine :

raw_decode on json.JSONDecoder does not handle leading whitespace. According to 
RFC 4627, section 2, whitespace can precede an object. With json.loads leading 
whitespace is handled just fine.

d = json.JSONDecoder()
d.raw_decode(' {}')
ValueError: No JSON object could be decoded

Tested with versions 2.6.7, 2.7.3, 3.1.3 and 3.2.3

--
components: Library (Lib)
messages: 165829
nosy: Antti Alien
priority: normal
severity: normal
status: open
title: JSONDecoder.raw_decode breaks on leading whitespace
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Antti Laine]

Antti Laine  added the comment:

My coworker just submitted a pull request for a possible fix to simplejson on 
github.

https://github.com/simplejson/simplejson/pull/38

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +ezio.melotti
versions: +Python 3.3, Python 3.4 -Python 2.6, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Jesús Cea Avión]

Jesús Cea Avión  added the comment:

Please, post a patch for 2.7 and 3.2/3.3, with a test. Seems quite easy.

If you hurry, this could go in 3.3.0.

--
keywords: +easy
nosy: +jcea
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Antti Laine]

Antti Laine  added the comment:

Suggestion for a patch for 3.3.0. I wasn't quite sure how the testcases were 
supposed to be loaded. Sorry if I made a mess ;)

--
keywords: +patch
Added file: 
http://bugs.python.org/file26434/json-raw-decoder-fix-whitespace.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Jesús Cea Avión]

Jesús Cea Avión  added the comment:

Antti, you are changing the signature of "decode()" and that would be a 
compatibility problem. Can you rewrite the patch to be more "compatible"?

In your test, please, use a bit more complicated object than "{}" :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[R. David Murray]

R. David Murray  added the comment:

IMO this is not a bug, according to the current documentation it is working as 
designed.  raw_decode says it decodes a string that *starts with* a json 
document.

To my understanding, raw_decode is designed to be used when parsing a stream 
containing more than just one json document, including possibly non-json text, 
in which case it makes sense that the calling application would be responsible 
for (and want to) handle the whitespace around any such document.

I recommend closing this as invalid.  If the consensus is otherwise and a 
change is made, I think it would be an enhancement, not a bug fix.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[R. David Murray]

R. David Murray  added the comment:

Ah, I see, you are thinking that "json document" includes the possibility of 
leading whitespace.  That is a reasonable interpretation...just make sure that 
we don't break backward compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Antti Laine]

Antti Laine  added the comment:

> you are changing the signature of "decode()" and that would be a 
> compatibility problem

I was changing the signature of raw_decode(), by adding a(n almost) private 
keyword. I really don't see how that would affect compatibility.

> you are thinking that "json document" includes the possibility of leading 
> whitespace

Yes. While JSON does not have a real standard to follow, I believe that going 
by the RFC is the best thing to do, and the RFC very clearly states, that 
objects can be surrounded with whitespace.

decode() depends on the functionality of raw_decode(), and its own parameter 
_w. I have no idea why _w is a parameter. It is not used anywhere in CPython, 
and, beginning with an underscore, it shouldn't be used from anywhere else. 
Still it offers the users a possibility to break the behaviour of decode(). If 
it's a performance hack, making the variable being initialized only once, then 
I think it's a very poor one.

Now the patch leaves decode() as it is, and checks for whitespace in 
raw_decode(), which leads to checking for whitespace twice when using decode().

I think that a more extensive cleanup would be in order.

--
Added file: 
http://bugs.python.org/file26450/json-raw-decoder-fix-whitespace.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Petri Lehtinen]

Petri Lehtinen added the comment:

I think skipping preceding whitespace in raw_decode() wouldn't hurt, but 
skipping following whitespace would be wrong.

David: Any examples of how this could break backwards compatibility?

--
nosy: +petri.lehtinen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[R. David Murray]

R. David Murray added the comment:

I didn't have anything specific in mind, just making a general comment about 
the care that needs to be taken in crafting the fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Bayard Randel]

Bayard Randel added the comment:

I've provided an updated patch for 3.6. Additionally I've updated the docstring 
for decoder.raw_decode to explain that whitespace at the beginning of the 
document will be ignored. I believe the concerns around the backwards 
incompatible change to the function signature were addressed by aalien's second 
patch.

--
nosy: +Bayard Randel
Added file: http://bugs.python.org/file44613/issue15393_3.6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Robert Collins]

Robert Collins added the comment:

I think the patch should either be rejected, or also handle trailing spaces: if 
we're taking the RFC definition of whitespace not being structural then we 
should also eat trailing space, which will change the  check for extra data in 
decode to just checking end == s.len().

Obviously a test for the trailing case needs to be added as well.

--
nosy: +rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Petri Lehtinen]

Changes by Petri Lehtinen :


--
nosy:  -petri.lehtinen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[R. David Murray]

R. David Murray added the comment:

Robert: as noted, skipping trailing whitespace would be breaking raw_decode's 
contract.  It is designed to parse a json document and return the position of 
the last character of that document.  If there is then another json document in 
the stream, parsing the remainder of the stream would skip that whitespace.  If 
it is some other data, then the whitespace should be preserved for whatever is 
parsing the remainder of the stream (if anything).

In any case that behavior cannot be changed because it would break backward 
compatibility.

I would not object to rejecting the change, but it does make raw_decode 
slightly more awkward to use if it does not skip the leading whitespace.  So I 
guess I'm +0.5 on accepting it as is.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[R. David Murray]

Changes by R. David Murray :


--
versions: +Python 3.5, Python 3.6, Python 3.7 -Python 2.7, Python 3.2, Python 
3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15393] JSONDecoder.raw_decode breaks on leading whitespace

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com