[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Terry J. Reedy]

Terry J. Reedy added the comment:

I am surprised that dumping to a string and to a file give different answers.  
This strikes me as a bug in any case.  I would expect the the main difference 
would be file.write(chunk) versus temlist.append(chunk).

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[R. David Murray]

R. David Murray added the comment:

If you break the invariants (in this case: a list has an accurate len) code 
that expects lists is only going to work by accident.

What you really want to do is define your own json encoder for your type.  If 
that isn't possible for a streamed sequence of undefined length, then enhancing 
json's extension machinery to allow it would be a good feature request.

That said, could json's ability to handle this be improved?  Possibly.  I think 
we would accept a patch if it doesn't make the code more complicated.  Ideally 
handling certain cases as "don't care" makes the code simpler, but that may or 
may not be the case here.  Or it may come out as a consequence of the 
enhancement.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

Grigory Statsenko added the comment:

With streaming you never know the real length before you're done iterating.

Anyway, the fix really shouldn't be that complicated:
In _iterencode_list just yield  the '[' instead of saving it to buf

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

Grigory Statsenko added the comment:

If __len__ is not defined, then the iterator is considered empty and is always 
rendered as [] even if it really isn't empty

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[SilentGhost]

SilentGhost added the comment:

The question is why are you defining __len__ if you don't know the size of your 
final object? Or at least, why are you starting with a potentially wrong 
initial value? This issue doesn't exist if you either don't define the method 
or return correct value.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

Grigory Statsenko added the comment:

My bad - it doesn't work with non-empty iterators if you set len to 0, so not a 
solution

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

Grigory Statsenko added the comment:

Actually, it does work with len = 0
even if the iterator is not empty. So, I guess that is a solution.
But still, I think the more correct way would be to make it work with > 0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

Grigory Statsenko added the comment:

I can't do that if I don't know how many entries there will be ahead of time. 
In my real-life situation I'm fetching the data from a database not knowing how 
many entries I'll get before I actually get them (in the iterator). In most 
cases there are huge amounts of entries that take up too much memory - that's 
why I need to stream it. But sometimes the result set is empty - and that's 
when everything fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[SilentGhost]

SilentGhost added the comment:

Why does your __len__ method returns 1? Shouldn't it be 0 since this is an 
empty iterator? Changing it to zero seems to fix the "issue" too.

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27613] Empty iterator is rendered as a single bracket ] when using json's iterencode

[Grigory Statsenko]

New submission from Grigory Statsenko:

JSONEncoder.iterencode doesn't work with empty iterators correctly.
Steps:
1. Define an iterator that is recognized by json as a list (inherit from list 
and define nonzero __len__).
2. Use json.dump with data containing an empty iterator defined as described in 
step#1 (but doesn't generate any items)

Expected result: it should be rendered as an empty list: '[]'

Actual result: it is rendered as ']' (only the closing bracket)
interestingly enough this behavior is not reproduced when using the dumps 
function.
I tried other alternatives to the standard json module: simplejson, ujson, hjson
All of them work as expected in this case (both brackets are rendered).

Here is an example of the code that demonstrates this error (compares the 
results of the dump and dumps functions):


import json as json
import io

class EmptyIterator(list):
def __iter__(self):
while False:
yield 1
def __len__(self):
return 1

def dump_to_str(data):
return json.dumps(data)

def dump_to_file(data):
stream = io.StringIO()
json.dump(data, stream)
return stream.getvalue()


data = {'it': EmptyIterator()}
print('to str: {0}'.format(dump_to_str(data)))
print('to file: {0}'.format(dump_to_file(data)))



This prints:
to str: {"it": []}
to file: {"it": ]}

--
messages: 271249
nosy: altvod
priority: normal
severity: normal
status: open
title: Empty iterator is rendered as a single bracket ] when using json's 
iterencode
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com