[issue45929] extend json.tool --json-lines to ignore empty rows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The current implementation allows for the final character of the input to be a 
newline. It does not allow double newlines. In the original example 

   echo -e '{"foo":1}\n{"bar":2}\n'

the echo command adds a newline to the output (which already contains the 
trailing newline), so the result ends with two newlines. Use option -n to 
disable adding newline in echo.

I afraid that if we add support of empty lines, soon we will get requests for 
supporting comments, encoding cookies, single-quote strings, non-quoted keys, 
hexadecimal integers and other possible JSON extensions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Łukasz Langa]

Łukasz Langa  added the comment:

I agree with Serhiy that in general the fact that json.tool crashes here is 
useful: similarly to an exception in Python code, it can inform the user that 
some data they feed to json.tool is invalid.

At the same time, I find it a bit obnoxious that the current implementation 
doesn't allow for the *final character* of the input to be a newline (or 
"characters" in case of \r\n... but it should still only ignore *a single 
effective newline*).

I mean, if the user starts spewing newlines in the middle of the file... that 
might easily be an error. If the file ends with 5 empty lines, that might 
easily be an error. But, if the file really is:

{'line': 1, 'data': ...}\n
{'line': 2, 'data': ...}\n
{'line': 3, 'data': ...}\n

I think that should be pragmatically accepted by json.tool, especially since 
many text editors now add newline characters at file ends.

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Alex Waygood]

Change by Alex Waygood :


--
nosy:  -AlexWaygood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Vito De Tullio]

Vito De Tullio  added the comment:

My final goal is to preserve the empty lines - I think I can do some bash 
magic, but maybe something more complex that a simple sed call.

FWIW on https://jsonlines.org/#line-separator-is-n I see "The last character in 
the file may be a line separator, and it will be treated the same as if there 
was no line separator present.".
And on https://github.com/ndjson/ndjson-spec#32-parsing I see "The parser MAY 
silently ignore empty lines, e.g. \n\n. This behavior MUST be documented and 
SHOULD be configurable by the user of the parser.".

While I get this choice can be "on a grey area", I think this is a known 
"dialect" of the jsonl specs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Both JSON Lines (https://jsonlines.org/) and Newline Delimited JSON 
(http://ndjson.org/) formats require that Each Line is a Valid JSON Value.

If you want to ignore empty lines you can filter them out with `sed /^$/d`.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Alex Waygood]

Alex Waygood  added the comment:

I am changing the "version" field to 3.11, as enhancement proposals are 
generally only considered for unreleased versions of Python.

--
nosy: +AlexWaygood, bob.ippolito, ezio.melotti, rhettinger
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Vito De Tullio]

Change by Vito De Tullio :


--
keywords: +patch
pull_requests: +28086
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29858

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45929] extend json.tool --json-lines to ignore empty rows

[Vito De Tullio]

New submission from Vito De Tullio :

It would be useful to let json.tool support empty rows during handling of json 
lines
generally speaking, this tolerance is already present in parsers like srsly and 
jsonlines

actual behavior:

# happy scenario
$ echo -e '{"foo":1}\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}
{
"bar": 2
}
$

# spurious EOL at EOF
$ echo -e '{"foo":1}\n{"bar":2}\n' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}
{
"bar": 2
}
Expecting value: line 2 column 1 (char 1)
$

# two groups of "rows" in jsonl <- my current usecase
$ echo -e '{"foo":1}\n\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}
Expecting value: line 2 column 1 (char 1)
$


my desired outcome is to preserve the EOLs, so to have something like:

# happy scenario
$ echo -e '{"foo":1}\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}
{
"bar": 2
}
$

# spurious EOL at EOF
$ echo -e '{"foo":1}\n{"bar":2}\n' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}
{
"bar": 2
}

$

# two groups of "rows" in jsonl
$ echo -e '{"foo":1}\n\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
"foo": 1
}

{
"bar": 2
}
$

--
components: Library (Lib)
messages: 407289
nosy: ZeD
priority: normal
severity: normal
status: open
title: extend json.tool --json-lines to ignore empty rows
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com