[issue34529] add the option for json.dumps to return newline delimited json

Change by Thibault Molleman : -- nosy: +Thibault Molleman ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34529] add the option for json.dumps to return newline delimited json

ron added the comment: Well... when handling GBs of data - it's preferred to generate the file directly in the required format rather than doing conversions. The new line is a format... protocols don't matter here... I still think the library should allow the user to create this format

[issue34529] add the option for json.dumps to return newline delimited json

Bob Ippolito added the comment: I suggested that each module would likely implement its own functions tailored to that project's IO and error handling requirements. The implementation may differ slightly depending on the protocol. This is consistent with how JSON is typically dealt with

[issue34529] add the option for json.dumps to return newline delimited json

ron added the comment: I'm a bit confused here. On one hand you say it's two lines of code. On other hand you suggest that each service provider will implement it's own functions. What's the harm from adding - small , unbreakable functionality? Your points for small code could have also

[issue34529] add the option for json.dumps to return newline delimited json

Serhiy Storchaka added the comment: This format is known as JSON Lines: http://jsonlines.org/. Its support in the user code is trivial -- just one or two lines of code. Writing: for item in items: json.dump(item, file) or jsondata = ''.join(json.dumps(item) for item in

[issue34529] add the option for json.dumps to return newline delimited json

Bob Ippolito added the comment: I think the best start would be to add a bit of documentation with an example of how you could work with newline delimited json using the existing module as-is. On the encoding side you need to ensure that it's a compact representation without embedded

[issue34529] add the option for json.dumps to return newline delimited json

Raymond Hettinger added the comment: > The main difference between Json and new line delimited json is that new line > contains valid json in each line. It is up to Bob to decide whether this feature request is within the scope of the module. -- assignee: -> bob.ippolito

[issue34529] add the option for json.dumps to return newline delimited json

ron added the comment: Raymond Hettinger answer is incorrect. The main difference between Json and new line delimited json is that new line contains valid json in each line. Meaning you can do to line #47 and what you will have in this line is a valid json. Unlike the regular json where if

[issue34529] add the option for json.dumps to return newline delimited json

Raymond Hettinger added the comment: Would this need be fulfilled by the *separators* option? >>> from json import dumps >>> query = dict(system='primary', action='load') >>> print(dumps(query, separators=(',\n', ': '))) {"system": "primary", "action": "load"} --

[issue34529] add the option for json.dumps to return newline delimited json

Michał Radwański added the comment: So this format is just a series of JSON, delimited by a newline. Instead of changing API, you might consider this piece of code: def ndjsondump(objects): return '\n'.join(json.dumps(obj) for obj in objects) Conversely, first you `str.splitlines`, then

[issue34529] add the option for json.dumps to return newline delimited json

New submission from ron : Many service providers such as Google BigQuery do not accept Json. They accept newline delimited Json. https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations please allow to receive this format directly from the dump. -- messages: