Re: Bulletproof json.dump?

2020-07-09 Thread Adam Funk
On 2020-07-07, Stephen Rosen wrote: > On Mon, Jul 6, 2020 at 6:37 AM Adam Funk wrote: > >> Is there a "bulletproof" version of json.dump somewhere that will >> convert bytes to str, any other iterables to list, etc., so you can >> just get your data into a file & keep working? >> > > Is the data

Re: Bulletproof json.dump?

2020-07-07 Thread J. Pic
Try jsonlight.dumps it'll just work. Le mar. 7 juil. 2020 à 12:53, Adam Funk a écrit : > On 2020-07-06, Adam Funk wrote: > > > On 2020-07-06, Chris Angelico wrote: > >> On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list > >> wrote: > > >>> While I agree entirely with your point, there i

Re: Bulletproof json.dump?

2020-07-07 Thread Stephen Rosen
On Mon, Jul 6, 2020 at 6:37 AM Adam Funk wrote: > Is there a "bulletproof" version of json.dump somewhere that will > convert bytes to str, any other iterables to list, etc., so you can > just get your data into a file & keep working? > Is the data only being read by python programs? If so, cons

Re: Bulletproof json.dump?

2020-07-07 Thread Adam Funk
On 2020-07-06, Adam Funk wrote: > On 2020-07-06, Chris Angelico wrote: >> On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list >> wrote: >>> While I agree entirely with your point, there is however perhaps room >>> for a bit more helpfulness from the json module. There is no sensible >>> r

Re: Bulletproof json.dump?

2020-07-06 Thread J. Pic
You can achieve round-tripping by maintaining a type mapping in code, for a single datatype it would look like: newloads(datetime, newdumps(datetime.now()) If those would rely on __dump__ and __load__ functions in the fashion of pickle then nested data structures would also be easy: @dataclass c

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Chris Angelico wrote: > On Tue, Jul 7, 2020 at 12:01 AM Jon Ribbens via Python-list > wrote: >> I think what you're saying is, if we do: >> >> json1 = json.dumps(foo) >> json2 = json.dumps(json.loads(json1)) >> assert json1 == json2 >> >> the assertion should never fail

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Tue, Jul 7, 2020 at 12:01 AM Jon Ribbens via Python-list wrote: > > On 2020-07-06, Chris Angelico wrote: > > I think that even in non-strict mode, round-tripping should be > > achieved after one iteration. That is to say, anything you can > > JSON-encode will JSON-decode to something that woul

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Chris Angelico wrote: > I think that even in non-strict mode, round-tripping should be > achieved after one iteration. That is to say, anything you can > JSON-encode will JSON-decode to something that would create the same > encoded form. Not sure if there's anything that would viol

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Frank Millman wrote: > On 2020-07-06 3:08 PM, Jon Ribbens via Python-list wrote: >> On 2020-07-06, Frank Millman wrote: >>> On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote: While I agree entirely with your point, there is however perhaps room for a bit more helpf

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Mon, Jul 6, 2020 at 11:39 PM Adam Funk wrote: > > Aha, I think the default=repr option is probably just what I need; > maybe (at least in the testing stages) something like this: > > try: > with open(output_file, 'w') as f: > json.dump(f) > except TypeError: > print('unexpected

Re: Bulletproof json.dump?

2020-07-06 Thread Adam Funk
On 2020-07-06, Chris Angelico wrote: > On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list > wrote: >> >> On 2020-07-06, Chris Angelico wrote: >> > On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: >> >> Is there a "bulletproof" version of json.dump somewhere that will >> >> convert bytes

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Mon, Jul 6, 2020 at 11:31 PM Jon Ribbens via Python-list wrote: > > On 2020-07-06, Chris Angelico wrote: > > On Mon, Jul 6, 2020 at 11:06 PM Jon Ribbens via Python-list > > wrote: > >> The 'json' module already fails to provide round-trip functionality: > >> > >> >>> for data in ({True: 1}

Re: Bulletproof json.dump?

2020-07-06 Thread Adam Funk
On 2020-07-06, Frank Millman wrote: > On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote: >> On 2020-07-06, Chris Angelico wrote: >>> On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: Is there a "bulletproof" version of json.dump somewhere that will convert bytes to str, any other

Re: Bulletproof json.dump?

2020-07-06 Thread Adam Funk
On 2020-07-06, Chris Angelico wrote: > On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: >> >> Hi, >> >> I have a program that does a lot of work with URLs and requests, >> collecting data over about an hour, & then writing the collated data >> to a JSON file. The first time I ran it, the json.dum

Re: Bulletproof json.dump?

2020-07-06 Thread Frank Millman
On 2020-07-06 3:08 PM, Jon Ribbens via Python-list wrote: On 2020-07-06, Frank Millman wrote: On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote: While I agree entirely with your point, there is however perhaps room for a bit more helpfulness from the json module. There is no sensible re

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Chris Angelico wrote: > On Mon, Jul 6, 2020 at 11:06 PM Jon Ribbens via Python-list > wrote: >> The 'json' module already fails to provide round-trip functionality: >> >> >>> for data in ({True: 1}, {1: 2}, (1, 2)): >> ... if json.loads(json.dumps(data)) != data: >>

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, J. Pic wrote: > Well I made a suggestion on python-ideas and a PyPi lib came out of it, but > since you can't patch a lot of internal types it's not so useful. > > Feel free to try it out: > > https://yourlabs.io/oss/jsonlight/ While I applaud your experimentation, that is not suit

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Mon, Jul 6, 2020 at 11:06 PM Jon Ribbens via Python-list wrote: > > On 2020-07-06, Chris Angelico wrote: > > On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list > > wrote: > >> While I agree entirely with your point, there is however perhaps room > >> for a bit more helpfulness from th

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Frank Millman wrote: > On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote: >> While I agree entirely with your point, there is however perhaps room >> for a bit more helpfulness from the json module. There is no sensible >> reason I can think of that it refuses to serialize se

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Chris Angelico wrote: > On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list > wrote: >> While I agree entirely with your point, there is however perhaps room >> for a bit more helpfulness from the json module. There is no sensible >> reason I can think of that it refuses to

Re: Bulletproof json.dump?

2020-07-06 Thread J. Pic
Well I made a suggestion on python-ideas and a PyPi lib came out of it, but since you can't patch a lot of internal types it's not so useful. Feel free to try it out: https://yourlabs.io/oss/jsonlight/ -- https://mail.python.org/mailman/listinfo/python-list

Re: Bulletproof json.dump?

2020-07-06 Thread Frank Millman
On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote: On 2020-07-06, Chris Angelico wrote: On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: Is there a "bulletproof" version of json.dump somewhere that will convert bytes to str, any other iterables to list, etc., so you can just get your da

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Mon, Jul 6, 2020 at 10:11 PM Jon Ribbens via Python-list wrote: > > On 2020-07-06, Chris Angelico wrote: > > On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: > >> Is there a "bulletproof" version of json.dump somewhere that will > >> convert bytes to str, any other iterables to list, etc., so

Re: Bulletproof json.dump?

2020-07-06 Thread Jon Ribbens via Python-list
On 2020-07-06, Chris Angelico wrote: > On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: >> Is there a "bulletproof" version of json.dump somewhere that will >> convert bytes to str, any other iterables to list, etc., so you can >> just get your data into a file & keep working? > > That's the PHP d

Re: Bulletproof json.dump?

2020-07-06 Thread Chris Angelico
On Mon, Jul 6, 2020 at 8:36 PM Adam Funk wrote: > > Hi, > > I have a program that does a lot of work with URLs and requests, > collecting data over about an hour, & then writing the collated data > to a JSON file. The first time I ran it, the json.dump failed because > there was a bytes value ins

Bulletproof json.dump?

2020-07-06 Thread Adam Funk
Hi, I have a program that does a lot of work with URLs and requests, collecting data over about an hour, & then writing the collated data to a JSON file. The first time I ran it, the json.dump failed because there was a bytes value instead of a str, so I had to figure out where that was coming fr