Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Peter Uhnák]

>
> The class comment clearly states:
>

The docs also say "When using JSON mode the reference policy should be set
to #error or #ignore for full JSON compatibility." But demonstrably setting
it to #error does not give me full JSON compatibility.


If I feed it valid input, then I expect valid output. Now it throws up.
This is a bug.

If I feed it infinite input, then I cannot be surprised it will start
producing infinite output. Having a cycle detection is a nice feature, but
I cannot sacrifice functionality for it.

In the end this is your call, maybe it makes sense for STON's json to be
closer to STON's behavior, even if it sometimes breaks down. And I can
always switch to NeoJSON if I want to be sure.

Peter



> ...
> - referencePolicy <#normal|#ignore|#error> default is #normal
> if #normal, track and count object references and use references
> to implement sharing and break cycles
> if #error, track object references and signal STONWriterError when
> a shared reference is encountered
> if #ignore, don't track object references which might loop forever
> on cycles
>
> ...
>
> For JSON production (not a main goes of STON), it was set to #error, but
> like I showed, it you could prefer #ignore. You have to know what to feed
> it.
>
> Cycle/shared structure checking is expensive, i.e. slow(er) and uses more
> memory.
>
> I am not sure it is easy to make a difference between a cycle and a shared
> structure, I would have to think about that.
>
> And yes, NeoJSON will go happily in cycles.
>
> > On 27 Mar 2018, at 18:52, Peter Uhnák  wrote:
> >
> > Ah sorry, I'm blind.
> >
> > Yes, ideally it should throw up an error in case of recursive.
> > And I see that NeoJSON runs into infinite cycle, so I guess this should
> be addressed there too.
> >
> >
> >
> > On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík 
> wrote:
> >
> >
> > Peter Uhnák wrote:
> > Hi Sven,
> >
> > shouldn't it be set to #ignore by default for JSON then? Or is there a
> > use case where it makes sense to have something else for JSON?
> >
> > To keep STON practices / assumptions while using JSON to hold the data
> where STON is not supported yet, maybe? IIRC STON is positioning itself as
> JSON superset; but maybe only syntactic superset, while semantically it can
> be a bit different (so STONize to more low-level, JSON, syntax via STON;
> JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just
> making things up).
> >
> > Herbert: I am not talking about (infinitely) recursive references, but
> >
> > I know; Sven explicitly asked what to do with cycles in that case,
> though.
> >
> > referencing the same object, e.g.
> >
> >
> > Peter
> >
> > On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík  > > wrote:
> >
> >
> >
> > Sven Van Caekenberghe wrote:
> >
> > Hi Peter,
> >
> > Both cases will pass using this expression:
> >
> > String streamContents: [ :out |
> >  STON jsonWriter
> >  referencePolicy: #ignore;
> >  on: out;
> >  nextPut: d ].
> >
> > The problem is what to do with cycles.
> >
> >
> > Be faithful to what JSON.stringify does in JS. See attachment. :-)
> >
> > Herby
> >
> >
> > Sven
> >
> > On 27 Mar 2018, at 17:43, Peter Uhnák > >  wrote:
> >
> > Hi,
> >
> > I don't think this should throw an error when I am producing
> > JSON.
> >
> > ```
> > d := {
> > 'a' ->  #().
> > 'b' ->  #().
> > } asDictionary.
> >
> > STON toJsonString: d.
> > ```
> >
> > dtto this
> >
> > ```
> > a := {'hm'}.
> >
> > d := {
> > 'a' ->  a.
> > 'b' ->  a.
> > } asDictionary.
> >
> > STON toJsonString: d.
> > ```
> >
> > Maybe I should forgo using STON>>toJson* out of laziness (it
> > is preinstalled) and always go for NeoJSON?
> >
> > Peter
> >
> >
> >
> >
> >
> >
>
>
>

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Herbert Vojčík]

Sven Van Caekenberghe wrote:

The JSON spec does (of course) not deal with cycles nor structure sharing, STON 
does (by design).


That surprises me. I thought cycles are explicitly prohibited in JSON.


The class comment clearly states:

...
- referencePolicy<#normal|#ignore|#error>  default is #normal
if #normal, track and count object references and use references to 
implement sharing and break cycles
if #error, track object references and signal STONWriterError when a 
shared reference is encountered
if #ignore, don't track object references which might loop forever on 
cycles

...

For JSON production (not a main goes of STON), it was set to #error, but like I 
showed, it you could prefer #ignore. You have to know what to feed it.

Cycle/shared structure checking is expensive, i.e. slow(er) and uses more 
memory.


Maybe not. You do not need to _count_ references actually, just 
mark/unmark on enter/leave (that is, no (Identity)Dictionary is needed, 
just an (Identity)Set). Assuming you do depth-first.



I am not sure it is easy to make a difference between a cycle and a shared 
structure, I would have to think about that.

And yes, NeoJSON will go happily in cycles.


See above.


On 27 Mar 2018, at 18:52, Peter Uhnák  wrote:

Ah sorry, I'm blind.

Yes, ideally it should throw up an error in case of recursive.
And I see that NeoJSON runs into infinite cycle, so I guess this should be 
addressed there too.



On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík  wrote:


Peter Uhnák wrote:
Hi Sven,

shouldn't it be set to #ignore by default for JSON then? Or is there a
use case where it makes sense to have something else for JSON?

To keep STON practices / assumptions while using JSON to hold the data where 
STON is not supported yet, maybe? IIRC STON is positioning itself as JSON 
superset; but maybe only syntactic superset, while semantically it can be a bit 
different (so STONize to more low-level, JSON, syntax via STON; JSONize 
faithfully to JSON semantics using NeoJSON; but maybe I am just making things 
up).

Herbert: I am not talking about (infinitely) recursive references, but

I know; Sven explicitly asked what to do with cycles in that case, though.

referencing the same object, e.g.


Peter

On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík>  wrote:



 Sven Van Caekenberghe wrote:

 Hi Peter,

 Both cases will pass using this expression:

 String streamContents: [ :out |
  STON jsonWriter
  referencePolicy: #ignore;
  on: out;
  nextPut: d ].

 The problem is what to do with cycles.


 Be faithful to what JSON.stringify does in JS. See attachment. :-)

 Herby


 Sven

 On 27 Mar 2018, at 17:43, Peter Uhnák>   wrote:

 Hi,

 I don't think this should throw an error when I am producing
 JSON.

 ```
 d := {
 'a' ->   #().
 'b' ->   #().
 } asDictionary.

 STON toJsonString: d.
 ```

 dtto this

 ```
 a := {'hm'}.

 d := {
 'a' ->   a.
 'b' ->   a.
 } asDictionary.

 STON toJsonString: d.
 ```

 Maybe I should forgo using STON>>toJson* out of laziness (it
 is preinstalled) and always go for NeoJSON?

 Peter

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Sven Van Caekenberghe]

The JSON spec does (of course) not deal with cycles nor structure sharing, STON 
does (by design).

The class comment clearly states:

...
- referencePolicy <#normal|#ignore|#error> default is #normal
if #normal, track and count object references and use references to 
implement sharing and break cycles
if #error, track object references and signal STONWriterError when a 
shared reference is encountered
if #ignore, don't track object references which might loop forever on 
cycles

...

For JSON production (not a main goes of STON), it was set to #error, but like I 
showed, it you could prefer #ignore. You have to know what to feed it.

Cycle/shared structure checking is expensive, i.e. slow(er) and uses more 
memory.

I am not sure it is easy to make a difference between a cycle and a shared 
structure, I would have to think about that.

And yes, NeoJSON will go happily in cycles.

> On 27 Mar 2018, at 18:52, Peter Uhnák  wrote:
> 
> Ah sorry, I'm blind.
> 
> Yes, ideally it should throw up an error in case of recursive.
> And I see that NeoJSON runs into infinite cycle, so I guess this should be 
> addressed there too.
> 
> 
> 
> On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík  wrote:
> 
> 
> Peter Uhnák wrote:
> Hi Sven,
> 
> shouldn't it be set to #ignore by default for JSON then? Or is there a
> use case where it makes sense to have something else for JSON?
> 
> To keep STON practices / assumptions while using JSON to hold the data where 
> STON is not supported yet, maybe? IIRC STON is positioning itself as JSON 
> superset; but maybe only syntactic superset, while semantically it can be a 
> bit different (so STONize to more low-level, JSON, syntax via STON; JSONize 
> faithfully to JSON semantics using NeoJSON; but maybe I am just making things 
> up).
> 
> Herbert: I am not talking about (infinitely) recursive references, but
> 
> I know; Sven explicitly asked what to do with cycles in that case, though.
> 
> referencing the same object, e.g.
> 
> 
> Peter
> 
> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík  > wrote:
> 
> 
> 
> Sven Van Caekenberghe wrote:
> 
> Hi Peter,
> 
> Both cases will pass using this expression:
> 
> String streamContents: [ :out |
>  STON jsonWriter
>  referencePolicy: #ignore;
>  on: out;
>  nextPut: d ].
> 
> The problem is what to do with cycles.
> 
> 
> Be faithful to what JSON.stringify does in JS. See attachment. :-)
> 
> Herby
> 
> 
> Sven
> 
> On 27 Mar 2018, at 17:43, Peter Uhnák >  wrote:
> 
> Hi,
> 
> I don't think this should throw an error when I am producing
> JSON.
> 
> ```
> d := {
> 'a' ->  #().
> 'b' ->  #().
> } asDictionary.
> 
> STON toJsonString: d.
> ```
> 
> dtto this
> 
> ```
> a := {'hm'}.
> 
> d := {
> 'a' ->  a.
> 'b' ->  a.
> } asDictionary.
> 
> STON toJsonString: d.
> ```
> 
> Maybe I should forgo using STON>>toJson* out of laziness (it
> is preinstalled) and always go for NeoJSON?
> 
> Peter
> 
> 
> 
> 
> 
> 

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Peter Uhnák]

Ah sorry, I'm blind.

Yes, ideally it should throw up an error in case of recursive.
And I see that NeoJSON runs into infinite cycle, so I guess this should be
addressed there too.



On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík  wrote:

>
>
> Peter Uhnák wrote:
>
>> Hi Sven,
>>
>> shouldn't it be set to #ignore by default for JSON then? Or is there a
>> use case where it makes sense to have something else for JSON?
>>
>
> To keep STON practices / assumptions while using JSON to hold the data
> where STON is not supported yet, maybe? IIRC STON is positioning itself as
> JSON superset; but maybe only syntactic superset, while semantically it can
> be a bit different (so STONize to more low-level, JSON, syntax via STON;
> JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just
> making things up).
>
> Herbert: I am not talking about (infinitely) recursive references, but
>>
>
> I know; Sven explicitly asked what to do with cycles in that case, though.
>
> referencing the same object, e.g.
>>
>>
>> Peter
>>
>> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík > > wrote:
>>
>>
>>
>> Sven Van Caekenberghe wrote:
>>
>> Hi Peter,
>>
>> Both cases will pass using this expression:
>>
>> String streamContents: [ :out |
>>  STON jsonWriter
>>  referencePolicy: #ignore;
>>  on: out;
>>  nextPut: d ].
>>
>> The problem is what to do with cycles.
>>
>>
>> Be faithful to what JSON.stringify does in JS. See attachment. :-)
>>
>> Herby
>>
>>
>> Sven
>>
>> On 27 Mar 2018, at 17:43, Peter Uhnák> >  wrote:
>>
>> Hi,
>>
>> I don't think this should throw an error when I am producing
>> JSON.
>>
>> ```
>> d := {
>> 'a' ->  #().
>> 'b' ->  #().
>> } asDictionary.
>>
>> STON toJsonString: d.
>> ```
>>
>> dtto this
>>
>> ```
>> a := {'hm'}.
>>
>> d := {
>> 'a' ->  a.
>> 'b' ->  a.
>> } asDictionary.
>>
>> STON toJsonString: d.
>> ```
>>
>> Maybe I should forgo using STON>>toJson* out of laziness (it
>> is preinstalled) and always go for NeoJSON?
>>
>> Peter
>>
>>
>>
>>
>>
>

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Herbert Vojčík]

Peter Uhnák wrote:

Hi Sven,

shouldn't it be set to #ignore by default for JSON then? Or is there a
use case where it makes sense to have something else for JSON?


To keep STON practices / assumptions while using JSON to hold the data 
where STON is not supported yet, maybe? IIRC STON is positioning itself 
as JSON superset; but maybe only syntactic superset, while semantically 
it can be a bit different (so STONize to more low-level, JSON, syntax 
via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe 
I am just making things up).



Herbert: I am not talking about (infinitely) recursive references, but


I know; Sven explicitly asked what to do with cycles in that case, though.


referencing the same object, e.g.


Peter

On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík > wrote:



Sven Van Caekenberghe wrote:

Hi Peter,

Both cases will pass using this expression:

String streamContents: [ :out |
 STON jsonWriter
 referencePolicy: #ignore;
 on: out;
 nextPut: d ].

The problem is what to do with cycles.


Be faithful to what JSON.stringify does in JS. See attachment. :-)

Herby


Sven

On 27 Mar 2018, at 17:43, Peter Uhnák>  wrote:

Hi,

I don't think this should throw an error when I am producing
JSON.

```
d := {
'a' ->  #().
'b' ->  #().
} asDictionary.

STON toJsonString: d.
```

dtto this

```
a := {'hm'}.

d := {
'a' ->  a.
'b' ->  a.
} asDictionary.

STON toJsonString: d.
```

Maybe I should forgo using STON>>toJson* out of laziness (it
is preinstalled) and always go for NeoJSON?

Peter

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Peter Uhnák]

Hi Sven,

shouldn't it be set to #ignore by default for JSON then? Or is there a use
case where it makes sense to have something else for JSON?

Herbert: I am not talking about (infinitely) recursive references, but
referencing the same object, e.g.


Peter

On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík  wrote:

>
>
> Sven Van Caekenberghe wrote:
>
>> Hi Peter,
>>
>> Both cases will pass using this expression:
>>
>> String streamContents: [ :out |
>> STON jsonWriter
>> referencePolicy: #ignore;
>> on: out;
>> nextPut: d ].
>>
>> The problem is what to do with cycles.
>>
>
> Be faithful to what JSON.stringify does in JS. See attachment. :-)
>
> Herby
>
>
> Sven
>>
>> On 27 Mar 2018, at 17:43, Peter Uhnák  wrote:
>>>
>>> Hi,
>>>
>>> I don't think this should throw an error when I am producing JSON.
>>>
>>> ```
>>> d := {
>>> 'a' ->  #().
>>> 'b' ->  #().
>>> } asDictionary.
>>>
>>> STON toJsonString: d.
>>> ```
>>>
>>> dtto this
>>>
>>> ```
>>> a := {'hm'}.
>>>
>>> d := {
>>> 'a' ->  a.
>>> 'b' ->  a.
>>> } asDictionary.
>>>
>>> STON toJsonString: d.
>>> ```
>>>
>>> Maybe I should forgo using STON>>toJson* out of laziness (it is
>>> preinstalled) and always go for NeoJSON?
>>>
>>> Peter
>>>
>>
>>
>>

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Herbert Vojčík]

Sven Van Caekenberghe wrote:

Hi Peter,

Both cases will pass using this expression:

String streamContents: [ :out |
STON jsonWriter
referencePolicy: #ignore;
on: out;
nextPut: d ].

The problem is what to do with cycles.


Be faithful to what JSON.stringify does in JS. See attachment. :-)

Herby


Sven


On 27 Mar 2018, at 17:43, Peter Uhnák  wrote:

Hi,

I don't think this should throw an error when I am producing JSON.

```
d := {
'a' ->  #().
'b' ->  #().
} asDictionary.

STON toJsonString: d.
```

dtto this

```
a := {'hm'}.

d := {
'a' ->  a.
'b' ->  a.
} asDictionary.

STON toJsonString: d.
```

Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) 
and always go for NeoJSON?

Peter

Re: [Pharo-users] STON shouldn't care about shared references in JSON mode

[Sven Van Caekenberghe]

Hi Peter,

Both cases will pass using this expression:

String streamContents: [ :out | 
STON jsonWriter 
referencePolicy: #ignore; 
on: out; 
nextPut: d ].

The problem is what to do with cycles.

Sven

> On 27 Mar 2018, at 17:43, Peter Uhnák  wrote:
> 
> Hi,
> 
> I don't think this should throw an error when I am producing JSON.
> 
> ```
> d := { 
>   'a' -> #().
>   'b' -> #().
> } asDictionary.
> 
> STON toJsonString: d.
> ```
> 
> dtto this
> 
> ```
> a := {'hm'}.
> 
> d := { 
>   'a' -> a.
>   'b' -> a.
> } asDictionary.
> 
> STON toJsonString: d.
> ```
> 
> Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) 
> and always go for NeoJSON?
> 
> Peter

[Pharo-users] STON shouldn't care about shared references in JSON mode

[Peter Uhnák]

Hi,

I don't think this should throw an error when I am producing JSON.

```
d := {
'a' -> #().
'b' -> #().
} asDictionary.

STON toJsonString: d.
```

dtto this

```
a := {'hm'}.

d := {
'a' -> a.
'b' -> a.
} asDictionary.

STON toJsonString: d.
```

Maybe I should forgo using STON>>toJson* out of laziness (it is
preinstalled) and always go for NeoJSON?

Peter