Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
On Sun, Nov 27, 2011 at 06:39:52PM +0100, Moritz Heidkamp wrote:
> Vok Vojwo writes:
> > I think the Medea egg intends to do it the right way. But it seems to be
> > buggy.
> >
> > And it has a voracious appetite
>
> It does indeed :-)
>
>
> > Medea fails to parse the data:
> >
> > (use medea)
> > (read-json json) ;; => #f
>
> Thanks for the hint. I managed to bisect it down to a Unicode character
> in one of the strings ("’"). Looking at medea's test suite I found this:
>
> ;; (test-read '#("Дҫ") "[\"Дҫ\"]") ; FIXME genturfahi needs utf8 support for
> that to work
>
> So thanks for the reminder, I should mention this limitation in medea's
> documentation. Alan, would it be possible to make genturfahi UTF-8
> aware?
>
Yes. I will accomplish it over the hackathon weekend. My first
attempt didn't go well, though it was because I was having trouble
getting basic UTF-like things working in Chicken at all. I actually
have some of the required support ready to be checked in here in my
local repository.
-Alan
--
.i ma'a lo bradi cu penmi gi'e du
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
Moritz Heidkamp writes:
> Thanks for the hint. I managed to bisect it down to a Unicode character
> in one of the strings ("’"). Looking at medea's test suite I found this:
>
> ;; (test-read '#("Дҫ") "[\"Дҫ\"]") ; FIXME genturfahi needs utf8 support for
> that to work
>
> So thanks for the reminder, I should mention this limitation in medea's
> documentation.
OK, the documentation mentions this now. Note by the way that Unicode
escape sequences (\u) work.
Moritz
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
Vok Vojwo writes:
> I think the Medea egg intends to do it the right way. But it seems to be
> buggy.
>
> And it has a voracious appetite
It does indeed :-)
> Medea fails to parse the data:
>
> (use medea)
> (read-json json) ;; => #f
Thanks for the hint. I managed to bisect it down to a Unicode character
in one of the strings ("’"). Looking at medea's test suite I found this:
;; (test-read '#("Дҫ") "[\"Дҫ\"]") ; FIXME genturfahi needs utf8 support for
that to work
So thanks for the reminder, I should mention this limitation in medea's
documentation. Alan, would it be possible to make genturfahi UTF-8
aware?
Moritz
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
John Cowan writes: > They differ in their representation of JSON null, however: json-abnf > uses 'null, whereas medea's default is () the other way around, actually :-) Moritz ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
Vok Vojwo scripsit: > I am a bit confused by the way the JSON egg maps JSON structures to > Scheme values. [...] > Is there any reason why the JSON egg creates vectors of pairs? The problem is that JSON arrays and objects are disjoint, and the egg uses lists to represent JSON arrays. Since lists and a-lists are obviously not disjoint, something else has to be done. One alternative would be to represent JSON objects as a-lists and JSON arrays as vectors. This is the approach taken by the json-abnf egg, and by the medea egg by default, although medea's representation is customizable. They differ in their representation of JSON null, however: json-abnf uses 'null, whereas medea's default is (); the json egg uses the undefined value. I am going to be adding JSON support to my JSO egg when I can steal a few cycles for it. JSOs are specially marked a-lists that work like JavaScript objects, so I will use them to represent JSON objects. JSON null will be represented by a unique empty JSO. I haven't yet settled on the representation of JSON arrays; I might use specially marked JSOs, or I might use lists (which would require the caller to test for arrays before objects) or even vectors. -- John Cowan http://www.ccil.org/~cowan co...@ccil.org "After all, would you consider a man without honor wealthy, even if his Dinar laid end to end would reach from here to the Temple of Toplat?" "No, I wouldn't", the beggar replied. "Why is that?" the Master asked. "A Dinar doesn't go very far these days, Master.--Kehlog Albran Besides, the Temple of Toplat is across the street." The Profit ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
2011/11/27 Moritz Heidkamp : > > You may be interested in two alternative JSON eggs, namely json-abnf egg > (GPLed) which represents both JSON objects as tagged lists alists > (i.e. they have a symbol `object' their car) and arrays as vectors or > the medea egg (BSD licensed) which uses alists for objects and vectors > for arrays by default but can be parameterized to use whatever > representation you prefer. I think the Medea egg intends to do it the right way. But it seems to be buggy. And it has a voracious appetite ; loading /usr/local/lib/chicken/6/chicken.import.so ... ; loading /usr/local/lib/chicken/6/data-structures.import.so ... ; loading /usr/local/lib/chicken/6/defstruct.so ... ; loading /usr/local/lib/chicken/6/extras.import.so ... ; loading /usr/local/lib/chicken/6/genturfahi.import.so ... ; loading /usr/local/lib/chicken/6/genturfahi-peg.so ... ; loading /usr/local/lib/chicken/6/genturfahi.so ... ; loading /usr/local/lib/chicken/6/irregex.import.so ... ; loading /usr/local/lib/chicken/6/iset.import.so ... ; loading /usr/local/lib/chicken/6/iset.so ... ; loading /usr/local/lib/chicken/6/lolevel.import.so ... ; loading /usr/local/lib/chicken/6/matchable.import.so ... ; loading /usr/local/lib/chicken/6/matchable.so ... ; loading /usr/local/lib/chicken/6/medea.import.so ... ; loading /usr/local/lib/chicken/6/medea.so ... ; loading /usr/local/lib/chicken/6/ports.import.so ... ; loading /usr/local/lib/chicken/6/posix.import.so ... ; loading /usr/local/lib/chicken/6/regex.import.so ... ; loading /usr/local/lib/chicken/6/regex.so ... ; loading /usr/local/lib/chicken/6/sandbox.so ... ; loading /usr/local/lib/chicken/6/scheme.import.so ... ; loading /usr/local/lib/chicken/6/srfi-13.import.so ... ; loading /usr/local/lib/chicken/6/srfi-14.import.so ... ; loading /usr/local/lib/chicken/6/srfi-18.import.so ... ; loading /usr/local/lib/chicken/6/srfi-1.import.so ... ; loading /usr/local/lib/chicken/6/srfi-4.import.so ... ; loading /usr/local/lib/chicken/6/srfi-69.import.so ... ; loading /usr/local/lib/chicken/6/unicode-char-sets.import.so ... ; loading /usr/local/lib/chicken/6/unicode-char-sets.so ... ; loading /usr/local/lib/chicken/6/utf8-case-map.import.so ... ; loading /usr/local/lib/chicken/6/utf8-case-map.so ... ; loading /usr/local/lib/chicken/6/utf8.import.so ... ; loading /usr/local/lib/chicken/6/utf8-lolevel.import.so ... ; loading /usr/local/lib/chicken/6/utf8-lolevel.so ... ; loading /usr/local/lib/chicken/6/utf8.so ... ; loading /usr/local/lib/chicken/6/utf8-srfi-13.import.so ... ; loading /usr/local/lib/chicken/6/utf8-srfi-13.so ... ; loading /usr/local/lib/chicken/6/utf8-srfi-14.import.so ... ; loading /usr/local/lib/chicken/6/utf8-srfi-14.so ... ; loading /usr/local/lib/chicken/6/vector-lib.import.so ... ; loading /usr/local/lib/chicken/6/vector-lib.so ... compared to the Json egg: ; loading /usr/local/lib/chicken/6/json.import.so ... ; loading /usr/local/lib/chicken/6/scheme.import.so ... ; loading /usr/local/lib/chicken/6/chicken.import.so ... ; loading /usr/local/lib/chicken/6/ports.import.so ... ; loading /usr/local/lib/chicken/6/srfi-1.import.so ... ; loading /usr/local/lib/chicken/6/srfi-69.import.so ... ; loading /usr/local/lib/chicken/6/packrat.import.so ... ; loading /usr/local/lib/chicken/6/json.so ... ; loading /usr/local/lib/chicken/6/packrat.so ... I am testing with Googles developer calendar: (use http-client) (define json (with-input-from-request "http://www.google.com/calendar/feeds/developer-calen...@google.com/public/full?alt=json"; #f read-string)) Medea fails to parse the data: (use medea) (read-json json) ;; => #f Json works but it is quite slow: (use json) (time (begin (json-read (open-input-string json)) (if #f #f))) 0.228s CPU time, 0.12s GC time (major), 135063 mutations, 6/1835 GCs (major/minor) 0.236s CPU time, 0.128s GC time (major), 135045 mutations, 6/1835 GCs (major/minor) compared to Json-abnf: (use json-abnf) (time (begin (parser json) (if #f #f))) 0.04s CPU time, 2380 mutations, 0/782 GCs (major/minor) 0.04s CPU time, 2398 mutations, 0/782 GCs (major/minor) I am completely astonished that the performance difference is more than 500%. Is this the typical packrat memorization penalty? So in the end I have - Medea which has the most useful data representation but does not work. - Json which need the fewest number of modules, but which is slow and produces strange pairs of vectors. - Json-abnf which is the fastest but produces annoying object tags and can not read from ports. If it is possible to get rid of the object tags Json-abnf seems to be the best choice right now. ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
Hi Vok,
Vok Vojwo writes:
> I am a bit confused by the way the JSON egg maps JSON structures to
> Scheme values. The JSON egg maps a structure to a vector:
>
> (use json)
> (with-input-from-string "{\"pi\":3.14,\"e\":2.71}" json-read)
> ;; => #(("pi" . 3.14) ("e" . 2.71))
I agree, this is indeed a strange choice. My guess is that there is no
way to reliably detect whether a list is an alist or not. And since the
json egg maps lists to JSON arrays already, "avectors" were the only
choice left, it seems.
You may be interested in two alternative JSON eggs, namely json-abnf egg
(GPLed) which represents both JSON objects as tagged lists alists
(i.e. they have a symbol `object' their car) and arrays as vectors or
the medea egg (BSD licensed) which uses alists for objects and vectors
for arrays by default but can be parameterized to use whatever
representation you prefer.
Moritz
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
* Vok Vojwo [27 14:37]:
> 2011/11/27 Christian Kellermann :
> >
> > My guess is that it makes it possible to write back to json, since
> > alists are already used for representing a different datatype.
>
> But json-write does not accept alists as values. This
>
> (json-write (vector (cons "x" (list (cons "a" 1) (cons "b" 2)
>
> throws the following error:
>
> Error: Invalid JSON object in json-write: ("a" . 1)
This is true. I mistook them for lists, which are already used.
Sorry for the noise.
--
Who can (make) the muddy water (clear)? Let it be still, and it will
gradually become clear. Who can secure the condition of rest? Let
movement go on, and the condition of rest will gradually arise.
-- Lao Tse.
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
Even though I agree that it should be schemish,
it must be consistent.
How would you deal with
{"1":[2,3],"4":[5,6]}
and
[["1",2,3],["4",5,6]]
that must be distinguished?
Furthermore, if we were to change the spec of the json egg,
we should be aware with backward compatibility issues.
Otherwise, I need to change my all json-related code.
Best,
Daishi
At Sun, 27 Nov 2011 14:16:38 +0100,
Vok Vojwo wrote:
>
> I am a bit confused by the way the JSON egg maps JSON structures to
> Scheme values. The JSON egg maps a structure to a vector:
>
> (use json)
> (with-input-from-string "{\"pi\":3.14,\"e\":2.71}" json-read)
> ;; => #(("pi" . 3.14) ("e" . 2.71))
>
> This makes it impossible to use the standard Scheme function assoc to
> read the data.
>
> The following functions fix the problem:
>
> (define (json->alist arg)
> (cond
>((vector? arg)
> (map (lambda (pair)
>(cons (car pair) (json->alist (cdr pair
> (vector->list arg)))
>((list? arg)
> (list->vector (map json->alist arg)))
>(else arg)))
>
> (define (alist->json arg)
> (cond
>((list? arg)
> (list->vector (map (lambda (pair)
> (cons (car pair) (alist->json (cdr pair
>arg)))
>((vector? arg)
> (map alist->json (vector->list arg)))
>(else arg)))
>
> A small verification test:
>
> (use http-client)
>
> (define (with-input-from-url url chunk)
> (with-input-from-request url #f chunk))
>
> (define json
> (with-input-from-url
>
> "http://www.google.com/calendar/feeds/developer-calen...@google.com/public/full?alt=json";
>json-read))
>
> (equal? json (alist->json (json->alist json)))
> ;; => #t
>
> By using alists it is possible to use assoc to access the data:
>
> (define (assoc* alist . path)
> (let assoc* ((path path)
>(alist alist))
> (if (null? path)
> alist
> (assoc* (cdr path)
> (cdr (assoc (car path) alist))
>
> (assoc* (json->alist json) "feed" "title" "$t")
> ;; => "Official Google External Developer Events"
>
> Is there any reason why the JSON egg creates vectors of pairs?
>
> If not I would suggest to fix it to make it more schemish.
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
2011/11/27 Christian Kellermann :
>
> My guess is that it makes it possible to write back to json, since
> alists are already used for representing a different datatype.
But json-write does not accept alists as values. This
(json-write (vector (cons "x" (list (cons "a" 1) (cons "b" 2)
throws the following error:
Error: Invalid JSON object in json-write: ("a" . 1)
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
* Vok Vojwo [27 14:16]: > (assoc* (json->alist json) "feed" "title" "$t") > ;; => "Official Google External Developer Events" > > Is there any reason why the JSON egg creates vectors of pairs? > > If not I would suggest to fix it to make it more schemish. I agree with you and I can only guess the original author's intentions. My guess is that it makes it possible to write back to json, since alists are already used for representing a different datatype. Surely the choice could have been made in a more convenient way... Kind regards, Christian -- Who can (make) the muddy water (clear)? Let it be still, and it will gradually become clear. Who can secure the condition of rest? Let movement go on, and the condition of rest will gradually arise. -- Lao Tse. ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
[Chicken-users] Why does the JSON egg map JSON structs to Scheme vectors instead of alists?
I am a bit confused by the way the JSON egg maps JSON structures to
Scheme values. The JSON egg maps a structure to a vector:
(use json)
(with-input-from-string "{\"pi\":3.14,\"e\":2.71}" json-read)
;; => #(("pi" . 3.14) ("e" . 2.71))
This makes it impossible to use the standard Scheme function assoc to
read the data.
The following functions fix the problem:
(define (json->alist arg)
(cond
((vector? arg)
(map (lambda (pair)
(cons (car pair) (json->alist (cdr pair
(vector->list arg)))
((list? arg)
(list->vector (map json->alist arg)))
(else arg)))
(define (alist->json arg)
(cond
((list? arg)
(list->vector (map (lambda (pair)
(cons (car pair) (alist->json (cdr pair
arg)))
((vector? arg)
(map alist->json (vector->list arg)))
(else arg)))
A small verification test:
(use http-client)
(define (with-input-from-url url chunk)
(with-input-from-request url #f chunk))
(define json
(with-input-from-url
"http://www.google.com/calendar/feeds/developer-calen...@google.com/public/full?alt=json";
json-read))
(equal? json (alist->json (json->alist json)))
;; => #t
By using alists it is possible to use assoc to access the data:
(define (assoc* alist . path)
(let assoc* ((path path)
(alist alist))
(if (null? path)
alist
(assoc* (cdr path)
(cdr (assoc (car path) alist))
(assoc* (json->alist json) "feed" "title" "$t")
;; => "Official Google External Developer Events"
Is there any reason why the JSON egg creates vectors of pairs?
If not I would suggest to fix it to make it more schemish.
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
