Re: [Web-SIG] Time a for JSON parser in the standard library?

[Benji York]

Bob Ippolito wrote:
> On Wed, Apr 9, 2008 at 8:04 PM, John Millikin <[EMAIL PROTECTED]> wrote:

>>  For "exception by default" behavior, the library can call
>>  warnings.simplefilter ("error", JSONWarning) on initialization. Then
>>  we just define various subclasses of JSONWarning for the different
>>  warning types (trailing comma, illegal whitespace, NaN etc). If users
>>  want to ignore some or all of the warnings, they can specify which (or
>>  all). Best of all, it wouldn't require long parameter lists.
> 
> That sounds like a really bad idea

Indeed.
-- 
Benji York
http://benjiyork.com
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[Bob]
>  simplejson would give you an error and tell you exactly where the
>  problem was,

Another good point.

Other JSON modules should follow simplejson's lead, and provide access
to the location in the document where the lexical or parse error
occurred, so that the offending document can be opened in a text
editor to determine the source of the problem, and perhaps fix it.

This should also apply to "junk" after the document object, i.e. JSON
expressions present in the document after the main document has been
successfully parsed. A strict interpretation of the spec is that such
"junk" is not permitted, and makes the JSON document broken, even
though the main object representation is valid.

Simplejson has an option for the user to control this, and jyson does
too; I don't know about the others.

[Bob]
> but there isn't currently a non-strict mode and honestly
>  nobody has asked for it.

If we only need "strict mode", then why do all of our parsers have options?

Isn't "permissive mode" just a way of setting all of the parse options
to liberal, in one go?

Alan.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Ian Bicking]

John Millikin wrote:
> On Wed, Apr 9, 2008 at 10:15 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>>  That sounds like a really bad idea, if there is an option to change
>>  the behavior it shouldn't live in module state.
>>
> Would you rather have strictness controls as parameters? demjson
> currently has seventeen of those. Maybe we could have loads(bytes) and
> loads_broken(bytes, allow_trailing_comma, allow_all_whitespace,
> allow_comments, ...) functions, one for parsing JSON, the other for
> parsing garbage.  There's no real way to hide or remove the complexity
> in parsing invalid data, so both warnings and parameters will cause
> the implementation to be much larger, but at least having to call
> warnings.filter ("ignore", JSONWarning) might serve to make some users
> think twice.

What reason is there for all the different flags?  Why not just strict 
and loose?


-- 
Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Wed, Apr 9, 2008 at 11:01 PM, John Millikin <[EMAIL PROTECTED]> wrote:
> On Wed, Apr 9, 2008 at 10:15 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  >  That sounds like a really bad idea, if there is an option to change
>  >  the behavior it shouldn't live in module state.
>  >
>  Would you rather have strictness controls as parameters? demjson
>  currently has seventeen of those. Maybe we could have loads(bytes) and
>  loads_broken(bytes, allow_trailing_comma, allow_all_whitespace,
>  allow_comments, ...) functions, one for parsing JSON, the other for
>  parsing garbage.  There's no real way to hide or remove the complexity
>  in parsing invalid data, so both warnings and parameters will cause
>  the implementation to be much larger, but at least having to call
>  warnings.filter ("ignore", JSONWarning) might serve to make some users
>  think twice.
>

Yeah just like sys.setdefaultencoding, look at how well that worked out.

simplejson has a number of options that you can pass to loads and
dumps, but if you don't explicitly use any of them it does what most
people want by default. I don't see a "loose=True" being much worse, I
probably wouldn't allow people to pick and choose exactly which
garbage to allow (except for the spec ambiguities that it already
gives you control over).

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

On Wed, Apr 9, 2008 at 10:15 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  That sounds like a really bad idea, if there is an option to change
>  the behavior it shouldn't live in module state.
>
Would you rather have strictness controls as parameters? demjson
currently has seventeen of those. Maybe we could have loads(bytes) and
loads_broken(bytes, allow_trailing_comma, allow_all_whitespace,
allow_comments, ...) functions, one for parsing JSON, the other for
parsing garbage.  There's no real way to hide or remove the complexity
in parsing invalid data, so both warnings and parameters will cause
the implementation to be much larger, but at least having to call
warnings.filter ("ignore", JSONWarning) might serve to make some users
think twice.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Wed, Apr 9, 2008 at 8:04 PM, John Millikin <[EMAIL PROTECTED]> wrote:
> On Wed, Apr 9, 2008 at 7:20 PM, Ian Bicking <[EMAIL PROTECTED]> wrote:
>  >  I prefer liberal, with an option to turn on warnings or be strict.
>  >  Potentially warnings could be turned on by default, and turned off with
>  >  the warnings module.  (Is there a way to have a warning raise an
>  >  exception by default?  I only know of the command-line option to the
>  >  interpreter.)
>  >
>  I forgot about the warnings module.
>
>  For "exception by default" behavior, the library can call
>  warnings.simplefilter ("error", JSONWarning) on initialization. Then
>  we just define various subclasses of JSONWarning for the different
>  warning types (trailing comma, illegal whitespace, NaN etc). If users
>  want to ignore some or all of the warnings, they can specify which (or
>  all). Best of all, it wouldn't require long parameter lists.

That sounds like a really bad idea, if there is an option to change
the behavior it shouldn't live in module state.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

On Wed, Apr 9, 2008 at 7:20 PM, Ian Bicking <[EMAIL PROTECTED]> wrote:
>  I prefer liberal, with an option to turn on warnings or be strict.
>  Potentially warnings could be turned on by default, and turned off with
>  the warnings module.  (Is there a way to have a warning raise an
>  exception by default?  I only know of the command-line option to the
>  interpreter.)
>
I forgot about the warnings module.

For "exception by default" behavior, the library can call
warnings.simplefilter ("error", JSONWarning) on initialization. Then
we just define various subclasses of JSONWarning for the different
warning types (trailing comma, illegal whitespace, NaN etc). If users
want to ignore some or all of the warnings, they can specify which (or
all). Best of all, it wouldn't require long parameter lists.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Ian Bicking]

Alan Kennedy wrote:
> [John]
>>  Sounds like a case *for* strict parsing, in my opinion. PHP's loose
>>  parsing made it difficult to figure out why the JSON was invalid. If
>>  trailing comma handling is to try to work around copy-paste errors, -1
>>  from me.
> 
> No, the PHP lib did exactly what it should, IMHO. The PHP lib was
> liberal in what it consumed (a dangling comma), and strict in what it
> produced (no dangling comma).
> 
> It accepted my broken document with a dangling-comma, and emitted a
> strictly conformant document with the offending comma removed, which
> enabled my co-worker to proceed with his job.

I prefer liberal, with an option to turn on warnings or be strict. 
Potentially warnings could be turned on by default, and turned off with 
the warnings module.  (Is there a way to have a warning raise an 
exception by default?  I only know of the command-line option to the 
interpreter.)

   Ian
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Wed, Apr 9, 2008 at 5:48 PM, Alan Kennedy <[EMAIL PROTECTED]> wrote:
> [Alan]
>  >> [hand written JSON containing a] hard-to-spot dangling comma, from all the
>
> >>  copying and pasting. That broke his javascript library; he solved the
>  >>  problem by passing it through a PHP JSON codec on his local Apache. It
>  >>  worked, i.e. his problem disappeared, but he didn't know why (the PHP
>  >>  lib had eliminated the dangling comma). Which all goes to confirm,
>  >>  IMHO, that you should be liberal in what you consume and strict in
>  >>  what you produce.
>
>  [John]
>
> >  Sounds like a case *for* strict parsing, in my opinion. PHP's loose
>  >  parsing made it difficult to figure out why the JSON was invalid. If
>  >  trailing comma handling is to try to work around copy-paste errors, -1
>  >  from me.
>
>  No, the PHP lib did exactly what it should, IMHO. The PHP lib was
>  liberal in what it consumed (a dangling comma), and strict in what it
>  produced (no dangling comma).
>
>  It accepted my broken document with a dangling-comma, and emitted a
>  strictly conformant document with the offending comma removed, which
>  enabled my co-worker to proceed with his job.
>
>  +1 from me.
>
>  Other opinions?

simplejson would give you an error and tell you exactly where the
problem was, but there isn't currently a non-strict mode and honestly
nobody has asked for it.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[Alan]
>> [hand written JSON containing a] hard-to-spot dangling comma, from all the
>>  copying and pasting. That broke his javascript library; he solved the
>>  problem by passing it through a PHP JSON codec on his local Apache. It
>>  worked, i.e. his problem disappeared, but he didn't know why (the PHP
>>  lib had eliminated the dangling comma). Which all goes to confirm,
>>  IMHO, that you should be liberal in what you consume and strict in
>>  what you produce.

[John]
>  Sounds like a case *for* strict parsing, in my opinion. PHP's loose
>  parsing made it difficult to figure out why the JSON was invalid. If
>  trailing comma handling is to try to work around copy-paste errors, -1
>  from me.

No, the PHP lib did exactly what it should, IMHO. The PHP lib was
liberal in what it consumed (a dangling comma), and strict in what it
produced (no dangling comma).

It accepted my broken document with a dangling-comma, and emitted a
strictly conformant document with the offending comma removed, which
enabled my co-worker to proceed with his job.

+1 from me.

Other opinions?

Alan.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

On Wed, Apr 9, 2008 at 10:05 AM, Ian Bicking <[EMAIL PROTECTED]> wrote:
> I strongly prefer we stick to the conventional names of 
> dump/dumps/load/loads, for consistency with other serialization libraries 
> already in Python.
>

On Wed, Apr 9, 2008 at 10:27 AM, Benji York <[EMAIL PROTECTED]> wrote:
> +1
>

On Wed, Apr 9, 2008 at 10:28 AM, Duncan McGreggor
<[EMAIL PROTECTED]> wrote:
> +1 for me too.
>

PEP updated to use dump/dumps/load/loads

On Wed, Apr 9, 2008 at 11:38 AM, Alan Kennedy <[EMAIL PROTECTED]> wrote:
>  Answer #2: I'm working (i.e. day job) with JSON at the moment: a
>  javascript client talking to a java server. The JS guy had a problem
>  last week with a sample JSON document I gave him to prototype on. I
>  wrote the sample by hand (it later became my freemarker template), and
>  so inadvertently left in a hard-to-spot dangling comma, from all the
>  copying and pasting. That broke his javascript library; he solved the
>  problem by passing it through a PHP JSON codec on his local Apache. It
>  worked, i.e. his problem disappeared, but he didn't know why (the PHP
>  lib had eliminated the dangling comma). Which all goes to confirm,
>  IMHO, that you should be liberal in what you consume and strict in
>  what you produce.
>
Sounds like a case *for* strict parsing, in my opinion. PHP's loose
parsing made it difficult to figure out why the JSON was invalid. If
trailing comma handling is to try to work around copy-paste errors, -1
from me.

>  I'm beginning to think that any putative JSON API should permit the
>  user to specify which class will be used to instantiate JSON objects.
>  If the users can specify their own classes, that might go a long way
>  way resolve issues such as "I need my javascript client to communicate
>  Numbers representing radians to my python server which uses Decimal
>  because it works better with my geo-positioning library". Standard
>  libraries should provide their own set of default instantiation
>  classes, which the user could override.
>
This is the float v. Decimal thing again -- load(s) might grow a
parameter for that, since it's hard to be both fast and correct. But
what is the use case for overriding the mappings for other JSON types,
like arrays or objects? If given the choice, I'd rather have a very
simple API in the stdlib that can be wrapped or implemented by third
parties if they need something weird, than a large API that is
difficult to implement fully.
PEP: XXX
Title: A JSON handling library
Version: $Revision$
Last-Modified: $Date$
Author: John Millikin <[EMAIL PROTECTED]>
Discussions-To: web-sig@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 05-Apr-2008
Python-Version: 2.6


Abstract


This PEP describes a proposed library for parsing and generating
data in the `JSON` [1]_ format. JSON stands for "JavaScript Object
Notation", and is described by RFC 4627 [2]_.

Rationale
=

JSON is a widely-used data interchange format, often used for sending
data to and from a web browser using Javascript. Its simplicity and
ease of use has lead to various implementations with varying degrees
of compliance to the RFC. By bundling a capable implementation in
Python's standard library, I hope to reduce or eliminate the need
for choosing a JSON library.

Existing Public libraries
=

* Bob Ippolito's simplejson [3]_
* Deron Meranda's demjson [4]_
* John Millikin's jsonlib [5]_
* Alan Kennedy mentioned on web-sig [6]_ that he has written
  an implementation for Jython, named jyson, but has not released
  the source code.

Each of these have different APIs, different degrees of strictness,
and different qualities of error handling.

Module Interface


Parsing
---

Encoding Autodetection
''

The RFC requires that JSON is encoded in one of the Unicode encodings.
Because the first two bytes in a valid JSON expression are always from
the ASCII set, it is possible to reliably determine the encoding of
input data. Functions for autodetecting encoding exist in jsonlib and
demjson.

Parsing API
'''

A JSON expression may be parsed using the ``load`` or ``loads`` functions::

  load (file)
  loads (bytes_or_string)

If the input is encoded as a byte stream, the encoding should be auto-detected
as above. If input has been recieved in a non-standard encoding, it can
be manually decoded and passed to ``parse`` as a string. The return
value is either a sequence or mapping, depending on the input.

Serialization
-

Python objects may be serialized using the ``dump`` and ``dumps``
functions::

  dump (obj, file, indent = None, ascii_only = True, encoding = 'utf-8')
  dumps (obj, indent = None, ascii_only = True, encoding = 'utf-8')

``indent`` is used to control pretty-printing. If ``None``, no pretty
printing will be performed and the output will be maximally compact.
If ``indent`` is a string, that string will be used for indenting
nested value

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[John]
> I'm interested in whether you generally use JSON to communicate with a
> JavaScript client, or another JSON library. Both the demjson and simplejson
> libraries are written with the assumption that they are to be used to
> interact with JavaScript.

Answer #1: My motive is simply to implement the JSON spec, in a
[j|p]ythonic way. If the ideal of JSON is to be realised, then the
producer of the document is not relevant: it is only the document
itself that matters.

Answer #2: I'm working (i.e. day job) with JSON at the moment: a
javascript client talking to a java server. The JS guy had a problem
last week with a sample JSON document I gave him to prototype on. I
wrote the sample by hand (it later became my freemarker template), and
so inadvertently left in a hard-to-spot dangling comma, from all the
copying and pasting. That broke his javascript library; he solved the
problem by passing it through a PHP JSON codec on his local Apache. It
worked, i.e. his problem disappeared, but he didn't know why (the PHP
lib had eliminated the dangling comma). Which all goes to confirm,
IMHO, that you should be liberal in what you consume and strict in
what you produce.

[John]
> You mentioned in an earlier e-mail that jyson supports reading arrays with
> trailing commas -- is this intentional, or accidental? Do you read them with
> Python or JavaScript semantics?

Went out of my way to accept them, with python semantics.

Javascript semantics differ. Last time I tested, FireFox and IE
interpreted "[1,2,3,]" differently as [1,2,3] and [1,2,3,null].
Although that may have changed during the meanwhilst.

[Alan]
> > 2. To have a native-code implementation, customised for jython.

[John]
> Did you encounter any particular issues related to implementing a JSON
> library in Jython that would affect how a standard library implementation's
> API should be designed?

Jython is changing rapidly. It is evolving from a 2.2 stage ("from
__future__ import generators") to a 2.5 stage in one leap. Jython 2.5
is built with java 1.5 (1.5 is where java grew annotations and
generics). Between 2.2. and 2.5, python has grown Decimal's, generator
comprehensions, decorators, context managers, bi-directional
generators, etc. I prefer for a pure java implementation of a JSON
codec to remain flexible in terms of the way that it maps
"fundamental" JSON types into the jython type hierarchy and
interpreter machinery[1].

I'm beginning to think that any putative JSON API should permit the
user to specify which class will be used to instantiate JSON objects.
If the users can specify their own classes, that might go a long way
way resolve issues such as "I need my javascript client to communicate
Numbers representing radians to my python server which uses Decimal
because it works better with my geo-positioning library". Standard
libraries should provide their own set of default instantiation
classes, which the user could override.

Regards,

Alan.

[1] There is an argument that a pure java JSON parser for jython is
not worth the effort, in performance terms at least. JVM optimisation
is very sophisticated these days, and it is conceivable that pure
python (byte)code could run as fast or faster on a JVM than equivalent
java code. Think PyPy. So maybe a single well-designed pure-python
JSON module in the cpython standard library is the way to go.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Duncan McGreggor]

On Wed, Apr 9, 2008 at 12:27 PM, Benji York <[EMAIL PROTECTED]> wrote:
> Ian Bicking wrote:
>  >>   parse (bytes_or_string)
>  >>   generate (obj, indent = None, ascii_only = True, encoding = 'utf-8')
>  >
>  > I strongly prefer we stick to the conventional names of
>  > dump/dumps/load/loads, for consistency with other serialization
>  > libraries already in Python.
>
>  +1

+1 for me too.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Benji York]

Ian Bicking wrote:
>>   parse (bytes_or_string)
>>   generate (obj, indent = None, ascii_only = True, encoding = 'utf-8')
> 
> I strongly prefer we stick to the conventional names of 
> dump/dumps/load/loads, for consistency with other serialization 
> libraries already in Python.

+1
-- 
Benji York
http://benjiyork.com
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Ian Bicking]

>   parse (bytes_or_string)
>   generate (obj, indent = None, ascii_only = True, encoding = 'utf-8')

I strongly prefer we stick to the conventional names of 
dump/dumps/load/loads, for consistency with other serialization 
libraries already in Python.

   Ian
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Robert Brewer]

John Millikin wrote:
> On Sat, Apr 5, 2008 at 7:01 PM, Robert Brewer <[EMAIL PROTECTED]> wrote:
> > Re: Representation of Fractional Numbers, there are two solutions. If you
> > return decimals, people using JS on the other end are going to call 
> > float(d).
> > If you return floats, people not using JS on the other end are going to go
> > use a different library. I suggest the former is more acceptable than the
> > latter for a stdlib offering. Allowing the caller of parse() to choose
> > would be even better.
> 
> I don't understand what you mean, here. generate ([decimal.Decimal ('1.1')])
> -> '[1.1]', so a JavaScript user calling eval() on it would get a standard
> JavaScript float object without having to call float() explicitly.

Sorry, I wasn't describing what anyone would do in Javascript. Pythonistas 
receiving JSON numbers from a JS *sender*, who want Python floats, can call 
float(d) if they like if you hand them a Decimal object. Annoying but easy. 
People receiving JSON numbers from, say, a Python sender, can't call Decimal(f) 
if you hand them a float instance, at least not reliably. So they'll either go 
use some other jsonlib (bad) or start passing numbers in strings (worse).


Robert Brewr
[EMAIL PROTECTED]

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

(messed up CC on last email, re-sending to list)

On Sat, Apr 5, 2008 at 7:01 PM, Robert Brewer <[EMAIL PROTECTED]> wrote:

> Re: Representation of Fractional Numbers, there are two solutions. If you
> return decimals, people using JS on the other end are going to call
> float(d). If you return floats, people not using JS on the other end are
> going to go use a different library. I suggest the former is more acceptable
> than the latter for a stdlib offering. Allowing the caller of parse() to
> choose would be even better.


I don't understand what you mean, here. generate ([decimal.Decimal ('1.1')])
-> '[1.1]', so a JavaScript user calling eval() on it would get a standard
JavaScript float object without having to call float() explicitly.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Robert Brewer]

John Millikin wrote:
> I've written a rough draft of a PEP for standard library inclusion,
> attached to this email. Comments/improvements welcome - I tried to
> leave most of the differences between modules in the "Issues" section.

Re: Representation of Fractional Numbers, there are two solutions. If you 
return decimals, people using JS on the other end are going to call float(d). 
If you return floats, people not using JS on the other end are going to go use 
a different library. I suggest the former is more acceptable than the latter 
for a stdlib offering. Allowing the caller of parse() to choose would be even 
better.


Robert Brewer
[EMAIL PROTECTED]

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

I've written a rough draft of a PEP for standard library inclusion, attached
to this email. Comments/improvements welcome - I tried to leave most of the
differences between modules in the "Issues" section.
PEP: XXX
Title: A JSON handling library
Version: $Revision$
Last-Modified: $Date$
Author: John Millikin <[EMAIL PROTECTED]>
Discussions-To: web-sig@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 05-Apr-2008
Python-Version: 2.6
Post-History: XXX


Abstract


This PEP describes a proposed library for parsing and generating
data in the `JSON` [1]_ format. JSON stands for "JavaScript Object
Notation", and is described by RFC 4627 [2]_.

Rationale
=

JSON is a widely-used data interchange format, often used for sending
data to and from a web browser using Javascript. Its simplicity and
ease of use has lead to various implementations with varying degrees
of compliance to the RFC. By bundling a capable implementation in
Python's standard library, I hope to reduce or eliminate the need
for choosing a JSON library.

Existing Public libraries
=

* Bob Ippolito's simplejson [3]_
* Deron Meranda's demjson [4]_
* John Millikin's jsonlib [5]_
* Alan Kennedy mentioned on web-sig [6]_ that he has written
  an implementation for Jython, but I couldn't find source code for
  it.

Each of these have different APIs, different degrees of strictness,
and different qualities of error handling.

Module Interface


Parsing
---

Encoding Autodetection
''

The RFC requires that JSON is encoded in one of the Unicode encodings.
Because the first two bytes in a valid JSON expression are always from
the ASCII set, it is possible to reliably determine the encoding of
input data. Functions for autodetecting encoding exist in jsonlib and
demjson.

Parsing API
'''

A JSON expression may be parsed using the ``parse`` function::

  parse (bytes_or_string)

If the input is a ``bytes`` object, the encoding should be auto-detected
as above. If input has been recieved in a non-standard encoding, it can
be manually decoded and passed to ``parse`` as a string. The return
value is either a sequence or mapping, depending on the input.

Serialization
-

Python objects may be serialized using the ``generate`` function::

  generate (obj, indent = None, ascii_only = True, encoding = 'utf-8')
  
``indent`` is used to control pretty-printing. If ``None``, no pretty
printing will be performed and the output will be maximally compact.
If ``indent`` is a string, that string will be used for indenting
nested values. The only values allowed in ``indent`` are those that
are valid JSON whitespace; these are U+0009, U+000A, U+000D, and U+0020.

``ascii_only`` controls whether the output may contain characters above
the ASCII set. If ``True``, all non-ASCII characters must be escaped
using \\u syntax. Otherwise, non-ASCII characters will be included
without escaping. Depending on the output encoding and values of the
characters, this might be more size-efficient.

``encoding`` specifies how the output is to be encoded. If ``None``,
the output will be a Unicode string. By default, JSON is encoded in
UTF-8.

Note: this is the set of options generally supported by implementations.
For a full treatment of other options, see `Options for Serialization`_.

Other
-

XXX Should the encoding autodetection function be a part of the
public API?

Issues
==

Representation of Fractional Numbers


The author of jsonlib feels that fractional numbers should be parsed
into an instance of ``decimal.Decimal``, to avoid issues with values
that cannot be represented exactly by the ``float`` type
[7]_.

  The spec does not require a decimal, but I dislike losing information
  in the parsing stage. Any implementation in the standard library
  should, in my opinion, at least offer a parameter for lossless parsing
  of number values.

The author of simplejson disagrees [8]_, saying that:

  Practically speaking I've tried using decimal instead of float for
  JSON and it's generally The Wrong Thing To Do. The spec doesn't say
  what to do about numbers, but for proper JavaScript interaction you
  want to do things that approximate what JS is going to do: 64-bit
  floating point.

demjson appears to have some sort of float precision detection
mechanism, and returns instances of ``float`` only if they can
represent a value exactly.

Serializing User-defined Types
--

There should be some way for a user to specify how types not known
to the JSON library should be serialized. For example, django
needs to serialize types related to date and time.

* simplejson supports a ``default`` parameter to ``dump`` and
  ``dumps``, which should be a callable that accepts a value and
  returns a serializable object.
* demjson supports a ``json_equivalent`` method of objects to
  encode, or users ma

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Deron Meranda]

Bob and John, I downloaded the new versions of simplejson (1.8.1)
and jsonlib (1.2.7).  I will re-run tests and update my report
at http://deron.meranda.us/python/comparing_json_modules/
over the next couple days.  I went ahead and put a note on
there that new versions are available; until I get it all updated.

It's nice to see things tightening up.


On Sun, Mar 23, 2008 at 5:49 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  ... I think it's [simplejson] more or less ready to go into the stdlib
>  at this point.

Your code is certainly looking pretty strong now and I'm glad
you're willing to donate it.  I believe there may yet be some minor
changes we'd need to make before Guido and company would
be willing to accept it though.

Some of my suggestions for it to go into Python are mainly
just to really clean up the interface.  Some of these might
break exact API compatibility with simplejson, but if you're
donating this (fork of?) to Python then its the time to do this
before the official API gets blessed.

Rename module to just "json".  (This would obviously be
done when/if it goes into Python stdlib.)

Change how the separators parameter works.  I don't
like having the user having to pass in the punctuation
characters (when would those ever be different), or
other arbitrary characters.  Maybe something safer like
space_after_punctuation = (None | bool | int), similar to
how you did the indent parameter.

dump and load?  My preference is to not use the dumps
and loads function names (borrowed from the pickle module?).
I still don't think of JSON data streams as being strings;
and want to avoid confusion with pickle too.

What about documentation?  The docstrings are nice,
but I don't know what Guido needs in terms of other
module documentation to go in the manual.

There's probably some other minor things too.

Lastly, I'm still not completely sold that this, or any of the
other existing modules (mine included), is what we really
want in the stdlib.  There are at least three really good modules
now; but I have this feeling that an even better one could
be made (I'm talking about the user/caller interface, not
necesarily the internal implementation).  I'm playing with some
ideas, and may have some thoughts to share shortly.
-- 
Deron Meranda
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Robert Brewer]

Bob Ippolito wrote: 
> I chose to only support the basic types out of the box, but you can
> specialize the decoder and encoder any way you want, e.g. to provide a
> JSON serialization scheme where you get a deque out if you put one in.
> I can imagine scenarios where you would want to encode decimal as a
> string for example, because the other end is probably going to parse
> JSON numbers as doubles.

Thanks for allowing that; the last 3 JSON projects I've been involved
with have had Python on both ends, and always passed Decimal, never
floating-point.


Robert Brewer
[EMAIL PROTECTED]

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Sun, Mar 23, 2008 at 4:11 PM, John Millikin <[EMAIL PROTECTED]> wrote:
> I've added a few issues to your tracker, regarding encoding
>  autodetection, support for serializing stdlib classes like
>  collections.deque and decimal.Decimal, and a unittest test suite.
>  There don't seem to be any downloads of simplejson available at
>  .

There are no downloads on google code because the downloads are on
PyPI (and there's a link to that on the front page of the google code
project). There's little reason for me to bother uploading it to
google unless setuptools makes it easy like it does for PyPI.

There is a unittest-like suite, run python setup.py test or run
nosetest directly.

I chose to only support the basic types out of the box, but you can
specialize the decoder and encoder any way you want, e.g. to provide a
JSON serialization scheme where you get a deque out if you put one in.
I can imagine scenarios where you would want to encode decimal as a
string for example, because the other end is probably going to parse
JSON numbers as doubles.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

I've added a few issues to your tracker, regarding encoding
autodetection, support for serializing stdlib classes like
collections.deque and decimal.Decimal, and a unittest test suite.
There don't seem to be any downloads of simplejson available at
.

Mr. Meranda: a new version of jsonlib is released that fixes most of
the issues found in your comparison. It also has much nicer error
messages, and supports Python version 2.4.

On Sun, Mar 23, 2008 at 4:49 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  Ok, so I made these changes (parse_float/parse_int/parse_constant) and
>  a few others for simplejson 1.8.1 and moved it to google code.
>
>  http://code.google.com/p/simplejson/
>
>  Other changes:
>
>   * No longer escapes / by default, if you're embedding in HTML then
>  you'll have to escape that yourself, I got really tired of looking at
>  URLs with all of the /s escaped.
>   * Optional scanstring C speedup for decoding
>   * correct unicode surrogate pair decoding
>   * can be used from the command-line now to validate and pretty-print
>  JSON ("curl http://json/ | python -msimplejson")
>   * bug fix for ensure_ascii=False decoding
>
>  If anyone else has any complaints, bug reports, or feature requests
>  they'd like to address they should speak up soon either here or on the
>  issue tracker. I think it's more or less ready to go into the stdlib
>  at this point.
>
>  -bob
>
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Thu, Mar 20, 2008 at 8:38 PM, Robert Brewer <[EMAIL PROTECTED]> wrote:
>
> Bob Ippolito wrote:
>  > On Thu, Mar 20, 2008 at 5:50 PM, Robert Brewer <[EMAIL PROTECTED]>
>  > wrote:
>  > > Deron Meranda wrote:
>  > > > And even then, we're not just talking about a JSON parser.
>  > > > We're all doing more than that; we're mapping Python to JSON.
>  > > > And there is no definitive spec for that.  Just look at my
>  > > > numbers tests; there are a lot of differences in how numeric
>  > > > mappings are done, but yet many of them can be arguably
>  > > > "correct" while still doing things differently.
>  > >
>  > >  ...which IMO argues that any json implementation that goes
>  > > in the stdlib needs to at least allow access to the raw bytes
>  > > in both directions. For example, if you really want JSON
>  > > numerals to become Python decimals, you shouldn't be forced
>  > > to lose information just because the json decoder was only
>  > > designed to hand you a float. Arbitrary converter plugins would
>  > > be icing on the cake. A built in decimal converter would be
>  > > heaven. :)
>  >
>  > That can be easily done, but at the expense of speed or clarity in the
>  > implementation... I'd be willing to add some hooks to simplejson that
>  > allow people to pass in their own functions that turn JSON terms (as
>  > strings) into Python objects.
>
>  That'd be great! I expect a speed penalty of course, and IMO most of that 
> should be pushed onto anyone passing in functions, rather than making 
> everyone pay.

Ok, so I made these changes (parse_float/parse_int/parse_constant) and
a few others for simplejson 1.8.1 and moved it to google code.

http://code.google.com/p/simplejson/

Other changes:

 * No longer escapes / by default, if you're embedding in HTML then
you'll have to escape that yourself, I got really tired of looking at
URLs with all of the /s escaped.
 * Optional scanstring C speedup for decoding
 * correct unicode surrogate pair decoding
 * can be used from the command-line now to validate and pretty-print
JSON ("curl http://json/ | python -msimplejson")
 * bug fix for ensure_ascii=False decoding

If anyone else has any complaints, bug reports, or feature requests
they'd like to address they should speak up soon either here or on the
issue tracker. I think it's more or less ready to go into the stdlib
at this point.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Robert Brewer]

Bob Ippolito wrote:
> On Thu, Mar 20, 2008 at 5:50 PM, Robert Brewer <[EMAIL PROTECTED]>
> wrote:
> > Deron Meranda wrote:
> > > And even then, we're not just talking about a JSON parser.
> > > We're all doing more than that; we're mapping Python to JSON.
> > > And there is no definitive spec for that.  Just look at my
> > > numbers tests; there are a lot of differences in how numeric
> > > mappings are done, but yet many of them can be arguably
> > > "correct" while still doing things differently.
> >
> >  ...which IMO argues that any json implementation that goes
> > in the stdlib needs to at least allow access to the raw bytes
> > in both directions. For example, if you really want JSON
> > numerals to become Python decimals, you shouldn't be forced
> > to lose information just because the json decoder was only
> > designed to hand you a float. Arbitrary converter plugins would
> > be icing on the cake. A built in decimal converter would be
> > heaven. :)
> 
> That can be easily done, but at the expense of speed or clarity in the
> implementation... I'd be willing to add some hooks to simplejson that
> allow people to pass in their own functions that turn JSON terms (as
> strings) into Python objects.

That'd be great! I expect a speed penalty of course, and IMO most of that 
should be pushed onto anyone passing in functions, rather than making everyone 
pay.


Robert Brewer
[EMAIL PROTECTED]

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Thu, Mar 20, 2008 at 5:50 PM, Robert Brewer <[EMAIL PROTECTED]> wrote:
> Deron Meranda wrote:
>  > And even then, we're not just talking about a JSON parser.  We're all
>  > doing more than that; we're mapping Python to JSON.  And there is
>  > no definitive spec for that.  Just look at my numbers tests; there are
>  > a lot of differences in how numeric mappings are done, but yet many
>  > of them can be arguably "correct" while still doing things
>  differently.
>
>  ...which IMO argues that any json implementation that goes in the stdlib
>  needs to at least allow access to the raw bytes in both directions. For
>  example, if you really want JSON numerals to become Python decimals, you
>  shouldn't be forced to lose information just because the json decoder
>  was only designed to hand you a float. Arbitrary converter plugins would
>  be icing on the cake. A built in decimal converter would be heaven. :)

That can be easily done, but at the expense of speed or clarity in the
implementation... I'd be willing to add some hooks to simplejson that
allow people to pass in their own functions that turn JSON terms (as
strings) into Python objects.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Robert Brewer]

Deron Meranda wrote:
> And even then, we're not just talking about a JSON parser.  We're all
> doing more than that; we're mapping Python to JSON.  And there is
> no definitive spec for that.  Just look at my numbers tests; there are
> a lot of differences in how numeric mappings are done, but yet many
> of them can be arguably "correct" while still doing things
differently.

...which IMO argues that any json implementation that goes in the stdlib
needs to at least allow access to the raw bytes in both directions. For
example, if you really want JSON numerals to become Python decimals, you
shouldn't be forced to lose information just because the json decoder
was only designed to hand you a float. Arbitrary converter plugins would
be icing on the cake. A built in decimal converter would be heaven. :)


Robert Brewer
[EMAIL PROTECTED]

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Deron Meranda]

On Thu, Mar 20, 2008 at 6:48 PM, John Millikin <[EMAIL PROTECTED]> wrote:
>  This is fantastic. My knowledge of other JSON modules was based mainly
>  on the comparison page from json.org, and yours is much more complete
>  and informative.

Well that's understandable.  What's on json.org is great to give somebody
an easy to understand overview (ever looked at YAML?); but it wasn't until
Douglas wrote the RFC that there was anything detailed enough to base
an implementation upon without a lot of guesswork.  And I still had to
poke around in the ECMAScript, IEEE 754, and Unicode standards to
fill in some gaps.  And JSON is the *easy* format!

And even then, we're not just talking about a JSON parser.  We're all
doing more than that; we're mapping Python to JSON.  And there is
no definitive spec for that.  Just look at my numbers tests; there are
a lot of differences in how numeric mappings are done, but yet many
of them can be arguably "correct" while still doing things differently.

>  You could try adding a section to the numbers area about
>  Arabic/Chinese/whatever numbers, such as U+0661. These are not allowed
>  in JSON, but are accepted by parsers that use \d regex patterns with
>  the re.UNICODE flag set.

Good call.  I forgot all about that possibility, probably because
I'm not using regexes.

>  For strings, I would like to suggest that escaping "/" to "\\/" be
>  considered the norm, with deviations from this marked on the table.
>  This is to protect against foolish website authors including JSON
>  directly using a 

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Deron Meranda]

On Thu, Mar 20, 2008 at 7:50 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  >  Anyway, to get JSON in UTF-8, I'm calling it like this:
>  >
>  >   simplejson.dumps( ["\x1a"], ensure_ascii=False ).encode('utf8')
>  >
>  >  which on my system outputs this:   '["\x1a"]'
>  >  rather than this:  '["\\u001a"]'
>  >
>  >  If I change the ensure_ascii to its default of True, then I do get
>  >  the correct results.
>
>  Okay, so I was using the default settings, which is ensure_ascii=True,
>  and I was getting correct results. I didn't know you were testing with
>  ensure_ascii=False.

I almost missed that one myself too.  That's why I finally decided to
get so detailed, to find little things like that.  I also found a few bugs
in mine as well by doing this exercise (although I had the advantage
of fixing mine before anybody could see :)

I assume that will be a really easy fix for you.  But, I am using it the
right way though, aren't I?
-- 
Deron Meranda
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Thu, Mar 20, 2008 at 3:38 PM, Deron Meranda <[EMAIL PROTECTED]> wrote:
> On Thu, Mar 20, 2008 at 6:03 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
>  > On Thu, Mar 20, 2008 at 1:37 PM, Deron Meranda <[EMAIL PROTECTED]> wrote:
>
> >  >  I finally sat down and put the five or so top JSON libraries
>  >  >  to the test so we can all see what's what.  I've put everything
>  >  >  in a report here:
>  >  >
>  >  >  http://deron.meranda.us/python/comparing_json_modules/
>
>
> >  This is very cool, but it's not reproducible. If you published the
>  >  code that you used then it would help everyone out considerably.
>
>  Yes, I will try to do that.  I just need to clean it up a bit so it's
>  a bit less manual.
>
>
>  > For example, with simplejson 1.7.5 I'm unable to reproduce the problems
>  >  you're experiencing with
>  >  http://deron.meranda.us/python/comparing_json_modules/strings in Table
>  >  1 (other than UserString of course).
>
>  Do you mean tests 1-12 and 1-13 where simepljson is seemingly
>  not \u escaping U+001A through U+001F ?
>
>  I think my table might be confusing.  In those sets of tests I'm looking at
>  the JSON after its been converted into UTF-8 (because while it is
>  sitting in memory as a "unicode" string, its not technically JSON).
>  I talk about that on the next page (Unicode), but that's not the same
>  order you read things in...my fault; I'll try to rewrite that a bit so its
>  clearer.
>
>  Anyway, to get JSON in UTF-8, I'm calling it like this:
>
>   simplejson.dumps( ["\x1a"], ensure_ascii=False ).encode('utf8')
>
>  which on my system outputs this:   '["\x1a"]'
>  rather than this:  '["\\u001a"]'
>
>  If I change the ensure_ascii to its default of True, then I do get
>  the correct results.
>
>  Interestingly it works correctly either way for all characters
>  at or less than U+0019.
>
>  So is it just the way I'm calling it (perhaps not as intended),
>  or do we still see a difference between your system and mine?
>
>
>  I'll try to get some test framework code up within the next day.

Okay, so I was using the default settings, which is ensure_ascii=True,
and I was getting correct results. I didn't know you were testing with
ensure_ascii=False.

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

On Thu, Mar 20, 2008 at 12:37 PM, Deron Meranda <[EMAIL PROTECTED]> wrote:
>  I finally sat down and put the five or so top JSON libraries
>  to the test so we can all see what's what.  I've put everything
>  in a report here:
>
>  http://deron.meranda.us/python/comparing_json_modules/
>
This is fantastic. My knowledge of other JSON modules was based mainly
on the comparison page from json.org, and yours is much more complete
and informative.

You could try adding a section to the numbers area about
Arabic/Chinese/whatever numbers, such as U+0661. These are not allowed
in JSON, but are accepted by parsers that use \d regex patterns with
the re.UNICODE flag set.

For strings, I would like to suggest that escaping "/" to "\\/" be
considered the norm, with deviations from this marked on the table.
This is to protect against foolish website authors including JSON
directly using a 

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Deron Meranda]

On Thu, Mar 20, 2008 at 6:03 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
> On Thu, Mar 20, 2008 at 1:37 PM, Deron Meranda <[EMAIL PROTECTED]> wrote:
>  >  I finally sat down and put the five or so top JSON libraries
>  >  to the test so we can all see what's what.  I've put everything
>  >  in a report here:
>  >
>  >  http://deron.meranda.us/python/comparing_json_modules/

>  This is very cool, but it's not reproducible. If you published the
>  code that you used then it would help everyone out considerably.

Yes, I will try to do that.  I just need to clean it up a bit so it's
a bit less manual.

> For example, with simplejson 1.7.5 I'm unable to reproduce the problems
>  you're experiencing with
>  http://deron.meranda.us/python/comparing_json_modules/strings in Table
>  1 (other than UserString of course).

Do you mean tests 1-12 and 1-13 where simepljson is seemingly
not \u escaping U+001A through U+001F ?

I think my table might be confusing.  In those sets of tests I'm looking at
the JSON after its been converted into UTF-8 (because while it is
sitting in memory as a "unicode" string, its not technically JSON).
I talk about that on the next page (Unicode), but that's not the same
order you read things in...my fault; I'll try to rewrite that a bit so its
clearer.

Anyway, to get JSON in UTF-8, I'm calling it like this:

  simplejson.dumps( ["\x1a"], ensure_ascii=False ).encode('utf8')

which on my system outputs this:   '["\x1a"]'
rather than this:  '["\\u001a"]'

If I change the ensure_ascii to its default of True, then I do get
the correct results.

Interestingly it works correctly either way for all characters
at or less than U+0019.

So is it just the way I'm calling it (perhaps not as intended),
or do we still see a difference between your system and mine?


I'll try to get some test framework code up within the next day.
-- 
Deron Meranda
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Thu, Mar 20, 2008 at 1:37 PM, Deron Meranda <[EMAIL PROTECTED]> wrote:
> On Sun, Mar 16, 2008 at 7:18 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>  >  I'm reading Alan's post as saying that he has a competing implementation.
>
>  Yes, there are several JSON implementation now, some
>  better than others.
>
>  I finally sat down and put the five or so top JSON libraries
>  to the test so we can all see what's what.  I've put everything
>  in a report here:
>
>  http://deron.meranda.us/python/comparing_json_modules/
>
>  I have tried to be very rigorous.  There's probably mistakes
>  in there, so let me know if anybody finds any.  Also if any
>  of you module authors update your code and want me to
>  re-do the tests against a newer version let me know.

This is very cool, but it's not reproducible. If you published the
code that you used then it would help everyone out considerably. For
example, with simplejson 1.7.5 I'm unable to reproduce the problems
you're experiencing with
http://deron.meranda.us/python/comparing_json_modules/strings in Table
1 (other than UserString of course). I've tried this both on 32-bit
UCS-2 python (OS X) as well as 64-bit UCS4 python (x86-64 linux) and I
get correct output. I'm still looking through the rest of the tests,
so that might not be the only supposed problem that I'm unable to
reproduce.

>  I do think though that if this is targeted for Python 3, that
>  none of the modules really works well.  We should really
>  design an interface that uses the bytes type rather
>  than str for pushing around encoded JSON data.

I can certainly agree with that. The scope of the changes wouldn't be
that big though, at least for simplejson's default behavior (escape
everything to ASCII).

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Deron Meranda]

On Sun, Mar 16, 2008 at 7:18 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>  I'm reading Alan's post as saying that he has a competing implementation.

Yes, there are several JSON implementation now, some
better than others.

I finally sat down and put the five or so top JSON libraries
to the test so we can all see what's what.  I've put everything
in a report here:

http://deron.meranda.us/python/comparing_json_modules/

I have tried to be very rigorous.  There's probably mistakes
in there, so let me know if anybody finds any.  Also if any
of you module authors update your code and want me to
re-do the tests against a newer version let me know.

I do have to say I'm glad to see that many of the
implementations have been getting much much better since
I first checked out the scene a year ago.

Yes, my module is among those, but I don't particularly
care who's we use (or derive from) for inclusion in Python,
as long as we can clean up any warts or issues with
non-conformance.  Seeing all these compared in one place
might give all of us ideas for a better approach before
anything becomes an official Python component. I've
certainly learned a few things I could do better with mine
by looking at everybody else's.

I do think though that if this is targeted for Python 3, that
none of the modules really works well.  We should really
design an interface that uses the bytes type rather
than str for pushing around encoded JSON data.


>  But I believe that both approaches Alan offers are overkill: he
>  proposes something like the db-API, where there are multiple
>  implementations of the same API (except in the case of the db-API they
>  aren't necessarily 100% interchangeable).

I agree, let's not go down the db-API path!  It's full of mud.

-- 
Deron Meranda
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[John Millikin]

I am the author of jsonlib. Apologies for not replying to this thread
earlier; I had not realized that JSON was considered a web-side
technology, as I use it primarily for serialization of simple data
sets between Python, Java, and C. First, I would like to state that my
purpose for writing jsonlib was not to become the maintainer of yet
another (de)serializer, but merely to solve my immediate problem of no
JSON libraries that handle Unicode correctly. Any solution that
results in a standard JSON library I fully support. At the worst, if
Python decides to go down the route of the super-forgiving parsers I
will simply maintain jsonlib as a separate library.

> I think the author could very well be incorrect in believing that the
> spec requires using a decimal
>
The spec does not require a decimal, but I dislike losing information
in the parsing stage. Any implementation in the standard library
should, in my opinion, at least offer a parameter for lossless parsing
of number values.

> On the encoding side, I simply make the assumption that all character
> transcoding has happened before the JSON text reaches the JSON parser.
> (I think this is a reasonable assumption, given that byte streams are
> always associated with file storage, network transmission, etc, and
> only the programmer has access to the relevant encoding information).
> But given that RFC 4627 specifies how to guess encoding of JSON byte
> streams, I'll probably change that policy.
>
jsonlib has an encoding autodetection feature. It's not particularly
hard, except that Python 2.5 (the version I use) does not have a codec
for UTF-32, so I had to hand-roll one using the struct module.

Finally, as regards APIs, I very much like the simplejson API and have
attempted to model the options of jsonlib's write() function after
simplejson's dumps. My reason for using read() and write() is due to
python-json, which is the first JSON library I used.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Guido van Rossum]

On Sun, Mar 16, 2008 at 5:52 PM, Bob Ippolito <[EMAIL PROTECTED]> wrote:
> On Sun, Mar 16, 2008 at 5:35 PM, Alan Kennedy <[EMAIL PROTECTED]> wrote:
>  > [Deron]
>  >  > (I just joined this list, so this reponse may not be threaded properly)
>  >
>  >  [Bob]
>  >
>  > > I wasn't subscribed to the list at the time this came up, but I'm all
>  >  > for getting simplejson into the stdlib.
>  >
>  >  Well, it appears we have a quorum of JSON<->python codec writers,
>  >  since I've written a jython module that I'd like to interoperate with
>  >  cpython codecs. I think it's appropriate for any discussions of JSON
>  >  to take place on the web-sig.
>  >
>  >  I've been thinking about how to take this forward. I see two ways
>
>  Are there *really* competing implementations? I mean, it seems that
>  pretty much everyone uses simplejson, at least in the web framework
>  world. If someone writes a test for BMP, we'll fix it. As far as byte
>  encoding detection I think that's beyond the scope of a JSON
>  implementation and I think it's unnecessary in the first place.

I'm reading Alan's post as saying that he has a competing implementation.

But I believe that both approaches Alan offers are overkill: he
proposes something like the db-API, where there are multiple
implementations of the same API (except in the case of the db-API they
aren't necessarily 100% interchangeable). It'm sorry for Alan, but I'd
much rather pick one implementation (simplejson), fix 1-2 minor issues
with it, and put it into the standard library. Anything else is just a
lot of opportunity for endless debate without much benefit.

>  As far as Jython support goes, I suppose that's probably fixable
>  without too much effort. I would imagine that the problems are just in
>  the decoder, because of the sre_* module (ab)use. Was there some other
>  reason for writing a Jython-specific codec?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

On Sun, Mar 16, 2008 at 5:35 PM, Alan Kennedy <[EMAIL PROTECTED]> wrote:
> [Deron]
>  > (I just joined this list, so this reponse may not be threaded properly)
>
>  [Bob]
>
> > I wasn't subscribed to the list at the time this came up, but I'm all
>  > for getting simplejson into the stdlib.
>
>  Well, it appears we have a quorum of JSON<->python codec writers,
>  since I've written a jython module that I'd like to interoperate with
>  cpython codecs. I think it's appropriate for any discussions of JSON
>  to take place on the web-sig.
>
>  I've been thinking about how to take this forward. I see two ways

Are there *really* competing implementations? I mean, it seems that
pretty much everyone uses simplejson, at least in the web framework
world. If someone writes a test for BMP, we'll fix it. As far as byte
encoding detection I think that's beyond the scope of a JSON
implementation and I think it's unnecessary in the first place.

As far as Jython support goes, I suppose that's probably fixable
without too much effort. I would imagine that the problems are just in
the decoder, because of the sre_* module (ab)use. Was there some other
reason for writing a Jython-specific codec?

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[Deron]
> (I just joined this list, so this reponse may not be threaded properly)

[Bob]
> I wasn't subscribed to the list at the time this came up, but I'm all
> for getting simplejson into the stdlib.

Well, it appears we have a quorum of JSON<->python codec writers,
since I've written a jython module that I'd like to interoperate with
cpython codecs. I think it's appropriate for any discussions of JSON
to take place on the web-sig.

I've been thinking about how to take this forward. I see two ways

Formal approach


Introduce a "Standards Track" Library PEP, which is designed for the
purpose of bringing a new module through a full peer-review process
and into the python standard library. (Which means we in jython and
ironpython land should also then provide it). This would have the
following outcomes

 - Result in a single JSON implementation going into the cpython
standard library, possibly in Python 3000
 - Expose the new module to full community review/bug-tracking/modification
 - Opportunity to thrash out all of the finer points of JSON<->python
transcoding, including but not limited to
  - NaN, Infinity, etc
  - What is the most appropriate number/integer/float/double/decimal
representation
  - Structural strictness, e.g. junk after document body, dangling commas, etc.
  - BMP support
  - Byte encoding detection
  - Python 3000 support
 - Standardise the interface, de facto

However, this option is somewhat complicated by the fact that we seem
to have TWO quality cpython implementations competing for a place in
the cpython standard library. Also, I think the PEP process might be a
little cumbersome for this topic, given that the PEP process involves
commit rights to the cpython source tree (since the proposal for a new
module should be accompanied by the source code of the proposed
implementation).

Informal approach
=

Develop and document a standard interface, and ensure that all of our
modules support it. This interface would define method, class and
exception names. Standard methods would probably "load" and "dump"
objects, possibly creating "JSONEncoder"s and "JSONDecoder"s to do the
job: "JSONException" and subclasses thereof would signify errors.
Perhaps a standard mechanism to retrieve the location of errors, e.g.
line and column, would be appropriate? Perhaps a standard set of
feature/option names could be agreed, e.g. "accept_NaN", etc.

User code written to this standard could move reasonably easily
between implementations, or indeed between platforms. This approach
has the benefits that

 - Authors are free to interpret edge cases as they see fit, and
provide options.
 - Competing implementations can continue to improve in the field
 - Changing implementations could be as simple as using a different egg
   (Although an exhaustive set of test cases covering the required
behaviour is recommended)

We could call it PAJ, Python Api for Json, or some such.

I feel the informal option is more appropriate. It could be
effectively managed on a wiki page. Or perhaps a ticketing system
(e.g. TRAC) would be good for tracking detailed discussions of JSON's
many edge cases, etc. I would be willing to start a wiki page with
details about a putative module interface.

Finally, at this stage I think speed is less of a concern; correctness
is more important for now. As Aahz is fond of quoting, "It is easier
to optimize correct code than to correct optimized code".

Thoughts?

Alan.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bob Ippolito]

I wasn't subscribed to the list at the time this came up, but I'm all
for getting simplejson into the stdlib. We use it a lot here at Mochi
Media and we're willing to support it in or out of the stdlib. I'm
willing to sign over copyright to the PSF and/or relicense and help
with whatever else needs to happen, just let me know.

As far as the reading of non-BMP unicode goes, give me a test and
we'll fix it. So far nobody has sent any bug reports related to
unicode, so I'm willing to wager that nobody actually cares, but I'd
still like to do it correctly. I haven't tried very hard because I use
a UCS2 build of Python on my workstation and I don't know enough about
unicode edge cases to craft a proper suite.

Practically speaking I've tried using decimal instead of float for
JSON and it's generally The Wrong Thing To Do. The spec doesn't say
what to do about numbers, but for proper JavaScript interaction you
want to do things that approximate what JS is going to do: 64-bit
floating point. Encoding floats with repr might be the wrong thing to
do because the representation is long, but ideally repr in Python
would use a better algorithm[1]. Changing from repr to str is trivial
and beneficial so I've already put that on the trunk.

[1] http://www.cs.indiana.edu/~burger/fp/index.html

-bob
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[Massimo]
> It would also be nice to have a common interface to all modules that
> do serialization. For example pickle, cPickle, marshall has dumps, so
> json should also have dumps.

Indeed, this is my primary concern also.

The reason is that I have a pure-java JSON codec for jython, that I
will either publish separately or contribute to jython itself.

If we're going to have the facility in both cpython and jython (and
probably ironpython, etc), then it would be optimal to have a
compatible API so that we have full interoperability. And given that
we in jython land are always left implementing cpython APIs (which are
not necessarily always the optimal design for jython) it would be nice
if we could agree on APIs, etc, *before* stuff goes into the standard
library.

The API for my codec is slightly different from simplejson, although
it could be made the same with a little work, including exception
signatures, etc.

But there are some things about my own design that I like. For
example, simplejson allows override of the JSON output representing
certain objects, by the use of subclasses of JSONEncoder. My design
does it differently; it simply looks for a "__json__()" callable on
every object being serialised, and if found, calls it and uses its
return value to represent the object. I have no equivalent of
simplejson's decoding extensions.

Another difference is the set of options. Simplejson has options to
control parsing and generation, and so does mine. But the sets of
options are different, e.g. simplejson has no option to permit/reject
dangling commas (e.g. "[1,2,3,]")*, whereas mine has no support for
accepting NaN, infinity, etc, etc.

On the encoding side, I simply make the assumption that all character
transcoding has happened before the JSON text reaches the JSON parser.
(I think this is a reasonable assumption, given that byte streams are
always associated with file storage, network transmission, etc, and
only the programmer has access to the relevant encoding information).
But given that RFC 4627 specifies how to guess encoding of JSON byte
streams, I'll probably change that policy.

Lastly, another area of potential cooperation is testing: I have over
100 unit-tests, with fairly extensive coverage. I think that test
coverage is very important in the case of JSON; you can never have too
many tests.

So, what is the best way to go about agreeing on the best API?

1. Discussion on web-sig?
2. Discussion on stdlib-sig?
3. Collaborative authoring/discussion on a WIKI page?
4. 

Regards,

Alan.

* Which can mean different things to different software. Some
javascript interpreters interpret it as a 4 element list (inferring
the last object between the comma and the closing square bracket as a
null) , others as a 3 element list. Python obviously interprets it as
a 3-element list. So the general internet maxim "be liberal in what
you accept and strict in what produce" applies. My API gives control
of this strictness/relaxedness to the user.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Graham Dumpleton]

On 11/03/2008, Alan Kennedy <[EMAIL PROTECTED]> wrote:
> [Graham]
>
> >  The problem areas were, different interpretations of what could be
>  >  supplied in an error response. Whether an integer, string or arbitrary
>  >  object could be supplied as the id attribute in a request. Finally,
>  >  some JavaScript clients would only work with a server side
>  >  implementation which provided introspection methods as they would
>  >  dynamically create a JavaScript proxy object based on a call of the
>  >  introspection methods.
>
>
> These are JSON-RPC concerns, and nothing to do with JSON text 
> de/serialization.
>
>  I do believe we're only discussing JSON<->python objects
>  transformation, in this thread at least.

Okay. No problem then. :-)

Graham

>  >  Unfortunately the JSON 1.1 draft specification didn't necessarily make
>  >  things better.
>
>
> There is no JSON 1.1 spec; but there is a JSON-RPC 1.1 spec.
>
>  http://json-rpc.org/wiki/specification
>
>
>  >  Thus my question is, what version of the JSON specification are you
>  >  intending to support.
>
>
> The one specified in RFC 4627
>
>  http://www.ietf.org/rfc/rfc4627.txt
>
>  Regards,
>
>
>  Alan.
>
> ___
>  Web-SIG mailing list
>  Web-SIG@python.org
>  Web SIG: http://www.python.org/sigs/web-sig
>  Unsubscribe: 
> http://mail.python.org/mailman/options/web-sig/graham.dumpleton%40gmail.com
>
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Alan Kennedy]

[Graham]
>  The problem areas were, different interpretations of what could be
>  supplied in an error response. Whether an integer, string or arbitrary
>  object could be supplied as the id attribute in a request. Finally,
>  some JavaScript clients would only work with a server side
>  implementation which provided introspection methods as they would
>  dynamically create a JavaScript proxy object based on a call of the
>  introspection methods.

These are JSON-RPC concerns, and nothing to do with JSON text de/serialization.

I do believe we're only discussing JSON<->python objects
transformation, in this thread at least.

>  Unfortunately the JSON 1.1 draft specification didn't necessarily make
>  things better.

There is no JSON 1.1 spec; but there is a JSON-RPC 1.1 spec.

http://json-rpc.org/wiki/specification

>  Thus my question is, what version of the JSON specification are you
>  intending to support.

The one specified in RFC 4627

http://www.ietf.org/rfc/rfc4627.txt

Regards,

Alan.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Titus Brown]

On Mon, Mar 10, 2008 at 11:55:04PM -0500, Massimo Di Pierro wrote:
-> It would also be nice to have a common interface to all modules that  
-> do serialization. For example pickle, cPickle, marshall has dumps, so  
-> json should also have dumps.

Doesn't it?  Or did you want something additional?

http://svn.red-bean.com/bob/simplejson/tags/simplejson-1.7/docs/index.html

--titus
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Massimo Di Pierro]

It would also be nice to have a common interface to all modules that  
do serialization. For example pickle, cPickle, marshall has dumps, so  
json should also have dumps.

Massimo

On Mar 10, 2008, at 9:51 PM, Mark Ramm wrote:

>>  Well, so fix this. How hard can it be?
>
> A google search for "jsonlib rfc"  turns up this article:
>
> http://jmillikin.blogspot.com/2008/02/python-json-catastrophe.html
>
> And it looks like the two issues with simplejson are:
>
> 1) decoding JSON with unicode code-points outside the Basic  
> Multilingual Plane
> 2) Decoding the json 1.1 as a python float rather than a decimal
>
> I think the author could very well be incorrect in believing that the
> spec requires using a decimal, so that just leaves the Unicode issue,
> which isn't very clearly defined in the article, but ought to be
> reasonably easy to fix.
>
> The spec isn't as clear as it could be about a lot of issues, but I've
> had no problems with simplejson and interoperability with the various
> javascript libraries.   I've also used simplejson with one or two of
> the ruby json libraries, flex's json lib, and a couple of libraries
> for other languages, with no particular issues.   That's not to say
> there are no bugs, but I don't think there are too many issues buried
> in there.
>
> --Mark
> ___
> Web-SIG mailing list
> Web-SIG@python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/ 
> mdipierro%40cti.depaul.edu

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Mark Ramm]

>  Well, so fix this. How hard can it be?

A google search for "jsonlib rfc"  turns up this article:

http://jmillikin.blogspot.com/2008/02/python-json-catastrophe.html

And it looks like the two issues with simplejson are:

1) decoding JSON with unicode code-points outside the Basic Multilingual Plane
2) Decoding the json 1.1 as a python float rather than a decimal

I think the author could very well be incorrect in believing that the
spec requires using a decimal, so that just leaves the Unicode issue,
which isn't very clearly defined in the article, but ought to be
reasonably easy to fix.

The spec isn't as clear as it could be about a lot of issues, but I've
had no problems with simplejson and interoperability with the various
javascript libraries.   I've also used simplejson with one or two of
the ruby json libraries, flex's json lib, and a couple of libraries
for other languages, with no particular issues.   That's not to say
there are no bugs, but I don't think there are too many issues buried
in there.

--Mark
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[James Bennett]

On Mon, Mar 10, 2008 at 9:32 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>  Well, so fix this. How hard can it be?

A bit of poking around turned up the post I was looking for:

http://jmillikin.blogspot.com/2008/02/python-json-catastrophe.html

Seems like his beef with simplejson is mostly Unicode/encoding
handling; the floating-point stuff is a bit more debatable wrt to the
spec, because rfc4627 doesn't say anything about how to handle these
aside from saying that a "number" in JSON is allowed to contain a
decimal point followed by more digits.

Since the post is only a couple weeks old, I'm assuming that the
Unicode stuff is current, so if the consensus is in favor of
simplejson I suppose that'd be the area to concentrate on.


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Graham Dumpleton]

On 11/03/2008, James Bennett <[EMAIL PROTECTED]> wrote:
> On Mon, Mar 10, 2008 at 8:37 AM, Mark Ramm <[EMAIL PROTECTED]> wrote:
>  >  I would definitely support the incusion of a JSON library in the
>  >  standard lib.   And, I think that it should be simplejson which is
>  >  used by TurboGears, Pylons, and bundled with Django.
>
>
> I'd tentatively agree, though I recall seeing a post not long ago
>  (which I am currently unable to find) from the author of jsonlib
>  lamenting the fact that most of the other JSON modules for Python had
>  various significant inconsistencies with the RFC. While authors of
>  competing tools should be taken with a grain of salt, I do think
>  compliance with the spec is an important factor for any particular
>  module that might be blessed with stdlib membership, and so should
>  play a bigger role in any such decision than mere benchmark speed.

The problem with the JSON 1.0 specification was that it wasn't always
as clear as could have been. As a result different server side
implementations interpreted it differently, as did the JavaScript
clients. I'll admit that it has been a while since I looked at it and
maybe things have improved, but certainly it used to be the case that
finding a JavaScript library that talked to a specific server side
implementation wasn't always easy. End result was that the JavaScript
library would often only work with the specific web framework it was
originally designed for and nothing else.

The problem areas were, different interpretations of what could be
supplied in an error response. Whether an integer, string or arbitrary
object could be supplied as the id attribute in a request. Finally,
some JavaScript clients would only work with a server side
implementation which provided introspection methods as they would
dynamically create a JavaScript proxy object based on a call of the
introspection methods.

Unfortunately the JSON 1.1 draft specification didn't necessarily make
things better. Rather than creating a proper layered specification
which separate lower level transport and encoding concerns from higher
level application concepts such as introspection they bundle it all
together. Thus they try to enforce that a server must support
introspection even though doing so may be totally impractical
depending on what the JSON server adapter is hooking in to. They also
introduced all this muck about having to support both positional and
named parameters at the same time.

As well as all that, I also recollect seeing some complaints about
servers handing character encoding wrongly. This may be the thing you
are talking about.

Thus my question is, what version of the JSON specification are you
intending to support.

Graham
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Guido van Rossum]

On Mon, Mar 10, 2008 at 7:23 PM, James Bennett <[EMAIL PROTECTED]> wrote:
> On Mon, Mar 10, 2008 at 8:37 AM, Mark Ramm <[EMAIL PROTECTED]> wrote:
>  >  I would definitely support the incusion of a JSON library in the
>  >  standard lib.   And, I think that it should be simplejson which is
>  >  used by TurboGears, Pylons, and bundled with Django.
>
>  I'd tentatively agree, though I recall seeing a post not long ago
>  (which I am currently unable to find) from the author of jsonlib
>  lamenting the fact that most of the other JSON modules for Python had
>  various significant inconsistencies with the RFC. While authors of
>  competing tools should be taken with a grain of salt, I do think
>  compliance with the spec is an important factor for any particular
>  module that might be blessed with stdlib membership, and so should
>  play a bigger role in any such decision than mere benchmark speed.

Well, so fix this. How hard can it be?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[James Bennett]

On Mon, Mar 10, 2008 at 8:37 AM, Mark Ramm <[EMAIL PROTECTED]> wrote:
>  I would definitely support the incusion of a JSON library in the
>  standard lib.   And, I think that it should be simplejson which is
>  used by TurboGears, Pylons, and bundled with Django.

I'd tentatively agree, though I recall seeing a post not long ago
(which I am currently unable to find) from the author of jsonlib
lamenting the fact that most of the other JSON modules for Python had
various significant inconsistencies with the RFC. While authors of
competing tools should be taken with a grain of salt, I do think
compliance with the spec is an important factor for any particular
module that might be blessed with stdlib membership, and so should
play a bigger role in any such decision than mere benchmark speed.


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Massimo Di Pierro]

I agree. simplejson is used by web2py as well.

Massimo

On Mar 10, 2008, at 8:37 AM, Mark Ramm wrote:

>>  Is it time there was a JSON codec included in the python standard  
>> library?
>
> I would definitely support the incusion of a JSON library in the
> standard lib.   And, I think that it should be simplejson which is
> used by TurboGears, Pylons, and bundled with Django.
>
>>  Choosing a Python JSON Translator
>>  http://blog.hill-street.net/?p=7
>
> This blog is a year old, and isn't quite accurate anymore.   in
> particular simplejson sprouted some optional C code, and is now a lot
> faster.
>
> --Mark Ramm
> ___
> Web-SIG mailing list
> Web-SIG@python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/ 
> mdipierro%40cti.depaul.edu

___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Guido van Rossum]

On Mon, Mar 10, 2008 at 9:08 AM, Jonathan Ellis
<[EMAIL PROTECTED]> wrote:
> On Mon, 10 Mar 2008 09:37:35 -0400, "Mark Ramm"
>  <[EMAIL PROTECTED]> said:
>  > >  Is it time there was a JSON codec included in the python standard 
> library?
>  >
>  > I would definitely support the incusion of a JSON library in the
>  > standard lib.   And, I think that it should be simplejson which is
>  > used by TurboGears, Pylons, and bundled with Django.
>
>  +1

+1 here too.

Brett should probably figure out where to put it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Bill Janssen]

> Is it time there was a JSON codec included in the python standard library?

Great idea.  In fact, I'd support including a whole ECMAscript
interpreter module, much as we have XML parsers.

Bill
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Jonathan Ellis]

On Mon, 10 Mar 2008 09:37:35 -0400, "Mark Ramm"
<[EMAIL PROTECTED]> said:
> >  Is it time there was a JSON codec included in the python standard library?
> 
> I would definitely support the incusion of a JSON library in the
> standard lib.   And, I think that it should be simplejson which is
> used by TurboGears, Pylons, and bundled with Django.

+1
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Jodok Batlogg]

On 10.03.2008, at 14:37, Mark Ramm wrote:

Is it time there was a JSON codec included in the python standard  
library?


I would definitely support the incusion of a JSON library in the
standard lib.   And, I think that it should be simplejson which is
used by TurboGears, Pylons, and bundled with Django.


+1 for simplejson





Choosing a Python JSON Translator
http://blog.hill-street.net/?p=7


This blog is a year old, and isn't quite accurate anymore.   in
particular simplejson sprouted some optional C code, and is now a lot
faster.


yes. i second that as well.

jodok batlogg




--Mark Ramm
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/jodok%40lovelysystems.com


--
"Beautiful is better than ugly."
  -- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzhütterstraße 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060



smime.p7s
Description: S/MIME cryptographic signature
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Time a for JSON parser in the standard library?

[Mark Ramm]

>  Is it time there was a JSON codec included in the python standard library?

I would definitely support the incusion of a JSON library in the
standard lib.   And, I think that it should be simplejson which is
used by TurboGears, Pylons, and bundled with Django.

>  Choosing a Python JSON Translator
>  http://blog.hill-street.net/?p=7

This blog is a year old, and isn't quite accurate anymore.   in
particular simplejson sprouted some optional C code, and is now a lot
faster.

--Mark Ramm
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com