Re: [PHP-DEV] Removing basic types from our JSON parser
mike wrote: On Sun, Dec 14, 2008 at 12:39 AM, Rasmus Lerdorf ras...@lerdorf.com wrote: Eh? Read what you wrote there. If json wasn't pure javascript, how in the world would eval() work on it? Sorry. I guess I meant it didn't execute by itself, but needed to be interpreted using something like eval and/or thrown into a variable before it was useful. Otherwise it's just text. I'm not sure why we had to do that workaround I previously mentioned, and I cannot find a repository with that old code, but nowadays json_encode seems to work seamlessly for all the data we exchange (which are strings, ints, arrays, no objects, possibly not even booleans) and just wanted to voice off if there was any possible incompatibility that may be introduced here. I'll go back in my corner now :) I thought I explained that a few times now. JSON is defined in the RFC as a subset of Javascript, so a JSON Parser doesn't need to have all the capabilities of the javascript parser. Things break if you pass the output of json_encode() to a JSON parser that follows the RFC to the letter. When we do json_encode(123) we spit out just: 123 or json_encode(abc) we produce just: abc this is obviously valid Javascript so both eval and direct injection into a script block (which is the same thing) will work fine. But the RFC says we should be wrapping either array or object notation around it. As in [123] and [abc] to make it valid JSON. Direct injection into scriptvar foo = ?php echo json_encode(abc)?;/script will still work, of course, the only change is that foo now becomes an array with a single string element as opposed to just the string itself, so making this change will break existing javascript that relies on PHP producing unwrapped basic types. Now, in most cases when you are passing stuff around via json, you don't do that for a single value, so in most cases you end up passing an array anyway, and in that case nothing changes. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Removing basic types from our JSON parser
script var foo = ?php echo json_encode($foo)?; /script will always work. The only question is what sort of variable foo will end up being. The RFC says we have to wrap basic types in an array or object, while currently we let the basic types through without the wrapper. He problem her eis: A lot of people use in a way and assume, that a basic type is serialized as a basic type in JSON/JS. If we change this, it could break a lot of apps. And the above is really nice to encode PHP strings to a secure JS variant. It is hard to do with addslashes etc. So I think, we should not change the way it works currently. I think in the docs for json_encode() should be a warning: JSON officially only supports Objects and Arrays as outer/top-level/... type for JSON serializations. So the argument of json_encode must be a PHP object or array to be fully JSON conformant. If you provide a simple type (string, number, Boolean), json_encode will generate a valid java script, but some conformant JSON parser may not be able to read it. How about this? Uwe -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Původní zpráva Předmět: [PHP-DEV] Removing basic types from our JSON parser Od: theta...@php.net (Uwe Schindler) Komu: Datum: 14.12.2008 10:40 script var foo = ?php echo json_encode($foo)?; /script will always work. The only question is what sort of variable foo will end up being. The RFC says we have to wrap basic types in an array or object, while currently we let the basic types through without the wrapper. He problem her eis: A lot of people use in a way and assume, that a basic type is serialized as a basic type in JSON/JS. If we change this, it could break a lot of apps. +1 In PHP there is no function for escaping JS strings and json_encode is useful compensation. I use it this way very often. David Grudl -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Removing basic types from our JSON parser
For reference I saw people use json_encode() to pass a string to javascript into their page while avoiding bugs/XSS with stuff like /script. var foo = ?=json_encode($my_string)?; ... (yes, they maybe heared somewhere that JSON is *not* javascript, I told 'em too). This is not correct. JSON *is* valid JavaScript, but the other way round is not correct. Not every Javascript is JSON. So it is perfectly legal, to do this. I think, this is really cool, using json_encode() as something like htmlspecialchars for correctly encoding strings inside JS for security. I use it, too. To the problem with basic types: I like the way of maybe giving a option to the function. If you want to be standards conformant, you have to remove the encoding parameter (which is also there), too. JSON has to be UTF-8, if you want to be conform to the specs. But for the example above (secure encoding of JS strings with json_encode), it is perfectly legal to use another encoding (in the above case, the encoding of the HTML page). Uwe -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
var foo = ?=json_encode($my_string)?;
... (yes, they maybe heared somewhere that JSON is *not* javascript, I
told 'em too).
JSON is a strict subset of JavaScript, and the above scenario is something I
also do often.
Since PHP generates web pages, any time I need to generate a JS literal
json_encode() is the natural equivalent of var_export(.., true) for PHP
literals.
Since it may not match the RFC apparently, I suggest a flag as some other
people said:
function json_encode($var, $allowScalar = false) {}
This way we gain best of both worlds without disrupting either specs nor
existing workflows.
Regards,
Stan Vassilev
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Stan Vassilev | FM wrote:
var foo = ?=json_encode($my_string)?;
... (yes, they maybe heared somewhere that JSON is *not* javascript, I
told 'em too).
JSON is a strict subset of JavaScript, and the above scenario is
something I also do often.
Since PHP generates web pages, any time I need to generate a JS literal
json_encode() is the natural equivalent of var_export(.., true) for PHP
literals.
Since it may not match the RFC apparently, I suggest a flag as some
other people said:
function json_encode($var, $allowScalar = false) {}
I'm not sure if that is the right way of looking at it though. I think
the only question is how to represent the scalar types, not whether they
should be allowed. As far as I am concerned it is obvious that they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:
JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).
The only question is how to serialize these into a JSON-text string.
Again, from the RFC:
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
This last statement is the issue. When serializing a basic type into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(), that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.
I'll talk to Douglas on Monday if he is in the office and get his opinion.
-Rasmus
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
I'm not sure if that is the right way of looking at it though. I think
the only question is how to represent the scalar types, not whether they
should be allowed. As far as I am concerned it is obvious that they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:
JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).
The only question is how to serialize these into a JSON-text string.
Again, from the RFC:
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
This last statement is the issue. When serializing a basic type into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(), that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.
I'll talk to Douglas on Monday if he is in the office and get his opinion.
-Rasmus
cool.
let's see how :D. Crockford explain it :D
but before he can react on this thread, i guess Rasmus is right
understanding the RFC.
json force your encoder to output the following in encoded form
1. array: multiple of unnamed values
2. object: multiple of named values
so it's right to wrap it in object or array (u can say remove basic types)
json_encode(array(1)); = [1]
json_encode(array(1, 2, 3, 1)); = [1,2,3,1]
json_encode(array('value1' = 1)); = {value1:1}
json_encode(array('a'=1, 'b'=2, 'c'=3, 'd'=1)); = {a:1,
b:2,c:3,d:1}
i'd prefer 2 new functions: js_encode (or js_export?) js_decode which
support basic types instead of having an extra optional parameter
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Sorry for top posting. I see no reason why php should not interop with
javascript 100% as it should. I do not see a reason to create new
functions, parameters, etc. The existing should be expected to both
encode and decode json with javascript properly. If there is a
mismatch right now, then it should be fixed.
On Dec 13, 2008, at 10:10 PM, moo.tinys moo.ti...@gmail.com wrote:
I'm not sure if that is the right way of looking at it though. I
think
the only question is how to represent the scalar types, not whether
they
should be allowed. As far as I am concerned it is obvious that
they be
allowed and the JSON spec is quite explicit that JSON can be used to
represent scalar types:
JSON can represent four primitive types (strings, numbers,
booleans, and null) and two structured types (objects and arrays).
The only question is how to serialize these into a JSON-text string.
Again, from the RFC:
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
This last statement is the issue. When serializing a basic type
into a
JSON text string, the RFC says we need to wrap it inside an array or
object. So, if we are going to add an option to json_encode(),
that's
what the option should do. Specify whether, and perhaps how, to wrap
basic types, not whether to allow or disallow them.
I'll talk to Douglas on Monday if he is in the office and get his
opinion.
-Rasmus
cool.
let's see how :D. Crockford explain it :D
but before he can react on this thread, i guess Rasmus is right
understanding the RFC.
json force your encoder to output the following in encoded form
1. array: multiple of unnamed values
2. object: multiple of named values
so it's right to wrap it in object or array (u can say remove
basic types)
json_encode(array(1)); = [1]
json_encode(array(1, 2, 3, 1)); = [1,2,3,1]
json_encode(array('value1' = 1)); = {value1:1}
json_encode(array('a'=1, 'b'=2, 'c'=3, 'd'=1)); = {a:1,
b:2,c:3,d:1}
i'd prefer 2 new functions: js_encode (or js_export?) js_decode which
support basic types instead of having an extra optional parameter
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
mike wrote: Sorry for top posting. I see no reason why php should not interop with javascript 100% as it should. I do not see a reason to create new functions, parameters, etc. The existing should be expected to both encode and decode json with javascript properly. If there is a mismatch right now, then it should be fixed. There is no mismatch with Javascript now at all. We spit out perfectly valid Javascript. There is only a slight mismatch with the JSON RFC which defines a stricter subset of Javascript than we are using. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
I can't give specifics (I'm mobile and I don't remember where the code would be anyway) but we had an issue with php and javascript exchange via json where the php output using json_encode and we had to wrap it with additional braces for it to be usable. Sorry for being vague, this thread reminded me of that which is why I wanted to voice off. Thanks for the clarification (and thank you so much for PHP :)) On Dec 13, 2008, at 10:39 PM, Rasmus Lerdorf ras...@lerdorf.com wrote: mike wrote: Sorry for top posting. I see no reason why php should not interop with javascript 100% as it should. I do not see a reason to create new functions, parameters, etc. The existing should be expected to both encode and decode json with javascript properly. If there is a mismatch right now, then it should be fixed. There is no mismatch with Javascript now at all. We spit out perfectly valid Javascript. There is only a slight mismatch with the JSON RFC which defines a stricter subset of Javascript than we are using. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Yes, but that would be going to a json_parse thing somewhere, not pure Javascript. -Rasmus mike wrote: I can't give specifics (I'm mobile and I don't remember where the code would be anyway) but we had an issue with php and javascript exchange via json where the php output using json_encode and we had to wrap it with additional braces for it to be usable. Sorry for being vague, this thread reminded me of that which is why I wanted to voice off. Thanks for the clarification (and thank you so much for PHP :)) On Dec 13, 2008, at 10:39 PM, Rasmus Lerdorf ras...@lerdorf.com wrote: mike wrote: Sorry for top posting. I see no reason why php should not interop with javascript 100% as it should. I do not see a reason to create new functions, parameters, etc. The existing should be expected to both encode and decode json with javascript properly. If there is a mismatch right now, then it should be fixed. There is no mismatch with Javascript now at all. We spit out perfectly valid Javascript. There is only a slight mismatch with the JSON RFC which defines a stricter subset of Javascript than we are using. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Fri, Dec 12, 2008 at 7:50 PM, Scott MacVicar sc...@macvicar.net wrote: The problem here is that none of the other JSON parsers in any other language support this and more importantly the browsers [2] which are now adding native JSON support. Users are frequently expecting the result from a json_encode in PHP to just work with JSON.parse() on the client side. [3] I have a patch for this that implements the RFC exactly and makes secure json_encode only works with objects and arrays. definately +1 we use json both encoding and decoding to interact with javascript and ensuring that PHP's built-in json is standards compliant is an absolute must. -please- make sure it works both encoding php - js, decoding js - php for all relevant types in both languages. thank you :) -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Le samedi 13 décembre 2008 à 03:50 +, Scott MacVicar a écrit :
Hi All,
Basic types were added to our JSON decoder to PHP 5.2.1, this allows
one to use json_encode / json_decode on any of our scalar types. Omar
correctly identified #38680 as not a bug but it appears that Ilia
added support for this anyway violating the RFC [1]. Maybe there was a
reason for this but I'm not sure why?
The reason was to make json_decode() able to decode anything encoded by
json_encode(). I believe many people may be depending on this (I saw a
few codes).
If json_decode() is made to only accept arrays/struct, the same should
be done to json_encode().
For reference I saw people use json_encode() to pass a string to
javascript into their page while avoiding bugs/XSS with stuff like
/script.
var foo = ?=json_encode($my_string)?;
... (yes, they maybe heared somewhere that JSON is *not* javascript, I
told 'em too).
I also saw people using json_encode/json_decode as an alternate for
serialize/unserialize.
Also, reading [2], I see about stringify If value is an object or
array, the structure will be visited recursively to determine the
serialization of each membr or element., this seems to assume that
value can be something else than an object or array.
At least on Firefox 3.2, it is not the case.
The problem here is that none of the other JSON parsers in any other
language support this and more importantly the browsers [2] which are
now adding native JSON support. Users are frequently expecting the
result from a json_encode in PHP to just work with JSON.parse() on the
client side. [3]
I did some tests with firefox minefield (3.2pre) :
javascript:alert(JSON.stringify({foo: bar}));
= {foo:bar}
javascript:alert(JSON.stringify(true));
= Invalid argument
javascript:alert(JSON.stringify(hello world));
= Invalid argument
javascript:alert(JSON.stringify([hello world]));
= [hello world]
javascript:alert(JSON.parse([\hello world\]));
= hello world
(bug? or alert() behaviour when receiving an array?)
javascript:alert(JSON.parse(\hello world\));
= Error parsing JSON
javascript:alert(JSON.parse(null));
= Error parsing JSON
I have a patch for this that implements the RFC exactly and makes
secure json_encode only works with objects and arrays.
This would be a change for 5.3+ and we should just document in 5.2
that what we did was a bad idea.
I'm +0 on this, because it will break PHP code on one side, but will
break on browser stuff on the other side.
I guess making 5.3 throwing E_WARNING (or E_DEPRECATED, if someone reads
it) and removing this support in HEAD could be the default behaviour
based on how we handled most features we wanted to remove when people
started relying on those.
Anyway, maintaining two versions of the JSON parser is not something we
want (I believe) so this should be removed as soon as possible. I'm just
not sure about breaking this features this fast.
Mark
[1] - http://www.ietf.org/rfc/rfc4627.txt?number=4627
[2] - http://wiki.ecmascript.org/doku.php?id=es3.1:json_support
[3] - https://developer.mozilla.org/en/Using_JSON_in_Firefox
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Removing basic types from our JSON parser
Mark Karpeles wrote: Le samedi 13 décembre 2008 à 03:50 +, Scott MacVicar a écrit : Hi All, Basic types were added to our JSON decoder to PHP 5.2.1, this allows one to use json_encode / json_decode on any of our scalar types. Omar correctly identified #38680 as not a bug but it appears that Ilia added support for this anyway violating the RFC [1]. Maybe there was a reason for this but I'm not sure why? The reason was to make json_decode() able to decode anything encoded by json_encode(). I believe many people may be depending on this (I saw a few codes). If json_decode() is made to only accept arrays/struct, the same should be done to json_encode(). For reference I saw people use json_encode() to pass a string to javascript into their page while avoiding bugs/XSS with stuff like /script. var foo = ?=json_encode($my_string)?; ... (yes, they maybe heared somewhere that JSON is *not* javascript, I told 'em too). I also saw people using json_encode/json_decode as an alternate for serialize/unserialize. Also, reading [2], I see about stringify If value is an object or array, the structure will be visited recursively to determine the serialization of each membr or element., this seems to assume that value can be something else than an object or array. At least on Firefox 3.2, it is not the case. I think it does more than assume that. The spec specifically says that the value can be any Javascript value in the prototype: JSON.stringify(value, replacer, space) value any JavaScript value, usually an object or array. replaceran optional parameter that determines how object values are stringified for objects without a toJSON method. It can be a function or an array. space an optional parameter that specifies the indentation of nested structures. If it is omitted, the text will be packed without extra whitespace. If it is a number, it will specify the number of spaces to indent at each level. If it is a string (such as '\t' or 'nbsp;'), it contains the characters used to indent at each level. I think we should follow the spec, not individual browser implementations. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
