Re: [Standards] Proposed XMPP Extension: JSON Containers
On 13 Oct 2013 13:56, "Waqas Hussain" wrote: > Would getting to write a MIME over XMPP spec be a suitable prize? But > wait, MIME has escape sequences (and also, what MIME exactly? there's > so much to choose from in there...). :) > The basic structure of MIME, in particular multipart/* and message/rfc822, is such that you can take one MIME message and embed it in another - however there's no escaping used. The way it works instead is to use boundaries which are simply very unlikely to occur within the encapsulated part. The boundaries aren't tested by MIME engines, as far as I'm aware. So forwarding this message as an attachment will fully encapsulate it, one text-based language in another, without any escaping. You can also sling (unencoded) JSON, XML, and so on inside body parts without escaping, by a similar method. In fact, SQL and Lua, both using '--' sequences for comments, are the riskiest I suppose, but in practise you don't get a random string as a comment... > If you have a structured language with delimiters, and the content > inside those delimiters can contain the delimiters themselves, you > need escaping (as we discovered in the entity caps preimage issue). > Ways around that include length-encoding (which I'd say is very > non-friendly to humans), or EOF as end delimiter (which greatly limits > your structure). Or alternately you use delimiters which are both relatively long, unpredictable, and unlikely to occur even if they were. The general rule is that your data must not be confusable as your delimiter. Or to put it another way, the encapsulation must be unambiguous. Of course, in *this* case, we probably want to use XML's own escaping, and this particular subthread is Waqas and I waxing theoretical. :-) Dave.
Re: [Standards] Proposed XMPP Extension: JSON Containers
On Sat, Oct 12, 2013 at 6:31 PM, Dave Cridland wrote: > > On 12 Oct 2013 17:17, "Waqas Hussain" wrote: >> As a general rule, if you embed one textual language inside another, >> escaping is pretty much mandatory. If you see a way around it, you are >> probably mistaken and missing the edge cases. > > MIME. > > Do I win a prize? > Would getting to write a MIME over XMPP spec be a suitable prize? But wait, MIME has escape sequences (and also, what MIME exactly? there's so much to choose from in there...). :) If you have a structured language with delimiters, and the content inside those delimiters can contain the delimiters themselves, you need escaping (as we discovered in the entity caps preimage issue). Ways around that include length-encoding (which I'd say is very non-friendly to humans), or EOF as end delimiter (which greatly limits your structure). -- Waqas
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 12 Oct 2013 17:17, "Waqas Hussain" wrote: > As a general rule, if you embed one textual language inside another, > escaping is pretty much mandatory. If you see a way around it, you are > probably mistaken and missing the edge cases. MIME. Do I win a prize? Dave.
Re: [Standards] Proposed XMPP Extension: JSON Containers
On Fri, Oct 11, 2013 at 9:22 PM, Field Tian (fitian) wrote: > Yea, cdata is a traditional way in XML. XMPP seems not prefer CDATA. I don’t > know what's the detailed reason. But I think CDATA has its advantage > sometimes. There is generally this assumption that CDATA would allow adding JSON to XML without any further escaping. This isn't true. XML 1.0 defines: CData ::= (Char* - (Char* ']]>' Char*)) So JSON containing "]]>" inside cannot be added to XML without escaping in some way. Also, XML 1.0 defines valid characters (all characters, in PCDATA, CDATA, even escaped via entities) as: Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10]/* any Unicode character, excluding the surrogate blocks, FFFE, and . */ While JSON (RFC 4627) defines unescaped characters as: unescaped = %x20-21 / %x23-5B / %x5D-10 Which are not the same. JSON can contain unescaped characters that are not allowed in a CDATA section. So you have to escape those anyway. As a general rule, if you embed one textual language inside another, escaping is pretty much mandatory. If you see a way around it, you are probably mistaken and missing the edge cases. -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: JSON Containers
On Oct 12, 2013, at 6:46 AM, Matthew Wild wrote: > On 12 October 2013 02:22, Field Tian (fitian) wrote: >> Yea, cdata is a traditional way in XML. XMPP seems not prefer CDATA. I don’t >> know what's the detailed reason. But I think CDATA has its advantage >> sometimes. > > Some discussion on it from 2007: > http://mail.jabber.org/pipermail/standards/2007-July/016098.html > > The specs have never been clear on cdata. By not forbidding it (things > like processing instructions are explicitly forbidden), it's > implicitly allowed. But I have strong doubts that many XMPP > implementations handle it properly, and nobody generates them for this > reason. Maybe it's time to do some testing, and get some hard facts :) > > Regards, > Matthew +1 to gathering data before we reopen that can of worms.
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 12 October 2013 02:22, Field Tian (fitian) wrote: > Yea, cdata is a traditional way in XML. XMPP seems not prefer CDATA. I don’t > know what's the detailed reason. But I think CDATA has its advantage > sometimes. Some discussion on it from 2007: http://mail.jabber.org/pipermail/standards/2007-July/016098.html The specs have never been clear on cdata. By not forbidding it (things like processing instructions are explicitly forbidden), it's implicitly allowed. But I have strong doubts that many XMPP implementations handle it properly, and nobody generates them for this reason. Maybe it's time to do some testing, and get some hard facts :) Regards, Matthew
Re: [Standards] Proposed XMPP Extension: JSON Containers
Yea, cdata is a traditional way in XML. XMPP seems not prefer CDATA. I don’t know what's the detailed reason. But I think CDATA has its advantage sometimes. Regards .:|:.:|:. Field Tian | CCATG HF Software Engineer | +865516636 -Original Message- From: standards-boun...@xmpp.org [mailto:standards-boun...@xmpp.org] On Behalf Of Matthew Wild Sent: 2013年10月11日 9:08 To: XMPP Standards Subject: Re: [Standards] Proposed XMPP Extension: JSON Containers Hi, On 11 October 2013 01:50, Field Tian (fitian) wrote: >If wrap json directly, we need to do xml escape, It is boring. or > XML Parser cannot parse the follow message(I put into the value of > status). It might be boring, but any XML/XMPP library will be able to do this for you already. I really hope XMPP developers aren't manually XML escaping in their day-to-day work - I know I'm not :) > Why not use the traditional way? For example, to extend an element > called , then whatever type of message can be put into. Please comment! It's definitely not "traditional" to use cdata in XMPP. I would worry about interoperability problems, and I don't think how you escape the data (through entities or cdata) is within the scope of this XEP anyway. Regards, Matthew
Re: [Standards] Proposed XMPP Extension: JSON Containers
Hi, On 11 October 2013 01:50, Field Tian (fitian) wrote: >If wrap json directly, we need to do xml escape, It is boring. or XML > Parser cannot parse the follow message(I put into the value of status). It might be boring, but any XML/XMPP library will be able to do this for you already. I really hope XMPP developers aren't manually XML escaping in their day-to-day work - I know I'm not :) > Why not use the traditional way? For example, to extend an element called > , then whatever type of message can be put into. Please comment! It's definitely not "traditional" to use cdata in XMPP. I would worry about interoperability problems, and I don't think how you escape the data (through entities or cdata) is within the scope of this XEP anyway. Regards, Matthew
Re: [Standards] Proposed XMPP Extension: JSON Containers
HI Peter, Matthew & All,
If wrap json directly, we need to do xml escape, It is boring. or XML Parser
cannot parse the follow message(I put into the value of status).
{ "name": "romeo", "age": "421", "status": "" }
Why not use the traditional way? For example, to extend an element called
, then whatever type of message can be put into. Please comment!
Regards
.:|:.:|:. Field Tian | CCATG HF Software Engineer | +865516636
-Original Message-
From: standards-boun...@xmpp.org [mailto:standards-boun...@xmpp.org] On Behalf
Of Peter Saint-Andre
Sent: 2013年10月11日 7:08
To: XMPP Standards
Subject: Re: [Standards] Proposed XMPP Extension: JSON Containers
Updated to version 0.0.2 on the website.
On 10/9/13 9:20 AM, XMPP Extensions Editor wrote:
> The XMPP Extensions Editor has received a proposal for a new XEP.
>
> Title: JSON Containers
>
> Abstract: This specification defines an element to be used for encapsulating
> JSON data in XMPP.
>
> URL: http://xmpp.org/extensions/inbox/json-containers.html
>
> The XMPP Council will decide in the next two weeks whether to accept this
> proposal as an official XEP.
>
Re: [Standards] Proposed XMPP Extension: JSON Containers
Updated to version 0.0.2 on the website. On 10/9/13 9:20 AM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: JSON Containers > > Abstract: This specification defines an element to be used for encapsulating > JSON data in XMPP. > > URL: http://xmpp.org/extensions/inbox/json-containers.html > > The XMPP Council will decide in the next two weeks whether to accept this > proposal as an official XEP. >
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 10/9/13 11:45 AM, Ralph Meijer wrote: > Doing a generic with mime type and encoding seems over-design. Agreed. That opens up too much flexibility, I think. Peter -- Peter Saint-Andre https://stpeter.im/
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 2013-10-09 19:45, Ralph Meijer wrote:
> Please don't top post. Reformatted below.
>
> On 2013-10-09 18:58, DOI Yusuke wrote:
>> On 2013-10-09 18:46, Maxim Ignatenko wrote:
>>> On 9 October 2013 16:20, XMPP Extensions Editor wrote:
Abstract: This specification defines an element to be used for
> encapsulating JSON data in XMPP.
>>>
>>> What's the intended purpose of embedding JSON into XMPP stanzas?
>>> Provide a standard encapsulation for things like Google Cloud
>>> Messaging?
>>
>> I feel it's better to define a standard method to encode arbitrary
>> content type {} than defining
>> tag specifically to say 'this is JSON'.
>> (I'm not sure if there are such spec already or not)
>
> First off, arbitrary content cannot be represented in XMPP like that, as
> it doesn't embed binary data nicely. But see Justin's reply for that, as
> well.
>
> The primary use case for me is in publish-subscribe. There, you must
> have a containing element for item payloads, and specifying one
> dedicated to JSON makes a lot of sense to me. If you read today's
> article by Justin [1], you can easily see why.
>
> As mentioned in the comments over at HN [2], if you want to make things
> easy for (web-)developers that like to deal with simple APIs without
> having to craft XML for payloads this really helps. Now the API is
> basically a destination address for the Publish-Subscribe service, a
> node identifier, optionally an item identifier and the payload as JSON.
> Doing a generic with mime type and encoding seems over-design.
[1] http://blog.fanout.io/2013/10/09/publishing-json-over-xmpp/
[2] https://news.ycombinator.com/item?id=6520524
--
ralphm
Re: [Standards] Proposed XMPP Extension: JSON Containers
Please don't top post. Reformatted below.
On 2013-10-09 18:58, DOI Yusuke wrote:
> On 2013-10-09 18:46, Maxim Ignatenko wrote:
>> On 9 October 2013 16:20, XMPP Extensions Editor wrote:
>>> Abstract: This specification defines an element to be used for
encapsulating JSON data in XMPP.
>>
>> What's the intended purpose of embedding JSON into XMPP stanzas?
>> Provide a standard encapsulation for things like Google Cloud
>> Messaging?
>
> I feel it's better to define a standard method to encode arbitrary
> content type {} than defining
> tag specifically to say 'this is JSON'.
> (I'm not sure if there are such spec already or not)
First off, arbitrary content cannot be represented in XMPP like that, as
it doesn't embed binary data nicely. But see Justin's reply for that, as
well.
The primary use case for me is in publish-subscribe. There, you must
have a containing element for item payloads, and specifying one
dedicated to JSON makes a lot of sense to me. If you read today's
article by Justin [1], you can easily see why.
As mentioned in the comments over at HN [2], if you want to make things
easy for (web-)developers that like to deal with simple APIs without
having to craft XML for payloads this really helps. Now the API is
basically a destination address for the Publish-Subscribe service, a
node identifier, optionally an item identifier and the payload as JSON.
Doing a generic with mime type and encoding seems over-design.
--
ralphm
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 10/09/2013 09:58 AM, DOI Yusuke wrote:
I feel it's better to define a standard method to encode arbitrary
content type {} than defining
tag specifically to say 'this is JSON'.
(I'm not sure if there are such spec already or not)
I like this. Kinda reminds me of what I suggested here:
http://mail.jabber.org/pipermail/standards/2013-January/027027.html
Justin
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 09/10/13 16:20, XMPP Extensions Editor wrote:
> Abstract: This specification defines an element to be used for encapsulating
> JSON data in XMPP.
I don't see how the example given,
http://example.com/user-queries";>
{ "name": "romeo" }
has any advantage over using
http://example.com/user-queries";>
{ "name": "romeo" }
and the namespace owner (in this case example.com) documenting that the
payload of a {http://example.com/user-queries}query element is JSON. If
the recipient understands the http://example.com/user-queries namespace,
then they know that its payload is JSON, and can perform whatever
validation is required. If the recipient doesn't understand it, they're
not going to be able to do anything useful with it anyway...
It would be useful for the text and/or examples to indicate how the
embedding in XML interacts with characters that are reserved/special in
XML. For instance, my interpretation is that this message:
{ "name": "romeo", "age": "421", "status": "I >3 Juliet" }
is expected to be interpreted as equivalent to receiving this
application/json from a web server or whatever:
{ "name": "romeo", "age": "421", "status": "I <3 Juliet" }
If I'm right, I think it'd be good to use an example like this that
illustrates that point. If I'm wrong, then I don't see how it could work
in a generic XML parser.
> The data MUST be encoded as UTF-8 (though officially unspecified,
> this is the de facto encoding for JSON today).
The JSON RFC, RFC 4627, does in fact say "The default encoding is
UTF-8", and could be interpreted as implying that the only valid
encodings are UTF-8, UTF-16 and UTF-32 - it states that the pattern of
nulls in the first 4 bytes is sufficient to detect the encoding, which
is only true if common single-byte encodings like Latin-1 are not allowed.
In any case, if the JSON is character data in an XMPP XML stream, then
it has to be UTF-8, whether this XEP says it is or not. Non-UTF-8 can
only appear in XMPP XML if it's encoded in Base64 or something; or if
you're less concerned with the wire protocol and more concerned with
semantics, you could say that XMPP is a transport for (XML containing)
Unicode text (encoded in UTF-8 for the wire protocol), not bytes.
S
h
Re: [Standards] Proposed XMPP Extension: JSON Containers
I feel it's better to define a standard method to encode arbitrary
content type {} than defining
tag specifically to say 'this is JSON'.
(I'm not sure if there are such spec already or not)
Yusuke
From: Maxim Ignatenko
Subject: Re: [Standards] Proposed XMPP Extension: JSON Containers
Date: Wed, 9 Oct 2013 17:46:38 +0100
> On 9 October 2013 16:20, XMPP Extensions Editor wrote:
>> Abstract: This specification defines an element to be used for encapsulating
>> JSON data in XMPP.
>
> What's the intended purpose of embedding JSON into XMPP stanzas?
> Provide a standard encapsulation for things like Google Cloud
> Messaging?
Re: [Standards] Proposed XMPP Extension: JSON Containers
On 9 October 2013 16:20, XMPP Extensions Editor wrote: > Abstract: This specification defines an element to be used for encapsulating > JSON data in XMPP. What's the intended purpose of embedding JSON into XMPP stanzas? Provide a standard encapsulation for things like Google Cloud Messaging?
