Dear Yusuke, Regarding your questions:
> Is a mail with the subject 'Requirement Clarification' sent to the list? I've not received one. > Stepping back to the requirements. For example, what I need is binary-only > bootstrap mechanism for EXI (not like XEP-0138) with least > negotiation/propose of compression parameter. Does your proposal cover such > use case? No. This XEP covers EXI negotiation in a way compliant with XEP-0138. >Let me ask a question: which strategy are you taking? > > 1) short target: find a way to encode XMPP stanza with existing > EXI format 1.0 > 2) long target: find a way to encode XMPP stanza ideally with > proposed currently-nonexistent ideal EXI 1.1 (or 2.0) The proposal supports different EXI versions. It's part of the negotiation, using the version attribute. > My concern on a-1 is about XML semantics. Does current server implementation > okay to have modified XML-level structure? On the other hand, re-use of > string table is possible only in a-3 (discussion below). Sorry, I don't understand your question. > For dynamic grammars, your proposal "reusing the same options as used by the > stream" (section 3.2) may not be adequate, because this means all stanzas > should be in the same schema and does not allow introduction of new schema on > the fly. However, I agree a > constrained node seldom updates its > functionality so fixed set of schema on C2S pair should be okay (for S2S > communication it's not good). I don't see why this should require all stanzas to be in the same schema. Schemas are handled by namespace, and a collection of schemas can be used in compression. Which schema namespaces are to be included is part of the negotiation. Sincerely, Peter Waher -----Original Message----- From: Yusuke DOI [mailto:yusuke....@toshiba.co.jp] Sent: den 16 mars 2013 01:10 To: Peter Waher Cc: Peter Saint-Andre; XMPP Standards; Joachim Lindborg (joachim.lindb...@sust.se); Takuki Kamiya (tkam...@us.fujitsu.com) Subject: Re: EXI extension proposal Dear Peter, (2013/03/15 12:53), Peter Waher wrote: >> I believe we need to clarify some of 'requirements' first. Maybe, >> there could be several approaches for EXI1.0 or maybe we need to >> propose something to EXI1.x(maybe x=1), upon such requirement >> discussion. >> >> Topics may involve: - Minimal client requirement - Minimal server >> requirement - Schema management - Efficiency requirement - ... >> >> I'll make another thread on requirement discussion following this >> mail. > > Sounds like a very good idea. I'll wait your coming mail, and then > create a new section for this. Is a mail with the subject 'Requirement Clarification' sent to the list? I have had mailer trouble so I'm not sure I've successfully sent out the mail. Stepping back to the requirements. For example, what I need is binary-only bootstrap mechanism for EXI (not like XEP-0138) with least negotiation/propose of compression parameter. Does your proposal cover such use case? >> Agreed. BTW, is the schema itself encoded in EXI or plaintext? The >> example seems to be in plaintext but it could be EXI-encoded binary. > > Do you mean with the uploadSchema command? In this case, it's simply > base64-encoded plain text. > >> Then upload in xs:base64binary will become straitforward Binary type >> upload, far more efficient. > > I agree. If packet size is of importance (which it probably is for > memory constrained devices), embedding the schemas might not even be > possible. In this case, downloadSchema is an option that requires very > few bytes. EXI-encoded schema exchange should be also good for EXI-only nodes. Anyway, if a node does not be capable of handling EXI, schema download/upload is useless. >>> The problem with this approach, is that name seldom change, >>> especially during development. And a slight change, a new attribute, >>> a new element, etc., will completely change the compression. >>> Furthermore, errors produced in this way will be extremely difficult >>> to debug. An efficient and fool-proof way to communicate using >>> different schema versions (having the same namespace and schema IDs) >>> is necessary. >> >>> §2.4 also proposes the possibility to install such schema files >>> manually on the server. The XEP allows for different scenarios. > >> Agreed. We need different mechanism for development stage and >> deployment stage. I believe schemaLocation and hash is sufficient >> enough. > > It will probably be enough. However, I used target namespace instead > of schema location, as the schema may be available in many places, and > it's the target namespace that is used during validation and > compression. I also included the byte size as a reference. I don't think the number of bytes is required in this case. Usually schema file itself has targetNameSpace so namespace is also redundunt. With schemaLocation, the other end of the node may choose to download the schema from the location, instead of returning a 'I don't have the schema please upload with our tiny sensor network bandwidth' error. > I had to rethink this a bit, after comments made by Takuki Kamiya. > I've rewritten §3.2 to clarify this. Is it clear from what I've > written? > > The EXI header and options are omitted in the communication, since > they are agreed upon during initial setup. Only EXI bodies are > written. Furthermore, I changed the operation to exclude the SD/ED > events and include a description of how to perform padding after each > last End Element EE at the end of each stanza, so EXI bodies are sent > at byte boundaries. Again, as Taki mentioned, we'd better not to alter EXI spec. Let me ask a question: which strategy are you taking? 1) short target: find a way to encode XMPP stanza with existing EXI format 1.0 2) long target: find a way to encode XMPP stanza ideally with proposed currently-nonexistent ideal EXI 1.1 (or 2.0) If you're taking strategy (2), I stongly argue it's too early to step into detailed spec. Even you and I does not have common view on this problem space. For strategy (1), there are at least two problems (let me start again): a) padding b) dynamic schema/grammar For padding problem there are three solutions, a-1) encode a stanza in a time, as you proposed a-2) use selfContained per stanza a-3) use byte-aligned My concern on a-1 is about XML semantics. Does current server implementation okay to have modified XML-level structure? On the other hand, re-use of string table is possible only in a-3 (discussion below). For dynamic grammars, your proposal "reusing the same options as used by the stream" (section 3.2) may not be adequate, because this means all stanzas should be in the same schema and does not allow introduction of new schema on the fly. However, I agree a constrained node seldom updates its functionality so fixed set of schema on C2S pair should be okay (for S2S communication it's not good). >> I think this makes far worse compression ratio (according to my >> experiment). > > This may depend on type of information sent. > > Example: > > <Fields> <Field name="Energy" unit="MWh" value="12.3" > timestamp="...."/> ... 100 times (inly Energy, MWh) </Fields> > > May be better compressed if string tables are renewed, since bit field > lengths decrease and strings are short, rather than if it has to share > space with, for instance: > > <Fields> <Field name="Energy" unit="MWh" value="12.3" > timestamp="...."/> <Field name="Volume" unit="m^3" value="12.3" > timestamp="...."/>... > > 20 times, with different field names and units. </Fields> > > Depends on use case. So manufacturer may have to create tests > depending on the use of the device, and schemas used. I agree we need test but I don't get your point in the example (looks like an extreme case with just slight better compression -- 40 of labels makes 6-bit string table identifier. The 100 records equiv. to 75 characters = re-use of 6-10 JIDs will compensate the cost). But anyway, >> We need to be careful to add something on existing standards. In >> other words, existing EXI processors should be applicable for this >> proposal. > > Correct. This option is for the XMPP implementation, on how to > use/configure existing EXP processors. It's not an option for the EXI > processor itself. I strongly oppose to this idea (sessionWideBuffer) as long as your proposal is to encode each stanza as standalone EXI document. Re-use of string table is not allowed in the EXI spec. Best Regards, // Yusuke DOI <yusuke....@toshiba.co.jp>
