Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Apr 09, 2009 at 07:19:11PM -0500, XMPP Extensions Editor wrote:
> Version 0.8 of XEP-0198 (Stream Management) has been released.
>
> Abstract: This specification defines an XMPP protocol extension for active
> management of an XML stream between two XMPP entities, including features for
> stanza acknowledgements and stream resumption.
>
> Changelog: [See revision history] (ff/jk/jjh/psa)
>
> Diff:
> http://svn.xmpp.org:18080/browse/XMPP/trunk/extensions/xep-0198.xml?%40diffMode=u&%40diffWrap=s&r1=2933&r2=3028&u=3&ignore=&k=
>
> URL: http://xmpp.org/extensions/xep-0198.html
>
Looks fine to me now, just a minor inconsistency (probably from editing):
4. Acks
...
An element MUST possess an 'h' attribute.
An element SHOULD NOT possess any attributes.
But later it says:
When an element ("request") is received, the recipient MUST acknowledge
it by sending an ack element (either or ) to the sender.
...
The response MUST contain a value of 'h' that is equal to the number of
stanzas handled by the recipient of the element.
Greetings,
Robin
--
Robin Redeker | Deliantra, the free code+content MORPG
el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net
http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Thu, Apr 09, 2009 at 10:01:09PM +0530, Mridul Muralidharan wrote: > [.snip.] > > To get it clarified (since I am looking at client libraries right now), that > is still within the component's domain right ? Not outside of it ? > > > What I mean is : > > user disco's/connects to component transport.server.domain component sends > back presence for us...@transport.server.domain, > us...@transport.server.domain/res , etc > > Or do you mean something different ? So in essence, I am trying to see if a > client can infer if the presence from a bare jid is coming from a component > (and so handled separately) or from an xmpp user where barejid does not make > sense. > If _presence_ semantics don't allow presence from bare JIDs it's wrong for components to try to fit the presence into them. I certainly don't want to build disco-hacks into presence handling just for some broken components who don't know how to gateway presence information in a well defined RFC compliant way. IMO either the RFC should specify presence from bare JIDs or components should introduce 'fake' resources. For the moment mapping presence from bare JIDs to an empty resource (resource string with length 0) is probably the best workaround a client (library) can do. And some special rules in general for bare JIDs (like unavailability from a bare JID stands for 'no resource online anymore'). Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Thu, Apr 09, 2009 at 11:25:15AM +0200, Jonathan Schleifer wrote: > Am 09.04.2009 um 00:11 schrieb Justin Karneges: > > > However, transport contacts often have no resource. These are not > > real logged > > in XMPP clients, but faked contacts from a server component. You > > will get > > and have to deal with it. > > > You usually get presence from screenn...@foo.transport/ - so it's not _that_ would be even worse than a bare JID, as the resource part MUST NOT be empty. A resource string is always at least 1 character long. > a bare JID, but a JID with empty resource. At least, I haven't seen > getting presence from a bare JID so far. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Wed, Apr 08, 2009 at 09:05:46AM -0600, Peter Saint-Andre wrote: > On 4/8/09 6:36 AM, Dave Cridland wrote: > > On Wed Apr 8 11:21:51 2009, Robin Redeker wrote: > >> Hi! > >> > >> I've recently stumbled across unavailable presence stanzas on s2s > >> connections > >> as replies to probes: > >> > >> > >> > >> > > Right, we hand those out too, whenever users are known to be offline. > > We'll also timestamp them, and include , if we have that > > information. > > Hmm, is this in response to a probe? That might make some sense, then. It is sent response to a probe in jabberd2. But I'm wondering about the general semantics of the unavailable presence from a bare JID of a contact (or even a non-contact). After thinking a bit more about presences I went into some other issues which I didn't find explained in the RFC or didn't make sense to me: 1. I'm also wondering about the general semantics of _available_ presence from a bare JID, like discussed in another branch of this thread. Imagine these presence stanzas are received by a client for contact a...@b: 10 xa away What should I display? Is the last presence from 'the "empty" resource'? Empty resources make no sense, as any resource's name must not be empty anyways (see BNF in RFC 3920). But whats the alternative interpretation of presence from a bare JID? Does it shadow any other presence? Is it guaranteed that a client will not receive presence for a bare JID and full JID from the same contact? If services send presence from bare JIDs, how should those be handled? What is the meaning of elements in those presences? 2. Another question I got is: RFC 3921 bis 4.5.4. Client Processing of Inbound Unavailable Presence http://xmpp.org/internet-drafts/draft-saintandre-rfc3921bis-07.html#presence-unavailable-client 1. If the user is in the contact's roster, the client MUST display the unavailable presence information in an appropriate roster interface. Imagine that some contact was online with 2 resources which become unavailable and send me their reason for becoming unavailable: 20 Off for vacation. 10 Off for picking girlfriend up. Ignore the contradicting offline status for now, but for which' of those should I display the unavailable information? For both resources? For the unavailable presence which has been received most recently? Or for the unavailable presence with the highest priority? Lets assume the client should display the unavailable presence for _all_ resources which went offline. Wouldn't that be inconsistent with presence probes where the server returns unavailable information for only one resource? For this read: 4.3.2. Server Processing of Inbound Presence Probe 2. Else, if the contact has no available resources, the server SHOULD reply to the presence probe by sending to the user the full XML of the last presence stanza of type "unavailable" received by the server from the contact ... This means a server only remembers the last received unavailable presence, but a client _must_ display all unavailable presences it receives. That feels a bit inconsistent to me. Also imagine what happens when people use random resources, then the list of unavailable presences (which MUST be displayed "appropriately") will grow indefinitely! Ok, this is maybe no "appropriate", but what _IS_ actually appropriate in this case? -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] unavailable presence from bare JID
Hi! I've recently stumbled across unavailable presence stanzas on s2s connections as replies to probes: RFC 3921 (bis) doesn't explicitly disallow this, and jabberd2 seems to generate these unavailable presences. How is a client supposed to handle it in case it has received available presence from some resources before, eg. like this: And then receives: Should the client assume that the resources 'bla' and 'foo' are offline now? Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Presence distribution
On Tue, Mar 31, 2009 at 05:36:59PM +0100, Dave Cridland wrote: > On Tue Mar 31 16:00:20 2009, Pedro Melo wrote: >> 1) as much as you have right now: remote servers can do whatever they >> want with your presence, you have no control after they leave your own >> server (a pessimist would even say that you don't have any control >> after they leave your client); >> >> > Well, there's trust, and responsibility. If a contact has an outright > evil server, there's little you can do - similarly if your own server, or > even client, is subverted. For things like this, it's really game over. > We generally choose to model a trusted client and a trusted server, > although we assume an untrusted server for content in the case of e2e > encryption. > > But then there's responsibility - right now, it's inarguably your local > server which enforces who gets your presence, and a reverse-roster lookup > causes immediate problems there - if people don't get to see your > presence (or worse, do when they shouldn't), you've then got two places > to debug instead of one. > > >> 2) if subscription state gets out of sync, the protocol will behave >> better or worse than the current version :), it really depends which >> roster is correct. There was a thread 2 or 3 weeks back with a >> tentative solution to keep subscription state in sync using the >> initial . >> >> > I'm not sure your initial analysis is correct, and it's related to the > above comments I make - if rosters are out of sync now, while it's tricky > to discover, you know where the blame lies instantly in each case. > > And a rough analysis suggests that in the current situation, you'll > either get presence when you *no longer* wanted it, or else you won't > get presence when you want it. With a reverse roster lookup, it's > possible to end up in a situation where you'll get presence when the If the server does route the wrong information to the wrong people it's buggy. With a reverse roster lookup on a corrupted database anything might happen. I don't get why this would justify the horrible amount of redundant information that is sent. I want to tell my 10 contacts on jabber.org that I am online. So sending 1 stanza will be 10 times less parsing, less bandwith, less CPU cost for compression, than sending 10. That factor scales with the number of contacts on a roster, which is linear, which is incredibly bad for something that can be accomplished with O(1) complexity. If the problem is synchronization of subscription states, well, then fix the problem with synchronization of subscription states. > sender doesn't want you to, which is substantially worse, and without > any bad actor involved. (I could be wrong here, please check this.) So we argue to "optimize" the protocol for this case? "Hmm, the other side might get information I didn't want to tell him, well, lets send it 10 times instead of once. I guess that will 'fix' the problem." I understand what you mean: Going from 10 to 1 message will indeed be a privacy problem in case of lost 'unsubscribed' stanzas. But the solution is to ask why the unsubscribed stanza was lost. XEP-0198 will address at least parts of this problem. >> But let me clarify something: although the bandwidth and processing >> savings of a multi-cast approach to presence distribution should be >> relevant, I don't believe this is compatible with a lot of other >> protocols that we have, so although I think multi-cast presence is a >> worthy long-term goal, I don't think we are ready to address it at >> this moment. > > We have this famous thing about ~87% (the figure varies) of traffic > being presence, and I'm afraid I think these figures skew vastly the > moment compression takes effect, since, by the very nature of the data > we're sending, it compresses really well. In fact, I'm not sure that it's > even worth worrying about as a problem, given how well this should > compress. Oh yay, let's burn some more CPU cycles, energy is sooo cheap. Everytime you send out 10 instead of 1 presence stanzas an additional amount of warmth is added to the global warming. Compression is a good optimization if you don't have anything else to save anymore and really want to pay for the CPU time. But in this case, wouldn't be preventing the redundancy in the first place be more beneficial in the overall performance? > I basically refuse to consider any solution seriously unless you're > measuring the effects of it post-compression - and I agree that's hard. Everytime you compress a redundant presence stanza a tree dies. Please think of the trees. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Tue, Mar 31, 2009 at 08:17:43AM -0700, Justin Karneges wrote: > On Tuesday 31 March 2009 08:07:15 Fabio Forno wrote: > > On Tue, Mar 31, 2009 at 4:05 PM, Peter Saint-Andre [.snip.] > > Of course we don't want to require acks unless they are explicitly requested. > > Let's not veer too far off the path here. All we're trying to fix is the > atomicity of 'u'. > > Proposed change: Don't send 'u' over the wire anymore. The value shall be > derived by the number of stanzas sent, starting from 0 (no stanzas sent yet). > > C: > C: > > S: > > We'd still have and ack elements. They'd just only ever contain 'h' > now. +1 - Sounds good and clean to me :) Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Question on 3921 and how the connection between servers and clients is maintained.
Hi Thomas! I'll try to answer some questions, but I'm just a client developer, so I might be wrong with some answers :) On Tue, Mar 24, 2009 at 10:20:10PM -0400, Thomas Bolioli wrote: > All, > I am chair of a standards body developing a standard for allowing > devices to maintain their presence on the network while in a low power > state. Here is more detail on what we are looking to accomplish. > http://www.ecma-international.org/memento/TC32-TG21.htm > http://www.ecma-international.org/memento/TC32-TG21-M.htm > > I am looking for some information on XMPP/Jabber and how it behaves in > certain circumstances. I have read through the RFCs and > http://xmpp.org/protocols/ but couldn't find the specific answer to my > question so I figured I would ask here. Most of the info I found > pertained to layers above layer 3/4 and it is the behavior of XMPP @ > layer 3/4 that is most interesting for us right now. > > So here goes: > Assuming for a moment this IM/presence scenario and keeping in mind that > IM can be replaced by any service/resource provided by an XMPP extension > and the use case should remain valid. I also understand XMPP to > implement a SIP like protocol. > > Use case: Julie has her IM client up on client A. Client A is connected > to server B. Jim is on client Z connected to Server X. > In todays world, if client A goes to sleep/standby after a period of > inactivity, her connection to Server B is severed. If Jim sends her a > chat request or other message, that message is either terminated @ > server B or held in lieu of Julie bringing the system back into a > working S0 state. Lets say for a hypothetical, that when Julie's system > went to sleep, it no longer dropped off of the network (ie; it > maintained it's IP, etc). Julie is waiting for directions for the > restaraunt where she will be meeting Joe. Julie is off reading a book > and is to leave once she gets word from Joe with the location. Yeah, > there will be better ways to get this info to Julie but lets assume cell > phones with SMS haven't been invented yet. :-) After 20 minutes her > system (client A) enters sleep and the IM client drops its connection to > the server (Q.1 Is there a connection to drop?). There will now be a Yes, Client A communicates in XMPP via a TCP connection with Server B. The problem is, that as soon as Client A drops the connection the resource of Client A is broadcasted as unavailable and all messages to Julie are either sent to other resources or stored on the Server B. > delay between when the information arrives on Server B and when Julie > gets it because Julie needs to wake up client A in order for the IM > client to check in and retrieve the cache of backlogged messages in the > queue on server B. Q2. Does XMPP have a facility for Server B to send a > message to client A when all are on public IPs? There is no communication channel between Server B and Client A other than the TCP connection that is initiated by Client A. I currently don't know about any servers or extension protocols that 'notify' the Client A when it is offline. > Q3. When client A is > behind a NAT (ie; does XMPPs version of SIP have nat traversal)? What do you mean btw. with 'SIP like protocol' above and 'XMPPs vrsion of SIP' here? (I'm not too familiar with SIP). All client2server connections are initated and maintained by the client software. The server has no way of contacting a client when it is offline. It's not like SIP that clients are always running and can be contacted directly when they are not connected. The only thing that afaik has to do with SIP in XMPP is the extension protocol for audio/video/file-transfer between two clients on the network. That protocol (Jingle) uses ICE to create a communication channel between Client A and Client Z. But no such protocol exists for Client A and Server B - at least as far as I know. > So if > the message is truly pushed to client A from Server B, client A simply > needs to wake up to gather/respond the incoming message and the PC > sitting next to the reading Julie will alert her within a short timespan > that Joe has sent his message to her. Since client A never left the > network, IP routing is still possible and the TCP SYN is routable to the > waiting client (assume a route was available to redirect from the router > to the PC in the case of a NAT scenario). > Q5. The client and server communicate via SOAP or a SOAP like XML based > transactional protocol over TCP, correct? Not at all. The protocol on the wire is a pseudo-XML "streaming" protocol, which uses the nesting of XML tags for framing. Messages, presence information and other queries are curr
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 05:03:29PM -0700, Justin Karneges wrote: > On Thursday 19 March 2009 16:52:14 Robin Redeker wrote: > > Thats the same as this IMO: > > > > C: > > C: > > C: > > C: > > C: > > > > S: > > That's not reliable though. What you want to do is this: > > C: > C: > C: > C: > C: > C: > C: > C: > > That way, if the connection dies in the middle somewhere, it can be resumed > properly. So, you want to be able to indicate 'u' without requiring that the > other side ack immediately. > I thought, from reading the XEP, that the sequence number indicates the number of stanzas sent. It seemed so obvious to me :) +1 for the idea in the other part of this thread about making the 'u' update implicit with each stanza. It never felt safe to me to seperate the request for acknowledgement from the thing that is to be sent over the stream. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 04:35:17PM -0700, Justin Karneges wrote: > On Thursday 19 March 2009 16:08:00 Robin Redeker wrote: > > On Thu, Mar 19, 2009 at 01:49:31PM -0500, XMPP Extensions Editor wrote: > > > Version 0.6 of XEP-0198 (Stream Management) has been released. > > > > My main problem is to understand the purpose of being able to > > include u _and_ h in a or element. If each has to be > > answered with an OR it seems a bit redundant to me to make > > two different elements. > > The reason there are two elements is because you may not want to request an > ack every time you bump the counter. For example, on an otherwise idle > connection, you could send followed by , and that's it, and the > other side has no obligation to ack anytime soon. Maybe after sending 50 > messages you'd throw an out there just so you can finally cross off the > messages in your outbound queue. This is more optimized than having the > other side ack every message. Well, if you don't want to request, just not send a new u attribute with the element I proposed: C: C: C: C: C: S: Thats the same as this IMO: C: C: C: C: C: S: As the RFC says, an can be acked by or an element with an 'h' attribute. So the real difference is only whether either an 'u' or an 'h' value is present in the or element. The receiving side of an u attribute knows that it has to tell the sender (with an h attribute) that it received that sequence number. Or do I miss something? Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 01:49:31PM -0500, XMPP Extensions Editor wrote: > Version 0.6 of XEP-0198 (Stream Management) has been released. My main problem is to understand the purpose of being able to include u _and_ h in a or element. If each has to be answered with an OR it seems a bit redundant to me to make two different elements. So in the end wouldn't a (standing for 'synchronize') element be enough: C: C: S: S: S: S: S: S: C:- here a client _might_ want to wait a certain amount of time, then, if the second message from the server has been received it could omit the synchronization element for the h='1' and could just send this one: C: C: C: That kind of combining the u and h attribute would require additional logic. So a primitive implementation might send a each time it received an with an u attribute: C: C: S: S: S: S: S: S: S: C: C: C: C: C: In the end it's just synchronization of two counters over a wire (along with keeping track what the other entity received if it received my counter value). Other than this I've found the XEP to answer all my immediate questions that arised when I read the previous version. I guess it's time to implement it :) Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Inconsistent Subscriptions in XMPP
On Tue, Feb 24, 2009 at 01:49:50AM +0100, Pavel Simerda wrote: > There are several cases when subscription databases in XMPP are > inconsistent. > > You may view subscription information as a global distributed database. > Subscription state between two JIDs, for example a...@a and b...@b are stored > in two places at the same time. Servers A and B maintain their own > copies of subscription state. The problem is not just subscription state, but any state that is not synchronized, basic presence information comes to mind, they also get out of sync sometimes. Maybe it's less of a problem, because a client logging in again, can refresh that state. Subscription information of course isn't as easily 'synchronized'. I don't know whether servers already do this, but having a consistent state for presence, would allow for caching the presence information on the receiving server. A generic synchronisation protocol would of course help any state information that has to be shared with other servers. I brought this topic up some weeks ago (and I think also a year ago), but I'm not a server developer (If I was, I would probably have already implemented some proof of concept protocol to do this). But these things don't seem to bother the (and/or developers) users much and there doesn't seem to be much (or enough) interest for consistent information sharing amog servers. Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] presences and server restarts
Hi! I've recently tripped over another really annoying ghost presence while I was chatting with someone. He always showed up as online, while he actually wasn't, just that I never got a 'unavailable' presence stanza and wondered whether my client was actually broken (which it wasn't as a glance on the stanza log indicated). It was a private server on the other end, so something that easily restarts/crashes from time to time. So it might be one of those ghost presences. The issue of ghost presences is of course known, and I don't know currently I was reminded of the concept of rpc.statd by a friend: It's there to inform other NFS server about unclean restarts. So, in XMPP that would be: A server telling the other server that he would love to receive a message when the other server restarts/crashed. Then the servers that receive that notice could send presence probes again for the contacts on the server that restarts. That would probably be at least less overhead than re-probing again and again, like it's suggested in rfc3921bis: In addition, a server MAY periodically send a presence probe to a contact if it has not received presence information or other traffic from the contact in some configurable amount of time; this can help to prevent "ghost" contacts who appear to be online but in fact are not. I don't know if this "problem" has already been solved, I couldn't find much on a quick look over the XEPs. Just wanted to throw in another idea, just in case... Greetings, Robin Redeker -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Messages to unsubscribed contacts and resources
Hi Joe, thanks for the reply! On Fri, Dec 14, 2007 at 07:09:26AM -0800, Joe Hildebrand wrote: > > On Dec 12, 2007, at 7:04 AM, Robin Redeker wrote: > >Ok, I have another question: > > > >We have: Contact A with one resource (resource is 'res1'): > > > > [EMAIL PROTECTED]/res1 > > > >And Contact B with two resources: > > > > [EMAIL PROTECTED]/res1 priority 20 > > Assume you mean contact_b. Yes, indeed :) > > > [EMAIL PROTECTED]/res2 priority 10 > > > >They are not subscribed to each other and do NOT share any presence > >information. > > Even directed presence? The cool side effect of sending a directed > presence is that everyone to whom you have sent directed presence will > get an unavail when you go offline. Yes, even directed presence not. I know that directed presence makes things easier in this case, but I assume here that there are still clients and people that don't do or don't want to send the presence. [.snip.] > >Should he make a second 'chat session', which is probably not very > >intuitive for the user because he just wanted to talk to contact B and > >when he got two options to send the message, what should he do (he > >doesn't even know which resource is the 'right' one, or whether 'res1' > >maybe went offline)? > > You're correct that there shouldn't be a new visible session. There > were clients that did that back in the day (or with each new thread!) > and it turned out to be pretty confusing. You should just rebind to > the new resource. Ok, that also makes things easier in the code. > >What if they share presence, should contact A's client determine from > >the highest priority resource where the message should go to? > > Absolutely not. You should just unbind at least when res1 goes > offline, and perhaps whenever you get a presence change from > contact_b. Keep in mind that b's server might be doing much more > sophisticated "most available" calculations. But what if contact B first replied from the resource 1 and also sent a directed presence to contact A, _and_ then changes to resource 2 to talk to contact A, should contact A rebind (I would guess that would make sense)? > Alternate question: What if contact_b's server is using RAP > (XEP-168)? Then contact_a can know what the new most available > resource is? My answer to that is that if the resource contact_a is > bound to was the primary resource, and now isn't, you should unbind, > otherwise not. Well, what in the case I explained in my last paragraph (about contact B, who changes the resource without changing presence settings)? Should contact A still send to the primary resource or still rebind to the last resource he got a message from? Sorry for my complicated and very technical questions, I just want to get it right :) Thanks for your time! Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Messages to unsubscribed contacts and resources
On Tue, Dec 11, 2007 at 10:28:55AM -0700, Peter Saint-Andre wrote: > Robin Redeker wrote: > > Hi! > > > > Some days ago I had a mail discussion on the jdev@ mailing list > > about messages to unsubscribed contacts and contacts in general. > > Tomasz said that messages should generally go to the bare JID instead of > > the full JID, and that the local routing is up to the server. > > > > He meant, chat sessions should be kept track of with the > > element, which completly makes sense to me. [.snip.] > > Please also note that the term 'chat session' in that paragraph is quite > > undefined, or at least it's meaning is a bit fuzzy to me. > > At its simplest, I think this is a chat session: > [.snip.] > > The sharing of directed presence gives both parties more knowledge about > availability, but I think it should be initiated by the recipient (and > subject to client prompt on a per-session basis, or configuration in the > client to auto-share directed presence). > [.snip.] > > Of course, If I don't have to keep track of the resources, that would > > _greatly_ simplify everything for me. Just sending to the bare JID and > > leaving the rest up to and the contacts routing settings > > would make enormous sense to me. > > Sending every message to the bare JID is not the custom. > > > Back to section 5.1.1, the sections somehow contradicts the section > > 8.3.1.1 (Message): > > > >For a message stanza of type "chat", "error", "groupchat", or > >"normal", the server SHOULD deliver the stanza to the > >highest-priority available resource. > > > > That 'feature' only makes sense if at least the initial message goes to > > a bare JID. But if it is routed to a resource by the server and I have > > no knowledge about the presence of that resource (eg. if I'm not > > subscribed), where should the next message go to, to the full JID I > > received a reply from? > > You should keep sending to the bare JID until you receive a reply from a > full JID, then start sending to the full JID. > Ok, I have another question: We have: Contact A with one resource (resource is 'res1'): [EMAIL PROTECTED]/res1 And Contact B with two resources: [EMAIL PROTECTED]/res1 priority 20 [EMAIL PROTECTED]/res2 priority 10 They are not subscribed to each other and do NOT share any presence information. Contact A sends a bare JID message to contact B. Contact B receives the message at res1 (due to highest priority), and answers from that resource. Contact A receives the full JID of that resource now and 'binds' the conversation to it and they chat. Later Contact B switches to his laptop, with another client bound to the other resource 'res2'. And he sends a message to the bare JID (because the client there doesn't know the full JID of contact A) of contact A. How should contact A proceed? Should he bind (and reply) the conversation to res2? Should he make a second 'chat session', which is probably not very intuitive for the user because he just wanted to talk to contact B and when he got two options to send the message, what should he do (he doesn't even know which resource is the 'right' one, or whether 'res1' maybe went offline)? What if they share presence, should contact A's client determine from the highest priority resource where the message should go to? Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Messages to unsubscribed contacts and resources
On Tue, Dec 11, 2007 at 10:28:55AM -0700, Peter Saint-Andre wrote: > Robin Redeker wrote: > > Hi! > > > > Some days ago I had a mail discussion on the jdev@ mailing list > > about messages to unsubscribed contacts and contacts in general. > > Tomasz said that messages should generally go to the bare JID instead of > > the full JID, and that the local routing is up to the server. > > [.snip.] > > Please also note that the term 'chat session' in that paragraph is quite > > undefined, or at least it's meaning is a bit fuzzy to me. > > At its simplest, I think this is a chat session: > [.interesting examples snipped.] Thanks for the examples, that was what I thought was the other option. > > The sharing of directed presence gives both parties more knowledge about > availability, but I think it should be initiated by the recipient (and > subject to client prompt on a per-session basis, or configuration in the > client to auto-share directed presence). > > > And is somewhere described how full JIDs and play together? If at > > all? > > http://www.xmpp.org/extensions/xep-0201.html Thanks :) > > If routing is up to the local server side, makes it sense to reveal > > resources at all? Wasn't there a progress towards randomized resources? > > http://www.xmpp.org/internet-drafts/draft-saintandre-rfc3920bis-04.html#bind-servergen Nice :) [.snip.] > > IMHO, threads are *not* required to have a chat session. Ok. > > But before I can implement anything resembling 'chat sessions' that > > term must be more explictly defined. > > See above for an example. I can easily write a formal definition too. Well, that example was what I imagined what a 'chat session' would be. I don't know how important this is for others, I guess most client developers don't have a problem with this. > > Of course, If I don't have to keep track of the resources, that would > > _greatly_ simplify everything for me. Just sending to the bare JID and > > leaving the rest up to and the contacts routing settings > > would make enormous sense to me. > > Sending every message to the bare JID is not the custom. Ok. > > Back to section 5.1.1, the sections somehow contradicts the section > > 8.3.1.1 (Message): > > > >For a message stanza of type "chat", "error", "groupchat", or > >"normal", the server SHOULD deliver the stanza to the > >highest-priority available resource. > > > > That 'feature' only makes sense if at least the initial message goes to > > a bare JID. But if it is routed to a resource by the server and I have > > no knowledge about the presence of that resource (eg. if I'm not > > subscribed), where should the next message go to, to the full JID I > > received a reply from? > > You should keep sending to the bare JID until you receive a reply from a > full JID, then start sending to the full JID. Ok, that makes much sense with the behaviour described in the URL you posted below. > > > Will my messages, if that contacts resource goes > > offline, be dropped without my knowledge? > > No, they will probably go to offline storage, because of this: > > http://www.xmpp.org/internet-drafts/draft-saintandre-rfc3921bis-04.html#rules-fulljid-availnomatch > > I agree that these customary practices are not spelled out very well, so > I will fix that in the next version of rfc3921bis. I guess that would be a nice addition. Thanks! Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] Messages to unsubscribed contacts and resources
Hi! Some days ago I had a mail discussion on the jdev@ mailing list about messages to unsubscribed contacts and contacts in general. Tomasz said that messages should generally go to the bare JID instead of the full JID, and that the local routing is up to the server. He meant, chat sessions should be kept track of with the element, which completly makes sense to me. On the other side RFC3921bis says: Section 5.1.1: If the message is being sent in reply to a message previously received from an address of the form <[EMAIL PROTECTED]/resource> (e.g., within the context of a chat session), the value of the 'to' address SHOULD be of the form <[EMAIL PROTECTED]/resource> rather than of the form <[EMAIL PROTECTED]> unless the sender has knowledge (via presence) that the intended recipient's resource is no longer available. That of course also makes sense, but my problem was: What if the sending contact does not know about the presence of that resource? When should he stop sending to [EMAIL PROTECTED]/resource? Should he, if he has no knowledge about the presence, send to [EMAIL PROTECTED] generally? Please also note that the term 'chat session' in that paragraph is quite undefined, or at least it's meaning is a bit fuzzy to me. And is somewhere described how full JIDs and play together? If at all? If routing is up to the local server side, makes it sense to reveal resources at all? Wasn't there a progress towards randomized resources? Sorry for so many questions, but I'm a little bit confused. I try to get the conversation aspect in my client right, and one problem I stumbled accross was the fact that my client has no 'windows' or 'tabs' which could control the extend of a 'chat session'. In my client there is always a 'tab' to a contact, and the extends of a 'chat session' are very fuzzy and undefined. If the term 'chat session' in XMPP means I have to keep track of the session via some special hacks with resources and , I would have to complicate the whole thing a bit. That is of course maybe only an issue with my special client. But before I can implement anything resembling 'chat sessions' that term must be more explictly defined. Of course, If I don't have to keep track of the resources, that would _greatly_ simplify everything for me. Just sending to the bare JID and leaving the rest up to and the contacts routing settings would make enormous sense to me. Back to section 5.1.1, the sections somehow contradicts the section 8.3.1.1 (Message): For a message stanza of type "chat", "error", "groupchat", or "normal", the server SHOULD deliver the stanza to the highest-priority available resource. That 'feature' only makes sense if at least the initial message goes to a bare JID. But if it is routed to a resource by the server and I have no knowledge about the presence of that resource (eg. if I'm not subscribed), where should the next message go to, to the full JID I received a reply from? Will my messages, if that contacts resource goes offline, be dropped without my knowledge? Thanks for your time :) Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Binary data over XMPP
On Fri, Nov 09, 2007 at 10:01:39AM -0700, Joe Hildebrand wrote: > > On Nov 9, 2007, at 8:47 AM, Tobias Markmann wrote: > > >There are already several binary-to-text encodings which perform a bit > >better than Base64, two of them are: > > > >1. http://en.wikipedia.org/wiki/ASCII85 invented by Adobe > >2. http://base91.sourceforge.net/ > > Both of those seem to allow < and &, which make them less than ideal > for embedding in XML. "XMPP is not XML" :-))) R
Re: [Standards] Binary data over XMPP
On Tue, Nov 06, 2007 at 10:16:19AM +, Dave Cridland wrote: [.snip.] > > You would no longer be able to do that with binary blobs; you > >would have to special-case blob stanzas fairly heavily, since I > >guarantee you that if the characters '<' or '>' appear un-escaped > >in the binary blob, Expat will choke and die. > > > > > Sure, but there's two options with an escaping mechanism - either > synchronized or non-synchronized - and they can be negotiated easily. > > With a non-synch mechanism, the sender just sends out the > element, then sends out the binary data, then continues with XML. It > can be done in a single TCP packet, but it requires that the receiver > processes the data into stanzas prior to processing through the XML > parser. Some receivers already do this, so it seems reasonable that > this can be an option. Also take into account that the sender also has to customize the XML writer to allow writing raw octets, which breaks multiple layers of nice cosy XML abstractions. (Of course with current XMPP you already need a XML writer that allows some customisations). > With a synch mechanism, the sender sends out a element, and > then waits. The receiver then says it's ready for binary data > (sending a stanza to indicate this), and the sender then sends the > binary data - followed immediately by more XML as required, since a > "binary parser" is going to be octet counting anyway. For people who > parse all the network traffic at once through a SAX-like parser, this > should work fine, at the expense of some efficiency. > > Note that anyone can send non-synchronized blobs, but not everyone > can receive them, so a client (for instance) which is built to stream > network data directly into a SAX parser can still *send* blobs > efficiently. How do you propose the receiver determines the end of the binary data? Is it going to be prefixed by a lenght? Generally: pumping binary crap through XMPP is another big step _away_ from XML compatibility. Also transforming a stream (TLS) into packets (stanzas) which are used to again emulate a stream (TLS) sounds crazy to me anyway :) To encrypt stanzas (packets) with TLS (stream encryption)... *shudder* :) Robin
Re: [Standards] Binary data over XMPP
On Mon, Nov 05, 2007 at 09:56:12AM -0800, Justin Karneges wrote: > On Monday 05 November 2007 3:40 am, Dave Cridland wrote: > > Now, we can't expect that the entire Internet will bend to our will > > and instantly upgrade, so we need a sane fallback - probably to IBB, > > or something fairly similar. The interesting question is whether we > > choose to have this negotiated end to end (which means we'll need to > > have each hop along the route tested), or whether we say that this > > down-conversion happens within servers. > > Binary over XMPP has been on my TODO for awhile now, and I have some notes > written up about it but nothing publicized. I think a hop-by-hop approach is > best, if we want to have any hope for compatibility. > > Comments on the two formatting approaches: > > 1) XML element to indicate binary mode: this is probably the least > destructive approach. Keep in mind that we already have an XML to binary > protocol change in XMPP: the TLS and SASL encryption layers. Your XML parser > needs to be able to stop on a dime when it sees that final '>' character, so > asking for that in this discussion should not be a big deal. Just keep in mind that we don't have a way to change "back". The current change is a very drastic one, like "flush the whole parser state and begin from start". Robin
Re: [Standards] Binary data over XMPP
On Mon, Nov 05, 2007 at 06:47:40PM +0100, Michal 'vorner' Vaner wrote: > Hello > > On Mon, Nov 05, 2007 at 06:27:33PM +0100, Robin Redeker wrote: > > On Mon, Nov 05, 2007 at 04:04:10PM +0100, Michal 'vorner' Vaner wrote: > > > > > > It is as much drastic approach as the blobs, it changes the protocol > > > from the very basic ground. Furthermore, you can extract the stanza and > > > feed it to any XML parser. > > > > +1 for "real" protocol frames! > > Actually, I was just showing, how deep change was the blobs thing. I'm > against changing the whole infrastructure inside out. I didn't mean to > propagate the frames, I just took them as example. Heh, ok. Just wanted to take the chance to promote the idea once more :) R
Re: [Standards] Binary data over XMPP
On Mon, Nov 05, 2007 at 04:04:10PM +0100, Michal 'vorner' Vaner wrote: > Hello > > On Mon, Nov 05, 2007 at 02:45:05PM +, Dave Cridland wrote: > > Another option would be to setup a distinct connection (and protocol) for > > routing blobs, and so send them through the server, yet not in-band. I'm > > not comfortable with this, because it means essentially duplicating all > > security information, and maintaining synchronization between two distinct > > streams. > > Or make the connection blobs by default, and some blobs could contain > complete XML documents, like this: > lenght of first block > > length of second block > > length of third block > some binary data. > > It is as much drastic approach as the blobs, it changes the protocol > from the very basic ground. Furthermore, you can extract the stanza and > feed it to any XML parser. +1 for "real" protocol frames! R
Re: [Standards] [Fwd: I-D Action:draft-melnikov-digest-to-historic-00.txt]
On Wed, Sep 12, 2007 at 01:53:37PM +0100, Ian Paterson wrote: > Peter Saint-Andre wrote: > >Ian Paterson wrote: > > > >>In real life servers will always be compromised (especially in cases > >>where the attacker is the service provider). So SASL PLAIN still > >>contains a serious vulnerability that is easily fixed in those cases > >>where DIGEST-MD5 is a practical option. > >> > > > >Except that DIGEST-MD5 is effectively being deprecated by the IETF. Thus > >the interest in SCRAM, YAP, and their ilk. > > > > With all due respect to the experts at the IETF, I feel (as a > non-expert) that they are trying to depricate DIGEST-MD5 before it has a > suitable replacement (i.e. another one that protects users' passwords > from a compromised server). I strongly agree we should recommend/require > SCRAM and/or YAP as soon as they are baked. But is that likely to happen > before 3920bis is puiblished? > > I agree that if we start recommending SASL PLAIN in addition to > DIGEST-MD5 now, *and if we continue to do so in the future*, then we can > ensure that current implementations will still be compatible with future > implementations that have removed support for DIGEST-MD5. > > However I don't understand why we are considering recommending weakening > the security of XMPP servers in the short and medium term by not > requiring any of DIGEST-MD5 or SCRAM or YAP. Are XMPP implementors > experiencing interoperability issues with DIGEST-MD5? One problem we encountered when talking about JID escaping and actually trying it in the real world: http://thread.gmane.org/gmane.network.jabber.standards-jig/16309/focus=16466 The most obvious solution would be: Clients and Servers that have broken DIGEST-MD5 implementations should fix them ASAP. Problem with that solution is: Even if the SASL implementations are fixed fast (within few weeks) there will still be a lot of old servers out there and old clients. That of course causes no direct interop problems as noone uses \ in their JIDs yet. Robin
Re: [Standards] file transfer
On Tue, Aug 28, 2007 at 09:21:43PM +0100, Richard Dobson wrote: > Peter Saint-Andre wrote: [.snip.] > >I understand. But SI and Jingle essentially perform the same functions, > >since they both enable negotiation. > > Yes but jingle also provides access to ICE and pseudo-tcp so it would be > helpful rather than have the entire file transfer negotiation using just > jingle if it was separated in two so that all the jingle side is doing > is negotiating a generic bytestream that can be then reused by things > like SI, to me this seems like a sensible compromise as there is no real > benefit that I can see of doing the whole thing in jingle as its just > duplicating effort, whereas creating a new bytestream method is > complimentory, anyone else think its a good idea? I generally thought that reimplementing all that with jingle is a bad idea. And I as client/library author surely don't like the idea of duplicating code or functinality. I agree completly with you. Robin
Re: [Standards] Comment to rfc3921 pt 11.1 : handling of messages to ressources with identical priorities
On Fri, Aug 24, 2007 at 12:11:09PM -0600, Peter Saint-Andre wrote: > Robin Redeker wrote: [.snipped arguments for broadcasting.] > > I actually mostly care about sane and _clear_ and simple semantics for > > handling cornercases. It clearly defined what users, developers and > > admins can expect in such cases. > > This feels like an implementation detail to me. At the least, I would > not want to say that if two resources have the same priority then the > server MUST deliver to both resources, because that closes the door to > more intelligent routing based on advanced knowledge that the server may > have from user-configured routing rules or the like. The RFC defines whether this ia an implementation detail or not. Currently the RFC delegates handling of this corner case to the implementors. If servers are free to handle this case any way they want they will handle it differently. Users won't have a clue what will happen. As someone on this thread posted: they even expected broadcasting in the first place. That expectation might be an exception, haven't made a poll about this yes :-) Without a SHOULD the users, developers and admins still can't rely on a defined behaviour. (defined in a sense that users (and also their clients) have no idea what behaviour the server is or might be implemented for this case). All I want is the ability to tell "Aunt Tillie" what she can expect when she has two clients with the same priority. If there is nothing to expect this will happen: Asking her: "Oh, I don't know what might happen, which server software and settings has the XMPP server you are using?" She: "Uhm, I don't know, there is no website." Me: "Oh, just ask the admin!" She: "Will do that..." *writes mail/xmpp message to admin* Admin: "We send the message to a random resource if the priorities are the same." She: "Thats bad, I don't want that. What should I do?" Me: "Easy, just pick a server which does the right thing." She: "Which would be a server that just broadcasts the messages to all resources?" Me: "Easy, just go to http://www.jabber.org/user/publicservers.shtml and send mails to all admins to ask them." She *does that* She: "Ok, got it now, 12jabber.net does it. I've created an account there now, but now I have to move my roster there and reprint my vcards..." I don't know what the user will expect, some seem to expect broadcasting other some clever algorithm like: "last resource that sent a message", or whatever. But they certainly won't have much abilities to find it out or rely on a default behaviour. Maybe best would be to make the default behaviour broadcasting and then specify a way to configure the handling of this case by eg. ad-hoc commands on the server for an account. Then the RFC could say: "Unless other behaviour was configured for an account the server SHOULD/MUST broadcast the message to all resources with same priorities." That might be a bit overkill, but it would be a defined, simple and most reliable default handling for that case. And with per account configuration it would even be open to future "more intelligent" routing. R
Re: [Standards] Comment to rfc3921 pt 11.1 : handling of messages to ressources with identical priorities
On Thu, Aug 23, 2007 at 04:44:38PM -0600, Joe Hildebrand wrote: > > On Aug 22, 2007, at 1:16 AM, Robin Redeker wrote: > >Will that algorithm be a 100% solution? > > There is no such thing. But it works really well in the real world, > with real customers, in real deployments. As well, please make sure > that you read the thread that started with JD's priority discussion > here: http://urltea.com/1av0; this is a problem more for client > developers and sysadmins configuring branding files than for end-users. This is a cornercase that must be handled in a sane way in servers IMO. That thread is interesting but doesn't solve the issue of two resources having the same priority. As far as I understood it was about priority dances in clients and such. You say sysadmins should configure it, are there configuration variables that enable 'sane' handling of same priority resources? And yes, I'm sure that heuristics and weird algorithms usually work in the real world out there, but if they don't users probably won't know what to do or what it could be. I actually mostly care about sane and _clear_ and simple semantics for handling cornercases. It clearly defined what users, developers and admins can expect in such cases. R
Re: [Standards] Comment to rfc3921 pt 11.1 : handling of messages to ressources with identical priorities
On Tue, Aug 21, 2007 at 11:02:09AM -0600, Joe Hildebrand wrote: > > On Aug 21, 2007, at 5:15 AM, Remko Tronçon wrote: > > >When 2 resources are logged in with the same priority, it's probably a > >lot worse to just let the server pick one, because it will pick the > >wrong one in 50% of the cases. > > Not if you use a better algorithm. You've got information from > presence that helps you make a better choice than coin-flip. > Timestamps, show, etc. are really good at tie-breaking in the real > world. Will that algorithm be a 100% solution or will it just reduce the chance of picking the wrong resource? I think that the solution to this problem should be a 100% one, because it's an essential part of XMPP to deliver messages. I would rather like a deterministic 'send to all resources with the same priority' than some heuristic (which can fail) on the server side the user doesn't know about. If the proposed algorithm goes into a 'best practices' document we will likely end up with only 'some' servers implementing leading to a inconsistent experience for the users of those servers. I don't expect users to read some website somewhere to figure out whether the server does what he expects. Some users of course don't even know what to expect. But the 'send to all resources with the same prio' approach will seem consistent to them and I argue that this will give the user a more safe feeling that he just gets the message than having to worry about servers doing funny stuff to figure out where a message should go to. About the nice 'sell 50 shares' example: There is delayed delivery indeed. Will your proposed algorithm provide a 100% solution to the problem and is also the most likely thing a user expects? Can it also be explained to a user easily? Or will we have to tell them "there is some algorithm which is around multiple pages long that determines whether you get that message or not"? Robin
Re: [Standards] IMML
On Mon, Aug 06, 2007 at 12:17:39PM -0700, Rachel Blackman wrote: > >XHTML-IM is for sending HTML messages. IMML is for sending modern > >Instant Messages. IMML intentionally leaves out most of the > >flexibility > >that XHTML-IM provides, most of which has no semantic meaning > >whatsoever. We might as well be using XSL-FO. > > > >Imposing rules such as you suggest for HTML a's just adds to the > >complexity of implementation, and illustrates that HTML in any form is > >simply the wrong tool for the job. > > This is an argument that will never be resolved. We have people who > want to be able to put every single HTML object (including Java, > Active X, DHTML and so on) into messages and feel XHTML-IM is far too > restrictive and anemic; > they argue you should be able to paste HTML > in from Firefox and have it display exactly as it was. Then every XMPP client would have to embed the firefox rendering engine for that. Displaying HTML (mostly any kind, XHTML, HTML 4...) in different browsers will lead to different results (in the real world). Maybe they should post screenshots instead :-) ? > And then we have people who feel XHTML-IM is too complex and is > overkill for the situation. > > XHTML-IM manages to strike a (somewhat sensible) middle ground, which > is probably why it will stick around. :) There is one big difference between XHTML-IM and IMML, which are simply a matter of semantic. XHTML-IM says how to exactly display a text, what color, what size, what padding, where to place an image, or whatever. IMML just says "display this with emphasis", "and this is an URL, display it approriate". Clients can then choose how to put emphasis on some text (eg. by rendering it bold). XHTML-IM doesn't leave such a choice, either the client can render something bold or he has no way to display a bold element. I also disable XHTML-IM always, because the people seem to never get it right to send just normal text. Often their text is smaller than mine and some even think it's funny or very readable to have some big and colorful text. While I agree that lots of todays youth just loves these gimmicks and that it might be important to support colors and images and all that, I also want to have a middle way between XHTML-IM and no formatting at all. IMML is seems to be restricted enough to provide a reliable way to display an instant message while not enforcing how something might be displayed. For XHTML-IM one must roll a more or less complete XHTML-IM parser and CSS1 parser. I agree that it is not impossible to implement it due to the very limited set of XHTML and CSS1. But XHTML-IM leaves the destination client almost no freedom in displaying the text. A console client will have no other choice than just ignoring the XHTML-IM content of a message or run some heuristics over the html and somehow find out what color or attributes a text might have. IMML is way easier to implement, and clients and users still have choice how to display something with emphasis. IMML moves the choice to the receiving end, XHTML-IM leaves the choice to the sender. It's mostly a semantic difference. And afterall, instant messages are usually not documents. Just my 2 cents. R
Re: [Standards] summary: allowable characters
On Sun, Aug 05, 2007 at 12:44:27PM +0200, Matthias Wimmer wrote: > Robin Redeker schrieb: > >But the problem that we have then is that a 1.0 server, which will only > >receive escaped JIDs, stores those (escaped) 1.1 JIDs in the 1.0 contacts > >roster as escaped JIDs (because he doesn't even know they are escaped > >JIDs!). > > > >That means that after migration (1.0 server upgrades software to a 1.1 > >capable one) those JIDs are still in their escaped form in the roster. > >And because there doesn't happen any escaping between 1.1 servers our > >recently upgraded 1.0 server has broken rosters. > > I'd say that this just means that when migrating from a 1.0 server to a > 1.1 server, that you have to convert the roster as it is stored on the > server. > > This is something the server implementation has to handle. E.g. by > flagging rosters that are created on the 1.1 software are that have > already been converted. Whenever the software reads a roster without > this flag, it knows it has to convert the roster first. > > Sorry, but I don't see the problem. Some sort of the same logic is also > needed for converting account names from a 1.0 server on upgrade to a > 1.1 server. > The problem is that you don't know which JID was a 1.1 JID before it was escaped. So you would unescape every JID in the roster, leading to falsely unescaped JIDs. Of course we currently don't have many JIDs with a \ in the nodepart, and probably even less which would represent an escaped JID. But you will have false positives, like jabber.org currently has '[EMAIL PROTECTED]', that will be unescaped with your logic when migrating to 1.1 to 'stpeter @jabber.org' which would be wrong IMO. Robin
Re: [Standards] summary: allowable characters
On Sun, Aug 05, 2007 at 05:10:13AM +0530, Mridul Muralidharan wrote: > Robin Redeker wrote: > >On Fri, Aug 03, 2007 at 04:29:15AM +0530, Mridul Muralidharan wrote: > >> > >>Just mentioning a basic problem which was discussed at jdev. > >> > >>If two 1.0 server move to 1.1, all the 'older' 1.0 jid's will become > >>unroutable - which are present in user > >>roster/affiliations/privacylists/etc. > >> > > > >Yes, this sounds like the death blow for escaping for backward > >compatibility. It will poison the old 1.0 servers and make whole roster > >subscriptions unusable once that server upgrades to 1.1. (Not to mention > >the JIDs in the private XML storage or other places you mentioned). > > > >Do you see any problem in just disallowing incompatible 1.1 JIDs to be > >able to communicate with 1.0 JIDs? The old 1.0-compatible JID accounts > >on a 1.1 server will of course still be able to talk with people on 1.0 > >servers. > > The problem is 1.1 JID's cant communicate with 1.1 contact JID's - > if user has [EMAIL PROTECTED], what will the 1.1 server do ? It could > either be pointing to a 1.1 [EMAIL PROTECTED] (route as-is), was a 1.0 > jid - convert to cont&[EMAIL PROTECTED] (needs transformation) or continues > to > be 1.0 [EMAIL PROTECTED] (route as-is) (all three as different cases, > though 1 and 3 look the same). I don't know exactly what you mean. There MUST NOT be done any escaping between two 1.1 servers. So a 1.1 contact on server A and another 1.1 contact on server B are communicating nicely with each other. And also the JIDs in their rosters will be a 1.1 JID. Escaping only happens between 1.1 and 1.0 servers, and only on those 1.1 vs. 1.0 borders. But the problem that we have then is that a 1.0 server, which will only receive escaped JIDs, stores those (escaped) 1.1 JIDs in the 1.0 contacts roster as escaped JIDs (because he doesn't even know they are escaped JIDs!). That means that after migration (1.0 server upgrades software to a 1.1 capable one) those JIDs are still in their escaped form in the roster. And because there doesn't happen any escaping between 1.1 servers our recently upgraded 1.0 server has broken rosters. > > > >The network won't be split the day servers start speaking XMPP 1.1. > >By preventing people with JIDs with incompatible characters to speak > >with 1.0 servers the 1.1 servers can prevent that split. > > Existing data will be present - and without jid meta-data, we cant > associate encoding info. > > One possible option would be to move to use uri scheme for jid's - (and > so this could be the differentiator for 1.1 vs 1.0). > More importantly, it would help in case of interop with other protocols. > > Last time I brought this up, it was considered a bit too disruptive, and > so dropped :-) Since Peter was considering 1.1 of xmpp, maybe this would > be a good time to rethink this idea ! You mean if a 1.1 server speaks with a 1.0 server it gives him an URL instead of an escaped JID? How will 1.0 servers handle that? They will have to code a little to handle that, don't they? (I like the idea of URLs btw., but changing a whole culture from their JIDs to URIs will be quite hard I guess :-) Robin
Re: [Standards] summary: allowable characters
On Fri, Aug 03, 2007 at 04:29:15AM +0530, Mridul Muralidharan wrote: > > > Just mentioning a basic problem which was discussed at jdev. > > If two 1.0 server move to 1.1, all the 'older' 1.0 jid's will become > unroutable - which are present in user roster/affiliations/privacylists/etc. > Yes, this sounds like the death blow for escaping for backward compatibility. It will poison the old 1.0 servers and make whole roster subscriptions unusable once that server upgrades to 1.1. (Not to mention the JIDs in the private XML storage or other places you mentioned). Do you see any problem in just disallowing incompatible 1.1 JIDs to be able to communicate with 1.0 JIDs? The old 1.0-compatible JID accounts on a 1.1 server will of course still be able to talk with people on 1.0 servers. The network won't be split the day servers start speaking XMPP 1.1. By preventing people with JIDs with incompatible characters to speak with 1.0 servers the 1.1 servers can prevent that split. The 1.1<->1.0 gap will grow with people who want to use the new characters in their JID, and hopefully the server administrators also upgrade their servers at the same speed that these people come. Clients would also have to take care whether they speak to a 1.0 or 1.1 server. A client error message like: "your server doesn't support these characters in the JID, convince the admin to upgrade!" will maybe even raise the pressure for admins a bit to upgrade :-) The problem with forcing admins to upgrade I see here is that they are maybe forced to upgrade to a unstable version or not so stable version as they had before. Robin
Re: [Standards] MUC rooms on roster.
On Fri, Aug 03, 2007 at 11:30:05AM +0200, Tomasz Sterna wrote: > Dnia 03-08-2007, pią o godzinie 11:10 +0200, Robin Redeker napisał(a): > > Hm, and what is about http://www.xmpp.org/extensions/xep-0048.html ? > > Hmm... I had always found this JEP awkward. > We have roster to store these kinds of things. > Early versions of jabberd/2 allowed storing an element with the > roster item, that clients could store the additional item information. > But it was explicitly forbidden by Standards JIG, thus removing support, > which in turn resulted in JEP-0048. Just some random thoughts here... I also find XEP-0048 a bit awkward, but the limitation of the roster for presence subscriptions seems to let no room for other informations stored there. See also http://www.xmpp.org/extensions/xep-0083.html (Nested Roster Groups) which I find equally awkward! See also http://www.xmpp.org/extensions/xep-0057.html, which is probably what you meant with storing additional elements with the roster. Also interesting is the way tkabber stores notes for roster items: test fwfwefew Private XML storage is IMO a good solution for that. But I've not seen any XEP for this kind of note storage (maybe I just couldn't find it). Applications have to sync the roster with other places where information for that JID is stored. I also miss a registrar for this kind of usage of private XML storage. Also note that the private XML storage will become more and more messy with the number of different programs storing information there. Does a way exist to purge all information in the XML storage? Or even list the stored information? Listing stored information would allow the removal of unknown elements by clients. > > That works (see below) but there is a more pressing problem here: > > [...] > > This is why I asked, to hear others feedback and issues. > Could we use standard roster for it, or we really have to stick with the > XEP-0048 defined alternative roster. Well, back to the issue: I would not like to join every MUC I have on my list just because I'm available. I would still like some MUCs only optionally joined and not automatically. How should that work? By unsubscribing from the MUC and not delete the roster item? Would work I guess. The problem here is XEP-0045 which defines the element for MUC-ability-signalling. I see no issues in storing many other things in the roster, but I see issues with making that work with MUC the way MUC is defined (which I don't really like either, but can't tell yet why :-) Robin
Re: [Standards] MUC rooms on roster.
On Fri, Aug 03, 2007 at 10:13:20AM +0200, Tomasz Sterna wrote: > I was talking with Grégoire Menuel (mu-conference developer) about > implementing Peter's idea of MUC rooms as items on the roster. > > Basically the idea is to teach MUC component to answer to subscription > requests. > So when you add [EMAIL PROTECTED]/nick to the roster and send the > subscribe request, MUC server replies subscribed+subscribe to you. When > you accept, you have a room on the roster with subscription "both". Next > time you broadcast your presence, you automatically join the room. Hm, and what is about http://www.xmpp.org/extensions/xep-0048.html ? That would still be useful to store non-autmatically-joined MUCs I guess? Also that XEP offers also a way to auto-join MUCs. > There is a problem on the client side though. > How would client know, that the presence from [EMAIL PROTECTED]/* are from > room participants, not from many resources of [EMAIL PROTECTED] user (not > on roster, thus ignored). That works (see below) but there is a more pressing problem here: Example 17. Jabber User Seeks to Enter a Room (Multi-User Chat) How does the client or the server know that he has to include such an element in the presence? If the client does send it's initial presence it of course won't include that element. So either the client sends directed presence or the server knows somehow what [EMAIL PROTECTED] is, so it can include the element. Exploiting the fact that presences without also work for joining a MUC would at least break this SHOULD in XEP-0045: MUC clients SHOULD signal their ability to speak the MUC protocol by including in the initial presence stanza an empty element qualified by the Also there is this issue: Before attempting to enter the room, a MUC-compliant client SHOULD first discover its reserved room nickname (if any) by following the protocol defined in the Discovering Reserved Room Nickname section of this document. The client has to do that before sending initial presence, right? If those issues, and some I don't know yet, are cleared up the client can easily recognize the presences from occupants by the element: Usually a participant presence looks like this: Or the update looks like: xa gone where the goblins go With the element the client can figure out that this is a presence that comes from a MUC. Robin
Re: [Standards] summary: allowable characters
(Warning, long mail ahead! Get a coffee and some time first :-) On Thu, Aug 02, 2007 at 03:34:30PM -0600, Peter Saint-Andre wrote: > Mridul Muralidharan wrote: > > The problem essentially is that any place where we have a JID persisted > > in the backend (user roster, acl's, affiliations, privacy lists/block > > lists, etc), it will become incompatible change. > > For example, what used to be [EMAIL PROTECTED] will now become > > contact&[EMAIL PROTECTED] - causing incompatibilities. > > Well we're having a long discussion about this in the jdev room > right now: > > http://www.jabber.org/muc-logs/[EMAIL PROTECTED]/2007-08-02.html > > I volunteer elmex for posting a summary once we're done. :) Yes, basically Mridul is completly right, we can't do much about the already deployed backslashes in JIDs. Especially in 1.0 server rosters. But... First we have to wonder whethere there are actually people with [EMAIL PROTECTED] in their roster, as registering a JID with a \ in the username is a considerable problem with XMPP 1.0 servers with SASL and DIGEST-MD5 (see some older message from me in the JID escaping thread). Of course that should be further investigaged as old-style IQ auth works with [EMAIL PROTECTED] and also some jabberd2 servers allow authentication as [EMAIL PROTECTED] without problems. But there exists a possibility to migrate our old JIDs to the 1.1 world and staying interoperable with 1.0 servers: First: A 1.1 server that is going to communicate with 1.0 server will escape the JIDs from his userbase when he SENDS to a 1.0 entity. Escaping can be performed as described in XEP-0106 (after dropping the silly \20 escaping rule). That will work great if the 1.1 server has NO old userbase. If we have for example jabber.org, a large userbase, and there is actually [EMAIL PROTECTED] as registered user in. And we want to upgrade to a 1.1 server then we will run into the problems Mridul pointed out: 1.0 servers have [EMAIL PROTECTED] in their roster, and if we have now 'stpeter @jabber.org' registering a new account he will collide with that, because his JID will be escaped to the in the 1.0 servers roster existing [EMAIL PROTECTED] Bang, we got a collision. There exists no real easy way to prevent that except just not allowing 'stpeter @jabber.org' to register. To detect a case like this, that a new user with a colliding JID registers, the 1.1 server needs to keep track of the old JIDs in his database. If the 1.1 server knows that [EMAIL PROTECTED] is a JID from the pre-1.1 times, he can assume that [EMAIL PROTECTED] is already in some rosters out there. So he MUST NOT allow anyone who might collide with that to register at jabber.org after the migration to 1.1. So when upgrading jabber.org could just mark all JIDs with a \ in their name to be a pre-1.1 JID and disallow anyone to register who might collide with one of the registered JIDs. This way ' [EMAIL PROTECTED]' can register if no '[EMAIL PROTECTED]' existed before (he knows that from his database with the marks of old JIDs). If ' [EMAIL PROTECTED]' now wants to talk with '[EMAIL PROTECTED]', it would look like this: As jabber.org (1.1) knows that chrome.pl (1.0) is in fact 1.0 he escapes like XEP-0106 recommends and sends actually: In [EMAIL PROTECTED]'s client will now popup a message from "[EMAIL PROTECTED]" and except some weird JID he can talk with him. Because if he sends a message back: Then jabber.org will unescape the to-field and deliver the message to ' [EMAIL PROTECTED]'. Of course this solution is not a perfect one for the end-users as I will describe below, but I argue that the incompatibilities will increase the pressure on developers a bit and on administrators to adapt XMPP 1.1. And thus that might speed up the migration while providing a compatibility-workaround for maybr 98-99% of the cases, or maybe even 99,% (this needs to be investigated a big IMO, maybe my assumptions are completly wrong). So much for the server-to-server interoperability. Now about 1.1 clients and 1.0 clients. 1.0 clients will have no way to reach ' [EMAIL PROTECTED]', which is fine, either the user knows that guy's JID needs to be escaped because he uses an old client, or he has to upgrade to a client with 1.1 capabilities (what this means is described below). Not being able to send a message to ' [EMAIL PROTECTED]' will increase the pressure on the client developers as stated above. So 1.0 clients are basically out of luck if the user don't know how to escape, however, tell em: "get a new client". Of course it's blunt to say that, but I guess we can assume that not our WHOLE old userbase without spaces and all those fancy characters in their JID are NOT GOING TO signup a new account. So the users with spaces and & or whatever in their JID will probably come as slow as the XMPP world migrates to 1.1, and thus not experience any weird things. Lets assume now our guy,
Re: [Standards] JID Escaping
On Thu, Aug 02, 2007 at 05:21:46PM +0200, Robin Redeker wrote: > It seems that at least the majority of the whidely deployed servers > support \ in JIDs in DIGEST-MD5 SASL authentication due to faulty SASL > DIGEST-MD5 implementations. Sorry, I meant: ... support NO \ in JIDs in DIGEST-MD5 ... Robin
Re: [Standards] JID Escaping
On Fri, Jul 27, 2007 at 02:32:39PM -0600, Peter Saint-Andre wrote: > Matthias Wimmer wrote: > > Robin Redeker schrieb: > >> I propose to rename the XEP to make clear that this escaping/unescaping > >> should > >> only happen in very rare cases (only at gateways or heavily specialized > >> client > >> frontends). And that the terms 'escaping' and 'unescaping' are replaced by > >> 'mapping' and 'unmapping', because thats what is happening here. > > > > +100 > > Well, it's interesting, on the ejabberd list today someone said they > have an existing database of 45k email users and they want to offer > Jabber services to that user population, but re-use the same usernames. > I'm sure they have some users in there with addresses containing > characters like single quote, e.g., tim.o'[EMAIL PROTECTED] In which > case I bet that they'll be interested in using JID Escaping. > > I really feel that this discussion is not going anywhere. The spec is > IMHO pretty clear. If you don't like the spec, don't implement it. Ok, update on \ quoting: I tried to register users with \ in their nodepart in servers and authenticate via SASL DIGEST-MD5. These servers didn't work: - ejabberd (eg. at jabber.org) - jabberd2 (with gnusasl as SASL backend, DIGEST-MD5 code broken, scod however works) - openfire at igniterealtime.org I already filed a bugreport against ejabberd, they seem to have also their own code to handle DIGEST-MD5 as far as I understood the code. I filed a bugreport to ejabberd and it seems to be fixed sometime ( see http://www.jabber.ru/bugzilla/show_bug.cgi?id=362 ). I also informed the jabberd2 author about the problem with gnusasl. It seems that there are versions of jabberd2 which support it, eg. the server jabber.wavenet.pl which uses: jabberd sm 2.0s6. I confirmed that that version uses their own sasl implementation. igniterealtime.org seems to use "Openfire Enterprise 3.4.0 Alpha 2". ejabberd seemed neither to work on jabber.org nor my local setup, neither versions 1.1.2 nor 1.1.3 seemed to work. If someone tests with ejabberd and does NOT get an error he should try to use a client that actually not has a quoting bug too. tkabber for example doesn't quote the \ in the sasl response, so as ejabberd doesn't unquote they play fine with each other. Of course tkabber will blow up on servers that do right unquoting (confirmed that with jabber.wavenet.pl). This is the state of \ in JIDs and DIGEST-MD5 in the current public server network, one might want to look at this statistic: http://www.ta-sa.org/files/txt/a5cc5ae8318c76f40f9912daff6ef6af.txt That are 680 version responses from various public servers advertised on xmpp.org.ru, xmpp.net and jabber.org. (btw. I sent requests to 713 servers, but 33 seemed not to be reachable or didn't support the software version feature). It seems that at least the majority of the whidely deployed servers support \ in JIDs in DIGEST-MD5 SASL authentication due to faulty SASL DIGEST-MD5 implementations. However, using the PLAIN authentication method seems to work! Robin
Re: [Standards] Proposed XMPP Extension: Stanza Size Limits
On Wed, Aug 01, 2007 at 10:40:41PM -0500, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Stanza Size Limits > > Abstract: This document specifies an XMPP protocol extension for limiting the > size of stanzas that an entity would like to receive from a server. > > URL: http://www.xmpp.org/extensions/inbox/stanzalimits.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > How are clients supposed to figure out what stanzas they can _send_ to the server btw.? Is that covered by this XEP or some other XEP? In case of eg. private XML storage (or any other stanza) it might be desireable for a client to know the limit _before_ it sends the XML to the server. If I imagine I sent a large stanza and the server suddently tells me it's too big, does the server disconnect me then? Does it ignore the rest of the too big stanza so that I can continue sending other smaller stanzas? How is the client supposed to react on that error? Robin
Re: [Standards] Proposed XMPP Extension: Message Stanza Profiles
On Wed, Aug 01, 2007 at 06:55:47PM -0700, Justin Karneges wrote: > On Wednesday 01 August 2007 1:44 pm, Robin Redeker wrote: > > On Wed, Aug 01, 2007 at 01:04:13PM -0700, Justin Karneges wrote: [.snip.] > > > A MUC invite should be displayed in the same context as the body that > > > accompanied it. If IBB and body are together, only IBB should be used. > > > If IBB and x:data are used together, it is an invalid stanza. > > > > Ok, then we really need an XEP clearing this up. The easiest for the > > receiving client would be a element with a specific namespace > > attribute specifying the meaning and purpose of the message then. > > (Unfortunately this would also mean new behaviour for the receiving > > side). > > This was suggested at the devcon, but we resolved that the elements already > present in the stanza were enough of an indicator. > [.snip.] > The fact that the former would be in the 'jabber:x:data' profile is easily > derivable without the extra element. > > Sure, the element might help clear up a situation like this: > [.snip.] > > But a good set of rules would work just as well, I think. I agree, but here some of my thoughts: Well, I also think that it's possible to come up with rules that divide the current XEPs into a set of disctinctive profiles. But will that also be true for future XEPs? Will the ruleset be extensible enough not to become too complex when more (yet undefined) XEPs joining in? I think atleast that the element is an easy and extensible way to ensure future compatibility. With a profile element it will also be very easy to make a registrar where one can simply create new profiles by defining a new namespace for the element. But I don't know how your solution to the registrar stuff looks like, maybe it works as well. Also note that old clients that don't support would of course still work. The problem with is, I guess, that maybe clients can't really rely on it because there will be still a lot of old clients not sending it. At least for those cases we might still need a bit disambiguation in the XEP. But IMO the division of possible children elements into profiles and the then required disambiguation 'heuristic' in the client is not optimal if the heuristic is the only way to determine it. So I guess that a element can be of much help for a (new) client to determine whether a disambiguation heuristic is neccessary. I know that it can be prevented by the design of the XEP that the client needs a 'heuristic' (heuristics can fail) for the decision algorithm for the profile of a message stanza. Whether it will remain possible or not to provide such a non-heuristic way might depends on the future XEP authors. They will have to always keep an eye on the message profile XEP and the always defined profiles when defining new elements. So they will have the choice of joining an existing profile and creating a new one. The problem with an existing profile is that all clients might need an update to recognize the new element. My point here is, that with the element the clients have an easy way to extend their profile decision logic. I'm unsure whether this way is easier than the way without , but I can imagine that the way without might become messier, if not now, maybe in future. Robin
Re: [Standards] Proposed XMPP Extension: Message Stanza Profiles
On Wed, Aug 01, 2007 at 01:04:13PM -0700, Justin Karneges wrote: > On Wednesday 01 August 2007 12:21 pm, Robin Redeker wrote: > > On Wed, Aug 01, 2007 at 11:46:43AM -0500, XMPP Extensions Editor wrote: > > > The XMPP Extensions Editor has received a proposal for a new XEP. > > > > > > Title: Message Stanza Profiles > > > > > > Abstract: This document specifies best practices for handling extended > > > content in XMPP message stanzas. > > > > > > URL: http://www.xmpp.org/extensions/inbox/messageprofiles.html > > > > > > The XMPP Council will decide at its next meeting whether to accept this > > > proposal as an official XEP. > > > > Hm, this is an interesting XEP. I can imagine how client _might_ get > > confused by such a message, but at least I wouldn't have much problems > > with it. > > > > I mean, I have to scan the message for all kinds of things anyway, eg. > > for the mood and all the other rubbish. > > > > I would then transmit each part to the program module that handles it > > and let it proceed. Eg. the dataform would be displayed, the message > > would be printed in the window, geoloc and mood will be updated and the > > message module will figure out that the message is urgent and will of > > course intercept the message if it has expired. > > This is precisely what I'm arguing against doing. > > A MUC invite should be displayed in the same context as the body that > accompanied it. If IBB and body are together, only IBB should be used. If > IBB and x:data are used together, it is an invalid stanza. Ok, then we really need an XEP clearing this up. The easiest for the receiving client would be a element with a specific namespace attribute specifying the meaning and purpose of the message then. (Unfortunately this would also mean new behaviour for the receiving side). [.snip.] > Receiving clients are not expected to have to find out what profile a message > uses, and therefore senders should not assume a receiver knows anything about > profiles. So you're right, this XEP is mainly a recommendation to senders. > However, knowing that messages must conform to a profile should allow for > tighter receiver implementations. Ah, ok. Robin
Re: [Standards] Proposed XMPP Extension: Message Stanza Profiles
On Wed, Aug 01, 2007 at 11:46:43AM -0500, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Message Stanza Profiles > > Abstract: This document specifies best practices for handling extended > content in XMPP message stanzas. > > URL: http://www.xmpp.org/extensions/inbox/messageprofiles.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > Hm, this is an interesting XEP. I can imagine how client _might_ get confused by such a message, but at least I wouldn't have much problems with it. I mean, I have to scan the message for all kinds of things anyway, eg. for the mood and all the other rubbish. I would then transmit each part to the program module that handles it and let it proceed. Eg. the dataform would be displayed, the message would be printed in the window, geoloc and mood will be updated and the message module will figure out that the message is urgent and will of course intercept the message if it has expired. How are receiving clients expected to find out which profile the message belongs to? Are they expected to find it out? Or are the profiles just a recommendation for the sender to send messages that gurantee that the message isn't confused by dumb clients? Robin
Re: [Standards] <[CDATA[ in XMPP
A small mistake I made: On Tue, Jul 31, 2007 at 02:02:32PM +0200, Robin Redeker wrote: > On Mon, Jul 30, 2007 at 06:49:45PM -0700, Rachel Blackman wrote: >stream->send ( > "" >); > >data = sprintf ( > "", And here: > get_int, chatsession->get_from_nick, chatsession->get_to_nick >); There one should still have to take care that no string with "]]>" is concationated into the CDATA block, as that would end the CDATA block. R
Re: [Standards] <[CDATA[ in XMPP
On Tue, Jul 31, 2007 at 02:07:43PM +0200, Jacek Konieczny wrote: [.snip.] > IMHO XMPP should become more XML compatible and not less. Both existing > XML parsers and serializers should be usable for XMPP. If something has > to be changed in the XMPP specification then it should be removing the > incompatibilities and not adding more. +100! Robin
Re: [Standards] <[CDATA[ in XMPP
On Tue, Jul 31, 2007 at 01:28:43PM +0200, Alexander Gnauck wrote: > Ralph Meijer schrieb: > >I am curious which software this holds for. Most XMPP implementations > >are based on expat, as far as I know. Sure they might have their own > >DOM's (my Twisted XMPP stuff does), but that is irrelevant for parsing. > > There is many software which is using expat, but there is also lots of > software which has its own simple parsers. > * I have my own XML parser in many projects > * The Tigase and gloox have their own lightweight XML parsers > * Lots of mobile clients have their own parsers > > Of course it may be possible to replace this parsers with feature > complete XML parsers or add CDATA to them if it doesnt exists already. > But there are reasons why the developers wrote their own parsers. The RFC3920 does not restrict usage of CDATA, so I always assumed it's a valid XML construct. But I would've never used it in my code just because I know that there are XMPP implementations which can't even read the ""XML"" described in RFC3920. > Today we talk about CDATA, hey but what is coming next? There seems to be an obvious large need for clearing up the CDATA case. It was never restricted by the RFC, some RFCs even used it: http://www.xmpp.org/rfcs/rfc3923.html 3.1. Process for Securing Messages ... a sending agent MUST use the following procedure: ... Provide the resulting signed and/or encrypted object within an XML CDATA section (see Section 2.7 of [XML] (Bray, T., Paoli, J., Sperberg-McQueen, C., and E. Maler, “Extensible Markup Language (XML) 1.0 (3rd ed),” February 2004.)) contained in an child of a stanza ... Of course that RFC was never relevant in any implementation as far as I know, but you couldn't know whether it might become adopted for e2e encryption. Within this perspective it becomes even less sane to rely on CDATA-free XML in XMPP stream parsers. R
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 06:49:45PM -0700, Rachel Blackman wrote:
>
> On Jul 30, 2007, at 6:28 PM, Justin Karneges wrote:
>
> >On Monday 30 July 2007 5:37 pm, Rachel Blackman wrote:
> >>If I send ' [EMAIL PROTECTED]' to the server in a
> >>roster add/remove request, it will almost certainly eat that
> >>whitespace at the beginning.
> >
> >It most certainly must not do that.
>
> I cannot remember which server it was that I encountered, but one
> consumed all 'extraneous' whitespace (even inside of stanzas) to deal
> with the common practice of sending whitespace to keepalive a
> connection.
The keepalive whitespace is afaik only outside of stanzas. A server
doesn't need to ignore all whitespace if it just has to ignore
whitespace in the toplevel element.
> I do not dispute that the behavior is likely erroneous, but it was in
> place in at least one server; I witnessed it when it would eat any
> leading spaces in a element.
Yes, Sounds like a big bug.
> >>But now let's say I do ' element when sending something.
> It can't be sent raw, either way, due to the ampersand.
In case of CDATA we can send it raw.
> My thought is that if we allow CDATA, it should only be allowed in
> message elements, because otherwise you end up with questions like
> this. Of course, that's an un-XML-ish limitation of CDATA, so...
I don't see any problems or open questions here? CDATA has clearly
defined semantics and syntax in http://www.w3.org/TR/REC-xml/#syntax and
it can appear in XML _ANYWHERE_ where usually normal character content
can appear.
When thinking about this please bear in mind that at the time the server
knows that it has to interpret a string as JID it is already XML
unescaped. This means in pseudo-code:
dom_node = parse (""
);
That code without CDATA would look like:
stream->send (
""
+ xml_escape (chatwindow->get_message_from_user)
+ ""
);
It saves us CPU time to escape the message. CDATA doesn't give us much,
but there are certainly uses where CDATA is more convenient than some
escaping method.
Imagine this code (sprintf can be found in most modern languages like
C/C++ or Perl, and even Java has similar constructs):
data = sprintf (
"",
get_int, chatsession->get_from_nick, chatsession->get_to_nick
);
That code looks clean nice and small and precise. With xml_escape it
will become more loaded:
data = sprintf (
"%d: %s/%s",
get_int, xml_escape (chatsession->get_from_nick),
xml_escape (chatsession->get_to_nick)
);
Of course these are verrry minor conveniences when writing code, I
barely use CDATA myself, but I can imagine others find it very
convenient. (Might those speak up and give me some example? I'm
interested just out of curiosity :)
R
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 05:12:16PM -0700, Rachel Blackman wrote: > Can I use <[CDATA[ in, say, roster additions or removals? If I'm > using it there, how do I need to process the text on the server-side > for the JIDs? If I send ' [EMAIL PROTECTED]' as a CDATA element -- > allowing the space in there -- how do I handle escaping it on the > server side? Do I just store it as ' [EMAIL PROTECTED]' in the > roster? Do I need to re-escape it before sending it back? Do I need > to determine that the JID requires escaping, and so send that roster > item as a <[CDATA[ block? Does it show up as the same JID or > different than [EMAIL PROTECTED] Etc. ' [EMAIL PROTECTED]' is not (yet) a valid JID. And you can already send such a JID to the server:
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 06:16:24PM -0500, Mickael Remond wrote: > Hello, > > Regarding this discussion, it seems that the topic started about the use > of CDATA. > So, we are talking about way to escape characters in an XML stream. My > view is that all way to escape characters are good, especially when they > are defined in XML, cited in XMPP RFC and are simple to implement (and > implemented by all parsers I know). > > I have tested the use of CDATA for example with Psi, Gajim and a few other > clients and all of them were fine. > As Peter said, it should probably be allowed if not explicitely forbidden. > > My view is that it should be allowed as, as I understand it, it is just > one of the possible way to escape characters (and not the worst one). I've been reading the XMPP RFC with an eye on exactly CDATA, and it says clearly that CDATA is allowed. An XML stanza exists at the direct child level of the root element and is said to be well-balanced if it matches the production [43] content of [XML]. See in the XML spec: [43]content::= CharData? ((element | Reference | CDSect | PI | Comment) CharData?)* CDSect is a CDATA Section. And the RFC3920(bis) doesn't say anything about excluding it. R
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 04:48:53PM -0600, Peter Saint-Andre wrote: > Robin Redeker wrote: > > If we don't call something XML which clearly is not XML, then there > > aren't any further confusions possible. > > > > But still the term 'XML' is quite often used in the XMPP specs. > > Interestingly, the first paragraph of the XML specification says: > > *** > > Extensible Markup Language, abbreviated XML, describes a class of data > objects called XML documents and partially describes the behavior of > computer programs which process them. XML is an application profile or > restricted form of SGML, the Standard Generalized Markup Language [ISO > 8879]. By construction, XML documents are conforming SGML documents. > > *** > > Is it true that "XML is SGML"? Obviously some SGML pedants would object > because XML is "an application profile or restricted form of SGML". Yet > "XML documents are conforming SGML documents"! So here we have a rough > definition of conformance, and a working example since XML conforms to SGML. > > Similarly, a complete XMPP session (with start and end stream tags) is a > conforming XML document. Just because the session does not include > comments, processing instructions, DTD subsets, entity references other > than those predefined in the XML spec, other other restricted features > does not mean that the session is not an XML document. > The difference is, which I consider the major confusing part for developers, that the XML recommendation doesn't always use the term 'SGML' when defining something. The XMPP spec very often uses 'XML' to describe things. > So we could say this in rfc3920bis: > > *** > > The Extensible Messaging and Presence Protocol (XMPP) defines a class of > data objects called XML streams as well as the behavior of computer > programs that process XML streams. XMPP is an application profile or > restricted form of the Extensible Markup Language (XML). A complete XML > stream (including start and end stream tags) is a conforming XML document. > > *** Sounds nice. > > (Please take this mail as serious as you like, I don't know how serious I > > mean it.) > > Don't worry, I take all of your emails that way. :P Just take care that some mails have the intend to point something out, which I would like to have fixed or I really feel could make the life of other developers easier. I of course can only offer an opinion w.r.t. technical aspects of some things. That some things can't be fixed because of backward compatibility is unfortunate. But other things (still) can, just make sure to take at least those serious "enough" :-) R
Re: [Standards] <[CDATA[ in XMPP
On Tue, Jul 31, 2007 at 02:13:36AM +0400, Sergei Golovan wrote: > On 7/31/07, Mickael Remond <[EMAIL PROTECTED]> wrote: > > XMPP is XML. > > XMPP is not XML. Looks like we need a bit more clarification about this issue in the upcoming rfc3920bis, no? I would even recommend inventing a new term for it: XMPPL If we don't call something XML which clearly is not XML, then there aren't any further confusions possible. But still the term 'XML' is quite often used in the XMPP specs. (Please take this mail as serious as you like, I don't know how serious I mean it.) R
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 05:00:09PM -0600, Peter Saint-Andre wrote: > Tobias S. Josefowitz wrote: > > On 7/31/07, Peter Saint-Andre <[EMAIL PROTECTED]> wrote: > > > >> Similarly, a complete XMPP session (with start and end stream tags) is a > >> conforming XML document. Just because the session does not include > >> comments, processing instructions, DTD subsets, entity references other > >> than those predefined in the XML spec, other other restricted features > >> does not mean that the session is not an XML document. > > > > Unless for example starttls comes into play... exceptions everywhere. > > Is that the existence of a rule I am sensing here? > > How does the inclusion of STARTTLS negotiation cause the complete XML > stream to not be an XML document? http://www.ta-sa.org/files/txt/ba19c3c423064ce6ca0a21b7dd7183d5.txt
Re: [Standards] <[CDATA[ in XMPP
On Mon, Jul 30, 2007 at 09:58:34PM +0200, Andreas Monitzer wrote: > On Jul 30, 2007, at 21:50, Sergei Golovan wrote: > > >I would say "It makes developing custom parsers more difficult". But > >any existing XML parser shouldn't have any problem. > > Nobody should write their own XML parser anyways, unless it's Your assumption is that XMPP is XML. I think it has been said in multiple discussions on this list and in the jdev chat room that that assumption is not correct :-) It's said that XMPP is just 'an application' or 'a subset' of XML. And I heard from some people that XMPP's "XML" was even simplified to allow writing of own "more efficient" ""XML""(XMPP) parsers. > absolutely necessary (e.g. programming for a platform where no XML > parser is available), since these kind of implementations are bound > to contain serious issues > (XML is much more complicated than people think). I very much agree with that observation :-) R
Re: [Standards] JID Escaping
On Mon, Jul 30, 2007 at 01:57:05PM -0600, Peter Saint-Andre wrote: > Matthias Wimmer wrote: > > - [EMAIL PROTECTED] is an allowed JID by XMPP, but somehow > > prohibited by JID escaping XEP. It is not very clear how to handle this > > when doing unescaping. Not unescaping? Then we need an exception for the > > escaping rules again, that when reescaping, this does not have to be > > encoded as [EMAIL PROTECTED] > > Yes, that case is messy. We added it to XEP-0106 mostly to handle an odd > gateway case for LDAP, not native XMPP addresses. I'm open to removing > it for native addresses. Sounds like a good plan, because the XEP doesn't define how to act if someone sends such a JID. I guess the displaying implications, which will kick in if it is decided that every client SHOULD do unescaping to display a contact, are clear. In case every client SHOULD perform unescaping to display JIDs or contacts, a paragraph might be added to '7. Security Considerations' advising clients to inform the user somehow that there were spaces in the nodepart of the JID. R
Re: [Standards] JID Escaping
On Fri, Jul 27, 2007 at 02:32:39PM -0600, Peter Saint-Andre wrote: > Matthias Wimmer wrote: > > Robin Redeker schrieb: > >> I propose to rename the XEP to make clear that this escaping/unescaping > >> should > >> only happen in very rare cases (only at gateways or heavily specialized > >> client > >> frontends). And that the terms 'escaping' and 'unescaping' are replaced by > >> 'mapping' and 'unmapping', because thats what is happening here. > > > > +100 > > Well, it's interesting, on the ejabberd list today someone said they > have an existing database of 45k email users and they want to offer > Jabber services to that user population, but re-use the same usernames. > I'm sure they have some users in there with addresses containing > characters like single quote, e.g., tim.o'[EMAIL PROTECTED] In which > case I bet that they'll be interested in using JID Escaping. If they can live with the fact that clients like: psim gajim, kopete, pidgin, tkabber, ... http://www.jabber.org/software/clients.shtml That clients like those display the guy as [EMAIL PROTECTED] And if they can teach their userbase that their JID contains \27 instead of '. > I really feel that this discussion is not going anywhere. The spec is > IMHO pretty clear. If you don't like the spec, don't implement it. I _certainly_ won't. But you are aware that you recommend every client author to implement XEP-0106? (Hint: http://www.xmpp.org/extensions/xep-0211.html ) XEP-0106 is certainly useful, and noone says it should go away, I would just know WHO is supposed to implement it and in WHICH cases. Mriduls clarification in http://article.gmane.org/gmane.network.jabber.standards-jig/16328 is pretty useful and I agree that the XEP-0106 partially lets shine through that it's not for everyone and in every occasion, but I'm afraid that with a 'RECOMMENDED' in XEP-0211 most client authors will try to implement it. But ok, lets stop here, I'm sure the client authors who want to implement XEP-0211 and like to also provide the recommended fields will read the mailing list. I will file a bug report against the first ordinary IM client that does unconditional unescaping of JIDs and displays the unescaped nodepart as main handle for a conversation. Ok, lets stop now really. Robin
Re: [Standards] XEP-0045: roomnick case
On Fri, Jul 27, 2007 at 09:17:03AM +0530, Mridul Muralidharan wrote: > Jack Moffitt wrote: > >>Currently in XEP-0045, roomnicks are case-sensitive. To be precise > >>roomnicks are handled according to the Resourceprep profile of > >>stringprep: > >> > >>http://www.xmpp.org/extensions/xep-0045.html#bizrules-jids > >> > >>This means that the following roomnicks are all different: > >> > >>StPeter > >>stpeter > >>STPETER > >> > >>Some people have pointed out that this can be confusing to end users. > > > >Not only this, but it's very easy to "steal" someone's identity when > >they are not there. You just log in with their normal nick. > > > >At Chesspark we make all public rooms non-anonymous and we always show > >the full jid or (if they are in your roster) the roster nick for that > >person. That way no one is ever confused who is who. So we've > >basically solved this in the client. All this nick stuff stuff is so > >IRCish anyway. It's nice if what you want is an anonymous room, but > >for everything else, it's just a mess IMO. > > > >After trying several solutions we found that this was least confusing > >to everyone and we have gotten no complaints except for the occasional > >jerk that signs up with a name like st.peter or stpeter_, but there's > >little you can do about that but ban the offenders. > > > >jack. > > Unless rooms are not anonymous, clients have no way to validate who the > participants are anyway - so it is not really stealing : you dont > register the nick, it is not tied to your jid in any way. > In anonymous rooms you dont know who the participants are - so relying > on nicks to give a hint of the identity would be wrong. I guess the problem here is to explain this to the users. Robin
Re: [Standards] JID Escaping
On Wed, Jul 25, 2007 at 03:22:04AM +0530, Mridul Muralidharan wrote: > > Hi Robin, > > You should analyze as to who will actually need to encode or decode nodes. > > 1) Gateway's. I know that JID escaping is for gateways, and it makes sense to define a mapping. But not for ordinary 'I want a JID with an @ in my name!'-user-usage. > For the purpose of mapping foreign uid's to xmpp nodes. > In this case, ONLY the gateway, will need to do encode/decode. > All other entities will treat is as a JID - without any transformation > to node/domain/resource. > It will encode a foreign id to node so that it becomes a valid xmpp JID. > It will decode node from an xmpp 'to' for obtaining the destination in > the foreign network. > Makes complete sense and seems to me as the only sane application. > 2) Client's and server's. > When the node is same as uid, and the uid of the user contains > prohibited characters. In this case, when client authorizes to server, > it would encode the uid to obtain the node - and then form the JID to be > used (Only the client encodes it as part of auth, and once authorization > is over, encoding of the JID is opaque to the client). > > At the server, it would decode the node to obtain the actual uid and use > that (backend store update, etc) - note that for purpose of routing, the > server would treat the jid as-is, only decode it to identify the backend > entry (database, ldap, file, etc) to which it has to persist/query data. > (This is one way in which you would model things - servers can ofcourse > come up with other alternate ways to do this). > > Other than for these two, I cant really envision any other important > usecase which requires any entity (client, gateway, component or server) > to encode or decode - the above are directly related to and required for > xmpp routing. > > Hopefully, from this point of view, the xep might make more sense. > I completly agree. > Robin Redeker wrote: > >On Tue, Jul 24, 2007 at 10:10:45PM +0530, Mridul Muralidharan wrote: > >>Robin Redeker wrote: > >>>On Sat, Jul 21, 2007 at 08:17:19PM -0600, Peter Saint-Andre wrote: > >>>>Robin Redeker wrote: > >>>>>On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote: > >>>>>>>I think the whole XEP should be renamed to something like: > >>>>>>> > >>>>>>> XEP-0106 - JID Mapping for Gateways > >>>>>>This would be better. But it breaks the generic usage of JIDs for > >>>>>>both users > >>>>>>and gateways. It will create a lot of trouble. > >>>>>> > >>>>>The XEP seems to already create a lot of trouble. Just remind me to > >>>>>register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-) > >>>>No problem. The spec says: > >>>> > >>>>"The character sequence \20 MUST NOT be the first or last character of > >>>>an escaped node identifier." > >>>> > >>>>But of course you can violate the spec if desired. ;-) > >>>I don't violate the RFC here. I violate some optional extension. > >>You cant specify the characters mentioned in the xep in the node without > >>escaping them - so trying to do so otherwise is a violation of the rfc. > >>Ofcourse you can come up with a custom encoding (as we used to have), > >>but this does not allow any arbitrary client/server combination to > >>interoperate. > > > >Of course I can? With an old client? And it's even allowed and sane and > >valid? Or do I miss something here? > > You cannot use prohibited characters - whether they are old clients or > bad clients. > The first conforming xmpp recipient (typically the server) would kick > your stream out with a stream error. But I am right that '[EMAIL PROTECTED]' is a completly valid JID. And that every server should accept it? > > > > >>>The XEP-0106 has to exclude the JIDs which start or end with '\20' in the > >>>nodepart from the escaping AND unescaping transformations. > >>This is already present. > > > >Great, JIDs with '\20' in the beginning and end have been deprecated then? > >Shouldn't the RFC be changed then? > > No, this is not related to the rfc. > The rfc specifies list of prohibited characters - and they MUST NOT be > in the node. > The xep allows a way of encoding these characters into the JID because > of requirements like what I mentioned above. Those tw
Re: [Standards] JID Escaping
On Tue, Jul 24, 2007 at 10:10:45PM +0530, Mridul Muralidharan wrote:
> Robin Redeker wrote:
> >On Sat, Jul 21, 2007 at 08:17:19PM -0600, Peter Saint-Andre wrote:
> >>Robin Redeker wrote:
> >>>On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote:
> >>>>>I think the whole XEP should be renamed to something like:
> >>>>>
> >>>>> XEP-0106 - JID Mapping for Gateways
> >>>>This would be better. But it breaks the generic usage of JIDs for both
> >>>>users
> >>>>and gateways. It will create a lot of trouble.
> >>>>
> >>>The XEP seems to already create a lot of trouble. Just remind me to
> >>>register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-)
> >>No problem. The spec says:
> >>
> >>"The character sequence \20 MUST NOT be the first or last character of
> >>an escaped node identifier."
> >>
> >>But of course you can violate the spec if desired. ;-)
> >
> >I don't violate the RFC here. I violate some optional extension.
>
> You cant specify the characters mentioned in the xep in the node without
> escaping them - so trying to do so otherwise is a violation of the rfc.
> Ofcourse you can come up with a custom encoding (as we used to have),
> but this does not allow any arbitrary client/server combination to
> interoperate.
Of course I can? With an old client? And it's even allowed and sane and
valid? Or do I miss something here?
> >The XEP-0106 has to exclude the JIDs which start or end with '\20' in the
> >nodepart from the escaping AND unescaping transformations.
>
> This is already present.
Great, JIDs with '\20' in the beginning and end have been deprecated then?
Shouldn't the RFC be changed then?
> >At the moment the paragraph says that it MUST NOT be first or last
> >in the node part, but it doesn't say WHAT to do when this perfectly
> >fine JID arrives from the line. Should the JID not be unescaped at all?
> >Should only the parts after and before '\20' be unescaped?
> >Should the client close the connection?
>
> It depends on who is doing what.
> If the recipient is expected to 'parse' the node, then it would return
> an error, else it would pass it on (directed packets through server for
> example).
To who will it return an error? Will it throw a pop-up at the user
"Someone with an invalid JID sent you something!"?
Or back to the sender? Why should it do that for a perfectly fine JID?
> >Do I miss something in the XEP? (If I do so please ignore the rest of
> >the mail.)
> >
> >Please also note the nice, but maybe not so important collision that
> >here happens when the client just doesn't unescape:
> >
> > unescape ("\5c20foobar\5c20") => "\20foobar\20"
> > unescape ("\20foobar\20") => "\20foobar\20"
> >
> >This is of course not really an important JID, and who cares about a few
> >optical collisions in clients which confuse the user. And these only
> >happens
> >once someone else decides to put '\20' at the beginning or end
> >of his name and why would someone do that?
> >
> >Hey, we could add security notes to all clients which tell the user:
> >
> > "Never attach '\20' to the beginning or end of your name, it is unsafe!"
> >
> >The U.S. Army will love this! (One might think of a case where they
> >actually
> >name their units by enumerating them with a \ in the end:
> >
> > Unescaped: Escaped: Unescaped:
> > "Tank\1" "Tank\5c1" "Tank\1"
> > "Tank\20" "Tank\20""Tank\20"
> > "Tank\22" "Tank\5c22" "Tank\22"
> > "Tank\5c20" "Tank\20" ... ps
> >
> >Ah... never... why would they do that... :-)
>
> These sort of problems are common to any form of encoding - example
> urlencoding.
> It is obviously expected that the client/gateway would do the right thing.
Yea, usually everyone does unescape and compare the unescaped stuff,
then no collisions happen (and urlencoding does not have this problem
because it doesn't compare escaped URLs afaik).
The problem here is that the server treats "Tank\20" and "Tank\5c20" as
different nodes, and thats of course the completly right thing to do,
because they are different.
If you m
Re: [Standards] JID Escaping
On Sat, Jul 21, 2007 at 08:17:19PM -0600, Peter Saint-Andre wrote:
> Robin Redeker wrote:
> > On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote:
> >>> I think the whole XEP should be renamed to something like:
> >>>
> >>> XEP-0106 - JID Mapping for Gateways
> >> This would be better. But it breaks the generic usage of JIDs for both
> >> users
> >> and gateways. It will create a lot of trouble.
> >>
> >
> > The XEP seems to already create a lot of trouble. Just remind me to
> > register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-)
>
> No problem. The spec says:
>
> "The character sequence \20 MUST NOT be the first or last character of
> an escaped node identifier."
>
> But of course you can violate the spec if desired. ;-)
I don't violate the RFC here. I violate some optional extension.
The XEP-0106 has to exclude the JIDs which start or end with '\20' in the
nodepart from the escaping AND unescaping transformations.
At the moment the paragraph says that it MUST NOT be first or last
in the node part, but it doesn't say WHAT to do when this perfectly
fine JID arrives from the line. Should the JID not be unescaped at all?
Should only the parts after and before '\20' be unescaped?
Should the client close the connection?
Do I miss something in the XEP? (If I do so please ignore the rest of
the mail.)
Please also note the nice, but maybe not so important collision that
here happens when the client just doesn't unescape:
unescape ("\5c20foobar\5c20") => "\20foobar\20"
unescape ("\20foobar\20") => "\20foobar\20"
This is of course not really an important JID, and who cares about a few
optical collisions in clients which confuse the user. And these only happens
once someone else decides to put '\20' at the beginning or end
of his name and why would someone do that?
Hey, we could add security notes to all clients which tell the user:
"Never attach '\20' to the beginning or end of your name, it is unsafe!"
The U.S. Army will love this! (One might think of a case where they actually
name their units by enumerating them with a \ in the end:
Unescaped: Escaped: Unescaped:
"Tank\1" "Tank\5c1" "Tank\1"
"Tank\20" "Tank\20""Tank\20"
"Tank\22" "Tank\5c22" "Tank\22"
"Tank\5c20" "Tank\20" ... ps
Ah... never... why would they do that... :-)
I propose to rename the XEP to make clear that this escaping/unescaping should
only happen in very rare cases (only at gateways or heavily specialized client
frontends). And that the terms 'escaping' and 'unescaping' are replaced by
'mapping' and 'unmapping', because thats what is happening here.
Robin
Re: [Standards] Handling for characters that have entities, but XML does not require them to be escaped
On Sun, Jul 22, 2007 at 11:25:23PM +0200, Matthias Wimmer wrote: > Hi Robin! > > > Robin Redeker schrieb: > >>Why at all do these characters have to be escaped? > >I guess because many people did implement their own broken XML parsers > >in the past and many couldn't handle real XML, so they enforced escaping > >that character for the backward compatibility. (just a guess) > > I can't beleave that there are any such problems. There is already > software producing XML, that is valid but not escaping all possible > characters. > > Examples for this are jabberd2 (but to a very new SVN version), jadc2s > (up to today), Psi (still not escaping " and ' in text nodes). Hm, ok, then I also have absolutely no idea why these chars have to be escaped. (And I'm also very curious!) > So there is out many software, that worked for years now, but > introducing this unneccessary restricting in RFC 3920 made them broken. > > >If you use expat you could get the original string from a text node > >and look for a '>' in that string. But this is an ugly hack that I also > >consider unneccessary. > > How do I do this with expat? I have never seen something like this. At > least normally expat is a SAX parser, that you set an > CharacterDataHandler. And the function you register as the > CharacterDataHandler gets passed unescaped UTF-8 data. Within the > CharacterDataHandler I see now way to determine if a > has been > transfered as > or as >. The function I mean is XML_GetInputContext, the Perl API uses that to get me the portion of the XML document that was parsed. If you got that you might be able to find out whether unescaped > is in that. (see eg. http://www.math.ucla.edu/computing/docindex/expat-html-2/reference.html#XML_GetInputContext ) You could also use XML_SetDefaultHandler and XML_DefaultCurrent to get nearly the same data without the limit of XML_CONTEXT_BYTES (I actually don't know very precise what the difference is, I'm not that intimate with the C API of expat). These are all of course very weird ways to get to the original XML character data of a recognized element, and I would love if XMPP wouldn't require me to even have need for such access to the my XML parser. All I want is a DOM tree and all I want to care about when writing XMPP out is passing my DOM tree to a XML generator without convincing the XML generator that _I_ know better than it how XML should be generated and look like so that a XMPP server doesn't get confused. (Lets not start with namespace madness here, we don't want to open the can of worms which is still left over from the 'About stream namespaces' discussion from 4 months ago. Hmm... IMO the issue should be brought up regularly even though that discussions about it end in: "We know, but we won't fix it, because the protocol should stay broken and underspecified because if we fix it we break the implementations."). > >The RFC should be fixed and software that doesn't parse unescaped > in > >text nodes should be fixed (noone is forced in todays world to write his > >own XML parser, libxml2 (afaik) and expat (for sure) can be convinced to > >handle partial transferred XML documents these days). > > Yes ... I'd also say that because of reusing standards and > implementations of them, we should not force software to not accept > unescaped entities. We should even encourage software to accept these > unescaped entities. I agree, we should encourage being conform to the W3C recommendation. Robin
Re: [Standards] Handling for characters that have entities, but XML does not require them to be escaped
On Sun, Jul 22, 2007 at 04:30:13PM +0200, Matthias Wimmer wrote: > Hi! > > There are several characters, that have predefined entities in XML, but > that do not need to be escaped in XML. > Examples for such characters are > ' and " in text nodes. > [...] > So I have two questions regarding this: > > Why at all do these characters have to be escaped? I guess because many people did implement their own broken XML parsers in the past and many couldn't handle real XML, so they enforced escaping that character for the backward compatibility. (just a guess) > I it really necessary, that RFC 3920bis mandates a server to reject such > XMPP streams? I very much dislike this requirement, as it would require > me to implement my own XML parser, as I don't know any parser I could > use, that could be configured to notice me that these characters have > been received unescaped. If you use expat you could get the original string from a text node and look for a '>' in that string. But this is an ugly hack that I also consider unneccessary. The RFC should be fixed and software that doesn't parse unescaped > in text nodes should be fixed (noone is forced in todays world to write his own XML parser, libxml2 (afaik) and expat (for sure) can be convinced to handle partial transferred XML documents these days). Robin
Re: [Standards] JID Escaping
On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote: > >I think the whole XEP should be renamed to something like: > > > > XEP-0106 - JID Mapping for Gateways > > This would be better. But it breaks the generic usage of JIDs for both users > and gateways. It will create a lot of trouble. > The XEP seems to already create a lot of trouble. Just remind me to register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-) Robin
Re: [Standards] JID Escaping
On Fri, Jul 20, 2007 at 02:41:32PM +0200, Mats Bengtsson wrote: > > >>But then the XEP is wrong since it includes the "/" character to be > >>> escaped. > >No its not, it is specifying escaping of the node portion of the JID, it > >specifically says you must not escape the resource. > > > > It is since you cannot uniquely separate the JID into its > node+domain+resource parts. > You cannot tell which the node part is. In my example > > User JID: [EMAIL PROTECTED]/[EMAIL PROTECTED]/[EMAIL PROTECTED]/coci > > the node part may be: mats\40home.se\2fcoci\40home.se\2fmats > The problem is escaping both the "@" and "/" and be able to uniquely > identify the > node+domain+resource parts of a JID. I think you can allow to escape either > of these characters but not both. If you installed some mind-reading device in the users computer you could determine what the user meant :-) Anyway, it's not defined how to escape a full JID, it's just defined how to escape the _node_ part of a JID. The client has to ask for the nodepart in a seperate entry box I guess (or use a MindReader(tm)). What do you need JID escaping for btw.? Robin
Re: [Standards] JID Escaping
Hi! I've been reading through the JID escaping once again. The escaping collisions are gone (with the \5c exception being gone) comparing to the last time I've read it. I think it should be clarified that the JID escaping should _only_ be used in gateways which want to map outside-strings to a JID node. The reason why I come to this conclusion is that it's not practical for regular IM clients to unescape a JID, see below: It should only be used in a E-Mail client that sends and retrieves mails via XMPP. All other clients will run into a name-spoofing problem (see below). In section '2. Requirements' is said: It MUST NOT be possible for clients to use this escaping mechanism to avoid the goal of stringprep; namely, that JIDs that look alike should have same character representation after being processed by stringprep. The whole purpose of the mechanism described in XEP-0106 is to _avoid_ the goal of stringprep (to be more exact, it avoids the goal of the nodeprep profile of stringprep). I understand that paragraph like the following: The JID escaping should not be used to allow users to enter node parts for the JID which contain invalid characters (like &, @, ...). It's only thought as a mapping between eg. e-mail addresses and JIDs for gateways. If that is not true and it actually is for using invalid characters in the node part of a JID, and clients unescape _every_ JID they see we will run into problems (see below). For escaping following rule has to be followed: * Note: The character sequence \20 MUST NOT be the first or last character of an escaped node identifier. This can only be seen as advice to the client authors not to allow spaces at the beginning and ending of names. But please observe that JIDs like this are still valid: [EMAIL PROTECTED] If now a client comes and unescapes that JID it will collide (visually) with '[EMAIL PROTECTED]'. There is no rule that unescaping such JIDs is not allowed, and such a rule would make no sense as it would break the display of perfectly fine JIDs. This is the main reason why escaping end unescaping MUST only be done for gateway applications, and there on the side of the gateway and the client. Unescaping should NOT apply for regular XMPP messaging in any form. When receiving a message from a gateway the client could check of course "is this i a email gateway?" and then perform unescaping of the node part and display the source email address the message came from. Regular clients, which don't implement any gateway specialcases, should NOT handle escaping at all. Please also note that the example '5.1 Jabber Identifiers' in the XEP is misleading: # User Input Escaped JID Client Display 1 space [EMAIL PROTECTED] [EMAIL PROTECTED] space [EMAIL PROTECTED] 2 call me "ishmael"@example.com [EMAIL PROTECTED] call me "ishmael"@example.com A user CAN'T input a broken JID, the client can't parse broken JIDs like that or should at least scream out loud if a user tries to enter such a JID. For escaping the user should get a form like this: Node: space cadet Domain: example.com Resource: The example is misleading because client author could think that they should implement a heuristic to parse broken JIDs and escape them if they are not valid JIDs. I think the whole XEP should be renamed to something like: XEP-0106 - JID Mapping for Gateways Maybe I got everything wrong, but this is the only way I am able to make sense of that XEP. A true escaping mechanism must be also understood by the server. And servers then have to unescape the JIDs to compare them. Robin
Re: [Standards] XEP-0045: roomnick case
On Thu, Jul 19, 2007 at 03:19:09PM -0600, Peter Saint-Andre wrote: > Chris Mullins wrote: > > > I think the IDEAL option is MEP: > > If we were redesigning things from scratch, probably. > > But we're not redesigning things from scratch. :) What are we're doing then? Patching/Hacking up nearly 10 year old standards so that they do something similiar to what we want? (I don't neccessarly vote for Nicks over Pubsub here, even though it's a nice application of Pubsub once again, I think it should at least be tried in a sample implementation. Unfortunately that would be a lot of work for nothing.) > Breaking changes are bad. > > How can we meet people's needs without breaking things? By breaking the people. Robin
Re: [Standards] securing in-band registration
On Wed, Jul 18, 2007 at 04:34:31PM -0600, Peter Saint-Andre wrote: > Part of the solution is requiring x:data forms for registration. Yes, as > Matthias pointed out this will make life difficult for existing clients. > So we need to define a transition strategy. Clearly define how the > x:data-only registration works and set some goals for deprecating the > old way of doing things. I thought those are already part of XEP-0077, is that another feature that noone implements these days in their clients? Or why do they make the life harder for clients? Robin
Re: [Standards] how to treat invalid XMPP?
On Mon, Jul 16, 2007 at 07:31:01PM +0200, Jakob Schroeter wrote: > Hi, > > Apparantly there is a number of software packages that generates invalid > XMPP. > I've seen at least unescaped ' and " in attribute values and character data, > respectively. > > http://www.xmpp.org/rfcs/rfc3920.html#xml states that an XMPP implementation > must not generate such unescaped characters, and when it "receives such > restricted XML data, it MUST ignore the data". You have to distinguish 'bad XML' and 'restricted XML' here. Unescaped or bad XML should lead to a disconnect. If you receive parts of the 'restricted XML', that means XML that is: * comments (as defined in Section 2.5 of [XML] (Bray, T., Paoli, J., Sperberg-McQueen, C., and E. Maler, “Extensible Markup Language (XML) 1.0 (2nd ed),” October 2000.)) * processing instructions (Section 2.6 therein) * internal or external DTD subsets (Section 2.8 therein) * internal or external entity references (Section 4.2 therein) with the exception of predefined entities (Section 4.6 therein) * character data or attribute values containing unescaped characters that map to the predefined entities (Section 4.6 therein); such characters MUST be escaped > So far I just throw a parse error and disconnect the stream (when I > implemented that, I never thought this would actually happen), but people > complain about that. Also, that makes the receiving client look bad. If you see a component or server that sends bad XML you should file a bugreport against them. Or inform the administrator to upgrade. Of course it looks bad to the user if the client bails out with "Got bad XML, disconnecting you from your precious contacts and ruining your IM flirt with that nice girl." But there is no other choice, or your client even displays corrupted data to the user, which would be even worse. Robin
Re: [Standards] XEP-0115 is harmful and should be deferred
Hi! On Thu, Jul 05, 2007 at 11:34:10PM +0200, Tomasz Sterna wrote: > Dnia 04-07-2007, śro o godzinie 21:03 -0600, Peter Saint-Andre > napisał(a): > > > base64(sha1(dave-formatted id/features)) > > Seems reasonable to me. > > I've picked this random post to reply but it does not concern this > particular post but the whole thread... > ...which I did not follow really, because I find this whole XEP and > concept of entity capabilities distributed with presence packed unneeded > and harmful. > > This XEP came out to solve a problem of jabber:iq:version storming on > the Psi client launch, which other clients blindly copied just to > show-off the remote client version at the fancy tooltip. > > So instead of pull-based mechanizm there was a push-based mechanizm > deployed. > > But I do see some inconsistency here. > We don't allow vCard hashes to be pushed with the presence. We do not > allow moods and "now-playing" to be pushed on us with the presence > packet, but we gladly allow unrequested capabilities to be pushed with > the presence?? More then, we're going to REQUIRE them? w.r.t. to vcard, this is just a small snipped from a recent debug log: 2007-06-30 13:12:30 +0200 | 2007-06-30 13:12:30 +0200 | i'm here 2007-06-30 13:12:30 +0200 | 1 2007-06-30 13:12:30 +0200 | http://gaim.sf.net/caps"; ver="2.0.0beta5" xmlns="http://jabber.org/protocol/caps"/> 2007-06-30 13:12:30 +0200 | 2007-06-30 13:12:30 +0200 | Quite some noise I get there with the presence, I completly agree with you on the following: > Excuse me. I've subscribed your PRESENCE information. I didn't ask for > your mood, tune, avatar nor capabilities. If I would need them, I will > ask and you may allow me to have them. > There's nothing special about the caps, that these would require special > privileges. I think this is a good point and well spotted :-) ! > And what's more - we invented a way for me to subscribe for all this > kinds of additional information. Using everyone's favorite PubSub. > Why don't we reuse it somehow? > > We didn't have PEP/PubSub deployed at all when we invented XEP-0115, but > we do have now and for sure can do better now. I, as client developer, would really like if PEP/PubSub became more widely-used. Recently when reading this list I became the feeling that PEP is being avoided. Eg. by the common "ironic joke": 'lets use PEP for this'. PEP/PubSub has many good applications IMO. And if you ask me, PEP is IMO essential enough to go into XEP-0073 (Basic IM Protocol Suite). (Well, it will have to go in there anyway if it becomes neccessary to promote client capabilities.) And I think announcing capabilities seems to be a great application of PEP/PubSub. I can already imagine the client setting: [X] Allow others to subscribe to your client's capabilities. or: [X] Don't publish this client's capabilities. Just my 1.999... Cents Greetings, Robin
