Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Apr 09, 2009 at 07:19:11PM -0500, XMPP Extensions Editor wrote: Version 0.8 of XEP-0198 (Stream Management) has been released. Abstract: This specification defines an XMPP protocol extension for active management of an XML stream between two XMPP entities, including features for stanza acknowledgements and stream resumption. Changelog: [See revision history] (ff/jk/jjh/psa) Diff: http://svn.xmpp.org:18080/browse/XMPP/trunk/extensions/xep-0198.xml?%40diffMode=u%40diffWrap=sr1=2933r2=3028u=3ignore=k= URL: http://xmpp.org/extensions/xep-0198.html Looks fine to me now, just a minor inconsistency (probably from editing): 4. Acks ... An a/ element MUST possess an 'h' attribute. An r/ element SHOULD NOT possess any attributes. But later it says: When an r/ element (request) is received, the recipient MUST acknowledge it by sending an ack element (either a/ or r/) to the sender. ... The response MUST contain a value of 'h' that is equal to the number of stanzas handled by the recipient of the r/ element. Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Thu, Apr 09, 2009 at 11:25:15AM +0200, Jonathan Schleifer wrote: Am 09.04.2009 um 00:11 schrieb Justin Karneges: However, transport contacts often have no resource. These are not real logged in XMPP clients, but faked contacts from a server component. You will get presence from='screenn...@aim.transport'/ and have to deal with it. You usually get presence from screenn...@foo.transport/ - so it's not _that_ would be even worse than a bare JID, as the resource part MUST NOT be empty. A resource string is always at least 1 character long. a bare JID, but a JID with empty resource. At least, I haven't seen getting presence from a bare JID so far. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Thu, Apr 09, 2009 at 10:01:09PM +0530, Mridul Muralidharan wrote: [.snip.] To get it clarified (since I am looking at client libraries right now), that is still within the component's domain right ? Not outside of it ? What I mean is : user disco's/connects to component transport.server.domain component sends back presence for us...@transport.server.domain, us...@transport.server.domain/res , etc Or do you mean something different ? So in essence, I am trying to see if a client can infer if the presence from a bare jid is coming from a component (and so handled separately) or from an xmpp user where barejid does not make sense. If _presence_ semantics don't allow presence from bare JIDs it's wrong for components to try to fit the presence into them. I certainly don't want to build disco-hacks into presence handling just for some broken components who don't know how to gateway presence information in a well defined RFC compliant way. IMO either the RFC should specify presence from bare JIDs or components should introduce 'fake' resources. For the moment mapping presence from bare JIDs to an empty resource (resource string with length 0) is probably the best workaround a client (library) can do. And some special rules in general for bare JIDs (like unavailability from a bare JID stands for 'no resource online anymore'). Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] unavailable presence from bare JID
Hi! I've recently stumbled across unavailable presence stanzas on s2s connections as replies to probes: presence from=b...@jid to=... type=unavailable / RFC 3921 (bis) doesn't explicitly disallow this, and jabberd2 seems to generate these unavailable presences. How is a client supposed to handle it in case it has received available presence from some resources before, eg. like this: presence from=t...@example.net/bla type=available / presence from=t...@example.net/foo type=available / And then receives: presence from=t...@example.net type=unavailable / Should the client assume that the resources 'bla' and 'foo' are offline now? Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] unavailable presence from bare JID
On Wed, Apr 08, 2009 at 09:05:46AM -0600, Peter Saint-Andre wrote: On 4/8/09 6:36 AM, Dave Cridland wrote: On Wed Apr 8 11:21:51 2009, Robin Redeker wrote: Hi! I've recently stumbled across unavailable presence stanzas on s2s connections as replies to probes: presence from=b...@jid to=... type=unavailable / Right, we hand those out too, whenever users are known to be offline. We'll also timestamp them, and include status/, if we have that information. Hmm, is this in response to a probe? That might make some sense, then. It is sent response to a probe in jabberd2. But I'm wondering about the general semantics of the unavailable presence from a bare JID of a contact (or even a non-contact). After thinking a bit more about presences I went into some other issues which I didn't find explained in the RFC or didn't make sense to me: 1. I'm also wondering about the general semantics of _available_ presence from a bare JID, like discussed in another branch of this thread. Imagine these presence stanzas are received by a client for contact a...@b: presence from=a...@b/X to=m...@jid/myres/ presence from=a...@b/Y to=m...@jid/myres priority10/priority showxa/show /presence presence from=a...@b to=m...@jid/myres showaway/show /presence What should I display? Is the last presence from 'the empty resource'? Empty resources make no sense, as any resource's name must not be empty anyways (see BNF in RFC 3920). But whats the alternative interpretation of presence from a bare JID? Does it shadow any other presence? Is it guaranteed that a client will not receive presence for a bare JID and full JID from the same contact? If services send presence from bare JIDs, how should those be handled? What is the meaning of priority/ elements in those presences? 2. Another question I got is: RFC 3921 bis 4.5.4. Client Processing of Inbound Unavailable Presence http://xmpp.org/internet-drafts/draft-saintandre-rfc3921bis-07.html#presence-unavailable-client 1. If the user is in the contact's roster, the client MUST display the unavailable presence information in an appropriate roster interface. Imagine that some contact was online with 2 resources which become unavailable and send me their reason for becoming unavailable: presence from=c...@d/X type=unavailable priority20/priority statusOff for vacation./status /presence presence from=c...@d/Y type=unavailable priority10/priority statusOff for picking girlfriend up./status /presence Ignore the contradicting offline status for now, but for which' of those should I display the unavailable information? For both resources? For the unavailable presence which has been received most recently? Or for the unavailable presence with the highest priority? Lets assume the client should display the unavailable presence for _all_ resources which went offline. Wouldn't that be inconsistent with presence probes where the server returns unavailable information for only one resource? For this read: 4.3.2. Server Processing of Inbound Presence Probe 2. Else, if the contact has no available resources, the server SHOULD reply to the presence probe by sending to the user the full XML of the last presence stanza of type unavailable received by the server from the contact ... This means a server only remembers the last received unavailable presence, but a client _must_ display all unavailable presences it receives. That feels a bit inconsistent to me. Also imagine what happens when people use random resources, then the list of unavailable presences (which MUST be displayed appropriately) will grow indefinitely! Ok, this is maybe no appropriate, but what _IS_ actually appropriate in this case? -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Presence distribution
On Tue, Mar 31, 2009 at 05:36:59PM +0100, Dave Cridland wrote: On Tue Mar 31 16:00:20 2009, Pedro Melo wrote: 1) as much as you have right now: remote servers can do whatever they want with your presence, you have no control after they leave your own server (a pessimist would even say that you don't have any control after they leave your client); Well, there's trust, and responsibility. If a contact has an outright evil server, there's little you can do - similarly if your own server, or even client, is subverted. For things like this, it's really game over. We generally choose to model a trusted client and a trusted server, although we assume an untrusted server for content in the case of e2e encryption. But then there's responsibility - right now, it's inarguably your local server which enforces who gets your presence, and a reverse-roster lookup causes immediate problems there - if people don't get to see your presence (or worse, do when they shouldn't), you've then got two places to debug instead of one. 2) if subscription state gets out of sync, the protocol will behave better or worse than the current version :), it really depends which roster is correct. There was a thread 2 or 3 weeks back with a tentative solution to keep subscription state in sync using the initial presence type=probe /. I'm not sure your initial analysis is correct, and it's related to the above comments I make - if rosters are out of sync now, while it's tricky to discover, you know where the blame lies instantly in each case. And a rough analysis suggests that in the current situation, you'll either get presence when you *no longer* wanted it, or else you won't get presence when you want it. With a reverse roster lookup, it's possible to end up in a situation where you'll get presence when the If the server does route the wrong information to the wrong people it's buggy. With a reverse roster lookup on a corrupted database anything might happen. I don't get why this would justify the horrible amount of redundant information that is sent. I want to tell my 10 contacts on jabber.org that I am online. So sending 1 stanza will be 10 times less parsing, less bandwith, less CPU cost for compression, than sending 10. That factor scales with the number of contacts on a roster, which is linear, which is incredibly bad for something that can be accomplished with O(1) complexity. If the problem is synchronization of subscription states, well, then fix the problem with synchronization of subscription states. sender doesn't want you to, which is substantially worse, and without any bad actor involved. (I could be wrong here, please check this.) So we argue to optimize the protocol for this case? Hmm, the other side might get information I didn't want to tell him, well, lets send it 10 times instead of once. I guess that will 'fix' the problem. I understand what you mean: Going from 10 to 1 message will indeed be a privacy problem in case of lost 'unsubscribed' stanzas. But the solution is to ask why the unsubscribed stanza was lost. XEP-0198 will address at least parts of this problem. But let me clarify something: although the bandwidth and processing savings of a multi-cast approach to presence distribution should be relevant, I don't believe this is compatible with a lot of other protocols that we have, so although I think multi-cast presence is a worthy long-term goal, I don't think we are ready to address it at this moment. We have this famous thing about ~87% (the figure varies) of traffic being presence, and I'm afraid I think these figures skew vastly the moment compression takes effect, since, by the very nature of the data we're sending, it compresses really well. In fact, I'm not sure that it's even worth worrying about as a problem, given how well this should compress. Oh yay, let's burn some more CPU cycles, energy is sooo cheap. Everytime you send out 10 instead of 1 presence stanzas an additional amount of warmth is added to the global warming. Compression is a good optimization if you don't have anything else to save anymore and really want to pay for the CPU time. But in this case, wouldn't be preventing the redundancy in the first place be more beneficial in the overall performance? I basically refuse to consider any solution seriously unless you're measuring the effects of it post-compression - and I agree that's hard. Everytime you compress a redundant presence stanza a tree dies. Please think of the trees. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Tue, Mar 31, 2009 at 08:17:43AM -0700, Justin Karneges wrote: On Tuesday 31 March 2009 08:07:15 Fabio Forno wrote: On Tue, Mar 31, 2009 at 4:05 PM, Peter Saint-Andre stpe...@stpeter.im [.snip.] Of course we don't want to require acks unless they are explicitly requested. Let's not veer too far off the path here. All we're trying to fix is the atomicity of 'u'. Proposed change: Don't send 'u' over the wire anymore. The value shall be derived by the number of stanzas sent, starting from 0 (no stanzas sent yet). C: message/ C: r/ S: a h='1'/ We'd still have r and a ack elements. They'd just only ever contain 'h' now. +1 - Sounds good and clean to me :) Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Question on 3921 and how the connection between servers and clients is maintained.
Hi Thomas! I'll try to answer some questions, but I'm just a client developer, so I might be wrong with some answers :) On Tue, Mar 24, 2009 at 10:20:10PM -0400, Thomas Bolioli wrote: All, I am chair of a standards body developing a standard for allowing devices to maintain their presence on the network while in a low power state. Here is more detail on what we are looking to accomplish. http://www.ecma-international.org/memento/TC32-TG21.htm http://www.ecma-international.org/memento/TC32-TG21-M.htm I am looking for some information on XMPP/Jabber and how it behaves in certain circumstances. I have read through the RFCs and http://xmpp.org/protocols/ but couldn't find the specific answer to my question so I figured I would ask here. Most of the info I found pertained to layers above layer 3/4 and it is the behavior of XMPP @ layer 3/4 that is most interesting for us right now. So here goes: Assuming for a moment this IM/presence scenario and keeping in mind that IM can be replaced by any service/resource provided by an XMPP extension and the use case should remain valid. I also understand XMPP to implement a SIP like protocol. Use case: Julie has her IM client up on client A. Client A is connected to server B. Jim is on client Z connected to Server X. In todays world, if client A goes to sleep/standby after a period of inactivity, her connection to Server B is severed. If Jim sends her a chat request or other message, that message is either terminated @ server B or held in lieu of Julie bringing the system back into a working S0 state. Lets say for a hypothetical, that when Julie's system went to sleep, it no longer dropped off of the network (ie; it maintained it's IP, etc). Julie is waiting for directions for the restaraunt where she will be meeting Joe. Julie is off reading a book and is to leave once she gets word from Joe with the location. Yeah, there will be better ways to get this info to Julie but lets assume cell phones with SMS haven't been invented yet. :-) After 20 minutes her system (client A) enters sleep and the IM client drops its connection to the server (Q.1 Is there a connection to drop?). There will now be a Yes, Client A communicates in XMPP via a TCP connection with Server B. The problem is, that as soon as Client A drops the connection the resource of Client A is broadcasted as unavailable and all messages to Julie are either sent to other resources or stored on the Server B. delay between when the information arrives on Server B and when Julie gets it because Julie needs to wake up client A in order for the IM client to check in and retrieve the cache of backlogged messages in the queue on server B. Q2. Does XMPP have a facility for Server B to send a message to client A when all are on public IPs? There is no communication channel between Server B and Client A other than the TCP connection that is initiated by Client A. I currently don't know about any servers or extension protocols that 'notify' the Client A when it is offline. Q3. When client A is behind a NAT (ie; does XMPPs version of SIP have nat traversal)? What do you mean btw. with 'SIP like protocol' above and 'XMPPs vrsion of SIP' here? (I'm not too familiar with SIP). All client2server connections are initated and maintained by the client software. The server has no way of contacting a client when it is offline. It's not like SIP that clients are always running and can be contacted directly when they are not connected. The only thing that afaik has to do with SIP in XMPP is the extension protocol for audio/video/file-transfer between two clients on the network. That protocol (Jingle) uses ICE to create a communication channel between Client A and Client Z. But no such protocol exists for Client A and Server B - at least as far as I know. So if the message is truly pushed to client A from Server B, client A simply needs to wake up to gather/respond the incoming message and the PC sitting next to the reading Julie will alert her within a short timespan that Joe has sent his message to her. Since client A never left the network, IP routing is still possible and the TCP SYN is routable to the waiting client (assume a route was available to redirect from the router to the PC in the case of a NAT scenario). Q5. The client and server communicate via SOAP or a SOAP like XML based transactional protocol over TCP, correct? Not at all. The protocol on the wire is a pseudo-XML streaming protocol, which uses the nesting of XML tags for framing. Messages, presence information and other queries are currently pushed from client to server and from server to client without any acknowledgements. Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 01:49:31PM -0500, XMPP Extensions Editor wrote: Version 0.6 of XEP-0198 (Stream Management) has been released. My main problem is to understand the purpose of being able to include u _and_ h in a r or a element. If each r has to be answered with an a OR r it seems a bit redundant to me to make two different elements. So in the end wouldn't a s (standing for 'synchronize') element be enough: C: message/ C: s u='1'/ S: message/ S: s h='1' u='1'/ S: message/ S: s u='2'/ S: message / S: s u='3'/ C: s h='1'/ - here a client _might_ want to wait a certain amount of time, then, if the second message from the server has been received it could omit the synchronization element for the h='1' and could just send this one: C: s h='2'/ C: message/ C: s u='2' h='3'/ That kind of combining the u and h attribute would require additional logic. So a primitive implementation might send a s each time it received an s with an u attribute: C: message/ C: s u='1'/ S: message/ S: s u='1'/ S: s h='1'/ S: message/ S: s u='2'/ S: message / S: s u='3'/ C: s h='1'/ C: s h='2'/ C: message/ C: s u='2'/ C: s h='3'/ In the end it's just synchronization of two counters over a wire (along with keeping track what the other entity received if it received my counter value). Other than this I've found the XEP to answer all my immediate questions that arised when I read the previous version. I guess it's time to implement it :) Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 04:35:17PM -0700, Justin Karneges wrote: On Thursday 19 March 2009 16:08:00 Robin Redeker wrote: On Thu, Mar 19, 2009 at 01:49:31PM -0500, XMPP Extensions Editor wrote: Version 0.6 of XEP-0198 (Stream Management) has been released. My main problem is to understand the purpose of being able to include u _and_ h in a r or a element. If each r has to be answered with an a OR r it seems a bit redundant to me to make two different elements. The reason there are two elements is because you may not want to request an ack every time you bump the counter. For example, on an otherwise idle connection, you could send message followed by a, and that's it, and the other side has no obligation to ack anytime soon. Maybe after sending 50 messages you'd throw an r out there just so you can finally cross off the messages in your outbound queue. This is more optimized than having the other side ack every message. Well, if you don't want to request, just not send a new u attribute with the s element I proposed: C: message/ C: message/ C: message/ C: message/ C: s u='4'/ S: s h='4'/ Thats the same as this IMO: C: message/ C: message/ C: message/ C: message/ C: r u='4'/ S: a h='4'/ As the RFC says, an r can be acked by a or an r element with an 'h' attribute. So the real difference is only whether either an 'u' or an 'h' value is present in the r or a element. The receiving side of an u attribute knows that it has to tell the sender (with an h attribute) that it received that sequence number. Or do I miss something? Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] UPDATED: XEP-0198 (Stream Management)
On Thu, Mar 19, 2009 at 05:03:29PM -0700, Justin Karneges wrote: On Thursday 19 March 2009 16:52:14 Robin Redeker wrote: Thats the same as this IMO: C: message/ C: message/ C: message/ C: message/ C: r u='4'/ S: a h='4'/ That's not reliable though. What you want to do is this: C: message/ C: a u='1'/ C: message/ C: a u='2'/ C: message/ C: a u='3'/ C: message/ C: a u='4'/ That way, if the connection dies in the middle somewhere, it can be resumed properly. So, you want to be able to indicate 'u' without requiring that the other side ack immediately. I thought, from reading the XEP, that the sequence number indicates the number of stanzas sent. It seemed so obvious to me :) +1 for the idea in the other part of this thread about making the 'u' update implicit with each stanza. It never felt safe to me to seperate the request for acknowledgement from the thing that is to be sent over the stream. Robin -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] presences and server restarts
Hi! I've recently tripped over another really annoying ghost presence while I was chatting with someone. He always showed up as online, while he actually wasn't, just that I never got a 'unavailable' presence stanza and wondered whether my client was actually broken (which it wasn't as a glance on the stanza log indicated). It was a private server on the other end, so something that easily restarts/crashes from time to time. So it might be one of those ghost presences. The issue of ghost presences is of course known, and I don't know currently I was reminded of the concept of rpc.statd by a friend: It's there to inform other NFS server about unclean restarts. So, in XMPP that would be: A server telling the other server that he would love to receive a message when the other server restarts/crashed. Then the servers that receive that notice could send presence probes again for the contacts on the server that restarts. That would probably be at least less overhead than re-probing again and again, like it's suggested in rfc3921bis: In addition, a server MAY periodically send a presence probe to a contact if it has not received presence information or other traffic from the contact in some configurable amount of time; this can help to prevent ghost contacts who appear to be online but in fact are not. I don't know if this problem has already been solved, I couldn't find much on a quick look over the XEPs. Just wanted to throw in another idea, just in case... Greetings, Robin Redeker -- Robin Redeker | Deliantra, the free code+content MORPG el...@ta-sa.org / r.rede...@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Messages to unsubscribed contacts and resources
Hi Joe, thanks for the reply! On Fri, Dec 14, 2007 at 07:09:26AM -0800, Joe Hildebrand wrote: On Dec 12, 2007, at 7:04 AM, Robin Redeker wrote: Ok, I have another question: We have: Contact A with one resource (resource is 'res1'): [EMAIL PROTECTED]/res1 And Contact B with two resources: [EMAIL PROTECTED]/res1 priority 20 Assume you mean contact_b. Yes, indeed :) [EMAIL PROTECTED]/res2 priority 10 They are not subscribed to each other and do NOT share any presence information. Even directed presence? The cool side effect of sending a directed presence is that everyone to whom you have sent directed presence will get an unavail when you go offline. Yes, even directed presence not. I know that directed presence makes things easier in this case, but I assume here that there are still clients and people that don't do or don't want to send the presence. [.snip.] Should he make a second 'chat session', which is probably not very intuitive for the user because he just wanted to talk to contact B and when he got two options to send the message, what should he do (he doesn't even know which resource is the 'right' one, or whether 'res1' maybe went offline)? You're correct that there shouldn't be a new visible session. There were clients that did that back in the day (or with each new thread!) and it turned out to be pretty confusing. You should just rebind to the new resource. Ok, that also makes things easier in the code. What if they share presence, should contact A's client determine from the highest priority resource where the message should go to? Absolutely not. You should just unbind at least when res1 goes offline, and perhaps whenever you get a presence change from contact_b. Keep in mind that b's server might be doing much more sophisticated most available calculations. But what if contact B first replied from the resource 1 and also sent a directed presence to contact A, _and_ then changes to resource 2 to talk to contact A, should contact A rebind (I would guess that would make sense)? Alternate question: What if contact_b's server is using RAP (XEP-168)? Then contact_a can know what the new most available resource is? My answer to that is that if the resource contact_a is bound to was the primary resource, and now isn't, you should unbind, otherwise not. Well, what in the case I explained in my last paragraph (about contact B, who changes the resource without changing presence settings)? Should contact A still send to the primary resource or still rebind to the last resource he got a message from? Sorry for my complicated and very technical questions, I just want to get it right :) Thanks for your time! Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Messages to unsubscribed contacts and resources
On Tue, Dec 11, 2007 at 10:28:55AM -0700, Peter Saint-Andre wrote: Robin Redeker wrote: Hi! Some days ago I had a mail discussion on the jdev@ mailing list about messages to unsubscribed contacts and contacts in general. Tomasz said that messages should generally go to the bare JID instead of the full JID, and that the local routing is up to the server. He meant, chat sessions should be kept track of with the thread element, which completly makes sense to me. [.snip.] Please also note that the term 'chat session' in that paragraph is quite undefined, or at least it's meaning is a bit fuzzy to me. At its simplest, I think this is a chat session: [.snip.] The sharing of directed presence gives both parties more knowledge about availability, but I think it should be initiated by the recipient (and subject to client prompt on a per-session basis, or configuration in the client to auto-share directed presence). [.snip.] Of course, If I don't have to keep track of the resources, that would _greatly_ simplify everything for me. Just sending to the bare JID and leaving the rest up to thread and the contacts routing settings would make enormous sense to me. Sending every message to the bare JID is not the custom. Back to section 5.1.1, the sections somehow contradicts the section 8.3.1.1 (Message): For a message stanza of type chat, error, groupchat, or normal, the server SHOULD deliver the stanza to the highest-priority available resource. That 'feature' only makes sense if at least the initial message goes to a bare JID. But if it is routed to a resource by the server and I have no knowledge about the presence of that resource (eg. if I'm not subscribed), where should the next message go to, to the full JID I received a reply from? You should keep sending to the bare JID until you receive a reply from a full JID, then start sending to the full JID. Ok, I have another question: We have: Contact A with one resource (resource is 'res1'): [EMAIL PROTECTED]/res1 And Contact B with two resources: [EMAIL PROTECTED]/res1 priority 20 [EMAIL PROTECTED]/res2 priority 10 They are not subscribed to each other and do NOT share any presence information. Contact A sends a bare JID message to contact B. Contact B receives the message at res1 (due to highest priority), and answers from that resource. Contact A receives the full JID of that resource now and 'binds' the conversation to it and they chat. Later Contact B switches to his laptop, with another client bound to the other resource 'res2'. And he sends a message to the bare JID (because the client there doesn't know the full JID of contact A) of contact A. How should contact A proceed? Should he bind (and reply) the conversation to res2? Should he make a second 'chat session', which is probably not very intuitive for the user because he just wanted to talk to contact B and when he got two options to send the message, what should he do (he doesn't even know which resource is the 'right' one, or whether 'res1' maybe went offline)? What if they share presence, should contact A's client determine from the highest priority resource where the message should go to? Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
[Standards] Messages to unsubscribed contacts and resources
Hi! Some days ago I had a mail discussion on the jdev@ mailing list about messages to unsubscribed contacts and contacts in general. Tomasz said that messages should generally go to the bare JID instead of the full JID, and that the local routing is up to the server. He meant, chat sessions should be kept track of with the thread element, which completly makes sense to me. On the other side RFC3921bis says: Section 5.1.1: If the message is being sent in reply to a message previously received from an address of the form [EMAIL PROTECTED]/resource (e.g., within the context of a chat session), the value of the 'to' address SHOULD be of the form [EMAIL PROTECTED]/resource rather than of the form [EMAIL PROTECTED] unless the sender has knowledge (via presence) that the intended recipient's resource is no longer available. That of course also makes sense, but my problem was: What if the sending contact does not know about the presence of that resource? When should he stop sending to [EMAIL PROTECTED]/resource? Should he, if he has no knowledge about the presence, send to [EMAIL PROTECTED] generally? Please also note that the term 'chat session' in that paragraph is quite undefined, or at least it's meaning is a bit fuzzy to me. And is somewhere described how full JIDs and thread play together? If at all? If routing is up to the local server side, makes it sense to reveal resources at all? Wasn't there a progress towards randomized resources? Sorry for so many questions, but I'm a little bit confused. I try to get the conversation aspect in my client right, and one problem I stumbled accross was the fact that my client has no 'windows' or 'tabs' which could control the extend of a 'chat session'. In my client there is always a 'tab' to a contact, and the extends of a 'chat session' are very fuzzy and undefined. If the term 'chat session' in XMPP means I have to keep track of the session via some special hacks with resources and thread, I would have to complicate the whole thing a bit. That is of course maybe only an issue with my special client. But before I can implement anything resembling 'chat sessions' that term must be more explictly defined. Of course, If I don't have to keep track of the resources, that would _greatly_ simplify everything for me. Just sending to the bare JID and leaving the rest up to thread and the contacts routing settings would make enormous sense to me. Back to section 5.1.1, the sections somehow contradicts the section 8.3.1.1 (Message): For a message stanza of type chat, error, groupchat, or normal, the server SHOULD deliver the stanza to the highest-priority available resource. That 'feature' only makes sense if at least the initial message goes to a bare JID. But if it is routed to a resource by the server and I have no knowledge about the presence of that resource (eg. if I'm not subscribed), where should the next message go to, to the full JID I received a reply from? Will my messages, if that contacts resource goes offline, be dropped without my knowledge? Thanks for your time :) Robin -- Robin Redeker | Deliantra, the free code+content MORPG [EMAIL PROTECTED] / [EMAIL PROTECTED] | http://www.deliantra.net http://www.ta-sa.org/ |
Re: [Standards] Binary data over XMPP
On Fri, Nov 09, 2007 at 10:01:39AM -0700, Joe Hildebrand wrote: On Nov 9, 2007, at 8:47 AM, Tobias Markmann wrote: There are already several binary-to-text encodings which perform a bit better than Base64, two of them are: 1. http://en.wikipedia.org/wiki/ASCII85 invented by Adobe 2. http://base91.sourceforge.net/ Both of those seem to allow and , which make them less than ideal for embedding in XML. XMPP is not XML :-))) R
Re: [Standards] Binary data over XMPP
On Tue, Nov 06, 2007 at 10:16:19AM +, Dave Cridland wrote: [.snip.] You would no longer be able to do that with binary blobs; you would have to special-case blob stanzas fairly heavily, since I guarantee you that if the characters '' or '' appear un-escaped in the binary blob, Expat will choke and die. Sure, but there's two options with an escaping mechanism - either synchronized or non-synchronized - and they can be negotiated easily. With a non-synch mechanism, the sender just sends out the blob/ element, then sends out the binary data, then continues with XML. It can be done in a single TCP packet, but it requires that the receiver processes the data into stanzas prior to processing through the XML parser. Some receivers already do this, so it seems reasonable that this can be an option. Also take into account that the sender also has to customize the XML writer to allow writing raw octets, which breaks multiple layers of nice cosy XML abstractions. (Of course with current XMPP you already need a XML writer that allows some customisations). With a synch mechanism, the sender sends out a blob/ element, and then waits. The receiver then says it's ready for binary data (sending a stanza to indicate this), and the sender then sends the binary data - followed immediately by more XML as required, since a binary parser is going to be octet counting anyway. For people who parse all the network traffic at once through a SAX-like parser, this should work fine, at the expense of some efficiency. Note that anyone can send non-synchronized blobs, but not everyone can receive them, so a client (for instance) which is built to stream network data directly into a SAX parser can still *send* blobs efficiently. How do you propose the receiver determines the end of the binary data? Is it going to be prefixed by a lenght? Generally: pumping binary crap through XMPP is another big step _away_ from XML compatibility. Also transforming a stream (TLS) into packets (stanzas) which are used to again emulate a stream (TLS) sounds crazy to me anyway :) To encrypt stanzas (packets) with TLS (stream encryption)... *shudder* :) Robin
Re: [Standards] Binary data over XMPP
On Mon, Nov 05, 2007 at 04:04:10PM +0100, Michal 'vorner' Vaner wrote: Hello On Mon, Nov 05, 2007 at 02:45:05PM +, Dave Cridland wrote: Another option would be to setup a distinct connection (and protocol) for routing blobs, and so send them through the server, yet not in-band. I'm not comfortable with this, because it means essentially duplicating all security information, and maintaining synchronization between two distinct streams. Or make the connection blobs by default, and some blobs could contain complete XML documents, like this: lenght of first block message to='bla' length of second block iq ... length of third block some binary data. It is as much drastic approach as the blobs, it changes the protocol from the very basic ground. Furthermore, you can extract the stanza and feed it to any XML parser. +1 for real protocol frames! R
Re: [Standards] Binary data over XMPP
On Mon, Nov 05, 2007 at 06:47:40PM +0100, Michal 'vorner' Vaner wrote: Hello On Mon, Nov 05, 2007 at 06:27:33PM +0100, Robin Redeker wrote: On Mon, Nov 05, 2007 at 04:04:10PM +0100, Michal 'vorner' Vaner wrote: It is as much drastic approach as the blobs, it changes the protocol from the very basic ground. Furthermore, you can extract the stanza and feed it to any XML parser. +1 for real protocol frames! Actually, I was just showing, how deep change was the blobs thing. I'm against changing the whole infrastructure inside out. I didn't mean to propagate the frames, I just took them as example. Heh, ok. Just wanted to take the chance to promote the idea once more :) R
Re: [Standards] file transfer
On Tue, Aug 28, 2007 at 09:21:43PM +0100, Richard Dobson wrote: Peter Saint-Andre wrote: [.snip.] I understand. But SI and Jingle essentially perform the same functions, since they both enable negotiation. Yes but jingle also provides access to ICE and pseudo-tcp so it would be helpful rather than have the entire file transfer negotiation using just jingle if it was separated in two so that all the jingle side is doing is negotiating a generic bytestream that can be then reused by things like SI, to me this seems like a sensible compromise as there is no real benefit that I can see of doing the whole thing in jingle as its just duplicating effort, whereas creating a new bytestream method is complimentory, anyone else think its a good idea? I generally thought that reimplementing all that with jingle is a bad idea. And I as client/library author surely don't like the idea of duplicating code or functinality. I agree completly with you. Robin
Re: [Standards] Comment to rfc3921 pt 11.1 : handling of messages to ressources with identical priorities
On Fri, Aug 24, 2007 at 12:11:09PM -0600, Peter Saint-Andre wrote: Robin Redeker wrote: [.snipped arguments for broadcasting.] I actually mostly care about sane and _clear_ and simple semantics for handling cornercases. It clearly defined what users, developers and admins can expect in such cases. This feels like an implementation detail to me. At the least, I would not want to say that if two resources have the same priority then the server MUST deliver to both resources, because that closes the door to more intelligent routing based on advanced knowledge that the server may have from user-configured routing rules or the like. The RFC defines whether this ia an implementation detail or not. Currently the RFC delegates handling of this corner case to the implementors. If servers are free to handle this case any way they want they will handle it differently. Users won't have a clue what will happen. As someone on this thread posted: they even expected broadcasting in the first place. That expectation might be an exception, haven't made a poll about this yes :-) Without a SHOULD the users, developers and admins still can't rely on a defined behaviour. (defined in a sense that users (and also their clients) have no idea what behaviour the server is or might be implemented for this case). All I want is the ability to tell Aunt Tillie what she can expect when she has two clients with the same priority. If there is nothing to expect this will happen: Asking her: Oh, I don't know what might happen, which server software and settings has the XMPP server you are using? She: Uhm, I don't know, there is no website. Me: Oh, just ask the admin! She: Will do that... *writes mail/xmpp message to admin* Admin: We send the message to a random resource if the priorities are the same. She: Thats bad, I don't want that. What should I do? Me: Easy, just pick a server which does the right thing. She: Which would be a server that just broadcasts the messages to all resources? Me: Easy, just go to http://www.jabber.org/user/publicservers.shtml and send mails to all admins to ask them. She *does that* She: Ok, got it now, 12jabber.net does it. I've created an account there now, but now I have to move my roster there and reprint my vcards... I don't know what the user will expect, some seem to expect broadcasting other some clever algorithm like: last resource that sent a message, or whatever. But they certainly won't have much abilities to find it out or rely on a default behaviour. Maybe best would be to make the default behaviour broadcasting and then specify a way to configure the handling of this case by eg. ad-hoc commands on the server for an account. Then the RFC could say: Unless other behaviour was configured for an account the server SHOULD/MUST broadcast the message to all resources with same priorities. That might be a bit overkill, but it would be a defined, simple and most reliable default handling for that case. And with per account configuration it would even be open to future more intelligent routing. R
Re: [Standards] IMML
On Mon, Aug 06, 2007 at 12:17:39PM -0700, Rachel Blackman wrote: XHTML-IM is for sending HTML messages. IMML is for sending modern Instant Messages. IMML intentionally leaves out most of the flexibility that XHTML-IM provides, most of which has no semantic meaning whatsoever. We might as well be using XSL-FO. Imposing rules such as you suggest for HTML a's just adds to the complexity of implementation, and illustrates that HTML in any form is simply the wrong tool for the job. This is an argument that will never be resolved. We have people who want to be able to put every single HTML object (including Java, Active X, DHTML and so on) into messages and feel XHTML-IM is far too restrictive and anemic; they argue you should be able to paste HTML in from Firefox and have it display exactly as it was. Then every XMPP client would have to embed the firefox rendering engine for that. Displaying HTML (mostly any kind, XHTML, HTML 4...) in different browsers will lead to different results (in the real world). Maybe they should post screenshots instead :-) ? And then we have people who feel XHTML-IM is too complex and is overkill for the situation. XHTML-IM manages to strike a (somewhat sensible) middle ground, which is probably why it will stick around. :) There is one big difference between XHTML-IM and IMML, which are simply a matter of semantic. XHTML-IM says how to exactly display a text, what color, what size, what padding, where to place an image, or whatever. IMML just says display this with emphasis, and this is an URL, display it approriate. Clients can then choose how to put emphasis on some text (eg. by rendering it bold). XHTML-IM doesn't leave such a choice, either the client can render something bold or he has no way to display a bold element. I also disable XHTML-IM always, because the people seem to never get it right to send just normal text. Often their text is smaller than mine and some even think it's funny or very readable to have some big and colorful text. While I agree that lots of todays youth just loves these gimmicks and that it might be important to support colors and images and all that, I also want to have a middle way between XHTML-IM and no formatting at all. IMML is seems to be restricted enough to provide a reliable way to display an instant message while not enforcing how something might be displayed. For XHTML-IM one must roll a more or less complete XHTML-IM parser and CSS1 parser. I agree that it is not impossible to implement it due to the very limited set of XHTML and CSS1. But XHTML-IM leaves the destination client almost no freedom in displaying the text. A console client will have no other choice than just ignoring the XHTML-IM content of a message or run some heuristics over the html and somehow find out what color or attributes a text might have. IMML is way easier to implement, and clients and users still have choice how to display something with emphasis. IMML moves the choice to the receiving end, XHTML-IM leaves the choice to the sender. It's mostly a semantic difference. And afterall, instant messages are usually not documents. Just my 2 cents. R
Re: [Standards] JID Escaping
On Fri, Jul 27, 2007 at 02:32:39PM -0600, Peter Saint-Andre wrote: Matthias Wimmer wrote: Robin Redeker schrieb: I propose to rename the XEP to make clear that this escaping/unescaping should only happen in very rare cases (only at gateways or heavily specialized client frontends). And that the terms 'escaping' and 'unescaping' are replaced by 'mapping' and 'unmapping', because thats what is happening here. +100 Well, it's interesting, on the ejabberd list today someone said they have an existing database of 45k email users and they want to offer Jabber services to that user population, but re-use the same usernames. I'm sure they have some users in there with addresses containing characters like single quote, e.g., tim.o'[EMAIL PROTECTED] In which case I bet that they'll be interested in using JID Escaping. I really feel that this discussion is not going anywhere. The spec is IMHO pretty clear. If you don't like the spec, don't implement it. Ok, update on \ quoting: I tried to register users with \ in their nodepart in servers and authenticate via SASL DIGEST-MD5. These servers didn't work: - ejabberd (eg. at jabber.org) - jabberd2 (with gnusasl as SASL backend, DIGEST-MD5 code broken, scod however works) - openfire at igniterealtime.org I already filed a bugreport against ejabberd, they seem to have also their own code to handle DIGEST-MD5 as far as I understood the code. I filed a bugreport to ejabberd and it seems to be fixed sometime ( see http://www.jabber.ru/bugzilla/show_bug.cgi?id=362 ). I also informed the jabberd2 author about the problem with gnusasl. It seems that there are versions of jabberd2 which support it, eg. the server jabber.wavenet.pl which uses: jabberd sm 2.0s6. I confirmed that that version uses their own sasl implementation. igniterealtime.org seems to use Openfire Enterprise 3.4.0 Alpha 2. ejabberd seemed neither to work on jabber.org nor my local setup, neither versions 1.1.2 nor 1.1.3 seemed to work. If someone tests with ejabberd and does NOT get an error he should try to use a client that actually not has a quoting bug too. tkabber for example doesn't quote the \ in the sasl response, so as ejabberd doesn't unquote they play fine with each other. Of course tkabber will blow up on servers that do right unquoting (confirmed that with jabber.wavenet.pl). This is the state of \ in JIDs and DIGEST-MD5 in the current public server network, one might want to look at this statistic: http://www.ta-sa.org/files/txt/a5cc5ae8318c76f40f9912daff6ef6af.txt That are 680 version responses from various public servers advertised on xmpp.org.ru, xmpp.net and jabber.org. (btw. I sent requests to 713 servers, but 33 seemed not to be reachable or didn't support the software version feature). It seems that at least the majority of the whidely deployed servers support \ in JIDs in DIGEST-MD5 SASL authentication due to faulty SASL DIGEST-MD5 implementations. However, using the PLAIN authentication method seems to work! Robin
Re: [Standards] JID Escaping
On Thu, Aug 02, 2007 at 05:21:46PM +0200, Robin Redeker wrote: It seems that at least the majority of the whidely deployed servers support \ in JIDs in DIGEST-MD5 SASL authentication due to faulty SASL DIGEST-MD5 implementations. Sorry, I meant: ... support NO \ in JIDs in DIGEST-MD5 ... Robin
Re: [Standards] summary: allowable characters
(Warning, long mail ahead! Get a coffee and some time first :-) On Thu, Aug 02, 2007 at 03:34:30PM -0600, Peter Saint-Andre wrote: Mridul Muralidharan wrote: The problem essentially is that any place where we have a JID persisted in the backend (user roster, acl's, affiliations, privacy lists/block lists, etc), it will become incompatible change. For example, what used to be [EMAIL PROTECTED] will now become contact[EMAIL PROTECTED] - causing incompatibilities. Well we're having a long discussion about this in the jdev room right now: http://www.jabber.org/muc-logs/[EMAIL PROTECTED]/2007-08-02.html I volunteer elmex for posting a summary once we're done. :) Yes, basically Mridul is completly right, we can't do much about the already deployed backslashes in JIDs. Especially in 1.0 server rosters. But... First we have to wonder whethere there are actually people with [EMAIL PROTECTED] in their roster, as registering a JID with a \ in the username is a considerable problem with XMPP 1.0 servers with SASL and DIGEST-MD5 (see some older message from me in the JID escaping thread). Of course that should be further investigaged as old-style IQ auth works with [EMAIL PROTECTED] and also some jabberd2 servers allow authentication as [EMAIL PROTECTED] without problems. But there exists a possibility to migrate our old JIDs to the 1.1 world and staying interoperable with 1.0 servers: First: A 1.1 server that is going to communicate with 1.0 server will escape the JIDs from his userbase when he SENDS to a 1.0 entity. Escaping can be performed as described in XEP-0106 (after dropping the silly \20 escaping rule). That will work great if the 1.1 server has NO old userbase. If we have for example jabber.org, a large userbase, and there is actually [EMAIL PROTECTED] as registered user in. And we want to upgrade to a 1.1 server then we will run into the problems Mridul pointed out: 1.0 servers have [EMAIL PROTECTED] in their roster, and if we have now 'stpeter @jabber.org' registering a new account he will collide with that, because his JID will be escaped to the in the 1.0 servers roster existing [EMAIL PROTECTED] Bang, we got a collision. There exists no real easy way to prevent that except just not allowing 'stpeter @jabber.org' to register. To detect a case like this, that a new user with a colliding JID registers, the 1.1 server needs to keep track of the old JIDs in his database. If the 1.1 server knows that [EMAIL PROTECTED] is a JID from the pre-1.1 times, he can assume that [EMAIL PROTECTED] is already in some rosters out there. So he MUST NOT allow anyone who might collide with that to register at jabber.org after the migration to 1.1. So when upgrading jabber.org could just mark all JIDs with a \ in their name to be a pre-1.1 JID and disallow anyone to register who might collide with one of the registered JIDs. This way ' [EMAIL PROTECTED]' can register if no '[EMAIL PROTECTED]' existed before (he knows that from his database with the marks of old JIDs). If ' [EMAIL PROTECTED]' now wants to talk with '[EMAIL PROTECTED]', it would look like this: message from= [EMAIL PROTECTED] to=[EMAIL PROTECTED] / As jabber.org (1.1) knows that chrome.pl (1.0) is in fact 1.0 he escapes like XEP-0106 recommends and sends actually: message from=[EMAIL PROTECTED] to=[EMAIL PROTECTED] / In [EMAIL PROTECTED]'s client will now popup a message from [EMAIL PROTECTED] and except some weird JID he can talk with him. Because if he sends a message back: message from=[EMAIL PROTECTED] to=[EMAIL PROTECTED] / Then jabber.org will unescape the to-field and deliver the message to ' [EMAIL PROTECTED]'. Of course this solution is not a perfect one for the end-users as I will describe below, but I argue that the incompatibilities will increase the pressure on developers a bit and on administrators to adapt XMPP 1.1. And thus that might speed up the migration while providing a compatibility-workaround for maybr 98-99% of the cases, or maybe even 99,% (this needs to be investigated a big IMO, maybe my assumptions are completly wrong). So much for the server-to-server interoperability. Now about 1.1 clients and 1.0 clients. 1.0 clients will have no way to reach ' [EMAIL PROTECTED]', which is fine, either the user knows that guy's JID needs to be escaped because he uses an old client, or he has to upgrade to a client with 1.1 capabilities (what this means is described below). Not being able to send a message to ' [EMAIL PROTECTED]' will increase the pressure on the client developers as stated above. So 1.0 clients are basically out of luck if the user don't know how to escape, however, tell em: get a new client. Of course it's blunt to say that, but I guess we can assume that not our WHOLE old userbase without spaces and all those fancy characters in their JID are NOT GOING TO signup a new account. So the users with spaces and or whatever in
Re: [Standards] [CDATA[ in XMPP
On Tue, Jul 31, 2007 at 01:28:43PM +0200, Alexander Gnauck wrote: Ralph Meijer schrieb: I am curious which software this holds for. Most XMPP implementations are based on expat, as far as I know. Sure they might have their own DOM's (my Twisted XMPP stuff does), but that is irrelevant for parsing. There is many software which is using expat, but there is also lots of software which has its own simple parsers. * I have my own XML parser in many projects * The Tigase and gloox have their own lightweight XML parsers * Lots of mobile clients have their own parsers Of course it may be possible to replace this parsers with feature complete XML parsers or add CDATA to them if it doesnt exists already. But there are reasons why the developers wrote their own parsers. The RFC3920 does not restrict usage of CDATA, so I always assumed it's a valid XML construct. But I would've never used it in my code just because I know that there are XMPP implementations which can't even read the XML described in RFC3920. Today we talk about CDATA, hey but what is coming next? There seems to be an obvious large need for clearing up the CDATA case. It was never restricted by the RFC, some RFCs even used it: http://www.xmpp.org/rfcs/rfc3923.html 3.1. Process for Securing Messages ... a sending agent MUST use the following procedure: ... Provide the resulting signed and/or encrypted object within an XML CDATA section (see Section 2.7 of [XML] (Bray, T., Paoli, J., Sperberg-McQueen, C., and E. Maler, “Extensible Markup Language (XML) 1.0 (3rd ed),” February 2004.)) contained in an e2e/ child of a message/ stanza ... Of course that RFC was never relevant in any implementation as far as I know, but you couldn't know whether it might become adopted for e2e encryption. Within this perspective it becomes even less sane to rely on CDATA-free XML in XMPP stream parsers. R
Re: [Standards] [CDATA[ in XMPP
On Tue, Jul 31, 2007 at 02:07:43PM +0200, Jacek Konieczny wrote: [.snip.] IMHO XMPP should become more XML compatible and not less. Both existing XML parsers and serializers should be usable for XMPP. If something has to be changed in the XMPP specification then it should be removing the incompatibilities and not adding more. +100! Robin
Re: [Standards] [CDATA[ in XMPP
A small mistake I made: On Tue, Jul 31, 2007 at 02:02:32PM +0200, Robin Redeker wrote: On Mon, Jul 30, 2007 at 06:49:45PM -0700, Rachel Blackman wrote: stream-send ( messagebody![CDATA[ Here: + chatwindow-get_message_from_user + ]]/body/message ); data = sprintf ( body![CDATA[%d: %s/%s]]/body, And here: get_int, chatsession-get_from_nick, chatsession-get_to_nick ); There one should still have to take care that no string with ]] is concationated into the CDATA block, as that would end the CDATA block. R
Re: [Standards] [CDATA[ in XMPP
On Mon, Jul 30, 2007 at 04:48:53PM -0600, Peter Saint-Andre wrote: Robin Redeker wrote: If we don't call something XML which clearly is not XML, then there aren't any further confusions possible. But still the term 'XML' is quite often used in the XMPP specs. Interestingly, the first paragraph of the XML specification says: *** Extensible Markup Language, abbreviated XML, describes a class of data objects called XML documents and partially describes the behavior of computer programs which process them. XML is an application profile or restricted form of SGML, the Standard Generalized Markup Language [ISO 8879]. By construction, XML documents are conforming SGML documents. *** Is it true that XML is SGML? Obviously some SGML pedants would object because XML is an application profile or restricted form of SGML. Yet XML documents are conforming SGML documents! So here we have a rough definition of conformance, and a working example since XML conforms to SGML. Similarly, a complete XMPP session (with start and end stream tags) is a conforming XML document. Just because the session does not include comments, processing instructions, DTD subsets, entity references other than those predefined in the XML spec, other other restricted features does not mean that the session is not an XML document. The difference is, which I consider the major confusing part for developers, that the XML recommendation doesn't always use the term 'SGML' when defining something. The XMPP spec very often uses 'XML' to describe things. So we could say this in rfc3920bis: *** The Extensible Messaging and Presence Protocol (XMPP) defines a class of data objects called XML streams as well as the behavior of computer programs that process XML streams. XMPP is an application profile or restricted form of the Extensible Markup Language (XML). A complete XML stream (including start and end stream tags) is a conforming XML document. *** Sounds nice. (Please take this mail as serious as you like, I don't know how serious I mean it.) Don't worry, I take all of your emails that way. :P Just take care that some mails have the intend to point something out, which I would like to have fixed or I really feel could make the life of other developers easier. I of course can only offer an opinion w.r.t. technical aspects of some things. That some things can't be fixed because of backward compatibility is unfortunate. But other things (still) can, just make sure to take at least those serious enough :-) R
Re: [Standards] [CDATA[ in XMPP
On Mon, Jul 30, 2007 at 06:16:24PM -0500, Mickael Remond wrote: Hello, Regarding this discussion, it seems that the topic started about the use of CDATA. So, we are talking about way to escape characters in an XML stream. My view is that all way to escape characters are good, especially when they are defined in XML, cited in XMPP RFC and are simple to implement (and implemented by all parsers I know). I have tested the use of CDATA for example with Psi, Gajim and a few other clients and all of them were fine. As Peter said, it should probably be allowed if not explicitely forbidden. My view is that it should be allowed as, as I understand it, it is just one of the possible way to escape characters (and not the worst one). I've been reading the XMPP RFC with an eye on exactly CDATA, and it says clearly that CDATA is allowed. An XML stanza exists at the direct child level of the root stream/ element and is said to be well-balanced if it matches the production [43] content of [XML]. See in the XML spec: [43]content::= CharData? ((element | Reference | CDSect | PI | Comment) CharData?)* CDSect is a CDATA Section. And the RFC3920(bis) doesn't say anything about excluding it. R
Re: [Standards] XEP-0045: roomnick case
On Fri, Jul 27, 2007 at 09:17:03AM +0530, Mridul Muralidharan wrote: Jack Moffitt wrote: Currently in XEP-0045, roomnicks are case-sensitive. To be precise roomnicks are handled according to the Resourceprep profile of stringprep: http://www.xmpp.org/extensions/xep-0045.html#bizrules-jids This means that the following roomnicks are all different: StPeter stpeter STPETER Some people have pointed out that this can be confusing to end users. Not only this, but it's very easy to steal someone's identity when they are not there. You just log in with their normal nick. At Chesspark we make all public rooms non-anonymous and we always show the full jid or (if they are in your roster) the roster nick for that person. That way no one is ever confused who is who. So we've basically solved this in the client. All this nick stuff stuff is so IRCish anyway. It's nice if what you want is an anonymous room, but for everything else, it's just a mess IMO. After trying several solutions we found that this was least confusing to everyone and we have gotten no complaints except for the occasional jerk that signs up with a name like st.peter or stpeter_, but there's little you can do about that but ban the offenders. jack. Unless rooms are not anonymous, clients have no way to validate who the participants are anyway - so it is not really stealing : you dont register the nick, it is not tied to your jid in any way. In anonymous rooms you dont know who the participants are - so relying on nicks to give a hint of the identity would be wrong. I guess the problem here is to explain this to the users. Robin
Re: [Standards] JID Escaping
On Wed, Jul 25, 2007 at 03:22:04AM +0530, Mridul Muralidharan wrote: Hi Robin, You should analyze as to who will actually need to encode or decode nodes. 1) Gateway's. I know that JID escaping is for gateways, and it makes sense to define a mapping. But not for ordinary 'I want a JID with an @ in my name!'-user-usage. For the purpose of mapping foreign uid's to xmpp nodes. In this case, ONLY the gateway, will need to do encode/decode. All other entities will treat is as a JID - without any transformation to node/domain/resource. It will encode a foreign id to node so that it becomes a valid xmpp JID. It will decode node from an xmpp 'to' for obtaining the destination in the foreign network. Makes complete sense and seems to me as the only sane application. 2) Client's and server's. When the node is same as uid, and the uid of the user contains prohibited characters. In this case, when client authorizes to server, it would encode the uid to obtain the node - and then form the JID to be used (Only the client encodes it as part of auth, and once authorization is over, encoding of the JID is opaque to the client). At the server, it would decode the node to obtain the actual uid and use that (backend store update, etc) - note that for purpose of routing, the server would treat the jid as-is, only decode it to identify the backend entry (database, ldap, file, etc) to which it has to persist/query data. (This is one way in which you would model things - servers can ofcourse come up with other alternate ways to do this). Other than for these two, I cant really envision any other important usecase which requires any entity (client, gateway, component or server) to encode or decode - the above are directly related to and required for xmpp routing. Hopefully, from this point of view, the xep might make more sense. I completly agree. Robin Redeker wrote: On Tue, Jul 24, 2007 at 10:10:45PM +0530, Mridul Muralidharan wrote: Robin Redeker wrote: On Sat, Jul 21, 2007 at 08:17:19PM -0600, Peter Saint-Andre wrote: Robin Redeker wrote: On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote: I think the whole XEP should be renamed to something like: XEP-0106 - JID Mapping for Gateways This would be better. But it breaks the generic usage of JIDs for both users and gateways. It will create a lot of trouble. The XEP seems to already create a lot of trouble. Just remind me to register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-) No problem. The spec says: The character sequence \20 MUST NOT be the first or last character of an escaped node identifier. But of course you can violate the spec if desired. ;-) I don't violate the RFC here. I violate some optional extension. You cant specify the characters mentioned in the xep in the node without escaping them - so trying to do so otherwise is a violation of the rfc. Ofcourse you can come up with a custom encoding (as we used to have), but this does not allow any arbitrary client/server combination to interoperate. Of course I can? With an old client? And it's even allowed and sane and valid? Or do I miss something here? You cannot use prohibited characters - whether they are old clients or bad clients. The first conforming xmpp recipient (typically the server) would kick your stream out with a stream error. But I am right that '[EMAIL PROTECTED]' is a completly valid JID. And that every server should accept it? The XEP-0106 has to exclude the JIDs which start or end with '\20' in the nodepart from the escaping AND unescaping transformations. This is already present. Great, JIDs with '\20' in the beginning and end have been deprecated then? Shouldn't the RFC be changed then? No, this is not related to the rfc. The rfc specifies list of prohibited characters - and they MUST NOT be in the node. The xep allows a way of encoding these characters into the JID because of requirements like what I mentioned above. Those two special cases you described there are not really pointed out in the XEP. The XEP makes it sounds as if the escaping is for the usual IM user that wants a ' in his nodepart of the JID. XEP-0106: JID Escaping This document specifies a mechanism that enables the display of Jabber Identifiers (JIDs) with characters disallowed by the Nodeprep profile of stringprep. The XEP should be renamed to not make it sound like we want enable our regular IM user to expand the allowed characterset of his name in the nodepart of the JID. At the moment the paragraph says that it MUST NOT be first or last in the node part, but it doesn't say WHAT to do when this perfectly fine JID arrives from the line. Should the JID not be unescaped at all? Should only the parts after and before '\20' be unescaped? Should the client close the connection? It depends on who is doing what. If the recipient is expected to 'parse
Re: [Standards] JID Escaping
On Tue, Jul 24, 2007 at 10:10:45PM +0530, Mridul Muralidharan wrote: Robin Redeker wrote: On Sat, Jul 21, 2007 at 08:17:19PM -0600, Peter Saint-Andre wrote: Robin Redeker wrote: On Sat, Jul 21, 2007 at 09:20:27AM +0200, Mats Bengtsson wrote: I think the whole XEP should be renamed to something like: XEP-0106 - JID Mapping for Gateways This would be better. But it breaks the generic usage of JIDs for both users and gateways. It will create a lot of trouble. The XEP seems to already create a lot of trouble. Just remind me to register '[EMAIL PROTECTED]' when every client unescapes JIDs ;-) No problem. The spec says: The character sequence \20 MUST NOT be the first or last character of an escaped node identifier. But of course you can violate the spec if desired. ;-) I don't violate the RFC here. I violate some optional extension. You cant specify the characters mentioned in the xep in the node without escaping them - so trying to do so otherwise is a violation of the rfc. Ofcourse you can come up with a custom encoding (as we used to have), but this does not allow any arbitrary client/server combination to interoperate. Of course I can? With an old client? And it's even allowed and sane and valid? Or do I miss something here? The XEP-0106 has to exclude the JIDs which start or end with '\20' in the nodepart from the escaping AND unescaping transformations. This is already present. Great, JIDs with '\20' in the beginning and end have been deprecated then? Shouldn't the RFC be changed then? At the moment the paragraph says that it MUST NOT be first or last in the node part, but it doesn't say WHAT to do when this perfectly fine JID arrives from the line. Should the JID not be unescaped at all? Should only the parts after and before '\20' be unescaped? Should the client close the connection? It depends on who is doing what. If the recipient is expected to 'parse' the node, then it would return an error, else it would pass it on (directed packets through server for example). To who will it return an error? Will it throw a pop-up at the user Someone with an invalid JID sent you something!? Or back to the sender? Why should it do that for a perfectly fine JID? Do I miss something in the XEP? (If I do so please ignore the rest of the mail.) Please also note the nice, but maybe not so important collision that here happens when the client just doesn't unescape: unescape (\5c20foobar\5c20) = \20foobar\20 unescape (\20foobar\20) = \20foobar\20 This is of course not really an important JID, and who cares about a few optical collisions in clients which confuse the user. And these only happens once someone else decides to put '\20' at the beginning or end of his name and why would someone do that? Hey, we could add security notes to all clients which tell the user: Never attach '\20' to the beginning or end of your name, it is unsafe! The U.S. Army will love this! (One might think of a case where they actually name their units by enumerating them with a \ in the end: Unescaped: Escaped: Unescaped: Tank\1 Tank\5c1 Tank\1 Tank\20 Tank\20Tank\20 Tank\22 Tank\5c22 Tank\22 Tank\5c20 Tank\20 ... ps Ah... never... why would they do that... :-) These sort of problems are common to any form of encoding - example urlencoding. It is obviously expected that the client/gateway would do the right thing. Yea, usually everyone does unescape and compare the unescaped stuff, then no collisions happen (and urlencoding does not have this problem because it doesn't compare escaped URLs afaik). The problem here is that the server treats Tank\20 and Tank\5c20 as different nodes, and thats of course the completly right thing to do, because they are different. If you meant before that Tank\20 should be rejected by the client, then I'm still wondering why something should reject a perfectly fine JID? Should I send a disco to the sending client and look whether it knows JID escaping and _after_ I know that perform the JID unescaping? And if that client sends me a JID with '\20' in the beginning or end should I send him a error back or blame my user that he has bad people speaking to him? Does that mean that people with a JID like [EMAIL PROTECTED] won't be able to send messages from their JID escaping enabled clients? Do we want to deprecate JIDs from the RFC which start and end with '\20'? I propose to rename the XEP to make clear that this escaping/unescaping should only happen in very rare cases (only at gateways or heavily specialized client frontends). And that the terms 'escaping' and 'unescaping' are replaced by 'mapping' and 'unmapping', because thats what is happening here. Not really very specialized clients - even common clients will need it. Example
Re: [Standards] Handling for characters that have entities, but XML does not require them to be escaped
On Sun, Jul 22, 2007 at 04:30:13PM +0200, Matthias Wimmer wrote: Hi! There are several characters, that have predefined entities in XML, but that do not need to be escaped in XML. Examples for such characters are ' and in text nodes. [...] So I have two questions regarding this: Why at all do these characters have to be escaped? I guess because many people did implement their own broken XML parsers in the past and many couldn't handle real XML, so they enforced escaping that character for the backward compatibility. (just a guess) I it really necessary, that RFC 3920bis mandates a server to reject such XMPP streams? I very much dislike this requirement, as it would require me to implement my own XML parser, as I don't know any parser I could use, that could be configured to notice me that these characters have been received unescaped. If you use expat you could get the original string from a text node and look for a '' in that string. But this is an ugly hack that I also consider unneccessary. The RFC should be fixed and software that doesn't parse unescaped in text nodes should be fixed (noone is forced in todays world to write his own XML parser, libxml2 (afaik) and expat (for sure) can be convinced to handle partial transferred XML documents these days). Robin
Re: [Standards] Handling for characters that have entities, but XML does not require them to be escaped
On Sun, Jul 22, 2007 at 11:25:23PM +0200, Matthias Wimmer wrote: Hi Robin! Robin Redeker schrieb: Why at all do these characters have to be escaped? I guess because many people did implement their own broken XML parsers in the past and many couldn't handle real XML, so they enforced escaping that character for the backward compatibility. (just a guess) I can't beleave that there are any such problems. There is already software producing XML, that is valid but not escaping all possible characters. Examples for this are jabberd2 (but to a very new SVN version), jadc2s (up to today), Psi (still not escaping and ' in text nodes). Hm, ok, then I also have absolutely no idea why these chars have to be escaped. (And I'm also very curious!) So there is out many software, that worked for years now, but introducing this unneccessary restricting in RFC 3920 made them broken. If you use expat you could get the original string from a text node and look for a '' in that string. But this is an ugly hack that I also consider unneccessary. How do I do this with expat? I have never seen something like this. At least normally expat is a SAX parser, that you set an CharacterDataHandler. And the function you register as the CharacterDataHandler gets passed unescaped UTF-8 data. Within the CharacterDataHandler I see now way to determine if a has been transfered as or as gt;. The function I mean is XML_GetInputContext, the Perl API uses that to get me the portion of the XML document that was parsed. If you got that you might be able to find out whether unescaped is in that. (see eg. http://www.math.ucla.edu/computing/docindex/expat-html-2/reference.html#XML_GetInputContext ) You could also use XML_SetDefaultHandler and XML_DefaultCurrent to get nearly the same data without the limit of XML_CONTEXT_BYTES (I actually don't know very precise what the difference is, I'm not that intimate with the C API of expat). These are all of course very weird ways to get to the original XML character data of a recognized element, and I would love if XMPP wouldn't require me to even have need for such access to the my XML parser. All I want is a DOM tree and all I want to care about when writing XMPP out is passing my DOM tree to a XML generator without convincing the XML generator that _I_ know better than it how XML should be generated and look like so that a XMPP server doesn't get confused. (Lets not start with namespace madness here, we don't want to open the can of worms which is still left over from the 'About stream namespaces' discussion from 4 months ago. Hmm... IMO the issue should be brought up regularly even though that discussions about it end in: We know, but we won't fix it, because the protocol should stay broken and underspecified because if we fix it we break the implementations.). The RFC should be fixed and software that doesn't parse unescaped in text nodes should be fixed (noone is forced in todays world to write his own XML parser, libxml2 (afaik) and expat (for sure) can be convinced to handle partial transferred XML documents these days). Yes ... I'd also say that because of reusing standards and implementations of them, we should not force software to not accept unescaped entities. We should even encourage software to accept these unescaped entities. I agree, we should encourage being conform to the W3C recommendation. Robin
Re: [Standards] securing in-band registration
On Wed, Jul 18, 2007 at 04:34:31PM -0600, Peter Saint-Andre wrote: Part of the solution is requiring x:data forms for registration. Yes, as Matthias pointed out this will make life difficult for existing clients. So we need to define a transition strategy. Clearly define how the x:data-only registration works and set some goals for deprecating the old way of doing things. I thought those are already part of XEP-0077, is that another feature that noone implements these days in their clients? Or why do they make the life harder for clients? Robin
Re: [Standards] XEP-0045: roomnick case
On Thu, Jul 19, 2007 at 03:19:09PM -0600, Peter Saint-Andre wrote: Chris Mullins wrote: I think the IDEAL option is MEP: If we were redesigning things from scratch, probably. But we're not redesigning things from scratch. :) What are we're doing then? Patching/Hacking up nearly 10 year old standards so that they do something similiar to what we want? (I don't neccessarly vote for Nicks over Pubsub here, even though it's a nice application of Pubsub once again, I think it should at least be tried in a sample implementation. Unfortunately that would be a lot of work for nothing.) Breaking changes are bad. How can we meet people's needs without breaking things? By breaking the people. Robin
Re: [Standards] how to treat invalid XMPP?
On Mon, Jul 16, 2007 at 07:31:01PM +0200, Jakob Schroeter wrote: Hi, Apparantly there is a number of software packages that generates invalid XMPP. I've seen at least unescaped ' and in attribute values and character data, respectively. http://www.xmpp.org/rfcs/rfc3920.html#xml states that an XMPP implementation must not generate such unescaped characters, and when it receives such restricted XML data, it MUST ignore the data. You have to distinguish 'bad XML' and 'restricted XML' here. Unescaped or bad XML should lead to a disconnect. If you receive parts of the 'restricted XML', that means XML that is: * comments (as defined in Section 2.5 of [XML] (Bray, T., Paoli, J., Sperberg-McQueen, C., and E. Maler, “Extensible Markup Language (XML) 1.0 (2nd ed),” October 2000.)) * processing instructions (Section 2.6 therein) * internal or external DTD subsets (Section 2.8 therein) * internal or external entity references (Section 4.2 therein) with the exception of predefined entities (Section 4.6 therein) * character data or attribute values containing unescaped characters that map to the predefined entities (Section 4.6 therein); such characters MUST be escaped So far I just throw a parse error and disconnect the stream (when I implemented that, I never thought this would actually happen), but people complain about that. Also, that makes the receiving client look bad. If you see a component or server that sends bad XML you should file a bugreport against them. Or inform the administrator to upgrade. Of course it looks bad to the user if the client bails out with Got bad XML, disconnecting you from your precious contacts and ruining your IM flirt with that nice girl. But there is no other choice, or your client even displays corrupted data to the user, which would be even worse. Robin
Re: [Standards] XEP-0115 is harmful and should be deferred
Hi! On Thu, Jul 05, 2007 at 11:34:10PM +0200, Tomasz Sterna wrote: Dnia 04-07-2007, śro o godzinie 21:03 -0600, Peter Saint-Andre napisał(a): base64(sha1(dave-formatted id/features)) Seems reasonable to me. I've picked this random post to reply but it does not concern this particular post but the whole thread... ...which I did not follow really, because I find this whole XEP and concept of entity capabilities distributed with presence packed unneeded and harmful. This XEP came out to solve a problem of jabber:iq:version storming on the Psi client launch, which other clients blindly copied just to show-off the remote client version at the fancy tooltip. So instead of pull-based mechanizm there was a push-based mechanizm deployed. But I do see some inconsistency here. We don't allow vCard hashes to be pushed with the presence. We do not allow moods and now-playing to be pushed on us with the presence packet, but we gladly allow unrequested capabilities to be pushed with the presence?? More then, we're going to REQUIRE them? w.r.t. to vcard, this is just a small snipped from a recent debug log: 2007-06-30 13:12:30 +0200 | presence from= to= 2007-06-30 13:12:30 +0200 | statusi'm here/status 2007-06-30 13:12:30 +0200 | priority1/priority 2007-06-30 13:12:30 +0200 | c node=http://gaim.sf.net/caps; ver=2.0.0beta5 xmlns=http://jabber.org/protocol/caps/ 2007-06-30 13:12:30 +0200 | x xmlns=vcard-temp:x:update/ 2007-06-30 13:12:30 +0200 | /presence Quite some noise I get there with the presence, I completly agree with you on the following: Excuse me. I've subscribed your PRESENCE information. I didn't ask for your mood, tune, avatar nor capabilities. If I would need them, I will ask and you may allow me to have them. There's nothing special about the caps, that these would require special privileges. I think this is a good point and well spotted :-) ! And what's more - we invented a way for me to subscribe for all this kinds of additional information. Using everyone's favorite PubSub. Why don't we reuse it somehow? We didn't have PEP/PubSub deployed at all when we invented XEP-0115, but we do have now and for sure can do better now. I, as client developer, would really like if PEP/PubSub became more widely-used. Recently when reading this list I became the feeling that PEP is being avoided. Eg. by the common ironic joke: 'lets use PEP for this'. PEP/PubSub has many good applications IMO. And if you ask me, PEP is IMO essential enough to go into XEP-0073 (Basic IM Protocol Suite). (Well, it will have to go in there anyway if it becomes neccessary to promote client capabilities.) And I think announcing capabilities seems to be a great application of PEP/PubSub. I can already imagine the client setting: [X] Allow others to subscribe to your client's capabilities. or: [X] Don't publish this client's capabilities. Just my 1.999... Cents Greetings, Robin
