Re: [Standards] stanza order and MUC
On 9/30/10 12:26 PM, Justin Karneges wrote: > On Thursday 30 September 2010 02:13:22 Dave Cridland wrote: >> On Thu Sep 30 00:12:59 2010, bschumac wrote: >>>Users a...@s/C and b...@s/C are both online and both users begin a >>>conversation with each other at the same time. The server >>> >>> receives >>> >>>messages in this order: >>> 1. a...@s/C sends "Yo" to b...@s (Bare JID) >>> 2. b...@s/C sends "How's it going?" to a...@s >>> 3. a...@s/C sends second "Terrible." to b...@s/C (Full JID -- >>> >>> resource >>> >>> locked) >>> >>>However server provides incorrect guarantee that only ensures >>>in-order between "fullest-possible" JID, the messages end up >>> >>>delivered out-of-order: >>>* b...@s/C receives "Terrible." >>>* b...@s/C receives "Yo." >>>* a...@s/C receives "How's it going?" >> >> I'm in awe of your programming skills. The majority of clients have >> to wait until they get the response before they resource-lock, and >> moreover, quite a few users tend to wait until they see a question >> before they answer it, too. Not me, of course - I wrote this message >> three weeks ago, and just didn't get around to sending it until now... > > At first I thought this example was sort of contrived (not that it matters to > me, contrived examples are the most fun when discussing protocol), but with > s2s negotiation lag you could replicate the above issue in real life. > Suppose > the outbound s2s channel of A's server takes longer (on the order of seconds) > to complete than the inbound channel from B's server. It is possible that A > is able to read B's message and reply to it, while A's initial message is > still queued waiting for the outbound s2s to be ready. This means there will > be two of A's messages queued to be sent (one to B's bare JID and the other > to > B's full JID), and the delivery order of these messages will be at the mercy > of A's server implementation. > >> I'll throw out an alternate ordering rule, and see how people react: >> >> Servers MUST ensure in-order processing of the stanzas received on a >> stream. Servers MUST resolve any ambiguity caused by processing >> requests (for example, those sent to the bare jid of a connected >> client); in the case where the result of a request could have an >> effect on subsequent stanzas, servers MUST suspend processing of >> stanzas until the request is completed. Servers MUST ensure that any >> stanzas forwarded from a stream to a particular remote domain are >> forwarded in order - in particular, this implies that for any given >> inbound stanza that is addressed to a remote domain, there SHOULD be >> at most one outbound stream, used consistently. > > Beautiful. :) I like this because it goes beyond merely delivery and also > addresses pipelining, to provide a complete "in-order processing" picture. > > In the context of the existing section 10.1, your suggestion is, > interestingly, neither Full JIDs pairs nor Bare JID pairs. It is basically > Bare JID destination (where it makes sense) and Stream source. Interesting. Unfortunately we're on the wrong list here. I'll start a thread on x...@ietf.org so we can bang this out. Peter -- Peter Saint-Andre https://stpeter.im/
Re: [Standards] stanza order and MUC
On Thursday 30 September 2010 02:13:22 Dave Cridland wrote: > On Thu Sep 30 00:12:59 2010, bschumac wrote: > >Users a...@s/C and b...@s/C are both online and both users begin a > >conversation with each other at the same time. The server > > > > receives > > > >messages in this order: > > 1. a...@s/C sends "Yo" to b...@s (Bare JID) > > 2. b...@s/C sends "How's it going?" to a...@s > > 3. a...@s/C sends second "Terrible." to b...@s/C (Full JID -- > > > > resource > > > > locked) > > > >However server provides incorrect guarantee that only ensures > >in-order between "fullest-possible" JID, the messages end up > > > >delivered out-of-order: > >* b...@s/C receives "Terrible." > >* b...@s/C receives "Yo." > >* a...@s/C receives "How's it going?" > > I'm in awe of your programming skills. The majority of clients have > to wait until they get the response before they resource-lock, and > moreover, quite a few users tend to wait until they see a question > before they answer it, too. Not me, of course - I wrote this message > three weeks ago, and just didn't get around to sending it until now... At first I thought this example was sort of contrived (not that it matters to me, contrived examples are the most fun when discussing protocol), but with s2s negotiation lag you could replicate the above issue in real life. Suppose the outbound s2s channel of A's server takes longer (on the order of seconds) to complete than the inbound channel from B's server. It is possible that A is able to read B's message and reply to it, while A's initial message is still queued waiting for the outbound s2s to be ready. This means there will be two of A's messages queued to be sent (one to B's bare JID and the other to B's full JID), and the delivery order of these messages will be at the mercy of A's server implementation. > I'll throw out an alternate ordering rule, and see how people react: > > Servers MUST ensure in-order processing of the stanzas received on a > stream. Servers MUST resolve any ambiguity caused by processing > requests (for example, those sent to the bare jid of a connected > client); in the case where the result of a request could have an > effect on subsequent stanzas, servers MUST suspend processing of > stanzas until the request is completed. Servers MUST ensure that any > stanzas forwarded from a stream to a particular remote domain are > forwarded in order - in particular, this implies that for any given > inbound stanza that is addressed to a remote domain, there SHOULD be > at most one outbound stream, used consistently. Beautiful. :) I like this because it goes beyond merely delivery and also addresses pipelining, to provide a complete "in-order processing" picture. In the context of the existing section 10.1, your suggestion is, interestingly, neither Full JIDs pairs nor Bare JID pairs. It is basically Bare JID destination (where it makes sense) and Stream source. -Justin
Re: [Standards] stanza order and MUC
On Thu Sep 30 14:59:02 2010, Ben Schumacher wrote: My basic point is that if you're using some sort of threading, queuing, sockets, quantum delivery in your server to make it "optimized, threaded, clustered, etc." and you choose which thread, queue, socket, particle based on whichever JID the stanza is address to, rather than a "sensible" approach that uses a bare JID (ignores the "resourcepart", if it exists), stanzas will get delivered in a different order than they were introduced into the system. It may not happen all the time, but it will eventually happen. If a user has two clients connected, there is no way you can make any assumptions on when two sequential stanzas sent to the users bare jid, but to different full jids, will be delivered. So yes, it will eventually happen - quite a lot. And I don't think we need to care. I'll throw out an alternate ordering rule, and see how people react: Servers MUST ensure in-order processing of the stanzas received on a stream. Servers MUST resolve any ambiguity caused by processing requests (for example, those sent to the bare jid of a connected client); in the case where the result of a request could have an effect on subsequent stanzas, servers MUST suspend processing of stanzas until the request is completed. Servers MUST ensure that any stanzas forwarded from a stream to a particular remote domain are forwarded in order - in particular, this implies that for any given inbound stanza that is addressed to a remote domain, there SHOULD be at most one outbound stream, used consistently. Seems okay to me with the caveat that a stream, when a user is involves, obviously requires a "resourcepart" and tying the wording too closely to the concept of a stream leaves cases open where I'm sending messages to another user which resource-locked and the users goes offline as I'm sending message (thus breaking the lock). Assuming that my messages are delivered within the context of my stream in-order and then handed off to some construct that represents the other user's stream, but the socket is close before they're delivered, what the expectation for the message with regards to offline storage? As the sender I'd expect the messages to end up in offline in the same order they were originally sent by me, but when the other stream disappears mid-conversation, what's our in-order semantics? I'm maybe a little too close to an implementation here, but I think it's less ambiguous to simply tie in-orderness to a (destination) bare JID rather than a stream. I think once you've "attempted delivery", you assume you've done your bit. If you don't, then I can concoct any number of cases where the in-order delivery rules will either break, or cause massive serialization. To pick on off the top of my head: u...@example.org has two 198-enabled clients, on resources 1 and 2. We send a stanza from u...@example.net to u...@example.org/1, and subsequently another one from u...@example.net to u...@example.org/2. After a period, u...@example.org/1 drops offline without acking the stanza. When we perform recovery on unacked stanzas - either because we don't support resume or because the resume timer expired - we'll redeliver the stanza to /2. The only way I see to avoid this is to - if 198 is supported - delay the delivery to /2 until /1 no longer has any unacked stanzas. I don't think this is a practical suggestion. Dave. -- Dave Cridland - mailto:d...@cridland.net - xmpp:d...@dave.cridland.net - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Re: [Standards] stanza order and MUC
On 9/30/10 3:13 AM, Dave Cridland wrote: On Thu Sep 30 00:12:59 2010, bschumac wrote: Users a...@s/C and b...@s/C are both online and both users begin a conversation with each other at the same time. The server receives messages in this order: 1. a...@s/C sends "Yo" to b...@s (Bare JID) 2. b...@s/C sends "How's it going?" to a...@s 3. a...@s/C sends second "Terrible." to b...@s/C (Full JID -- resource locked) However server provides incorrect guarantee that only ensures in-order between "fullest-possible" JID, the messages end up delivered out-of-order: * b...@s/C receives "Terrible." * b...@s/C receives "Yo." * a...@s/C receives "How's it going?" I'm in awe of your programming skills. The majority of clients have to wait until they get the response before they resource-lock, and moreover, quite a few users tend to wait until they see a question before they answer it, too. Not me, of course - I wrote this message three weeks ago, and just didn't get around to sending it until now... This is as it should be, though in this case I'm afraid you're simply failing to grok my example and I don't care enough to go through the trouble of making an ASCII sequence diagram. My basic point is that if you're using some sort of threading, queuing, sockets, quantum delivery in your server to make it "optimized, threaded, clustered, etc." and you choose which thread, queue, socket, particle based on whichever JID the stanza is address to, rather than a "sensible" approach that uses a bare JID (ignores the "resourcepart", if it exists), stanzas will get delivered in a different order than they were introduced into the system. It may not happen all the time, but it will eventually happen. I'll throw out an alternate ordering rule, and see how people react: Servers MUST ensure in-order processing of the stanzas received on a stream. Servers MUST resolve any ambiguity caused by processing requests (for example, those sent to the bare jid of a connected client); in the case where the result of a request could have an effect on subsequent stanzas, servers MUST suspend processing of stanzas until the request is completed. Servers MUST ensure that any stanzas forwarded from a stream to a particular remote domain are forwarded in order - in particular, this implies that for any given inbound stanza that is addressed to a remote domain, there SHOULD be at most one outbound stream, used consistently. Seems okay to me with the caveat that a stream, when a user is involves, obviously requires a "resourcepart" and tying the wording too closely to the concept of a stream leaves cases open where I'm sending messages to another user which resource-locked and the users goes offline as I'm sending message (thus breaking the lock). Assuming that my messages are delivered within the context of my stream in-order and then handed off to some construct that represents the other user's stream, but the socket is close before they're delivered, what the expectation for the message with regards to offline storage? As the sender I'd expect the messages to end up in offline in the same order they were originally sent by me, but when the other stream disappears mid-conversation, what's our in-order semantics? I'm maybe a little too close to an implementation here, but I think it's less ambiguous to simply tie in-orderness to a (destination) bare JID rather than a stream. Cheers, Ben
Re: [Standards] stanza order and MUC
On Thu Sep 30 00:12:59 2010, bschumac wrote: Users a...@s/C and b...@s/C are both online and both users begin a conversation with each other at the same time. The server receives messages in this order: 1. a...@s/C sends "Yo" to b...@s (Bare JID) 2. b...@s/C sends "How's it going?" to a...@s 3. a...@s/C sends second "Terrible." to b...@s/C (Full JID -- resource locked) However server provides incorrect guarantee that only ensures in-order between "fullest-possible" JID, the messages end up delivered out-of-order: * b...@s/C receives "Terrible." * b...@s/C receives "Yo." * a...@s/C receives "How's it going?" I'm in awe of your programming skills. The majority of clients have to wait until they get the response before they resource-lock, and moreover, quite a few users tend to wait until they see a question before they answer it, too. Not me, of course - I wrote this message three weeks ago, and just didn't get around to sending it until now... I'll throw out an alternate ordering rule, and see how people react: Servers MUST ensure in-order processing of the stanzas received on a stream. Servers MUST resolve any ambiguity caused by processing requests (for example, those sent to the bare jid of a connected client); in the case where the result of a request could have an effect on subsequent stanzas, servers MUST suspend processing of stanzas until the request is completed. Servers MUST ensure that any stanzas forwarded from a stream to a particular remote domain are forwarded in order - in particular, this implies that for any given inbound stanza that is addressed to a remote domain, there SHOULD be at most one outbound stream, used consistently. What this means in practical terms: 1) If you get a vCard request for a bare jid, and a message to that same bare jid, you needn't block the entire stream while you pootle off to your database (or, if you're as hip as us, your ACP-142 compliant X.500 directory) to fetch the info. Just kick of the vCard fetching, and forward the stanza to a suitable resource. If a sender cares about what the result of the vCard is prior to delivery of the message, they'd wait. And you certainly don't care - whatever their family name is, you're going to pick the same resource, and even if you waited, there's no certainty you'd manage to delivery the result before the message anyway. 2) If you get a XEP-0191 change request from a client, and, in the same TCP packet, a presence broadcast - hence both to the same bare jid - then absolutely you need to fully process the XEP-0191 request before the presence broadcast, because the order clearly matters. 3) Once you've forwarded a message, that counts as processing. So if two sequential messages require forwarding to two different streams, then you don't need to care about the order they're delivered. If two sequential messages require forwarding to the same stream, you forward them in turn (such that they'll arrive in-order). 3) If a MUC service is - sensibly - using a single stream to communicate with you, it all Just Works™. If it doesn't use a single stream, it becomes their fault and not yours. (But note that a MUC service could use a single stream per-participant 4) If you use multiple streams to communicate with an entity (ie, two S2S streams which you randomly use either of to send to a remote full jid), then everything will break, and this is entirely your fault. You can, however, consistently use one of any number of streams to a remote domain for stanzas from a particular stream. 5) Quantum clients capable of resource locking prior to locating a resource to lock will now work - this is an important demographic in some dimensions. Dave. -- Dave Cridland - mailto:d...@cridland.net - xmpp:d...@dave.cridland.net - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Re: [Standards] stanza order and MUC
> Right. Ordering is important enough that I think this issue deserves more > than one sentence in the RFC. > > It seems our choices are: > > 1) "Entity" is a Full JID. Resource locking (RFC 3921) and MUC (XEP-0045) > need repairs, or we just live with their problems and try not to repeat them. > > 2) "Entity" is a Bare JID. Everything works as Jeremy Himself intended. I think it needs a bit more explanation. I guess not everybody (myself for example) knows what Jeremy Himself intended. Specifically, the Bare JID may have a few connections to the server and there might be packets addresses to different resources. What the above sentence means in this context? > Can developers of various servers describe their current ordering policies? > It would be interesting to see just how the spec has been interpretted so far. I tried a few different approaches. It is quite easy to switch between them in the Tigase. In order processing per TCP/IP stream, per sender address, per received address (not bare JID though, I will try that next time I run tests). Even though it appears to be working correctly in the test system, nothing seems to guarantee 100% packet in order delivery in some border cases unless you switch to single threaded mode which is not feasible. Artur -- Artur Hefczyc http://www.tigase.org/ http://artur.hefczyc.net/
