Re: [jdev] XMPP Ping method?
On Wednesday 01 November 2006 9:17 am, Peter Saint-Andre wrote: > Justin Karneges wrote: > > This seems unlikely > > though, as I think the RFC revision has already been made, and there was > > no consideration for these features. > > It is not accurate to say that "the RFC revision has already been made". > I published -00 versions of rfc3920bis and rfc3921bis, but that is the > start of discussion, not the end. Well let's test the waters. :) I've put together some RFC text that unifies the acking mechanisms proposed by myself and Dave Cridland. I'll post it to standards-jig. -Justin
Re: [jdev] XMPP Ping method?
On Thu Nov 2 23:24:47 2006, Jesus Cea wrote: I'm a bit worried about CPU/bandwidth explosión, nevertheless. And mobile bandwidth, that pay per byte. Also on mobile, the battery drain for transmission outweighs everything else. The battery drain for receiving data isn't small either. In practise, this means three things: 1) You only generally want to send data when you absolutely have to. 2) You only want to send extra data when you're sending some already. 3) You don't want to recive data you don't need, either. So for keeping a connection live, whitespace is good - it's a single octet, which translates as about 40 bytes or so including the TCP overhead. On most mobile networks, you'd be needing to send these every N minutes, where N is around 4. Every one of those bytes you send, you pay for. For ACKs and restarts, you either want to wrap a sequence number into the TLE, or you want to append a new TLE to the end of each TCP packet. The ACK that comes back shouldn't be sent immediately, either - it should be safe to hang onto until the server *needs* to send something, or - for client ACKs to the server - you're going to send anyway. So a rough logic is that if the connection has been quiet for 4 minutes, the client sends an ACK if one is outstanding, or else you send a whitespace character. Ideally, we'd have the server send the whitespace pings, because they cost the device less in terms of battery. I imagine we'd use a stream feature for this negotiation, in which case the ACKs themselves might as well be a new TLE. Incidentally, the power concerns are true to a reasonable degree for WLAN, too, so this isn't a purely mobile thing. Dave. -- Dave Cridland - mailto:[EMAIL PROTECTED] - xmpp:[EMAIL PROTECTED] - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Re: [jdev] XMPP Ping method?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Campbell wrote: > If you can be assured of whitespace pings between the client and the > server, the server can simply queue up all stanzas as received and > forwarded, and delete from the queue any stanzas sent between successful > receipt of whitespace pings. Without protocol changes, the odd > duplicated stanza is probably the best of a known evil. You can get a "whitespace", but you don't know if the other side received all your data yet. The idea somebody proposed of stanza sequence numbering could solve that. More CPU/bytes, nevertheless. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ [EMAIL PROTECTED] http://www.argo.es/~jcea/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/ _/_/_/_/_/ _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRUqBr5lgi5GaxT1NAQJbowQAgrkLSew0DVaNQwxF/YhXCN4Y+5biXaoR LgnYv5s8hwO+4wGqZdbCJsbKlRygXts0Fu3qidDE83rSnNhWapr6w4ps1/N/cf8X jS1YIUpblFXA14jhA1PEXc9h8lW8Qf2fdQstXdKRXR9NG8VRFkwQCq0LlqmheSoY RKUdB61cJx0= =hWB/ -END PGP SIGNATURE-
Re: [jdev] XMPP Ping method?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Mullins wrote: >>From the very first day I wrote a Jabber packet, I was looking for a > Ping command. I would still like to see one. +1 Coming from an IRC devel background, I also miss "ping". You can't use TCP timeouts because they are very very very late. The faster TCP timeout would be about 20 minutes. I'm a bit worried about CPU/bandwidth explosión, nevertheless. And mobile bandwidth, that pay per byte. > > I usually end up using IQ:Time, or IQ:Version as a ping. Almost > everything supports this - clients, servers, bots, etc. > > I would still love to see IQ:Ping. > > -- > Chris Mullins > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf > Of Peter Saint-Andre > Sent: Wednesday, November 01, 2006 8:45 AM > To: Jabber software development list > Subject: Re: [jdev] XMPP Ping method? > > Scott Robinson wrote: >> What is the proper method of performing a ping across a client XMPP >> connection. That is, from a sever's perspective, if a client >> mysteriously and unexpectedly drops off the Internet, it won't know it >> until the TCP connection times out. > > Most servers use "whitespace pings". It would be good for us to document > that method in an XMPP extension. > > Peter > - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ [EMAIL PROTECTED] http://www.argo.es/~jcea/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/ _/_/_/_/_/ _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRUp+P5lgi5GaxT1NAQK+dwP5ATBJ7I26V8sUAkn33UU+ar522QUmylTm L2s5mEgrlOUJUMwxT85KyDJtM2ACc5YWeTupe/oO8o5l7O8Wr2dTRCS2TVqasGRb RyyiD2arV+lFM/bnZtXbvHHnB0wnIuKXiN8dXhc0t6uvqKZkTt6I7KkHkDIhff4h hhOKrhnm5rQ= =g9tW -END PGP SIGNATURE-
Re: [jdev] XMPP Ping method?
Off-topic: I'm cross-posting this again, since this seems to be a debate about whether this is a protocol or implementation issue (It's a protocol issue by the way ;)) On-topic: On 2 Nov 2006, at 10:27, Bruce Campbell wrote: Hop-by-hop tests are quite easy, too, but there's a gotcha - when they fail, you want to know which stanzas you need to resend. And XMPP does not provide any mechanism for that, and nor do pings. If you can be assured of whitespace pings between the client and the server, the server can simply queue up all stanzas as received and forwarded, and delete from the queue any stanzas sent between successful receipt of whitespace pings. Without protocol changes, the odd duplicated stanza is probably the best of a known evil. We discussed this idea the last time the reliable delivery discussion came around. Unfortunately, you can't do that because you have no idea what stanzas the other end received before they sent the whitespace ping, only what stanzas you had sent. For what it's worth, I very much want to get some ack system into widespread use. For some people, with reliable server software, hardware, network infrastructure and clients, XMPP seems to be reliable already but it's really just good fortune. Once one of these starts becoming a problem, reliability goes out of the window. Since the networking team at the site my primary account is hosted have been having problems over the last couple of weeks, I've lost count of the number of people I've sent messages to only for them to never be received (and for me to not know this until I speak to the person out-of-band). As such, I'm really suffering from the current lack of reliable delivery. Per-hop acks would guarantee end-to-end delivery and would strengthen xmpp considerably. Details, such as a new TLE, or are worth discussing, but this is the moment for finally making xmpp reliable. /K -- Kevin Smith Psi XMPP Client Project Leader (http://psi-im.org)
Re: [jdev] XMPP Ping method?
On Wed, Nov 01, 2006 at 06:07:39PM +0100, Tobias Markmann wrote: > Isn't that a TCP problem since that can happen to any protocol which is > based to TCP? You should diffentiate between interactive and bulk protocols that use TCP. The latter tend towards getting as much data in the shortest amount of time (eg HTTP and FTP request -> response setup), whilst the former (telnet/ssh, xmpp etc) tend to small amounts of data being sent at arbitary intervals. On Wed, 1 Nov 2006, Dave Cridland wrote: Hop-by-hop tests are quite easy, too, but there's a gotcha - when they fail, you want to know which stanzas you need to resend. And XMPP does not provide any mechanism for that, and nor do pings. If you can be assured of whitespace pings between the client and the server, the server can simply queue up all stanzas as received and forwarded, and delete from the queue any stanzas sent between successful receipt of whitespace pings. Without protocol changes, the odd duplicated stanza is probably the best of a known evil. -- Bruce Campbell.
Re: [jdev] XMPP Ping method?
On Wed, Nov 01, 2006 at 08:42:08PM +, Dave Cridland wrote: > On Wed Nov 1 17:07:11 2006, Michal 'vorner' Vaner wrote: > >On Wed, Nov 01, 2006 at 06:07:39PM +0100, Tobias Markmann wrote: > >> Isn't that a TCP problem since that can happen to any protocol > >which is > >> based to TCP? > > > >Well, it is partly implementation problem, many OSes (as I heard) > >are > >able to tell you how much was already delivered and if you remember > >what > >part of data was what stanza, you can resend it after reconnection. > > > >But that is bit more work, of course, and alot more data. > > No OS can tell you what's been delivered, but some might be able to > tell you what hasn't been sent, and what hasn't been acknowledged. Yes, but from that, you can deduce what _was_ acknowledged. And if you loose TCP ack, TCP will take care of it. If you loose datal-level ack, TCP will take care of it the same, there is no difference. > I > looked for how to do this on Linux, which usually provides the > richest API to the network layer, but I couldn't find anything to > tell me either. I guess it is not much used and documented, I just heard it exists. > But this isn't quite the same thing anyway - you want to know what > stanzas have been accepted - what happens if the ACKs get lost, or > the server dies? If they get lost, they will arrive later. If the server dies, same is if it did not have enough time to send the data-level one, no? But anyway, I just said it can be done, but I also said it would be difficult and wouldn't be the best thing anyway, I think. > -- BOFH Excuse #452: Somebody ran the operating system through a spelling checker. Michal 'vorner' Vaner pgpMnJLtagDyt.pgp Description: PGP signature
Re: [jdev] XMPP Ping method?
On Wed Nov 1 17:07:11 2006, Michal 'vorner' Vaner wrote: On Wed, Nov 01, 2006 at 06:07:39PM +0100, Tobias Markmann wrote: > Isn't that a TCP problem since that can happen to any protocol which is > based to TCP? Well, it is partly implementation problem, many OSes (as I heard) are able to tell you how much was already delivered and if you remember what part of data was what stanza, you can resend it after reconnection. But that is bit more work, of course, and alot more data. No OS can tell you what's been delivered, but some might be able to tell you what hasn't been sent, and what hasn't been acknowledged. I looked for how to do this on Linux, which usually provides the richest API to the network layer, but I couldn't find anything to tell me either. But this isn't quite the same thing anyway - you want to know what stanzas have been accepted - what happens if the ACKs get lost, or the server dies? Consider ESMTP, which has got data level acknowledgement. There's a long-known problem whereby after DATA (and these days, BDAT and BURL), there's a chance that you'll lose the connection before you get the 2xx acknowledgement from the server. This is on the increase again, partly due to the preference for protocol-level rejections instead of DSNs, partly due to the marked increase in usage of ESMTP over things like GPRS. It's important to note that this specifically is about hop-by-hop, and not end-to-end, which are different problems entirely. Finding out if the guy you're talking to is still connected is quite easy, just send an IQ (in principle *any* IQ), and you'll see. Hop-by-hop tests are quite easy, too, but there's a gotcha - when they fail, you want to know which stanzas you need to resend. And XMPP does not provide any mechanism for that, and nor do pings. My last suggestion - adding a sequence attribute to stanzas - didn't seem to impress most people, partly because it requires servers to rewrite stanzas between hops. If instead the sender appends a distinct stanza (which could be an iq, or could be something else) to every TCP segment sent, which itself contains a sequence, then that can be used as the restart token with almost precisely the same effect, and requires no rewriting of stanzas. So, the sender appends, for instance, id='ping123'> to each send() call's payload, and the receiver can then note this simply, and respond with an iq reply when it suits it, which also contains sent and recv sequence counts. Loosely, you'd add that to the end of each TCP packet, in practise about every 1.5k or at the end of each send() should be quite safe. Dave. -- Dave Cridland - mailto:[EMAIL PROTECTED] - xmpp:[EMAIL PROTECTED] - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Re: [jdev] XMPP Ping method?
On Wednesday 01 November 2006 20:08, Tomasz Sterna wrote: > On 11/1/06, Scott Robinson <[EMAIL PROTECTED]> wrote: > > What is the proper method of performing a ping across a client XMPP > > connection. That is, from a sever's perspective, if a client > > mysteriously and unexpectedly drops off the Internet, it won't know it > > until the TCP connection times out. > > Isn't TCP enough to handle the situation? > Why do you need to implement it on application level? If 2 machines are connected over TCP/IP connection and there is one or more firewalls in between, try to power-off one of the machines. Another side will not notice connection drop. Default TCP/IP timeout is 300sec. This kind of "power-off" disconnections happen surprisingly often. And this is especially problem for network servers serving many simultaneous connections as such clients reconnect in a few seconds so server ends up with many dead connections... Artur -- Artur Hefczyc http://www.tigase.org/ http://wttools.sf.net/
Re: [jdev] XMPP Ping method?
On 11/1/06, Scott Robinson <[EMAIL PROTECTED]> wrote: What is the proper method of performing a ping across a client XMPP connection. That is, from a sever's perspective, if a client mysteriously and unexpectedly drops off the Internet, it won't know it until the TCP connection times out. Isn't TCP enough to handle the situation? Why do you need to implement it on application level? -- smk
RE: [jdev] XMPP Ping method?
That's quite a bit more complicated that I envisioned. I was just thinking "IQ:Ping", and leave it there. For all of the use cases I've come across, this would be sufficient. -- Chris Mullins -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Karneges Sent: Wednesday, November 01, 2006 9:27 AM To: Jabber software development list Subject: Re: [jdev] XMPP Ping method? On Wednesday 01 November 2006 9:17 am, Peter Saint-Andre wrote: > But the latest discussions about this (a month or two ago?) were that > this is most appropriate as an XMPP extension (XEP), so please send me > the latest version and we'll get that on the docket for the next XMPP > Council meeting. http://www.xmpp.org/extensions/inbox/ack.html -Justin
RE: [jdev] XMPP Ping method?
>From the very first day I wrote a Jabber packet, I was looking for a Ping command. I would still like to see one. I usually end up using IQ:Time, or IQ:Version as a ping. Almost everything supports this - clients, servers, bots, etc. I would still love to see IQ:Ping. -- Chris Mullins -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Peter Saint-Andre Sent: Wednesday, November 01, 2006 8:45 AM To: Jabber software development list Subject: Re: [jdev] XMPP Ping method? Scott Robinson wrote: > What is the proper method of performing a ping across a client XMPP > connection. That is, from a sever's perspective, if a client > mysteriously and unexpectedly drops off the Internet, it won't know it > until the TCP connection times out. Most servers use "whitespace pings". It would be good for us to document that method in an XMPP extension. Peter -- Peter Saint-Andre Jabber Software Foundation http://www.jabber.org/people/stpeter.shtml
Re: [jdev] XMPP Ping method?
On Wednesday 01 November 2006 9:17 am, Peter Saint-Andre wrote: > But the latest discussions about this (a month or two ago?) were that > this is most appropriate as an XMPP extension (XEP), so please send me > the latest version and we'll get that on the docket for the next XMPP > Council meeting. http://www.xmpp.org/extensions/inbox/ack.html -Justin
Re: [jdev] XMPP Ping method?
Justin Karneges wrote: > On Tuesday 31 October 2006 4:49 pm, Scott Robinson wrote: >> What is the proper method of performing a ping across a client XMPP >> connection. > > There isn't. There was a JEP/XEP proposal to add Acking and Ping features to > the protocol, but it wasn't accepted as a XEP (that is, it died before it > even got a XEP number). The JSF Council recommendation was to take care of > these features in the XMPP-Core RFC instead of a XEP. But the latest discussions about this (a month or two ago?) were that this is most appropriate as an XMPP extension (XEP), so please send me the latest version and we'll get that on the docket for the next XMPP Council meeting. > This seems unlikely > though, as I think the RFC revision has already been made, and there was no > consideration for these features. It is not accurate to say that "the RFC revision has already been made". I published -00 versions of rfc3920bis and rfc3921bis, but that is the start of discussion, not the end. Peter -- Peter Saint-Andre Jabber Software Foundation http://www.jabber.org/people/stpeter.shtml smime.p7s Description: S/MIME Cryptographic Signature
Re: [jdev] XMPP Ping method?
On Wed, Nov 01, 2006 at 06:07:39PM +0100, Tobias Markmann wrote: > Isn't that a TCP problem since that can happen to any protocol which is > based to TCP? Well, it is partly implementation problem, many OSes (as I heard) are able to tell you how much was already delivered and if you remember what part of data was what stanza, you can resend it after reconnection. But that is bit more work, of course, and alot more data. -- This email has not been checked by an antivirus system. No virus found. Michal "vorner" Vaner pgp5kO5FcWw55.pgp Description: PGP signature
Re: [jdev] XMPP Ping method?
Isn't that a TCP problem since that can happen to any protocol which is based to TCP?On 11/1/06, Scott Robinson < [EMAIL PROTECTED]> wrote:What is the proper method of performing a ping across a client XMPP connection. That is, from a sever's perspective, if a clientmysteriously and unexpectedly drops off the Internet, it won't know ituntil the TCP connection times out.This obviously can lead to two inconvenient situations: 1. Dropped messages that could otherwise stored. 2. A stale presense of availablity.A quick Google search didn't result in anything useful, except a threadnoting how it _shouldn't_ be done.Any ideas? --Scott Robinson <[EMAIL PROTECTED]>http://quadhome.com/-BEGIN PGP SIGNATURE-Version: GnuPG v1.2.5 (GNU/Linux) iEYEARECAAYFAkVH7w8ACgkQ2wcaZqTSGsTUygCg1PrqPL5KBGe6kFEmAAL003yudjsAoKIAXlj00sRoFpI/WqIVUL9hpFOV=tucZ-END PGP SIGNATURE-
Re: [jdev] XMPP Ping method?
Cross-posting to sjig: On 1 Nov 2006, at 16:45, Peter Saint-Andre wrote: Scott Robinson wrote: What is the proper method of performing a ping across a client XMPP connection. That is, from a sever's perspective, if a client mysteriously and unexpectedly drops off the Internet, it won't know it until the TCP connection times out. Most servers use "whitespace pings". It would be good for us to document that method in an XMPP extension. Perhaps we could also revisit the ping/ack (J|X)EP, as the whitespace ping doesn't really address the issue as we might desire? /K -- Kevin Smith Psi XMPP Client Project Leader (http://psi-im.org)
Re: [jdev] XMPP Ping method?
Scott Robinson wrote: > What is the proper method of performing a ping across a client XMPP > connection. That is, from a sever's perspective, if a client > mysteriously and unexpectedly drops off the Internet, it won't know it > until the TCP connection times out. Most servers use "whitespace pings". It would be good for us to document that method in an XMPP extension. Peter -- Peter Saint-Andre Jabber Software Foundation http://www.jabber.org/people/stpeter.shtml smime.p7s Description: S/MIME Cryptographic Signature
Re: [jdev] XMPP Ping method?
On Tuesday 31 October 2006 4:49 pm, Scott Robinson wrote: > What is the proper method of performing a ping across a client XMPP > connection. There isn't. There was a JEP/XEP proposal to add Acking and Ping features to the protocol, but it wasn't accepted as a XEP (that is, it died before it even got a XEP number). The JSF Council recommendation was to take care of these features in the XMPP-Core RFC instead of a XEP. This seems unlikely though, as I think the RFC revision has already been made, and there was no consideration for these features. -Justin
