Hello Matthew Thanks for the response, with regards to IBB in the HTTP over XMPP proposal:
>> Hello Matt (again) and Council members. >> <> Seems I sent the last mail (below) too quickly, and started to think a little, with regards to IBB. I have a set of questions/reflections: >> >> The IBB requires an open iq-stanza (with response and possible handshake) >> and a closing iq-stanza (with response) for each transfer. For streamBase64 >> I see no problem in converting to IBB for transport. I agree that IBB is >> better suited for this, as it is already defined. >> >> But for normal chunked transfer, this seems very impractical. This may >> result in hundreds of additional messages that needs to be sent for a simple >> API-transaction, invoking a set of calls, etc. Or a normal web page >> (containing links to much embedded content). >> >> Note that chunked transfer is often used when content size (Content-Length) >> is not known, and the server decides to start transmitting anyway to >> maintain buffers small. This is often the case as I mentioned for dynamic >> content, not necessarily large, for instance in results to API calls. To add >> such a large extent of messages for each response will decrease performance >> radically without much gain. >> >> I therefore wanted to ask if it is possible to leave the chunkedBase64 >> encoding as it is, and only convert streamBase64 to IBB? I believe the >> overhead of messages merits a separate handling in this case. >> >> The chunk message contains a last attribute also that is a convenient way to >> avoid the last close iq stanza in IBB. This saves a lot during small >> transfers. Consider a relatively small HTTP GET with a response of 3 chunks. >> Using IBB it would require at least 2+2+3+2=9 messages (http get+reponse+ibb >> open+ibb open response+3 chunks+close+close response), while as it is now, >> only 5 messages are required. >> >> The streamBase64 part is different however, as it is probable that very few >> (probably at most one) stream is active from a client at a given time. >> Having the additional open and close commands here gives no visible >> additional cost in terms of messages. >> >> What is the opinion of the council: >> >> Is the use of IBB so important in this case that a performance degradation >> is preferred in the chunked case? >> >> Or is it sufficient to use IBB only in the streamBase64 case, and leave the >> chunkedBase64 case as is? >> > One of the things IBB worked through is data loss. The current definition > just assumes the HTTP/XMPP "client" is there and ready to receive the > <message/> stanzas containing the data chunk. While it should be, networks > and software is not 100% reliable and that "client" could have lost their > connection just after sending the <iq type='set'/>. The HTTP/XMPP "server" > is not going to have any clue things aren't going where it expects unless > it's receiving presence for the HTTP/XMPP "client". > >You could build this into HTTP/XMPP, but at some point you will get to the >point where it duplicates much of IBB (which is awfully small to begin with) >... so why not use IBB in the first place? When it comes to transfer of HTTP messages two things are very important: 1) Order of packets 2) Speed RFC 6120 guarantees that the order of messages are delivered in the same order as sent, which takes care of (1). Regarding speed, I believe it's important for the extension to avoid unnecessary messages, especially messages that require feedback. So, in the examples given above (9 instead of 5 messages, including a waiting period for a server-client-server roundtrip), most chunked responses will actually contain very few chunks. To insert extra messages, including a handshake message for each HTTP call I believe is simply not a good idea. It will increase the amount of messages, and also delay the sending of the messages until an intermediate response is returned. Furthermore, it will increase the need to buffer responses in another buffer, since the web server will not wait when it responds. It's much more attractive to just be able to send a sequence of messages, and let network buffers/queues and the XMPP server forward them as they can. This is how HTTP servers do it. I believe adding an additional handshake mechanism here for every such call, will just make the protocol unnecessarily slow, and give the impression XMPP is unnecessarily slow, when it doesn't have to be. Furthermore, trying to prevent data loss due connection loss might even be contra productive in this case: The specification explicitly states that any chunked or streamed transfer should be aborted if connection is lost, or of the receiving end goes off-line. The reason being that first, this is the way HTTP normally works (i.e. transfer aborted if connection is lost), secondly because handling of such things as connection loss is typically already implemented in clients and servers, and a well-defined method of detecting a connection loss is required for compatibilities (i.e. connection loss = presence offline). However, I'm not against the idea if we find this to be an important in the experimental phase. If we find that such connection loss is a problem, I propose to add a handshake mechanism that can be done once, and not once per method call, this limiting the number of telegrams added in a session. For instance: What about an optional attribute in the original HTTP request, stating the maximum buffer size allowed by the client? That would eliminate the need for a handshake, since the server could use that value or a smaller value, is it sees fit, and simply send messages. (IBB messages can be used if desired, instead of the chunk message type.) Also, as I stated above, I have no problems already now redefining streaming, so it uses IBB instead of the chunked message transfer. In this case, the overhead of a few telegrams is negligible. Would that be OK with all of you? Sincerely, Peter Waher
