Re: [Standards] Deprecating XEP-0138: Stream Compression
On Mon, Aug 29, 2016 at 09:29:23AM +0200, Florian Schmaus wrote: > On 29.08.2016 05:29, Sam Whited wrote: > > On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet > > wrote: > >> Two years late, but can we deprecate it XEP-0138 now, lest someones > >> comes along and implements/enables it in their client? > > > > Though I'm aware of the security risks, stream compression is still > > useful, and may even be necessary in some deployments. Maybe it would > > be better to just expand the security section to explain when stream > > compression might be a risk instead of deprecating the entire (still > > useful) XEP? > > Exactly. I don't think that XEP-0138 as whole should be deprecated. Not > every compression mechanism may be vulnerable to the class of attacks. > Even zlib can very likely be made secure by using "full flush". I also > think that the worsened compression ratio by doing so, can be cushioned > by only performing a full flush, i.e. dropping the > dictionary/compression state, if the receiving entity changes. > > […] > > Of course, both entities of an XMPP connection would need to perform > this on their outgoing stream. > > I don't think that we will reach consensus on deprecating XEP-0138. But > I think we all agree that the XEP should discuss the known security > issues in the "Security Considerations" section. So instead of focusing > on deprecating the XEP, I suggest we first add this information to the XEP. > > - Florian > I was reminded about the XEP while browsing PIFT [1], and while wondering if I should implement/enable it, I had a faint memory about security issues concerning compression. So, while searching my inbox for clues, I found this thread which didn’t evolve since 2014. I would be happy with a substantial update to the security considerations as well. Actually, if 0138 implementations were limited to the platforms described in the XEP introduction, it wouldn’t be an issue at all, which is why I agree that deprecacting the XEP might be overkill. [1] https://www.zash.se/xmpp-clients.html signature.asc Description: PGP signature ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Deprecating XEP-0138: Stream Compression
On 29 Aug 2016 04:30, "Sam Whited" wrote: > > On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet wrote: > > Two years late, but can we deprecate it XEP-0138 now, lest someones > > comes along and implements/enables it in their client? > > Though I'm aware of the security risks, stream compression is still > useful, and may even be necessary in some deployments. Maybe it would > be better to just expand the security section to explain when stream > compression might be a risk instead of deprecating the entire (still > useful) XEP? > Very much agree. Where radio is used, for example, encryption can be applied at the link layer, fully padded, which defeats all such attacks. In other cases, the general approach of stimulating known traffic from a particular entity can be used to gain information even in the absence of compression. As examples, you can determine if a particular client is in a chatroom by sending traffic to that chatroom, for example. My understanding of the attacks is that they leak metadata, in XMPP, rather than credentials as in HTTP - and they still require the ability to both stimulate traffic and observe the full packet flow. > —Sam > > > -- > Sam Whited > pub 4096R/54083AE104EA7AD3 > ___ > Standards mailing list > Info: https://mail.jabber.org/mailman/listinfo/standards > Unsubscribe: standards-unsubscr...@xmpp.org > ___ ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Deprecating XEP-0138: Stream Compression
On 29.08.2016 05:29, Sam Whited wrote: > On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet > wrote: >> Two years late, but can we deprecate it XEP-0138 now, lest someones >> comes along and implements/enables it in their client? > > Though I'm aware of the security risks, stream compression is still > useful, and may even be necessary in some deployments. Maybe it would > be better to just expand the security section to explain when stream > compression might be a risk instead of deprecating the entire (still > useful) XEP? Exactly. I don't think that XEP-0138 as whole should be deprecated. Not every compression mechanism may be vulnerable to the class of attacks. Even zlib can very likely be made secure by using "full flush". I also think that the worsened compression ratio by doing so, can be cushioned by only performing a full flush, i.e. dropping the dictionary/compression state, if the receiving entity changes. Pseudocode: --- So instead of List outgoingElements = ... for (StreamElement e : outgoingElements) { socket.write(e); // Drop compressor state. socket.fullFlush(); } we get List outgoingElements = ... Jid lastTo = null; for (StreamElement e : outgoingElements) { socket.write(e); if (lastTo != e.getTo()) { // Drop compressor state. socket.fullFlush(); } lastTo = e.getTo(); } Of course, both entities of an XMPP connection would need to perform this on their outgoing stream. I don't think that we will reach consensus on deprecating XEP-0138. But I think we all agree that the XEP should discuss the known security issues in the "Security Considerations" section. So instead of focusing on deprecating the XEP, I suggest we first add this information to the XEP. - Florian signature.asc Description: OpenPGP digital signature ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Deprecating XEP-0138: Stream Compression
On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet wrote: > Two years late, but can we deprecate it XEP-0138 now, lest someones > comes along and implements/enables it in their client? Though I'm aware of the security risks, stream compression is still useful, and may even be necessary in some deployments. Maybe it would be better to just expand the security section to explain when stream compression might be a risk instead of deprecating the entire (still useful) XEP? —Sam -- Sam Whited pub 4096R/54083AE104EA7AD3 ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Deprecating XEP-0138: Stream Compression
On Tue, Oct 14, 2014 at 02:48:42PM +0200, Thijs Alkemade wrote: > > On 9 okt. 2014, at 17:06, Peter Saint-Andre - &yet wrote: > > > On 10/9/14, 7:59 AM, Thijs Alkemade wrote: > >> Hello all, > >> > >> Stream compression is insecure, that was shown with CRIME and BREACH and > >> the > >> situation for XMPP isn't much different [1]. I think we should look at the > >> easiest way to deprecate XEP-0138 and move to something better. > >> > >> Using a "full flush" (in zlib terms) after every stanza would solve the > >> problem, as I can't find any realistic examples where an attacker could > >> insert > >> their own payload into the same stanza as something secret they want to > >> know. > >> However, clients and servers have no way to negotiate a mode like that, so > >> it's not possible to reject connections that won't do a per-stanza full > >> flush. > >> Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that > >> this > >> could be negotiated in WebSocket extension [2]. > >> > >> From my own (very small scale) tests with raw XMPP XML, it appears that > >> full > >> flushing after every stanza yields about the same compression ratio as > >> compressing each stanza separately. Doing that would have a number of > >> advantages: > >> > >> 1. Not relying on nothing leaking through the "full flush", which may be a > >> concept that other compression algorithms than zlib don't have or don't do > >> securely enough. > >> > >> 2. Practically no memory overhead in the server or client between messages. > >> There's no context to keep around, each new message can be decompressed > >> with a > >> fresh new context. Memory overhead for compression is a real concern for > >> servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS > >> compression was that it's impossible configure the memory use of TLS > >> compression to sane levels in OpenSSL. > >> > >> However, it also has downsides. It requires either: > >> > >> 1. That the concatenation of two compressed stanzas can be separated > >> unambiguously. > > > > Could you explain that a bit more? For example, are you talking about > > compressing two stanzas and sending them in the same TCP packet? > > Instead of sending: > > zlib(“...”) > > (Where you’d occasionally send the compressed data you have so far.) > > You'd send: > > zlib(“”) + zlib(“”) + zlib(“”) + zlib(“”) > > (Where + is concatenation.) > > This is easy in zlib because it’s possible to tell when a zlib stream ends > [1][2]. > > > > >> 2. Or that we apply framing outside of compression (which I expect to be > >> another can of worms). > > > > Yes, I'd expect so. I recall debates about framing (or the lack thereof) > > for XMPP on this very list from over 10 years ago. ;-) > > > > a> zlib has a header bit that indicates whether a block is the last block > > in a > >> stream, but again, that might be zlib-specific. > > > > Would it be worthwhile to investigate what the various compression > > algorithms support here? > > I've been trying to look into LZW, as it is described by XEP 0229, but while I > can find enough descriptions of the algorithm itself, I can't find much about > the output encoding. Most of the LZW API's I've seen also have no flush-method > or something similar. > > Regards, > Thijs > > [1] = http://zlib.net/manual.html: > > "If the flush parameter is Z_FINISH, the remaining data is written and the > gzip stream is completed in the output. If gzwrite() is called again, a new > gzip stream will be started in the output. gzread() is able to read such > concatented gzip streams." > > [2] = https://docs.python.org/2/library/zlib.html#zlib.Decompress.unused_data > Two years late, but can we deprecate it XEP-0138 now, lest someones comes along and implements/enables it in their client? Best regards, -- Mathieu Pasquet (mathieui), poezio developer signature.asc Description: PGP signature ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Deprecating XEP-0138: Stream Compression
On 9 okt. 2014, at 17:06, Peter Saint-Andre - &yet wrote: > On 10/9/14, 7:59 AM, Thijs Alkemade wrote: >> Hello all, >> >> Stream compression is insecure, that was shown with CRIME and BREACH and the >> situation for XMPP isn't much different [1]. I think we should look at the >> easiest way to deprecate XEP-0138 and move to something better. >> >> Using a "full flush" (in zlib terms) after every stanza would solve the >> problem, as I can't find any realistic examples where an attacker could >> insert >> their own payload into the same stanza as something secret they want to know. >> However, clients and servers have no way to negotiate a mode like that, so >> it's not possible to reject connections that won't do a per-stanza full >> flush. >> Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that >> this >> could be negotiated in WebSocket extension [2]. >> >> From my own (very small scale) tests with raw XMPP XML, it appears that full >> flushing after every stanza yields about the same compression ratio as >> compressing each stanza separately. Doing that would have a number of >> advantages: >> >> 1. Not relying on nothing leaking through the "full flush", which may be a >> concept that other compression algorithms than zlib don't have or don't do >> securely enough. >> >> 2. Practically no memory overhead in the server or client between messages. >> There's no context to keep around, each new message can be decompressed with >> a >> fresh new context. Memory overhead for compression is a real concern for >> servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS >> compression was that it's impossible configure the memory use of TLS >> compression to sane levels in OpenSSL. >> >> However, it also has downsides. It requires either: >> >> 1. That the concatenation of two compressed stanzas can be separated >> unambiguously. > > Could you explain that a bit more? For example, are you talking about > compressing two stanzas and sending them in the same TCP packet? Instead of sending: zlib(“...”) (Where you’d occasionally send the compressed data you have so far.) You'd send: zlib(“”) + zlib(“”) + zlib(“”) + zlib(“”) (Where + is concatenation.) This is easy in zlib because it’s possible to tell when a zlib stream ends [1][2]. > >> 2. Or that we apply framing outside of compression (which I expect to be >> another can of worms). > > Yes, I'd expect so. I recall debates about framing (or the lack thereof) for > XMPP on this very list from over 10 years ago. ;-) > > a> zlib has a header bit that indicates whether a block is the last block in a >> stream, but again, that might be zlib-specific. > > Would it be worthwhile to investigate what the various compression algorithms > support here? I've been trying to look into LZW, as it is described by XEP 0229, but while I can find enough descriptions of the algorithm itself, I can't find much about the output encoding. Most of the LZW API's I've seen also have no flush-method or something similar. Regards, Thijs [1] = http://zlib.net/manual.html: "If the flush parameter is Z_FINISH, the remaining data is written and the gzip stream is completed in the output. If gzwrite() is called again, a new gzip stream will be started in the output. gzread() is able to read such concatented gzip streams." [2] = https://docs.python.org/2/library/zlib.html#zlib.Decompress.unused_data signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Standards] Deprecating XEP-0138: Stream Compression
On 10/9/14, 7:59 AM, Thijs Alkemade wrote: Hello all, Stream compression is insecure, that was shown with CRIME and BREACH and the situation for XMPP isn't much different [1]. I think we should look at the easiest way to deprecate XEP-0138 and move to something better. Using a "full flush" (in zlib terms) after every stanza would solve the problem, as I can't find any realistic examples where an attacker could insert their own payload into the same stanza as something secret they want to know. However, clients and servers have no way to negotiate a mode like that, so it's not possible to reject connections that won't do a per-stanza full flush. Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that this could be negotiated in WebSocket extension [2]. From my own (very small scale) tests with raw XMPP XML, it appears that full flushing after every stanza yields about the same compression ratio as compressing each stanza separately. Doing that would have a number of advantages: 1. Not relying on nothing leaking through the "full flush", which may be a concept that other compression algorithms than zlib don't have or don't do securely enough. 2. Practically no memory overhead in the server or client between messages. There's no context to keep around, each new message can be decompressed with a fresh new context. Memory overhead for compression is a real concern for servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS compression was that it's impossible configure the memory use of TLS compression to sane levels in OpenSSL. However, it also has downsides. It requires either: 1. That the concatenation of two compressed stanzas can be separated unambiguously. Could you explain that a bit more? For example, are you talking about compressing two stanzas and sending them in the same TCP packet? 2. Or that we apply framing outside of compression (which I expect to be another can of worms). Yes, I'd expect so. I recall debates about framing (or the lack thereof) for XMPP on this very list from over 10 years ago. ;-) a> zlib has a header bit that indicates whether a block is the last block in a stream, but again, that might be zlib-specific. Would it be worthwhile to investigate what the various compression algorithms support here? Peter -- Peter Saint-Andre https://andyet.com/
[Standards] Deprecating XEP-0138: Stream Compression
Hello all, Stream compression is insecure, that was shown with CRIME and BREACH and the situation for XMPP isn't much different [1]. I think we should look at the easiest way to deprecate XEP-0138 and move to something better. Using a "full flush" (in zlib terms) after every stanza would solve the problem, as I can't find any realistic examples where an attacker could insert their own payload into the same stanza as something secret they want to know. However, clients and servers have no way to negotiate a mode like that, so it's not possible to reject connections that won't do a per-stanza full flush. Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that this could be negotiated in WebSocket extension [2]. From my own (very small scale) tests with raw XMPP XML, it appears that full flushing after every stanza yields about the same compression ratio as compressing each stanza separately. Doing that would have a number of advantages: 1. Not relying on nothing leaking through the "full flush", which may be a concept that other compression algorithms than zlib don't have or don't do securely enough. 2. Practically no memory overhead in the server or client between messages. There's no context to keep around, each new message can be decompressed with a fresh new context. Memory overhead for compression is a real concern for servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS compression was that it's impossible configure the memory use of TLS compression to sane levels in OpenSSL. However, it also has downsides. It requires either: 1. That the concatenation of two compressed stanzas can be separated unambiguously. 2. Or that we apply framing outside of compression (which I expect to be another can of worms). zlib has a header bit that indicates whether a block is the last block in a stream, but again, that might be zlib-specific. Thoughts? Comments? Best regards, Thijs Alkemade [1] https://blog.thijsalkema.de/blog/2014/08/07/https-attacks-and-xmpp-2-crime-and-breach/ [2] https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-18#section-8.1.1 signature.asc Description: Message signed with OpenPGP using GPGMail