Re: [Standards] Deprecating XEP-0138: Stream Compression

[Mathieu Pasquet]

On Mon, Aug 29, 2016 at 09:29:23AM +0200, Florian Schmaus wrote:
> On 29.08.2016 05:29, Sam Whited wrote:
> > On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet  
> > wrote:
> >> Two years late, but can we deprecate it XEP-0138 now, lest someones
> >> comes along and implements/enables it in their client?
> > 
> > Though I'm aware of the security risks, stream compression is still
> > useful, and may even be necessary in some deployments. Maybe it would
> > be better to just expand the security section to explain when stream
> > compression might be a risk instead of deprecating the entire (still
> > useful) XEP?
> 
> Exactly. I don't think that XEP-0138 as whole should be deprecated. Not
> every compression mechanism may be vulnerable to the class of attacks.
> Even zlib can very likely be made secure by using "full flush". I also
> think that the worsened compression ratio by doing so, can be cushioned
> by only performing a full flush, i.e. dropping the
> dictionary/compression state, if the receiving entity changes.
> 
> […]
> 
> Of course, both entities of an XMPP connection would need to perform
> this on their outgoing stream.
> 
> I don't think that we will reach consensus on deprecating XEP-0138. But
> I think we all agree that the XEP should discuss the known security
> issues in the "Security Considerations" section. So instead of focusing
> on deprecating the XEP, I suggest we first add this information to the XEP.
> 
> - Florian
> 

I was reminded about the XEP while browsing PIFT [1], and while
wondering if I should implement/enable it, I had a faint memory about
security issues concerning compression. So, while searching my inbox for
clues, I found this thread which didn’t evolve since 2014.

I would be happy with a substantial update to the security
considerations as well. Actually, if 0138 implementations were limited
to the platforms described in the XEP introduction, it wouldn’t be an
issue at all, which is why I agree that deprecacting the XEP might be
overkill.


[1] https://www.zash.se/xmpp-clients.html


signature.asc
Description: PGP signature
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Deprecating XEP-0138: Stream Compression

[Florian Schmaus]

On 29.08.2016 05:29, Sam Whited wrote:
> On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet  
> wrote:
>> Two years late, but can we deprecate it XEP-0138 now, lest someones
>> comes along and implements/enables it in their client?
> 
> Though I'm aware of the security risks, stream compression is still
> useful, and may even be necessary in some deployments. Maybe it would
> be better to just expand the security section to explain when stream
> compression might be a risk instead of deprecating the entire (still
> useful) XEP?

Exactly. I don't think that XEP-0138 as whole should be deprecated. Not
every compression mechanism may be vulnerable to the class of attacks.
Even zlib can very likely be made secure by using "full flush". I also
think that the worsened compression ratio by doing so, can be cushioned
by only performing a full flush, i.e. dropping the
dictionary/compression state, if the receiving entity changes.

Pseudocode:
---
So instead of

List outgoingElements = ...

for (StreamElement e : outgoingElements) {
   socket.write(e);
   // Drop compressor state.
   socket.fullFlush();
}

we get

List outgoingElements = ...
Jid lastTo = null;
for (StreamElement e : outgoingElements) {
   socket.write(e);
   if (lastTo != e.getTo()) {
  // Drop compressor state.
  socket.fullFlush();
   }
   lastTo = e.getTo();
}

Of course, both entities of an XMPP connection would need to perform
this on their outgoing stream.

I don't think that we will reach consensus on deprecating XEP-0138. But
I think we all agree that the XEP should discuss the known security
issues in the "Security Considerations" section. So instead of focusing
on deprecating the XEP, I suggest we first add this information to the XEP.

- Florian



signature.asc
Description: OpenPGP digital signature
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Deprecating XEP-0138: Stream Compression

[Sam Whited]

On Sun, Aug 28, 2016 at 2:53 PM, Mathieu Pasquet  wrote:
> Two years late, but can we deprecate it XEP-0138 now, lest someones
> comes along and implements/enables it in their client?

Though I'm aware of the security risks, stream compression is still
useful, and may even be necessary in some deployments. Maybe it would
be better to just expand the security section to explain when stream
compression might be a risk instead of deprecating the entire (still
useful) XEP?

—Sam


-- 
Sam Whited
pub 4096R/54083AE104EA7AD3
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Deprecating XEP-0138: Stream Compression

[Mathieu Pasquet]

On Tue, Oct 14, 2014 at 02:48:42PM +0200, Thijs Alkemade wrote:
> 
> On 9 okt. 2014, at 17:06, Peter Saint-Andre -   wrote:
> 
> > On 10/9/14, 7:59 AM, Thijs Alkemade wrote:
> >> Hello all,
> >> 
> >> Stream compression is insecure, that was shown with CRIME and BREACH and 
> >> the
> >> situation for XMPP isn't much different [1]. I think we should look at the
> >> easiest way to deprecate XEP-0138 and move to something better.
> >> 
> >> Using a "full flush" (in zlib terms) after every stanza would solve the
> >> problem, as I can't find any realistic examples where an attacker could 
> >> insert
> >> their own payload into the same stanza as something secret they want to 
> >> know.
> >> However, clients and servers have no way to negotiate a mode like that, so
> >> it's not possible to reject connections that won't do a per-stanza full 
> >> flush.
> >> Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that 
> >> this
> >> could be negotiated in WebSocket extension [2].
> >> 
> >> From my own (very small scale) tests with raw XMPP XML, it appears that 
> >> full
> >> flushing after every stanza yields about the same compression ratio as
> >> compressing each stanza separately. Doing that would have a number of
> >> advantages:
> >> 
> >> 1. Not relying on nothing leaking through the "full flush", which may be a
> >> concept that other compression algorithms than zlib don't have or don't do
> >> securely enough.
> >> 
> >> 2. Practically no memory overhead in the server or client between messages.
> >> There's no context to keep around, each new message can be decompressed 
> >> with a
> >> fresh new context. Memory overhead for compression is a real concern for
> >> servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS
> >> compression was that it's impossible configure the memory use of TLS
> >> compression to sane levels in OpenSSL.
> >> 
> >> However, it also has downsides. It requires either:
> >> 
> >> 1. That the concatenation of two compressed stanzas can be separated
> >> unambiguously.
> > 
> > Could you explain that a bit more? For example, are you talking about 
> > compressing two stanzas and sending them in the same TCP packet?
> 
> Instead of sending:
> 
> zlib(“...”)
> 
> (Where you’d occasionally send the compressed data you have so far.)
> 
> You'd send:
> 
> zlib(“”) + zlib(“”) + zlib(“”) + zlib(“”)
> 
> (Where + is concatenation.)
> 
> This is easy in zlib because it’s possible to tell when a zlib stream ends 
> [1][2].
> 
> > 
> >> 2. Or that we apply framing outside of compression (which I expect to be
> >> another can of worms).
> > 
> > Yes, I'd expect so. I recall debates about framing (or the lack thereof) 
> > for XMPP on this very list from over 10 years ago. ;-)
> > 
> > a> zlib has a header bit that indicates whether a block is the last block 
> > in a
> >> stream, but again, that might be zlib-specific.
> > 
> > Would it be worthwhile to investigate what the various compression 
> > algorithms support here?
> 
> I've been trying to look into LZW, as it is described by XEP 0229, but while I
> can find enough descriptions of the algorithm itself, I can't find much about
> the output encoding. Most of the LZW API's I've seen also have no flush-method
> or something similar.
> 
> Regards,
> Thijs
> 
> [1] = http://zlib.net/manual.html:
> 
> "If the flush parameter is Z_FINISH, the remaining data is written and the
> gzip stream is completed in the output. If gzwrite() is called again, a new
> gzip stream will be started in the output. gzread() is able to read such
> concatented gzip streams."
> 
> [2] = https://docs.python.org/2/library/zlib.html#zlib.Decompress.unused_data
> 

Two years late, but can we deprecate it XEP-0138 now, lest someones
comes along and implements/enables it in their client?

Best regards,

-- 
Mathieu Pasquet (mathieui), poezio developer


signature.asc
Description: PGP signature
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Deprecating XEP-0138: Stream Compression

[Thijs Alkemade]

On 9 okt. 2014, at 17:06, Peter Saint-Andre - yet pe...@andyet.net wrote:

 On 10/9/14, 7:59 AM, Thijs Alkemade wrote:
 Hello all,
 
 Stream compression is insecure, that was shown with CRIME and BREACH and the
 situation for XMPP isn't much different [1]. I think we should look at the
 easiest way to deprecate XEP-0138 and move to something better.
 
 Using a full flush (in zlib terms) after every stanza would solve the
 problem, as I can't find any realistic examples where an attacker could 
 insert
 their own payload into the same stanza as something secret they want to know.
 However, clients and servers have no way to negotiate a mode like that, so
 it's not possible to reject connections that won't do a per-stanza full 
 flush.
 Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that 
 this
 could be negotiated in WebSocket extension [2].
 
 From my own (very small scale) tests with raw XMPP XML, it appears that full
 flushing after every stanza yields about the same compression ratio as
 compressing each stanza separately. Doing that would have a number of
 advantages:
 
 1. Not relying on nothing leaking through the full flush, which may be a
 concept that other compression algorithms than zlib don't have or don't do
 securely enough.
 
 2. Practically no memory overhead in the server or client between messages.
 There's no context to keep around, each new message can be decompressed with 
 a
 fresh new context. Memory overhead for compression is a real concern for
 servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS
 compression was that it's impossible configure the memory use of TLS
 compression to sane levels in OpenSSL.
 
 However, it also has downsides. It requires either:
 
 1. That the concatenation of two compressed stanzas can be separated
 unambiguously.
 
 Could you explain that a bit more? For example, are you talking about 
 compressing two stanzas and sending them in the same TCP packet?

Instead of sending:

zlib(“message/iq/message/iq...”)

(Where you’d occasionally send the compressed data you have so far.)

You'd send:

zlib(“message/”) + zlib(“iq/”) + zlib(“message/”) + zlib(“iq”)

(Where + is concatenation.)

This is easy in zlib because it’s possible to tell when a zlib stream ends 
[1][2].

 
 2. Or that we apply framing outside of compression (which I expect to be
 another can of worms).
 
 Yes, I'd expect so. I recall debates about framing (or the lack thereof) for 
 XMPP on this very list from over 10 years ago. ;-)
 
 a zlib has a header bit that indicates whether a block is the last block in a
 stream, but again, that might be zlib-specific.
 
 Would it be worthwhile to investigate what the various compression algorithms 
 support here?

I've been trying to look into LZW, as it is described by XEP 0229, but while I
can find enough descriptions of the algorithm itself, I can't find much about
the output encoding. Most of the LZW API's I've seen also have no flush-method
or something similar.

Regards,
Thijs

[1] = http://zlib.net/manual.html:

If the flush parameter is Z_FINISH, the remaining data is written and the
gzip stream is completed in the output. If gzwrite() is called again, a new
gzip stream will be started in the output. gzread() is able to read such
concatented gzip streams.

[2] = https://docs.python.org/2/library/zlib.html#zlib.Decompress.unused_data



signature.asc
Description: Message signed with OpenPGP using GPGMail

[Standards] Deprecating XEP-0138: Stream Compression

[Thijs Alkemade]

Hello all,

Stream compression is insecure, that was shown with CRIME and BREACH and the
situation for XMPP isn't much different [1]. I think we should look at the
easiest way to deprecate XEP-0138 and move to something better.

Using a full flush (in zlib terms) after every stanza would solve the
problem, as I can't find any realistic examples where an attacker could insert
their own payload into the same stanza as something secret they want to know.
However, clients and servers have no way to negotiate a mode like that, so
it's not possible to reject connections that won't do a per-stanza full flush.
Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that this
could be negotiated in WebSocket extension [2].

From my own (very small scale) tests with raw XMPP XML, it appears that full
flushing after every stanza yields about the same compression ratio as
compressing each stanza separately. Doing that would have a number of
advantages:

1. Not relying on nothing leaking through the full flush, which may be a
concept that other compression algorithms than zlib don't have or don't do
securely enough.

2. Practically no memory overhead in the server or client between messages.
There's no context to keep around, each new message can be decompressed with a
fresh new context. Memory overhead for compression is a real concern for
servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS
compression was that it's impossible configure the memory use of TLS
compression to sane levels in OpenSSL.

However, it also has downsides. It requires either:

1. That the concatenation of two compressed stanzas can be separated
unambiguously.

2. Or that we apply framing outside of compression (which I expect to be
another can of worms).

zlib has a header bit that indicates whether a block is the last block in a
stream, but again, that might be zlib-specific.

Thoughts? Comments?

Best regards,
Thijs Alkemade

[1] 
https://blog.thijsalkema.de/blog/2014/08/07/https-attacks-and-xmpp-2-crime-and-breach/
[2] 
https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-18#section-8.1.1


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [Standards] Deprecating XEP-0138: Stream Compression

[Peter Saint-Andre - yet]

On 10/9/14, 7:59 AM, Thijs Alkemade wrote:

Hello all,

Stream compression is insecure, that was shown with CRIME and BREACH and the
situation for XMPP isn't much different [1]. I think we should look at the
easiest way to deprecate XEP-0138 and move to something better.

Using a full flush (in zlib terms) after every stanza would solve the
problem, as I can't find any realistic examples where an attacker could insert
their own payload into the same stanza as something secret they want to know.
However, clients and servers have no way to negotiate a mode like that, so
it's not possible to reject connections that won't do a per-stanza full flush.
Reading draft-ietf-hybi-permessage-compression-18, I was happy to see that this
could be negotiated in WebSocket extension [2].

 From my own (very small scale) tests with raw XMPP XML, it appears that full
flushing after every stanza yields about the same compression ratio as
compressing each stanza separately. Doing that would have a number of
advantages:

1. Not relying on nothing leaking through the full flush, which may be a
concept that other compression algorithms than zlib don't have or don't do
securely enough.

2. Practically no memory overhead in the server or client between messages.
There's no context to keep around, each new message can be decompressed with a
fresh new context. Memory overhead for compression is a real concern for
servers: one of the reasons Prosody was pushing for XEP-0138 to replace TLS
compression was that it's impossible configure the memory use of TLS
compression to sane levels in OpenSSL.

However, it also has downsides. It requires either:

1. That the concatenation of two compressed stanzas can be separated
unambiguously.


Could you explain that a bit more? For example, are you talking about 
compressing two stanzas and sending them in the same TCP packet?



2. Or that we apply framing outside of compression (which I expect to be
another can of worms).


Yes, I'd expect so. I recall debates about framing (or the lack thereof) 
for XMPP on this very list from over 10 years ago. ;-)


a zlib has a header bit that indicates whether a block is the last 
block in a

stream, but again, that might be zlib-specific.


Would it be worthwhile to investigate what the various compression 
algorithms support here?


Peter

--
Peter Saint-Andre
https://andyet.com/