Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Matthew Miller]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On Jun 12, 2012, at 13:57, Peter Saint-Andre wrote:

> On 6/12/12 1:44 PM, Justin Karneges wrote:
>> On Tuesday, June 12, 2012 12:10:30 PM Peter Saint-Andre wrote:
>>> On 5/24/12 8:44 AM, Peter Saint-Andre wrote:
 On 5/24/12 1:01 AM, Sergey Dobrov wrote:
> On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
>> Old thread alert!
>> 
>> 
>> Well, the spec says that the length restriction applies to the
>> base64-encoded data, so I think we meant that this does *not* apply to
>> the raw data. However, you seem to be saying that this might be
>> difficult to implement. Have other folks experienced this problem?
> 
> Have other folks experienced this problem implementing things right ;)
> 
> I know that some implementations (maybe all?) measure raw data and not
> base64.
 
 If that's what all code does now, then I think it's fine for us to
 update the spec so that it reflects reality.
>>> 
>>> Does anyone object to making this change in the spec?
>> 
>> I think measuring the raw (pre-base64-encoded) data is the most sensible, 
>> and 
>> likely what I had originally intended but poorly conveyed in the XEP. So 
>> this 
>> is fine by me, especially if implementations are working this way.
> 
> Super.
> 
> I propose the following change.
> 
> OLD
> The base64-encoded data to be sent, prior to any wrapping in the 
> element and IQ or message stanza, MUST NOT be larger than the
> 'block-size' determined in the bytestream negotiation.
> 
> NEW
> The data to be sent, prior to base64-encoding and prior to any wrapping
> in XML, MUST NOT be larger than the 'block-size' determined in the
> bytestream negotiation.
> 

WFM


- - m&m

Matthew A. Miller


-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJP159pAAoJEJq6Ou0cgrSPcNYH/0mZplvX4ra7LEVpsrLw0WTI
AS9P3xwqwFbVGvdfSIyGV4N9IZOQAn44/DanEK9kMbrxAaaDr83wGCCIpsnM1lgY
bILeet/v2yZh/vUezXJuCHISgPiJhozYBAraGJ2aPtRK4Pm2O5C+v0FUtv9eKs1K
eXdYoJgWQAdz5mV7tpVaFOTUvLKZhGvifcBPR5CXdjoIzdJ/NZLNjyG/jnn3s/jA
5G1n8vrW2i2B1bFl6MtiXAGXhdi9DhXEOklUYEYQKtmnXpKAFpE/vcjPDB+eDzUn
dyQJp43Ww24TjrJTaGi1madPh/xg1W54jDWbRZ7uarfZWAxOiozT57PeCfP2YIM=
=ir5O
-END PGP SIGNATURE-

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

On 6/12/12 1:44 PM, Justin Karneges wrote:
> On Tuesday, June 12, 2012 12:10:30 PM Peter Saint-Andre wrote:
>> On 5/24/12 8:44 AM, Peter Saint-Andre wrote:
>>> On 5/24/12 1:01 AM, Sergey Dobrov wrote:
 On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
> Old thread alert!
>
>
> Well, the spec says that the length restriction applies to the
> base64-encoded data, so I think we meant that this does *not* apply to
> the raw data. However, you seem to be saying that this might be
> difficult to implement. Have other folks experienced this problem?

 Have other folks experienced this problem implementing things right ;)

 I know that some implementations (maybe all?) measure raw data and not
 base64.
>>>
>>> If that's what all code does now, then I think it's fine for us to
>>> update the spec so that it reflects reality.
>>
>> Does anyone object to making this change in the spec?
> 
> I think measuring the raw (pre-base64-encoded) data is the most sensible, and 
> likely what I had originally intended but poorly conveyed in the XEP. So this 
> is fine by me, especially if implementations are working this way.

Super.

I propose the following change.

OLD
The base64-encoded data to be sent, prior to any wrapping in the 
element and IQ or message stanza, MUST NOT be larger than the
'block-size' determined in the bytestream negotiation.

NEW
The data to be sent, prior to base64-encoding and prior to any wrapping
in XML, MUST NOT be larger than the 'block-size' determined in the
bytestream negotiation.

/psa

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Justin Karneges]

On Tuesday, June 12, 2012 12:10:30 PM Peter Saint-Andre wrote:
> On 5/24/12 8:44 AM, Peter Saint-Andre wrote:
> > On 5/24/12 1:01 AM, Sergey Dobrov wrote:
> >> On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
> >>> Old thread alert!
> >>> 
> >>> 
> >>> Well, the spec says that the length restriction applies to the
> >>> base64-encoded data, so I think we meant that this does *not* apply to
> >>> the raw data. However, you seem to be saying that this might be
> >>> difficult to implement. Have other folks experienced this problem?
> >> 
> >> Have other folks experienced this problem implementing things right ;)
> >> 
> >> I know that some implementations (maybe all?) measure raw data and not
> >> base64.
> > 
> > If that's what all code does now, then I think it's fine for us to
> > update the spec so that it reflects reality.
> 
> Does anyone object to making this change in the spec?

I think measuring the raw (pre-base64-encoded) data is the most sensible, and 
likely what I had originally intended but poorly conveyed in the XEP. So this 
is fine by me, especially if implementations are working this way.

Justin

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

On 5/24/12 8:44 AM, Peter Saint-Andre wrote:
> On 5/24/12 1:01 AM, Sergey Dobrov wrote:
>> On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
>>> Old thread alert!
>>>
>>>
>>> Well, the spec says that the length restriction applies to the
>>> base64-encoded data, so I think we meant that this does *not* apply to
>>> the raw data. However, you seem to be saying that this might be
>>> difficult to implement. Have other folks experienced this problem?
>>
>> Have other folks experienced this problem implementing things right ;)
>>
>> I know that some implementations (maybe all?) measure raw data and not
>> base64.
> 
> If that's what all code does now, then I think it's fine for us to
> update the spec so that it reflects reality.

Does anyone object to making this change in the spec?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

On 5/24/12 1:01 AM, Sergey Dobrov wrote:
> On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
>> Old thread alert!
>>
>>
>> Well, the spec says that the length restriction applies to the
>> base64-encoded data, so I think we meant that this does *not* apply to
>> the raw data. However, you seem to be saying that this might be
>> difficult to implement. Have other folks experienced this problem?
> 
> Have other folks experienced this problem implementing things right ;)
> 
> I know that some implementations (maybe all?) measure raw data and not
> base64.

If that's what all code does now, then I think it's fine for us to
update the spec so that it reflects reality.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Sergey Dobrov]

On 05/24/2012 02:46 AM, Lance Stout wrote:
> For SleekXMPP, I applied the limit to the pre-base64 data because I 
> apparently didn't read the XEP closely enough.
> 
> I'll fix that shortly though using the 3*floor(X/4) formula.

But if you will control incoming stanzas starting with base64 length,
you will lost compatibility with many implementations. This is the main
problem, I think.

> 
> - Lance
> 


-- 
With best regards,
Sergey Dobrov,
XMPP Developer and JRuDevels.org founder.

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Sergey Dobrov]

On 05/23/2012 04:45 AM, Peter Saint-Andre wrote:
> Old thread alert!
> 
> 
> Well, the spec says that the length restriction applies to the
> base64-encoded data, so I think we meant that this does *not* apply to
> the raw data. However, you seem to be saying that this might be
> difficult to implement. Have other folks experienced this problem?

Have other folks experienced this problem implementing things right ;)

I know that some implementations (maybe all?) measure raw data and not
base64.

> 
> Peter
> 


-- 
With best regards,
Sergey Dobrov,
XMPP Developer and JRuDevels.org founder.

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Lance Stout]

For SleekXMPP, I applied the limit to the pre-base64 data because I apparently 
didn't read the XEP closely enough.

I'll fix that shortly though using the 3*floor(X/4) formula.

- Lance

On May 23, 2012, at 11:34 AM, Matthew Wild wrote:

> On 23 May 2012 02:52, Peter Saint-Andre  wrote:
>> On 5/22/12 4:13 PM, Matthew Wild wrote:
>>> Old draft alert! I really thought I had sent this... sorry.
>>> 
>>> On 31 January 2012 09:07, Sergey Dobrov  wrote:
 On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
> At its meeting on December 20, 2011, the XMPP Council agreed to issue
> a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
> preparation for perhaps advancing this specification from Draft to
> Final in the XSF's standards process. To help the Council decide
> whether this XEP is ready to advance to a status of Final, the Council
> would like to gather the following information:
> 
> 1. What software has implemented XEP-0047? Please note that the
> protocol must be implemented in at least two separate codebases (and
> preferably more).
 
 I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, I
 have my own implementation in my experimental library named "twilix"
 which is an high level wrapper on top of twisted library. This
 implementation is used now to pass attachments to my experimental
 microblogging platform implementation (named LiJ) which based on
 XEP-277. The source code of the implementation is placed here:
 https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py
 
> 
> 2. Have developers experienced any problems with the protocol as defined
> in XEP-0047? If so, please describe the problems and, if possible,
> suggested solutions.
 
 The only problem I remember with the XEP is in this paragraph:
 
> The base64-encoded data to be sent, prior to any wrapping in the  
> element and IQ or message stanza, MUST NOT be larger than the 
> 'block-size' determined in the bytestream negotiation.
 
 It's not clear for me when I have to check the length of the block: for
 raw data or for base64 encoded? In the last case it's hard to do such
 encoding. I implemented the last case for the first time but then I
 noticed that Vacuum-IM checks it for raw data and so my implementation
 said that Vacuum-IM sends inappropriate packets, so for now I am
 checking a raw data. Maybe it will be good to specify this thing more
 clearly.
 
>>> 
>>> It seems overall pretty clear to me that the XEP says the block-size
>>> limit applies to the data after base64-encoding. This does make it a
>>> bit more awkward than it should be in my opinion, and I'm not
>>> surprised clients interpret it differently.
>>> 
>>> The maximum amount of data you can fit into X bytes of base64 is
>>> 3*floor(X/4)... so for 4096 bytes of base64, 3072 bytes of data can be
>>> sent. Not hard to figure out, but perhaps "too hard", and for the sake
>>> of simplicity should be relaxed. If we do relax it (and make the limit
>>> apply to pre-base64 length) then we should decrease the recommended
>>> block size accordingly.
>> 
>> It seems that at least an implementation note about that little
>> calculation would be helpful. I don't particularly see a reason to
>> change the spec on this point, but it's not a hill for me to die on.
> 
> My preference (and I would imagine the preference of all developers)
> would be for the limit to apply to the input data. If people have
> already implemented with the current text though then I agree it makes
> sense to leave it.
> 
> On the other hand from the (little) evidence we have that it's not
> entirely clear what interpretation existing implementations have
> adopted. It wouldn't surprise me at all if the majority are applying
> the limit to the pre-base64 data, and in that case I think the XEP
> should be amended accordingly.
> 
> But we need that data before making such a move. People with IBB
> implementations, speak up!
> 
> Regards,
> Matthew



smime.p7s
Description: S/MIME cryptographic signature

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Todd Herman]

We use IBB (for Jingle and SI) in an XMPP product that we use internally and 
may release commercially soon.  A different developer, on my team, handled the 
IBB related tasking so I can't provide definitive information.  I can tell you 
that we can easily adjust to any change made at this point.  From a quick 
glance at the code, it looks like we are using block-size to control the amount 
of bytes read from the buffer at a time.  I would need to look at it further, 
but as I mentioned, we can make adjustments if anything changes (and if I 
determine we are not currently supporting it correctly).

I looked at the XEP as well and see how it can be interpreted as referring to 
the encoded data rather than the raw data.  I would have to make a  slight 
observation here that a statement in 2.2 ("stanza, MUST NOT be larger than the 
'block-size'") says "MUST NOT be larger" which implies it can be equal to or 
smaller.  This leaves room for interpretation.  I don't see anywhere that the 
block-size definitively refers to the raw or encoded data.  I would suggest the 
wording be more explicit here so interpretation is limited.  I would also say 
that block-size should definitively refer to the encoded data since that is 
what is stored in the "data" portion of the message.

Todd Herman


-Original Message-
From: standards-boun...@xmpp.org [mailto:standards-boun...@xmpp.org] On Behalf 
Of Matthew Wild
Sent: Wednesday, May 23, 2012 2:34 PM
To: Peter Saint-Andre
Cc: XMPP Standards
Subject: Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band 
Bytestreams) to Final

On 23 May 2012 02:52, Peter Saint-Andre  wrote:
> On 5/22/12 4:13 PM, Matthew Wild wrote:
>> Old draft alert! I really thought I had sent this... sorry.
>>
>> On 31 January 2012 09:07, Sergey Dobrov  wrote:
>>> On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
>>>> At its meeting on December 20, 2011, the XMPP Council agreed to 
>>>> issue a "Call for Experience" regarding XEP-0047 (In-Band 
>>>> Bytestreams), in preparation for perhaps advancing this 
>>>> specification from Draft to Final in the XSF's standards process. 
>>>> To help the Council decide whether this XEP is ready to advance to 
>>>> a status of Final, the Council would like to gather the following 
>>>> information:
>>>>
>>>> 1. What software has implemented XEP-0047? Please note that the 
>>>> protocol must be implemented in at least two separate codebases 
>>>> (and preferably more).
>>>
>>> I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, 
>>> I have my own implementation in my experimental library named "twilix"
>>> which is an high level wrapper on top of twisted library. This 
>>> implementation is used now to pass attachments to my experimental 
>>> microblogging platform implementation (named LiJ) which based on 
>>> XEP-277. The source code of the implementation is placed here:
>>> https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/b
>>> ase.py
>>>
>>>>
>>>> 2. Have developers experienced any problems with the protocol as 
>>>> defined in XEP-0047? If so, please describe the problems and, if 
>>>> possible, suggested solutions.
>>>
>>> The only problem I remember with the XEP is in this paragraph:
>>>
>>>> The base64-encoded data to be sent, prior to any wrapping in the  
>>>> element and IQ or message stanza, MUST NOT be larger than the 'block-size' 
>>>> determined in the bytestream negotiation.
>>>
>>> It's not clear for me when I have to check the length of the block: 
>>> for raw data or for base64 encoded? In the last case it's hard to do 
>>> such encoding. I implemented the last case for the first time but 
>>> then I noticed that Vacuum-IM checks it for raw data and so my 
>>> implementation said that Vacuum-IM sends inappropriate packets, so 
>>> for now I am checking a raw data. Maybe it will be good to specify 
>>> this thing more clearly.
>>>
>>
>> It seems overall pretty clear to me that the XEP says the block-size 
>> limit applies to the data after base64-encoding. This does make it a 
>> bit more awkward than it should be in my opinion, and I'm not 
>> surprised clients interpret it differently.
>>
>> The maximum amount of data you can fit into X bytes of base64 is 
>> 3*floor(X/4)... so for 4096 bytes of base64, 3072 bytes of data can 
>> be sent. Not hard to figure out, but perhaps "too hard", and for 

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Matthew Wild]

On 23 May 2012 02:52, Peter Saint-Andre  wrote:
> On 5/22/12 4:13 PM, Matthew Wild wrote:
>> Old draft alert! I really thought I had sent this... sorry.
>>
>> On 31 January 2012 09:07, Sergey Dobrov  wrote:
>>> On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
 At its meeting on December 20, 2011, the XMPP Council agreed to issue
 a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
 preparation for perhaps advancing this specification from Draft to
 Final in the XSF's standards process. To help the Council decide
 whether this XEP is ready to advance to a status of Final, the Council
 would like to gather the following information:

 1. What software has implemented XEP-0047? Please note that the
 protocol must be implemented in at least two separate codebases (and
 preferably more).
>>>
>>> I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, I
>>> have my own implementation in my experimental library named "twilix"
>>> which is an high level wrapper on top of twisted library. This
>>> implementation is used now to pass attachments to my experimental
>>> microblogging platform implementation (named LiJ) which based on
>>> XEP-277. The source code of the implementation is placed here:
>>> https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py
>>>

 2. Have developers experienced any problems with the protocol as defined
 in XEP-0047? If so, please describe the problems and, if possible,
 suggested solutions.
>>>
>>> The only problem I remember with the XEP is in this paragraph:
>>>
 The base64-encoded data to be sent, prior to any wrapping in the  
 element and IQ or message stanza, MUST NOT be larger than the 'block-size' 
 determined in the bytestream negotiation.
>>>
>>> It's not clear for me when I have to check the length of the block: for
>>> raw data or for base64 encoded? In the last case it's hard to do such
>>> encoding. I implemented the last case for the first time but then I
>>> noticed that Vacuum-IM checks it for raw data and so my implementation
>>> said that Vacuum-IM sends inappropriate packets, so for now I am
>>> checking a raw data. Maybe it will be good to specify this thing more
>>> clearly.
>>>
>>
>> It seems overall pretty clear to me that the XEP says the block-size
>> limit applies to the data after base64-encoding. This does make it a
>> bit more awkward than it should be in my opinion, and I'm not
>> surprised clients interpret it differently.
>>
>> The maximum amount of data you can fit into X bytes of base64 is
>> 3*floor(X/4)... so for 4096 bytes of base64, 3072 bytes of data can be
>> sent. Not hard to figure out, but perhaps "too hard", and for the sake
>> of simplicity should be relaxed. If we do relax it (and make the limit
>> apply to pre-base64 length) then we should decrease the recommended
>> block size accordingly.
>
> It seems that at least an implementation note about that little
> calculation would be helpful. I don't particularly see a reason to
> change the spec on this point, but it's not a hill for me to die on.

My preference (and I would imagine the preference of all developers)
would be for the limit to apply to the input data. If people have
already implemented with the current text though then I agree it makes
sense to leave it.

On the other hand from the (little) evidence we have that it's not
entirely clear what interpretation existing implementations have
adopted. It wouldn't surprise me at all if the majority are applying
the limit to the pre-base64 data, and in that case I think the XEP
should be amended accordingly.

But we need that data before making such a move. People with IBB
implementations, speak up!

Regards,
Matthew

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

On 5/22/12 4:13 PM, Matthew Wild wrote:
> Old draft alert! I really thought I had sent this... sorry.
> 
> On 31 January 2012 09:07, Sergey Dobrov  wrote:
>> On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
>>> At its meeting on December 20, 2011, the XMPP Council agreed to issue
>>> a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
>>> preparation for perhaps advancing this specification from Draft to
>>> Final in the XSF's standards process. To help the Council decide
>>> whether this XEP is ready to advance to a status of Final, the Council
>>> would like to gather the following information:
>>>
>>> 1. What software has implemented XEP-0047? Please note that the
>>> protocol must be implemented in at least two separate codebases (and
>>> preferably more).
>>
>> I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, I
>> have my own implementation in my experimental library named "twilix"
>> which is an high level wrapper on top of twisted library. This
>> implementation is used now to pass attachments to my experimental
>> microblogging platform implementation (named LiJ) which based on
>> XEP-277. The source code of the implementation is placed here:
>> https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py
>>
>>>
>>> 2. Have developers experienced any problems with the protocol as defined
>>> in XEP-0047? If so, please describe the problems and, if possible,
>>> suggested solutions.
>>
>> The only problem I remember with the XEP is in this paragraph:
>>
>>> The base64-encoded data to be sent, prior to any wrapping in the  
>>> element and IQ or message stanza, MUST NOT be larger than the 'block-size' 
>>> determined in the bytestream negotiation.
>>
>> It's not clear for me when I have to check the length of the block: for
>> raw data or for base64 encoded? In the last case it's hard to do such
>> encoding. I implemented the last case for the first time but then I
>> noticed that Vacuum-IM checks it for raw data and so my implementation
>> said that Vacuum-IM sends inappropriate packets, so for now I am
>> checking a raw data. Maybe it will be good to specify this thing more
>> clearly.
>>
> 
> It seems overall pretty clear to me that the XEP says the block-size
> limit applies to the data after base64-encoding. This does make it a
> bit more awkward than it should be in my opinion, and I'm not
> surprised clients interpret it differently.
> 
> The maximum amount of data you can fit into X bytes of base64 is
> 3*floor(X/4)... so for 4096 bytes of base64, 3072 bytes of data can be
> sent. Not hard to figure out, but perhaps "too hard", and for the sake
> of simplicity should be relaxed. If we do relax it (and make the limit
> apply to pre-base64 length) then we should decrease the recommended
> block size accordingly.

It seems that at least an implementation note about that little
calculation would be helpful. I don't particularly see a reason to
change the spec on this point, but it's not a hill for me to die on.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Matthew Wild]

Old draft alert! I really thought I had sent this... sorry.

On 31 January 2012 09:07, Sergey Dobrov  wrote:
> On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
>> At its meeting on December 20, 2011, the XMPP Council agreed to issue
>> a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
>> preparation for perhaps advancing this specification from Draft to
>> Final in the XSF's standards process. To help the Council decide
>> whether this XEP is ready to advance to a status of Final, the Council
>> would like to gather the following information:
>>
>> 1. What software has implemented XEP-0047? Please note that the
>> protocol must be implemented in at least two separate codebases (and
>> preferably more).
>
> I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, I
> have my own implementation in my experimental library named "twilix"
> which is an high level wrapper on top of twisted library. This
> implementation is used now to pass attachments to my experimental
> microblogging platform implementation (named LiJ) which based on
> XEP-277. The source code of the implementation is placed here:
> https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py
>
>>
>> 2. Have developers experienced any problems with the protocol as defined
>> in XEP-0047? If so, please describe the problems and, if possible,
>> suggested solutions.
>
> The only problem I remember with the XEP is in this paragraph:
>
>> The base64-encoded data to be sent, prior to any wrapping in the  
>> element and IQ or message stanza, MUST NOT be larger than the 'block-size' 
>> determined in the bytestream negotiation.
>
> It's not clear for me when I have to check the length of the block: for
> raw data or for base64 encoded? In the last case it's hard to do such
> encoding. I implemented the last case for the first time but then I
> noticed that Vacuum-IM checks it for raw data and so my implementation
> said that Vacuum-IM sends inappropriate packets, so for now I am
> checking a raw data. Maybe it will be good to specify this thing more
> clearly.
>

It seems overall pretty clear to me that the XEP says the block-size
limit applies to the data after base64-encoding. This does make it a
bit more awkward than it should be in my opinion, and I'm not
surprised clients interpret it differently.

The maximum amount of data you can fit into X bytes of base64 is
3*floor(X/4)... so for 4096 bytes of base64, 3072 bytes of data can be
sent. Not hard to figure out, but perhaps "too hard", and for the sake
of simplicity should be relaxed. If we do relax it (and make the limit
apply to pre-base64 length) then we should decrease the recommended
block size accordingly.

Regards,
Matthew

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

Old thread alert!

On 1/31/12 2:07 AM, Sergey Dobrov wrote:
> On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
>> At its meeting on December 20, 2011, the XMPP Council agreed to
>> issue a "Call for Experience" regarding XEP-0047 (In-Band
>> Bytestreams), in preparation for perhaps advancing this
>> specification from Draft to Final in the XSF's standards process.
>> To help the Council decide whether this XEP is ready to advance to
>> a status of Final, the Council would like to gather the following
>> information:
>> 
>> 1. What software has implemented XEP-0047? Please note that the 
>> protocol must be implemented in at least two separate codebases
>> (and preferably more).
> 
> I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also,
> I have my own implementation in my experimental library named
> "twilix" which is an high level wrapper on top of twisted library.
> This implementation is used now to pass attachments to my
> experimental microblogging platform implementation (named LiJ) which
> based on XEP-277. The source code of the implementation is placed
> here: 
> https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py

Thanks for letting us know!

>> 2. Have developers experienced any problems with the protocol as
>> defined in XEP-0047? If so, please describe the problems and, if
>> possible, suggested solutions.
> 
> The only problem I remember with the XEP is in this paragraph:
> 
>> The base64-encoded data to be sent, prior to any wrapping in the
>>  element and IQ or message stanza, MUST NOT be larger than
>> the 'block-size' determined in the bytestream negotiation.
> 
> It's not clear for me when I have to check the length of the block:
> for raw data or for base64 encoded? In the last case it's hard to do
> such encoding. I implemented the last case for the first time but
> then I noticed that Vacuum-IM checks it for raw data and so my
> implementation said that Vacuum-IM sends inappropriate packets, so
> for now I am checking a raw data. Maybe it will be good to specify
> this thing more clearly.

Well, the spec says that the length restriction applies to the
base64-encoded data, so I think we meant that this does *not* apply to
the raw data. However, you seem to be saying that this might be
difficult to implement. Have other folks experienced this problem?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Guillaume Desmottes]

Le lundi 30 janvier 2012 à 15:54 -0700, Peter Saint-Andre a écrit :
> 1. What software has implemented XEP-0047? Please note that the
> protocol must be implemented in at least two separate codebases (and
> preferably more).

We implemented it in telepathy-gabble with Stream Initiation (we'll
switch to Jingle at some point hopefully). It's currently used as a
fallback when S5B is failing with file transfers and tubes.

We also implemented a variant which is used in MUC for D-Bus tubes data
transfer, but that's probably out of scope for the original XEP.

> 2. Have developers experienced any problems with the protocol as defined
> in XEP-0047? If so, please describe the problems and, if possible,
> suggested solutions.

Tbh I implemented this almost 5 years ago so I don't really remember the
details. IIRC most of our concerns were about Stream Initiation not IBB
itself.

Looking at the code, we implement a "sending window" of 10 stanzas: we
send 10 stanzas and stop sending until we start receiving acks.
XEP says: "The sender of a data chunk need not wait for these
acknowledgements before sending further stanzas. However, it is
RECOMMENDED that the sender does wait in order to minimize the potential
for rate-limiting penalties or throttling."

Maybe it could suggest using such window to improve performance?



G.

Re: [Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Sergey Dobrov]

On 01/31/2012 05:54 AM, Peter Saint-Andre wrote:
> At its meeting on December 20, 2011, the XMPP Council agreed to issue
> a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
> preparation for perhaps advancing this specification from Draft to
> Final in the XSF's standards process. To help the Council decide
> whether this XEP is ready to advance to a status of Final, the Council
> would like to gather the following information:
> 
> 1. What software has implemented XEP-0047? Please note that the
> protocol must be implemented in at least two separate codebases (and
> preferably more).

I have tested XEP-47 implementations in tkabber and Vacuum-IM. Also, I
have my own implementation in my experimental library named "twilix"
which is an high level wrapper on top of twisted library. This
implementation is used now to pass attachments to my experimental
microblogging platform implementation (named LiJ) which based on
XEP-277. The source code of the implementation is placed here:
https://github.com/xamvz/twilix/blob/master/twilix/bytestreams/ibb/base.py

> 
> 2. Have developers experienced any problems with the protocol as defined
> in XEP-0047? If so, please describe the problems and, if possible,
> suggested solutions.

The only problem I remember with the XEP is in this paragraph:

> The base64-encoded data to be sent, prior to any wrapping in the  
> element and IQ or message stanza, MUST NOT be larger than the 'block-size' 
> determined in the bytestream negotiation.

It's not clear for me when I have to check the length of the block: for
raw data or for base64 encoded? In the last case it's hard to do such
encoding. I implemented the last case for the first time but then I
noticed that Vacuum-IM checks it for raw data and so my implementation
said that Vacuum-IM sends inappropriate packets, so for now I am
checking a raw data. Maybe it will be good to specify this thing more
clearly.

> 
> 3. Is the text of XEP-0047 clear and unambiguous? Are more examples
> needed? Is the conformance language (MAY/SHOULD/MUST) appropriate?
> Have developers found the text confusing at all? Please describe any
> suggestions you have for improving the text.
> 
> If you have any comments about advancing XEP-0047 from Draft to Final,
> please provide them by the close of business on Friday, February 24,
> 2012. After the Call for Experience, this XEP might undergo revisions
> to address feedback received, after which it will be presented to the
> XMPP Council for voting to a status of Final.
> 
> You can review the specification here:
> 
> http://www.xmpp.org/extensions/xep-0047.html
> 
> Please send all feedback to the standards@xmpp.org discussion list.
> 
> Thanks!
> 
> Peter
> 
> 

-- 
With best regards,
Sergey Dobrov,
XMPP Developer and JRuDevels.org founder.

[Standards] Call for Experience: Advancement of XEP-0047 (In-Band Bytestreams) to Final

[Peter Saint-Andre]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

At its meeting on December 20, 2011, the XMPP Council agreed to issue
a "Call for Experience" regarding XEP-0047 (In-Band Bytestreams), in
preparation for perhaps advancing this specification from Draft to
Final in the XSF's standards process. To help the Council decide
whether this XEP is ready to advance to a status of Final, the Council
would like to gather the following information:

1. What software has implemented XEP-0047? Please note that the
protocol must be implemented in at least two separate codebases (and
preferably more).

2. Have developers experienced any problems with the protocol as defined
in XEP-0047? If so, please describe the problems and, if possible,
suggested solutions.

3. Is the text of XEP-0047 clear and unambiguous? Are more examples
needed? Is the conformance language (MAY/SHOULD/MUST) appropriate?
Have developers found the text confusing at all? Please describe any
suggestions you have for improving the text.

If you have any comments about advancing XEP-0047 from Draft to Final,
please provide them by the close of business on Friday, February 24,
2012. After the Call for Experience, this XEP might undergo revisions
to address feedback received, after which it will be presented to the
XMPP Council for voting to a status of Final.

You can review the specification here:

http://www.xmpp.org/extensions/xep-0047.html

Please send all feedback to the standards@xmpp.org discussion list.

Thanks!

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/


-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8nH7UACgkQNL8k5A2w/vzSQwCffoBBmDnr+jJ3aOqrS6LF89AN
WpwAnA+1oPOYJ79N1NkK+1cd9dNu1HhL
=ld4P
-END PGP SIGNATURE-