[Standards] file transfer (was: Re: [LONG] Jeez, Sorry)

2007-08-27 Thread Peter Saint-Andre
Please stop top-posting. It makes the discussion hard to follow.

Jonathan Chayce Dickinson wrote:
 I'm sorry. Didn't mean any harm. 

I know you didn't. I appreciate your enthusiasm, I really do! But let's
temper it with some realism. :)

 I'm not too hot on bandwidth here, 

I'm not too hot on mental bandwidth here, either. :)

 so I
 didn't come across MTOM.

So you didn't research very thoroughly, did you?

If you click I'm Feeling Lucky at http://www.google.com/ when
searching for Direct Internet Message Encapsulation, you come to the
following page:

http://xml.coverpages.org/dime.html

And right there at the top you find this text:

Note: Microsoft has listed DIME among Superseded Specifications in its
Messaging Specifications Index Page. See Direct Internet Message
Encapsulation (DIME) [June 17, 2002]: This specification has been
superseded by the SOAP Message Transmission Optimization Mechanism
(MTOM) specification.

So here we find many things we need to know. In particular, the
technology was superseded and was never seriously pursued. So why are we
discussing it here?

However, your suggestion might lead to a productive discussion about
file transfer / file sharing, see below

 I wasn't wasting your time, quote: Didn't want to push this forward
 until I got some feedback. I don't like wasting people's time, hell, I
 love helping people out and solving problems: don't reply to this and
 you will never hear of it again; all it takes is nope, not a good
 idea, not something as violent as fanciful (ouch!).

It's fanciful to seriously suggest using a technology that was
superseded and abandoned over 5 years ago. Let's try to use more
standardized and widely-available technologies if possible.

 Read the spec and you will see that it has a lot to do with XMPP: Namely
 (quoted from my draft):
 
 1. DIME will rectify the absence of true In-Band Bytestreams, as it
 defines methods to send multiple payloads in a atomical message. Each
 payload in the message can reference other payloads in the message.
 
 XMPP needs, in my opinion, a way to negotiate real in-band bytestreams.

What are the problems with XEP-0047?

 Base64 seems like a cop-out to me, it really does.
 
 2. Furthermore compression methods used by servers (which tend to be
 fast instead of -small-effective) will, at best, return the data to near
 it's original length. If the data is presented to these algorithms in
 its original state they may be able to compress it even further.
 
 Not everyone has a T3 connection at home, some of us live in South
 Africa, and well, bandwidth here is rather expensive: and so is a static
 IP (yes, we differentiate between them here, we /actually/ have
 non-static ones too). When you have to weigh up the value of 1MB, you
 will know.
 
 3. Ultimately this protocol will smooth out the issues plaguing Jabber
 regarding file sharing.
 
 This was a hot topic on JDev not too long ago! Demand, supply. If the
 developers are demanding it, I ASSURE you, the users will be fast
 following (faster than any cheetah metaphores ;)
 
 4. Almost all the existing XEPs should be able to derive some further
 benefit from this extension (especially Jingle and SI).
 
 That is a bit fanciful and ambitious, I admit. Add XHTML to that list,
 and well there you go, not even nearly all, but 3, which is a start.
 
 (Just coming in on that regard, SOAP has nothing to do with XMPP, but
 neither did SASL (at the start), or XHTML... And there is SOAP over XMPP
 now)
 
 And, well, *both* are XML and *both* suffer from the fact that you can't
 package binary data reliably and efficiently in an XML document. 

This is very true.

Jabber was not designed for binary transfer. This is a strength for the
core use cases, but a weakness for things like file transfer and file
sharing and media sessions, as we have painfully discovered time and again.

However, personally I'd say we don't want to use an abandoned technology
to solve the problem.

Another approach would be to define a BEEP transport:

http://www.beepcore.org/

Yes, BEEP is not widely implemented either, but I at least there are
RFCs (and I am aware of a new push for wider implementation).

Maybe it would help if I finished writing up some results of the file
transfer discussions at the recent XMPP DevCon?

 MTOM
 wouldn't exist if it wasn't a real problem. It would have ended with DIME.

But why was DIME abandoned? (Not that I like MTOM, mind you!)

 A lot of protocols do it, hell, even your cellphone does something
 similar (which is why you can call someone and surf over GPRS/3G/HSDPA
 at the same time).
 
 In regard to Dave's email, about the negative compression etc. very
 true, didn't think of that. Nice, constructive criticism. Dave: the very
 reason I thought of it was because a P2P connection isn't always
 possible, as in my case (I don't need it, but the awareness is there). I
 was hoping to use the server to host the Peer-To-Host-To-Peer connection
 instead of some 3rd 

Re: [Standards] file transfer (was: Re: [LONG] Jeez, Sorry)

2007-08-27 Thread Hiếu Hoàng
On 8/28/07, Peter Saint-Andre [EMAIL PROTECTED] wrote:
[snip]

 That appears to be a sensible line of thinking. However, it seems to me
 that the wide availability of HTTP servers (which could be bundled with
 your favorite XMPP server) makes an HTTP approach even more appealing.

 Part of my question is: do we really need to stream files? Think about
 the difference between media streaming and media downloading. The use
 cases in which true streaming is needed seem few. Even things like
 podcasting are not truly casted in real time -- they are uploaded to an
 HTTP server and then downloaded, to be listened to on demand.

 Similarly, it seems to me that if I want to send you a file, I could
 upload it to an HTTP (associated with my XMPP server), get a URL for
 that file, then send you the URL. If I don't have an HTTP server at my
 disposal, I could fall back to in-band bytestreams. But let's face it,
 the HTTP community has file-upload and file-download down cold. Why not
 leverage all that? We don't need to do *everything* via XMPP! Do we
 really need streaming here, or does it just seem cool?

[huge snip]

 Again I think the fundamental question is: do we *really* need to stream
 things, or can we use sender-upload and receiver-download for many or
 most use cases, with in-band bytestreams as the fallback?

 Peter

Being a only newbie user, with expensive bandwidth, I like this path.
Currently, I know of disk.jabbim.cz service which provides HTTP
hosting for files, say http://disk.jabbim.cz/[EMAIL PROTECTED]/. That is
really convenient for me, either I can tell it to send the file to
some JID, or I can copy the URL of the file to post independently.
That's good (except for the Czechy messages, maybe someone can hack
xml:lang support into it).

The other booster is private storage, sharing the same hosting space
but the files don't have HTTP access. People can't just build up the
URLs and download the files from everywhere, abusing the hosting
service.

I know that this is using SI for sending between JIDs, but it provides
a nice alternative for people with expensive Internet access.

Hiếu