xxxterm http download weirdness

2012-03-07 Thread Mihai Popescu
Hello,

I was starting a download of ports from
http://ftp.openbsd.org/pub/openbsd/snapshots/ports.tar.gz to compare
it with an old archive from a mirror. As you can see, the file size is
listed as 20.7M, but what I got is a 220M file which is   corrupted as
reported by tar. The fact is that artfiles.org mirror works fine. If I
watch the bigger file inside download window, it shows 20.7M but after
that keeps incrementing till ~ 220M.

Is anyone experiencing this ?

xxxterm-1.10.0v0 (without configuration file)
OpenBSD 5.1 (GENERIC) #160: Sun Feb 12 09:46:33 MST 2012
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC



Re: xxxterm http download weirdness

2012-03-07 Thread Remco
Mihai Popescu wrote:

 Hello,
 
 I was starting a download of ports from
 http://ftp.openbsd.org/pub/openbsd/snapshots/ports.tar.gz to compare
 it with an old archive from a mirror. As you can see, the file size is
 listed as 20.7M, but what I got is a 220M file which is   corrupted as
 reported by tar. The fact is that artfiles.org mirror works fine. If I
 watch the bigger file inside download window, it shows 20.7M but after
 that keeps incrementing till ~ 220M.
 
 Is anyone experiencing this ?
 

Haven't tried this myself but I suspect that your HTTP client (xxxterm)
tries to decompress the file in transit.

AFAICT the HTTP protocol supports compression on the wire.
e.g.:
- you request index.html on a server
- the server compresses the file using gzip
- the compressed file is sent to your browser
- your browser decompresses the gzipped content
   and renders the resulting HTML so you can view it

I suspect the ports.tar.gz is seen as gzipped HTTP content and your browser
tries to decompress it before doing something with it.

If this is the case, I don't know how to avoid this. Maybe xxxterm has an
option to avoid decompressing content that was already compressed before it
was put on the wire. Or maybe someone can recommend another tool.
(e.g., does 'ftp http://ftp.openbsd.org/pub/openbsd/snapshots/ports.tar.gz'
suffer from this as well ?)