Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Grant Edwards]

On 2009-09-25, Wu, Yue  wrote:

> Still want to say, if manual.txt.gzip isn't a real plain text,
> it should be renamed to a more proper and clear name. And
> please put a full version of 
> manual.txt[http://www.mutt.org/doc/devel/manual.txt] on the
> offical website.
>
> What do you think?

I agree completely.  A user shouldn't have to jump through
hoops to look at the "text" version of the manual.  It should
be viewable using less, more, cat, vi, emacs, ed, mutt's pager,
or anything else.  It should be plain text without
printer-control-codes, vt100 escape sequences, postscript
commands, embedded bitmapped fonts, HPCL, or anything else.

Unless there are overriding reasons not to, I'd recommend
sticking with 7-bit ASCII or Latin-1 encoding.

If fancy formatting is desired, then generate a PDF, RTF, or
Postscript file.

We can live without a copyright symbol in the manual.  Really.

-- 
Grant Edwards   grante Yow! I KAISER ROLL?!
  at   What good is a Kaiser Roll
   visi.comwithout a little COLE SLAW
   on the SIDE?

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Wu, Yue]

On Fri, Sep 25, 2009 at 01:57:45AM -0400, James Michael Fultz wrote:
> * "Wu, Yue"  [2009-09-25 13:22 +0800]:
> [ ... ]
> > Hi James,
> > 
> > I've tried `col -b` and `col -bp`, both produce errors and stop immatiatly.
> > 
> > y...@bsd ~ > cat manual.txt | col -b -p
> > 
> >9.14. Default editor Function Bindings
> > 
> >col: Illegal byte sequence
> 
> Curious.  While the '-p' option to col may not be a cure-all for
> dealing with unsupported character sets, I don't see any such
> character at that location either.  Looking at a hexdump of that
> section of the manual, aside from the embedded backspace
> characters, it appears to be ASCII.
> 
> Perhaps your copy of the manual is still yet corrupted?  Here are
> the MD5 checksums for the files I have downloaded.
> 
> dff620858d4aa4d9df7b46a02ede482b  manual.txt
> 832c15a5325130eaa5d81d038e373aaa  manual.txt.gz
> 
> 'manual.txt' being a decompressed copy of the downloaded
> 'manual.txt.gz'.

y...@bsd ~ > md5 manual.txt.gz
MD5 (manual.txt.gz) = 832c15a5325130eaa5d81d038e373aaa
y...@bsd ~ > md5 manual.txt
MD5 (manual.txt) = dff620858d4aa4d9df7b46a02ede482b
y...@bsd ~ > cat manual.txt | col -b -p
[...omitted most parts]
   9.14. Default editor Function Bindings

   col: Illegal byte sequence
  List_of_Examplesy

using col -b without -p produces the same error.

Still want to say, if manual.txt.gzip isn't a real plain text, it should be
renamed to a more proper and clear name. And please put a full version of
manual.txt[http://www.mutt.org/doc/devel/manual.txt] on the offical website.

What do you think?

-- 
Hi,
Wu, Yue

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[James Michael Fultz]

* "Wu, Yue"  [2009-09-25 13:22 +0800]:
[ ... ]
> Hi James,
> 
> I've tried `col -b` and `col -bp`, both produce errors and stop immatiatly.
> 
> y...@bsd ~ > cat manual.txt | col -b -p
> 
>9.14. Default editor Function Bindings
> 
>col: Illegal byte sequence

Curious.  While the '-p' option to col may not be a cure-all for
dealing with unsupported character sets, I don't see any such
character at that location either.  Looking at a hexdump of that
section of the manual, aside from the embedded backspace
characters, it appears to be ASCII.

Perhaps your copy of the manual is still yet corrupted?  Here are
the MD5 checksums for the files I have downloaded.

dff620858d4aa4d9df7b46a02ede482b  manual.txt
832c15a5325130eaa5d81d038e373aaa  manual.txt.gz

'manual.txt' being a decompressed copy of the downloaded
'manual.txt.gz'.

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Wu, Yue]

On Thu, Sep 24, 2009 at 10:40:15PM -0400, James Michael Fultz wrote:
> 
> There's an explanation to be found in less' man page and Mutt's
> manual.
> 
> 
> 
> "Also, the internal pager supports a couple other advanced
> features. For one, it will accept and translate the ``standard''
> nroff sequences for bold and underline. These sequences are a
> series of either the letter, backspace (^H), the letter again for
> bold or the letter, backspace, ``_'' for denoting underline. ..."
> 
> You can convert those sequences to regular text using col with
> its '-b' option.  Due to some versions of col's inability to
> handle UTF-8, the '-p' option may be necessary as well.

Hi James,

I've tried `col -b` and `col -bp`, both produce errors and stop immatiatly.

y...@bsd ~ > cat manual.txt | col -b -p

   9.14. Default editor Function Bindings

   col: Illegal byte sequence

-- 
Hi,
Wu, Yue

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[James Michael Fultz]

* "Wu, Yue"  [2009-09-25 07:49 +0800]:
[ ... ]
> Also, I don't know how to view the file as a plain text, when I
> try to view it by vim, the garbage charactors full-filled with
> my screen.
> 
> `less` is ok. Maybe the file contains many control sequences
> that vim can't recorgnize but `less` can? If so, then I think
> it should be renamed to manual.less or other thing to make it
> clear.

There's an explanation to be found in less' man page and Mutt's
manual.



"Also, the internal pager supports a couple other advanced
features. For one, it will accept and translate the ``standard''
nroff sequences for bold and underline. These sequences are a
series of either the letter, backspace (^H), the letter again for
bold or the letter, backspace, ``_'' for denoting underline. ..."

You can convert those sequences to regular text using col with
its '-b' option.  Due to some versions of col's inability to
handle UTF-8, the '-p' option may be necessary as well.

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Wu, Yue]

On Thu, Sep 24, 2009 at 03:20:39PM +0200, Joost Kremers wrote:
> On Thu, Sep 24, 2009 at 03:12:53PM +0200, Michael Wagner wrote:
> > * Wu, Yue  24.09.2009
> > > y...@bsd ~/gtd/stuff > gzip -tv manual.txt.gz
> > > gzip: manual.txt.gz: not in gzip format
> > > manual.txt.gz:NOT OK
> > 
> > I don't understand it. Here is all ok.
> 
> I've seen it happen before that a browser silently unzips a file it downloads
> but doesn't bother to change the file ending.

Thanks for pointing me out of it, gzipped file issue has gone.

When I use

fetch http://www.mutt.org/doc/devel/manual.txt.gz

the file is ok(fetch is a wget-like app in FreeBSD).

> 
> Wu, try the `file' command on manual.txt.gz. If it says 'ASCII Text', then the
> file was decompressed.

Maybe opera does some suck thing on the gzipped file it
downloads:

y...@bsd ~ > file manual.txt.gz
manual.txt.gz: ISO-8859 English text, with overstriking

Don't know what's wrong with opera :(

Also, I don't know how to view the file as a plain text, when I try to view it
by vim, the garbage charactors full-filled with my screen.

`less` is ok. Maybe the file contains many control sequences that vim can't
recorgnize but `less` can? If so, then I think it should be renamed to
manual.less or other thing to make it clear.

-- 
Hi,
Wu, Yue

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Joost Kremers]

On Thu, Sep 24, 2009 at 03:12:53PM +0200, Michael Wagner wrote:
> * Wu, Yue  24.09.2009
> > y...@bsd ~/gtd/stuff > gzip -tv manual.txt.gz
> > gzip: manual.txt.gz: not in gzip format
> > manual.txt.gz:NOT OK
> 
> I don't understand it. Here is all ok.

I've seen it happen before that a browser silently unzips a file it downloads
but doesn't bother to change the file ending.

Wu, try the `file' command on manual.txt.gz. If it says 'ASCII Text', then the
file was decompressed.


-- 
Joost Kremers
Life has its moments

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Michael Wagner]

* Wu, Yue  24.09.2009
> On Wed, Sep 23, 2009 at 06:09:38PM +0200, Michael Wagner wrote:
 
> > the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
> > file. I've downloaded it for testing and everything is allright with the 
> > it. Also when I visited the side with different text browsers, they 
> > showed me all the correct file.
> > 
> > What browser do you use?
> 
> I'm using opera 10.0 for FreeBSD.
> 
> y...@bsd ~/gtd/stuff > gzip -tv manual.txt.gz
> gzip: manual.txt.gz: not in gzip format
> manual.txt.gz:NOT OK
> 
> y...@bsd ~/gtd/stuff > gzip --version
> FreeBSD gzip 20090621
> 
> I'm curious why not just use `tar jcvf manul.txt.bz2 manual.txt`?

Hello Wu,

I don't understand it. Here is all ok.

mich...@workstation tty2:~$ wget http://www.mutt.org/doc/devel/manual.txt.gz

mich...@workstation tty2:~$ file manual.txt.gz 
manual.txt.gz: gzip compressed data, was "manual.txt", from Unix,
last modified: Sun Jul  5 00:12:55 2009, max compression

mich...@workstation tty2:~$ gzip -tv manual.txt.gz
manual.txt.gz:   OK

Is it possible that 'opera' does an offending download? Download it one 
more time with 'wget' and look what happen. Sorry, when this will give 
you not the correct file, I have no idea what to do more. But I can send 
you the manual with email. Write to me, if you want it this way.

Michael

-- 
"Nowhere you can be that isn't where you were meant to be. It's easy."
   -- John Lennon


signature.asc
Description: Digital signature

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Wu, Yue]

On Wed, Sep 23, 2009 at 06:09:38PM +0200, Michael Wagner wrote:
> * Wu, Yue  23.09.2009
> > In mutt offical site, the documentation for devel version:
> > 
> > text version: http://www.mutt.org/doc/devel/manual.txt
> > 
> > is an uncompleted version, which just give a table of contents and the first
> > chapter.
> > 
> > text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
> > 
> > isn't a gzipped version actually, I can't gunzip with it.
> 
> Hello Wu,
> 
> the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
> file. I've downloaded it for testing and everything is allright with the 
> it. Also when I visited the side with different text browsers, they 
> showed me all the correct file.
> 
> What browser do you use?

I'm using opera 10.0 for FreeBSD.

y...@bsd ~/gtd/stuff > gzip -tv manual.txt.gz
gzip: manual.txt.gz: not in gzip format
manual.txt.gz:NOT OK

y...@bsd ~/gtd/stuff > gzip --version
FreeBSD gzip 20090621

I'm curious why not just use `tar jcvf manul.txt.bz2 manual.txt`?

-- 
Hi,
Wu, Yue

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[James Michael Fultz]

* Noah Sheppard  [2009-09-23 14:44 -0400]:
> On Wed, Sep 23, 2009 at 02:37:21PM -0400, James Michael Fultz
> wrote:
[ ... ]
> > Try less and 'col -b manual.txt | less' if the former doesn't
> > display cleanly.  The col command will strip embedded
> > backspace sequences.
> 
> That worked, thanks for the tip (although col complained
> "Invalid or incomplete multibyte or wide character").  Also,
> when I zcat the downloaded gz, and run it through col, I get
> truncated at exactly the same point (right after "Mutt is
> Copyright").  If, however, I zcat the file directly to less
> without col, everything displays correctly, with the bolding,
> and in its entirety, all the way to the end of chapter 10.

$ zcat manual.txt.gz | col -b | less

Works fine here -- Ubuntu 9.04 64-bit.  Maybe there's a bug in
your version col?  Could be a hint here as to the state of the
uncompressed copy of 'manual.txt' on the server.

Hmm.  Does appear that some versions of col don't handle UTF-8.
For example:


A workaround appears to be to use 'col -bxp'.

> Perhaps there is some character in manual.txt which is causing
> truncation somewhere, perhaps server-side, or perhaps in wget,
> less, and firefox (for the uncompressed manual.txt).  Maybe
> some library on our systems common to all those which does not
> like some special character?  Just a guess.

I think that the file as it is saved on the server is truncated.
I would also chance to guess that col is used to process it, and
the same bug or a similar bug to the one you've experienced
affects it there as well.

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Grant Edwards]

On 2009-09-23, Noah Sheppard  wrote:

> Perhaps there is some character in manual.txt which is causing
> truncation somewhere, perhaps server-side, or perhaps in wget, less, and
> firefox (for the uncompressed manual.txt).

I'm pretty sure it's not in wget, since you can use wget to
download files containing arbitrary binary data (and I've
probably used it to download gigabytes of stuff over the years
without any issues).

-- 
Grant Edwards   grante Yow! I'm dressing up in
  at   an ill-fitting IVY-LEAGUE
   visi.comSUIT!!  Too late...

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Noah Sheppard]

On Wed, Sep 23, 2009 at 02:44:40PM -0400, Noah Sheppard wrote:
> [ ... ] 
> Perhaps there is some character in manual.txt which is causing
> truncation somewhere, perhaps server-side, or perhaps in wget, less, and
> firefox (for the uncompressed manual.txt).  Maybe some library on our
> systems common to all those which does not like some special character?
> Just a guess.

There is a (C) (in a single character) right about where the truncation
happens.

-- 
Noah Sheppard
Assistant Computer Resource Manager
Taylor University CSE Department
nshep...@cse.taylor.edu

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Noah Sheppard]

On Wed, Sep 23, 2009 at 02:37:21PM -0400, James Michael Fultz wrote:
> [ ... ] 
> > When I download manual.txt.gz with wget and then unzip it and
> > vim the result, I again see alternating valid character and
> > <##> (I have "set display+=uhex in my .vimrc).  This may very
> > well be a configuration problem on my side, that I don't see
> > non-utf8 files correctly.
> 
> Try less and 'col -b manual.txt | less' if the former doesn't
> display cleanly.  The col command will strip embedded backspace
> sequences.

That worked, thanks for the tip (although col complained "Invalid or
incomplete multibyte or wide character").  Also, when I zcat the
downloaded gz, and run it through col, I get truncated at exactly the
same point (right after "Mutt is Copyright").  If, however, I zcat the
file directly to less without col, everything displays correctly, with
the bolding, and in its entirety, all the way to the end of chapter 10.

Perhaps there is some character in manual.txt which is causing
truncation somewhere, perhaps server-side, or perhaps in wget, less, and
firefox (for the uncompressed manual.txt).  Maybe some library on our
systems common to all those which does not like some special character?
Just a guess.

-- 
Noah Sheppard
Assistant Computer Resource Manager
Taylor University CSE Department
nshep...@cse.taylor.edu

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[James Michael Fultz]

* Noah Sheppard  [2009-09-23 12:36]:
> On Wed, Sep 23, 2009 at 06:09:38PM +0200, Michael Wagner wrote:
> > * Wu, Yue  23.09.2009
> > > In mutt offical site, the documentation for devel version:
> > > 
> > > text version: http://www.mutt.org/doc/devel/manual.txt
> > > 
> > > is an uncompleted version, which just give a table of
> > > contents and the first chapter.
> > > 
> > > text gzipped version:
> > > http://www.mutt.org/doc/devel/manual.txt.gz
> > > 
> > > isn't a gzipped version actually, I can't gunzip with it.
[ ... ]
> 
> I can confirm the first problem using Firefox 3.0.14 on Gentoo
> Linux, 64-bit. As far as the second, I'm not sure if I'm
> experiencing what Wu is.  I cannot reproduce the "unable to
> gzip" problem, at least.

Firefox 3.0.14 on Ubuntu 9.04 64-bit here.  The uncompressed
manual appears to be to truncated when displayed in Firefox.
Same thing with ELinks 0.12pre2.

When running 'gzip -tv manual.txt.gz', it reports "OK".

When viewing the directory listing on the server at
 with either Firefox or ELinks,
the file size for the uncompressed manual appears to be smaller
than the file size for the compressed manual.  Leads me to think
that the uncompressed manual stored on the server is in fact
truncated.

[ ... ]
> When viewed in firefox or downloaded with wget, manual.txt is
> only the table of contents + first chapter.
> 
> When viewed in firefox, manual.txt.gz displays as a text/plain
> file, but it's apparently got some kind of unicode going on,
> because valid characters alternate with little squares
> containing numbers (utf16 character codes or something similar,
> I presume).  It appears as though Firefox is automatically
> gunzip'ing the file for me.

I think that it's formatted using embedded backspace characters
to simulate bolded text.  Displays as such when downloaded an
viewed with less or when viewed within ELinks.

Not sure whether it is the client (in this case Firefox or
ELinks) or the server performing decompression, but I know that
data can be compressed on-the-fly using gzip server-side.

> When I download manual.txt.gz with wget and then unzip it and
> vim the result, I again see alternating valid character and
> <##> (I have "set display+=uhex in my .vimrc).  This may very
> well be a configuration problem on my side, that I don't see
> non-utf8 files correctly.

Try less and 'col -b manual.txt | less' if the former doesn't
display cleanly.  The col command will strip embedded backspace
sequences.

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Michael Wagner]

* Gary Johnson  23.09.2009
> On 2009-09-23, Michael Wagner wrote:
> > * Wu, Yue  23.09.2009
> > > In mutt offical site, the documentation for devel version:
> > > 
> > > text version: http://www.mutt.org/doc/devel/manual.txt
> > > 
> > > is an uncompleted version, which just give a table of contents and the 
> > > first
> > > chapter.
> > > 
> > > text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
> > > 
> > > isn't a gzipped version actually, I can't gunzip with it.
> > 
> > Hello Wu,
> > 
> > the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
> > file. I've downloaded it for testing and everything is allright with the 
> > it. Also when I visited the side with different text browsers, they 
> > showed me all the correct file.
> > 
> > What browser do you use?
> 
> FWIW, I get the same, truncated results that Wu Yue reported from
> http://www.mutt.org/doc/devel/manual.txt using w3m and Firefox on
> different computers with different routes to the Internet.  The last
> line is
> 
>Mutt is Copyright
>
> I didn't check the .gz file.

Hello Gary,

but I spoke only from the .gz file. The other file is here also only the 
table of contents and the first chapter.

Michael

-- 
"An eye for an eye will only make the entire world blind."
   --Mohandas Karamchand Gandhi


signature.asc
Description: Digital signature

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Noah Sheppard]

On Wed, Sep 23, 2009 at 06:09:38PM +0200, Michael Wagner wrote:
> * Wu, Yue  23.09.2009
> > In mutt offical site, the documentation for devel version:
> > 
> > text version: http://www.mutt.org/doc/devel/manual.txt
> > 
> > is an uncompleted version, which just give a table of contents and the first
> > chapter.
> > 
> > text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
> > 
> > isn't a gzipped version actually, I can't gunzip with it.
> 
> Hello Wu,
> 
> the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
> file. I've downloaded it for testing and everything is allright with the 
> it. Also when I visited the side with different text browsers, they 
> showed me all the correct file.

I can confirm the first problem using Firefox 3.0.14 on Gentoo Linux,
64-bit. As far as the second, I'm not sure if I'm experiencing what Wu
is.  I cannot reproduce the "unable to gzip" problem, at least.

Also, to the best of my knowledge, I have never before accessed
manual.txt, so I do not believe caching is an issue.  A couple
shift+refreshes did not lead me to the entire manual file.

When viewed in firefox or downloaded with wget, manual.txt is only the
table of contents + first chapter.

When viewed in firefox, manual.txt.gz displays as a text/plain file,
but it's apparently got some kind of unicode going on, because valid
characters alternate with little squares containing numbers (utf16
character codes or something similar, I presume).  It appears as though
Firefox is automatically gunzip'ing the file for me.

When I download manual.txt.gz with wget and then unzip it and vim the
result, I again see alternating valid character and <##> (I have "set
display+=uhex in my .vimrc).  This may very well be a configuration
problem on my side, that I don't see non-utf8 files correctly.

-- 
Noah Sheppard
Assistant Computer Resource Manager
Taylor University CSE Department
nshep...@cse.taylor.edu

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Grant Edwards]

On 2009-09-23, Gary Johnson  wrote:
> On 2009-09-23, Michael Wagner wrote:
>> * Wu, Yue  23.09.2009
>> > In mutt offical site, the documentation for devel version:
>> > 
>> > text version: http://www.mutt.org/doc/devel/manual.txt
>> > 
>> > is an uncompleted version, which just give a table of contents and the 
>> > first
>> > chapter.
>> > 
>> > text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
>> > 
>> > isn't a gzipped version actually, I can't gunzip with it.
>> 
>> Hello Wu,
>> 
>> the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
>> file. I've downloaded it for testing and everything is allright with the 
>> it. Also when I visited the side with different text browsers, they 
>> showed me all the correct file.
>> 
>> What browser do you use?
>
> FWIW, I get the same, truncated results that Wu Yue reported from
> http://www.mutt.org/doc/devel/manual.txt using w3m and Firefox on
> different computers with different routes to the Internet.  The last
> line is
>
>Mutt is Copyright

Same here.  Nothing after Chapter 1, section 7.

Identical results with Firefox, wget, links, etc.

-- 
Grant Edwards   grante Yow! I feel better about
  at   world problems now!
   visi.com

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Gary Johnson]

On 2009-09-23, Michael Wagner wrote:
> * Wu, Yue  23.09.2009
> > In mutt offical site, the documentation for devel version:
> > 
> > text version: http://www.mutt.org/doc/devel/manual.txt
> > 
> > is an uncompleted version, which just give a table of contents and the first
> > chapter.
> > 
> > text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
> > 
> > isn't a gzipped version actually, I can't gunzip with it.
> 
> Hello Wu,
> 
> the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
> file. I've downloaded it for testing and everything is allright with the 
> it. Also when I visited the side with different text browsers, they 
> showed me all the correct file.
> 
> What browser do you use?

FWIW, I get the same, truncated results that Wu Yue reported from
http://www.mutt.org/doc/devel/manual.txt using w3m and Firefox on
different computers with different routes to the Internet.  The last
line is

   Mutt is Copyright
   
I didn't check the .gz file.

Regards,
Gary

Re: Offical site's uncomplete manual.txt and broken manual.txt.gz

[Michael Wagner]

* Wu, Yue  23.09.2009
> In mutt offical site, the documentation for devel version:
> 
> text version: http://www.mutt.org/doc/devel/manual.txt
> 
> is an uncompleted version, which just give a table of contents and the first
> chapter.
> 
> text gzipped version: http://www.mutt.org/doc/devel/manual.txt.gz
> 
> isn't a gzipped version actually, I can't gunzip with it.

Hello Wu,

the version at http://www.mutt.org/doc/devel/manual.txt.gz is a gzipped 
file. I've downloaded it for testing and everything is allright with the 
it. Also when I visited the side with different text browsers, they 
showed me all the correct file.

What browser do you use?

Hth Michael

-- 
Having more money does not insure happiness. People with ten million
dollars are no happier than people with nine million dollars.
-Hobart Brown


signature.asc
Description: Digital signature