Re: mhl nocomponent

[Philipp]

[2023-04-02 14:04] Michael Richardson 
> I use MH-E, which does it's header display/hiding outside of MH.
> In general, I like to see extra headers, but they have gotten way out of hand
> from the MS/Outlook space.
> We probably need a better list of common "this is junk" header list that 
> probably
> has to have wildcards in it.

I have a patch for simple globbing in ignores. If you are intrested,
I can port it.

Philipp

Re: mhl nocomponent

[David Levine]

Ken wrote:

> I am thinking maybe we should change the nmh default mhl files as well
> to recognize current reality

+1 to removing "extras:nocomponent".

David

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>  | Given my druthers I think I'd rather do the last one, since this kind
>  | of seems like a table!
>
>I would do it that way (now) too, either that way, or just use mdoc
>primitives - an appropriate layout could probably be achieved using
>the list macros (with tags) in compact mode.

Fair enough!  I'll take a look and see if I can convert those to either
a mdoc primitive or something using tbl.

Are we all in agreement that (when possible) we should change man pages
over to mdoc and new man pages should be written in mdoc?

--Ken

Re: Unsupported nroff macros on MacOS X

[Ralph Corderoy]

Hi Ken,

> This manpage is not compatible with mandoc(1) and might display
> incorrectly.

mandoc is a pain.  It's one of many programs which attempt to interpret
man pages whilst being an incomplete implementation.  I hang out in
places which like to talk about troff/nroff, including for man pages,
and mandoc's flaws crop up a lot.

> After some digging, it turns out man(1) is a shell script and to make
> a long story short is running this command:
>
>   mandoc -Tlint -Wunsupp

I'd suggest disabling that if it's possible.

> And some quick googling suggests that in fact the mandoc macros

I understand it's a program which tries to interpret files intended for
a troff/nroff rather than a set of macros.

> .fc aint one of them.

Ignore mandoc.  :-)

> I will admit that my roff-fu is not very good, but I took a look at
> this.  It seems this is a common idiom for nmh man pages.
> Specifically (this is from packf(1) but it's similar everywhere else):
>
> .SH "PROFILE COMPONENTS"
> .fc ^ ~
> .nf
> .ta 2.4i
> .ta \w'ExtraBigProfileName  'u
> ^Path:~^To determine the user's nmh directory
> ^Current\-Folder:~^To find the default current folder
> ^Msg\-Protect:~^To set mode when creating a new `file'
> .fi
>
> So as I vaguely understand it, the .fc line sets '^ as a field
> delimeter and '~' as the character to pad a field.

Yes, i.e. ‘~’ is where to share out the stretchable space within the
field.

> .nf sets the following lines to no-fill mode.

Yes, so don't join input lines together to form output lines.

> ".ta 2.4i" sets the tab stop to 2.4 inches.

Yes, just a single tab stop.

$ printf '%s\n' '.ta 2.4i' $'a\tb\tc\td' '.pl \n(nlu' | nroff
a   bcd
$

> The following line sets the tab stop to the width of
> "ExtraBigProfileName" and 'u is the default horizontal span for the
> terminal

Not quite.  The single tab stop is set to the width of the string
‘ExtraBigProfileName  ’ including a couple of spaces.  \w'' gives the
width of the string in ‘basic units’.  The ‘ta’ request assumes numbers
are in ems so the ‘u’ overrides this to show the number is in basic
units.

> (now that I look at it, I'm not sure why there are two .ta lines right
> after another).

Agreed.

> This isn't wonderful and I'd like to fix it, but I'm not sure what
> to do.  Ideas include:
>
> - Tell MacOS X users to install groff(1) (the man(1) script will try
>   to call groff if it encounters a warning like this from mandoc)

This will fix things for macOS users viewing non-nmh too.  +1

> - Switch to some other roff construct; I was under the impression that
>   actual tabs would work?  I'm not sure why tabs aren't used here.

Perhaps a carry over from more complex layouts in other pages which use
the same valid technique.

> - Switch to tbl(1) macros which as far as I can tell are supported by
>   mandoc and seem to work everywhere.

tbl(1) is a preprocessor which reads the man-page source, leaves most
well alone, and turns what's in between lines of .TS to .TE into roff
for processing by troff/nroff.  Other programs which read man pages do
try to emulate it as part of processing the file, typically with the
same gotchas and noise as they trigger with the rest of the file.

man pages are defined as what troff/nroff handle with the man(7) set of
macros.  It's a complex language.  Pretenders start on a sticky wicket.

The .TS and .TE are copied through allowing a macro set, e.g. man, to
define them if it wants.  This allows further processing of tbl's output
which now lies between them.

A man page which uses tbl should indicate this by the first line being

'\" t

so man knows to use tbl(1) as well as troff/nroff.

> Given my druthers I think I'd rather do the last one, since this kind
> of seems like a table!

It's not the least work.  You may find you have to then muck around with
tbl's commands a bit to get a nice table for ASCII and UTF-8 on the TTY
and PostScript/PDF for the page or graphical viewers.

Others have mentioned the mdoc macros.  These are an alternative to man.
Their big selling point is semantic mark-up rather than presentation.
But they have their own problems, they're YA-standard, and I'd avoid
them.  Stick with simple man macros and troff/nroff with ASCII, UTF-8,
and PostScript/PDF as targets.

-- 
Cheers, Ralph.

Re: Unsupported nroff macros on MacOS X

[Ralph Corderoy]

Hi Ken,

> Are we all in agreement that (when possible) we should change man
> pages over to mdoc and new man pages should be written in mdoc?

No, see my other email which crossed.

-- 
Cheers, Ralph.

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>mandoc is a pain.  It's one of many programs which attempt to interpret
>man pages whilst being an incomplete implementation.  I hang out in
>places which like to talk about troff/nroff, including for man pages,
>and mandoc's flaws crop up a lot.

So  I'll admit my ignorance here.  What's the difference between
mandoc and mdoc?  It seems like mandoc is just the program that
interprets nroff source?  A lesser implementation, as you say.

>Others have mentioned the mdoc macros.  These are an alternative to man.
>Their big selling point is semantic mark-up rather than presentation.
>But they have their own problems, they're YA-standard, and I'd avoid
>them.  Stick with simple man macros and troff/nroff with ASCII, UTF-8,
>and PostScript/PDF as targets.

Sigh.  Well, I'll admit my biases here.  In my youth somehow I missed
the troff boat; we used a lot of SunOS 4 and that didn't seem to have a
complete troff implementation (nor any documentation), and everyone a
bit older than me had System V experience which did come with a complete
troff manual and somehow it seemed like everybody else had figured out
troff, but other than modifying the occasional man page (which was mostly
done by copying other man pages) I never did really grok troff.

I realize that this is all solvable and the original troff manuals are
available online.  But ... well, I'm busy and all that, and I kind of
view it like learning Morse code; yes, it would be cool, but I just
don't have the time.

I did a quick Google, and I found this answer which is similar to
my question:

https://unix.stackexchange.com/questions/391399/what-is-the-difference-between-the-mdoc-macro-set-for-troff-and-the-plain-man-ma

Now obviously that person has their own biases, but this paragraph is
very telling to me:

  Unless you're an experienced manpage author with a sound
  understanding of Roff grammar and the pipeline's mechanics, you
  really shouldn't be using anything other than mdoc for authoring your
  manual-pages. Veteran troff users may find mdoc to be needlessly
  verbose or restrictive, finding man to be lighter and less
  intrusive. However, these authors are experienced enough to know damn
  well what they're doing ― so unless you're a grizzled veteran, just
  stick to using mdoc.

And that describes me exactly!  The frustrating thing for _me_ is that
doing simple stuff (like formatting command options) in man pages seems
kinda complicated and mdoc has macros which deal with that exactly.  Like
it does all of the stuff I'd ever want to do in man pages.  And it seems
like it is everywhere now?  Is there a reason to NOT use it?

I am open to more input on this topic.  My specific question is: should
we replace the .fc macros in nmh man pages with mdoc macros?  It seems like
that should Just Work on all common platforms today.

--Ken

Re: Unsupported nroff macros on MacOS X

[Ralph Corderoy]

Hi Ken,

I don't have time to reply fully now.

> My specific question is: should we replace the .fc macros in nmh man
> pages with mdoc macros?

The question is flawed.

You could replace .fc plus *all* the man macros with all-mdoc macros in
all the nmh man pages.  It's man ^ mdoc == 1.

But that would be a daft waste of time.

-- 
Cheers, Ralph.

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>You could replace .fc plus *all* the man macros with all-mdoc macros in
>all the nmh man pages.  It's man ^ mdoc == 1.

Ah, poop.  I see what you mean; you can't mix and match macros across
packages.  Dang it.

--Ken

Re: Unsupported nroff macros on MacOS X

[Ralph Corderoy]

Hi Ken,

> I see what you mean; you can't mix and match macros across packages.

...across macro sets like man and mdoc.  No.

The status quo is fine.  It doesn't require understanding all of troff.
Just man(7) plus the odd bit here and there.  It's sufficed for nmh for
ages.  Any occasional issue can be discussed on the list.  man(7) still
serves well for many other mainstream packages with copious man pages.

Silencing the .fc warning has easy fixes.

- Require groff.  A win all round.
- Replace .fc with plain tabs if that gives suitable results for all .fc
  uses.
- Experiment with the .HP, .IP, or .TP man macros to see if they're
  suitable.
- Introduce tbl(1).

  $ tbl | nroff
  .TS
  tab(,);
  l l.
  foo,bar
  a longer foo,and another bar
  .TE
  .pl \n(plu
  foobar
  a longer foo   and another bar
  ^D
  $

-- 
Cheers, Ralph.

Re: Unsupported nroff macros on MacOS X

[Anthony J. Bentley]

Ralph Corderoy writes:
> Silencing the .fc warning has easy fixes.
>
> - Require groff.  A win all round.

I don't agree. The ideal would be for mandoc to gain support for .fc
(listed in its TODO as a desired but low-importance [read: rarely used]
manpage feature), but despite its imperfect compatibility here, mandoc
is a widely used program, the default manpage formatter in most BSDs,
some Linux distributions, popular web frontends like manpages.debian.org
and man.archlinux.org, and apparently now macOS. It is worth catering for.

Considering equivalent results can be achieved with a straightforward
conversion to tbl, a widely supported and appropriately venerable tool,
I see no reason to avoid doing so.

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>The status quo is fine.  It doesn't require understanding all of troff.
>Just man(7) plus the odd bit here and there.

Sigh.  The "odd bit" unfortunately, for me, requires a lot of knowledge
that seems to take some serious roff-fu.

Let's take the example you gave where the first line for a man
page that uses tbl should contain:

'\" t

So, my question is ... what does this mean?  I understand that \" is
a comment, but I'm confused about the leading single quote.  As a random
note, this string is rather hard to search for.  Also, I don't really
understand how single quotes are used in roff, I guess; all of the
documentation seems to assume you already know this.  And it's this
way for me for EVERY SINGLE BIT of "odd bit"; there's a huge pile of
knowledge that is assumed you know.  It's not that I'm afraid of digging
into hours-long rat holes; I write programs using OpenSSL, after all.
It's just a lot to deal with when I am just trying to format a nice
table.  And I do it so infrequently that I have to re-learn it all
every time I want to edit a man page.

(I did play around with tbl, and it seems like that is actually very
easy so I am thinking that Anthony is is right and we should just switch
to that).

--Ken

Re: Unsupported nroff macros on MacOS X

[Robert Elz]

Date:Mon, 03 Apr 2023 17:58:44 +0100
From:Ralph Corderoy 
Message-ID:  <20230403165844.e9a5f21...@orac.inputplus.co.uk>

  | - Require groff.  A win all round.

If you mean "require a *roff compat text processor" then OK, but if
you really mean "groff" then no, never.   MH (and nmh) is a nice free,
available to everyone, mail system.   Removing that by requiring anything
encumbered by any kind of commercial licence (including the GPL) would be
a huge step into the darkness.

Certainly converting to mdoc is not a trivial task, though once done it
tends to produce better results than man for many situations (better HTML
versions for example - man likes to paginate, always).   I used to be
(kind of) in your camp, and resist mandoc because it couldn't do all the
small fiddles that are possible using *roff, but I have changed my mind.
There are still some things I wish mdoc could do, which it cannot (and
there seems to be no interest in upgrading it - the argument being that
it would take forever to get a version with a new feature installed
everywhere, so no man pages can actually use the new thing, so there's
no point adding it if no-one can use it).   But overall, if a new man
page is being written, or an old one is being completely overhauled,
using mdoc (processing with either mandoc or *roff -mdoc) tends to get
what you want done, and looking good in all of text (for grep or similar)
terminal (for viewing using "man") pdf (for printing, or whatever else)
and html (you know what for - and it is not e-mail).

Actually taking a large set of man pages written for -man and converting
them to -mdoc just so they are mdoc rather than man however is almost
certainly a waste of time, unless someone with mdoc knowledge has a large
amount of available time, and nothing productive do with it, and also
doesn't mind a large amount of (often) fairly mindless grunt work.

kre

Re: Unsupported nroff macros on MacOS X

[doug dougwellington . com]

Sorry if I jumped into the middle and missed something, but what about using 
this to convert once?


groff -Thtml


From: nmh-workers-bounces+doug=dougwellington@nongnu.org 
 on behalf of Ken 
Hornstein 
Sent: Monday, April 3, 2023 11:40 AM
To: nmh-workers@nongnu.org 
Subject: Re: Unsupported nroff macros on MacOS X

>The status quo is fine.  It doesn't require understanding all of troff.
>Just man(7) plus the odd bit here and there.

Sigh.  The "odd bit" unfortunately, for me, requires a lot of knowledge
that seems to take some serious roff-fu.

Let's take the example you gave where the first line for a man
page that uses tbl should contain:

'\" t

So, my question is ... what does this mean?  I understand that \" is
a comment, but I'm confused about the leading single quote.  As a random
note, this string is rather hard to search for.  Also, I don't really
understand how single quotes are used in roff, I guess; all of the
documentation seems to assume you already know this.  And it's this
way for me for EVERY SINGLE BIT of "odd bit"; there's a huge pile of
knowledge that is assumed you know.  It's not that I'm afraid of digging
into hours-long rat holes; I write programs using OpenSSL, after all.
It's just a lot to deal with when I am just trying to format a nice
table.  And I do it so infrequently that I have to re-learn it all
every time I want to edit a man page.

(I did play around with tbl, and it seems like that is actually very
easy so I am thinking that Anthony is is right and we should just switch
to that).

--Ken

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>Sorry if I jumped into the middle and missed something, but what about
>using this to convert once?
>
>groff -Thtml

I guess my next question is ... what do we do after that?

I am assuming that we still want to ship man pages; do we use some tool
to convert them back?  Do we have to make man page modifications to HTML?
I am kinda against depending on some third-party tool; I'm not sure what
the options are in this space.  I was under the impression that most things
go from man pages TO HTML, not the other way around, or they use some
other markup format that can output man pages and HTML.

--Ken

Re: Unsupported nroff macros on MacOS X

[Robert Elz]

Date:Mon, 03 Apr 2023 11:35:35 -0400
From:Ken Hornstein 
Message-ID:  <20230403153539.de26b1ef...@pb-smtp21.pobox.com>

  | What's the difference between mandoc and mdoc?
  | It seems like mandoc is just the program that interprets nroff source?
  | A lesser implementation, as you say.

A restricted implementation, that interprets just the parts of general *roff
that (is believed) man page authors ought be including in man page sources.

mdoc is a *roff macro package which implements a set of macros useful for
making documentation (in particular, manual pages).  A replacement for man(7).

  | I never did really grok troff.

Pity - it can be lots of fun (and like many things highly general and
very powerful, also quite frustrating at times).

  | And that describes me exactly!  The frustrating thing for _me_ is that
  | doing simple stuff (like formatting command options) in man pages seems
  | kinda complicated and mdoc has macros which deal with that exactly.  Like
  | it does all of the stuff I'd ever want to do in man pages.

Yes, that is its point.

  | And it seems like it is everywhere now?

Just about, yes.

  | Is there a reason to NOT use it?

For writing something new, none at all.   But for converting something
someone has already made work, and is full of low level *roff noise to
achieve peculiar effects (that you first have to understand the point of,
then decide if you really need to keep, and then work out how do achieve
with mdoc) then, yes, lots of reasons...

  | My specific question is: should
  | we replace the .fc macros in nmh man pages with mdoc macros?

As I believe you're now aware, you cannot just do that.   It is all or
nothing (as with almost any other *roff macro package) - you cannot take
pieces from one and simply use them with another (though a macro package
can be built as an extension to another macro set - when I was teaching
(a horrible time for the students involved) I had a set of exam paper
macros, which were built on top of ms to produce (several types of) exam
papers - with the ability to include the answers in line, so those are
available for pre-exam review and then for the markers, without being
printed for the students...

If you create something new, use mdoc, it is far easier (particularly once
you get used to its "parsable" and "callable" macro conventions).  What's
more, when you use -mdoc (or mandoc) the macros are smart enough to detect
whether the man page is really -mdoc or is actually -man and use the 
appropriate set of macros for that particular page, so no-one needs to
try to work out whether they should be using -mdoc or -man on the command
line, just use -mdoc always.

kre

Summary regarding NUL in email messages

[Ken Hornstein]

I meant to send this out earlier, but I wanted to summarize what I think
are the prevailing attitudes about NUL characters in email messages
that I asked about in February:

- It seems like NULs are just not that common in the real world, so no
  urgent changes are required
- We should probably try to do better when code is updated/modernized.

Any dissenters from this view?

--Ken

Re: Unsupported nroff macros on MacOS X

[doug dougwellington . com]

Ken wrote:
> I am assuming that we still want to ship man pages

To keep with the original intent of the project, I suspect that is the thing to 
do.

> I am kinda against depending on some third-party tool

Where does built-in turn into third-party?  With all the modern package 
managers, it's trivial to install other tools as needed.

> I'm not sure what the options are in this space [snip] some
> other markup format that can output man pages and HTML.

Having been in the python world for so long, I had to deal with markdown, 
restructured text, docutils, sphinx, etc.  Then there's the beast that is 
Doxygen.  They all have benefits and tradeoffs.  How hard do we hold onto the 
old ways vs. embracing new tools?

From: nmh-workers-bounces+doug=dougwellington@nongnu.org 
 on behalf of Robert 
Elz 
Sent: Monday, April 3, 2023 12:13 PM
To: Ken Hornstein 
Cc: nmh-workers@nongnu.org 
Subject: Re: Unsupported nroff macros on MacOS X

Date:Mon, 03 Apr 2023 11:35:35 -0400
From:Ken Hornstein 
Message-ID:  <20230403153539.de26b1ef...@pb-smtp21.pobox.com>

  | What's the difference between mandoc and mdoc?
  | It seems like mandoc is just the program that interprets nroff source?
  | A lesser implementation, as you say.

A restricted implementation, that interprets just the parts of general *roff
that (is believed) man page authors ought be including in man page sources.

mdoc is a *roff macro package which implements a set of macros useful for
making documentation (in particular, manual pages).  A replacement for man(7).

  | I never did really grok troff.

Pity - it can be lots of fun (and like many things highly general and
very powerful, also quite frustrating at times).

  | And that describes me exactly!  The frustrating thing for _me_ is that
  | doing simple stuff (like formatting command options) in man pages seems
  | kinda complicated and mdoc has macros which deal with that exactly.  Like
  | it does all of the stuff I'd ever want to do in man pages.

Yes, that is its point.

  | And it seems like it is everywhere now?

Just about, yes.

  | Is there a reason to NOT use it?

For writing something new, none at all.   But for converting something
someone has already made work, and is full of low level *roff noise to
achieve peculiar effects (that you first have to understand the point of,
then decide if you really need to keep, and then work out how do achieve
with mdoc) then, yes, lots of reasons...

  | My specific question is: should
  | we replace the .fc macros in nmh man pages with mdoc macros?

As I believe you're now aware, you cannot just do that.   It is all or
nothing (as with almost any other *roff macro package) - you cannot take
pieces from one and simply use them with another (though a macro package
can be built as an extension to another macro set - when I was teaching
(a horrible time for the students involved) I had a set of exam paper
macros, which were built on top of ms to produce (several types of) exam
papers - with the ability to include the answers in line, so those are
available for pre-exam review and then for the markers, without being
printed for the students...

If you create something new, use mdoc, it is far easier (particularly once
you get used to its "parsable" and "callable" macro conventions).  What's
more, when you use -mdoc (or mandoc) the macros are smart enough to detect
whether the man page is really -mdoc or is actually -man and use the
appropriate set of macros for that particular page, so no-one needs to
try to work out whether they should be using -mdoc or -man on the command
line, just use -mdoc always.

kre

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>> I am kinda against depending on some third-party tool
>
>Where does built-in turn into third-party?  With all the modern package
>managers, it's trivial to install other tools as needed.

That's a fair question!  And one I struggle with.  One thing that is
common is we do make a distinction between what DEVELOPERS are required
to have versus what end-users are required to have.  E.g.: you need
autoconf/automake/flex to build it from the git repo, but you do not
need any of that to build from a distribution tar file.  Something
similar could be done with man pages.  Nmh is a "traditional" tool
and we generally have the goal of sticking to "traditional" support
tools, but I acknowledge there is no good definition here and it's
kind of arbitrary what is traditional and what isn't.

In a more practical sense, I am not sure there is anyone with the free
cycles to convert the current man pages into some other markup language.
Yes we could convert the man pages to HTML, but I am not sure there's
a way to convert back.  If I am wrong please let me know.

--Ken

Re: Unsupported nroff macros on MacOS X

[Steven Winikoff]

>In a more practical sense, I am not sure there is anyone with the free
>cycles to convert the current man pages into some other markup language.

This seems like the sort of thing that should be possible to automate, and
that question has been raised before.  A quick search turned up the
following, among others:

   
https://stackoverflow.com/questions/13433903/convert-all-linux-man-pages-to-text-html-or-markdown
   https://jeromebelleman.gitlab.io/posts/publishing/manpages/

 - Steven
-- 
___
Steven Winikoff  | "Science is built upon facts, as a house is
Montreal, QC, Canada |  built of stones; but an accumulation of
s...@smwonline.ca |  facts is no more a science than a heap of
http://smwonline.ca  |  stones is a house."
 |   - Henri Poincaré

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>>In a more practical sense, I am not sure there is anyone with the free
>>cycles to convert the current man pages into some other markup language.
>
>This seems like the sort of thing that should be possible to automate, and
>that question has been raised before.  A quick search turned up the
>following, among others:
>
>   
> https://stackoverflow.com/questions/13433903/convert-all-linux-man-pages-to-text-html-or-markdown
>   https://jeromebelleman.gitlab.io/posts/publishing/manpages/

I am ... concerned about depending on pandoc, because of this:

  Pandoc is available in lxplus, aiadm and most RPM repositories. It's
  written in Haskell, which means it relies on hundreds of megabytes of
  library dependencies.

That screams to me, "this is going to bitrot at some point in the
future".  There seems to be a wealth of man/mdoc/roff converters and I
feel like there are always going to be some kind of nroff converter, at
least until Y2038..  I have no objection to Markdown but I'm not sure
what it would gain us exactly, other than maybe someone younger than 35
could edit the documentation.

--Ken

Re: Unsupported nroff macros on MacOS X

[Steven Winikoff]

>I am ... concerned about depending on pandoc, because of this:
>
>  Pandoc is available in lxplus, aiadm and most RPM repositories. It's
>  written in Haskell, which means it relies on hundreds of megabytes of
>  library dependencies.

That's certainly fair, but wouldn't it need to be used only once, after
which the documentation could be maintained in markdown format?  I suppose
that would require a tool to go from markdown to man, but at least it's a
thought.


>I have no objection to Markdown but I'm not sure what it would gain us
>exactly, other than maybe someone younger than 35 could edit the
>documentation.

That may be the point -- or not, I suppose, depending on one's point of
view.  (I'm far past the point of being under 35 myself, for what that's
worth.)

 - Steven
-- 
___
Steven Winikoff  |
Montreal, QC, Canada | "The cure for boredom is curiousity.
s...@smwonline.ca |  There is no cure for curiousity."
http://smwonline.ca  |
 |  - Dorothy Parker

Re: Unsupported nroff macros on MacOS X

[doug dougwellington . com]

>In a more practical sense, I am not sure there is anyone with the free
>cycles to convert the current man pages into some other markup language.

Interesting that you mention that.  I retired last year.  I haven't written a 
line of code in over a year.  Crazy!

Things got out of hand with email for me, I have accounts with Microsoft, 
Apple, Google, and three domain providers.  I'm hoping to figure out some way 
to mirror everything to my local linux box and use nmh and other unix tools 
again.  Once I'm happy with the mirror, I figure I'll delete everything online 
and start pulling everything to my local box and only deal with it there.

Anway, I'm sure I'll have a few cycles that I can devote to documentation...

From: Steven Winikoff 
Sent: Monday, April 3, 2023 1:18 PM
To: Ken Hornstein 
Cc: nmh-workers@nongnu.org ; doug dougwellington.com 

Subject: Re: Unsupported nroff macros on MacOS X

>In a more practical sense, I am not sure there is anyone with the free
>cycles to convert the current man pages into some other markup language.

This seems like the sort of thing that should be possible to automate, and
that question has been raised before.  A quick search turned up the
following, among others:

   
https://stackoverflow.com/questions/13433903/convert-all-linux-man-pages-to-text-html-or-markdown
   https://jeromebelleman.gitlab.io/posts/publishing/manpages/

 - Steven
--
___
Steven Winikoff  | "Science is built upon facts, as a house is
Montreal, QC, Canada |  built of stones; but an accumulation of
s...@smwonline.ca |  facts is no more a science than a heap of
http://smwonline.ca  |  stones is a house."
 |   - Henri Poincaré

Re: Unsupported nroff macros on MacOS X

[doug dougwellington . com]

> That screams to me, "this is going to bitrot at some point in the future".

Anybody else remember the NPM debacle?

How One Dev Broke Node and Thousands of Projects In 11 Lines of JavaScript - 
Slashdot
[https://a.fsdn.com/sd/topics/programming_64.png]
How One Dev Broke Node and Thousands of Projects In 11 Lines of JavaScript - 
Slashdot
An anonymous reader quotes an article written by Chris Williams for The 
Register: Programmers were left staring at broken builds and failed 
installations on Tuesday after someone toppled the Jenga tower of JavaScript. A 
couple of hours ago, Azer Koculu unpublished more than 250 of his modules from 
N...
developers.slashdot.org

> I have no objection to Markdown but I'm not sure what it would gain us 
> exactly,
> other than maybe someone younger than 35 could edit the documentation.

LOL.  OK. I suppose I should comb the archives, but how many of us are actually 
using nmh these days?  I mean, I don't REALLY want to know whether I'm as much 
of a dinosaur as I fear, but who are we maintaining nmh for?

From: nmh-workers-bounces+doug=dougwellington@nongnu.org 
 on behalf of Ken 
Hornstein 
Sent: Monday, April 3, 2023 1:30 PM
To: nmh-workers@nongnu.org 
Subject: Re: Unsupported nroff macros on MacOS X

>>In a more practical sense, I am not sure there is anyone with the free
>>cycles to convert the current man pages into some other markup language.
>
>This seems like the sort of thing that should be possible to automate, and
>that question has been raised before.  A quick search turned up the
>following, among others:
>
>   
> https://stackoverflow.com/questions/13433903/convert-all-linux-man-pages-to-text-html-or-markdown
>   https://jeromebelleman.gitlab.io/posts/publishing/manpages/

I am ... concerned about depending on pandoc, because of this:

  Pandoc is available in lxplus, aiadm and most RPM repositories. It's
  written in Haskell, which means it relies on hundreds of megabytes of
  library dependencies.

That screams to me, "this is going to bitrot at some point in the
future".  There seems to be a wealth of man/mdoc/roff converters and I
feel like there are always going to be some kind of nroff converter, at
least until Y2038..  I have no objection to Markdown but I'm not sure
what it would gain us exactly, other than maybe someone younger than 35
could edit the documentation.

--Ken

Re: Unsupported nroff macros on MacOS X

[George Michaelson]

On Tue, 4 Apr 2023, 06:48 doug dougwellington.com, 
wrote:

> > , but who are we maintaining nmh for?
>

Us.

>

Re: Unsupported nroff macros on MacOS X

[doug dougwellington . com]

> Us.

Of course.  🙂  There are 26 members on Savannah.  Is that all of us that are 
left?

From: George Michaelson 
Sent: Monday, April 3, 2023 2:01 PM
To: doug dougwellington.com 
Cc: nmh-workers@nongnu.org 
Subject: Re: Unsupported nroff macros on MacOS X

On Tue, 4 Apr 2023, 06:48 doug dougwellington.com, 
mailto:d...@dougwellington.com>> wrote:
> , but who are we maintaining nmh for?

Us.

Re: Unsupported nroff macros on MacOS X

[George Michaelson]

On Tue, 4 Apr 2023, 07:10 doug dougwellington.com, 
wrote:

> > Us.
>
> Of course.  🙂  There are 26 members on Savannah.  Is that all of us that
> are left?
>

I wouldn't personally put it much more than 10x that number. You could
argue 100x but it would be insane to argue 1000x.

mhonarc has significant usage.  Does that count?

G

>

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>>  Pandoc is available in lxplus, aiadm and most RPM repositories. It's
>>  written in Haskell, which means it relies on hundreds of megabytes of
>>  library dependencies.
>
>That's certainly fair, but wouldn't it need to be used only once, after
>which the documentation could be maintained in markdown format?  I suppose
>that would require a tool to go from markdown to man, but at least it's a
>thought.

Well, to me the PRIMARY way that nmh users interact with the
documentation is via man pages, viewed with man(1).  So being able
to generate that format is crucial.  If the point of a conversion to
Markdown (or anything else) is just so my dumb ass doesn't have to learn
random troff requests ... well, that seems like overkill.  If it's so
other people don't have to learn random troff requests, that's fair ...
but if we could solve this just by switching to tbl for tables then that
seems more sensible.  My dumb ass was able to figure out how to convert
those .fc requests to tbl tables in a short amount of time so maybe that
makes the most sense.

>>I have no objection to Markdown but I'm not sure what it would gain us
>>exactly, other than maybe someone younger than 35 could edit the
>>documentation.
>
>That may be the point -- or not, I suppose, depending on one's point of
>view.  (I'm far past the point of being under 35 myself, for what that's
>worth.)

I mean, yeah, I hear you.  Same for me!  I am just wondering out loud
if any of these up-and-coming young punks are interested in editing
man pages directly.

Doug writes:
>Anway, I'm sure I'll have a few cycles that I can devote to documentation...

We welcome the help!  Really, any way anyone wants to contribute is welcome.

>Anybody else remember the NPM debacle?

Oh yes, that's was a classic one!

>> I have no objection to Markdown but I'm not sure what it would gain
>> us exactly, other than maybe someone younger than 35 could edit the
>> documentation.
>
>LOL.  OK. I suppose I should comb the archives, but how many of us are
>actually using nmh these days?  I mean, I don't REALLY want to know
>whether I'm as much of a dinosaur as I fear, but who are we maintaining
>nmh for?

It's a fair question!  I mean, why does ANYONE maintain a particular
open-source software package if they aren't getting paid for it?

For me, I figure as long as I still use nmh, I'm gonna keep on
developing it.

--Ken

Re: Unsupported nroff macros on MacOS X

[Bakul Shah]

Why not just add a note in man pages affected by the .fc problem
that if the tables are not properly lined up, the user must install
groff (or plan9ports, where you also get troff)?

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>Why not just add a note in man pages affected by the .fc problem
>that if the tables are not properly lined up, the user must install
>groff (or plan9ports, where you also get troff)?

I don't necessarily object to this, but ... well, that troff request is
weird.  Like I'm still not quite sure what is going on there; I don't
understand why you need that fill space.  Also you manually need to
specify the maximum cell size when you make those tables by hand.

tbl (which seems like it has been supported ... forever?) does the hard
work of creating tables for you.  It seems like the right tool for the
job; even I could figure it out.  As Anthony has pointed out mandoc
is the default man page formatter for BSD systems, now MacOS X, and
some Linux distributions.  Seems to me that switching to tbl for those
things is a no-brainer unless I am missing something, which is always
possible!

--Ken

Re: Unsupported nroff macros on MacOS X

[Michael Richardson]

Ken Hornstein  wrote:
>> Sorry if I jumped into the middle and missed something, but what about
>> using this to convert once?
>>
>> groff -Thtml

> I guess my next question is ... what do we do after that?

I thought if we ran it through with man (nroff/groff) to ascii, then we'd get
asciidoc format essentially.  At which point there are tools that deal with
further transformations.




signature.asc
Description: PGP signature

Re: Unsupported nroff macros on MacOS X

[Bakul Shah]

On Apr 3, 2023, at 2:40 PM, Ken Hornstein  wrote:
> 
> tbl (which seems like it has been supported ... forever?) does the hard
> work of creating tables for you.  It seems like the right tool for the
> job; even I could figure it out.  As Anthony has pointed out mandoc
> is the default man page formatter for BSD systems, now MacOS X, and
> some Linux distributions.  Seems to me that switching to tbl for those
> things is a no-brainer unless I am missing something, which is always
> possible!

I don't see a tbl command on MacOS (or freebsd, except if you
installed groff (or plan9port -- ignore the troff comment!).

Unless you mean replacing the current setup with the output of
equivalent tbl syntax and then checking that in? I tried that.
mandoc doesn't understand tbl output for the simple test i tried.

/usr/local/bin/tbl <

Re: Unsupported nroff macros on MacOS X

[Robert Elz]

Date:Mon, 3 Apr 2023 15:10:15 -0700
From:Bakul Shah 
Message-ID:  

  | mandoc doesn't understand tbl output for the simple test i tried.

No, in general mandoc doesn't process *roff input at all.   Its input
syntax is similar, and it implements a few *roff commands, but in general
it simply processes mdoc (or man) macros directly - as in it knows what
to do with a TH or TS macro call (and all the others - including the few
primitive *roff forms it has been taught - like \*X and .br), but cannot
possibly cope with the the actual *roff low level code that the man macros
use to implenent those, and which the *roff formatters actually process.

  | But groff's (& plan9port's) nroff deals with this fine.

They actually implement *roff - mandoc is smaller, and faster, because it
doesn't.

kre

ps: Ken, while I am here, forget about translating HTML back into anything
rational - that's akin to translating assembler code back into C.   You can
do it, generally manually, if you really have to, but it isn't something to
ever deliberately plan to need doing.

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>I don't see a tbl command on MacOS (or freebsd, except if you
>installed groff (or plan9port -- ignore the troff comment!).

At least on MacOS X, 'man tbl' actually works (but there is not a separate
tbl command, true).  The man page says:

 The tbl language formats tables.  It is used within mdoc(7) and man(7)
 pages.  This manual describes the subset of the tbl language accepted by
 the mandoc(1) utility.

>Unless you mean replacing the current setup with the output of
>equivalent tbl syntax and then checking that in? I tried that.
>mandoc doesn't understand tbl output for the simple test i tried.
>
>/usr/local/bin/tbl <.TS
>c c c .
>This is   centered
>Well, this also
>.TE
>EOF

It looks like to me that tbl format is handled directly by mandoc.  E.g.,
this works:

mandoc -man <

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>>> Sorry if I jumped into the middle and missed something, but what about
>>> using this to convert once?
>>>
>>> groff -Thtml
>
>> I guess my next question is ... what do we do after that?
>
>I thought if we ran it through with man (nroff/groff) to ascii, then we'd get
>asciidoc format essentially.  At which point there are tools that deal with
>further transformations.

I ... am not sure that is correct?  The man page examples I see suggest
it is closer to something like Markdown.  E.g.:

https://docs.asciidoctor.org/asciidoctor/latest/manpage-backend/

--Ken

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Bakul Shah wrote in
 :
 |On Apr 3, 2023, at 2:40 PM, Ken Hornstein  wrote:
 |> 
 |> tbl (which seems like it has been supported ... forever?) does the hard
 |> work of creating tables for you.  It seems like the right tool for the
 |> job; even I could figure it out.  As Anthony has pointed out mandoc
 |> is the default man page formatter for BSD systems, now MacOS X, and
 |> some Linux distributions.  Seems to me that switching to tbl for those
 |> things is a no-brainer unless I am missing something, which is always
 |> possible!
 |
 |I don't see a tbl command on MacOS (or freebsd, except if you
 |installed groff (or plan9port -- ignore the troff comment!).

mandoc has primitivized versions built-in.

 |Unless you mean replacing the current setup with the output of
 |equivalent tbl syntax and then checking that in? I tried that.
 |mandoc doesn't understand tbl output for the simple test i tried.
 |
 |/usr/local/bin/tbl <

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Robert Elz wrote in
 <8706.1680561...@jacaranda.noi.kre.to>:
 |Date:Mon, 3 Apr 2023 15:10:15 -0700
 |From:Bakul Shah 
 |Message-ID:  
 |
 || mandoc doesn't understand tbl output for the simple test i tried.
 |
 |No, in general mandoc doesn't process *roff input at all.   Its input
 ...
 |ps: Ken, while I am here, forget about translating HTML back into anything
 ...

Its HTML conversion is much superior compared to the groff one.
This is hard to understand actually, i think this is a Brit
Professor who did it, and i have read the presentation, and it was
just brilliant.  The realization however soiled the entire
codebase (this is my personal opinion .. of someone who is neither
professor nor have some really good "hammers" out there, to earn
the merits to say something like this, that is to say).

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20230403223814.692aa1f2...@pb-smtp21.pobox.com>:
 |>>> Sorry if I jumped into the middle and missed something, but what about
 |>>> using this to convert once?
 |>>>
 |>>> groff -Thtml
 |>
 |>> I guess my next question is ... what do we do after that?
 |>
 |>I thought if we ran it through with man (nroff/groff) to ascii, then \
 |>we'd get
 |>asciidoc format essentially.  At which point there are tools that \
 |>deal with
 |>further transformations.
 |
 |I ... am not sure that is correct?  The man page examples I see suggest
 |it is closer to something like Markdown.  E.g.:
 |
 |https://docs.asciidoctor.org/asciidoctor/latest/manpage-backend/

Mind you there is this terrible trend in the OSS scene to not even
include manuals no more.  I am *so* delighted that the BTRFS guys
do again for their "progs" (and i could exaggerate a bit that it
possibly has something to do with me asking for this in IRC).
But they all convert to other formats and then leave it to
packagers to convert that to manuals .. or not even that.  Others
do so themselves at least, and provide the balls (git for
example).

This leads to cynical situations.  For example "ipcalc" of RedHat:

  0c09c6c78920290aaa1c3b91c596834080fbf0b0
  2020-06-18 14:50:15 +0200
  ipcalc.1: converted to markdown

  b2003892e7ddaed77a309882bcba06cea31cda0a
  2020-06-24 21:51:54 +0200
  ipcalc.1: removed
...
  322294d7bf61f9fdf0e45e9b6c6013a7c6a35bfd
  2020-10-14 18:33:24 +
  Do not fail to build if ronn is missing
[..the converter that is..]
...
  f14f19f7fbdb83d7ba2cef5de758c649abdce1e8
  2022-11-12 11:07:24 +0100
  updated manual to comply with ronn syntax

HA-HA-HA!!  Somewhat broken for 2 1/2 years.
What a mess.

Good night from Germany.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[George Michaelson]

Not to prolong the agony, I tried the example on OSX for man tbl:

  .TS
  tab(@);
  ccc.
  This@is@centered
  Well,@this@also
  .TE

It didn't work with the nroff -man they supply. It did work with mandoc

So my summary of understanding so far:

FreeBSD: does not honour .fc no longer has functional tbl or nroff in base,
does have mandoc which honours .TS/.TE
OSX: does not honour .fc, no longer has functional tbl, nroff is braindead,
does have mandoc which honours .TS/TE
debian: has nroff and tbl.  nroff appears to be groff. honours .fc. The
minimal install does not have mandoc. apt install mandoc and it works.

Re: Unsupported nroff macros on MacOS X

[Ken Hornstein]

>Not to prolong the agony, I tried the example on OSX for man tbl:
>
>  .TS
>  tab(@);
>  ccc.
>  This@is@centered
>  Well,@this@also
>  .TE
>
>It didn't work with the nroff -man they supply. It did work with mandoc

Silly question: what version of MacOS X do you have?  On my Ventura
install I get:

% which nroff
nroff: Command not found.

(I did find a Monterey install and it does have an nroff on it)

I think as long as whatever "man" ends up invoking on a particlar system
supports tbl(1) macros we are fine.  And I believe that is true!  Even
on the aforementioned Monterey system if I put those contents in "foo.1"
and run "man ./foo.1" then the right thing happens.

--Ken

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

George Michaelson wrote in
 :
 |Not to prolong the agony, I tried the example on OSX for man tbl:
 |
 |  .TS
 |  tab(@);
 |  ccc.
 |  This@is@centered
 |  Well,@this@also
 |  .TE
 |
 |It didn't work with the nroff -man they supply. It did work with mandoc

It surely works with "tbl FILE|nroff|more" if there is tbl and
nroff.  Or solely "groff -tTascii FILE|more".

 |So my summary of understanding so far:

Yes i think so.  Note that in

  Hello world.$
  $
  This^Iis^Icentered$
  .sp 0$
  This^Iis^Icentered$
  $
  Well,^Ithis^Ialso$
  $
  Hey.$
  This not$

mandoc deviates from groff unless .ta is used as in 

  Hello world.$
  $
  .ta 4 8$
  This^Iis^Icentered$
  .sp 0$
  This^Iis^Icentered$
  $
  Well,^Ithis^Ialso$
  $
  Hey.$
  This not$

Ie better set tabs explicitly.  Now.  I am not an expert, and my
mandoc is very old, things might have changed.

  $ ll $(command -v mandoc)
  -rwxr-xr-x 1 steffen steffen 1176552 Apr 13  2019 /usr/local/bin/mandoc*

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[George Michaelson]

I have ventura but not a clean install. it found homebrew nroff.

/usr/bin/mandoc is in the base. Thats probably why the man command worked.

what you said otherwise, I concur with.

-G

Re: Unsupported nroff macros on MacOS X

[George Michaelson]

You're using macports, homebrew or fink. If the command is in
/usr/local/bin then "its not in OSX" stands as a problem. Its not in $PATH
for a native install without adjunct s/w

All of this can be avoided if nmh adopts "install package x" requirements
because groff does handle .fc.

The point is, (I believe) Ken is (rightly) trying to minimise the surface
of required dependencies to install, to ones in the base distro of most
Unixen.

-G

Re: Unsupported nroff macros on MacOS X

[Anthony J. Bentley]

Ken Hornstein writes:
> Let's take the example you gave where the first line for a man
> page that uses tbl should contain:
>
> '\" t
>
> So, my question is ... what does this mean?  I understand that \" is
> a comment, but I'm confused about the leading single quote.

According to mandoc's roff(7) manual:

 The single quote ("'") is accepted as an alternative control
 character, treated by mandoc(1) just like ‘.’

And later:

 Text following an escaped double-quote ‘\"’, whether in a request,
 macro, or text line, is ignored to the end of the line.  A request
 line beginning with a control character and comment escape ‘.\"’ is
 also ignored.

As for why the 't' is there, it's strictly a manpage practice, not
a general roff authoring practice. man(1) peeks at the comment (man-db
does anyway; pretty sure mandoc doesn't need it) to determine which
preprocessors to run. E.g., a manual that needs the eqn preprocessor
would have an 'e' instead.

Normally the person piping a document through troff would be the author,
and would already know that the document contains tables that require
the source to be preprocessed with tbl. But manuals are formatted by
users; the man(1) program doesn't know the document features in advance,
and wants to avoid running preprocessors unless they're necessary.