Re: self-hosted man.openbsd.org script?
Hello,
A very good explanation. All crystal clear now, i thank you for your time.
I had a strange idea in the late night about replacing something with
something else, but i see it is not the case and there is no need to
elaborate.
Thank you.
On Thu, Dec 28, 2023 at 3:46 AM Ingo Schwarze wrote:
>
> Hi Mihai,
>
> Mihai Popescu wrote on Thu, Dec 28, 2023 at 01:32:34AM +0200:
>
> > [ removed elaborate instruction about going html from almost txt with
> > man pages ]
> >
> > All this to jump in html boat? Or I got it wrong?
> > Are old man pages deprecated?
>
> Manual pages are not restricted to a specific output format.
>
> THE traditional output format, as far as such a thing exists, is
> black markings on dead trees. That output format later evolved into
> PostScript format and still later into PDF format. Let's call all
> these output modes "typeset output".
>
> Another very old output format is video terminal (CRT) output.
> That's still used a lot, though mostly via virtual terminals nowadays
> rather than on CRTs. But all that is clearly younger than typeset
> output mode.
>
> It has already been explained why nowdays (meaning: during the
> last three decades) it has become convenient to also be able to
> access information remotely via the WWW. The simplest format to
> facilitate that has always been HTML.
>
> Even more recently (as in: during the last decade) source code
> management platforms have become popular that require documentation
> in Markdown format. So that's just another output mode for manual
> pages, see for example
>
> https://github.com/Tairokiya/rfind
> (even though using manual page format isn't widespread on that
>particular platform yet, and this example is of poor quality
>in several respects)
>
> or
>
> https://codeberg.org/fobser/gelatod
> (even though the software used by Codeberg, Forgejo, is currently
>haunted by a bug that prevents correct rendering of the
>standard-conforming Markdown (specifically, CommonMark) code
>generated from manual pages, as you can see - but that bug is
>already fixed and will be gone from the next Forgejo release)
>
>
> So, if the output format is *not* what defines manual pages, then is it
> that defines them?
>
> 1. The idea of having one self-contained document ("manual page")
> for each interface (specifically, CLI command in sections 1 and 8,
> system call in section 2, API function in section 3, kernel driver
> in section 4, configuration file in section 5, domain specific
> language in section 7, kernel function in section 9)
>
> 2. Each of these documents being complete but concise, i.e. not
> mixing in explanations of required prior knowledge about the
> foundations the interface is built on, nor containing tutorial-
> style instructions
>
> 3. A common structure consisting of
> - a one-line description (name section)
> - an informal (i.e. non-BNF), human readable syntax display
>(synopsis section)
> - a description of the arguments and behaviour (description section)
> - various standardized auxiliary sections like "return values",
>"environment", "files", "exit status", "errors", "see also",
>"standards" and so on, to help quickly locate information
>that often needs to be looked up
>
> 4. Universal tygographic conventions helping readability of the
> page for anyone familiar with other manual pages, no matter
> who wrote the particular page currently being looked at
>
> Using HTML output format for reading manual pages allows taking full
> advantage of these concepts just like any other output format does.
> So there is really no need to bash HTML as a manual page output
> format. Also remeber that HTML output format may be particularly
> convenient for people having specific accessibility requirements,
> for example blind people.
>
>
> Maybe what you had in mind is that some software authors abuse HTML as
> an *input* format for documentation, that they write documentation for
> their software directly in HTML format (instead of writing manual pages,
> which can then trivially be convented to HTML if desired, and additionally
> to many other formats). *Maintaining* documentation in HTML format is
> indeed a very bad idea. That usually results in documentation that is
> disorganized, sprawling rather than concise, hinders locating information,
> is riddled with idiosyncratic formatting choices, and next to impossible
> to convert to any other format.
>
> Yours,
> Ingo
Re: self-hosted man.openbsd.org script?
Hi Mihai,
Mihai Popescu wrote on Thu, Dec 28, 2023 at 01:32:34AM +0200:
> [ removed elaborate instruction about going html from almost txt with
> man pages ]
>
> All this to jump in html boat? Or I got it wrong?
> Are old man pages deprecated?
Manual pages are not restricted to a specific output format.
THE traditional output format, as far as such a thing exists, is
black markings on dead trees. That output format later evolved into
PostScript format and still later into PDF format. Let's call all
these output modes "typeset output".
Another very old output format is video terminal (CRT) output.
That's still used a lot, though mostly via virtual terminals nowadays
rather than on CRTs. But all that is clearly younger than typeset
output mode.
It has already been explained why nowdays (meaning: during the
last three decades) it has become convenient to also be able to
access information remotely via the WWW. The simplest format to
facilitate that has always been HTML.
Even more recently (as in: during the last decade) source code
management platforms have become popular that require documentation
in Markdown format. So that's just another output mode for manual
pages, see for example
https://github.com/Tairokiya/rfind
(even though using manual page format isn't widespread on that
particular platform yet, and this example is of poor quality
in several respects)
or
https://codeberg.org/fobser/gelatod
(even though the software used by Codeberg, Forgejo, is currently
haunted by a bug that prevents correct rendering of the
standard-conforming Markdown (specifically, CommonMark) code
generated from manual pages, as you can see - but that bug is
already fixed and will be gone from the next Forgejo release)
So, if the output format is *not* what defines manual pages, then is it
that defines them?
1. The idea of having one self-contained document ("manual page")
for each interface (specifically, CLI command in sections 1 and 8,
system call in section 2, API function in section 3, kernel driver
in section 4, configuration file in section 5, domain specific
language in section 7, kernel function in section 9)
2. Each of these documents being complete but concise, i.e. not
mixing in explanations of required prior knowledge about the
foundations the interface is built on, nor containing tutorial-
style instructions
3. A common structure consisting of
- a one-line description (name section)
- an informal (i.e. non-BNF), human readable syntax display
(synopsis section)
- a description of the arguments and behaviour (description section)
- various standardized auxiliary sections like "return values",
"environment", "files", "exit status", "errors", "see also",
"standards" and so on, to help quickly locate information
that often needs to be looked up
4. Universal tygographic conventions helping readability of the
page for anyone familiar with other manual pages, no matter
who wrote the particular page currently being looked at
Using HTML output format for reading manual pages allows taking full
advantage of these concepts just like any other output format does.
So there is really no need to bash HTML as a manual page output
format. Also remeber that HTML output format may be particularly
convenient for people having specific accessibility requirements,
for example blind people.
Maybe what you had in mind is that some software authors abuse HTML as
an *input* format for documentation, that they write documentation for
their software directly in HTML format (instead of writing manual pages,
which can then trivially be convented to HTML if desired, and additionally
to many other formats). *Maintaining* documentation in HTML format is
indeed a very bad idea. That usually results in documentation that is
disorganized, sprawling rather than concise, hinders locating information,
is riddled with idiosyncratic formatting choices, and next to impossible
to convert to any other format.
Yours,
Ingo
Re: self-hosted man.openbsd.org script?
Mihai Popescu writes: [ removed elaborate instruction about going html from almost txt with man pages ] All this to jump in html boat? Or I got it wrong? Are old man pages deprecated? It can be convenient to have Web-based access to the man pages for systems or software that one isn't currently using (or that isn't already installed). Gentoo is my daily driver, and as it turns out, the OpenBSD man pages are available as a package in the GURU repo, so i have that package installed to allow me to read those man pages on my Gentoo box: $ man cat will get me the man page for GNU coreutils 'cat', but: $ man 1bsd cat will get me the man page for OpenBSD's 'cat'. In the absence of such packaging, a Web-based UI for reading the man pages of other systems can be really helpful (e.g. for checking what functionality is and isn't implemented in a given system's implementation of a particular program, for portability reasons). Alexis.
Re: self-hosted man.openbsd.org script?
[ removed elaborate instruction about going html from almost txt with man pages ] All this to jump in html boat? Or I got it wrong? Are old man pages deprecated?
Re: self-hosted man.openbsd.org script?
Hi Paul,
Paul Pace wrote on Sun, Dec 24, 2023 at 05:25:55AM -0800:
> I have this vague memory of reading someone who posted a script, IIRC,
> to convert the system's man pages to HTML, or similar, into somewhere
> under /var/www and the pages worked just like the highly useful
> man.openbsd.org, and not like the plain text pages that everyone always
> posts to their websites.
>
> Does someone happen to know where that is?
I don't know about any such "script" and believe using a script for it
would be a bad idea - a dirty hack at best.
Converting a whole tree of manual pages to a different format
sounds like the job for the tradition catman(8) utility program
that Christoph Robitschko first implemented in 1993. NetBSD
and FreeBSD contain implementations by various other authors.
OpenBSD does not contain the catman(8) utility because it is rarely
needed by ordinary users and we tend to only include code in the base
system that is useful for many people.
However, the portable mandoc distribution does contain a version
that i wrote together with Michael Stapelberg (of the Debian project)
in 2017:
https://mandoc.bsd.lv/
https://mandoc.bsd.lv/man/catman.8.html
For your purpose, you may want to replace to line
options.fragment = 1;
in mandocd.c by something like
{ char style[] = "/usr/share/misc/mandoc.css";
options.style = style; }
before compiling such that you get complete HTML code including
, , , and elements.
Be careful to not clobber the system mandoc(1) installation by blindly
running "make install". Instead, just manually installing the
binary "mandocd" anywhere in the $PATH is enough. Installing
the "catman" binary in not necessary but won't hurt either.
Also note that viewing the results with a monster browser like
firefox or chrome may require some tweaking with respect to unveil(2),
see the respective files below /usr/local/share/doc/pkg-readmes/.
I freely admit all this is not particularly user-friendly but more
geared towards the needs of server admins. For example, the
server manpages.debian.org is essentially using something similar
to this method. For them, it's critical that this implementation
of catman(8) is much more efficient than the NetBSD and FreeBSD
implementations: it saves lots of time because it does *not* fork
and exec a new parser/formatter process for every manual page
but instead merely reinitialized and resuses the same parser and
formatter process over and over again, which is *much* faster.
With the huge amount of manual pages Debian has to format very
often, their server would not be able to keep up without that
optimization.
Yours,
Ingo
Re: self-hosted man.openbsd.org script?
Paul Pace writes: I have this vague memory of reading someone who posted a script, IIRC, to convert the system's man pages to HTML, or similar, into somewhere under /var/www and the pages worked just like the highly useful man.openbsd.org, and not like the plain text pages that everyone always posts to their websites. Does someone happen to know where that is? Not sure if this is what you were specifically thinking of, but there's man.cgi(8): https://mandoc.bsd.lv/man/man.cgi.8.html Alexis.
Re: self-hosted man.openbsd.org script?
On 12/24/23 08:25, Paul Pace wrote: I have this vague memory of reading someone who posted a script, IIRC, to convert the system's man pages to HTML, or similar, into somewhere under /var/www and the pages worked just like the highly useful man.openbsd.org, and not like the plain text pages that everyone always posts to their websites. Does someone happen to know where that is? /usr/src/usr.bin/mandoc is where the source for man.cgi resides. Frequent small updates take place, I believe it would be good to use the -current source code if you wish to play with it. It has very simple dependencies, so should be no issue running -current man.cgi code compiled on a -stable. ... however ... being that UofT might be down for a few days, I have lit up a VPS with cvsweb and man content on them. https://cvsweb.egoslike.us/ https://man.egoslike.us/ And look, my backups and notes don't suck. :) These are not official, but they are run by one of the people who run the official sites. They will go away once the official site is back up and running. Nick. On 12/23/23 11:16 AM, Nick Holland wrote: On 12/19/23 15:38, Nick Holland wrote: Hello, man.openbsd.org, cvsweb.openbsd.org, openbsd.cs.toronto.edu and obsdacvs.cs.toronto.edu will be unavailable for site maintenance starting Thursday, December 21 about 6:00am ET (UTC-5) and hopefully be back up and running by Saturday, December 23, 6:00am ET. Sorry for any inconvenience. Nick. Unfortunately, it seems there's a problem impacting our servers, and everyone is celebrating the holiday. So ... return of man.openbsd.org, cvsweb.openbsd.org and the install and anoncvs mirrors will be delayed. Nick.
Re: self-hosted man.openbsd.org script?
On Sun, Dec 24, 2023 at 05:25:55AM -0800, Paul Pace wrote: > I have this vague memory of reading someone who posted a script, IIRC, to > convert the system's man pages to HTML, or similar, into somewhere under > /var/www and the pages worked just like the highly useful man.openbsd.org, > and not like the plain text pages that everyone always posts to their > websites. > > Does someone happen to know where that is? Not a script, but there are instructions in the lower half of usr.bin/mandoc/Makefile and you can read the man.cgi manual with $ mandoc -a usr.bin/mandoc/man.cgi.8 https://github.com/openbsd/src/blob/master/usr.bin/mandoc/Makefile#L43-L77
Re: self-hosted man.openbsd.org script?
On Sun, Dec 24, 2023 at 05:25:55AM -0800, Paul Pace wrote: > I have this vague memory of reading someone who posted a script, IIRC, to > convert the system's man pages to HTML, or similar, into somewhere under > /var/www and the pages worked just like the highly useful man.openbsd.org, > and not like the plain text pages that everyone always posts to their > websites. > > Does someone happen to know where that is? My guess would be 'mandoc -T html' is the core utility used. -Otto > > On 12/23/23 11:16 AM, Nick Holland wrote: > > On 12/19/23 15:38, Nick Holland wrote: > > > Hello, > > > > > > man.openbsd.org, cvsweb.openbsd.org, openbsd.cs.toronto.edu > > > and obsdacvs.cs.toronto.edu will be unavailable for site > > > maintenance starting Thursday, December 21 about 6:00am ET > > > (UTC-5) and hopefully be back up and running by Saturday, > > > December 23, 6:00am ET. > > > > > > Sorry for any inconvenience. > > > > > > Nick. > > > > > > > Unfortunately, it seems there's a problem impacting our servers, > > and everyone is celebrating the holiday. > > > > So ... return of man.openbsd.org, cvsweb.openbsd.org and > > the install and anoncvs mirrors will be delayed. > > > > Nick. >
self-hosted man.openbsd.org script?
I have this vague memory of reading someone who posted a script, IIRC, to convert the system's man pages to HTML, or similar, into somewhere under /var/www and the pages worked just like the highly useful man.openbsd.org, and not like the plain text pages that everyone always posts to their websites. Does someone happen to know where that is? On 12/23/23 11:16 AM, Nick Holland wrote: On 12/19/23 15:38, Nick Holland wrote: Hello, man.openbsd.org, cvsweb.openbsd.org, openbsd.cs.toronto.edu and obsdacvs.cs.toronto.edu will be unavailable for site maintenance starting Thursday, December 21 about 6:00am ET (UTC-5) and hopefully be back up and running by Saturday, December 23, 6:00am ET. Sorry for any inconvenience. Nick. Unfortunately, it seems there's a problem impacting our servers, and everyone is celebrating the holiday. So ... return of man.openbsd.org, cvsweb.openbsd.org and the install and anoncvs mirrors will be delayed. Nick.
