Re: [bug #64301] [troff] susceptible to integer overflow

[Colin Watson]

On Tue, Jul 16, 2024 at 06:59:08AM -0500, G. Branden Robinson wrote:
> At 2024-07-15T17:03:53-0700, Collin Funk wrote:
> > Not directly related, but I noticed that groff uses a ./bootstrap
> > script which is 2 years behind gnulib updates.
> > 
> > Whenever you update the gnulib git submodule I would recommend running:
> > 
> > $ ./bootstrap --bootstrap-sync
> 
> Ah, thank you.  I noticed Colin Watson had recently achieved that update
> for man-db and was going to ask him how to do it.

I must confess that I'd generally just been copying the file by hand
from the gnulib checkout (build-aux/bootstrap).  I will try to remember
that the --bootstrap-sync option exists!

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Debian Salsa and groff license

[Colin Watson]

On Fri, Jul 05, 2024 at 08:38:04PM -0500, G. Branden Robinson wrote:
> See attached screenshot.
> 
> This seems a bit dubious.  I'm logged in and can't see how to fix it.
> 
> I guess GitLab is doing their part is the glorious struggle to defeat
> copyleft (they'll catch up to Microsoft GitHub yet!).
> 
> Nevertheless it seems like an accuracy problem to me.
> 
> What can we do?

Yes.  I don't know how to fix this; as far as I'm aware it's not within
the power of project owners to set this manually (much though that seems
completely unreasonable).

It might be worth filing a support issue with salsa.debian.org's admins
to see whether there's some piece of the licence detection machinery
that they can kick.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: vox clamantis in deserto

[Colin Watson]

On Wed, Jul 03, 2024 at 05:20:06PM -0500, G. Branden Robinson wrote:
> At 2024-07-03T22:32:06+0100, Colin Watson wrote:
> > I think this is https://savannah.gnu.org/bugs/?64438, right?  If
> > experts advise that this patch is safe to cherry-pick in isolation (I
> > know neither -ms nor pic well),
> 
> While shy of the term "expert", I can affirm thus.  The fix is not
> dependent on post-1.23.0 groff development in any way.

Good stuff.  Done (as per the other more detailed thread on post-1.23.0
regressions), thanks.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: most widespread complaints about groff 1.23.0

[Colin Watson]

On Wed, Jul 03, 2024 at 05:13:14PM -0500, G. Branden Robinson wrote:
> I've prepared a GNU Savannah query for all resolved tickets of Important
> severity whose fixes are expected in groff 1.24.[7]  Some are fixes for
> regressions.  Doug's message about "groff -ms -p" not working is one.
> 
> If a distributor were looking for patches to backport, I'd start there.

I cherry-picked a couple of what seemed to me to be the most urgent
patches and uploaded them to Debian as 1.23.0-5.  This includes a fix
for the problem Doug reported.

What are the plans for a 1.24 release?  In general I think it's bad to
drift into a habit of allowing too much time and too many commits to
pass between releases; it makes it more difficult to work out what went
wrong when users report regressions, and it invites a "just one more
fix" mindset which tends to be never-ending until the complaints pile up
high enough.

Thanks,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: vox clamantis in deserto

[Colin Watson]

On Wed, Jul 03, 2024 at 02:25:05PM -0400, Douglas McIlroy wrote:
> When I picked up a copy of 1.23.0 barely a week after its release, I found
> (and reported) that [groff -ms -p] had a fatal auto-immune disease, in
> which -ms diagnosed customary pic output as invalid. Nearly a year later,
> our departmental computer center just caught the disease from Ubuntu. If
> complaints of unusability have not been piling up in the interim, it seems
> that users like me, with gobs of -ms -p documents, are a relict endangerd
> species.

I think this is https://savannah.gnu.org/bugs/?64438, right?  If experts
advise that this patch is safe to cherry-pick in isolation (I know
neither -ms nor pic well), then I'd be happy to upload it to Debian, and
that would at least trickle into future Ubuntu releases even if there
doesn't happen to be a proper groff release in the interim.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: gropdf (1.23.0): incorrect CreationDate/ModDate

[Colin Watson]

On Fri, Apr 05, 2024 at 02:27:15PM +0200, Christof Meerwald wrote:
> On Thu, Apr 04, 2024 at 06:21:37PM -0500, G. Branden Robinson wrote:
> > Colin, can you double-check whether you're producing a date format
> > conformant with §7.9.4 of ISO 32000?  I'm attaching the relevant pages
> > of Adobe's gratis version of the standard.
> 
> I am attaching a patch that should address all three bugs.

Thanks for this patch, and sorry for messing this up in the first place.
After some thought, I agree with it, and have committed it (with
additional commentary) as
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=0815e503dba8d5c05921d68c6c718fe8f8440ee8.
I'll also fix it in Debian.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PATCH] Distribute bootstrap and bootstrap.conf

[Colin Watson]

On Thu, Apr 04, 2024 at 10:47:29PM -0500, Dave Kemper wrote:
> On Sun, Mar 31, 2024 at 5:31 AM Colin Watson  wrote:
> > I've omitted README.git to ensure that we still warn people who don't
> > know what they're doing that running "./bootstrap" may not be the right
> > place to start.
> 
> *raises hand*  I am one of those who don't know what they're doing --
> at least as regards the build system.  When I build groff from git, I
> blindly follow the recipe Bertrand posted a few years ago
> (http://lists.gnu.org/r/groff/2020-11/msg00068.html), which starts
> with the "./bootstrap" step.  Is the above saying this step should be
> removed, or replaced with something else?

No, it's still the right thing to do when building from git.

> Or does this change only apply to tarballs, not git pulls?

That.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PATCH] Distribute bootstrap and bootstrap.conf

[Colin Watson]

On Thu, Apr 04, 2024 at 12:45:42AM -0700, Collin Funk wrote:
> Hi Colin (nice name),

:-)

> > I looked into what it would take for Debian's groff package to do a full
> > rebootstrap from its packaged version of gnulib.  It seems relatively
> > straightforward, but it requires including bootstrap and bootstrap.conf
> > in tarballs so that we know what modules to use.
> 
> I'm testing gnulib-tool.py on packages at the moment and stumbled upon
> this commit in groff. Another option worth considering is using the
> gnulib-cache.m4 file.

Yeah, I know of gnulib-cache.m4.  However, IMO there's no reason to drop
down to that level when the bootstrap script is in use, because
bootstrap generates gnulib-cache.m4.  Also, ./bootstrap typically does a
bit more, sometimes including extra package-specific hooks defined in
bootstrap.conf that can't be expressed in gnulib-cache.m4.

Thanks,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PATCH] Distribute bootstrap and bootstrap.conf

[Colin Watson]

On Sun, Mar 31, 2024 at 06:04:47AM -0500, G. Branden Robinson wrote:
> At 2024-03-31T11:30:25+0100, Colin Watson wrote:
> > I looked into what it would take for Debian's groff package to do a
> > full rebootstrap from its packaged version of gnulib.  It seems
> > relatively straightforward, but it requires including bootstrap and
> > bootstrap.conf in tarballs so that we know what modules to use.
> 
> 2 lines of diff naming the two files!  I don't think it _gets_ more
> straightforward.
> 
> It's so close to April Fool's Day, I would have been tickled if you'd
> submitted it more like this.

:-)

> They say this was a "sophisticated attacker", but it also appears to be
> one who didn't grasp that "> /dev/null" is redundant with "grep -q".

Some of the sophistication was burying the actual exploit in confusion,
of course ...

> > I've omitted README.git to ensure that we still warn people who don't
> > know what they're doing that running "./bootstrap" may not be the
> > right place to start.
> 
> I approve of this change.  Push it whenever you're ready unless you
> would like to await feedback from others.  (Hard to imagine a case
> against, though.)

Done, thanks.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

[PATCH] Distribute bootstrap and bootstrap.conf

[Colin Watson]

With the recent xz-utils backdoor, there's been more focus on cases
where build systems rely on files produced by "make dist" and included
in release tarballs.  It's already fairly standard practice for
distributions to rebuild configure scripts using autoreconf, but less so
to rebuild the files that are produced by gnulib.

I looked into what it would take for Debian's groff package to do a full
rebootstrap from its packaged version of gnulib.  It seems relatively
straightforward, but it requires including bootstrap and bootstrap.conf
in tarballs so that we know what modules to use.

I've omitted README.git to ensure that we still warn people who don't
know what they're doing that running "./bootstrap" may not be the right
place to start.

* Makefile.am (EXTRA_DIST): Add "bootstrap" and "bootstrap.conf".
---
 Makefile.am | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Makefile.am b/Makefile.am
index e15a8ff0f..d41d4ee1d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -797,6 +797,8 @@ endif
 
 # Other files that should be present in the distribution tarball.
 EXTRA_DIST += \
+  bootstrap \
+  bootstrap.conf \
   BUG-REPORT \
   ChangeLog.old \
   ChangeLog.111 \
-- 
2.43.0


signature.asc
Description: PGP signature

Re: Version string with git(1) (-dirty)

[Colin Watson]

On Fri, Feb 23, 2024 at 03:25:05PM +0100, Alejandro Colomar wrote:
> On Fri, Feb 23, 2024 at 03:20:58PM +0100, Alejandro Colomar wrote:
> > So, how do you get that "-dirty" suffix?  Is it autotools?  Or git?  Or
> > is it specific to the groff build system?  I could use some shell code
> > in the makefile to get it, but was asking mainly in case git already
> > gives that somehow.
> 
> Never mind, I should RTFM.  :)
> 
> $ MANWIDTH=72 man git-describe | sed -n '/^ *--dirty/,/^$/p'
>  --dirty[=], --broken[=]
>  Describe the state of the working tree. When the working tree
>  matches HEAD, the output is the same as "git describe HEAD".
>  If the working tree has local modification "-dirty" is
>  appended to it. If a repository is corrupt and Git cannot
>  determine if there is local modification, Git will error out,
>  unless ‘--broken’ is given, which appends the suffix
>  "-broken" instead.

But also, gnulib's git-version-gen (which groff uses) does this
manually.  See:

  
https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=build-aux/git-version-gen

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Raw colored ASCII/UTF art for manpages

[Colin Watson]

On Thu, Dec 28, 2023 at 10:59:08PM +0100, Oliver Corff wrote:
> how can we inform the manpage maintainer for man(1) that groff has
> evolved beyond 1.17?

Please could you report this here so I don't forget about it?

  https://gitlab.com/man-db/man-db/-/issues

(Branden's other followup to your message seems to mix up man(1) and
man(7).)

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

[PATCH] [tests]: Tolerate U register defaulting to 1.

[Colin Watson]

If tmac/man.local is patched to set the U register to 1, then one test
fails.  That test is concerned specifically with the rendering of long
URLs when they are not turned into OSC 8 hyperlinks, so it makes sense
to disable OSC 8 hyperlinks for that test.

* tmac/tests/an_UE-breaks-before-long-URIs.sh: Run groff with -rU0.
---
 tmac/tests/an_UE-breaks-before-long-URIs.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tmac/tests/an_UE-breaks-before-long-URIs.sh 
b/tmac/tests/an_UE-breaks-before-long-URIs.sh
index 0b151fd8d..f39ceaea1 100755
--- a/tmac/tests/an_UE-breaks-before-long-URIs.sh
+++ b/tmac/tests/an_UE-breaks-before-long-URIs.sh
@@ -48,10 +48,10 @@ wail () {
 fail=yes
 }
 
-output=$(printf "%s" "$input" | "$groff" -Tascii -P-cbou -man)
+output=$(printf "%s" "$input" | "$groff" -Tascii -P-cbou -rU0 -man)
 echo "$output"
 error=$(printf "%s" "$input" \
-| "$groff" -Tascii -P-cbou -man -ww -z 2>&1)
+| "$groff" -Tascii -P-cbou -rU0 -man -ww -z 2>&1)
 
 echo "testing that no diagnostic messages are produced" >&2
 test -z "$error" || wail
-- 
2.39.2

Re: man-db test failures [was: unrecognized X command 'sgr 0' ignored (was: early adopters of] groff 1.23.0)

[Colin Watson]

On Tue, Jul 11, 2023 at 12:42:49AM +0200, Alexis wrote:
> The NixOS package for groff 1.23.0 is still seeing man-db test-failures
> with groff 1.23.0 (https://github.com/NixOS/nixpkgs/pull/241870).

Please report this to https://gitlab.com/man-db/man-db/-/issues and we
can have a look there.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Sun, Jul 09, 2023 at 07:13:54PM +0100, Deri wrote:
> On Sunday, 9 July 2023 17:59:04 BST G. Branden Robinson wrote:
> > I have no objection to this patch.  I think Colin is right that none
> > but reproducible-build mavens are going to use $SOURCE_DATE_EPOCH.
[...]
> It looks Ok, not tested. I'm happy for you to apply.

Thanks for the +1s.  I've gone ahead and pushed this to master (after a
slight rebase), and will cherry-pick it into Debian's 1.23.0 packaging.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [groff] 22/127: [build]: Install PDF documents better.

[Colin Watson]

On Mon, Jul 10, 2023 at 04:30:24AM -0400, G. Branden Robinson wrote:
> commit 838a36711cff1daff2ac01abf0462b3d24a74aac
> Author: G. Branden Robinson 
> AuthorDate: Mon Apr 3 23:23:24 2023 -0500
> 
> [build]: Install PDF documents better.
> 
> Ship only one copy of "automake.pdf", and install it and the new
> "groff-man-pages.pdf" in the "pdf/" subdirectory of the destination
> "doc" directory.

This is belated, but does it actually make sense to install automake.pdf
at all?  It seems to be entirely aimed at people working on the groff
code base itself; I can't see any reason it would be interesting to
include it in "make install".

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Sun, Jul 09, 2023 at 07:13:54PM +0100, Deri wrote:
> It looks Ok, not tested. I'm happy for you to apply.

Thanks.

> Once gropdf moves to the font subsetting version it will cause a
> problem for reproducible builds, since the gubbins inside the pdf
> changes enormously, but it should look identical.

The object of reproducible builds isn't to ensure that output never
changes - it's understood that different versions of tools frequently
generate different output.  Rather, it's to ensure that if two different
people run builds of the same inputs in sufficiently similar
environments, they get the same outputs.  (Characterizing "sufficiently
similar" of course takes considerable work, but it usually involves at
least pinning the same versions of all the build-dependencies.)

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Sun, Jul 09, 2023 at 11:59:04AM -0500, G. Branden Robinson wrote:
> At 2023-07-09T13:24:27+0100, Colin Watson wrote:
> > On Sun, Jul 09, 2023 at 02:42:55AM +0100, Colin Watson wrote:
> > > So, how about the attached patch?
> > 
> > I somehow neglected to update doc/groff.texi.  Here's an updated
> > version.
> 
> I have no objection to this patch.  I think Colin is right that none
> but reproducible-build mavens are going to use $SOURCE_DATE_EPOCH.
> 
> I am curious to see some concrete examples of the sorts of shenanigans
> TZ is used for in reproducible build testing, which foreclose the groff
> 1.23.0 approach as far as Debian (and presumably others) are concerned.

https://salsa.debian.org/reproducible-builds/reprotest/-/blob/master/reprotest/build.py
is the only thing I have to hand; if you search for "TZ" in that you'll
find code that runs a pair of builds with TZ=GMT+12 and TZ=GMT-14 to try
to catch some extremes.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Sun, Jul 09, 2023 at 02:42:55AM +0100, Colin Watson wrote:
> So, how about the attached patch?

I somehow neglected to update doc/groff.texi.  Here's an updated
version.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]
>From 522c4beb20414e8ceb6b321bbc96df6f35a81a10 Mon Sep 17 00:00:00 2001
From: Colin Watson 
Date: Sun, 9 Jul 2023 13:23:21 +0100
Subject: [PATCH] Display time from SOURCE_DATE_EPOCH in UTC.

The semantics imposed in 1.23.0 are unsuitable for use with
reproducible-builds harnesses, since those specifically want to vary the
TZ environment variable to shake out other problems in build systems.
However, my patch that Debian has been carrying for a while is
unsuitable for general use, since most people expect the time displayed
in output to use local time.

A viable compromise seems to be to force UTC _only_ when
SOURCE_DATE_EPOCH is set.  That will keep reproducible-builds harnesses
working with no extra effort, while also preserving the expected
behaviour for typical users of groff that don't go out of their way to
set that environment variable.

As a bonus, this corrects the behaviour of gropdf when the local offset
from UTC is not a whole number of hours.

* src/include/curtime.h (current_time): Return a `struct tm *`.
  Document behaviour.
* src/libs/libgroff/curtime.cpp (current_time): If SOURCE_DATE_EPOCH is
  set, return the overridden time after passing it through `gmtime`.
  Otherwise, pass the current time through `localtime`.

* src/devices/grohtml/post-html.cpp (html_printer::do_file_components,
  html_printer::~html_printer):
* src/devices/grops/ps.cpp (ps_printer::~ps_printer):
* src/roff/troff/input.cpp (init_registers): Adjust to new
  `current_time` signature.

* src/devices/gropdf/gropdf.pl: If SOURCE_DATE_EPOCH is set, return the
  overridden time after passing it through `gmtime`.  Otherwise, pass
  the current time through `localtime`.
  (PDFDate): Fix output in the case where the local offset from UTC is
  not a whole number of hours.  (Previously, the minutes offset field
  was always set to zero.)

* doc/groff.texi (Environment):
* src/devices/grohtml/grohtml.1.man (Environment):
* src/devices/gropdf/gropdf.1.man (Environment):
* src/devices/grops/grops.1.man (Environment):
* src/roff/groff/groff.1.man (Environment):
* src/roff/troff/troff.1.man (Environment): Update.

* NEWS: Document this.
---
 NEWS  | 17 +
 doc/groff.texi| 13 +++--
 src/devices/grohtml/grohtml.1.man | 12 +++-
 src/devices/grohtml/post-html.cpp | 16 
 src/devices/gropdf/gropdf.1.man   | 10 +-
 src/devices/gropdf/gropdf.pl  | 16 ++--
 src/devices/grops/grops.1.man | 12 +++-
 src/devices/grops/ps.cpp  |  9 ++---
 src/include/curtime.h | 19 +++
 src/libs/libgroff/curtime.cpp | 23 +--
 src/roff/groff/groff.1.man| 12 +++-
 src/roff/troff/input.cpp  | 24 +---
 src/roff/troff/troff.1.man| 12 +++-
 13 files changed, 110 insertions(+), 85 deletions(-)

diff --git a/NEWS b/NEWS
index 85b034be3..6b8e0cd36 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,23 @@
 This file describes recent user-visible changes in groff.  Bug fixes are
 not described.  There are more details in the man and info pages.
 
+VERSION 1.23.1
+==
+
+Miscellaneous
+-
+
+o If groff programs have their current time overridden by the SOURCE_DATE_EPOCH
+  environment variable, then that time is always displayed in UTC.  That
+  environment variable is normally only set when specifically requesting build
+  systems to produce reproducible output, and it is useful for reproducibility
+  test harnesses to vary the TZ environment variable and ensure that it does
+  not affect the output of the build; those harnesses have no way to set TZ=UTC
+  only for groff programs.  People setting SOURCE_DATE_EPOCH are likely to be
+  more in the "system programmer" camp as described in the release notes for
+  1.23.0, so it is easier to defend time-zone-invariant output to them.  In all
+  other cases, the current time remains displayed in local time.
+
 VERSION 1.23.0
 ==
 
diff --git a/doc/groff.texi b/doc/groff.texi
index 2a6635e9d..bcea4f3e7 100644
--- a/doc/groff.texi
+++ b/doc/groff.texi
@@ -1389,15 +1389,16 @@ overrides @env{GROFF_TYPESETTER}.
 @tindex SOURCE_DATE_EPOCH@r{, environment variable}
 A timestamp (expressed as seconds since the Unix epoch) to use as the
 output creation timestamp in place of the current time.  The time is
-converted to human-readable form using @cite{localtime@r{(3)}} when the
-formatter starts up and stored in registers usable by documents and
-macro packages (@pxref{Built-in Registers}).
+converted to human-readable form using @cite{gmtime@r{(3)}} and
+@cite{asctime@r{(3)}} when the form

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Tue, Dec 29, 2020 at 03:15:35PM +1100, G. Branden Robinson wrote:
> This issue is still open and tagged as a blocker for release, however.
> To resolve it we should settle on the time zone semantics of the system
> time retrieved by groff components (the formatter, the output drivers).
> We should also attempt to resolve these differing semantics between
> groff as distributed by GNU and by the Debian Project (and its many
> derivatives, none of which--to my knowledge--reverse the patch).
> 
> It appears to me that the consensus on this list is for groff to
> represent the system time in the local time zone, as date(1) does,
> whatever that may be.  People requiring reproducible builds should set
> TZ=UTC as well as SOURCE_EPOCH_DATE.

My apologies for dropping the ball on this thread.  I think I did a bad
job of communicating the problem.

Unfortunately, the solution above is not viable for Debian.  Harnesses
that test whether packages build reproducibly vary the TZ environment
variable intentionally (see
https://salsa.debian.org/reproducible-builds/reprotest/-/blob/master/reprotest/build.py),
as I understand it for good reasons: it's common for
inattentively-written build systems to have hidden dependencies on TZ
that affect things in surprising ways, and those harnesses are trying to
shake that sort of thing out.  They don't have a way to set TZ=UTC only
for groff.  I added this patch in Debian because the blast radius of
packages with reproducibility issues due to including groff output was
otherwise considerable, and it was by far the most economical way to
eliminate a class of reproducibility issues.

However, I made a strategic error in how I went about my previous
implementation.  My approach was just to use UTC unconditionally, since
I underestimated how much people would care about the current time
visible to groff programs.  Since it turns out that people do in fact
care, it seems to me that a better compromise would be to display the
time in UTC if the time was acquired from SOURCE_DATE_EPOCH, and
otherwise to display it in local time.  The broader user base who don't
care about reproducible builds won't be setting the SOURCE_DATE_EPOCH
environment variable anyway.  Anyone who is setting it is probably doing
so because they want to achieve some kind of reproducibility, and so is
rather more likely to be in the "systems programmer" camp; at the very
least, it ought to be easy for them to grasp why an invariant time zone
is useful in this context.

So, how about the attached patch?  I believe the results will not cause
the objections that were quite reasonably levelled at my previous
attempt, and it meets the reproducible-builds requirements that I know
about.

Thanks,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]
>From ba220ef16734eb45e246ff10523423e39603e168 Mon Sep 17 00:00:00 2001
From: Colin Watson 
Date: Sun, 9 Jul 2023 02:04:58 +0100
Subject: [PATCH] Display time from SOURCE_DATE_EPOCH in UTC.

The semantics imposed in 1.23.0 are unsuitable for use with
reproducible-builds harnesses, since those specifically want to vary the
TZ environment variable to shake out other problems in build systems.
However, my patch that Debian has been carrying for a while is
unsuitable for general use, since most people expect the time displayed
in output to use local time.

A viable compromise seems to be to force UTC _only_ when
SOURCE_DATE_EPOCH is set.  That will keep reproducible-builds harnesses
working with no extra effort, while also preserving the expected
behaviour for typical users of groff that don't go out of their way to
set that environment variable.

As a bonus, this corrects the behaviour of gropdf when the local offset
from UTC is not a whole number of hours.

* src/include/curtime.h (current_time): Return a `struct tm *`.
  Document behaviour.
* src/libs/libgroff/curtime.cpp (current_time): If SOURCE_DATE_EPOCH is
  set, return the overridden time after passing it through `gmtime`.
  Otherwise, pass the current time through `localtime`.

* src/devices/grohtml/post-html.cpp (html_printer::do_file_components,
  html_printer::~html_printer):
* src/devices/grops/ps.cpp (ps_printer::~ps_printer):
* src/roff/troff/input.cpp (init_registers): Adjust to new
  `current_time` signature.

* src/devices/gropdf/gropdf.pl: If SOURCE_DATE_EPOCH is set, return the
  overridden time after passing it through `gmtime`.  Otherwise, pass
  the current time through `localtime`.
  (PDFDate): Fix output in the case where the local offset from UTC is
  not a whole number of hours.  (Previously, the minutes offset field
  was always set to zero.)

* src/devices/grohtml/grohtml.1.man (Environment):
* src/devices/gropdf/gropdf.1.man (Environment):
* src/devices/grops/grops.1.man (Environment):
* src/roff/groff/groff.1.man (Environment):
* src/roff/troff/troff.1.man (Environment): Update.

* NEWS: Document t

Re: Behaviour of .so differs between mandoc and groff

[Colin Watson]

On Sun, Apr 30, 2023 at 07:05:55AM -0500, G. Branden Robinson wrote:
> The latter choice is a better one from a design perspective, in my
> opinion, because it is more general.  On the other hand, man pages
> sourcing the text of pages from other sections on the manual seems about
> as unlikely as a page in /usr/share/man sourcing one in /opt/man, which
> I dismissed as unworthy of support above.

It is, however, a real thing that happens.  Examples from a quick search
on my system:

  man3/XCompose.3.gz:.so man5/Compose.5
  man3/queue.3.gz:.so man7/queue.7
  man3/sigevent.3type.gz:.so man7/system_data_types.7
  man3/siginfo_t.3type.gz:.so man7/system_data_types.7
  man3/sigset_t.3type.gz:.so man7/system_data_types.7
  man3/sigval.3type.gz:.so man7/system_data_types.7
  man3/stpecpy.3.gz:.so man7/string_copying.7
  man3/stpecpyx.3.gz:.so man7/string_copying.7
  man3/ustpcpy.3.gz:.so man7/string_copying.7
  man3/ustr2stp.3.gz:.so man7/string_copying.7
  man3/zustr2stp.3.gz:.so man7/string_copying.7
  man3/zustr2ustp.3.gz:.so man7/string_copying.7
  man4/console_ioctl.4.gz:.so man2/ioctl_console.2
  man4/tty_ioctl.4.gz:.so man2/ioctl_tty.2
  man7/bash-builtins.7.gz:.so man1/bash.1
  man7/builtins.7.gz:.so man1/bash.1

I haven't exhaustively checked these, but at least some of them seem to
be internal to a given package.  My approach in man-db, though not an
especially scientific one, is to make reasonable efforts to support
observed usage if it isn't obviously unsupportable or entirely
unreasonable.

> In practice, as I understand it, `so` doesn't achieve anything for man
> pages that can't be done with symbolic links and (importantly) a man
> page indexer that is symlink-aware.

It is worth noting that today there are at least some real-world cases
where it isn't being used as a mere symlink; bash-builtins(7) is one
such.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Compressed man pages (was: Accessibility of man pages (was: Playground pager lsp(1)))

[Colin Watson]

On Sun, Apr 09, 2023 at 02:05:08PM +0200, Alejandro Colomar wrote:
> Important note: Sam, are you sure you want your pages compressed
> with bz2?  Have you seen the 10 seconds it takes man-db's man(1) to
> find a word in the pages?  I suggest that at least you try to
> reproduce these tests in your machine, and see if it's just me or
> man-db's man(1) is pretty bad at non-gz pages.

man-db is significantly slower with bzip2 than gzip these days, because
much of the performance work I did in 2.10.0 only applies to gzip:
there's in-process support for decompressing gzip, but we use
subprocesses for bzip2.  IMO the relatively small difference in
compressed size doesn't justify the effort of building in-process
support for multiple compression algorithms.  I recommend that
distributions just use gzip; but if distributions _really_ want to use
something else for whatever reason, then perhaps they should contribute
code to man-db to ensure similar performance to gzip.  I'm happy to give
pointers if there's a sufficiently compelling reason to make it worth
the effort.

> -  man-db's man(1) is slower with plain man(7) source than with .gz
>pages for some misterious reason.

Maybe CPU is sufficiently cheaper than I/O that the fact of reading less
data from disk dominates.


(Can I request that any concrete actions that need to be taken based on
this thread be split out to separate bug reports or something, please?
This thread is long and I don't really want to have lots of meandering
discourse in my inbox going back over the tired old man vs. info debate
or whatever, but if there are actual things I need to fix in man-db then
I'd rather not miss them.)

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Accessibility of man pages (was: Playground pager lsp(1))

[Colin Watson]

On Sat, Apr 08, 2023 at 03:02:59PM +0200, Alejandro Colomar wrote:
> Colin, I've had a feeling for a long time that compressed pages are
> not very useful.  These days, storage is cheap.  How would you feel
> about having the man pages installed uncompressed in Debian?  That
> would allow running text tools directly in /usr/share/man/.

I'm not personally all that bothered either way, but it's a
distribution-wide policy decision rather than something I'd decide on.
I suspect there are still some people who would push back against the
space cost.

> I've had to do that several times, and lucky me that I have the source
> code of the Linux man-pages checked out in my computers, but other
> users don't and they might have trouble finding for example which
> pages talk about RLIMIT_NOFILE.  The only way I know of is:

man -Kaw RLIMIT_NOFILE

(This looks at the page source rather than the rendered output, so
sometimes it over-reports if your search term matches a groff macro,
etc.  But that's true of your approach too.)

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: man page rendering speed (was: Playground pager lsp(1))

[Colin Watson]

On Fri, Apr 07, 2023 at 09:43:19AM -0500, G. Branden Robinson wrote:
> At 2023-04-07T09:36:10+0300, Eli Zaretskii wrote:
> > > From: "G. Branden Robinson" 
> [re-running *roff when a viewing a man page and resizing the terminal]
> > > Seems like it shouldn't be impossible to me, but what I imagine
> > > would require a little reëngineering of man(1), perhaps to spawn a
> > > little custom program to manage zcat/nroff pipeline it constructs.
> > > This little program's sole job could be to be aware of this pipeline
> > > and listen for SIGWINCH; if it happens, kill the rest of the
> > > pipeline and reëxecute it.

I didn't see the rest of the thread, but one significant complexity here
would be interacting with the pager to arrange for the viewing position
to be returned to where it was pre-SIGWINCH; bear in mind that the pager
is user-configurable (less(1) is common but not universal) and isn't
directly part of man(1).

> > This should be possible, but it flies in the face of the feature
> > whereby formatted man pages are kept for future perusal, which is
> > therefore faster:
> 
> You're referring to cat pages.  As far as I know, these are on their way
> out if not already gone.  Colin Watson, who has maintained the man-db
> implementation of man(1)[1] for something like 20 years, can speak more
> authoritatively to this, but as I understand it, the advent of resizable
> xterm windows started to kill the utility of cat pages decades ago and
> the increasing importance of desktop environments accelerated their
> demise.

Another major change in that period was the general though gradual move
to UTF-8, making it somewhat unclear for some time which encoding should
be preferred when rendering cat pages.  (Since 2010, man-db always saves
cat pages in UTF-8 and converts to the proper encoding at display time,
but it took a while to settle on this approach and in the meantime there
were perhaps four or five years when cat pages were commonly unavailable
in practice.  Even then, very few people cared enough to complain.)

Furthermore, the traditional approach to saving system-wide cat pages
involved having man(1) be set-id.  From a modern standpoint, this was
obviously problematic, and it caused both security vulnerabilities and
more ordinary bugs.  There are ways in which this might have been
rearranged to be less of a serious problem, but if you can avoid
bothering with set-id at all then that's clearly safer.

My general approach to cat pages for at least the last ten years has
been to put as little effort into them as possible.  This has so far
included not outright removing support for them (since dealing with the
resulting support load, even if small, would itself be effort), but if
an improvement to man(1) has some kind of degradation of cat pages as a
side-effect then I usually won't hesitate to make it anyway.

> ...which brings me to the other factor, of which I'm more confident: man
> page rendering times are much lower than they were in Unix's early days.

Indeed, and it's been the case for at least a decade that rendering
times have been short enough that they can largely be considered
negligible.  (For most of that time my own equipment has not been
particularly on the bleeding edge of performance.)

> The bottom line is that, even on BSD systems (where mdoc(7) is preferred
> to man(7)), a user can expect a man page to render from *roff source in
> less than, say, half a second in the worst case, and the median
> GNU/Linux user can expect to start reading a man page "instantaneously":

The other thing to note explicitly here is that what normally matters
most is the time to _start_ reading, not the time to render the whole
page.  My usual example for where this makes a difference is zshall(1),
which is a concatenation of several other pages and comes to about 3
lines of 80-column rendered output; on my system this takes about 0.6
seconds to render in its entirety, but typing "man zshall" nevertheless
shows the first page subjectively instantaneously.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Chapters of the manual (was: Bug#1018737: /usr/bin/rst2man: rst2man: .TH 5th field shouldn't be empty)

[Colin Watson]

[Sorry for the delay; a succession of business travel, holidays, and
Covid have together eaten up much of my time recently.]

On Thu, Nov 17, 2022 at 01:28:12AM +0100, Alejandro Colomar wrote:
> On 11/17/22 01:06, G. Branden Robinson wrote:
> > I think the adoption of the term (sub)chapter has a potential benefit in
> > that it removes a terminological collision with (sub)sections as
> > subdivisions of individual man pages (man: SH, SS; mdoc: Sh, Ss).
> >
> > If this terminological reform is adopted, I think it should be done
> > across all of (1) Linux man-pages, (2) groff, (3) mandoc, and (4)
> > man-db.  If we can't speak with one voice on this then I think it's
> > better not to undertake that reform at all, to avoid frustrating the
> > discoverabilty of man pages.
> >
> > Possibly the biggest barrier to this is the mnemonic and documentation
> > of the man(1) '-s' option.  In man-db man, '-C' and '-c' are both
> > already in use.
> 
> That can be documented as a historical detail in the documentation of the
> option itself, which makes sense, as to avoid programmers that have heard of
> sections to try to grep section and find nothing.
> 
> > Probably a good idea to loop Colin Watson in on this proposal of yours,
> > which is strictly speaking severable from the below.
> 
> Yes, especially since part of the discussion is in linux-man@ (I'm not sure
> if he reads it; I think not) and not in groff@ (which he reads, AFAIK).  So,
> I'll merge the 2 discussions about this by forwarding the 2 most interesting
> other emails below.

I'm not subscribed to linux-man@, and while I am technically subscribed
to groff@ I read it very infrequently these days, so thanks for
explicitly copying me in.

> So, does it make sense to all of us to start using the term chapter for
> divisions of the man-pages single volume, so that the manual pages in Linux
> are organized from now on in chapters 1 to 9 instead of sections 1 to 9?

I find myself relatively agnostic on this whole discussion.  There are
good reasons for reform, and also some good reasons to wonder whether
the grass will in fact be greener on the other side (given the necessity
to keep many bits of "section" terminology around in things like
man(1)'s option handling and the man-db configuration file more or less
indefinitely).

I'm not going to veto it, but I also have no great enthusiasm for the
upheaval.  If the community consensus is for it to happen, then I'd ask
that somebody who is enthusiastic about it put the work into the various
necessary updates to man-db's code and documentation and send an
appropriate merge request.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: All caps .TH page title

[Colin Watson]

On Fri, Jul 22, 2022 at 01:16:49AM +0200, Alejandro Colomar wrote:
> On 7/21/22 20:36, G. Branden Robinson wrote:
> > At 2022-07-21T16:29:21+0200, Alejandro Colomar wrote:
> > > Also, does it have any functional implications?  I'm especially
> > > interested in knowing if that may affect in any way the ability of
> > > man(1) to find a page when invoked as `man TIMESPEC` for example.
> > 
> > My understanding is that mandb(8) indexes based solely on the second
> > argument to the `TH` macro call and (what it interprets as) the contents
> > of the "Name" (or "NAME") section of the page.  It parses *roff itself
> > as best it can to determine this.  So the fact that the _first_ argument
> > to `TH` might be in full caps doesn't deter it.  (It might in fact have
> > made mandb(8) authors' job easier if an "honest lettercase" practice had
> > arisen back in the day--but it didn't).
> 
> Okay.
> 
> > Since he's a mandb(8) author/maintainer, I would again defer to Colin
> > Watson's knowledge and expertise in this area.
> 
> Added to CC, in case he wants to intervene.

The above is not quite correct.  man-db doesn't index on the .TH section
at all, and I don't believe I've encountered the practice of doing so in
other indexers (I could be wrong, but I think that's something I would
have remembered if I'd noticed it).  Rather, it parses the "NAME" (or
"Name", or a number of localized variants) section of pages using the
man macro set for "foo \- description" lines and uses the left-hand side
of those for page names, or equivalently looks for .Nm requests in pages
using the mdoc macro set.

With the exception of handling localized variants of that section name,
which is a pretty ugly pile of special cases, I believe this to be
fairly traditional behaviour.  I can't say I would have done it that way
if I'd been designing the system from scratch since it really involves
far too much half-arsed parsing, but it seemed to be the usual thing to
do when I came on the scene.

Changing the arguments to .TH won't bother man-db at all.

> > > I'm not saying necessarily that I'd like to keep that behavior.  I
> > > wouldn't mind breaking it, if it means that users will be able to
> > > differentiate upper- and lowercase pages.  We're not in Windows (nor
> > > MacOS), anyway.

man-db's man(1) performs case-insensitive lookups by default, which I've
found to be more useful behaviour.  Case-sensitive lookup can be
requested using the -I/--match-case option.

As far as I know this was an innovation when I introduced it in 2002, so
I don't know how widespread this behaviour is among man(1)
implementations.  You probably can't rely on it.

But in any case, as above, changing the arguments to .TH doesn't affect
this.  I haven't noticed it being widespread practice to spuriously
capitalize the name part of lines in the "NAME" section.

Cheers,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [PROPOSAL] time zones and reproducible builds (was: How to make groff use local timezone?)

[Colin Watson]

On Tue, Dec 22, 2020 at 01:29:44PM +, Deri wrote:
> Please can someone explain why reproducible builds are important.

Hi,

It's probably best to simply refer you to
https://reproducible-builds.org/ for general motivation, and then you
can come back with any questions you have.

> What is the output of groff we should be testing.

Without loss of generality: the files that end up being distributed in
packaged form to users.

> Since these are essentially source files which are intended to be run
> at some point, diffing them just tells us that there has been a change
> in grops or gropdf, not whether that output, when run, has changed.

It's not a problem if a change in grops or gropdf (or whatever) induces
a change in the output: this is to be expected for almost any software.
The point is rather that you should be able to install the same versions
of the various bits of software involved in the build toolchain,
construct a suitably-documented build environment, and get bit-for-bit
identical output.  Whether the software involved is groff or TeX or gcc
or an artisanally-crafted pile of Python or whatever is immaterial: if
you can reproduce a build and produce bit-for-bit identical output, then
that helps to assure that the build infrastructure that produced the
binary packages you're using is sound.

For example, if somebody has replaced gropdf on some bit of build
infrastructure with gropdf-but-insert-evil-attack, then that can be
noticed quite easily if gropdf would ordinarily produce bit-for-bit
identical results across multiple runs.  But if gropdf inserts extra
information from its environment into the output, then the problem
becomes more difficult: now you have to work out how to filter out the
"expected" differences, and that problem is compounded if what you're
looking at isn't a pair of PDF files but rather a pair of .debs or RPMs
or MSI files or whatever that contain some PDFs somewhere inside them.
Bear in mind that this is the sort of problem that people want to tackle
in bulk at the scale of a whole software distribution, not at the level
of comparing individual rendered PDF files by hand.

Now, there is absolutely room for debate and compromise on exactly what
sorts of environmental constraints one needs to apply when reproducing a
build, hence things like
https://reproducible-builds.org/docs/source-date-epoch/ and working out
to what extent timezones should be taken into consideration.  As I
mentioned I'm certainly open to the possibility that when I patched
Debian's groff to in some sense force TZ=UTC I did so at the wrong
layer.  But I hope this explains why at least the principle is
important.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Is .rd implemented at all?

[Colin Watson]

On Wed, Dec 16, 2020 at 08:30:33PM -0600, Dave Kemper wrote:
> On 12/15/20, Dorai Sitaram  wrote:
> >  Thanks Dave, for the suggestion. That doesn't seem to be the problem,
> > however, on my machine (Ubuntu 20.10). No-argument cat works as expected.
> 
> Then something peculiar to the Ubuntu groff seems to be behind this.
> Ubuntu is a Debian-derived distro, and the concurrent "local timezone"
> thread here points out Debian's proclivity to customize groff in
> incompatible ways.

The example near the head of the thread works fine for me on both Debian
and Ubuntu, and this is not something that we customise.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: How to make groff use local timezone?

[Colin Watson]

On Wed, Dec 16, 2020 at 11:35:12AM +1100, G. Branden Robinson wrote:
> What do people think about a GROFF_USE_UTC environment variable that
> causes troff to call gmtime() instead of localtime()?

How would this differ from just setting TZ=UTC?

Also, this conversation seems to overlap substantially with
https://savannah.gnu.org/bugs/?57218, so I thought I'd remind people of
the existence of that report.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: How to make groff use local timezone?

[Colin Watson]

On Mon, Dec 14, 2020 at 02:12:28PM +1100, G. Branden Robinson wrote:
> At 2020-12-12T21:19:38-0800, Jim Avera wrote:
> > I suspect groff is always in UTC (but haven't confirmed).  
> 
> No, that is not true, at least not in groff as provided by GNU.  A
> downstream distributor, like a GNU/Linux distribution, could alter this.

Debian currently does
(https://salsa.debian.org/debian/groff/blob/master/debian/patches/display-utc-times.patch).
But I'm unsure about the correctness of this, which is why I hadn't
pushed it upstream; it was something of a desperation measure to get
reproducible builds working in more places, but changing the semantics
of things like \n[hours] is not ideal.

Perhaps playing whack-a-mole with TZ=UTC in more places would be better
in this case, although it seems unlikely to be much fun - I'm not sure.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Bug#284002: [PATCH] mdoc: Update operating system release numbers

[Colin Watson]

On Mon, Nov 23, 2020 at 03:02:49AM +1100, G. Branden Robinson wrote:
> At 2020-11-22T16:08:56+0100, Ingo Schwarze wrote:
> > Sure.  I dislike the concept of mdoc.local for more than one reason,
> > but probably it is good enough for this purposes if there is no
> > better way in Debian.  If mdoc.local gets automatically updated
> > during system updates, the proposed string also seems fine.  If it
> > is considered a user config file and does *not* get updated
> > automatically, then something like just "Debian GNU/Linux" might
> > be even better.
> 
> Hahaha. This is Debian...it's _both_ automatically updated during system
> updates _and_ considered a user config file!
> 
>   https://www.debian.org/doc/debian-policy/ap-pkg-conffiles.html
> 
> "conffiles" have been a dpkg feature for something like 25 years now.
> Every Debian sysadmin is familiar with the dpkg conffile prompt.  :D
[...]
> I suspect most people don't touch mdoc.local, so it will be
> automatically updated for them.
> 
> Colin, what do you think?

Putting this in mdoc.local would more or less work, but if I had to do
this then I'd lean slightly towards just patching mdoc itself to say the
right thing for the operating system.

However, if that were the recommendation for distributors then it would
seem likely to result in a lack of consistency.  Maybe it's worth having
a little bit of explicit support in upstream groff for what distributors
are supposed to do?  I'd suggest that groff's configure could gain a
--with-os-name option whose argument becomes the default value of .Os;
it can carry on defaulting to "BSD", or the output of uname(3), or
whatever else as you see fit.  That would (gently) encourage
distributors to set this in a systematic way.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: FWD: [bug #59276] [PATCH] #include "config.h" before

[Colin Watson]

On Thu, Oct 15, 2020 at 03:39:29PM +0200, Ingo Schwarze wrote:
> I'm looking for an OK to push the patch appended below.
> I think it can (and should) go in before release because
> including "config.h" in a few additional files is unlikely
> to cause regressions with compilers that do not require it.

This patch LGTM.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Releasing groff 1.22.5?

[Colin Watson]

On Sat, Oct 10, 2020 at 08:56:49PM +1100, G. Branden Robinson wrote:
> If anyone feels we haven't yet satisfied some technical goal that we
> should have accomplished before branding something "1.22.5", please
> speak up now.  I have some things I'd like to accomplish before a
> release[1], but based on my experience with groff 1.22.4, I don't think
> they'd interfere a beta or release-candidate cycle.

I generally think "release early, release often" is a good plan.

May I suggest bumping to 1.23?  I know groff doesn't practise strict
semver, but this would be justified in that scheme (there are several
new features).  More generally, I feel that there's no particular reason
to be nervous of bumping the minor number, especially given how long
it's been since 1.22.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [groff] 04/28: groff_char(7): Revise glyph descriptions.

[Colin Watson]

On Wed, Sep 02, 2020 at 11:36:36PM +1000, G. Branden Robinson wrote:
> > > -\[char223] \[char223] 223 germandbls u00DF German double s (sharp s)
> > > +\[char223] \[char223] 223 germandbls u00DF lowercase sharp s
> 
> [there is no uppercase Eszett]
> 
> Au contraire!  It's U+1E9E, added in Unicode 5.0.[1]

Also adopted by the Council for German Orthography in 2017
(https://www.rechtschreibrat.com/DOX/rfdr_Regeln_2016_veroeffentlicht_2017.pdf,
2.3 § 25).  It's not part of the orthography I learned in school either,
and I imagine there are a fair few native speakers who prefer not to use
it (it is ever thus), but it seems reasonable for groff to follow the
official rules.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: I'd like to help kill off some man(1) hacks

[Colin Watson]

On Mon, Aug 24, 2020 at 04:39:32PM -0500, Dave Kemper wrote:
> On 8/24/20, G. Branden Robinson  wrote:
> > If we can get the PS-and-PDF-font-embedding situation worked out, maybe
> > that would merit a version bump to 1.23,
> 
> Does GNU in general, or groff in particular, have hard and fast rules
> about what level of change warrants bumping which element of the
> version number?  The next groff release will already change the
> interpretation of some undelimited \s arguments (commit 0b9aaca0),
> which has the potential to affect some back compatibility.  By the
> strictest reading of the rules of semantic versioning
> (http://semver.org/), this requires bumping the major version number.

groff hasn't generally done semver for this sort of relatively limited
potential compatibility break, but I certainly think we shouldn't be
scared of bumping the minor number.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: [man-db, groff] I'd like to help kill off some man(1) hacks

[Colin Watson]

On Sat, Aug 22, 2020 at 04:42:34AM +1000, G. Branden Robinson wrote:
> In man-db's src/man.c, we see the following:
[...]
> The above is to support the --nh and --nj options.
> 
> I've pushed some changes to groff's man macros that enable greatly
> improved user control of adjustment and hyphenation.  However, it
> interacts badly with the above because of the redefinition of the .hy
> and .ad requests.
> 
> It should be possible in the future for man-db man, #ifdef
> TROFF_IS_GROFF, to discard injection of requests into groff's input
> stream in favor of simply calling it with options.
> 
>   man --nh -> groff -man -rHY=0
>   man --nj -> groff -man -dAD=l
> 
> I get the feeling you'd have happily used these facilities if they
> existed in 2008-2009 when you implemented the man(1) options.

I'd definitely be in favour of simplifying this if possible.

What about mdoc, though?  man doesn't know which macro set the manual
page uses when it invokes groff.  (It could try to discover it, but
that's a whole new set of complexity.)  Admittedly, --no-justification
is a no-op for mdoc anyway, but --no-hyphenation isn't.

> What do you think?  No one has mentioned a release time table for groff
> 1.22.5, and Bertrand is pretty busy with life and fatherhood lately, so
> there should be plenty of time to coördinate migration strategies.  I'm
> also mindful of the possibility that it would suck for you to have to
> query an installed groff for its version string, parse it, and adapt to
> it, so I'm open to suggestions.

Yeah, this is a bit tricky.  But ... although it does suck, it's not
unheard of for man to use \n[.x], \n[.y], and \n[.Y] to work out the
groff version at run-time (see the horror that is locale_macros).
Perhaps we could do that here so that the injected requests are a no-op
with a sufficiently new version of groff, and then drop those at some
point in the future when old versions of groff are no longer interesting
to support?

Thanks,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Groff macro to make .UR and .UE links clickable in PDF?

[Colin Watson]

On Fri, Jul 10, 2020 at 11:26:46AM -0400, Steve Izma wrote:
> I think it's an abomination that a man page extends it's line
> length to fit the width of the terminal; built into the macros
> should be a 65- or 70 character maximum width.

I'd be willing to take a bug report about the way that man-db does this
by default; it's a change I adopted from Andries Brouwer's man way back
in 2001, and I'm certainly prepared to entertain the idea that a change
I made nearly half my life ago might have been wrong.  (However, I would
like it as a bug report on e.g.
https://savannah.nongnu.org/bugs/?group=man-db rather than buried in a
mailing list thread, because my ability to keep track of random threads
isn't quite what it used to be.)

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: weird \s

[Colin Watson]

On Wed, May 20, 2020 at 09:48:27PM +, Bjarni Ingi Gislason wrote:
[...]

Would you please consider the great merits of brevity, specifically of
not appending over a hundred lines of condescending quotations to your
emails?  And I think the level of aggression on display here is well out
of order for this list.  It is not necessary to treat other contributors
this way.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: Weird font issue with .fam BM with groff -Tpdf

[Colin Watson]

On Tue, Apr 28, 2020 at 10:32:01AM -0400, T. Kurt Bond wrote:
> I applied the patch and did a make clean and make and make install and
> tried formatting the file again with -Tpdf and the fonts are right.
> Thanks very much!

Great, thanks.  I've applied this patch to master.

-- 
Colin Watson   [cjwat...@debian.org]

Re: Weird font issue with .fam BM with groff -Tpdf

[Colin Watson]

On Mon, Apr 27, 2020 at 06:30:07PM -0400, T. Kurt Bond wrote:
> I have a troff file:
[...]
> When I do
> 
> groff -Tps try-bm.tr | ps2pdf - try-bm-Tps.pdf
> 
> there is no issue with the resulting PDF file.
> 
> When I do
> 
> groff -Tpdf try-bm.tr >try-bm-Tpdf.pdf
> 
> the PDF file that results has the text that starts out "This is
> Regular." set in italic and the text that starts "This is Italic." set
> in Regular/Roman.

Thanks for reporting this.  Some font file names in devpdf seem to be
switched around for no reason I can readily discern.  Can you try the
following patch, please?  (You may need to do a clean build, at least of
font/devpdf/; some of the Makefile dependencies don't seem entirely
correct for doing an incremental build after changing this file.)

diff --git a/font/devpdf/Foundry.in b/font/devpdf/Foundry.in
index 70ff7246..64ad3b42 100644
--- a/font/devpdf/Foundry.in
+++ b/font/devpdf/Foundry.in
@@ -31,8 +31,8 @@ 
AI|NURWGothic-BookOblique.t1!URWGothic-BookOblique!URWGothicL-BookObli!a0100
 AR|NURWGothic-Book.t1!URWGothic-Book!URWGothicL-Book!a010013l.pfb
 BMB|NURWBookman-Demi.t1!URWBookman-Demi!URWBookmanL-DemiBold!b018015l.pfb
 
BMBI|NURWBookman-DemiItalic.t1!URWBookman-DemiItalic!URWBookmanL-DemiBoldItal!b018035l.pfb
-BMI|NURWBookman-Light.t1!URWBookman-Light!URWBookmanL-LighItal!b018032l.pfb
-BMR|NURWBookman-LightItalic.t1!URWBookman-LightItalic!URWBookmanL-Ligh!b018012l.pfb
+BMI|NURWBookman-LightItalic.t1!URWBookman-LightItalic!URWBookmanL-LighItal!b018032l.pfb
+BMR|NURWBookman-Light.t1!URWBookman-Light!URWBookmanL-Ligh!b018012l.pfb
 CB|YNimbusMonoPS-Bold.t1!NimbusMonoPS-Bold!NimbusMonL-Bold!n022004l.pfb
 
CBI|YNimbusMonoPS-BoldItalic.t1!NimbusMonoPS-BoldItalic!NimbusMonL-BoldObli!n022024l.pfb
 
CI|YNimbusMonoPS-Italic.t1!NimbusMonoPS-Italic!NimbusMonL-ReguObli!n022023l.pfb

-- 
Colin Watson   [cjwat...@debian.org]

Re: discrepant groff configurations

[Colin Watson]

On Fri, Mar 27, 2020 at 01:28:43PM +, Ralph Corderoy wrote:
> (Actually, the existence of /usr/share/groff/current seems a bit odd.
> Most packages don't have a symbolic link to support multiple installed
> versions and users that want multiple versions of a program installed
> can use GNU Stow or similar to maintain the symlinks if their package
> manager doesn't provide it.)

Really, the weird thing is not so much the existence of the current
symlink, as the fact that the directories under $(dataprogramdir) are
versioned.  The current symlink is an attempt to cope with that so that
other bits of software that for some reason need to look at files in
tmac/ can do so without having to keep track of the version number; but
it's quite possible that it would have been better to abolish the
versioned directories.

Anyway, groff's build system was more or less completely rewritten
between 1.22.3 and 1.22.4 (Bertrand's automake conversion), so that will
certainly confound any attempts to compare different distributions if
you aren't comparing like version for like.

-- 
Colin Watson   [cjwat...@debian.org]

Re: eqn sqrt and pdf?

[Colin Watson]

On Sun, Mar 01, 2020 at 12:27:47PM +, Deri wrote:
> The download file is created by the program BuildFoundries which used to be 
> left in the font/devpdf/util directory after install, no longer. Perhaps this 
> ought to be installed in the bin directory during install. If the ghostscript 
> version changes then you could cd into the the font/devpdf directory and run 
> BuildFoundries.

Thanks, but this is unlikely to be a very helpful approach for files
built as part of distribution-managed packages, where the shipped files
are normally expected to be immutable (for example, checksumming the
installed files would fail after doing that kind of thing).

It would be preferable to make the download file independent of the
ghostscript version, if possible.  If that isn't possible, I'll need to
figure out some way to annotate the implied dependency in my packages'
metadata, in order that we can know when we need to rebuild groff.

-- 
Colin Watson   [cjwat...@debian.org]

Re: eqn sqrt and pdf?

[Colin Watson]

On Sat, Feb 29, 2020 at 06:41:16PM +, Ralph Corderoy wrote:
> Hi Marc,
> > locate -bei StandardSymbolsPS
> > /usr/share/ghostscript/9.27/Resource/Font/StandardSymbolsPS
> >
> > you fixed this by seding the download file which doesn't exist in the
> > debian distro.

Not so:

  $ dpkg -L groff | grep download
  /usr/share/groff/1.22.4/font/devpdf/download

> > Now i feel this is worth filing 2 bugs on the debian tracker:
> ...
> > * troff should use the current version of ghostscript
> 
> I'd guess so.
> 
> Debian's groff maintainer, Colin Watson, is a subscriber here, and may
> pipe up soon.

Apparently the current ghostscript version at build time gets built into
/usr/share/groff/1.22.4/font/devpdf/download by way of groff's build
process.  I hadn't noticed that before.  Do please file a Debian bug
report.  For stable update purposes, a simple rebuild will suffice.  For
ongoing purposes, we may want to look at figuring out how to avoid
encoding the version in font/devpdf/download.

-- 
Colin Watson   [cjwat...@debian.org]

Re: man Macro Package and pdfmark

[Colin Watson]

On Mon, Feb 17, 2020 at 08:56:25PM +1100, John Gardner wrote:
> That's not what I'm talking about. In Emacs, I'm used to smashing `c-h o`
> to bring up the documentation for the symbol at point. In info(1), I've no
> idea where or what to even begin searching for to find a symbol's
> documentation. info(1) without Emacs feels like a fish out of the water.

I mean, I'm not info(1)'s greatest fan, but it's really not that bad if
you spend a trivial amount of time reading its own docs, which it points
you at at the bottom of the screen when you start the program.  For
example, under "Index Commands", "i", "I", and "M-x index-apropos" seem
helpful.

FWIW, I'm not even slightly an emacs fan (nor user).

-- 
Colin Watson   [cjwat...@debian.org]

Re: mdoc patch: accept any number of arguments for .Dd

[Colin Watson]

On Thu, Jan 16, 2020 at 06:56:09PM +0100, Ingo Schwarze wrote:
> TLDR: The point is that i have seen several manuals in a number of
> different operating systems, both historical (around 1990) and modern
> (around 2020) that give one or two arguments to .Dd.  From the
> content of these arguments, it is obvious that they all expect the
> date to be passed through unchanged.  I don't recall ever having
> seen a manual page that passes one, two, or four arguments and
> expects the current behaviour of replacing the arguments with the
> date of formatting.

The behaviour change as described seems reasonable to me, although I
haven't reviewed the code.  However, is there some good reason why this
patch doesn't include a matching change to the documentation in
tmac/groff_mdoc.7.man?

-- 
Colin Watson   [cjwat...@debian.org]

Re: pic anomalies

[Colin Watson]

On Mon, Dec 30, 2019 at 06:31:34PM +, Ralph Corderoy wrote:
> It's idiomatic in C to write just `foo' to test it against its `zero
> value' whether it's an int, pointer, etc.

This argument holds less well in this case because do_sprintf has
several other comparisons to '\0'; so I'd be inclined to agree with Ingo
that an explicit != '\0' is better at least in this case.  I have a
patch in progress that I'll get round to sending, modulo holidays.

I confess a decreasing interest in general code style debates and an
increasing interest in maintaining local code style consistency as I get
older. :-)

-- 
Colin Watson   [cjwat...@debian.org]

Re: GNUism in groff tests

[Colin Watson]

On Mon, Dec 30, 2019 at 06:12:58PM +0100, Ingo Schwarze wrote:
> here is a patch to make the test suite portable.
> 
> Do you think something like this should be done?

I'm generally in favour of making shell scripts be /bin/sh where
feasible.  Your patch looks fine to me (though I think I agree with
Ralph's suggested improvement).

-- 
Colin Watson   [cjwat...@debian.org]

Re: pic anomalies

[Colin Watson]

On Mon, Dec 30, 2019 at 01:39:38PM +0100, Ingo Schwarze wrote:
> An extremely small number of tests (14 tests grand total) is scattered
> all around the groff source tree, for example:
> 
>  - contrib/gdiffmk/tests/gdiffmk_tests.sh
>  - contrib/hdtbl/examples/test-hdtbl.sh.in
>  - contrib/mom/examples/test-mom.sh.in
>  - src/roff/groff/tests/
>  - tmac/tests/
> 
> You can run that suite with "make check" from the top directory,
> but it is not much use because it is very fragile.  For example,
> right now, after "git pull" and building from source, nine out
> of the fourteen tests fail for me on OpenBSD-current, so it's at
> least about 65% broken.

FWIW, I run these tests as part of the Debian package build and enforce
that they pass.  This is obviously only of limited use since it's only
after releases, but it's better than nothing.  (I would hope that
maintainers also run "make check" prior to release, perhaps via "make
distcheck", although I don't actually know.)

I agree that some of these tests tend to be somewhat fragile,
particularly those that look at properties of generated PDFs.  That
said, with current git master on Debian unstable, all the tests pass for
me right now.

-- 
Colin Watson   [cjwat...@debian.org]

Re: pic anomalies

[Colin Watson]

On Sun, Dec 29, 2019 at 04:33:59PM +0100, Ingo Schwarze wrote:
> Regarding the bugfix in the code, thanks to Larry McVoy for
> providing feedback directly to me.  In particular, he suggested
> that while my fix wasn't incorrect, the two lines immediately
> following it can be simplified, making the code easier to read.
> 
> There may be opportunity for major cleanup in this code, too,
> but i don't think that should be mixed into a bugfix.
> The simplification of these two lines, however, is so closely
> related to the bugfix that it seems reasonable to include it.
> 
> Any OKs for the patch in the following form?

LGTM as far as it goes.  If I were doing it I think I'd take the
opportunity to simplify it a little further into this sort of structure:

  if (*form == '%') {
form++;
result += '%';
  }
  else {
...
snprintf(sprintf_buf, sizeof(sprintf_buf),
 one_format.contents(), v[i++]);
result += sprintf_buf;
  }
  one_format.clear();

But I understand if you'd prefer not to do that here.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [PATCH] mdoc: Update operating system release numbers

[Colin Watson]

On Sat, Dec 21, 2019 at 02:51:23PM +0100, Ingo Schwarze wrote:
> Consequently, i just pushed the patch to the upstream groff repository,
> with the following tweaks:
> 
>  * I included the following versions which appeared to be missing:
> - NetBSD 6.0.6 and 7.2
> - DragonFly 3.0.2 and 3.2.2
> 
>  * I did *not* include the following releases because x.y.0 are not
>included for any other DragonFly releases: DragonFly 3.6.0 and 3.8.0.
>I'm not a DragonFly user and i'm not completely sure what would make
>most sense for that system, so i just stuck to existing practice as
>best i could.

Thanks.  I have no complaints with these - I wasn't completely sure what
I was doing and am happy for the corrections.

(There was a small typo, which I've fixed on master.  It's very easy for
one's eyes to swim when dealing with these macros.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [PATCH] mdoc: Update operating system release numbers

[Colin Watson]

On Tue, Dec 17, 2019 at 01:14:06PM +, Colin Watson wrote:
> Based on a patch from Guillem Jover .
> 
> * tmac/doc-common-u: Update NetBSD, OpenBSD, FreeBSD, Darwin, and
> DragonFly version strings.
> 
> * tmac/groff_mdoc.7.man: Synchronize.

Side note: I am not the biggest fan of this business of encoding a bunch
of other projects' release history in groff, so please don't take me as
an advocate of that.  However, I am generally an advocate of the
position that if one is going to encode this sort of thing then it makes
sense to keep it up to date.

-- 
Colin Watson   [cjwat...@debian.org]

[PATCH] mdoc: Update operating system release numbers

[Colin Watson]

Based on a patch from Guillem Jover .

* tmac/doc-common-u: Update NetBSD, OpenBSD, FreeBSD, Darwin, and
DragonFly version strings.

* tmac/groff_mdoc.7.man: Synchronize.

Fixes: https://bugs.debian.org/867123
---
 tmac/doc-common-u | 120 ++
 tmac/groff_mdoc.7.man |  35 +++-
 2 files changed, 143 insertions(+), 12 deletions(-)

diff --git a/tmac/doc-common-u b/tmac/doc-common-u
index 0d2e418d..1da0470f 100644
--- a/tmac/doc-common-u
+++ b/tmac/doc-common-u
@@ -504,6 +504,15 @@
 .ds doc-operating-system-NetBSD-6.1.2 6.1.2
 .ds doc-operating-system-NetBSD-6.1.3 6.1.3
 .ds doc-operating-system-NetBSD-6.1.4 6.1.4
+.ds doc-operating-system-NetBSD-6.1.5 6.1.5
+.ds doc-operating-system-NetBSD-7.0   7.0
+.ds doc-operating-system-NetBSD-7.0.1 7.0.1
+.ds doc-operating-system-NetBSD-7.0.2 7.0.2
+.ds doc-operating-system-NetBSD-7.1   7.1
+.ds doc-operating-system-NetBSD-7.1.1 7.1.1
+.ds doc-operating-system-NetBSD-7.1.2 7.1.2
+.ds doc-operating-system-NetBSD-8.0   8.0
+.ds doc-operating-system-NetBSD-8.1   8.1
 .
 .ds doc-operating-system-OpenBSD-2.0  2.0
 .ds doc-operating-system-OpenBSD-2.1  2.1
@@ -542,6 +551,16 @@
 .ds doc-operating-system-OpenBSD-5.4  5.4
 .ds doc-operating-system-OpenBSD-5.5  5.5
 .ds doc-operating-system-OpenBSD-5.6  5.6
+.ds doc-operating-system-OpenBSD-5.7  5.7
+.ds doc-operating-system-OpenBSD-5.8  5.8
+.ds doc-operating-system-OpenBSD-5.9  5.9
+.ds doc-operating-system-OpenBSD-6.0  6.0
+.ds doc-operating-system-OpenBSD-6.1  6.1
+.ds doc-operating-system-OpenBSD-6.2  6.2
+.ds doc-operating-system-OpenBSD-6.3  6.3
+.ds doc-operating-system-OpenBSD-6.4  6.4
+.ds doc-operating-system-OpenBSD-6.5  6.5
+.ds doc-operating-system-OpenBSD-6.6  6.6
 .
 .ds doc-operating-system-FreeBSD-1.0 1.0
 .ds doc-operating-system-FreeBSD-1.1 1.1
@@ -608,6 +627,16 @@
 .ds doc-operating-system-FreeBSD-9.2 9.2
 .ds doc-operating-system-FreeBSD-9.3 9.3
 .ds doc-operating-system-FreeBSD-10.010.0
+.ds doc-operating-system-FreeBSD-10.110.1
+.ds doc-operating-system-FreeBSD-10.210.2
+.ds doc-operating-system-FreeBSD-10.310.3
+.ds doc-operating-system-FreeBSD-10.410.4
+.ds doc-operating-system-FreeBSD-11.011.0
+.ds doc-operating-system-FreeBSD-11.111.1
+.ds doc-operating-system-FreeBSD-11.211.2
+.ds doc-operating-system-FreeBSD-11.311.3
+.ds doc-operating-system-FreeBSD-12.012.0
+.ds doc-operating-system-FreeBSD-12.112.1
 .
 .ds doc-operating-system-Darwin-8.0.0  8.0.0
 .ds doc-operating-system-Darwin-8.1.0  8.1.0
@@ -654,6 +683,44 @@
 .ds doc-operating-system-Darwin-13.3.0 13.3.0
 .ds doc-operating-system-Darwin-13.4.0 13.4.0
 .ds doc-operating-system-Darwin-14.0.0 14.0.0
+.ds doc-operating-system-Darwin-14.1.0 14.1.0
+.ds doc-operating-system-Darwin-14.2.0 14.2.0
+.ds doc-operating-system-Darwin-14.3.0 14.3.0
+.ds doc-operating-system-Darwin-14.4.0 14.4.0
+.ds doc-operating-system-Darwin-14.5.0 14.5.0
+.ds doc-operating-system-Darwin-15.0.0 15.0.0
+.ds doc-operating-system-Darwin-15.1.0 15.1.0
+.ds doc-operating-system-Darwin-15.2.0 15.2.0
+.ds doc-operating-system-Darwin-15.3.0 15.3.0
+.ds doc-operating-system-Darwin-15.4.0 15.4.0
+.ds doc-operating-system-Darwin-15.5.0 15.5.0
+.ds doc-operating-system-Darwin-15.6.0 15.6.0
+.ds doc-operating-system-Darwin-16.0.0 16.0.0
+.ds doc-operating-system-Darwin-16.1.0 16.1.0
+.ds doc-operating-system-Darwin-16.2.0 16.2.0
+.ds doc-operating-system-Darwin-16.3.0 16.3.0
+.ds doc-operating-system-Darwin-16.4.0 16.4.0
+.ds doc-operating-system-Darwin-16.5.0 16.5.0
+.ds doc-operating-system-Darwin-16.6.0 16.6.0
+.ds doc-operating-system-Darwin-17.0.0 17.0.0
+.ds doc-operating-system-Darwin-17.1.0 17.1.0
+.ds doc-operating-system-Darwin-17.2.0 17.2.0
+.ds doc-operating-system-Darwin-17.3.0 17.3.0
+.ds doc-operating-system-Darwin-17.4.0 17.4.0
+.ds doc-operating-system-Darwin-17.5.0 17.5.0
+.ds doc-operating-system-Darwin-17.6.0 17.6.0
+.ds doc-operating-system-Darwin-17.7.0 17.7.0
+.ds doc-operating-system-Darwin-18.0.0 18.0.0
+.ds doc-operating-system-Darwin-18.1.0 18.1.0
+.ds doc-operating-system-Darwin-18.2.0 18.2.0
+.ds doc-operating-system-Darwin-18.3.0 18.3.0
+.ds doc-operating-system-Darwin-18.4.0 18.4.0
+.ds doc-operating-system-Darwin-18.5.0 18.5.0
+.ds doc-operating-system-Darwin-18.6.0 18.6.0
+.ds doc-operating-system-Darwin-18.7.0 18.7.0
+.ds doc-operating-system-Darwin-19.0.0 19.0.0
+.ds doc-operating-system-Darwin-19.1.0 19.1.0
+.ds doc-operating-system-Darwin-19.2.0 19.2.0
 .
 .ds doc-operating-system-DragonFly-1.01.0
 .ds doc-operating-system-DragonFly-1.11.1
@@ -688,14 +755,67 @@
 .ds doc-operating-system-DragonFly-2.12   2.12
 .ds doc-operating-system-DragonFly-2.13   2.13
 .ds doc-operating-system-DragonFly-3.03.0
+.ds doc-operating-system-DragonFly-3.0.1  3.0.1
 .ds doc-operating-system-DragonFly-3.13.1
 .ds doc-operating-system-DragonFly-3.23.2
+.ds doc-operating-system-DragonFly-3.2.1  3.2.1
 .ds doc-operating-system-DragonFly-3.3   

Re: Problem with configure

[Colin Watson]

On Tue, Nov 05, 2019 at 12:39:21PM -0600, Blake McBride wrote:
> I am attempting to build the latest rpo copy of groff on 64-bit LinuxMint
> 19.2.  I am running:
> 
> ./bootstrap
> 
> ./configure --prefix=/usr
[...]
> config.status: error: cannot find input file: `Makefile.in'

I expect that this failed simply because the previous bootstrap command
had failed, as described in the other thread you posted today.  If you
git pull, it should work now.

-- 
Colin Watson   [cjwat...@debian.org]

Re: Error in bootstrap

[Colin Watson]

On Tue, Nov 05, 2019 at 12:31:45PM -0600, Blake McBride wrote:
> While trying to build the latest git repo on a 64 bit Linux machine, I get
> the following when running bootstrap:
> 
> contrib/mom/mom.am:109: error: 'nodist_momprocessedexample_DATA' is used
> but 'momprocessedexampledir' is undefined

Thanks for your report.  This was a clear typo in
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=e80baeaf2654feee6e69c489ea763d11968153aa,
so I went ahead and pushed a fix:

  
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=da42ae7cf8d36fb689f5109dbdcb72b074b99517

CCing Peter, FYI.

-- 
Colin Watson   [cjwat...@debian.org]

Re: Two questons: Norwgegian characters and space when switching typeface

[Colin Watson]

On Mon, Nov 04, 2019 at 01:31:06AM -0800, Dale Snell wrote:
> For the Norwegian characters, you'll need to use constructions like "\[o/]"
> and "\[ae]".

Alternatively, run your input file through "preconv" as the first
preprocessor.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Getting refer to pass through special Unicode characters

[Colin Watson]

On Tue, Sep 24, 2019 at 02:21:29PM +, Dorai Sitaram via groff wrote:
> I have some some Unicode characters with codes higher than 256 (e.g.,
> smart quotes) that 'refer' chokes on with the message "invalid input
> character code".  
> Is there a way to tell refer to just pass them through to stdout? The
> error happens even if the offending characters don't occur inside the
> refer-specific .[ and .] or in the bibliography database. They are
> just in the "surrounding" text that isn't relevant to refer anyway.

troff itself does this too, doesn't it?  I think the way you're supposed
to handle this at the moment is to run preconv before other
preprocessors, including refer.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] 04/05: {g, n}roff.1.man: Give assistance to pager users.

[Colin Watson]

On Wed, Jul 03, 2019 at 11:39:32AM +1000, John Gardner wrote:
> I've lost the link, but I remember somebody got hold of a hard-copy
> terminal somehow and used it to display his Twitter's newsfeed by threading
> output through a serial port hooked up to his Linux workstation.
> 
> The output wasn't much to look at, but the effort somebody went to to
> connect two technologies invented 50 years apart was seriously inspiring.

In university we did a group project to build an EDSAC emulator, as part
of the Cambridge Computer Lab's celebrations of the 50th anniversary of
EDSAC's first running programs.  (This is probably enough information to
deduce my age quite accurately: I suspect younger than many here.)

When the original machine was built, debuggers weren't a thing yet, so
as a small affordance for users they connected a speaker to the
accumulator, and for example when you ran the prime-listing program you
could reportedly hear it counting off the rhythm of the primes.  As far
as we could tell nobody had ever made a recording of this, so we could
only guess at what it might have sounded like, but we certainly wanted
this feature in our emulator, even though this was a thing written 50
years later by a bunch of second-years in Java with only the most
tenuous connection to the original.

Best guess: shove the bits out what amounted to a serial port connected
to the sound card and cross our fingers, and it was enough to be able to
likewise hear the primes being counted off.  We eventually got somebody
who'd worked on the original machine to come and listen to it: he told
us that it really sounded nothing like the original but it clearly
fulfilled the same purpose.  Good enough for us considering what we had
to work with.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [patch] do not strip mdoc macros

[Colin Watson]

On Thu, Mar 14, 2019 at 12:43:53PM -0400, Doug McIlroy wrote:
> Bjarni Ingi Gislason wrote:
> > It is objectively correct to strip the macro files of bytes that are
> > meaningless for the program "groff"
> >
> > It is objectively correct to _not_ strip the macro files of bytes
> > that have a meaning for humans.
> >
> > So provide both versions.
> 
> The conclusion is compatible with the premises but doesn't 
> follow from them.
> 
> For human consumption, the comments are desirable. For machine
> consumption the comments are harmless. So there is no compelling
> need for the latter. For maintainability and intelligibility of
> the groff distro, smaller is better.

Quite.

It would follow from Bjarni's argument, if accepted, that it is
objectively correct (a strong claim!) to install programs in scripting
languages on user systems only after first stripping comments from them
(or perhaps even only after also running them through a compiler, where
available).  Now, there certainly are people who take that position; but
it is very far from being the dominant or accepted position at least in
free software circles, much less something that one can claim with a
straight fact to be "objectively correct".

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [PATCH] Avoid Perl's unsafe "<>" operator

[Colin Watson]

On Sun, Mar 03, 2019 at 01:21:49PM +, Ralph Corderoy wrote:
> Hi Colin,
> 
> > Perl's "open" documentation even notes for the "< $file\0" trick that
> > "this may not work on some bizarre filesystems", which suggests to me
> > that such a robust proof isn't possible.
> 
> No, that's a misquote.
> 
>   $file =~ s#^(\s)#./$1#;
>   open(my $fh, "< $file\0")
>   || die "Can't open $file: $!";
> 
> (this may not work on some bizarre filesystems).
> 
> The `bizarre' bit is referring to non-POSIX filesystems that don't like
> `./' as a prefix for the current directory.

Thanks for the correction (I would say it was a misreading rather than a
misquote, since it genuinely wasn't clear to me).

It's true that the ./ part doesn't need to be considered in our case.

> > we're trying to come up with ways to add extra characters to its input
> > that suppress that magic.
> 
> We're using the method described in documentation written by Tom
> Christiansen and his pedigree says to me that's sufficient.  I agree in
> general one looks for a certain way to be safe, but this seems the
> accepted Perl idiom.

Tom indeed has a fine pedigree, but even very competent people can be
wrong, which is why I look for convincing proofs instead.  To be honest,
even with a proof I think I would still prefer my manual construction,
since it requires less effort to convince oneself that it's safe.

> 
> https://groups.google.com/forum/message/raw?msg=comp.lang.perl.misc/0tqkwns3aaw/auWXakdD1S0J
> 
> If magic `open' is a bit too magical for you, you don't have to turn
> to `sysopen'.

It's worth noting that Tom posted this in 1999, which was before the
three-argument form of open was introduced in Perl 5.6 (March 2000).
Nowadays it would be strange (perhaps even a strawman) to make this
argument, because if "magic" two-argument open is too magical for you,
then the obvious next thing to use is the non-magical three-argument
open, not sysopen.

So, even if that document from 1999 remains authoritative on how to
perform safe escaping, I don't think it's any longer authoritative on
style.  The modern perlopentut(1) still carries Tom's copyright notice
(now from 2013) and doesn't mention the two-argument form of open at
all; that's been relegated to more detailed reference documentation.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [PATCH] Avoid Perl's unsafe "<>" operator

[Colin Watson]

Thanks for the review.

On Fri, Mar 01, 2019 at 04:41:36PM +, Deri wrote:
> On Thursday, 28 February 2019 19:42:45 GMT Colin Watson wrote:
> > On Thu, Jan 24, 2019 at 02:34:35PM +0000, Colin Watson wrote:
> > > The "<>" operator is implemented using the two-argument form of "open",
> > > which interprets magic such as pipe characters, allowing execution of
> > > arbitrary commands which is unlikely to be expected.  Perl >= 5.22 has a
> > > "<<>>" operator which avoids this, but also forbids the use of "-" to
> > > mean the standard input, which is a facility that the affected groff
> > > programs document.
> > 
> > [...]
> > 
> > Has anyone had a chance to review this patch (also in
> > https://savannah.gnu.org/bugs/?7, after Deri's suggestion)?  Should
> > I just go ahead and commit it?
> > 
> > I'm going to upload this patch to Debian unstable shortly in the cause
> > of getting release-critical bug fixes in ahead of our upcoming full
> > freeze, but it would be better to get it into upstream as well.
> 
> Hi Colin,
> 
> There appear to be a lot of extra changes in the patch which are not to do 
> with what we are trying to fix.

No, despite appearances there are no unrelated changes here.  I
explained what's going on in detail in the patch's commit message:
because my change involves introducing an extra level of indentation, it
looks like there's a lot going on when you look at the raw patch, but
the commit message explains the actual effective changes, and if you
apply it to a local branch and use something like "git show -b" you
should see something that's easier to review.

> There may also be a problem with the gropdf patch. One aspect of using "<>" 
> is 
> that if there are multiple files on the command line an eof is not signalled 
> between the files, i.e. after reading the last line of the first file the 
> next 
> read will be the first line of the next file. This may not have an impact but 
> the read in the LoadAhead subroutine may be done on a file which is at eof, 
> rather than the first line of the next file. I admit this may not cause an 
> issue in normal operation but is a change in behaviour.

Hm, OK.  Could you suggest some example input that I can use to provoke
this?  gropdf is the only groff program that splits up the input reading
in such a way as to have this kind of problem, but I'd be happy to try
to work out a fix given a test case.

> I prefer the first solution you suggested, upon which my code was based, 
> because there will be no change of behaviour. I have been unable to find a 
> way 
> of defeating this protection method to make "<>" safe. Do you know of a way 
> to 
> circumvent it?

I don't have a specific way to circumvent it, and it may well be correct
in most cases.  It's true that it's an easier change.  However, I think
the phrasing of your question puts the burden of proof in the wrong
place.  The problem at hand is that argument handling is more magical
than expected.  Can you *prove* (not just by absence of evidence) that
the approach of pre-escaping @ARGV effectively disables the dangerous
parts of <>'s magic in the fact of arbitrary malicious arguments,
ensuring that each argument is only ever interpreted as a file of the
exact same name, or standard input in the case of "-"?  Is such a proof
likely to be robust against future innocent changes to the code by
people who don't remember this thread?  Perl's "open" documentation even
notes for the "< $file\0" trick that "this may not work on some bizarre
filesystems", which suggests to me that such a robust proof isn't
possible.

What we have here is a particularly magical (in the sense often used by
the Perl documentation) language facility with an assortment of special
behaviours, and we're trying to come up with ways to add extra
characters to its input that suppress that magic.  I argue that - if
we're trying to construct a secure system, which I hope we are - this is
fundamentally the wrong approach.  It's instead much better to use
language facilities that don't have these magical behaviours in the
first place.  This is true even if it takes a little extra effort to
come up with something that exactly matches the subset of the magical
behaviours that we actually want, because then at least our uses of
those behaviours will be explicit and we'll be able to understand them.

An analogy: by now it's fairly well-understood that Perl programmers
should avoid "system $string" (which passes $string to "sh -c") and
prefer "system @list

Re: [groff] [PATCH] Avoid Perl's unsafe "<>" operator

[Colin Watson]

On Thu, Jan 24, 2019 at 02:34:35PM +, Colin Watson wrote:
> The "<>" operator is implemented using the two-argument form of "open",
> which interprets magic such as pipe characters, allowing execution of
> arbitrary commands which is unlikely to be expected.  Perl >= 5.22 has a
> "<<>>" operator which avoids this, but also forbids the use of "-" to
> mean the standard input, which is a facility that the affected groff
> programs document.
[...]

Has anyone had a chance to review this patch (also in
https://savannah.gnu.org/bugs/?7, after Deri's suggestion)?  Should
I just go ahead and commit it?

I'm going to upload this patch to Debian unstable shortly in the cause
of getting release-critical bug fixes in ahead of our upcoming full
freeze, but it would be better to get it into upstream as well.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Upgrade to 1.22.4 breaks PDF font

[Colin Watson]

On Tue, Feb 19, 2019 at 05:00:43PM +, Richard Morse wrote:
> Hi! I just upgraded to groff 1.22.4, and suddenly my files are not
> turning into PDFs properly. The error is "can't find font NR" (and
> "NI"). This is on Mac OS X 10.13, with the groff installed by
> Homebrew.
> 
> Looking at /usr/local/share/groff/1.22.4/font/devpdf, especially when
> comparing it with devps, shows that there are many fewer fonts for the
> PDF than for the PS device (which I don't think is supposed to
> happen). The "NR" and "NI" fonts were definitely there in 1.22.3.

It's likely that this is a bug in the Homebrew formula: you can confirm
this by looking for "URW fonts for pdf" in the configure output (it
should say "yes", but I bet it says "no").  The formula needs to make
sure that whatever provides a010013l.pfb is installed when groff is
built.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Loss of MSVC support

[Colin Watson]

On Thu, Feb 14, 2019 at 07:30:13PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 14 Feb 2019 12:50:09 +
> > From: Colin Watson 
> > 
> > If it's just the runtime, then Gnulib should be able to paper over a
> > pretty fair number of the differences, and groff already uses that.
> 
> Up to a degree.  There's no fork for Windows, for example, and many
> other functions are missing.

Yeah, I'm not saying it'll cover everything, but it doesn't have to
cover everything to be an improvement.

Gnulib also doesn't operate simply in terms of one-to-one function
replacement; for example, it does already provide some process-spawning
functions that are implemented in terms of Windows or Unix APIs as
appropriate, although I haven't looked into whether they'd meet groff's
requirements.

> > (It's possible that some of the _WIN32 conditionals can be supplied by
> > Gnulib these days, but there's also no great urgency to remove them,
> > IMO.)
> 
> I doubt that.  The vast majority of those I see in the current sources
> deal with issues that Gnulib cannot fix:

Hmm.  Most of the ones you mention look like things that appear to be
either very much fixed by Gnulib or at least tractable.

>   . absence of SIGPIPE

  
https://www.gnu.org/software/gnulib/manual/html_node/signal_002eh.html#signal_002eh
  https://www.gnu.org/software/gnulib/MODULES.html#module=sigpipe

(groff is actually doing something a bit more complicated involving
error detection on an output stream, but I wouldn't want to bet that
Gnulib couldn't handle it, perhaps via something like the "fwriteerror"
module.)

>   . backslashes as directory separators

This ifdef could probably be eradicated using:

  https://www.gnu.org/software/gnulib/MODULES.html#module=filename

Also, one of the relevant #ifdefs is in
src/libs/libgroff/localcharset.c, which seems to be a somewhat old copy
of https://www.gnu.org/software/gnulib/MODULES.html#module=localcharset;
using the latter would likely make it easier to keep up to date.

>   . differences in how argv[0] is presented to 'main'

I haven't found the bit of groff you're referring to here, but it sounds
like the sort of thing that Gnulib could fix if its "getprogname" module
were ported.

>   . different names of environment variables, like TEMP vs TMPDIR

Gnulib doesn't handle this today, but it's clear that it could if the
package using it were using something like the "tmpfile" module.

>   . quoting of command arguments 'like this' that isn't supported on
> Windows

This seems like:

  https://www.gnu.org/software/gnulib/MODULES.html#module=system-quote


I appreciate this may seem like pedantry, but people who care about
portability to a given platform should actively prefer things like this
to be handled by a portability library wherever possible, because then
it's easier to apply them to more packages.  Further, reducing the
forest of #ifdefs makes it easier to follow the essential logic of the
application code rather than having to wade through platform minutiae
when not actually doing porting work.

Importing more modules from Gnulib is typically a matter of adding them
to gnulib_modules in bootstrap.conf and rerunning ./bootstrap, so it
should be quite an accessible thing for somebody to try who has access
to both Windows and non-Windows test platforms and wants to try to
reduce the Windows support burden in groff.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Loss of MSVC support

[Colin Watson]

On Wed, Feb 13, 2019 at 06:58:18PM +0200, Eli Zaretskii wrote:
> Just to make what Keith says (and I concur) crystal clear: there's a
> need to distinguish between C99 compliance of the compiler and the
> C99/Posix compliance of the C runtime.  We can assume the former,
> certainly when using MinGW GCC, but we cannot assume the latter when
> building a native MS-Windows port (as opposed to Cygwin port) of
> Groff.

If it's just the runtime, then Gnulib should be able to paper over a
pretty fair number of the differences, and groff already uses that.  It
may just be a matter of somebody who can do test-builds on Windows
making sure that we're importing the right set of Gnulib modules.

(It's possible that some of the _WIN32 conditionals can be supplied by
Gnulib these days, but there's also no great urgency to remove them,
IMO.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [patch] modernize -T ascii rendering of opening single quote

[Colin Watson]

On Sat, Feb 02, 2019 at 05:57:31PM +0100, Ingo Schwarze wrote:
> Ralph Corderoy wrote on Sat, Feb 02, 2019 at 02:59:21PM +:
> > Ingo Schwarze wrote:
> >> It doubt that the benefit of avoiding the ugly ` ' in ASCII output is
> >> worth these (at least) three downsides.
> 
> > Whom is this change is meant to benefit?  I've lost track.
> 
> People reading roff(7) documents with nroff(1) or man(1) in a terminal
> window while they have LC_CTYPE=C set and while they are using a modern
> font.

One common real-world example I'd offer would be anyone who has a modern
terminal/font/etc. setup locally but who is reading a man page while
SSHed into a server that doesn't have a matching locale (it's not at all
unusual for me to find that en_GB.UTF-8 hasn't been generated on some
server or other).  Unfortunately we don't have a robust mechanism for
sshd to pick some other suitable locale that at least has a matching
encoding, and the details of this would no doubt be difficult to define
anyway; so the upshot is that somebody in this situation ends up with
the equivalent of LC_ALL=C on the remote system even though they
normally prefer UTF-8 locally.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] UTF8 characters to pdf

[Colin Watson]

On Mon, Jan 28, 2019 at 07:59:00PM +1100, John Gardner wrote:
> *> Would anyone who uses a Mac and is reasonably Homebrew-literate like to
> pick this up?*
> 
> Done! See https://github.com/Homebrew/homebrew-core/pull/36469 (I CC'd you
> on GitHub anyway)

Thanks!

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] UTF8 characters to pdf

[Colin Watson]

On Thu, Jan 24, 2019 at 04:21:51PM +, Colin Watson wrote:
> On Fri, Jan 25, 2019 at 02:10:32AM +1100, John Gardner wrote:
> > There was an attempt to get man-db
> > <https://github.com/Homebrew/homebrew-core/pull/25376> accepted in
> > Homebrew, but a patch was needed for macOS support, which Homebrew required
> > upstream to accept before they'd support it.
> 
> FWIW, this was gnulib upstream, not man-db / libpipeline upstream (i.e.
> me).
> 
> If somebody could point me to where I could get SSH access to a macOS
> box with the right set of build tools installed, I'd be happy to try to
> get it working properly.

We got this sorted out so that libpipeline and man-db build without
patches in Homebrew:

  https://github.com/ylluminarious/homebrew-man-db/issues/1

However, the original submitter is reluctant to submit these formulae to
the Homebrew maintainers again due to their earlier conflict (see the
end of that issue).  I'm not a good person to do it since I don't use a
Mac.  Would anyone who uses a Mac and is reasonably Homebrew-literate
like to pick this up?  I don't think the conflict should recur since the
upstream build is clean now, and I'd be happy to do whatever other
upstream support is needed.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] UTF8 characters to pdf

[Colin Watson]

On Fri, Jan 25, 2019 at 02:10:32AM +1100, John Gardner wrote:
> There was an attempt to get man-db
> <https://github.com/Homebrew/homebrew-core/pull/25376> accepted in
> Homebrew, but a patch was needed for macOS support, which Homebrew required
> upstream to accept before they'd support it.

FWIW, this was gnulib upstream, not man-db / libpipeline upstream (i.e.
me).

If somebody could point me to where I could get SSH access to a macOS
box with the right set of build tools installed, I'd be happy to try to
get it working properly.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [PATCH] Avoid Perl's unsafe "<>" operator

[Colin Watson]

On Thu, Jan 24, 2019 at 03:38:38PM +, Deri wrote:
> Have you seen:-
> 
> https://savannah.gnu.org/bugs/?7
> 
> Perhaps the patch could be uploaded there?

Thanks, I hadn't seen that.  Done.

-- 
Colin Watson   [cjwat...@debian.org]

[groff] [PATCH] Avoid Perl's unsafe "<>" operator

[Colin Watson]

The "<>" operator is implemented using the two-argument form of "open",
which interprets magic such as pipe characters, allowing execution of
arbitrary commands which is unlikely to be expected.  Perl >= 5.22 has a
"<<>>" operator which avoids this, but also forbids the use of "-" to
mean the standard input, which is a facility that the affected groff
programs document.

ARGV::readonly would probably also fix this, but I fundamentally dislike
the approach of escaping data in preparation for a language facility to
unescape it, especially when the required escaping is as non-obvious as
it is here.  (For the same reason, I prefer to use subprocess invocation
facilities that allow passing the argument list as a list rather than as
a string to be interpreted by the shell.)  So I've abandoned this
dubious convenience and changed the affected programs to iterate over
command-line arguments manually using the three-argument form of open.

This change involves an extra level of indentation, so it's a little
awkward to review.  It consists of changing this form:

  while (<>) {  # or foreach, which is similar but less efficient
...
  }

... into this:

  unshift @ARGV, '-' unless @ARGV;
  foreach my $filename (@ARGV) {
my $input;
if ($filename eq '-') {
  $input = \*STDIN;
} elsif (not open $input, '<', $filename) {
  warn $!;
  next;
}
while (<$input>) {
  ...
}
  }

Local variations: glilypond doesn't need the initial unshift since
that's already handled in contrib/glilypond/args.pl; gropdf declares
$input in a slightly different way since it's also used in the LoadAhead
function.

Fixes: https://bugs.debian.org/920269
---
 contrib/glilypond/glilypond.pl | 128 +++---
 contrib/gperl/gperl.pl | 188 +
 contrib/gpinyin/gpinyin.pl |  88 ---
 src/devices/gropdf/gropdf.pl   |  99 +
 tmac/hyphenex.pl   |  86 ---
 5 files changed, 318 insertions(+), 271 deletions(-)

diff --git a/contrib/glilypond/glilypond.pl b/contrib/glilypond/glilypond.pl
index 868801b2..f2a76158 100755
--- a/contrib/glilypond/glilypond.pl
+++ b/contrib/glilypond/glilypond.pl
@@ -565,73 +565,81 @@ our $Read =
 ); # end definition %lilypond_args
 
 
- LILYPOND: foreach (<>) {
-chomp;
-my $line = $_;
+ LILYPOND: foreach my $filename (@ARGV) {
+my $input;
+if ($filename eq '-') {
+  $input = \*STDIN;
+} elsif (not open $input, '<', $filename) {
+  warn $!;
+  next;
+}
+while (<$input>) {
+  chomp;
+  my $line = $_;
 
 
-# now the lines with '.lilypond ...'
+  # now the lines with '.lilypond ...'
 
-if ( /
-  ^
-  [.']
-  \s*
-  lilypond
-  (
-.*
-  )
-  $
-/x ) { # .lilypond ...
-  my $args = $1;
-  $args =~ s/
- ^
- \s*
-   //x;
-  $args =~ s/
- \s*
- $
-   //x;
-  $args =~ s/
- ^
- (
-   \S*
- )
- \s*
-   //x;
-  my $arg1 = $1; # 'start', 'end' or 'include'
-  $args =~ s/["'`]//g;
-  my $arg2 = $args; # file argument for '.lilypond include'
-
-  if ( exists $lilypond_args{$arg1} ) {
-   $lilypond_args{$arg1}->($arg2);
-   next;
-  } else {
-   # not a suitable argument of '.lilypond'
-   $stderr->print( "Unknown command: '$arg1' '$arg2':  '$line'" );
-  }
-
-  next LILYPOND;
-} # end if for .lilypond
+  if ( /
+^
+[.']
+\s*
+lilypond
+(
+  .*
+)
+$
+  /x ) { # .lilypond ...
+   my $args = $1;
+   $args =~ s/
+   ^
+   \s*
+ //x;
+   $args =~ s/
+   \s*
+   $
+ //x;
+   $args =~ s/
+   ^
+   (
+ \S*
+   )
+   \s*
+ //x;
+   my $arg1 = $1; # 'start', 'end' or 'include'
+   $args =~ s/["'`]//g;
+   my $arg2 = $args; # file argument for '.lilypond include'
+
+   if ( exists $lilypond_args{$arg1} ) {
+ $lilypond_args{$arg1}->($arg2);
+ next;
+   } else {
+ # not a suitable argument of '.lilypond'
+ $stderr->print( "Unknown command: '$arg1' '$arg2':  '$line'" );
+   }
 
+   next LILYPOND;
+  } # end if for .lilypond
 
-if ( $lilypond_mode ) { # do lilypond-mode
-  # see '.lilypond start'
-  $ly->print( $line );
-  next LILYPOND;
-} # do lilypond-mode
 
-# unknown line without lilypond
-unless ( /
-  ^
-  [.']
-  \s*
-  lilypond
-/x ) { # not a '.lilypond' line
- 

Re: [groff] UTF8 characters to pdf

[Colin Watson]

On Thu, Jan 24, 2019 at 05:05:25AM +, jonathan ahumada wrote:
> I’m trying to get into groff  with the hopes that I could print my own
> literary drafts. However, I write in spanish and have had problems
> with accented characters such as (á, é, ó , etc). I have been
> searching and apparently groff doesn’t support utf-8, which is the
> format my files are written in. I have seen some people use a
> `preconv`command or a `-k` flag to convert their files for utf-8 into
> latin1 within the groff command.

Sort of; it preprocesses UTF-8 into ASCII plus \[u] escapes.  Using
preprocessors to transform the input in various ways is a routine
practice in groff.

On the phrasing in your last couple of sentences, I would rather say
that groff supports UTF-8, but only via preconv (which is part of
groff).

> But the build I have on my Mac computer doesn’t seem to have this
> (version 1.19.2).

Yeah, that is pretty prehistoric; groff 1.20 was released a full ten
years ago, and the current release is 1.22.4.  I'd really recommend you
try upgrading.

I'm not a Mac user so I don't know which you might prefer or what the
tradeoffs are, but MacPorts has 1.22.3 and Homebrew has 1.22.4.  Perhaps
you might try installing a newer version of groff from one of those?

> I know you can use escape sequences in order to tell the troff
> formatter to use certain characters, but modifying my files with a
> sort of tailored script to replace all my accents seems burdensome. 

You wouldn't actually modify your files, but just have the build script
you use to render them add the -k option.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Bug#920269: Bug#920269: [vinc...@vinc17.net: Bug#920269: groff: gropdf can execute arbitrary commands]

[Colin Watson]

On Wed, Jan 23, 2019 at 03:02:26PM +, Ralph Corderoy wrote:
> Hi Colin,
> 
> > So this option would also lose support for Debian 8 (currently
> > oldstable).
> 
> Also, `<<>>' doesn't support `-' to mean stdin.
> That might affect users of the groff Perl scripts that use `<>'.

Ah, yes - in that case I agree that that isn't a good option.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Bug#920269: [vinc...@vinc17.net: Bug#920269: groff: gropdf can execute arbitrary commands]

[Colin Watson]

On Wed, Jan 23, 2019 at 03:21:53PM +0100, Jakub Wilk wrote:
> * Colin Watson , 2019-01-23, 13:56:
> > Perl >= 5.20 has the safer <<>> operator,
> 
> It was actually added only in Perl 5.22.

Sorry, indeed so - I grepped the perl*delta man pages for it but misread
"5220" as "5200".  So this option would also lose support for Debian 8
(currently oldstable).

-- 
Colin Watson   [cjwat...@debian.org]

[groff] [vinc...@vinc17.net: Bug#920269: groff: gropdf can execute arbitrary commands]

[Colin Watson]

Hi,

I'm not quite sure of the circumstances in which an attacker (presumably
the author of a document you've received) might be able to control the
arguments to gropdf; but regardless, this does seem to be undesirable
command-line handling and I think we should fix it.

  $ find -name \*.pl | xargs grep -- '<>'
  ./src/devices/gropdf/gropdf.pl:while (<>)
  ./src/devices/gropdf/gropdf.pl: my $lin=<>;
  ./tmac/hyphenex.pl:while (<>) {
  ./contrib/gpinyin/gpinyin.pl:foreach (<>) { # get line from input
  ./contrib/gperl/gperl.pl:foreach (<>) {
  ./contrib/glilypond/glilypond.pl: LILYPOND: foreach (<>) {
  ./contrib/glilypond/glilypond.pl:  } # end foreach <>

What I'm not quite sure of is what the best fix would be, since my Perl
is a bit rusty.  Perl >= 5.20 has the safer <<>> operator, which only
interprets file names, but I don't know whether it would be acceptable
to bump groff's minimum Perl version from its current value of 5.6.1.
In my own wheelhouse, that would drop support for everything up to and
including Debian 7 (currently oldoldstable) and Ubuntu 14.04 (which goes
out of regular support this April), which does include some stuff that
we'd want to issue security updates for, so maybe that isn't the best
option.

Alternatively, perhaps we could just copy ARGV::readonly from CPAN into
the start of all our Perl scripts?  It's sufficiently small that it
might not be worth getting too worked up about the code duplication:

  https://metacpan.org/source/DAVIDNICO/ARGV-readonly-0.01/lib/ARGV/readonly.pm

Any other ideas welcome.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]
--- Begin Message ---
Package: groff
Version: 1.22.4-2
Severity: grave
Tags: security
Justification: user security hole

According to the gropdf(1) man page:

   gropdf [-dels] [-F dir] [-I dir] [-p paper-size] [-u [cmapfile]]
  [-y foundry] [file ...]

but providing a "filename" with a pipe character can yield an
arbitrary command execution:

$ touch foo
$ ls foo
foo
$ gropdf "rm foo|"
$ ls foo
ls: cannot access 'foo': No such file or directory
$ 

The reason is that gropdf is a Perl script that uses the insecure
null filehandle "<>". The perlop(1) man page says:

  Since the null filehandle uses the two argument form of "open" in
  perlfunc it interprets special characters, so if you have a script like
  this:

  while (<>) {
  print;
  }

  and call it with "perl dangerous.pl 'rm -rfv *|'", it actually opens a
  pipe, executes the "rm" command and reads "rm"'s output from that pipe.

BTW, I fear that's not the only Perl script that is affected by such
a bug.

-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 
'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.19.0-1-amd64 (SMP w/12 CPU cores)
Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX 
(charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages groff depends on:
ii  groff-base  1.22.4-2
ii  libc6   2.28-5
ii  libgcc1 1:8.2.0-14
ii  libice6 2:1.0.9-2
ii  libsm6  2:1.2.2-1+b3
ii  libstdc++6  8.2.0-14
ii  libx11-62:1.6.7-1
ii  libxaw7 2:1.0.13-1+b2
ii  libxmu6 2:1.1.2-2
ii  libxt6  1:1.1.5-1

Versions of packages groff recommends:
ii  ghostscript  9.26~dfsg-0+deb9u2
ii  imagemagick  8:6.9.10.23+dfsg-2
ii  imagemagick-6.q16 [imagemagick]  8:6.9.10.23+dfsg-2
ii  libpaper11.1.26
ii  netpbm   2:10.0-15.3+b2
ii  perl 5.28.1-3
ii  psutils  1.17.dfsg-4

groff suggests no packages.

-- no debconf information


--- End Message ---

Re: [groff] [akalka...@hotmail.com: Bug#919890: groff-base: Incorrect page numbers position when exporting BSD-style (mdoc) documents to other formats (ps, etc)]

[Colin Watson]

On Mon, Jan 21, 2019 at 03:04:33PM +0100, Ingo Schwarze wrote:
> Colin Watson wrote on Sun, Jan 20, 2019 at 04:18:29PM +:
> > but much the same problem exists; page numbers are printed on the left
> > of odd-numbered pages and on the right of even-numbered pages,
> 
> Isn't that what one would most often want?  When somebody prints a
> manual page putting two virtual pages side-by-side on each sheet
> in landscape orientation, then page 1 ends up on the left side and
> page 2 on the right side of the first sheet, so the odd numbers
> should be on the left, as indeed they are.
[...]
> Maybe we should change the man macros rather than the mdoc macros?

I don't think I have any stake in this either way, except that man and
mdoc should be mutually-consistent, and IMO we should document the
choice made in both groff_man(7) and groff_mdoc(7) with a brief comment
on the rationale to head off future questions.

-- 
Colin Watson   [cjwat...@debian.org]

[groff] [akalka...@hotmail.com: Bug#919890: groff-base: Incorrect page numbers position when exporting BSD-style (mdoc) documents to other formats (ps, etc)]

[Colin Watson]

Hi groff@,

Would any mdoc folks like to have a look at this?  Details differ in
1.22.4 and this patch doesn't quite apply directly, but much the same
problem exists; page numbers are printed on the left of odd-numbered
pages and on the right of even-numbered pages, when one would normally
expect the opposite.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]
--- Begin Message ---
Package: groff-base
Version: 1.22.3-9
Severity: minor
Tags: patch

Dear Maintainer,

I used the following to export man pages (ie ssh_config, written in
mdoc-style) to a pdf file for double-sided printing:
zcat /usr/share/man/man5/ssh_config.5.gz |groff -man -rD1 -Tps - |ps2pdf
- ssh_config.pdf

The page numbers in the resulting pdf file are placed incorrectly.

This does not affect documents written using the "man" format, for example
it will not affect the man.pdf resulting from the following:
zcat /usr/share/man/man1/man.1.gz |groff -man -rD1 -Tps - |ps2pdf
- man.pdf


-- System Information:
Debian Release: 9.6
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 4.9.0-8-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages groff-base depends on:
ii  libc6   2.24-11+deb9u3
ii  libgcc1 1:6.3.0-18+deb9u1
ii  libstdc++6  6.3.0-18+deb9u1

groff-base recommends no packages.

Versions of packages groff-base suggests:
pn  groff  

-- no debconf information
@@ -721,7 +721,7 @@
 .setup-page-layout
 .sp \n[footer-space]u
 .ie \n[D] \{\
-.ie o \
+.ie e \
 .tl 
%\*[caption-font2]\*[date-string]\f[]\*[caption-font]\*[operating-system]\f[]
 .el \
 .tl 
\*[caption-font]\*[operating-system]\f[]\*[caption-font2]\*[date-string]\f[]%
--- End Message ---

Re: [groff] GNU troff version 1.22.4

[Colin Watson]

On Mon, Dec 24, 2018 at 11:44:54PM +1100, John Gardner wrote:
> Kudos to everyone's hard work! I did my part
> <https://github.com/Homebrew/homebrew-core/pull/35385>. :-)

And me:

  https://tracker.debian.org/pkg/groff

All the best,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] mom manpage

[Colin Watson]

On Thu, Dec 06, 2018 at 06:38:40PM -0500, Peter Schaffter wrote:
> +@display
> +@file{/usr/share/doc/groff-base/html/mom/toc.html}
> +@end display
[...]
> +@display
> +@file{/usr/share/doc/groff-base/pdf/mom-pdf.pdf}
> +@end display
[...]
> I'm not sure whether it's okay for entry points to local files to
> have literal paths or whether they need to be rendered symbolically
> for build/install purposes, so I won't commit without your go-ahead.

These paths are *very* Debian-and-derivatives-specific (even leaving
aside the hardcoded /usr/share/doc prefix, the split between groff-base
and groff is a Debianism), so they can't be committed upstream like
that.  Either the local references will need to be omitted, or we'll
need some kind of arrangement along the lines of the existing
@HTMLDOCDIR@ and @PDFDOCDIR@ in groff's man pages (I'm not sure how
practical that is in texinfo).

-- 
Colin Watson   [cjwat...@debian.org]

[groff] gratitude for man pages (was: mom manpage)

[Colin Watson]

I've been staying out of this because I don't really have a position on
it either way; I don't share the apparent position that all
documentation should come in the form of man pages, but I also don't
know enough about mom to have any real idea about what might be best for
it.  But I couldn't resist replying to this bit:

On Tue, Dec 04, 2018 at 03:16:50PM -0500, Peter Schaffter wrote:
> How much a part has the documentation played in her adoption?  Again,
> hard to judge, but users actually thank me for it from time to time.
> (I leave open the question of why mom's documentation inspires
> gratitude where manpages do not.)

I reasonably often get emails from users who have the idea that, because
I happen to maintain an implementation of man(1) that's in some popular
distributions, I'm a good contact point for all the man pages they read.
(I generally try to redirect them as appropriate and as time permits, of
course.)

One pleasant side-effect of this mild confusion is that I get thank-you
letters about man pages in general from time to time.  Here are a few
samples, shorn of any personal information:

  "I've recently started using Ubuntu and I simply wanted to say thank
  you for your continued work on the 'man' pages... they're so helpful
  with getting me up and running with bash commands."

  "I recently switched to linux and my favorite thing so far is the
  vastly informative database of man pages. These man pages have been
  invaluable in my journey to learn more about Linux."

  "Thanks for up-keeping the man page database! I bet you dont get that
  alot so I just wanted to say THANKS!"

(I think most of the gratitude is not really deservedly directed at me,
so I don't feel bad about sharing emails like this in some form.)

I don't wish to set up any kind of competitiveness with mom here; if you
get unsolicited thank-you mails about it, it sounds like it's doing OK!
But my inbox suggests that man pages do in fact inspire quite a bit of
gratitude too, enough that I get a sense of it from just its misdirected
overspill.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] 1.22.4.rc4 - Final RC before official 1.22.4

[Colin Watson]

On Fri, Nov 30, 2018 at 01:00:59AM +0100, Bertrand Garrigues wrote:
> To all the developers: please hold on your commits now, this release is
> for final sanity checks, only critical fixes should be commited from
> now.

I pushed a couple of corrections to error paths in the new
contrib/mom/examples/test-mom.sh.in script.  Not perhaps quite critical,
but since it was only added a week ago and the changes can only possibly
affect "make check" runs that are already about to fail, it seemed
reasonable.

Incidentally, the reason I ran into this was because I was building with
pdfinfo et al installed (having noticed that recent configure.ac change)
but didn't have URW fonts installed, and hadn't noticed the configure
warning.  Now, since this meant that my builds weren't generating the
U-* fonts, it's good that this caused me to notice and fix it.  However,
the reported symptom was as follows:

  Checking number of pages of 
/build/groff-9tRn1u/groff-1.22.4~rc4/debian/build/contrib/mom/examples/typesetting.pdf
Error: expected 3 pages, found 2 pages

This was a pretty opaque symptom, and it took me a little while to find
the real cause.  Should this be reported more clearly, perhaps by having
this test exit 77 (which would cause Automake to produce a SKIP result)
if configure didn't find any URW fonts?  At the moment we have a
supported configuration (configure explicitly tests for URW fonts but
chooses only to issue a warning if they're absent, presumably because
you can use most of groff without them) which then causes the test suite
to fail, which doesn't seem ideal.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] How to show all hyphenation points?

[Colin Watson]

On Thu, Nov 08, 2018 at 09:50:31AM -0500, Doug McIlroy wrote:
> What I see as the typical use is a nonce question about a 
> single word. That is trivially handled by
>   groff -a
>   .ll 1u
>   recapitulation
> The result is marrred with "can't break line" diagnostics,
> but it's quick and intelligible.

This is perhaps me being Captain Obvious, but FWIW adding the -Wbreak
option suppresses those diagnostics.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Release Candidate 1.22.4.rc2

[Colin Watson]

On Tue, Jun 19, 2018 at 12:49:03AM +0200, Bertrand Garrigues wrote:
> I will try now to finish the release now, and I'm currently checking all
> my unread mails.  I've also checked the recent commits and as far as I
> can see, there is no reason not to included them in the next release.

I hate to nag, but is there any likelihood of a 1.22.4 release in the
somewhat near future?  My next relevant deadline is that the freeze for
the next Debian release will begin in January 2019.  It would be good if
it could include a recent version of groff, but I'm not enthusiastic
about pushing in a release candidate.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] improve a few terminal renderings of special characters

[Colin Watson]

On Sat, Sep 01, 2018 at 01:41:04PM +0200, Ingo Schwarze wrote:
>   http://mandoc.bsd.lv/man/man.options.1.html#l

This is an excellent resource that I hadn't seen before.  Thanks for
putting it together.  I'll certainly pay attention to it if I have cause
to add more short options to man-db (although I don't exactly expect
this to be frequent).

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] gxditview Run by man-db Suffers SIGSYS; core dumped.

[Colin Watson]

On Thu, Aug 30, 2018 at 03:09:30PM +0100, Ralph Corderoy wrote:
> This warren I'm in is John's fault.
> 
> Talking of man-db and gxditview...
> 
> $ pacman -Q man-db groff
> man-db 2.8.4-1
> groff 1.22.3-7
> $ man -X100 core
> groff: gxditview: Signal 31 (core dumped)

Can you send me a bug report about this
(https://savannah.nongnu.org/bugs/?group=man-db or whatever other method
is convenient), ideally with "strace -f" output demonstrating the
failure?  SIGSYS probably has something to do with man-db's seccomp
sandbox, in which I suspect I've shamefully neglected to consider the
gxditview case.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [off-topic] Reliable use of errno

[Colin Watson]

On Sun, Aug 19, 2018 at 03:08:26PM +0100, Ralph Corderoy wrote:
> Hi Colin,
> 
> > Writing "== -1" implies a reading that any other value would indicate
> > success;
> 
> Or it implies it's being coded to the spec.

If your program explicitly handles unspecified return values and handles
those as an error case but without making reference to errno, then you
might have a point, but in practice I've never seen anyone doing this,
probably because it's excessively pedantic and noisy for no real
benefit.

Otherwise, your programming style is implicitly choosing whether you
handle unspecified return values as success or failure; for instance, if
you only handle errors within an "== -1" condition, you're implicitly
saying that unspecified return values are successes.  IME, most
programmers don't care either way, and I would argue they are correct
not to waste brain cells or program text on it.

> > writing "!= 0" implies a reading that any other value would indicate
> > an error; writing "< 0" implies a reading that it depends on the sign
> > of the hypothetical not-zero-or-minus-one return value.
> 
> Both of these allow for more values than -1, `stuttering' the human
> reader, and potentially troubling the static analyser.

Since both the practices in my quoted text above are extremely
widespread, I dispute your unsupported claim that this could trouble
static analysers: any static analyser capable of handling typical C code
must be able to handle both.  (I would be inclined to say the same for
human readers, TBH, although as usual picking one style within a project
and sticking to it is helpful.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [off-topic] Reliable use of errno

[Colin Watson]

On Sun, Aug 19, 2018 at 02:32:49PM +0200, Carsten Kunze wrote:
> Many system calls (e.g. close(2),
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/close.html)
> return 0 on success and -1 on error and set errno.
> 
> My understanding is that errno is only reliably set when the system
> call returns -1, not for < -1 nor for > 0, so I do only check the
> error condition with "if (close(fd) == -1)" ignoring all other values.
> Some are testing for "< 0" but would it not be consequent for them to
> check for "!= 0"? So when can I rely on errno to be set, for -1, for
> "< 0" or for "!= 0"?

These kinds of system calls only ever return 0 or -1, so it doesn't much
matter which way you do the test.

It depends which way you read the implicit gap in the specification; it
explicitly enumerates all the possible return values, but what would any
other return value mean if it could happen?  Writing "== -1" implies a
reading that any other value would indicate success; writing "!= 0"
implies a reading that any other value would indicate an error; writing
"< 0" implies a reading that it depends on the sign of the hypothetical
not-zero-or-minus-one return value.

You could perhaps argue that you ought to explicitly test for some value
other than 0 or -1 and raise some kind of generic error that doesn't
make use of errno in that case; but in practice people don't want to
scatter all this untestable error handling code all over their programs
for a condition that's already specified not to happen, and possibly
printing a stale errno value in a hypothetical situation is a good
trade-off for writing less code (and thus, statistically, fewer bugs).

The "< 0" idiom probably arises from system calls such as read(2), which
return either zero for end-of-file, a positive value for the number of
bytes read on success, or -1 on error.  In that case "< 0" covers all
the return values that aren't defined to be successful.  As a result, my
habit is to use that form for everything that returns -1 on error unless
I have a good reason not to (e.g. if -2 were defined to mean something
else then I'd explicitly test for -1).

In short, you're asking for a specification for unspecified behaviour,
and you aren't going to get one unless the POSIX committee chooses to
clarify it.  But POSIX also says that the other return values you're
concerned about simply don't happen, so I'd suggest just picking
whatever syntax you prefer for testing for the cases that do happen and
moving on.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] groff build problem

[Colin Watson]

On Sun, Aug 19, 2018 at 02:15:07PM +0200, Heinz-Jürgen Oertel wrote:
> using for the first time now OpenSuse Thumbleweed. Installed all tools (I 
> hope)
> ./configure runs without problems. Had to do:
> sudo ln -sf /usr/bin/aclocal /usr/bin/aclocal-1.13

As well as what other people have said, randomly dropping symlinks into
bits of /usr/ is a bad habit that I strongly recommend you avoid in
future (though I more usually see people attempting this with different
versions of libraries).  I notice that your error message included
mention of aclocal-1.16 as well, which means that the build system has
likely ended up with a mix of different versions of bits of automake,
and that's not going to be healthy.

I'd strongly advise removing /usr/bin/aclocal-1.13 (assuming it was
previously absent) before you do anything else, and then following the
other advice you've had here to rebootstrap the build system properly.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Spooky action at a distance in line adjustment...sometimes

[Colin Watson]

On Wed, Jun 27, 2018 at 12:39:47PM +0100, Ralph Corderoy wrote:
> Hi Doug,
> 
> > But it soon became old hat and people migrated back to ragged right
> > margins, which may not look as neat from afar, but also seem to be
> > easier to read both because of even spacing and because the variable
> > margin provides distinguishablility to help a reader track vertical
> > position on the page.
> 
> Right.  That's why I have `--nj' in the MANOPT environment variable.
> I'm surprised more distributions don't turn off justification by default
> as most users are unaware that man(1) might have a configuration option.
> Colin?

Mostly conservatism, I guess.  I personally prefer full justification in
manual pages, but probably just because I've got used to it over the
years, and I only added the options in 2009
(https://bugs.debian.org/440047).

There's no good (i.e. configuration-file-based) way for a distribution
to change the default.  Could you file a bug report for me as a
reminder?

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Brian Kernighan on the evoution of eqn, pic, grap, into troff

[Colin Watson]

On Sat, May 05, 2018 at 11:35:14AM +0200, Andreas Eder wrote:
> On Sa 05 Mai 2018 at 01:15, Tadziu Hoffmann  
> wrote:
> > (BTW, most Germans who use it also pronounce it wrong,
> > namely as in "ich" instead of as in "Bach".)
> 
> What do you mean by that? The sound of the 'ch' is the same in both
> cases.

It depends on the dialect, but
https://en.wikipedia.org/wiki/Standard_German_phonology#Ich-Laut_and_ach-Laut
describes the difference in "standard German" - it's essentially about
whether the sound is produced towards the front or back of the mouth.

> 'Andreas ( a german native speaker :-) )

I don't know which dialect you speak; many southern dialects use [x] for
both "ich" and "Bach", rather than [ç] for "ich" and [x] for "Bach".
But as a native English speaker, I can certainly say that there are
aspects of my own language's phonology that weren't obvious to me
without a lot of attention that didn't come automatically!

The Scottish "loch", and hence TeX per Knuth, is indeed pronounced with
the [x] sound.  (In fact, Irish and Scottish Gaelic both have a similar
scheme for pronouncing "ch" as standard German does: if it's surrounded
by front vowels, traditionally called "slender", then it's [ç], while if
it's surrounded by back or "broad" vowels then it's [x].  I don't know
whether the similarity to German is one of common evolution, borrowing,
or just coincidence.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Savannah bug field usage protocol

[Colin Watson]

On Sat, Apr 21, 2018 at 05:43:12PM -0400, G. Branden Robinson wrote:
> At 2018-04-21T14:21:03+0100, Ralph Corderoy wrote:
> > Some projects maintain a `pending' release notes so that significant
> > fixes and changes get added to as they're implemented.  This might avoid
> > the need to keep some bugs non-closed until the release.  Come release,
> > the file's reviewed, some minor things with highsight ditched, and the
> > commit log scanned for anything missing.
> 
> Our commit discipline is already supposed to include updating the
> ChangeLog, and, where appropriate, NEWS files as part of the commit
> where the underlying change is made.  Do you think that should suffice
> for aiding the preparation of Release Notes?

I agree with the sentiment, but do the ~40 lines of NEWS entries since
1.22.3 really reflect the ~1900 lines of ChangeLog for the same period?
I'm not really qualified to prepare updates there, but the ratio seems
unusually low to me.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] groff as the basis for comprehensive documentation?

[Colin Watson]

On Thu, Apr 19, 2018 at 06:48:19PM +0200, Ingo Schwarze wrote:
> Colin Watson wrote on Thu, Apr 19, 2018 at 10:06:28AM +0100:
> > "man ./apropos.1", as Nate pointed out.  man-db's heuristic is that if
> > the page name contains a slash then it's surely a path name instead and
> > should be treated as such; I think that's a reasonable one.
> 
> Thank you for explaining the heuristic and for pointing out the
> missing feature in mandoc.  Given the existence of the -l option,
> having the heuristic is maybe not absolutely required, but i
> agree that it is not unreasonable, and we have seen that the absence
> of the heuristic can confuse casual users who are used to man-db.
> 
> So with the commit below, i added the same heuristic to mandoc.

Thanks.

> By the way, the old version of man-db in jessie exhibits a strange
> behaviour in that case, but probably that has been fixed long ago:
> 
>$ lsb_release -d
>   Description:Debian GNU/Linux 8.10 (jessie)
>$ dpkg-query -l man-db | tail -n 1
>   ii  man-db 2.7.0.2-5i386 on-line manual pager
>$ man --version
>   man 2.7.0.2
>$ man -w man ./man.1
>   man: man-./man.1: No such file or directory
>   man: man_./man.1: No such file or directory
>   /usr/share/man/man1/man.1.gz
>   ./man.1

This is still incorrect in current versions: man(1)'s command-line
parsing is not quite as elegantly well-factored as it ought to be ...  I
don't quite have time to sort it out just now, but I've filed
https://savannah.nongnu.org/bugs/index.php?53708 so that I don't forget
about it.  Thanks for the report!

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] groff as the basis for comprehensive documentation?

[Colin Watson]

On Thu, Apr 19, 2018 at 02:28:36AM +0200, Ingo Schwarze wrote:
> Nate Bargmann wrote on Wed, Apr 18, 2018 at 06:52:44PM -0500:
> > I was disappointed that unlike "man" that I find on Slackware or
> > Debian, I had to add an uninstalled man page into the db in order
> > for "mandoc" to open it.  Perhaps I missed an option, but the "man"
> > I'm familiar with is capable of opening a file simply by giving it
> > a path name.
> 
> That is non-standard behaviour even on GNU/Linux:
> 
>$ lsb_release -d
>   Description:Debian GNU/Linux 8.10 (jessie)
>$ dpkg-query -l man-db | tail -n 1
>   ii  man-db 2.7.0.2-5i386 on-line manual pager
>$ ls *.1
>   apropos.1  demandoc.1  man.1  man.options.1  mandoc.1  soelim.1
>$ man apropos.1# the man-db man(1) implementation
>   No manual entry for apropos.1

"man ./apropos.1", as Nate pointed out.  man-db's heuristic is that if
the page name contains a slash then it's surely a path name instead and
should be treated as such; I think that's a reasonable one.

> I think it is good that the '-l' ("local") option is required when
> giving a file name in the local directory, or an absolute path.

I agree that man should not try to guess the difference between a page
name and an unadorned file name in the local directory (with no slash),
but when it's an absolute path or a relative path containing a slash,
the situation seems quite clear.

> Besides, with the mandoc implementation of man(1), it is not
> absolutely required to update the database even after installing a
> new page into /usr/share/man/.

You can also leave out the qualifier clause there: this is true of the
man-db implementation too.  (It used to not be true, but I fixed that
long ago, back in 2002 or so.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] PSPIC vs PDFPIC: adjust documentation to reality, or the reverse?

[Colin Watson]

On Tue, Apr 10, 2018 at 03:57:19PM +0100, Deri James wrote:
> To avoid making existing documents render incorrectly I propose to allow the 
> existing behaviour to be selected. Adding this to the NEWS file:-
> 
> 
> PDFPIC has now been corrected, so the behaviour is the same whether you use 
> the postscript or pdf drivers. However, this means that any documents which 
> were written using the old behaviour will not be rendered correctly if using 
> the pdf driver with the new version.
> 
> The change would mean that documents which relied on the previous behaviour 
> are likely to have a gap underneath the image which was not there before. If 
> you see this effect there are three ways you can restore the previous 
> behaviour (listed in order of priority):-
> 
> A) Add the line ".nr PDFPIC_LEGACY 1" to the document before the first call 
> to 
> .PDFPIC.
> 
> B) If it is just a single document which exhibits this behaviour you can run 
> groff adding "-rPDFPIC_LEGACY=1" to the command-line.

I don't have a specific name suggestion, and I'm aware that this is a
bikeshed, but can I suggest a more explicit variable name?  Otherwise
the next time some old behaviour needs to be switchably deprecated we're
in for some confusion.

(In my third year in college, I lived in "New Court", built in 1825.)

-- 
Colin Watson   [cjwat...@debian.org]

[groff] [anth...@derobert.net: Bug#892423: man -Tutf8 /usr/share/man/ja/man5/apt_preferences.5.gz (and several others) enter infinite loop]

[Colin Watson]

Hi,

The attached input file, when processed as follows:

  LC_ALL=ja_JP.UTF-8 groff -mtty-char -mandoc -Tutf8 --- Begin Message ---
Package: man-db
Version: 2.8.2-1
Severity: normal

Running:

man -Tutf8 /usr/share/man/ja/man5/apt_preferences.5.gz

starts outputting the manpage, but then at some point switches over to
outputting an (so far as I can tell) infinite loop of newlines:

   sources.list(5) ファイルに列挙された場所から取得した Packages ファイル
   や Release ファイルはすべて、/var/lib/apt/lists ディレクトリ
   や、apt.conf ファイルの Dir::State::Lists 変数で指定した場所に取得され
   ます。例え




Attaching the requested debug output is... difficult, but attached are
the first 100,000 lines:

   man --debug -Tutf8 /usr/share/man/ja/man5/apt_preferences.5.gz |& head -n 
10 | gzip -v9 > /tmp/man-debug.txt.gz

I've also attached the apt_preferences page, just in case you have a
different version of apt; but note that other pages do this too (e.g.,
Japanese dos2unix page).

-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'testing-debug'), (500, 
'testing'), (200, 'unstable'), (150, 'stable'), (100, 'experimental'), (1, 
'experimental-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.14.0-2-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en_GB (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages man-db depends on:
ii  bsdmainutils   11.1.2
ii  debconf [debconf-2.0]  1.5.66
ii  dpkg   1.19.0.5
ii  groff-base 1.22.3-10
ii  libc6  2.27-1
ii  libgdbm5   1.14.1-4
ii  libpipeline1   1.5.0-1
ii  libseccomp22.3.1-2.1
ii  zlib1g 1:1.2.8.dfsg-5

man-db recommends no packages.

Versions of packages man-db suggests:
pn  apparmor
ii  chromium [www-browser]  62.0.3202.89-1
ii  elinks [www-browser]0.12~pre6-13
ii  firefox [www-browser]   58.0.1-1+b1
ii  google-chrome-stable [www-browser]  65.0.3325.146-1
ii  groff   1.22.3-10
ii  konqueror [www-browser] 4:17.08.3-2
ii  less487-0.1
ii  links2 [www-browser]2.14-5
ii  lynx [www-browser]  2.8.9dev16-3
ii  w3m [www-browser]   0.5.3-36

-- debconf information:
  man-db/auto-update: true
* man-db/install-setuid: false


man-debug.txt.gz
Description: application/gzip


apt_preferences.5.gz
Description: application/gzip
--- End Message ---
.mso ja.tmac
.TH "APT_PREFERENCES" "5" "15\ \&8 \[u6708]\ \&2015" "APT 1.6~beta1" "APT"
.nh
.SH "\[u540D]\[u524D]"
apt_preferences \- APT 
\[u7528]\[u9078]\[u629E]\[u5236]\[u5FA1]\[u30D5]\[u30A1]\[u30A4]\[u30EB]
.SH "\[u8AAC]\[u660E]"
.PP
\fBsources.list\fR(5)
\[u30D5]\[u30A1]\[u30A4]\[u30EB]\[u306B]\[u5217]\[u6319]\[u3055]\[u308C]\[u305F]\[u5834]\[u6240]\[u304B]\[u3089]\[u53D6]\[u5F97]\[u3057]\[u305F]
Packages
\[u30D5]\[u30A1]\[u30A4]\[u30EB]\[u3084]
Release
\[u30D5]\[u30A1]\[u30A4]\[u30EB]\[u306F]\[u3059]\[u3079]\[u3066]\[u3001]/var/lib/apt/lists
\[u30C7]\[u30A3]\[u30EC]\[u30AF]\[u30C8]\[u30EA]\[u3084]\[u3001]apt\&.conf
\[u30D5]\[u30A1]\[u30A4]\[u30EB]\[u306E]
Dir::State::Lists
\[u5909]\[u6570]\[u3067]\[u6307]\[u5B9A]\[u3057]\[u305F]\[u5834]\[u6240]\[u306B]\[u53D6]\[u5F97]\[u3055]\[u308C]\[u307E]\[u3059]\[u3002]\[u4F8B]\[u3048]\[u3070]\[u3001]debian\&.lcs\&.mit\&.edu_debian_dists_unstable_contrib_binary\-i386_Release
'\" t
.\" Title: apt_preferences
.\"Author: [FAMILY Given]
.\" Generator: DocBook XSL Stylesheets v1.79.1 
.\"  Date: 15\ \&8 月\ \&2015
.\"Manual: APT
.\"Source: APT 1.6~beta1
.\"  Language: Japanese
.\"
.TH "APT_PREFERENCES" "5" "15\ \&8 月\ \&2015" "APT 1.6~beta1" "APT"
.\" -
.\" * Define some portability stuff
.\" -
.\" ~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~
.ie \n(.g .ds Aq \(aq
.el   .ds Aq '
.\" -
.\" * set default formatting
.\" -
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -
.\" * MAIN CONTENT STARTS HERE *
.\" -
.SH "名前"
apt_preferences \- APT 用選択制御ファイル
.SH "説明"
.PP
APT プリファレンスファイル
/etc/apt/preferences
と
/etc/apt/preferences\&.d/
フォルダにある断片ファイルは、インストールするパッケージのバージョンを制御するのに使用します。
.PP
\fBs

Re: [groff] \n[.Y] in release candidates

[Colin Watson]

On Sun, Apr 01, 2018 at 07:34:44AM +0200, Werner LEMBERG wrote:
> And here (hopefully!) a final version that properly takes the `.g'
> flag into account.

Thanks!  I've applied this (with one refinement: I added ".rm Ystring"
to the end, for hygiene).

-- 
Colin Watson   [cjwat...@debian.org]

[groff] \n[.Y] in release candidates

[Colin Watson]

In groff 1.22.4.rc2:

  $ echo '\n[.Y]' | nroff | grep .
  4.rc2

I can see why this happens, of course; but it seems odd for a number
register to contain non-numeric data.  Would it be better to change this
to chop off the dot and everything after it?

Failing that, can anyone suggest an improved version of this code
emitted by man-db that doesn't trip over this ("(4.rc2 >= 2)" in a
conditional doesn't work and ends up emitting noise to the output)?  I
know that testing versions rather than features isn't great, but I
couldn't find a better method in this case.  The purpose of this code is
to load per-locale macros while disabling warnings of the 'file'
category, in order that we don't need to first test whether the relevant
macro file exists, and to only do this with groff >= 1.20.2 because
otherwise we can't disable the resulting warning.  The appropriate
language code is of course substituted for %s.

  .if (\n[.g] & ((\n[.x] > 1) : ((\n[.x] == 1) & (\n[.y] > 20)) : ((\n[.x] == 
1) & (\n[.y] == 20) & (\n[.Y] >= 2 \{\
  .  warn (\n[.warn] - (\n[.warn] / 1048576 % 2 * 1048576))
  .  mso %s.tmac
  .\}

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Release Candidate 1.22.4.rc2

[Colin Watson]

On Thu, Mar 29, 2018 at 08:05:48PM +0200, Ingo Schwarze wrote:
> OpenBSD 6.3 still contains groff-1.22.3p8, we missed the deadline
> for the release, but i will commit the update to OpenBSD-current
> as soon as groff-1.22.4 is officially released, so it will be in
> OpenBSD 6.4 this autumn.

Similarly, I have this RC in Debian experimental and will upload it to
unstable once officially released, so it should be in Debian 10
"buster".  Downstream of that, it's missed the deadline for Ubuntu
18.04, but I expect it to be in Ubuntu 18.10.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Release Candidate 1.22.3.rc1

[Colin Watson]

On Sun, Mar 11, 2018 at 11:11:20PM +0100, Bertrand Garrigues wrote:
> You can also commit these two:
> 
> https://lists.gnu.org/archive/html/groff/2018-03/msg8.html
> https://lists.gnu.org/archive/html/groff/2018-02/msg00026.html

Thanks.  I've done so now.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] Warning in lj4_font(5)

[Colin Watson]

On Sat, Mar 10, 2018 at 09:16:24AM -0500, G. Branden Robinson wrote:
> This one's my fault.  I introduced it in
> a4f9b86065c02f6b2f385c85526a175ab13ae361 on 20 November.  Bag my head.
> 
> I have a patch ready (attached), which is basically yours but uses .IR
> for font-switching instead of embedded font escapes.

Cool, thanks.  Your patch seems reasonable to me.

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] [PATCH] Remove #! lines from more non-executable files

[Colin Watson]

On Fri, Mar 09, 2018 at 07:44:36PM +, Ralph Corderoy wrote:
> Hi Colin,
> 
> > As with my similar previous patch, these are always invoked using the
> > necessary interpreter, so the #!  lines were unnecessary, and caused
> > Debian's `lintian' tool to complain about installed non-executable
> > scripts.
> ...
> >  contrib/gpinyin/subs.pl  |  6 ++
> >  contrib/groffer/main_subs.pl |  6 ++
> >  contrib/groffer/man.pl   |  2 --
> >  contrib/groffer/split_env.sh |  2 --
> >  contrib/groffer/subs.pl  |  2 --
> >  contrib/groffer/version.sh   |  2 --
> >  src/roff/grog/subs.pl|  1 -
> 
> I realise you're doing the minimally intrusive edit to fix the problem,
> but I've always found these suffixes perverse, and wondered why it's not
> groff.exe, etc., to match.

For actual executables, I entirely agree with you, of course.  An
extension on those requires the user to type additional characters (at
least on Unix; Windows at least has the justification that the user
doesn't have to type the ".exe" part), and it encodes the implementation
in the interface in an inappropriate way that can cause problems later.

The same considerations don't really apply to internal library files,
though, which these are.  If subs.pl were rewritten in Python for some
reason, then groffer would have to do a whole lot more than changing
"require 'subs.pl'" to "require 'subs.py'". :-)  It's useful to have
exactly one of a #! line or a file extension, because they serve the
purpose of cluing editors into which syntax highlighting mode to enable
without the need for embedding editor-specific metadata in the file.
And both .pl and .sh have a long history in their respective communities
of indicating a library file in the relevant language that's loaded by
some means other than fork/exec.  So, for all these reasons, I wouldn't
be in favour of removing the extensions here.

> Is another solution to remove these silly, foreign-platform, suffixes,
> and make the files executable?

For the most part, they definitely shouldn't be executable, because
they're not intended to be executed directly: for example, the Perl
components are libraries loaded using 'require'.  Setting the executable
bit on them would be quite misleading.

There is one possible exception in the above list: split_env.sh is at
least theoretically useful when executed directly, even though groffer
specifically invokes it using 'sh'.  But it's in fact an internal helper
for groffer, so I don't think it really makes sense to suggest that its
executability is part of its interface.

Thanks,

-- 
Colin Watson   [cjwat...@debian.org]