Re: [BUG] italics run past where they should

[Alejandro Colomar]

Hi Branden,

On 8/17/22 07:34, G. Branden Robinson wrote:

One difference to keep in mind is that Debian's groff is patched to
recognize a "GROFF_SGR" environment variable, and disables SGR output
from grotty(1) by default.  So perhaps you could see if you can
reproduce it in Debian stable using its groff, but with GROFF_SGR=1 in
the environment.


Hmm, you found it.

$ GROFF_SGR=1 man membarrier

with Debian's groff 1.22.4 triggers the bug in Sid, but not in stable.
What does this tell you?

Cheers,

Alex



Regards,
Branden


--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: Savannah bug->email gateway problems affecting groff

[Bob Proulx]

G. Branden Robinson wrote:
> [CCing Bob Proulx]
> 
> This message just is not showing up on the savannah-hackers-public list
> (after two attempts 6 hours apart), so maybe there is more than one
> email problem going on.

I see one of the messages in the archive okay.


https://lists.gnu.org/archive/html/savannah-hackers-public/2022-08/msg6.html

I am thinking that your message was simply held for new sender
moderation.  Since all new senders to a mailing list, and that is
every mailing list is independent because that's how Mailman does
things, are held for human moderation as an anti-spam measure.  Having
been approved however all subsequent messages will be sent through
without delay.  I know I have been busy in real life and haven't been
able to attend the keyboard much the past few days.  But others in the
team should be able to handle things.  I don't know why a second
message would not have gone through.  But perhaps if the human thought
it was a duplicate they might have discarded it while trying to be
helpful.  I don't know.

In any case I made a reply to that message a few minutes ago.

> groff list subscribers may wish to consult
> https://lists.gnu.org/archive/html/savannah-hackers-public/2022-08/index.html
> for context.
> 
> I'd like to push some commits but I'm worried that gateway might be
> affected too.

When you say gateway are you talking about the git server?  I don't
usually think of a git server as a gateway.  It's just another server
system.  The git server is a separate virtual machine server from the
Savannah web frontend virtual machine server.  This is done to
separate services which helps against malicious action from the
hostile Internet.  Like we are experiencing now on the web frontend
system.  If you are talking about git hook commit email notifications
then that comes directly from the git server.  Commit mail
notifications are sent directly from the git server itself.

Bob

Re: Savannah bug->email gateway problems affecting groff

[Dave Kemper]

On 8/17/22, Bob Proulx  wrote:
> I am thinking that your message was simply held for new sender
> moderation.

Hi Bob,

In the past couple days I've made several updates to groff Savannah
tickets, which have always generated an immediate email to me and, not
too long after, one in the lists.gnu.org archive of the bug-groff
email list.  Neither of these has been getting the emails since August
12.  (See, for instance, http://savannah.gnu.org/bugs/?62901, which
has three comments by longtime groff project members, requiring no
moderation; none of these appear in the email archive at
http://lists.gnu.org/r/bug-groff/2022-08/.)

Similarly, a comment I added to
http://savannah.nongnu.org/support/?110692 never showed up in my inbox
(past comments I've submitted here have always shown up immediately)
and is not reflected in
http://lists.gnu.org/r/savannah-hackers/2022-08/.

> I know I have been busy in real life and haven't been
> able to attend the keyboard much the past few days.  But others in the
> team should be able to handle things.

I'm happy to redirect this query, but I don't know who else to contact
about this, and there seems little point in filing a Savannah ticket
over it, since no one is getting notified about them while email
notifications are down.

Re: [BUG] italics run past where they should

[G. Branden Robinson]

Hi Alex,

At 2022-08-17T09:52:27+0200, Alejandro Colomar wrote:
> On 8/17/22 07:34, G. Branden Robinson wrote:
> > One difference to keep in mind is that Debian's groff is patched to
> > recognize a "GROFF_SGR" environment variable, and disables SGR
> > output from grotty(1) by default.  So perhaps you could see if you
> > can reproduce it in Debian stable using its groff, but with
> > GROFF_SGR=1 in the environment.
> 
> Hmm, you found it.
> 
> $ GROFF_SGR=1 man membarrier
> 
> with Debian's groff 1.22.4 triggers the bug in Sid, but not in stable.
> What does this tell you?

That I don't have enough information yet.

Render the page directly with groff.

(export GROFF_SGR=1; zcat $(man -w membarrier) | groff -Tutf8 -man > grotty.out)

Use a text editor or hex dumper to confirm that SGR escape sequences are
present in "grotty.out".

Then view the file with a variety of pagers to see which, if any,
misbehave.

Regards,
Branden


signature.asc
Description: PGP signature

Re: [BUG] italics run past where they should

[Alejandro Colomar]

On 8/17/22 13:01, G. Branden Robinson wrote:

That I don't have enough information yet.

Render the page directly with groff.

(export GROFF_SGR=1; zcat $(man -w membarrier) | groff -Tutf8 -man > grotty.out)


I guess you don't mind that I use groff 1.23.0 since both seem to behave 
the same.  It'll be simpler for me.


Will come back with the results.


OpenPGP_signature
Description: OpenPGP digital signature

Re: [BUG] italics run past where they should

[Alejandro Colomar]

Hi Branden,

On 8/17/22 13:01, G. Branden Robinson wrote:

$ GROFF_SGR=1 man membarrier

with Debian's groff 1.22.4 triggers the bug in Sid, but not in stable.
What does this tell you?


That I don't have enough information yet.

Render the page directly with groff.

(export GROFF_SGR=1; zcat $(man -w membarrier) | groff -Tutf8 -man > grotty.out)

Use a text editor or hex dumper to confirm that SGR escape sequences are
present in "grotty.out".

Then view the file with a variety of pagers to see which, if any,
misbehave.


Reproduced:

 1986  <$(man -w membarrier) groff -Tutf8 -man > grotty.out
 1987  less grotty.out
 1988  less -R grotty.out
 1989  batcat grotty.out


The page I used is not compressed, since it's installed from source. 
zcatting Debian's page should produce the same.


Both less and batcat reproduce the bug.  I attached the page so that you 
can inspect it (it would be interesting to know if your less(1) with my 
grotty.out reproduces the bug).  I;ll also show inline here the relevant 
part of the file:


$ grep SYS_membarrier   20 20 20 20 20 20 20 1b  5b 31 6d 69 6e 74 20 73  | 
.[1mint s|
0010  79 73 63 61 6c 6c 28 53  59 53 5f 6d 65 6d 62 61 
|yscall(SYS_memba|
0020  72 72 69 65 72 2c 20 69  6e 74 20 1b 5b 34 6d 1b  |rrier, int 
.[4m.|
0030  5b 32 32 6d 63 6d 64 1b  5b 32 34 6d 1b 5b 31 6d 
|[22mcmd.[24m.[1m|
0040  2c 20 75 6e 73 69 67 6e  65 64 20 69 6e 74 20 1b  |, unsigned 
int .|
0050  5b 34 6d 1b 5b 32 32 6d  66 6c 61 67 73 1b 5b 32 
|[4m.[22mflags.[2|
0060  34 6d 1b 5b 31 6d 2c 20  69 6e 74 20 1b 5b 34 6d  |4m.[1m, int 
.[4m|
0070  1b 5b 32 32 6d 63 70 75  5f 69 64 1b 5b 32 34 6d 
|.[22mcpu_id.[24m|

0080  1b 5b 31 6d 29 3b 1b 5b  30 6d 0a |.[1m);.[0m.|
008b


Cheers,

Alex


--
Alejandro Colomar

MEMBARRIER(2)  Linux Programmer’s Manual 
MEMBARRIER(2)

NAME
   membarrier - issue memory barriers on a set of threads

LIBRARY
   Standard C library (libc, -lc)

SYNOPSIS
   #include  /* Definition of MEMBARRIER_* 
constants */
   #include   /* Definition of SYS_* 
constants */
   #include 

   int syscall(SYS_membarrier, int cmd, unsigned int 
flags, int cpu_id);

   Note: glibc provides no wrapper for membarrier(), 
necessitating the use
   of syscall(2).

DESCRIPTION
   The  membarrier() system call helps reducing the overhead of 
the memory
   barrier instructions required to order memory  accesses  on  multi‐core
   systems.   However,  this system call is heavier than a memory barrier,
   so using it effectively is not as simple as replacing  memory  
barriers
   with this system call, but requires understanding of the details below.

   Use of memory barriers needs to be done taking into account that a mem‐
   ory  barrier  always needs to be either matched with its memory barrier
   counterparts, or that the architecture’s memory model  doesn’t  require
   the matching barriers.

   There  are cases where one side of the matching barriers (which we will
   refer to as "fast side") is executed much more  often  than  the  other
   (which  we  will  refer to as "slow side").  This is a prime target for
   the use of membarrier().  The key idea is to replace, for these 
 match‐
   ing  barriers,  the fast‐side memory barriers by simple compiler barri‐
   ers, for example:

   asm volatile ("" : : : "memory")

   and replace the slow‐side memory barriers by calls to 
membarrier().

   This will add overhead to the slow side, and remove overhead  from  the
   fast side, thus resulting in an overall performance increase as long as
   the  slow  side  is  infrequent enough that the overhead of the 
membar‐
   rier() calls does not outweigh the performance gain on the fast 
side.

   The cmd argument is one of the following:

   MEMBARRIER_CMD_QUERY (since Linux 4.3)
  Query the set of supported commands.  The return  value  of  the
  call is a bit mask of supported commands.  
MEMBARRIER_CMD_QUERY,
  which  has the value 0, is not itself included in this bit mask.
  This command is always supported (on kernels where  
membarrier()
  is provided).

   MEMBARRIER_CMD_GLOBAL (since Linux 4.16)
  Ensure  that  all  threads from all processes on the system pass
  through a state where all  memory  accesses  to  user‐space  ad‐
  dresses match program order between entry to and return from the
  membarrier()  system  call.   All th

Re: deroff availability

[DJ Chase]

On Wed Aug 17, 2022 at 1:27 AM EDT, Jeff Conrad wrote:
> Oops ...
>
> Scratch what I was going to suggest ...
>
> I only asked because you mentioned that roff2text(1) does a poor job of
> formatting.  Reading more carefully, formatting wasn’t your objective.

I suppose I could have been clearer there, sorry.

> I assume you’ve tried something like
>
> roff2text  | tr -cs "[:alpha:]" "[\n*]"
>
> or
>
> nroff   | tr -cs "[:alpha:]" "[\n*]"
>
> But this obviously won’t work for diction or style.

It works for style, but it’s really a problem with diction. Diction
outputs line numbers of problematic sentences, which obviously don’t
corrispond to the source line-numbers.

Cheers,
-- 
DJ Chase
They, Them, Theirs

Re: deroff availability

[Damian McGuckin]

On Wed, 17 Aug 2022, DJ Chase wrote:


It works for style, but it?s really a problem with diction. Diction
outputs line numbers of problematic sentences, which obviously don?t
corrispond to the source line-numbers.


I have never refered to the line number in over 30 years of using diction.

Stay safe - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer

Re: [BUG] italics run past where they should

[Alejandro Colomar]

Hi Branden,

On 8/17/22 13:24, Alejandro Colomar wrote:

Reproduced:

  1986  <$(man -w membarrier) groff -Tutf8 -man > grotty.out
  1987  less grotty.out
  1988  less -R grotty.out
  1989  batcat grotty.out


The page I used is not compressed, since it's installed from source. 
zcatting Debian's page should produce the same.


Both less and batcat reproduce the bug.  I attached the page so that you 
can inspect it (it would be interesting to know if your less(1) with my 
grotty.out reproduces the bug).  I;ll also show inline here the relevant 
part of the file:


$ grep SYS_membarrier 0010  79 73 63 61 6c 6c 28 53  59 53 5f 6d 65 6d 62 61 
|yscall(SYS_memba|
0020  72 72 69 65 72 2c 20 69  6e 74 20 1b 5b 34 6d 1b  |rrier, int 
.[4m.|
0030  5b 32 32 6d 63 6d 64 1b  5b 32 34 6d 1b 5b 31 6d 
|[22mcmd.[24m.[1m|
0040  2c 20 75 6e 73 69 67 6e  65 64 20 69 6e 74 20 1b  |, unsigned 
int .|
0050  5b 34 6d 1b 5b 32 32 6d  66 6c 61 67 73 1b 5b 32 
|[4m.[22mflags.[2|
0060  34 6d 1b 5b 31 6d 2c 20  69 6e 74 20 1b 5b 34 6d  |4m.[1m, int 
.[4m|
0070  1b 5b 32 32 6d 63 70 75  5f 69 64 1b 5b 32 34 6d 
|.[22mcpu_id.[24m|

0080  1b 5b 31 6d 29 3b 1b 5b  30 6d 0a |.[1m);.[0m.|
008b



I made the reproducer smaller, so that it's reasier to investigate.
I produced a one-liner file, and then script(1)ed the reproduction of 
the bug (I made the terminal shorter, 80x3, to get less blank lines from 
less(1)):


$ grep SYS_membarrier offense_line.out
$ script
$ less -R offense_line.out
/cmd[ENTER]
q
$ ^D


Attached you'll find the offense line, and the typescript log.

Cheers,

Alex

--
Alejandro Colomar

   int syscall(SYS_membarrier, int cmd, unsigned int 
flags, int cpu_id);


typescript
Description: Binary data


OpenPGP_signature
Description: OpenPGP digital signature

Re: deroff availability

[DJ Chase]

On Wed Aug 17, 2022 at 1:16 AM EDT, Laurens Kils-Hütten wrote:
> The Arch Linux User Repository (AUR) is a valuable source here.
>
> I found https://aur.archlinux.org/packages/deroff 
> which in turn points to http://www.moria.de/~michael/deroff/
> et voilà!

Thank you. I’ll try compiling this later. If it works, I’ll try
packaging it for Fedora in case other users have the same problem.

Cheers,
-- 
DJ Chase
They, Them, Theirs

Re: [BUG] italics run past where they should

[G. Branden Robinson]

Hi Alex,

Is the subject line still accurate?  I thought the problem we were
chasing at this point was that all character attributes got shut off
after the end of a highlighted match when using the pager to search for
text in a man page.

Regardless, I'm not able to reproduce _any_ misrendering.

At 2022-08-17T13:24:17+0200, Alejandro Colomar wrote:
> Reproduced:
> 
>  1986  <$(man -w membarrier) groff -Tutf8 -man > grotty.out
>  1987  less grotty.out
>  1988  less -R grotty.out
>  1989  batcat grotty.out
> 
> The page I used is not compressed, since it's installed from source.
> zcatting Debian's page should produce the same.

Yeah, I'm not too worried about that part of the procedure.

> Both less and batcat reproduce the bug.  I attached the page so that
> you can inspect it (it would be interesting to know if your less(1)
> with my grotty.out reproduces the bug).

It does not reproduce the issue for me.  Your grotty.out file works fine
with both less 551 and less 581.2 (self-compiled), and with batcat
0.12.1, on my system.

I didn't mess with running less without its `-R` flag.  That is known
and documented to not handle SGR escape sequences on input.

> I;ll also show inline here the relevant part of the file:

> $ grep SYS_membarrier    20 20 20 20 20 20 20 1b  5b 31 6d 69 6e 74 20 73  | .[1mint s|
> 0010  79 73 63 61 6c 6c 28 53  59 53 5f 6d 65 6d 62 61  |yscall(SYS_memba|
> 0020  72 72 69 65 72 2c 20 69  6e 74 20 1b 5b 34 6d 1b  |rrier, int .[4m.|
> 0030  5b 32 32 6d 63 6d 64 1b  5b 32 34 6d 1b 5b 31 6d  |[22mcmd.[24m.[1m|
> 0040  2c 20 75 6e 73 69 67 6e  65 64 20 69 6e 74 20 1b  |, unsigned int .|
> 0050  5b 34 6d 1b 5b 32 32 6d  66 6c 61 67 73 1b 5b 32  |[4m.[22mflags.[2|
> 0060  34 6d 1b 5b 31 6d 2c 20  69 6e 74 20 1b 5b 34 6d  |4m.[1m, int .[4m|
> 0070  1b 5b 32 32 6d 63 70 75  5f 69 64 1b 5b 32 34 6d  |.[22mcpu_id.[24m|
> 0080  1b 5b 31 6d 29 3b 1b 5b  30 6d 0a |.[1m);.[0m.|
> 008b

I'm afraid this doesn't illuminate much; it looks congruent with my
dissection of similar output earlier in the thread[1][2], and moreover,
I thought the problem only manifested when you performed a search in the
pagers?  Any successful match in a pager that highlights matches will
(1) certainly change the series of escape sequences sent to the terminal
and (2) not be reflected in the grotty.out file.

I guess it is time to script(1) your pager session now.  Use grotty.out
since it is known to be sane.  We are, however, at this point, not
troubleshooting groff, but the pagers and/or terminal emulators.

Regards,
Branden

[1] https://lists.gnu.org/archive/html/groff/2022-08/msg00132.html
[2] We can observe one "new" sequence (due to more input context), a
parameter of zero to the CSI "m" sequence.  This is also known as
"SGR 0" and it turns off all graphic renditions that the terminal
regards as non-default.


signature.asc
Description: PGP signature

Re: [BUG] italics run past where they should

[G. Branden Robinson]

Hi Alex,

Good news--I finally see a problem.

At 2022-08-17T13:49:35+0200, Alejandro Colomar wrote:
> I made the reproducer smaller, so that it's reasier to investigate.  I
> produced a one-liner file, and then script(1)ed the reproduction of
> the bug (I made the terminal shorter, 80x3, to get less blank lines
> from less(1)):

In the future, when sending around a typescript that includes
screen-clearing escape sequences, it might be a good idea to capture
timing information.  Modern script(1) supports this, or you can use the
classic tool beloved of NetHack players, ttyrec(1).

> Attached you'll find the offense line, and the typescript log.

Yes, I see trouble.

Script started on 2022-08-17 13:47:53+02:00 [TERM="xterm-256color" 
TTY="/dev/pts/4" COLUMNS="80" LINES="3"]
ESC[?2004hESC]0;alx@asus5775: 
~/tmp^GESC[01;32malx@asus5775ESC[00m:ESC[01;34m~/tmpESC[00m$ less -R 
offense_line.out
ESC[?2004l^MESC[?1049hESC[22;0;0tESC[?1hESC=^M   ESC[1mint 
syscall(SYS_membarrier, int ESC[4mESC[22mcmdESC[24mESC[1m, unsigned int 
ESC[4mESC[22mflagsESC[24mESC[1m, int 
ESESC[4mESC[22mcpu_idESC[24mESC[1m);ESC[0mESC[m

Okay, that looks like a riot of noise, but the important thing is that
we can see around "cmd" pair of CSI escape sequences, 22 m and 24 m.

As discussed earlier in the thread, "22" restores the "intensity" to
"normal" (i.e., in this case, turns bold off) and "24" turns off
underlining.  A CSI 1 m sequence, which turns bold back on, follows.

ESC[7moffense_line.out 
(END)ESC[27mESC[K^MESC[K/ESC[KcESC[KmESC[Kd^MESC[K...skipping...

This is you doing your search.

   ESC[1mint syscall(SYS_membarrier, int 
ESC[4mESC[22mESC[7mcmdESC[27mESC[1m, unsigned int 
ESC[4mESC[22mflagsESC[24mESC[1m, int 
ESC[4mESC[22mcpu_idESC[24mESC[1m);ESC[0mESESC[m

This is the same line from the document, with the match now highlighted.

We see that after CSI 22 m, the pager has interpolated a CSI 7 m
sequence before the match and a CSI 27 m afterward.  This makes sense;
again as discussed before, these are "inverse" and "positive"
respectively.

But where's the CSI 24 m sequence we originally had?

This is the bug.

Regards,
Branden


signature.asc
Description: PGP signature

Re: deroff availability

[DJ Chase]

On Wed Aug 17, 2022 at 7:44 AM EDT, Damian McGuckin wrote:
> On Wed, 17 Aug 2022, DJ Chase wrote:
>
> > It works for style, but it?s really a problem with diction. Diction
> > outputs line numbers of problematic sentences, which obviously don?t
> > corrispond to the source line-numbers.
>
> I have never refered to the line number in over 30 years of using diction.

Interesting. What do you do for really long files that aren’t split into
chapters?

Cheers,
-- 
DJ Chase
They, Them, Theirs

Re: [BUG] italics run past where they should

[Alejandro Colomar]

Hi Branden,

On 8/17/22 14:12, G. Branden Robinson wrote:

Hi Alex,

Good news--I finally see a problem.

At 2022-08-17T13:49:35+0200, Alejandro Colomar wrote:

I made the reproducer smaller, so that it's reasier to investigate.  I
produced a one-liner file, and then script(1)ed the reproduction of
the bug (I made the terminal shorter, 80x3, to get less blank lines
from less(1)):


In the future, when sending around a typescript that includes
screen-clearing escape sequences, it might be a good idea to capture
timing information.  Modern script(1) supports this, or you can use the
classic tool beloved of NetHack players, ttyrec(1).


Attached you'll find the offense line, and the typescript log.


Yes, I see trouble.

Script started on 2022-08-17 13:47:53+02:00 [TERM="xterm-256color" TTY="/dev/pts/4" 
COLUMNS="80" LINES="3"]
ESC[?2004hESC]0;alx@asus5775: 
~/tmp^GESC[01;32malx@asus5775ESC[00m:ESC[01;34m~/tmpESC[00m$ less -R 
offense_line.out
ESC[?2004l^MESC[?1049hESC[22;0;0tESC[?1hESC=^M   ESC[1mint 
syscall(SYS_membarrier, int ESC[4mESC[22mcmdESC[24mESC[1m, unsigned int 
ESC[4mESC[22mflagsESC[24mESC[1m, int 
ESESC[4mESC[22mcpu_idESC[24mESC[1m);ESC[0mESC[m

Okay, that looks like a riot of noise, but the important thing is that
we can see around "cmd" pair of CSI escape sequences, 22 m and 24 m.

As discussed earlier in the thread, "22" restores the "intensity" to
"normal" (i.e., in this case, turns bold off) and "24" turns off
underlining.  A CSI 1 m sequence, which turns bold back on, follows.

ESC[7moffense_line.out 
(END)ESC[27mESC[K^MESC[K/ESC[KcESC[KmESC[Kd^MESC[K...skipping...

This is you doing your search.

ESC[1mint syscall(SYS_membarrier, int 
ESC[4mESC[22mESC[7mcmdESC[27mESC[1m, unsigned int 
ESC[4mESC[22mflagsESC[24mESC[1m, int 
ESC[4mESC[22mcpu_idESC[24mESC[1m);ESC[0mESESC[m

This is the same line from the document, with the match now highlighted.

We see that after CSI 22 m, the pager has interpolated a CSI 7 m
sequence before the match and a CSI 27 m afterward.  This makes sense;
again as discussed before, these are "inverse" and "positive"
respectively.

But where's the CSI 24 m sequence we originally had?

This is the bug.


It seems the bug is only present in less 590.  I compiled from source 
less 581.2 and a few previous to that, and they don't have it, and less 
608, which is also free of the bug.


It seems to be the same issue reported here:
.

It's curious that batcat had the same bug at the same time.
I reported it here:



Cheers,

Alex


--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: [BUG] italics run past where they should

[Alejandro Colomar]

Hi Magnus,

On 8/17/22 15:45, Magnus Nordenadler wrote:

On Wed Aug 17, 2022 at 3:08 PM CEST, Alejandro Colomar wrote:

It's curious that batcat had the same bug at the same time.


Is it? Bat (or batcat in Debian) is a replacement for cat, not less; it
doesn't provide its own pager. From the readme:


Ahh, that makes more sense.  I didn't know about batcat before, and I 
found random people on the internet talking about it as a pager, so I 
trusted that blindly.





By default, bat pipes its own output to a pager (e.g. less) if the
output is too large for one screen.


/Magnus


Cheers,

Alex

--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: [BUG] italics run past where they should

[Magnus Nordenadler]

On Wed Aug 17, 2022 at 3:08 PM CEST, Alejandro Colomar wrote:
> It's curious that batcat had the same bug at the same time.

Is it? Bat (or batcat in Debian) is a replacement for cat, not less; it
doesn't provide its own pager. From the readme:

> By default, bat pipes its own output to a pager (e.g. less) if the
> output is too large for one screen.

/Magnus

Re: deroff availability

[Damian McGuckin]

On Wed, 17 Aug 2022, DJ Chase wrote:


I have never refered to the line number in over 30 years of using diction.


Interesting. What do you do for really long files that aren?t split into
chapters?


Split them into chapters.

How do you handle a program with 1 lines. Hopefully you split into 
modules and indivual routines.  A document is a program.


Stay safe - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 
2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not 
wanted here Views & opinions here are mine and not those of any past or 
present employer

Re: About groff and dformat

[Thomas Dupond]

Riza Dindir  wrote:

> Hello Thomas,

Hi!

> Finally I was able to use the dformat without problems.

Congrats! :)

> The problem was the order in the pipe. In the original paper the
> author provided an example on the order of the preprocessors. I did
> not have the correct order. It should have been
> 
> !sh dformat.sh format-test.ms | pic | tbl | eqn | groff -ms > main.ps

I have the same result as you have. I don't know why the order matters,
my knowledge of prepocessors is lacking.

However I recreated your document easily with:

$ dformat.awk format-test.ms | groff -ms -pe -Tps > main.ps

I did not use tbl (with the -t switch) and it worked perfectly.  You
don't need tbl for this particular document.

I noted that when producing the pdf (either with groff -Tpdf or ps2pdf),
the dotted lines appear thinner than in the postscript file.  To be
honest, I would not have noticed them if I only looked at the pdf file.

> That solved the problem, and I can create the dformat in the document.
> 
> Thanks for the help.
> 
> Regards,
> Riza
> 
> On Wed, Aug 17, 2022 at 6:59 AM Riza Dindir  wrote:
> >
> > Hello Thomas,
> >
> > To make sure, I have this in my dformat.sh
> >
> > awk -f dformat.awk $*
> >
> > I tried with the dformat.awk that I have found
> > (https://github.com/arnoldrobbins/dformat) and the dformat.awk that
> > you have pointed out (https://github.com/sathlan/dformat). But still I
> > am getting the same error message and the dformat picture is displayed
> > partially.
> >
> > What is the version of groff (mine is 1.22.4), and awk (mine is
> > 20210215) that you are using? Clearly I must be doing something wrong.
> >
> > Although I could just remove inline equations from the dformat spec in
> > the document, I would like to understand what is wrong.
> >
> > Regards,
> > Riza
> >
> > On Tue, Aug 16, 2022 at 9:59 PM Thomas Dupond  wrote:
> > >
> > > Riza Dindir  wrote:
> > >
> > > > Hello All,
> > >
> > > Hi!
> > >
> > > > I am trying to get dformat to work. What I did was this.
> > > >
> > > > I first tried to include an example dformat script into my document. I
> > > > got the first example from the troff.org site
> > > > (https://troff.org/prog.html). The first example for the dformat
> > > > processor. That example has been using inline equations/math
> > > > expressions.
> > > >
> > > > Then I went and got the dformat awk script. That script I got from
> > > > (https://github.com/arnoldrobbins/dformat) which might not be the
> > > > original script.
> > >
> > > I know the following github repo is a transcript of the dformat by
> > > Jon L. Bentley:
> > >
> > > https://github.com/sathlan/dformat
> > >
> > > I had no problem generating your document with it. :)
> > >
> > > > Anyways I have this simple document.
> > > >
> > > > .TL
> > > > Some format
> > > > .NH
> > > > deneme
> > > > .EQ
> > > > delim @@
> > > > .EN
> > > > .LP
> > > > .begin dformat
> > > > style bitwid 0.08
> > > > style charwid 0
> > > > style recspread 0.3
> > > > noname
> > > > --16 Frame
> > > > --16 Frame
> > > >   A1:   --16 Frame
> > > > --16 Frame
> > > > --8-dashed ...
> > > > noname
> > > >   A2:   8--8 Flags
> > > > 8--8 Status
> > > > --8 @roman Chunk sub 1@
> > > >   B1:   --8 @roman Chunk sub 2@
> > > > --8-dashed ...
> > > > --8 @roman Chunk sub m@
> > > > 16--16 CRC
> > > >   A3:   8--8 Flags
> > > > noname
> > > >   B2:   8--8 @roman Data sub 1@
> > > > 8--8 @roman Data sub 2@
> > > > 8--8 @roman Data sub 3@
> > > > 8--8 @roman Data sub 4@
> > > > --8-dashed ...
> > > > 8--8 @roman Data sub n-1@
> > > > 8--8 @roman Data sub n@
> > > > 6--6 Length
> > > >   B3:   10--10 Channel #
> > > > pic line dotted from A1.sw to A2.nw
> > > > pic line dotted from A1.se to A3.ne
> > > > pic line dotted from B1.sw to B2.nw
> > > > pic line dotted from B1.se to B3.ne
> > > > .end
> > > >
> > > > And am running the command line as
> > > >
> > > > !sh dformat.sh format-test.ms | eqn | pic | groff -ms > main.ps
> > > >
> > > > The command says this
> > > >
> > > > pic::98: syntax error before '\'
> > > > pic::98: giving up on this picture
> > > >
> > > > Does anybody use dformat? Which dformat script are you using?
> > > >
> > > > Regards,
> > > > Riza
> > > >
> > > >
> > >
> > > --
> > > Regards,
> > > Thomas

-- 
Regards,
Thomas

Re: About groff and dformat

[Riza Dindir]

Hello Thomas

I did need tbl, abd chem for the original document, where I wanted to
create a document that had all the elements and preprocessors used in
troff.

One thing that I realized is this. When you use chem, you can not have
the # character in the dformat or other preprocessor blocks, since the
chem macro package does take # as a comment and erases all the
characters to the end of the line. Now in that particular dformat
example that I used, there was an expression "Channel #". When you
process this with dformat and then with chem, or the other way around,
or have chem in the pipeline, chem will erase all the characters
including the # from the generated troff document. Hence the error.

So if chem is  being used, better change the # character to something
else, or use \N'35' (ascii 35 is the # character). But this might not
work because the font used might not have that character, or the
character with ascii code 35 might not be existing. Since I do not
know how groff works at this level, I am not sure.

If one wants to enter a character and has the ascii code for that,
would this be the correct usage? That is entering a character that one
knows the ascii numeric value of, can one use the \N'xx' control
sequence? Would that change based on the font, or something else? Or
will the control sequence \N'35' always be mapped to the character #?

Regards,
Riza

On Wed, Aug 17, 2022 at 6:48 PM Thomas Dupond  wrote:
>
> Riza Dindir  wrote:
>
> > Hello Thomas,
>
> Hi!
>
> > Finally I was able to use the dformat without problems.
>
> Congrats! :)
>
> > The problem was the order in the pipe. In the original paper the
> > author provided an example on the order of the preprocessors. I did
> > not have the correct order. It should have been
> >
> > !sh dformat.sh format-test.ms | pic | tbl | eqn | groff -ms > main.ps
>
> I have the same result as you have. I don't know why the order matters,
> my knowledge of prepocessors is lacking.
>
> However I recreated your document easily with:
>
> $ dformat.awk format-test.ms | groff -ms -pe -Tps > main.ps
>
> I did not use tbl (with the -t switch) and it worked perfectly.  You
> don't need tbl for this particular document.
>
> I noted that when producing the pdf (either with groff -Tpdf or ps2pdf),
> the dotted lines appear thinner than in the postscript file.  To be
> honest, I would not have noticed them if I only looked at the pdf file.
>
> > That solved the problem, and I can create the dformat in the document.
> >
> > Thanks for the help.
> >
> > Regards,
> > Riza
> >
> > On Wed, Aug 17, 2022 at 6:59 AM Riza Dindir  wrote:
> > >
> > > Hello Thomas,
> > >
> > > To make sure, I have this in my dformat.sh
> > >
> > > awk -f dformat.awk $*
> > >
> > > I tried with the dformat.awk that I have found
> > > (https://github.com/arnoldrobbins/dformat) and the dformat.awk that
> > > you have pointed out (https://github.com/sathlan/dformat). But still I
> > > am getting the same error message and the dformat picture is displayed
> > > partially.
> > >
> > > What is the version of groff (mine is 1.22.4), and awk (mine is
> > > 20210215) that you are using? Clearly I must be doing something wrong.
> > >
> > > Although I could just remove inline equations from the dformat spec in
> > > the document, I would like to understand what is wrong.
> > >
> > > Regards,
> > > Riza
> > >
> > > On Tue, Aug 16, 2022 at 9:59 PM Thomas Dupond  wrote:
> > > >
> > > > Riza Dindir  wrote:
> > > >
> > > > > Hello All,
> > > >
> > > > Hi!
> > > >
> > > > > I am trying to get dformat to work. What I did was this.
> > > > >
> > > > > I first tried to include an example dformat script into my document. I
> > > > > got the first example from the troff.org site
> > > > > (https://troff.org/prog.html). The first example for the dformat
> > > > > processor. That example has been using inline equations/math
> > > > > expressions.
> > > > >
> > > > > Then I went and got the dformat awk script. That script I got from
> > > > > (https://github.com/arnoldrobbins/dformat) which might not be the
> > > > > original script.
> > > >
> > > > I know the following github repo is a transcript of the dformat by
> > > > Jon L. Bentley:
> > > >
> > > > https://github.com/sathlan/dformat
> > > >
> > > > I had no problem generating your document with it. :)
> > > >
> > > > > Anyways I have this simple document.
> > > > >
> > > > > .TL
> > > > > Some format
> > > > > .NH
> > > > > deneme
> > > > > .EQ
> > > > > delim @@
> > > > > .EN
> > > > > .LP
> > > > > .begin dformat
> > > > > style bitwid 0.08
> > > > > style charwid 0
> > > > > style recspread 0.3
> > > > > noname
> > > > > --16 Frame
> > > > > --16 Frame
> > > > >   A1:   --16 Frame
> > > > > --16 Frame
> > > > > --8-dashed ...
> > > > > noname
> > > > >   A2:   8--8 Flags
> > > > > 8--8 Status
> > > > > --8 @roman Chunk sub 1@
> > > > >   B1:   --8 @roman Chunk sub 2@
> > > > > --8-dashed ...
> > > > >

Updated install-font.sh

[Peter Schaffter]

The install-font script at

  https://www.schaffter.ca/mom/mom-05.html#install-font

isn't officially part of groff or mom, but I'm posting this on the
list anyway.  Someday, someone will grab the damned thing and make
it part of groff.

Robert Goulding discovered that contents of the
system font/devps/download file must be present in
site-font/devps/download.  Similarly, freeeuro.pfa and EURO need
to be copied into site/font/devpdf/ and registered in its download
file.

I've updated the script so that it takes care of these details when
being run for the first time.

I'll also be amending the long-form font installation instructions
in the appendices to the momdocs.

-- 
Peter Schaffter
https://www.schaffter.ca

Re: Updated install-font.sh

[Robert Goulding]

Thanks, Peter  - and I will second the plea that it be made somehow part of
the official distribution. It is really a fantastic little tool - Robert.

On Wed, Aug 17, 2022 at 2:29 PM Peter Schaffter  wrote:

> The install-font script at
>
>   https://www.schaffter.ca/mom/mom-05.html#install-font
>
> isn't officially part of groff or mom, but I'm posting this on the
> list anyway.  Someday, someone will grab the damned thing and make
> it part of groff.
>
> Robert Goulding discovered that contents of the
> system font/devps/download file must be present in
> site-font/devps/download.  Similarly, freeeuro.pfa and EURO need
> to be copied into site/font/devpdf/ and registered in its download
> file.
>
> I've updated the script so that it takes care of these details when
> being run for the first time.
>
> I'll also be amending the long-form font installation instructions
> in the appendices to the momdocs.
>
> --
> Peter Schaffter
> https://www.schaffter.ca
>
>

-- 
Robert Goulding
Director, John J. Reilly Center for Science, Technology, and Values;
Director, Program in History and Philosophy of Science;
Assoc. Professor, Program of Liberal Studies,
Fellow, Medieval Institute,
University of Notre Dame.

Re: Updated install-font.sh

[Dave Kemper]

On 8/17/22, Robert Goulding  wrote:
> Thanks, Peter  - and I will second the plea that it be made somehow part of
> the official distribution.

The steps needed to make this happen are outlined in the feature
request at http://savannah.gnu.org/bugs/?60930.  T. Kurt Bond started
on this work but has had limited time to finish it.  However, not all
the tasks need to be done by one person, so if anyone wants to
contribute by taking one on, that ticket is the best place to
coordinate that and ask about what still needs to be done.

Re: Savannah bug->email gateway problems affecting groff

[Dave Kemper]

Fixed now.  See
http://lists.gnu.org/r/savannah-hackers/2022-08/msg00013.html for the
gory details.  Thank you, Bob!

Re: Savannah bug->email gateway problems affecting groff

[Bob Proulx]

Dave Kemper wrote:
> In the past couple days I've made several updates to groff Savannah
> tickets, which have always generated an immediate email to me and, not
> too long after, one in the lists.gnu.org archive of the bug-groff
> email list.  Neither of these has been getting the emails since August
> 12.  (See, for instance, http://savannah.gnu.org/bugs/?62901, which
> has three comments by longtime groff project members, requiring no
> moderation; none of these appear in the email archive at
> http://lists.gnu.org/r/bug-groff/2022-08/.)

This was due to a misconfiguration which took me way too long to
figure out.  Fixed now as of a few minutes ago.

Found the systemd config included NoNewPrivileges=yes which completely breaks
things.  It prevents all suid in child processes such as /usr/sbin/sendmail and
anything else too.  Completely breaking it.

I removed that restriction and all is working again.  Re-Upgraded all
to the latest security releases.  I had downgraded for testing if
that was the problem.  All seems okay now.

The file with this configuration was set up May 27 therefore presumably apache
had not been restarted since then.  That's about the time this new web
server was brought online.  So I presume the dust was still settling.

On the 12th I applied the recent Trisquel point release upgrades which
included apache and the restart of apache on the 12th got that config
setup from May 27th for the first time.  This added to the confusion
since from my perspective only the security patch release had happened
recently and the change from May 27th was over the horizon old.  But
it was just a long "hang-fire".

See also https://savannah.nongnu.org/support/?110692 where other
people were impacted too.

> Similarly, a comment I added to
> http://savannah.nongnu.org/support/?110692 never showed up in my inbox
> (past comments I've submitted here have always shown up immediately)
> and is not reflected in
> http://lists.gnu.org/r/savannah-hackers/2022-08/.

The application of NoNewPrivileges=yes "broke things good".  This
prevented apache from sending email since sending email requires suid
functionality to work.  Lots of the system requires set-uid to work.
The system needs setuid in order to remain secure.  Disabling it is
very much like the classic problem with premature optimization.  It's
probably not the root of *all* evil but definitely the root of this evil.

> > I know I have been busy in real life and haven't been
> > able to attend the keyboard much the past few days.  But others in the
> > team should be able to handle things.
>
> I'm happy to redirect this query, but I don't know who else to contact
> about this, and there seems little point in filing a Savannah ticket
> over it, since no one is getting notified about them while email
> notifications are down.

It is tragic that the other couple of tickets that were filed with the
ticket system could not send an email notification to the Savannah
team.  I only saw them after fixing the problem and then going looking
for new tickets.

For my part I feel that it took me much too long to debug the problem.
I was completely unaware that systemd could set NoNewPrivileges=yes
and was not thinking of looking there for problems.  It just isn't
something I would have ever guessed happening!  I can only say that
eventually I got there in the end.

In the debugging of this I found it interesting that apachectl calls
systemctl and systemctl calls apachectl.  They break the loop by using
an internal variable as a flag to stop calling the other.  A comment
says they do this otherwise systemd gets confused.  This only happens
during start and both stop and restart are normal.  Isn't it a
wonderful world these days?  Not!  Hopefully that was read with a
heavy level of sarcasm.

Bob

Re: Updated install-font.sh

[Deri]

On Wednesday, 17 August 2022 19:28:28 BST Peter Schaffter wrote:
> Robert Goulding discovered that contents of the
> system font/devps/download file must be present in
> site-font/devps/download.  Similarly, freeeuro.pfa and EURO need
> to be copied into site/font/devpdf/ and registered in its download
> file.

Hi Peter,

I don't think the site_font download file for devpdf requires the Euro stuff 
repeated, so long as it is in the system download file already. Gropdf already 
searches in all expected font directories for download files and builds a map 
in memory.

There is probably no problem with duplicating the entry. Currently duplicates 
overwrite and it searches in the order -F dir, $GROFF_FONTPATH (from the 
environment), then the "standard" places, site_font, system font, /usr/lib/
font.

Joerg said he would report if it didn't work in this way, i.e. unnecessary to 
duplicate download entries. So far I have not heard anything.

Cheers 

Deri

Savannah bug->email outage

[G. Branden Robinson]

[Savannah hackers: see possible query bug at the end of this mail.]

This message is to compensate for some auto-generated traffic to the
bug-groff list that would have happened over the past several days but
for a problem with the Savannah ticket tracker to email service.

You can read more about the service outage on the
savannah-hackers-public mailing list.

https://lists.gnu.org/archive/html/savannah-hackers-public/2022-08/index.html

I performed an "advanced" query in the Savannah ticket browser for the
groff project[1], and it looks to me like 9 tickets were updated during
the service outage.

Here they are.

bug #24527: equations in .tl confuses grohtml
https://savannah.gnu.org/bugs/index.php?24527

bug #58736: [me] footnote breaks two-column output
https://savannah.gnu.org/bugs/index.php?58736

bug #59738: mdoc: .Lk suppresses end-of-sentence recognition
https://savannah.gnu.org/bugs/index.php?59738

bug #60504: [PATCH] document .PF macro in pic(1) and add it to -mpic
https://savannah.gnu.org/bugs/index.php?60504

bug #60571: Footnote markers defeat end-of-sentence recognition
https://savannah.gnu.org/bugs/index.php?60571

bug #61028: [refer-ms] sets comma after title in italics, should be roman
https://savannah.gnu.org/bugs/index.php?61028

bug #61935: [troff] \k not ignored after \c
https://savannah.gnu.org/bugs/index.php?61935

bug #62890: [pic] want option for SVG output
https://savannah.gnu.org/bugs/index.php?62890

bug #62901: [pic] accept `.PY` as synonym for `.PF`
https://savannah.gnu.org/bugs/index.php?62901

Nevertheless, the query missed at least one ticket.  Maybe because it
was merely "created", not "modified"?

bug #62900: [mm] want pic flyback support
https://savannah.gnu.org/bugs/index.php?62900

Regards,
Branden

[1] 
https://savannah.gnu.org/bugs/index.php?go_report=Apply&group=groff&func=browse&set=custom&msort=0&report_id=101&advsrch=0&status_id=0&resolution_id=0&submitted_by=0&assigned_to=0&category_id=0&bug_group_id=0&severity=0&summary=&details=&sumORdet=0&history_search=1&history_field=0&history_event=modified&history_date_dayfd=11&history_date_monthfd=8&history_date_yearfd=2022&chunksz=200&spamscore=5&boxoptionwanted=1#options


signature.asc
Description: PGP signature

Re: Updated install-font.sh

[Peter Schaffter]

Deri --

> I don't think the site_font download file for devpdf requires the
> Euro stuff repeated, so long as it is in the system download file
> already.  Gropdf already searches in all expected font directories
> for download files and builds a map in memory.

Odd.  At my end, gropdf doesn't recognise \[eu] without the Euro
stuff.  No problem with €, though.

> There is probably no problem with duplicating the entry.  Currently
> duplicates overwrite and it searches in the order -F dir,
> $GROFF_FONTPATH (from the environment), then the "standard"
> places, site_font, system font, /usr/lib/ font.

I haven't experienced any problem with the duplication.

-- 
Peter Schaffter
https://www.schaffter.ca

Re: Savannah bug->email gateway problems affecting groff

[Werner LEMBERG]

>> In the past couple days I've made several updates to groff Savannah
>> tickets, which have always generated an immediate email to me and,
>> not too long after, one in the lists.gnu.org archive of the
>> bug-groff email list.  Neither of these has been getting the emails
>> since August 12.  (See, for instance,
>> http://savannah.gnu.org/bugs/?62901, which has three comments by
>> longtime groff project members, requiring no moderation; none of
>> these appear in the email archive at
>> http://lists.gnu.org/r/bug-groff/2022-08/.)
> 
> This was due to a misconfiguration which took me way too long to
> figure out.  Fixed now as of a few minutes ago.

Thanks from me, too!


Werner