How to avoid fdopen redefinition?

[Gavin Smith]

When using a stdin.h from gnulib with Perl's extension headers (e.g.
XSUB.h), under MinGW there can be compiler warnings due to
redefinition of an fdopen symbol. This was reported here (point 1):

https://lists.gnu.org/archive/html/bug-texinfo/2021-02/msg00152.html

We don't use fdopen in the program so it would be better if we didn't
have to interact with this part of Gnulib. Is there any way of turning
this off?

The relevant part of stdin.in.h is the following:

#elif @GNULIB_MDA_FDOPEN@
/* On native Windows, map 'fdopen' to '_fdopen', so that -loldnames is not
   required.  In C++ with GNULIB_NAMESPACE, avoid differences between
   platforms by defining GNULIB_NAMESPACE::fdopen always.  */
# if defined _WIN32 && !defined __CYGWIN__
#  if !(defined __cplusplus && defined GNULIB_NAMESPACE)
#   undef fdopen
#   define fdopen _fdopen
#  endif
_GL_CXXALIAS_MDA (fdopen, FILE *, (int fd, const char *mode));
# else
_GL_CXXALIAS_SYS (fdopen, FILE *, (int fd, const char *mode));
# endif
_GL_CXXALIASWARN (fdopen);
#endif

Is there any way of turning this off? We didn't use the stdio module
explicitly but it was a dependency of other Gnulib modules.

I saw this message from when some of the current code was introduced:

https://lists.gnu.org/archive/html/bug-gnulib/2020-12/msg00220.html

It says in that message,

Conditionalize each of the Windows oldnames workarounds through a
GNULIB_MDA_ symbol.

What would be the recommended way of doing this? Is it
AC_SUBST([GNULIB_MDA_FDOPEN]) or something else?

I don't know what MDA means and don't want to be hooking into Gnulib
internals with a process of sheer trial and error.

TIA

Re: How to avoid fdopen redefinition?

[Gavin Smith]

On Mon, Mar 1, 2021 at 4:46 PM Bruno Haible  wrote:
> > Is there any way of turning this off?
>
> Yes: In your configure.ac, after the invocation of gl_INIT, add an assignment
>
>   GNULIB_MDA_FDOPEN=0
>
> > https://lists.gnu.org/archive/html/bug-gnulib/2020-12/msg00220.html
> >
> > It says in that message,
> >
> > Conditionalize each of the Windows oldnames workarounds through a
> > GNULIB_MDA_ symbol.
>
> Yes, that's the commit where GNULIB_MDA_FDOPEN was introduced.
>
> > I don't know what MDA means
>
> MDA means "Microsoft deprecated alias". 'fdopen' is POSIX, and after using
> POSIX as the base for some of their APIs for years, they decided to deprecate
> these functions that are in POSIX but not in ISO C++. (They don't care
> about ISO C.)

Thanks for the advice. I've made this change to configure.ac so
hopefully this will fix the problem.

__nonnull on DragonFlyBSD 5.9

[Gavin Smith]

I'm hoping for some help tracking down an error trying to compile Texinfo
on DragonFlyBSD 5.9.  The error message is as follows:

depbase=`echo basename-lgpl.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`; gcc 
-DHAVE_CONFIG_H -I. -I../..  -g -O2 -MT basename-lgpl.o -MD -MP -MF 
$depbase.Tpo -c -o basename-lgpl.o basename-lgpl.c && mv -f $depbase.Tpo 
$depbase.Po
depbase=`echo malloc/dynarray_at_failure.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`; 
gcc -DHAVE_CONFIG_H -I. -I../..  -g -O2 -MT malloc/dynarray_at_failure.o 
-MD -MP -MF $depbase.Tpo -c -o malloc/dynarray_at_failure.o 
malloc/dynarray_at_failure.c && mv -f $depbase.Tpo $depbase.Po
In file included from ./libc-config.h:33,
 from ./dynarray.h:22,
 from malloc/dynarray_at_failure.c:19:
/usr/include/stdlib.h:182:46: error: expected ')' before numeric constant
 int  posix_memalign(void **, size_t, size_t) __nonnull(1); /* (ADV) */
  ^
../../config.h:1233:71: error: expected ',' or ';' before ')' token
 # define _GL_ATTRIBUTE_NONNULL(args) __attribute__ ((__nonnull__ args))
   ^
   ^

The malloc/dynarray_at_failure.c file is fairly new in Gnulib
(added 2021-01-08) but I don't think it is a problem with that file
specifically.  The code around the line in config.h is the following:

#if _GL_HAS_ATTRIBUTE (nonnull)
# define _GL_ATTRIBUTE_NONNULL(args) __attribute__ ((__nonnull__ args))
#else
# define _GL_ATTRIBUTE_NONNULL(args)
#endif

and there is also a definition in cdefs.h:

/* The nonnull function attribute marks pointer parameters that
   must not be NULL.  */
#ifndef __nonnull
# if __GNUC_PREREQ (3,3) || __glibc_has_attribute (__nonnull__)
#  define __nonnull(params) __attribute__ ((__nonnull__ params))
# else
#  define __nonnull(params)
# endif
#elif !defined __GLIBC__
# undef __nonnull
# define __nonnull(params) _GL_ATTRIBUTE_NONNULL (params)
#endif


I've spent quite a while trying to isolate the problem but couldn't
get it down to a very small input file.  When I commented out the last
two but one lines in the code above from cdefs.h, the compilation continued.
I guess on this system, __GLIBC__ is not defined, but this is not a
correct definition of __nonnull.

There was the following recent change that could be responsible:

2021-01-23  Paul Eggert  

libc-config: port to Xcode 7
Problem reported by Mattias Engdegård in:
https://lists.gnu.org/r/emacs-devel/2021-01/msg01089.html
* lib/cdefs.h (__nonnull): If already defined but glibc is not in
use, override the definition with Gnulib’s _GL_ATTRIBUTE_NONNULL.
This is needed for Xcode 7, which has a ‘#define __nonnull
_Nonnull’ builtin for backwards-compatibility with an older Xcode
syntax that GNUish code never uses.

I don't know how to find what the definition of __nonnull is on DragonFlyBSD
but if I put the following in test.c:

#include 

__nonnull(1)

and then run "gcc -E test.c", the output ends with

# 2 "tmp.c" 2

__attribute__((__nonnull__(
# 3 "tmp.c"
1
# 3 "tmp.c" 3 4
)))

Maybe the difference is the brackets around the argument to __nonnull__,
which seem to be missing in the definition of _GL_ATTRIBUTE_NONNULL.  Is
there any other information I could provide to help solve the problem?

$ gcc --version
gcc 8.3 [DragonFly] Release/2019-02-22
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 

Fwd: bug#50116: Text on GNU grep webpage far too big

[Gavin Smith]

Does anybody have access to
https://www.gnu.org/software/gnulib/manual.css and could the CSS lines
mentioned below removed? These CSS lines would be a problem with any
Texinfo manual built with Texinfo 6.8 due to changes in the structure
of the HTML (enclosing an entire section in 
rather than just the section heading).

-- Forwarded message -
From: Steve Ward 
Date: Thu, Aug 19, 2021 at 12:56 PM
Subject: Re: bug#50116: Text on GNU grep webpage far too big
To: Gavin Smith 
Cc: <50...@debbugs.gnu.org>


On Thu, Aug 19, 2021 at 2:12 AM Gavin Smith  wrote:
>
> https://www.gnu.org/software/grep/manual/grep.html
>
> This documentation is not usable because the text is far too big.  I assume
> it is being enlarged with a CSS file.
>
>
>

In the manual.css file [0], commenting out the font-size property in
these blocks makes the page look normal to me.

.settitle, .top, .chapter
/*font-size: 250%;*/

.section, .subsection, .subsubsection
/*font-size: 150%; */


[0] <https://www.gnu.org/software/gnulib/manual.css>

Re: conditional "LIBUNISTRING_COMPILE_UNIWIDTH_WIDTH" was never defined.

[Gavin Smith]

On Fri, Jan 14, 2022 at 11:24:57PM +0100, Bruno Haible wrote:
> The following patch fixes it. The cause is that texinfo is using the
> gnulib-tool option --conditional-dependencies, and a couple of days
> ago I made this option more effective.
> 
> 
> 2022-01-14  Bruno Haible  
> 
>   Avoid error "conditional LIBUNISTRING_COMPILE_... was never defined"
>   when option --conditional-dependencies is used (regression 2022-01-09).
>   Reported by Gavin Smith  in
>   <https://lists.gnu.org/archive/html/bug-gnulib/2022-01/msg00099.html>.
>   * m4/libunistring-base.m4 (gl_LIBUNISTRING_MODULE): Use gl_CONDITIONAL
>   instead of AM_CONDITIONAL.

Thanks for the quick investigation and fix.  It appears to work now.

Avoid gnulib redefinitions - MDA, free-posix

[Gavin Smith]

Previously in the Texinfo project, we added variables to configure.ac to
stop the redefinition of "Microsoft deprecated aliases":

https://lists.gnu.org/archive/html/bug-gnulib/2021-03/msg4.html

For example, GNULIB_MDA_FDOPEN to stop the redefinition of 'fdopen'.

Recently, it was reported that this didn't work when building on
MS-Windows.  I found that it should maybe be GL_GNULIB_MDA_FDOPEN instead:

https://lists.gnu.org/archive/html/bug-texinfo/2022-10/msg00180.html

I had identified the following change as potentially being responsible:

2021-04-11  Bruno Haible  

Support several gnulib-tool invocations under the same configure.ac.
Reported by Reuben Thomas  in
.  
This is done by defining the Gnulib module indicator variables per
gnulib-tool invocation. So that a generated .h file is no longer
influenced by the set of modules used in other gnulib-tool invocations. 
* gnulib-tool (func_compute_include_guard_prefix): Set
module_indicator_prefix.

Should we use the variables with the GL_ prefix now and is this something
we can rely on?  Or should we simply #undef fdopen and the other symbols?

We don't use fdopen, putenv or mktemp in the library being built, but these
are pulled in by Gnulib dependencies.

I don't know if it is possible at all but it would seem to be better
for Gnulib not to redefine symbols that are not actually used in the
program.  I'd like to keep the use of Gnulib code to a minimum, only
using it where there is a significant portability benefit.

Likewise, there is a warning about the redefinition of "free", that
comes from the free-posix module.  The issue is, when building a
Perl extension module, some source files include Perl headers that
also redefine "free".

free-posix is to work around the possibility that free overwrites
the value of errno.  It's unlikely that this would cause a problem
in our code, but free-posix is required internally by Gnulib code.

It seems undesirable to override such a simple function as 'free',
when this is not something the user has asked for - as in our case,
it caused a conflict with other uses (Perl headers).

This may be hard for you to change, though.  Here is one idea.  When
using a module like 'free-posix', if it is loaded as a dependency only,
use the redefinition in Gnulib code only, but do not override symbols
in user code.  It would be as if there were two modules, say
gl-free-posix and free-posix, where gl-free-posix made the redefinition
of rpl_gl_free and free-posix (requested by the user) redirected rpl_free
to rpl_gl_free.

Re: Avoid gnulib redefinitions - MDA, free-posix

[Gavin Smith]

On Sat, Oct 29, 2022 at 09:58:25AM -0700, Paul Eggert wrote:
> If it's just 'free', I might prefer the latter solution, to underline the
> special case and to avoid even more complexity in gnulib-tool. Is that
> something you could write?

Here is a first attempt.  I am not very familiar with the Gnulib code.

The main idea is to put a define in AM_CPPFLAGS in the Makefile that
gnulib-tool generates.  Then the gnulib headers can distinguish whether
they are being used as part of building gnulib, or building the user's
program.

This avoided looking through all the gnulib code and deciding where
to replace "free" with "gl_internal_free" or whatever the replacement
would be called.

I tried to minimize the number of lines of code that would change, but
if this approach was adopted, then the following changes should also be
made:
* Rename free-posix to free-posix-internal and free-posix-public to free-posix
* REPLACE_FREE to be changed to REPLACE_FREE_INTERNAL throughout gnulib,
  likewise REPACE_FREE_PUBLIC to REPLACE_FREE.

I tested this on a project with a gnulib import.

Let me know if this is the right approach and if there are changes to
make to the patch.

diff --git a/ChangeLog b/ChangeLog
index 6f4bea5c1c..994f457e70 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,21 @@
+2020-10-29  Gavin Smith  
+
+   Avoid unrequested 'free' redefinition
+
+   * gnulib-tool (func_emit_lib_Makefile_am):
+   Add -DGNULIB_INTERNAL to AM_CPPFLAGS.
+   * m4/free-public.m4, modules/free-posix-public:
+   New module to set @REPLACE_FREE_PUBLIC@ from @REPLACE_FREE@.
+   * modules/stdlib, m4/stdlib_h.m4:
+   Substitute for @REPLACE_FREE_PUBLIC@.
+   * lib/stdlib.in.h <@GNULIB_FREE_POSIX@>: Only replace 'free' if
+   both GNULIB_INTERNAL and @REPLACE_FREE@ are set, or if
+   @REPLACE_FREE_PUBLIC@ is set.
+
+   This allows Gnulib to use the free redefinition internally without
+   replacing it in user code, unless they explicitly request the
+   module.
+
 2022-10-23  Bruno Haible  
 
assert-h: Make static_assert work on Solaris 11.4.
diff --git a/gnulib-tool b/gnulib-tool
index 028bcf36ad..84a77171d7 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -3945,7 +3945,7 @@ func_emit_lib_Makefile_am ()
   fi
   if test -z "$makefile_name"; then
 echo
-echo "AM_CPPFLAGS =$cppflags_part1$cppflags_part2"
+echo "AM_CPPFLAGS = -DGNULIB_INTERNAL$cppflags_part1$cppflags_part2"
 echo "AM_CFLAGS ="
   else
 if test -n "$cppflags_part1$cppflags_part2"; then
diff --git a/lib/stdlib.in.h b/lib/stdlib.in.h
index 8e0a609f1f..f75b8621f4 100644
--- a/lib/stdlib.in.h
+++ b/lib/stdlib.in.h
@@ -179,7 +179,7 @@ _GL_WARN_ON_USE (_Exit, "_Exit is unportable - "
 
 
 #if @GNULIB_FREE_POSIX@
-# if @REPLACE_FREE@
+# if @REPLACE_FREE@ && defined(GNULIB_INTERNAL) || @REPLACE_FREE_PUBLIC@
 #  if !(defined __cplusplus && defined GNULIB_NAMESPACE)
 #   undef free
 #   define free rpl_free
diff --git a/m4/free-public.m4 b/m4/free-public.m4
new file mode 100644
index 00..497abf81ae
--- /dev/null
+++ b/m4/free-public.m4
@@ -0,0 +1 @@
+AC_DEFUN([gl_PREREQ_FREE_PUBLIC], [:])
diff --git a/m4/stdlib_h.m4 b/m4/stdlib_h.m4
index 9e2096976f..b096054284 100644
--- a/m4/stdlib_h.m4
+++ b/m4/stdlib_h.m4
@@ -171,6 +171,7 @@ AC_DEFUN([gl_STDLIB_H_DEFAULTS],
   REPLACE_CALLOC_FOR_CALLOC_POSIX=0;  
AC_SUBST([REPLACE_CALLOC_FOR_CALLOC_POSIX])
   REPLACE_CANONICALIZE_FILE_NAME=0;  AC_SUBST([REPLACE_CANONICALIZE_FILE_NAME])
   REPLACE_FREE=0;AC_SUBST([REPLACE_FREE])
+  REPLACE_FREE_PUBLIC=0; AC_SUBST([REPLACE_FREE_PUBLIC])
   REPLACE_INITSTATE=0;   AC_SUBST([REPLACE_INITSTATE])
   REPLACE_MALLOC_FOR_MALLOC_GNU=0;AC_SUBST([REPLACE_MALLOC_FOR_MALLOC_GNU])
   REPLACE_MALLOC_FOR_MALLOC_POSIX=0;  
AC_SUBST([REPLACE_MALLOC_FOR_MALLOC_POSIX])
diff --git a/modules/free-posix-public b/modules/free-posix-public
new file mode 100644
index 00..050a06bf76
--- /dev/null
+++ b/modules/free-posix-public
@@ -0,0 +1,27 @@
+Description:
+Work around systems where free clobbers errno.
+
+Files:
+m4/free-public.m4
+
+Depends-on:
+free-posix
+
+configure.ac:
+REPLACE_FREE_PUBLIC=$REPLACE_FREE
+gl_CONDITIONAL([GL_COND_OBJ_FREE_PUBLIC], [test $REPLACE_FREE_PUBLIC = 1])
+AM_COND_IF([GL_COND_OBJ_FREE_PUBLIC], [
+  gl_PREREQ_FREE_PUBLIC
+])
+gl_STDLIB_MODULE_INDICATOR([free-posix-public])
+
+Makefile.am:
+
+Include:
+
+
+License:
+LGPLv2+
+
+Maintainer:
+None
diff --git a/modules/stdlib b/modules/stdlib
index 45d8f59331..2266e68d52 100644
--- a/modules/stdlib
+++ b/modules/stdlib
@@ -131,6 +131,7 @@ stdlib.h: stdlib.in.h $(top_builddir)/config.status 
$(CXXDEFS_H) \
  -e 
's|@''REPLACE_CALLOC_FOR_CALLOC_POSIX''@|$(REPLACE_CALLOC_FOR_CALLOC_POSIX)|g' \
  -e 
's|@''REPLACE

Re: Avoid gnulib redefinitions - MDA

[Gavin Smith]

On Sat, Oct 29, 2022 at 11:48:41PM +0200, Bruno Haible wrote:
> > Should we use the variables with the GL_ prefix now and is this something
> > we can rely on?  Or should we simply #undef fdopen and the other symbols?
> 
> The way to avoid a particular MDA symbol definition (GNULIB_MDA_FDOPEN=0
> before April 2021, GL_GNULIB_MDA_FDOPEN=0 after April 2021) is an undocumented
> functionality. It is not expected that it will break soon. The 2021-04-11
> change that you cited above was a once-in-a-decade change. But it may break
> theoretically, since it is not in the form of a stable functionality.
> (A stable, supported functionality would be something like a gnulib-tool
> option and/or a module name.)

Right, so it sounds like we are better off using #undef before
including the Perl headers to avoid depending on undocumented
functionalities.  Thanks.

Re: Avoid gnulib redefinitions - MDA

[Gavin Smith]

On Sun, Oct 30, 2022 at 12:09:50AM +0200, Bruno Haible wrote:
> Gavin Smith wrote:
> > so it sounds like we are better off using #undef before
> > including the Perl headers to avoid depending on undocumented
> > functionalities.  Thanks.
> 
> Using #undef means to decline all corrections done by Gnulib.
> These are documented in the manual.

In our case, using #undef only avoids a compilation warning about
symbols being redefined.  Perl itself redefines fdopen and other
functions, on MS-Windows at least.  (I haven't checked why.)  We
don't use fdopen, but if we did, it's very likely we would need to
use Perl's redefinition for the module to build properly.  I don't
think we'd be able to use Gnulib redefinitions in any case.

The one redefined function from Perl that we do use is "free".
Apparently it's necessary to use Perl's version here (on MS-Windows),
otherwise there can be weird errors.

Re: -Wlto-type-mismatch warning in error()

[Gavin Smith]

On Wed, Dec 07, 2022 at 08:57:45PM +, Sam James wrote:

> ../gnulib/lib/error.h:33:13: error: type of ‘error’ does not match original 
> declaration [-Werror=lto-type-mismatch]
>33 | extern void error (int __status, int __errnum, const char *__format, 
> ...)
>   | ^
> install-info.c:218:1: note: type mismatch in parameter 1
>   218 | error (const char *fmt, ...)
>   | ^
> install-info.c:218:1: note: ‘error’ was previously declared here
> install-info.c:218:1: note: code may be misoptimized unless 
> ‘-fno-strict-aliasing’ is used
> lto1: some warnings being treated as errors
> lto-wrapper: fatal error: //usr/bin/x86_64-pc-linux-gnu-gcc returned 1 exit 
> status
> compilation terminated.
> /usr/lib/gcc/x86_64-pc-linux-gnu/12/../../../../x86_64-pc-linux-gnu/bin/ld: 
> error: lto-wrapper failed
> collect2: error: ld returned 1 exit status
> make[3]: *** [Makefile:1514: ginstall-info] Error 1
> ```
> 
> This is with GCC 12.2.1 20221203.
> 
> We've also seen this in GNU make, so not sure if it's a
> gnulib problem or not, as it may be in lib/error.h (hence CCing bug-gnulib):
> - https://bugs.gentoo.org/863713 (texinfo)
> - https://bugs.gentoo.org/863824 (make)

I expect it is not a gnulib problem.  install-info/install-info.c declares
a function called "error" which appears to clash with a function from glibc.

The function called "error" in install-info is correctly used when
e.g. "install-info one two three" is run, printing

install-info: excess command line argument `three'

It may depend on the configuration process and what parts of gnulib are
being used.  The "error" module is brought in by "xalloc" (which we
ask for explicitly).

>From the Gentoo bug report
> -Werror=lto-type-mismatch:
> User to find possible runtime issues in packages. It likely means the package 
> is unsafe to build & use with LTO.
> For projects using the same identifier but with different types across 
> different files, they must be fixed to be consistent across the codebase.

The simplest way to fix this problem would probably be to rename the "error"
function in install-info.c.  Perhaps this issue has never come up before
because people have not used the LTO options for building.

Re: -Wlto-type-mismatch warning in error()

[Gavin Smith]

On Thu, Dec 08, 2022 at 09:25:01AM +0100, Arsen Arsenović wrote:
> > Shouldn't we report this to the GCC folks?  It could be a bug in lto,
> > no?  I mean, 'error' is not a symbol that applications cannot use, and
> > if an application defines a function by that name, it should not be
> > imported from the standard library.  Right?
> 
> I believe this would make them part of the same program.  On top of
> that, Gnulib is pulling in error anyway:
> 
> $ nm ./gnulib/lib/libgnu.a | grep error
>  U error
> $ nm install-info.o ../gnulib/lib/libgnu.a |& grep '\'
>  T error
>  U error
> 
> My guess is that libgnu_a-xalloc-die.o (the file emitting the U error
> symbol) includes gnulib/lib/error.h, GCC records that declaration
> (through it's use in xalloc_die), and then detects a mismatch with the
> one emitted by install-info.o (the T error symbol) and hence warns.
> 
> I imagine this would result is some very strange runtime failures if
> anyone ever observed install-info hit an xalloc_die condition.

Your analysis makes sense here.  I have committed a change to make it
static.  I have also cherry-picked this commit to the release/7.0
branch in case another release is made from this branch.

Unless there is some way in Gnulib to prefer the glibc symbols when
linking, this seems unavoidable.  Defining the Gnulib symbols as "weak"
wouldn't help; as Arsen has said, it is the definition in install-info.c
itself that shouldn't be used when resolving the reference to "error"
in libgnu.a.

Making the symbols provided by install-info.c weak might work,
so one idea is that when a program uses Gnulib, all of the global
symbols from the program (excluding Gnulib) should be marked as weak
in produced object files, so that Gnulib code preferentially uses
code from glibc or other libraries.  I have no idea what would be
needed to achieve this or what other implications there might be.
(This won't help if the symbol is weak in those libraries too, though.)

I consulted some documentation on the ELF format but there appears
only to be one type of "undefined" symbol - it wouldn't be possible
to make the undefined symbols in libgnu.a preferentially resolve to
glibc symbols rather than other files in the program.   I'm very ignorant
of these matters though so it possible I missed something.

Likewise something might also be possible with how the linker (ld) is run
but somebody would have to research this.

> Since I went through the process of making all the symbols in that file
> (besides main) local, here's the patch that does that

Thanks but no thanks.  install-info.c is a single-file program so there's
no point in adding the static keyword everywhere.

Re: -Wlto-type-mismatch warning in error()

[Gavin Smith]

On Sat, Dec 10, 2022 at 01:48:23AM +, Gavin Smith wrote:
> Making the symbols provided by install-info.c weak might work,
> so one idea is that when a program uses Gnulib, all of the global
> symbols from the program (excluding Gnulib) should be marked as weak
> in produced object files, so that Gnulib code preferentially uses
> code from glibc or other libraries.  I have no idea what would be
> needed to achieve this or what other implications there might be.
> (This won't help if the symbol is weak in those libraries too, though.)

On second thought this would be a bad idea in case the program uses
symbols that clash with library symbols that are marked weak.

getpt support?

[Gavin Smith]

I was wondering what the chances would be of getting support for the
getpt function in gnulib? It's a GNU stdlib.h extension to get a file
descriptor for a master pseudo-terminal. As far as I can tell it does
something very similar to posix_openpt, and there is already a gnulib
posix_openpt module.

Thanks,
Gavin

Re: getpt support?

[Gavin Smith]

On Mon, Jun 2, 2014 at 4:47 PM, Paul Eggert  wrote:
> Why not just use posix_openpt?

If the two functions are exactly the same (I think they are), this
would be acceptable. There are two weak reasons to use getpt instead:
(i) posix_openpt isn't documented in the glibc manual (but I've raised
a bug for this), and (ii) posix_openpt has an ugly name. There is
probably not a strong need for it but I would use it if it were there.

Call to locale_charset within wcwidth

[Gavin Smith]

I am working on a program that uses wcwidth from gnulib on many
characters in a buffer, however this is reported to be slow under some
platforms.[1]

The reason appears to be the call to locale_charset within the wcwidth
implementation. I found some discussion of speeding this up in the
bug-gnulib archives [2][3], has there been any progress on this?

The charset could be cached either in wcwidth itself, or there could
be some caching in locale_charset.

It appears you can have thread-local variables with gcc using
"__thread", for example, "static __thread const char *encoding". (I
don't know about other compilers.) Would something like this work? As
well as threads, updating the cached encoding when the encoding
changes would be a problem.

[1] http://lists.gnu.org/archive/html/bug-texinfo/2015-01/msg4.html
[2] http://lists.gnu.org/archive/html/bug-gnulib/2011-12/msg00083.html
[3] http://lists.gnu.org/archive/html/bug-gnulib/2013-12/msg00087.html

Re: Call to locale_charset within wcwidth

[Gavin Smith]

On Sun, Feb 1, 2015 at 10:07 AM, Daiki Ueno  wrote:
> Eli Zaretskii  writes:
>
>> Perhaps we could call _configthreadlocale to see if the value of
>> setlocale can be cached?
>
> Yes, but then we would need to hook setlocale to invalidate the cache,
> and currently the setlocale module is not listed in the dependencies of
> localcharset.  I'm wondering if it is worth adding the dependency for
> this particular performance problem.  Perhaps it might be easier to add
> a new API to localcharset for single-threaded / single-locale consumer.
>

Here's a simple solution. The caller of the new function would call
locale_encoding themselves and save the result. Maybe somebody will
have a better idea. If this is too specialized to go into gnulib,
would it be easy for a project to use its own replacement functions
instead of the ones from gnulib?

*** wcwidth.c   2014-04-22 02:56:36.0 +0100
--- wcwidth.c-NEW   2015-02-01 14:53:41.0 +
***
*** 27,37 
  #include "uniwidth.h"

  int
! wcwidth (wchar_t wc)
! #undef wcwidth
  {
/* In UTF-8 locales, use a Unicode aware width function.  */
-   const char *encoding = locale_charset ();
if (STREQ_OPT (encoding, "UTF-8", 'U', 'T', 'F', '-', '8', 0, 0, 0 ,0))
  {
/* We assume that in a UTF-8 locale, a wide character is the same as a
--- 27,35 
  #include "uniwidth.h"

  int
! wcwidth_given_encoding (wchar_t wc, const char *encoding)
  {
/* In UTF-8 locales, use a Unicode aware width function.  */
if (STREQ_OPT (encoding, "UTF-8", 'U', 'T', 'F', '-', '8', 0, 0, 0 ,0))
  {
/* We assume that in a UTF-8 locale, a wide character is the same as a
***
*** 48,50 
--- 46,56 
  #endif
  }
  }
+
+ int
+ wcwidth (wchar_t wc)
+ #undef wcwidth
+ {
+   const char *encoding = locale_charset ();
+   return wcwidth_given_encoding (wc, encoding);
+ }

Re: Compilation warnings on MinGW due to Gnulib's wcwidth

[Gavin Smith]

On 18 June 2015 at 16:26, Eli Zaretskii  wrote:
> Building the current Texinfo trunk tip with MinGW produces a barrage
> of compiler warnings like these:
>
>   gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../..-Id:/usr/include  -gdwarf-4 
> -g3 -O2 -MT mbscasecmp.o -MD -MP -MF $depbase.Tpo -c -o mbscasecmp.o 
> mbscasecmp.c &&\
>   mv -f $depbase.Tpo $depbase.Po
>   In file included from mbuiter.h:106:0,
>from mbscasecmp.c:27:
>   mbchar.h: In function 'mb_width_aux':
>   mbchar.h:249:3: warning: implicit declaration of function 'wcwidth' 
> [-Wimplicit-function-declaration]
>  int w = wcwidth (wc);
>  ^

I don't suppose you have any idea when this problem could have
started? For example, did you manage to compile the texinfo 5.9.93
pretest okay?

You think that the #include_next line is being executed, right, and
you think this is the case because _GL_ALREADY_INCLUDING_WCHAR_H is
defined? It looks like this is only defined if wchar.h in gnulib
indirectly includes itself.

Looking at http://svn.savannah.gnu.org/viewvc/trunk/gnulib/m4/?root=texinfo
and http://svn.savannah.gnu.org/viewvc/trunk/gnulib/lib/?root=texinfo,
nothing jumps out there as a recent change that could explain the
declaration of wcwidth being missing.

It could be helpful if you posted the config.log file, and
gnulib/lib/wchar.h as well.

Re: Compilation warnings on MinGW due to Gnulib's wcwidth

[Gavin Smith]

On 19 June 2015 at 07:48, Eli Zaretskii  wrote:
> There was no wcwidth.h in texinfo-5.9.93's gnulib; now there is.  So
> the arrangement of how that declaration is pulled in changed
> significantly since then.

I downloaded the texinfo-5.9.94 pretest
(http://alpha.gnu.org/gnu/texinfo/), and it doesn't have wcwidth.h
either: the file is in the repository, but isn't being distributed.

Re: Compilation warnings on MinGW due to Gnulib's wcwidth

[Gavin Smith]

On 19 June 2015 at 13:12, Gavin Smith  wrote:
> On 19 June 2015 at 07:48, Eli Zaretskii  wrote:
>> There was no wcwidth.h in texinfo-5.9.93's gnulib; now there is.  So
>> the arrangement of how that declaration is pulled in changed
>> significantly since then.
>
> I downloaded the texinfo-5.9.94 pretest
> (http://alpha.gnu.org/gnu/texinfo/), and it doesn't have wcwidth.h
> either: the file is in the repository, but isn't being distributed.

According to gnulib's ChangeLog, wcwidth.h was removed in July 2007.
It looks as if we have out-of-date files in the repository.

2007-07-07  Bruno Haible  

* lib/wchar_.h: Include the GL_LINK_WARNING macro.
(wcwidth): New declaration.
* m4/wchar.m4 (gl_WCHAR_MODULE_INDICATOR, gl_WCHAR_H_DEFAULTS): New
macros.
(gl_WCHAR_H): Require gl_WCHAR_H_DEFAULTS. Don't set WCHAR_H to empty
here. Prepare for creating  unconditionally.
* modules/wchar (Depends-on): Add link-warning.
(Makefile.am): Substitute also GNULIB_WCWIDTH, HAVE_DECL_WCWIDTH,
REPLACE_WCWIDTH, and GL_LINK_WARNING.
* lib/wcwidth.h: Remove file.
* m4/wcwidth.m4 (gl_FUNC_WCWIDTH): Require gl_WCHAR_H_DEFAULTS. Set
HAVE_DECL_WCWIDTH, REPLACE_WCWIDTH, WCHAR_H.
* modules/wcwidth (Files): Remove lib/wcwidth.h.
(configure.ac): Invoke gl_WCHAR_MODULE_INDICATOR.
(Include): Replace wcwidth.h with .
* lib/wcwidth.c: Include  instead of wcwidth.h.
* lib/mbchar.h: Don't include wcwidth.h.
* lib/mbswidth.c: Likewise.
* NEWS: Mention the change.

2007-07-07  Bruno Haible  

* lib/wcwidth.c: New file, extracted from lib/wcwidth.h.
* lib/wcwidth.h: Don't include wctype.h. Replace inline function
definition with an external declaration.
* m4/wcwidth.m4 (gl_FUNC_WCWIDTH): Use wcwidth.c when wcwidth is not
defined as a function. Remove AC_C_INLINE requirement.
* modules/wcwidth (Files): Add lib/wcwidth.c.
(Makefile.am): Remove redundant statement.

Re: Compilation warnings on MinGW due to Gnulib's wcwidth

[Gavin Smith]

On 19 June 2015 at 13:36, Eli Zaretskii  wrote:
>> Date: Fri, 19 Jun 2015 13:12:00 +0100
>> From: Gavin Smith 
>> Cc: bug-gnulib@gnu.org, texinfo-de...@gnu.org
>>
>> On 19 June 2015 at 07:48, Eli Zaretskii  wrote:
>> > There was no wcwidth.h in texinfo-5.9.93's gnulib; now there is.  So
>> > the arrangement of how that declaration is pulled in changed
>> > significantly since then.
>>
>> I downloaded the texinfo-5.9.94 pretest
>> (http://alpha.gnu.org/gnu/texinfo/), and it doesn't have wcwidth.h
>> either: the file is in the repository, but isn't being distributed.
>
> Then I guess it isn't related to the problem at hand.  Sorry for
> confusing everybody.
>
> However, the original issue still needs to be resolved.

I wonder if you could try compiling again now the file is gone from
the repository.

Re: Compilation warnings on MinGW due to Gnulib's wcwidth

[Gavin Smith]

On 19 June 2015 at 13:36, Eli Zaretskii  wrote:
>> Date: Fri, 19 Jun 2015 13:12:00 +0100
>> From: Gavin Smith 
>> Cc: bug-gnulib@gnu.org, texinfo-de...@gnu.org
>>
>> On 19 June 2015 at 07:48, Eli Zaretskii  wrote:
>> > There was no wcwidth.h in texinfo-5.9.93's gnulib; now there is.  So
>> > the arrangement of how that declaration is pulled in changed
>> > significantly since then.
>>
>> I downloaded the texinfo-5.9.94 pretest
>> (http://alpha.gnu.org/gnu/texinfo/), and it doesn't have wcwidth.h
>> either: the file is in the repository, but isn't being distributed.
>
> Then I guess it isn't related to the problem at hand.  Sorry for
> confusing everybody.
>
> However, the original issue still needs to be resolved.

It's very hard for me to guess what might be going on on your system.
I suggest adding warning directives to wchar.h, like the following:

Index: gnulib/lib/wchar.in.h
===
--- gnulib/lib/wchar.in.h   (revision 6344)
+++ gnulib/lib/wchar.in.h   (working copy)
@@ -43,10 +43,12 @@
  the latter includes .  But here, we have no way to detect whether
   is completely included or is still being included.  */

+#warning ""
 #@INCLUDE_NEXT@ @NEXT_WCHAR_H@

 #else
 /* Normal invocation convention.  */
+#warning "BBB"

 #ifndef _@GUARD_PREFIX@_WCHAR_H

which would confirm which parts of the file were being included, and
the files that were being included at the time, e.g.:

gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -Wall -g -O0 -MT
printf-args.o -MD -MP -MF $depbase.Tpo -c -o printf-args.o
printf-args.c &&\
mv -f $depbase.Tpo $depbase.Po
In file included from printf-args.h:41:0,
 from printf-args.c:29:
./wchar.h:52:2: warning: #warning "BBB" [-Wcpp]
 #warning "BBB"
  ^

configure script does not give error if "ar" is not found

[Gavin Smith]

Hello,

I was trying to build Texinfo on a SunOS 5.10 sparc system, but got an
error message when it tried to build gnulib, that "ar" didn't exist.
Looking at config.log I have:

configure:5731: checking for egrep
configure:5793: result: /usr/xpg4/bin/grep -E
configure:5800: checking for Minix Amsterdam compiler
configure:5824: result: no
configure:5876: checking for ar
configure:5906: result: no
configure:5983: checking for ranlib
configure:6013: result: no
configure:6037: checking for ANSI C header files


I found the following in gnulib-common.m4 that is probably responsible

  if test -z "$AR"; then
if test $gl_cv_c_amsterdam_compiler = yes; then
  AR='cc -c.a'
  if test -z "$ARFLAGS"; then
ARFLAGS='-o'
  fi
else
  dnl Use the Automake-documented default values for AR and ARFLAGS,
  dnl but prefer ${host}-ar over ar (useful for cross-compiling).
  AC_CHECK_TOOL([AR], [ar], [ar])
  if test -z "$ARFLAGS"; then
ARFLAGS='cru'
  fi
fi
  else
if test -z "$ARFLAGS"; then
  ARFLAGS='cru'
fi
  fi

The problem is with the use of AC_CHECK_TOOL. The third argument given
is a default value, that doesn't exist. I suggest doing, instead
(untested):

AC_CHECK_TOOL([AR], [ar], []])
if test -z "$AR"; then
  AC_MSG_ERROR([ar program not found - cannot create static archives])
fi

(Please cc me on any responses)

Re: configure script does not give error if "ar" is not found

[Gavin Smith]

On 23 June 2015 at 20:37, Tim Rice  wrote:
>> I was trying to build Texinfo on a SunOS 5.10 sparc system, but got an
>> error message when it tried to build gnulib, that "ar" didn't exist.
>> Looking at config.log I have:
>
> Is /usr/ccs/bin in your PATH?
> What does "whereis ar" say?
>

It wasn't. After I added it to the PATH, running "make" worked fine. I
would have liked to have been told by the configure script so it would
have been easier for me to work out that I had to add it to PATH.

Thanks,
Gavin

How to find out which modules have brought in a particular module

[Gavin Smith]

I'm interested in reducing the number of checks done in a configure
script: one way could be to make more use of conditional dependencies
between modules. gnulib-tool --add-import lists the modules which were
used and which were brought in as dependencies. However, there are
some things I'd like to do that I couldn't find options for. One is to
see which modules brought in a module. The way I was doing this was by
going into the "gnulib/modules" directory in the Gnulib checkout and
grepping for the name of the module. It would be nice to be able to
get this information automatically, and in a way that takes into
account indirect dependencies. Maybe there should be an option for
gnulib-tool that can list the modules that have been brought in, and
for each of them, list the modules that have been explicitly asked for
that depend on the module, directly or indirectly, and also list the
modules that depend on the module conditionally.

For example I wondered why the checks for the unistd module were being
run. I found that getopt-posix had a dependency on it. I edited the
module file to make this a conditional dependency, reran gnulib-tool
--add-import, ran "make configure" to remake the configure script: but
when I ran "configure" again, the tests were still run. I expected
that there was another module also depending on unistd, but it wasn't
immediately obvious which one, because several modules that could have
been imported depended on unistd. That's as far as I got investigating
the matter. Is there an easier way to investigate this kind of thing,
that I've been missing? Or would gnulib-tool benefit from the extra
functionality I've suggested?

AC_SUBST of pkglibexecdir in 'configmake' module

[Gavin Smith]

Hello,

The configmake module uses AC_SUBST for pkglibexecdir, which overrides
Automake's default definition. If gl_CONFIGMAKE_PREP isn't run then
the substituted value is empty.

This was a problem with Texinfo, reported here:
http://lists.gnu.org/archive/html/bug-texinfo/2016-01/msg00046.html

Maybe the assignment could be made unconditionally, or removed.

Thanks,

Gavin

Re: How to find out which modules have brought in a particular module

[Gavin Smith]

On 21 October 2015 at 02:42, Pádraig Brady  wrote:
> There was some work on displaying a graph previously.
> http://lists.gnu.org/archive/html/bug-gnulib/2011-03/msg00276.html
> Something like this is worth adding I think.
>
> cheers,
> Pádraig

Here's basic functionality that shows direct dependencies. Indirect
dependencies and noting conditional dependencies are a possible
extension, but I think that this output as it is could be useful for
seeing the interdependencies between modules, and potentially
eliminating some modules that make checks you don't care about. If
some code like this is included, it should of course be controlled by
a command-line option.

Anybody please feel free to suggest directions I should take this code in.

PS I was discouraged after I sent my first mail when I realised that
gnulib-tool was written in shell script (I thought it was in Perl for
some reason).

diff --git a/gnulib-tool b/gnulib-tool
index 426ace1..d0593fa 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -2885,6 +2885,9 @@ func_modules_transitive_closure ()
 done
 if $inc && func_acceptable "$dep"; then
   func_append inmodules " $dep"
+  cachevar=c_`echo "$dep" | LC_ALL=C sed -e 's/[^a-zA-Z0-9_]/_/g'`
+  # fixme: should use func_cache_var $dep instead
+  func_append ${cachevar}_demanded_by " $module"
   if test "$cond_dependencies" = true; then
 escaped_dep=`echo "$dep" | sed -e "$sed_escape_dependency"`
 sed_extract_condition1='/^ *'"$escaped_dep"' *$/{
@@ -2940,11 +2943,20 @@ func_show_module_list ()
 bold_on=
 bold_off=
   fi
+  (for m in $modules; do
+cachevar=c_`echo "$m" | LC_ALL=C sed -e 's/[^a-zA-Z0-9_]/_/g'`
+# fixme: should use func_cache_var $dep instead
+eval echo \"${m}\|\" \" \[\" \$${cachevar}_demanded_by \"\]\"
+  done) >"$tmp"/modules-with-reasons
   echo "Module list with included dependencies (indented):"
-  echo "$specified_modules" | sed -e '/^$/d' -e 's/$/| /' >
"$tmp"/specified-modules
-  echo "$modules" | sed -e '/^$/d' \
+  echo "$specified_modules" | sed -e '/^$/d' -e 's/$/|0/' >
"$tmp"/specified-modules
+  cat "$tmp"/modules-with-reasons \
+| sed -e '/^$/d' \
 | LC_ALL=C join -t '|' -a2 "$tmp"/specified-modules - \
-| sed -e 's/^\(.*\)|.*/|\1/' -e 's/^//' -e 's/^|\(.*\)$/
'"${bold_on}"'\1'"${bold_off}"'/'
+| sed -e 's/^\(.*\)|0|.*/|\1/' \
+  -e 's/^//' \
+  -e 's/^|\(.*\)$/  '"${bold_on}"'\1'"${bold_off}"'/' \
+  -e 's/|/\n/'
 }

 # func_modules_transitive_closure_separately

===

Sample output from "gnulib-tool --add-import --dry-run"

Module list with included dependencies (indented):
absolute-header
  [ include_next ]
alloca-opt
  [ regex malloca vasnprintf ]
  argz
btowc
  [ regex ]
configmake
  [ localcharset ]
dosname
  [ lstat stat ]
errno
  [ strerror vasprintf strerror-override vasnprintf ]
error
  [ xalloc-die ]
exitfail
  [ xalloc-die ]
extensions
  [ gettext mbchar mbswidth memrchr mkstemp regex vasprintf
getopt-posix iswblank mbrtowc mbsinit memmem-simple mempcpy stdio
stpcpy strcasestr-simple string strndup strnlen tempname wcrtomb
wcwidth fcntl-h langinfo localcharset memchr secure_getenv locale time
]
extern-inline
  [ mbchar mbiter xalloc mbuiter string wctype-h unistd xsize ]
fcntl-h
  [ tempname ]
float
  [ vasnprintf ]
  getopt-gnu
getopt-posix
  [ getopt-gnu ]
  gettext
gettext-h
  [ gettext regex getopt-posix xalloc-die ]
gettimeofday
  [ tempname time ]
havelib
  [ gettext iconv threadlib ]
  iconv
include_next
  [ stdarg errno getopt-posix stdint stdio stdlib string wchar
wctype-h fcntl-h float langinfo stddef sys_stat sys_time sys_types
unistd locale strings time ]
intprops
  [ strerror ]
iswblank
  [ mbchar ]
langinfo
  [ nl_langinfo ]
largefile
  [ mkstemp lstat stat ]
localcharset
  [ mbrtowc wcwidth ]
locale
  [ localeconv ]
localeconv
  [ nl_langinfo ]
lock
  [ regex ]
lstat
  [ tempname ]
malloc-posix
  [ strdup-posix ]
malloca
  [ mbsstr ]
  mbchar
  mbiter
mbrtowc
  [ mbiter mbswidth regex mbsinit mbuiter mbtowc ]
  mbscasecmp
  mbschr
mbsinit
  [ mbiter mbswidth regex mbrtowc mbuiter wcrtomb ]
  mbslen
  mbsncasecmp
  mbsstr
  mbswidth
mbtowc
  [ btowc ]
mbuiter
  [ mbscasecmp mbschr mbslen mbsncasecmp mbsstr ]
memchr
  [ memmem-simple strcasestr-simple vasnprintf strnlen1 strstr-simple ]
  memmem
memmem-simple
  [ memmem ]
mempcpy
  [ argz ]
  memrchr
  mkstemp
msvc-inval
  [ msvc-nothrow ]
msvc-nothrow
  [ error ]
multiarch
  [ stdint ]
nl_langinfo
  [ regex ]
nocrash
  [ getopt-gnu ]
pathmax
  [ stat ]
  regex
secure_get

Re: How to find out which modules have brought in a particular module

[Gavin Smith]

> Here's basic functionality that shows direct dependencies

Also sending as an attachment in case the diff was mangled in the email.


diff
Description: Binary data

nl_langinfo.m4: HAVE_LANGINFO_CODESET unquoted

[Gavin Smith]

In the nl_langinfo.m4 file, there's a test:

 if test $HAVE_LANGINFO_CODESET = 1 && test $HAVE_LANGINFO_ERA = 1 \
&& test $FUNC_NL_LANGINFO_YESEXPR_WORKS = 1; then
  :
else

This gave an (apparently harmless) error message when configuring
under OpenIndiana 11:

./configure[22339]: test: argument expected

Maybe the variables being tested need to have their expansions
surrounded in double quotes, or whatever reason these variables aren't
set should be fixed.

Re: nl_langinfo.m4: HAVE_LANGINFO_CODESET unquoted

[Gavin Smith]

On 11 January 2016 at 23:08, Paul Eggert  wrote:
> On 01/11/2016 12:32 PM, Gavin Smith wrote:
>>

> Let's do the latter, since these variables should all be set to 0 or 1 by
> then.
>
> The macro in question AC_REQUIREs gl_LANGINFO_SET, which should set
> HAVE_LANGINFO_CODESET and HAVE_LANGINFO_ERA to either 0 or 1. The macro also
> sets FUNC_NL_LANGINFO_YESEXPR_WORKS to either 0 or 1, before using it. So,
> can you investigate which of these variables is not set as expected, and
> why?

Neither HAVE_LANGINFO_CODESET nor HAVE_LANGINFO_ERA is set.
FUNC_NL_LANGINFO_YESEXPR_WORKS is set.

gl_LANGINFO_H_DEFAULTS is expanded in func_gl_gnulib_m4code_langinfo,
which I believe contains the code from the langinfo module.
func_gl_gnulib_m4code_langinfo is only called from
func_gl_gnulib_m4code_nl_langinfo, which is from the nl_langinfo
module, which was the only module imported explicitly.

func_gl_gnulib_m4code_nl_langinfo only calls
func_gl_gnulib_m4code_langinfo after the code that tries to use
HAVE_LANGINFO_CODESET, thus the error. It calls it because of the
dependencies between the modules:

Description:
nl_langinfo() function: query locale dependent information.

Files:
lib/nl_langinfo.c
m4/nl_langinfo.m4

Depends-on:
langinfo
localeconv  [test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1]

and

  func_gl_gnulib_m4code_langinfo
  if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
func_gl_gnulib_m4code_localeconv
  fi


Maybe AC_REQUIRE isn't hoisting the code far enough? The code is
remaining in the function corresponding to a gnulib module's code, and
not going to the very top level. The code is expanded, but is executed
too late.

Re: nl_langinfo.m4: HAVE_LANGINFO_CODESET unquoted

[Gavin Smith]

On 12 January 2016 at 00:53, Gavin Smith  wrote:
> Maybe AC_REQUIRE isn't hoisting the code far enough? The code is
> remaining in the function corresponding to a gnulib module's code, and
> not going to the very top level. The code is expanded, but is executed
> too late.

So gnulib-m4.comp, which gnulib-tool creates, looks like this:

  func_gl_gnulib_m4code_langinfo ()
  {
if ! $gl_gnulib_enabled_langinfo; then
  gl_LANGINFO_H
  gl_gnulib_enabled_langinfo=true
fi
  }
  func_gl_gnulib_m4code_nl_langinfo ()
  {
if ! $gl_gnulib_enabled_nl_langinfo; then
  gl_FUNC_NL_LANGINFO
  if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
AC_LIBOBJ([nl_langinfo])
  fi
  gl_LANGINFO_MODULE_INDICATOR([nl_langinfo])
  gl_gnulib_enabled_nl_langinfo=true
  func_gl_gnulib_m4code_langinfo
  if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
func_gl_gnulib_m4code_localeconv
  fi
fi
  if test $ac_use_included_regex = yes; then
func_gl_gnulib_m4code_nl_langinfo
  fi

I think because gl_LANGINFO_H is used earlier in the file, in the body
of func_gl_gnulib_m4code_langinfo, the macro that is AC_REQUIRE'd is
expanded there, and not later in gl_FUNC_NL_LANGINFO.

Re: nl_langinfo.m4: HAVE_LANGINFO_CODESET unquoted

[Gavin Smith]

On 12 January 2016 at 12:28, Gavin Smith  wrote:
>   func_gl_gnulib_m4code_nl_langinfo ()
>   {
> if ! $gl_gnulib_enabled_nl_langinfo; then
>   gl_FUNC_NL_LANGINFO
>   if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
> AC_LIBOBJ([nl_langinfo])
>   fi
>   gl_LANGINFO_MODULE_INDICATOR([nl_langinfo])
>   gl_gnulib_enabled_nl_langinfo=true
>   func_gl_gnulib_m4code_langinfo
>   if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
> func_gl_gnulib_m4code_localeconv
>   fi
> fi
>   if test $ac_use_included_regex = yes; then
> func_gl_gnulib_m4code_nl_langinfo
>   fi
>
> I think because gl_LANGINFO_H is used earlier in the file, in the body
> of func_gl_gnulib_m4code_langinfo, the macro that is AC_REQUIRE'd is
> expanded there, and not later in gl_FUNC_NL_LANGINFO.

I wonder if the code should instead be

>   func_gl_gnulib_m4code_nl_langinfo ()
>   {
> if ! $gl_gnulib_enabled_nl_langinfo; then
>   func_gl_gnulib_m4code_langinfo
>   if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
> func_gl_gnulib_m4code_localeconv
>   fi
>   gl_FUNC_NL_LANGINFO
>   if test $HAVE_NL_LANGINFO = 0 || test $REPLACE_NL_LANGINFO = 1; then
> AC_LIBOBJ([nl_langinfo])
>   fi
>   gl_LANGINFO_MODULE_INDICATOR([nl_langinfo])
>   gl_gnulib_enabled_nl_langinfo=true
> fi
>  }

It could be because nl_langinfo is a conditional module (in my import)
(on the "regex" module), so nl_langinfo's dependencies shouldn't be
run unconditionally. That's why the call to the code for nl_langinfo's
dependencies is inside func_gl_gnulib_m4code_nl_langinfo: my guess is
that the dependency code should come first.

Maybe this hasn't been a problem before because the combination of
conditional module dependencies and AC_REQUIRE isn't that common? If
the langinfo module had been unconditional, I'd guess its code would
have been executed first.

The code for this output is in func_emit_autoconf_snippets; it
shouldn't be hard to swap them around. I can try to post a patch if
others agree this is the reason for the problem.

regex error with NetBSD pcc

[Gavin Smith]

Using the "regex" module, I had the error

 #error "Add case for new bitset_word_t size"

from regex_internal.h when using pcc under NetBSD.

$ pcc --version
pcc 1.2.0.DEVEL 20141228 for x86_64--netbsd

The problem appears to be that the C preprocessor thinks that
ULONG_MAX is a negative value. With

#if 0xUL < 0
#warning OK
#endif

the message "OK" is printed. This appears to be the same bug mentioned here:

http://lists.gnu.org/archive/html/bug-gnulib/2009-02/msg00045.html

I recommend that NOTHING BE DONE about this failure. I write this
email, not to get a fix, but to stop others spending time if they have
the same problem.

The amount of workarounds for bugs and non-comformance to standards
should reduce over time, not increase (maybe one beautiful day there
won't even be a need for gnulib), which won't happen if people keep on
writing new buggy, defective compilers from scratch. Adding
portability workarounds for these new compilers just encourages people
to use them.

Earlier on pcc on this mailing list:

http://lists.gnu.org/archive/html/bug-gnulib/2011-11/msg00205.html
http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00110.html

Re: texinfo.tex 2016-05-07.20: "Tex capacity exceeded" with Emacs manuals

[Gavin Smith]

On 25 May 2016 at 19:08, Paul Eggert  wrote:
> texinfo.tex version 2016-05-07.20 (the current version in Gnulib) causes
> what appears to be runaway recursion when used to make PDF files for GNU
> Emacs. The Emacs bug report is here:
>
> http://bugs.gnu.org/23611
>
> To reproduce the problem on a Fedora 23 x86-64 machine with build tools
> installed, you can run these shell commands:
>
> git clone git://git.savannah.gnu.org/emacs.git emacs-bug
> cd emacs-bug
> git checkout 1ee91bf89176251f6e399c8436dca0248cdd6f6b
> ./autogen.sh all
> ./configure
> make pdf

These instructions download several 100's of MB of a git repository
and I wasn't willing to let it finish. I tried the file
"emacs-lisp-intro.texi" from
http://git.savannah.gnu.org/cgit/emacs.git/tree/doc/lispintro, and
after editing the file a bit (so it didn't need other files), I
processed it with that version of texinfo.tex. Unfortunately, I wasn't
able to reproduce the error. I did find one problem, though: some
entries in the tables of contents, like 13.1, have "{No value for
``COUNT-WORDS''} " instead of the proper value. That's likely been
brought in by the same change, and needs to be fixed. The change was
to expand @value when the table of contents file was read in, and not
when it was written out: however, in this file the @value is only
given a value (with @set) after the @insertcopying command. I
understand that this should work. I'll have a look at it. If I
remember correctly, I made this change in the first place to prevent a
similar infinite recursion error happening when writing an undefined
@value to an auxiliary file.

Re: texinfo.tex 2016-05-07.20: "Tex capacity exceeded" with Emacs manuals

[Gavin Smith]

On 26 May 2016 at 16:48, Paul Eggert  wrote:
> OK, here's a smaller example taken from the Emacs source code. Unpack the
> attached tarball in a fresh directory, copy the latest Gnulib texinfo.tex
> into it, and run the shell command 'texi2pdf bovine.texi'. On my platform
> (Fedora 23 x86-64, texi2pdf (GNU Texinfo 6.0) 6363) the output says
> './bovine.toc:1: TeX capacity exceeded, sorry [input stack size=5000].' Full
> output log attached.

OK, I got the same problem with the files you sent. I've committed and
uploaded a new version that reverses the change, so @value is expanded
at the time of writing again. Undefined @value's, when used in index
entries, should not cause failures, either. It's quite possible that
there is some combination that breaks, but it seems to work better for
the files I tried.

Remove dependency of getdelim on realloc-posix

[Gavin Smith]

This patch is intended to remove a warning due to redefining realloc
twice when building Perl extension modules, once by gnulib, a second
time by Perl's headers. Reported here

http://lists.gnu.org/archive/html/bug-texinfo/2016-08/msg00021.html

diff --git a/ChangeLog b/ChangeLog
index 2414e8e..9033e14 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2016-08-13  Gavin Smith  
+
+   * modules/getdelim: Remove dependency on realloc-posix.
+   * lib/getdelim.c (getdelim): Set errno to ENOENT if realloc failed,
+   as is done in lib/canonicalize-lgpl.c (__realpath).
+
 2016-07-15  Paul Eggert  

obstack: pacify GCC 6 with -Wnull-dereference
diff --git a/lib/getdelim.c b/lib/getdelim.c
index 68a6f34..8d2dbbe 100644
--- a/lib/getdelim.c
+++ b/lib/getdelim.c
@@ -75,6 +75,9 @@ getdelim (char **lineptr, size_t *n, int delimiter, FILE *fp)
   if (new_lineptr == NULL)
 {
   result = -1;
+  /* It's easier to set errno to ENOMEM than to rely on the
+ 'realloc-posix' gnulib module.  */
+  errno = ENOMEM;
   goto unlock_return;
 }
   *lineptr = new_lineptr;
diff --git a/modules/getdelim b/modules/getdelim
index e84558c..ee71165 100644
--- a/modules/getdelim
+++ b/modules/getdelim
@@ -9,7 +9,6 @@ Depends-on:
 stdio
 extensions
 stdint  [test $HAVE_GETDELIM = 0 || test $REPLACE_GETDELIM = 1]
-realloc-posix   [test $HAVE_GETDELIM = 0 || test $REPLACE_GETDELIM = 1]
 errno   [test $HAVE_GETDELIM = 0 || test $REPLACE_GETDELIM = 1]

 configure.ac:

Re: Remove dependency of getdelim on realloc-posix

[Gavin Smith]

On 13 August 2016 at 15:08, Andreas Schwab  wrote:
> On Aug 13 2016, Gavin Smith  wrote:
>
>> +2016-08-13  Gavin Smith  
>> +
>> +   * modules/getdelim: Remove dependency on realloc-posix.
>> +   * lib/getdelim.c (getdelim): Set errno to ENOENT if realloc failed,
>
> s/ENOENT/ENOMEM/
>

Thank you, that was a mistake.

regex.c value out of range on Solaris compiler

[Gavin Smith]

Hello,

When I try to compile a program using the 'regex' module on Solaris 10, 
I get the output

/opt/solarisstudio12.3/bin/c99 -Xc -D_XPG6 -DHAVE_CONFIG_H -I. -I../..   
-D_REENTRANT  -I/opt/csw/include -c -o regex.o regex.c
"regex_internal.h", line 105: warning: macro redefined: gettext
"regex_internal.h", line 734: warning: token-less macro argument
Assembler: "/tmp/yabeAAAiUaidC", line 19010 : Value out of range
"/tmp/yabeAAAiUaidC", line 19031 : Value out of range
Failure in /opt/solarisstudio12.3/prod/bin/fbe, status = 0x7f00
Fatal Error exec'ing /opt/solarisstudio12.3/prod/bin/fbe
c99: acomp failed for regex.c

The warnings about macros are harmless, but the "Value out of range" 
errors are fatal.

I re-ran with the -S option to get assembler output:

/opt/solarisstudio12.3/bin/c99 -Xc -D_XPG6 -DHAVE_CONFIG_H -I. -I../..   
-D_REENTRANT  -I/opt/csw/include -c -o regex.o regex.c -S

Then if I try to compile the regex.o file:

mv regex.o regex.S
/opt/solarisstudio12.3/bin/c99 -Xc -D_XPG6 -DHAVE_CONFIG_H -I.  -I../..   
-D_REENTRANT  -I/opt/csw/include -c -o regex.o regex.S
Assembler: "regex.S", line 21273 : Value out of range
"regex.S", line 21294 : Value out of range
c99: fbe failed for regex.S

Line 21273 is
subl$4294967296,%eax

Line 21294 is
subl$4294967296,%eax

I think the error is that these constants are too large. 4,294,967,296
is 2 ** 32 which is too big a value to fit in 32 bits.

When I try again with the Gnulib from 2016-12-15, this file compiles OK, 
but when I apply the patch from

http://lists.gnu.org/archive/html/emacs-devel/2016-12/msg00666.html

the problem occurs. I believe that the faulty code is coming from the 
use of INT_ADD_WRAPV from intprops.h. I haven't investigated how exactly 
this macro leads to the faulty code. Hopefully I am on the right track 
here and someone will have an idea of how to fix it. If not, I will have 
to study what this macro does and why it is being used here.

Thanks,
Gavin

PS please keep me CC'd in any responses

Re: regex.c value out of range on Solaris compiler

[Gavin Smith]

On Fri, Apr 14, 2017 at 10:27:45PM +0100, Gavin Smith wrote:
> Line 21273 is
> subl$4294967296,%eax
> 
> Line 21294 is
> subl$4294967296,%eax
> 
> 
> the problem occurs. I believe that the faulty code is coming from the 
> use of INT_ADD_WRAPV from intprops.h. I haven't investigated how exactly 
> this macro leads to the faulty code.

The macro expands to a lot of code, but I simplified the code that gives
the same error:

$ cat tmp.c
int main (void)
{
int length1 = 0, length2 = 0;

if ( ( int ) ( ( ( unsigned int ) ( length1 ) + ( unsigned int ) ( 
length2 ) ) - ( ( - 2147483647 - 1 ) ) ) + ( ( - 2147483647 - 1 ) ) )
1;
}

$ /opt/solarisstudio12.3/bin/c99 -Xc -D_XPG6 -DHAVE_CONFIG_H -I. -I../..  
-D_REENTRANT  tmp.c
Assembler: "/tmp/yabeAAAUsaiac", line 23 : Value out of range
 Failure in /opt/solarisstudio12.3/prod/bin/fbe, status = 0x7f00
 Fatal Error exec'ing /opt/solarisstudio12.3/prod/bin/fbe
 c99: acomp failed for tmp.c

The same
subl$4294967296,%eax
line occurs in the assembly output.

Can this be fixed in intprops.h?

Re: regex.c value out of range on Solaris compiler

[Gavin Smith]

On Fri, Apr 14, 2017 at 07:00:06PM -0700, Paul Eggert wrote:
> On 04/14/2017 02:27 PM, Gavin Smith wrote:
> > /opt/solarisstudio12.3/bin/c99 -Xc -D_XPG6 -DHAVE_CONFIG_H -I. -I../..
> > -D_REENTRANT  -I/opt/csw/include -c -o regex.o regex.c
> > "regex_internal.h", line 105: warning: macro redefined: gettext
> 
> That's odd. Where was the gettext macro originally defined?

/opt/csw/include/libintl.h, line 132: #define gettext libintl_gettext

(I got this using the  -xdumpmacros=defs,loc option.)

> > Assembler: "/tmp/yabeAAAiUaidC", line 19010 : Value out of range
> > "/tmp/yabeAAAiUaidC", line 19031 : Value out of range
> 
> Ouch. This appears to be a compiler bug, as a compiler should never generate
> invalid assembly-language code. I cannot reproduce the problem on Solaris 10
> sparc (32-bit) with Oracle Developer Studio 12.5. You are on Solaris 10 x86
> (32-bit) and are using 12.3. Do you observe the same compiler bug with 
> 12.5 or 12.6 beta? If so, I suggest reporting it as a bug to Oracle.

I only have access to 12.4 on the system I am using, but that appears to
compile the test file I posted without error. Solaris Studio 12.2 gives 
the same error as 12.3.

> What happens if you compile with -xO2?

The errors from the assembler do not occur and the compilation finishes 
successfully.

> 
> Although I cannot reproduce the problem in my environment, I think I see why
> it's happening. I installed the attached patches into Gnulib; do they fix
> your problem?

Yes, the error goes away with the change you made. It's good that you 
managed to fix this.

Re: regex.c value out of range on Solaris compiler

[Gavin Smith]

On Sat, Apr 15, 2017 at 02:40:49PM -0700, Paul Eggert wrote:
> Sorry, I don't understand how this could be. regex_internal.h's "#define
> gettext(msgid) msgid" line is in the else-part of the #if that #includes
> libintl.h in its then-part, so how can libintl.h's #define for gettext be
> active when regex_internal's #define is processed?

libintl.h is being included via a system header, /usr/include/locale.h.  
The system header is included from gnulib's locale.h.  locale.h is 
included at the beginning of regex_internal.h.  locale.h is generated 
from locale.in.h.  There's a chance there is something wrong with the 
way I am compiling because headers under /opt/csw/include are being 
checked as well as /usr/share/include.  The log at

https://buildfarm.opencsw.org/buildbot/builders/texinfo-solaris10-i386/builds/1263/steps/shell_2/logs/stdio

doesn't have the same warning, and that is more likely to have been done 
with a correct environment for compiling.  The other logs at 
https://buildfarm.opencsw.org/buildbot/waterfall?category=texinfo
don't have the warning either, so it is probably okay.

Re: Gnulib's wctype.h fails to compile with MinGW

[Gavin Smith]

On Thu, Apr 27, 2017 at 08:50:25PM +0300, Eli Zaretskii wrote:
> > From: Bruno Haible 
> > Date: Thu, 27 Apr 2017 19:03:35 +0200
> > 
> > Thanks for the report and suggested fix.
> > 
> > The #ifdefology here seems a bit fragile to me (will likely break in other
> > forks of mingw), therefore I'm using an Autoconf test instead:
> 
> I only know of 2 flavors of MinGW (and wish there were only one), but
> of course an Autoconf test is even better.  Thanks.
> 
> Gavin, could this be part of the next pretest, please?

Yes, I will make sure the Gnulib files have been updated.

getopt _getopt_internal declaration clash on Solaris 5.10

[Gavin Smith]

On Sun, Apr 23, 2017 at 08:20:20PM -0400, Assaf Gordon wrote:
> ---
> 
> On Solaris 5.10, compilation fails due to a conflict in gnulib's getopt 
> module.
> See full build log here: 
>  https://pretest.housegordon.org/g/4602/logs/make.log?inlined=1
> (search for "getopt_int" to find the error).
> Snippet below:
> ===
> gcc -DHAVE_CONFIG_H -I. -I../..-D_REENTRANT  -g -O2 -MT getopt.o -MD -MP 
> -MF $depbase.Tpo -c -o getopt.o getopt.c &&\
> mv -f $depbase.Tpo $depbase.Po
> In file included from getopt.c:80:0:
> getopt_int.h:25:12: error: conflicting types for '_getopt_internal'
>  extern int _getopt_internal (int ___argc, char **___argv,
> ^
> In file included from ./getopt.h:34:0,
>  from getopt.h:34,
>  from getopt.c:24:
> /opt/csw/include/getopt.h:118:12: note: previous declaration of 
> '_getopt_internal' was here
>  extern int _getopt_internal (int argc, char *const *argv,
> ^
> getopt.c:702:1: error: conflicting types for '_getopt_internal'
>  _getopt_internal (int argc, char **argv, const char *optstring,
>  ^
> In file included from ./getopt.h:34:0,
>  from getopt.h:34,
>  from getopt.c:24:
> /opt/csw/include/getopt.h:118:12: note: previous declaration of 
> '_getopt_internal' was here
>  extern int _getopt_internal (int argc, char *const *argv,
> ^
> *** Error code 1
> make: Fatal error: Command failed for target `getopt.o'

On a Solaris 5.10 system I have access to, /opt/csw/include/getopt.h
is a GNU libc include file from 1997. It is probably the same file that
you are using, as the declaration is on the same line (118).

I know it is possible to compile on this system with the right 
environment because it's done successfully on the OpenCSW buildfarm.  
(See https://buildfarm.opencsw.org/buildbot/waterfall?category=texinfo
for logs).

I tried compiling Texinfo 6.3 the same way and didn't get the same 
error, so it is probably due to a change in Gnulib.

I found in the older gnulib, the _getopt_internal had been redefined
as rpl_getopt_internal after the system header had been read, so the
definition clash did not occur.

I'm copying bug-gnulib into this email to see if the Gnulib developers 
can take this any further.

I believe the section that #define's the function names is now in
getopt-pfx-ext.h. _getopt_internal is not redefined there. Formerly,
I believe it was in getopt.h that _getopt_internal was redefined,
like this:

# define _getopt_internal __GETOPT_ID (getopt_internal)

gt_INTL_MACOSX not defined by gettext.m4

[Gavin Smith]

gettext.m4 from gnulib uses a macro gt_INTL_MACOSX, but running
gnulib-tool --add-import doesn't cause this macro to be defined in the
produced configure script.

There is a definition in intlmacosx.m4 but this file is not pulled in.
I have a backup of this file intlmacosx.m4~ which was probably created
by gnulib-tool when I updated the gnulib files. My guess is that the
dependencies for the gettext module have changed so it is not pulling
in the definition of this macro.

Originally reported here:
http://lists.gnu.org/archive/html/bug-texinfo/2017-07/msg00026.html

Re: gt_INTL_MACOSX not defined by gettext.m4

[Gavin Smith]

On Fri, Jul 14, 2017 at 12:57:43AM +0200, Bruno Haible wrote:
> Gavin Smith wrote:
> > Originally reported here:
> > http://lists.gnu.org/archive/html/bug-texinfo/2017-07/msg00026.html
> 
> Looking at the gnulib-cache.m4 and gnulib-comp.m4 from said package:
>   * gettext.m4 does NOT come from gnulib.
>   * You are using gnulib module 'gettext-h', not 'gettext'. (Which is 
> perfectly
> OK, since the 'gettext' module is now marked obsolete.)
> 

I see.  Thanks a lot for working this out.  I had forgotten I had removed the 
gettext module from gnulib.

I ran gettextize again, but it didn't install the extra files:

$   gettextize -f --po-dir=po --po-dir=po_document
Copying file ABOUT-NLS
Copying file build-aux/config.rpath
Not copying intl/ directory.
Copying file po/Makefile.in.in
Copying file po/Makevars.template
Copying file po/Rules-quot
Copying file po/boldquot.sed
Copying file po/en@boldquot.header
Copying file po/en@quot.header
Copying file po/insert-header.sin
Copying file po/quot.sed
Copying file po/remove-potcdate.sin
Copying file po_document/Makefile.in.in
Copying file po_document/Makevars.template
Copying file po_document/Rules-quot
Copying file po_document/boldquot.sed
Copying file po_document/en@boldquot.header
Copying file po_document/en@quot.header
Copying file po_document/insert-header.sin
Copying file po_document/quot.sed
Copying file po_document/remove-potcdate.sin
Copying file otherm4/gettext.m4
Copying file otherm4/iconv.m4
Copying file otherm4/lib-ld.m4
Copying file otherm4/lib-link.m4
Copying file otherm4/lib-prefix.m4
Copying file otherm4/nls.m4
Copying file otherm4/po.m4
Copying file otherm4/progtest.m4
Adding an entry to ChangeLog (backup is in ChangeLog~)

Please run 'aclocal -I otherm4 -I gnulib/m4' to regenerate the aclocal.m4 file.
You need aclocal from GNU automake 1.9 (or newer) to do this.
Then run 'autoconf' to regenerate the configure file.

You might also want to copy the convenience header file gettext.h
from the /usr/local/share/gettext directory into your package.
It is a wrapper around  that implements the configure --disable-nls
option.

Press Return to acknowledge the previous two paragraphs.

$

I understand this is not a gnulib problem.  I will continue to research
gettextize and why it is not adding the files it needs.

Take account of splitting option in gendocs.sh

[Gavin Smith]

The patch at the bottom of this mail makes gendocs.sh take account
of the --split option.  Texinfo uses this as one of its manuals
(texi2any_internals) uses --split=chapter for the HTML output.  Up to
this point we have been getting around this problem by using a custom
gendocs template.

It appears that the existing gendocs template was written to support
different splitting possibilities but that gendocs.sh didn't use this
fully.

Please let me know if any further work needs to be done on this patch.

I notice that this script also still supports texi2html, which is no
longer developed.  I could produce a patch to remove this support if
it was very likely that nobody was relying on it.

diff --git a/ChangeLog b/ChangeLog
index 9477335e75..d67b9b2f0a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2023-02-26  Gavin Smith 
+
+   gendocs: support chapter- and section-level split
+   * build-aux/gendocs.sh:
+   Strip out parts of the template depending on --split.
+   * doc/gendocs_template: Add lines to mark parts of file to output
+   only when splitting HTML by node.
+
 2023-02-26  Bruno Haible  
 
dfa: Avoid warnings with some Apple clang versions.
diff --git a/build-aux/gendocs.sh b/build-aux/gendocs.sh
index 1ad5cf4f5d..ca813bffe3 100755
--- a/build-aux/gendocs.sh
+++ b/build-aux/gendocs.sh
@@ -465,12 +465,20 @@ fi
 
 # 
 printf "\nMaking index.html for %s...\n" "$PACKAGE"
-if test -z "$use_texi2html"; then
-  CONDS="/%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
- /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d"
+if test x$split = xnode; then
+  CONDS="/%%IF  *HTML_NODE%%/d;/%%ENDIF  *HTML_NODE%%/d;\
+ /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+ /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;"
+elif test x$split = xchapter; then
+  CONDS="/%%IF  *HTML_CHAPTER%%/d;/%%ENDIF  *HTML_CHAPTER%%/d;\
+ /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
+ /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
+elif test x$split = xsection; then
+  CONDS="/%%IF  *HTML_SECTION%%/d;/%%ENDIF  *HTML_SECTION%%/d;\
+ /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+ /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
 else
-  # should take account of --split here.
-  CONDS="/%%ENDIF.*%%/d;/%%IF  *HTML_SECTION%%/d;/%%IF  *HTML_CHAPTER%%/d"
+  CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;" # invalid split argument
 fi
 
 curdate=`$SETLANG date '+%B %d, %Y'`
diff --git a/doc/gendocs_template b/doc/gendocs_template
index a101977dfc..0b959f0379 100644
--- a/doc/gendocs_template
+++ b/doc/gendocs_template
@@ -22,8 +22,10 @@ without any warranty.
 
 HTML
 (%%HTML_MONO_SIZE%%K bytes) - entirely on one web page.
+%%IF HTML_NODE%%
 HTML - with one web page per
 node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML - with one web page per
 section.
@@ -35,9 +37,11 @@ without any warranty.
 HTML compressed
 (%%HTML_MONO_GZ_SIZE%%K gzipped characters) - entirely on
 one web page.
+%%IF HTML_NODE%%
 HTML compressed
 (%%HTML_NODE_TGZ_SIZE%%K gzipped tar file) -
 with one web page per node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML compressed
 (%%HTML_SECTION_TGZ_SIZE%%K gzipped tar file) -

Re: Take account of splitting option in gendocs.sh

[Gavin Smith]

On Sun, Feb 26, 2023 at 09:52:08PM +0100, Bruno Haible wrote:
> Gavin Smith wrote:
> > I notice that this script also still supports texi2html, which is no
> > longer developed.  I could produce a patch to remove this support if
> > it was very likely that nobody was relying on it.
> 
> A web search on www.gnu.org shows that the manuals of at least the following
> packages still were generated using texi2html:
>   anubis, auctex, autogen, cfengine, cflow, classpath, combine, complexity,
>   cpio, freeimpi, freetalk, gama, gnats, gnu-pw-mgr, gnugo, gnuit, gnupod,
>   hurd, indent, libcdio, libjit, oleo, plotutils, radius, sqltutor, tar.
> 
> That's definitely more than nobody.
> 
> Bruno

OK it doesn't do any harm to keep the texi2html support in.

The patch I sent applied for both texi2html and texi2any, although I
hadn't tested it with texi2html.

Re: Take account of splitting option in gendocs.sh

[Gavin Smith]

On Mon, Feb 27, 2023 at 07:30:37AM +, Gavin Smith wrote:
> OK it doesn't do any harm to keep the texi2html support in.
> 
> The patch I sent applied for both texi2html and texi2any, although I
> hadn't tested it with texi2html.

Is there any progress on applying this patch?

This would allow us to eliminate the custom gendocs template we use
in the Texinfo project for chapter-level splitting for one of our
manuals.

Re: Take account of splitting option in gendocs.sh

[Gavin Smith]

On Sun, Mar 26, 2023 at 06:53:21PM +0200, Bruno Haible wrote:
> > Is there any progress on applying this patch?
> 
> I can't review it, because the interplay between the script and the template
> is something I don't know about.
> 
> Also: What are the effects of the patch on the output, if no --split option
> is being passed to gendocs.sh?

It should be the same as if --split=node was given, as 'node' is the
default value of $split in this script.

I see now that for texi2html, all three splitting options are used
("node", "chapter" and "section").  This can be seen at the GNU
Anubis manuals page (https://www.gnu.org/software/anubis/manual/),
where all three of these versions of the manual are available.

Here's an updated patch that does not affect the behaviour when
used with texi2html.

diff --git a/ChangeLog b/ChangeLog
index a0de338759..41a27c6710 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2023-03-26  Gavin Smith 
+
+   gendocs: support chapter- and section-level split
+   * doc/gendocs_template: Add lines to mark parts of file to output
+   only when splitting HTML by node.
+   * build-aux/gendocs.sh
+   [!texi2html]: Strip out parts of the template depending on --split.
+   [texi2html] Include all of the template as before.
+
 2023-03-25  Bruno Haible  
 
stdlib: ISO C 23: Document issue with once_flag and call_once.
diff --git a/build-aux/gendocs.sh b/build-aux/gendocs.sh
index 1ad5cf4f5d..c7c9e2826c 100755
--- a/build-aux/gendocs.sh
+++ b/build-aux/gendocs.sh
@@ -466,11 +466,24 @@ fi
 # 
 printf "\nMaking index.html for %s...\n" "$PACKAGE"
 if test -z "$use_texi2html"; then
-  CONDS="/%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
- /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d"
+  if test x$split = xnode; then
+CONDS="/%%IF  *HTML_NODE%%/d;/%%ENDIF  *HTML_NODE%%/d;\
+   /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;"
+  elif test x$split = xchapter; then
+CONDS="/%%IF  *HTML_CHAPTER%%/d;/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
+   /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
+  elif test x$split = xsection; then
+CONDS="/%%IF  *HTML_SECTION%%/d;/%%ENDIF  *HTML_SECTION%%/d;\
+   /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
+  else
+CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;" # invalid split argument
+  fi
 else
   # should take account of --split here.
-  CONDS="/%%ENDIF.*%%/d;/%%IF  *HTML_SECTION%%/d;/%%IF  *HTML_CHAPTER%%/d"
+  CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;"
 fi
 
 curdate=`$SETLANG date '+%B %d, %Y'`
diff --git a/doc/gendocs_template b/doc/gendocs_template
index a101977dfc..0b959f0379 100644
--- a/doc/gendocs_template
+++ b/doc/gendocs_template
@@ -22,8 +22,10 @@ without any warranty.
 
 HTML
 (%%HTML_MONO_SIZE%%K bytes) - entirely on one web page.
+%%IF HTML_NODE%%
 HTML - with one web page per
 node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML - with one web page per
 section.
@@ -35,9 +37,11 @@ without any warranty.
 HTML compressed
 (%%HTML_MONO_GZ_SIZE%%K gzipped characters) - entirely on
 one web page.
+%%IF HTML_NODE%%
 HTML compressed
 (%%HTML_NODE_TGZ_SIZE%%K gzipped tar file) -
 with one web page per node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML compressed
 (%%HTML_SECTION_TGZ_SIZE%%K gzipped tar file) -

Re: Take account of splitting option in gendocs.sh

[Gavin Smith]

On Sun, Mar 26, 2023 at 04:11:45PM -0600, Karl Berry wrote:
> + gendocs: support chapter- and section-level split
> 
> Seems sensible to me. Basically making the html-by-node part of the
> template conditional along with the other split options?
> 
> (Overall, it seems to me that several of these variants are pretty
> pointless nowadays, but never mind.)
> 
> FWIW, in the past, only texi2html supported split=chapter. Hence the
> hardwiring of various things.

I've made minor changes to the patch as specified in Karl's response below.
Please can this be applied?

The Texinfo package is still using a custom gendocs template for some of
its manuals and I would like to get rid of it to simplify our build process.


diff --git a/ChangeLog b/ChangeLog
index e55bfec8ed..9c8804f3f6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2023-07-12  Gavin Smith 
+
+   gendocs: support chapter- and section-level split
+   * doc/gendocs_template: Add lines to mark parts of file to output
+   only when splitting HTML by node.
+   Remove obsolete comment about ranges of years in copyright notice
+   (as per Karl).
+   * build-aux/gendocs.sh
+   [!texi2html]: Strip out parts of the template depending on --split.
+   [texi2html] Include all of the template as before.
+
 2023-07-11  Bruno Haible  
 
mbsrtoc32s, mbsnrtoc32s: Small optimization.
diff --git a/build-aux/gendocs.sh b/build-aux/gendocs.sh
index 1ad5cf4f5d..0e470bf7ce 100755
--- a/build-aux/gendocs.sh
+++ b/build-aux/gendocs.sh
@@ -2,7 +2,7 @@
 # gendocs.sh -- generate a GNU manual in many formats.  This script is
 #   mentioned in maintain.texi.  See the help message below for usage details.
 
-scriptversion=2023-01-21.00
+scriptversion=2023-07-12.10
 
 # Copyright 2003-2023 Free Software Foundation, Inc.
 #
@@ -466,11 +466,25 @@ fi
 # 
 printf "\nMaking index.html for %s...\n" "$PACKAGE"
 if test -z "$use_texi2html"; then
-  CONDS="/%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
- /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d"
+  if test x$split = xnode; then
+CONDS="/%%IF  *HTML_NODE%%/d;/%%ENDIF  *HTML_NODE%%/d;\
+   /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;"
+  elif test x$split = xchapter; then
+CONDS="/%%IF  *HTML_CHAPTER%%/d;/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_SECTION%%/,/%%ENDIF  *HTML_SECTION%%/d;\
+   /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
+  elif test x$split = xsection; then
+CONDS="/%%IF  *HTML_SECTION%%/d;/%%ENDIF  *HTML_SECTION%%/d;\
+   /%%IF  *HTML_CHAPTER%%/,/%%ENDIF  *HTML_CHAPTER%%/d;\
+   /%%IF  *HTML_NODE%%/,/%%ENDIF  *HTML_NODE%%/d;"
+  else
+CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;" # invalid split argument
+  fi
 else
-  # should take account of --split here.
-  CONDS="/%%ENDIF.*%%/d;/%%IF  *HTML_SECTION%%/d;/%%IF  *HTML_CHAPTER%%/d"
+  # for texi2html, we do not take account of --split and simply output
+  # all variants
+  CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;"
 fi
 
 curdate=`$SETLANG date '+%B %d, %Y'`
diff --git a/doc/gendocs_template b/doc/gendocs_template
index a101977dfc..0e6e8e767b 100644
--- a/doc/gendocs_template
+++ b/doc/gendocs_template
@@ -22,8 +22,10 @@ without any warranty.
 
 HTML
 (%%HTML_MONO_SIZE%%K bytes) - entirely on one web page.
+%%IF HTML_NODE%%
 HTML - with one web page per
 node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML - with one web page per
 section.
@@ -35,9 +37,11 @@ without any warranty.
 HTML compressed
 (%%HTML_MONO_GZ_SIZE%%K gzipped characters) - entirely on
 one web page.
+%%IF HTML_NODE%%
 HTML compressed
 (%%HTML_NODE_TGZ_SIZE%%K gzipped tar file) -
 with one web page per node.
+%%ENDIF HTML_NODE%%
 %%IF HTML_SECTION%%
 HTML compressed
 (%%HTML_SECTION_TGZ_SIZE%%K gzipped tar file) -
@@ -72,9 +76,7 @@ script.)
 
+ with the webmasters first.  -->
 
 
 
@@ -87,7 +89,7 @@ the FSF.  Broken links and other corrections or suggestions 
can be sent
 to mailto:%%EMAIL%%";><%%EMAIL%%>.
 
 
-Copyright © 2022 Free Software Foundation, Inc.
+Copyright © 2023 Free Software Foundation, Inc.
 
 This page is licensed under a https://creativecommons.org/licenses/by-nd/3.0/us/";>Creative




> 
>  else
># should take account of --split here.
> -  CONDS="/%%ENDIF.*%%/d;/%%IF  *HTML_SECTION%%/d;/%%IF  
> *HTML_CHAPTER%%/d"
> +  CONDS="/%%IF.*%%/d;/%%ENDIF.*%%/d;"
> 
> Should the comment now be something like
> # No split output, so delete all conditionals.
> ?
> 
> --
> 
> Copyright © 2022 Free Software Foundation, Inc.
> 
> On the usual triviality front, the copyright year in gendocs_template
> appar

Re: Take account of splitting option in gendocs.sh

[Gavin Smith]

On Wed, Jul 12, 2023 at 04:02:34PM +0200, Bruno Haible wrote:
> Gavin Smith wrote:
> > I've made minor changes to the patch as specified in Karl's response below.
> > Please can this be applied?
> 
> Thanks for having taken into account both my and Karl's comments. I have
> applied your patch.
> 
> Bruno

Thank you!

Re: c32width gives incorrect return values in C locale

[Gavin Smith]

On Sat, Nov 11, 2023 at 09:06:41PM +0100, Bruno Haible wrote:
> [CCing bug-gnulib]
> Indeed, the c32* functions by design work only on those Unicode characters
> that can be represented as multibyte sequences in the current locale.
> 
> I'll document this better in the Gnulib manual.
> 
> Since you want texinfo to work on UTF-8 encoded text with characters outside
> the repertoire of the current locale, you'll need the libunistring functions,
> documented in
> .
> Namely, replace c32width with uc_width.

Thanks, that seems to work perfectly.

I also changed c32isupper to uc_is_upper.  The gnulib manual stated
(node "isupper"):

  ‘c32isupper’
   This function operates in a locale dependent way, on 32-bit wide
   characters.  In order to use it, you first have to convert from
   multibyte to 32-bit wide characters, using the ‘mbrtoc32’ function.
   It is provided by the Gnulib module ‘c32isupper’.
  
  ...
  
  ‘uc_is_upper’
   This function operates in a locale independent way, on Unicode
   characters.  It is provided by the Gnulib module
   ‘unictype/ctype-upper’.

- and we wanted the "locale independent way".

I did not understand why uc_width was said to be "locale dependent":

  "These functions are locale dependent."

- from 
.

I also don't understand the purpose of the "encoding" argument -- can this
always be "UTF-8"?

I'm also unclear on the exact relationship between the types char32_t,
ucs4_t and uint32_t.  For example, uc_width takes a ucs4_t argument
but u8_mbtouc writes to a char32_t variable.  In the code I committed,
I used a cast to ucs4_t when calling uc_width.

Re: c32width gives incorrect return values in C locale

[Gavin Smith]

On Sat, Nov 11, 2023 at 11:54:52PM +0100, Bruno Haible wrote:
> [CCing bug-libunistring]
> Gavin Smith wrote:
> > I did not understand why uc_width was said to be "locale dependent":
> > 
> >   "These functions are locale dependent."
> > 
> > - from 
> > <https://www.gnu.org/software/libunistring/manual/html_node/uniwidth_002eh.html#index-uc_005fwidth>.
> 
> That's because some Unicode characters have "ambiguous width" — width 1 in
> Western locales, width 2 is East Asian locales (for historic and font choice
> reasons).
> 
> > I also don't understand the purpose of the "encoding" argument -- can this
> > always be "UTF-8"?
> 
> Yes, it can be always "UTF-8"; then uc_width will always choose width 1 for
> these characters.
> 
> > I'm also unclear on the exact relationship between the types char32_t,
> > ucs4_t and uint32_t.  For example, uc_width takes a ucs4_t argument
> > but u8_mbtouc writes to a char32_t variable.  In the code I committed,
> > I used a cast to ucs4_t when calling uc_width.
> 
> These types are all identical. Therefore you don't even need to cast.
> 
>   - char32_t comes from  (ISO C 11 or newer).
>   - ucs4_t comes from GNU libunistring.
>   - uint32_t comes from .

Thanks for the advice.

Re: obstack module has alignment issues on sparc? (Re: set_labels_identifiers_target -fsanitize=undefined error)

[Gavin Smith]

On Tue, Nov 14, 2023 at 10:16:33PM +0100, Bruno Haible wrote:
> Sam James wrote:
> > > It appears that the obstack gnulib module is the culprit.
> 
> I replied:
> > Therefore, if it is a bug in gnulib, it is also a bug in glibc.
> 
> Sam was right. I was wrong. It is a bug in the 'obstack' gnulib module.

It is surprising it worked as well as it did with mixing the two different
struct definitions.

Thanks for the detailed investigation.  Will this be fixed in gnulib at
some point?

>  The root cause
>  --
> 
> Gnulib generally uses idioms for overriding functions that are safe to use
> in shared libraries and will avoid collisions. This is the business with
>   REPLACE_FOO=1
> and
>   #define foo rpl_foo
> and so on.
> 
> But the Gnulib module 'obstack' has never been updated to use these idioms.
> It is still at the state of 1997 and uses a clunky _OBSTACK_INTERFACE_VERSION
> mechanism.

I get the impression it's not a very frequently used part of glibc.

> --- texinfo-7.1/tp/Texinfo/XS/gnulib/lib/obstack.h.bak2023-08-13 
> 22:10:03.0 +0200
> +++ texinfo-7.1/tp/Texinfo/XS/gnulib/lib/obstack.h2023-11-14 
> 20:30:55.584463250 +0100
> @@ -164,6 +164,12 @@
>  # endif
>  #endif
>  
> +#define _obstack_begin rpl_obstack_begin
> +#define _obstack_newchunk rpl_obstack_newchunk
> +#define _obstack_allocated_p rpl_obstack_allocated_p
> +#define _obstack_free rpl_obstack_free
> +#define _obstack_memory_used rpl_obstack_memory_used
> +
>  #ifdef __cplusplus
>  extern "C" {
>  #endif

That's great, I'll apply this patch to the release branch and revert
the other change (using obstack_alignment_mask).

Re: c32width gives incorrect return values in C locale

[Gavin Smith]

On Wed, Nov 15, 2023 at 09:42:21AM +0100, Patrice Dumas wrote:
> On Wed, Nov 15, 2023 at 12:22:24AM -0800, Paul Eggert wrote:
> > On 2023-11-13 01:28, Patrice Dumas wrote:
> > > According to your mail
> > > https://lists.gnu.org/archive/html/bug-libunistring/2023-11/msg0.html
> > > 
> > > char32_t is less portable
> > 
> > That should be OK, if Gnulib provides a char32_t substitute that works well
> > enough. The mail you refer to merely says that literals like U'x' don't
> > work, but this is not a show-stopper for char32_t.
> 
> Indeed, the solaris10 automatic build now compiles ok with char32_t with
> Gnulib uchar after the changes Gavin made.

Is this the OpenCSW buildbot?  How are you checking this?

Everytime I have checked

https://buildfarm.opencsw.org/buildbot/waterfall?tag=texinfo

recently, I have a page with a bunch of error messages on it:


web.Server Traceback (most recent call last):
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: 
: (OperationalError) unable to open 
database file None None /opt/csw/lib/python2.7/threading.py:774:__bootstrap 
/opt/csw/lib/python2.7/threading.py:801:__bootstrap_inner 
/opt/csw/lib/python2.7/threading.py:754:run ---  --- 
/opt/csw/lib/python2.7/site-packages/twisted/python/threadpool.py:191:_worker 
/opt/csw/lib/python2.7/site-packages/twisted/python/context.py:118:callWithContext
 
/opt/csw/lib/python2.7/site-packages/twisted/python/context.py:81:callWithContext
 /opt/csw/lib/python2.7/site-packages/buildbot/db/pool.py:185:__thd 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/threadlocal.py:61:contextual_connect
 /opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:272:connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:431:__init__ 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:867:_do_get 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:225:_create_connection 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:318:__init__ 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:379:__connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py:80:connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/default.py:283:connect ]]
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: 
: (OperationalError) unable to open 
database file None None /opt/csw/lib/python2.7/threading.py:774:__bootstrap 
/opt/csw/lib/python2.7/threading.py:801:__bootstrap_inner 
/opt/csw/lib/python2.7/threading.py:754:run ---  --- 
/opt/csw/lib/python2.7/site-packages/twisted/python/threadpool.py:191:_worker 
/opt/csw/lib/python2.7/site-packages/twisted/python/context.py:118:callWithContext
 
/opt/csw/lib/python2.7/site-packages/twisted/python/context.py:81:callWithContext
 /opt/csw/lib/python2.7/site-packages/buildbot/db/pool.py:185:__thd 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/threadlocal.py:61:contextual_connect
 /opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:272:connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:431:__init__ 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:867:_do_get 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:225:_create_connection 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:318:__init__ 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/pool.py:379:__connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py:80:connect 
/opt/csw/lib/python2.7/site-packages/sqlalchemy/engine/default.py:283:connect ]]

gperf dependency not documented

[Gavin Smith]

The DEPENDENCIES file in gnulib states a module that depends on the gperf
program:

* GNU gperf 3.0.1 or newer.
  + 3.0.1 or newer is mandatory, but 3.1 or newer is recommended.
Needed if you use the 'iconv_open' module.

However, it is not the only module that uses this program.  gperf is
also listed as a dependency of the following modules:

unicase/locale-language
unicase/special-casing
uninorm/composition
unictype/joiningtype-byname
unictype/category-byname
unictype/combining-class-byname
unictype/property-byname
unictype/joininggroup-byname
unictype/incb-byname
unictype/bidiclass-byname
unictype/scripts

Should these be added to DEPENDENCIES?

Variant wcwidth results on Solaris

[Gavin Smith]

I wondered if there has been any change since this report:

https://lists.gnu.org/archive/html/bug-gnulib/2008-08/msg00209.html

The issue is that on Solaris 10 and Solaris 11, wcwidth returns 2 
instead of 1 for some characters, for example, opening double quote “.

Some of these characters are listed as "ambiguous" under 
http://unicode.org/reports/tr11-2/, but having different results on 
different systems makes a test suite almost useless on at least one of 
the systems.

Would it be possible for gnulib to override the system wcwidth if it has
unusual return values?

wcwidth is used by the texi2any program in Texinfo to output to 
monospaced plain text for use in a terminal emulator.  It is used for 
wrapping lines.

Re: Variant wcwidth results on Solaris

[Gavin Smith]

Thank you for your detailed response.

> Different systems have different terminal emulators, and accordingly 
> also different wcwidth functions.

Without knowing anything about the terminal emulators in question, it 
seems to me likely that there would be several terminal emulators with 
different behaviour on the same system, all using the same libc.

> > Would it be possible for gnulib to override the system wcwidth if it has
> > unusual return values?
> 
> When you use a gnulib module that overrides a system function, you
> can influence its behaviour by setting the cache variable before gl_INIT.
> In this case, if you set
>   gl_cv_func_wcwidth_works=no
> you will get the gnulib override on all platforms.
> 
> Or you can modify the test program locally, for your package. [2]

This may be the best approach.

> > wcwidth is used by the texi2any program in Texinfo to output to 
> > monospaced plain text for use in a terminal emulator.  It is used for 
> > wrapping lines.
> 
> Ah, but wcwidth is a poor algorithm for line-breaking. The modern
> algorithms are specified in Unicode, and you have an implementation
> of the essential part of this algorithm in gnulib and in libunistring.

The way that texi2any does it is probably wrong for some languages.
Using the proper algorithms would be a larger change, so at the moment I 
just want to fix the line length issue.

> In gnulib it is in the modules
>   unilbrk/ulc-width-linebreaks
>   unilbrk/u8-width-linebreaks
>   unilbrk/u16-width-linebreaks
>   unilbrk/u32-width-linebreaks
> Pick the one that fits your needs.

I don't know how anybody is supposed to find out about these modules.
They are not documented anywhere in gnulib.texi.  Is one just supposed 
to look through the files under lib/ and modules/ in the gnulib 
repository?

I looked through the files in the gnulib repository and eventually found 
the lib/unilbrk.in.h file documenting the functions that are provided.  
It says in it "assuming the uc_width" function.  This seems to perform 
the same function as wcwidth.  If we could replace wcwidth with uc_width 
and uc_width from gnulib (or the same function from libunistring) and 
that was consistent across platforms, then the issue would be solved.

I don't see any reason why uc_width from gnulib/libunistring should be 
consistent across platforms and the results from the wcwidth function 
when the wcwidth module is in use should not be.  If the system wcwidth 
was worth respecting shouldn't uc_width use its values?

It appears that lib/wcwidth.c does actually use uc_width.  So the best 
solution might be to carry on using the wcwidth function in the source 
code for texi2any (instead of changing this to uc_width), and change the 
configure-time checks to make sure that the replacement function is used 
when the values from the system wcwidth are unusual.

> Another approach is to not include the code from gnulib, but instead
> rely on a preinstalled GNU libunistring. For this, use the gnulib
> module 'libunistring'.

libunistring looks like a very useful library, and I was not aware of 
it.  There is no reference to this library in the gnulib manual or the 
glibc manual.  Should a note be added to 
doc/posix-functions/wcwidth.texi saying that libunistring should be used 
if you want to get consistent results across systems?

Re: Variant wcwidth results on Solaris

[Gavin Smith]

On Sat, Mar 30, 2019 at 12:33:32PM +0100, Bruno Haible wrote:
> > It appears that lib/wcwidth.c does actually use uc_width.  So the best 
> > solution might be to carry on using the wcwidth function in the source 
> > code for texi2any (instead of changing this to uc_width), and change the 
> > configure-time checks to make sure that the replacement function is used 
> > when the values from the system wcwidth are unusual.
> 
> This is one way to solve your immediate problem, yes.
> 
> > Should a note be added to 
> > doc/posix-functions/wcwidth.texi saying that libunistring should be used 
> > if you want to get consistent results across systems?
> 
> wcwidth is a POSIX function. The idea of wanting to get the same result
> across platforms, for a POSIX function, is a too strong wish. POSIX allows
> different implementations. Gnulib does overrides only when
>   - the platform's implementation is clearly buggy, or
>   - the glibc implementation is vastly superior in all aspects, or
>   - the differences between platforms are unacceptable.
> 

What I've done is change the test case to avoid the line breaking issue.  
This avoids the need for any special check for this issue, while keeping 
the test suite useful on Solaris.  The system wcwidth will still be used 
on Solaris, so differing results are still possible, just no cases that 
lead to this are present in the test suite.

Re: autoconf-2.69c released [beta]

[Gavin Smith]

On Sat, Sep 26, 2020 at 07:20:40PM -0700, Paul Eggert wrote:
> On 9/26/20 10:47 AM, Zack Weinberg wrote:
> 
> > > Would it be right to say that this should be changed in Gnulib,
> > > otherwise any project using that file with Gnulib will have errors
> > > from the new autoconf?
> > 
> > Yes, indeed.  All of the Gnulib and Autoconf Macro Archive macros need
> > to be checked for problems like this.  I'm cc:ing gnulib-bugs.
> 
> Thanks for reporting that. I installed the attached patch into Gnulib; it
> works for me on simple tests (I did reproduce the bug).
> 
> Gavin, could you please give the latest Gnulib a try?

Thank you, that does resolve that warning.

There are also warnings about AC_PROG_CC_STDC, AC_HELP_STRING, 
AC_HEADER_STDC, and $as_echo_n from other Gnulib files:

%

AC_PROG_CC_STDC from stdarg.m4 (via gl_PROG_CC_C99 in gnulib-common.m4):

# gl_PROG_CC_C99
# Modifies the value of the shell variable CC in an attempt to make $CC
# understand ISO C99 source code.
# This is like AC_PROG_CC_C99, except that
# - AC_PROG_CC_C99 does not mix well with AC_PROG_CC_STDC
#   <https://lists.gnu.org/r/bug-gnulib/2011-09/msg00367.html>,
#   but many more packages use AC_PROG_CC_STDC than AC_PROG_CC_C99
#   <https://lists.gnu.org/r/bug-gnulib/2011-09/msg00441.html>.
# Remaining problems:
# - When AC_PROG_CC_STDC is invoked twice, it adds the C99 enabling options
#   to CC twice
#   <https://lists.gnu.org/r/bug-gnulib/2011-09/msg00431.html>.
# - AC_PROG_CC_STDC is likely to change now that C11 is an ISO standard.
AC_DEFUN([gl_PROG_CC_C99],
[
  dnl Change that version number to the minimum Autoconf version that supports
  dnl mixing AC_PROG_CC_C99 calls with AC_PROG_CC_STDC calls.
  m4_version_prereq([9.0],
[AC_REQUIRE([AC_PROG_CC_C99])],
[AC_REQUIRE([AC_PROG_CC_STDC])])
])

I'll leave it to someone else to try to fix this.

%

AC_HELP_STRING from threadlib.m4.  Also used in libgcrypt.m4.  Running 
autoupdate on those files and editing the result gives the patch I've 
attached.

%

$as_echo_n is used in gnulib-common.m4 around a call to AC_CACHE_VAL:

# gl_CACHE_VAL_SILENT(cache-id, command-to-set-it)
# is like AC_CACHE_VAL(cache-id, command-to-set-it), except that it does not
# output a spurious "(cached)" mark in the midst of other configure output.
# This macro should be used instead of AC_CACHE_VAL when it is not surrounded
# by an AC_MSG_CHECKING/AC_MSG_RESULT pair.
AC_DEFUN([gl_CACHE_VAL_SILENT],
[
  saved_as_echo_n="$as_echo_n"
  as_echo_n=':'
  AC_CACHE_VAL([$1], [$2])
  as_echo_n="$saved_as_echo_n"
])

AC_CACHE_VAL calls _AS_ECHO_N which is defined in m4sh.sh from autoconf.  
The as_echo_n shell variable is not referred to at all, so this 
redefinition appears to be ineffectual.  gl_CACHE_VAL_SILENT is not used 
from gnulib-common.m4 itself but is used from a handful of other 
modules.  It is used in the stdarg module, but the issue of spurious 
output only arises if the "va_copy" function was not found.
The other files using gl_CACHE_VAL_SILENT were

floorf.m4, floorl.m4, floor.m4, ceilf.m4, ceilr.m4, ceil.m4

but I haven't tested whether these can produce spurious output.

%

I am going to try to run autoupdate on the other gnulib m4 files and see 
what happens.
commit f077b04f4f694059180a37d1720685b2a1bac544
Author: Gavin Smith 
Date:   Sun Sep 27 10:20:06 2020 +0100

    AS_HELP_STRING

diff --git a/ChangeLog b/ChangeLog
index 8c06171aa..c77f19a68 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2020-09-27  Gavin Smith  
+
+	threadlib, libcrypt: use AS_HELP_MESSAGE instead of AC_HELP_MESSAGE.
+	* m4/threadlib.m4, m4/libgcrypt.m4: Update macro use.
+
 2020-09-26  Paul Eggert  
 
 	extensions: require AC_GNU_SOURCE only for <=2.63
diff --git a/m4/libgcrypt.m4 b/m4/libgcrypt.m4
index f56cc1bde..80947fbb6 100644
--- a/m4/libgcrypt.m4
+++ b/m4/libgcrypt.m4
@@ -1,5 +1,6 @@
 # libgcrypt.m4 - Autoconf macros to detect libgcrypt
 # Copyright (C) 2002, 2003, 2004, 2011, 2014, 2018 g10 Code GmbH
+# Copyright (C) 2020 Free Software Foundation
 #
 # This file is free software; as a special exception the author gives
 # unlimited permission to copy and/or distribute it, with or without
@@ -9,7 +10,6 @@
 # WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
 # implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 #
-# Last-changed: 2018-11-13
 
 
 dnl AM_PATH_LIBGCRYPT([MINIMUM-VERSION,
@@ -30,7 +30,7 @@ dnl
 AC_DEFUN([AM_PATH_LIBGCRYPT],
 [ AC_REQUIRE([AC_CANONICAL_HOST])
   AC_ARG_WITH(libgcrypt-prefix,
-AC_HELP_STRING([--with-libgcrypt-prefix=PFX],
+AS_HELP_STRING([--with-libgcrypt-prefix=PFX],
[prefix where LIBGCRYPT is installed (optional)]),
  libgcrypt_config_prefix="$withval", libgcrypt_config_prefix="")
   if test x"

Re: autoconf-2.69c released [beta]

[Gavin Smith]

> -AC_PREREQ([2.60])
> +AC_PREREQ([2.69])

I left this in by mistake from autoupdate; I'd guess it should be left 
as 2.60.

Re: autoconf-2.69c released [beta]

[Gavin Smith]

> diff --git a/ChangeLog b/ChangeLog
> index 8c06171aa..c77f19a68 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,8 @@
> +2020-09-27  Gavin Smith  
> +
> + threadlib, libcrypt: use AS_HELP_MESSAGE instead of AC_HELP_MESSAGE.
> + * m4/threadlib.m4, m4/libgcrypt.m4: Update macro use.
> +

Sorry, this should have been AS_HELP_MESSAGE and AC_HELP_MESSAGE.

Results from running autoupdate on gnulib m4 files

[Gavin Smith]

Because of the upcoming autoconf 2.70 release gives more warnings, I 
checked the other files in gnulib in the m4 directory with

autoupdate -v *.m4 2>&1 | tee log

Fortunately, there are not too many issues (on top of the other issues I 
already reported).  Here are the relevant excerpts from the output:

-
gnulib-common.m4:486: warning: AC_PROG_CC_C99 is obsolete; use AC_PROG_CC
gnulib-common.m4:487: warning: AC_PROG_CC_STDC is obsolete; use AC_PROG_CC
autoupdate: 'gnulib-common.m4' is updated

realloc.m4:11: warning: The preprocessor macro `STDC_HEADERS' is obsolete.
  Except in unusual embedded environments, you can safely include all
  ISO C90 headers unconditionally.
autoupdate: 'realloc.m4' is updated

std-gnu11.m4:457: warning: AC_PROG_CC_C89 is obsolete; use AC_PROG_CC
std-gnu11.m4:459: warning: AC_PROG_CC_C99 is obsolete; use AC_PROG_CC
std-gnu11.m4:459: warning: AC_PROG_CC_STDC is obsolete; use AC_PROG_CC
autoupdate: 'std-gnu11.m4' is updated
-

I went through the changes that autoupdate made and reverted what seemed to be
incorrect changes (e.g. AC_PREREQ version numbers).  The remaining 
changes are almost certainly incorrect but it shows where other problems 
are likely to arise:

* Use of AC_DECL_SYS_SIGLIST in strsignal.m4
* Redefinition of AC_CHECKING in several files to silence AC_CHECK_PROG
* AC_HEADER_STDC in realloc.m4 and malloc.m4

I've attached the diff to this email.

diff --git a/m4/csharpcomp.m4 b/m4/csharpcomp.m4
index 77819d683..dcab1b754 100644
--- a/m4/csharpcomp.m4
+++ b/m4/csharpcomp.m4
@@ -15,12 +15,12 @@ AC_DEFUN([gt_CSHARPCOMP],
   AC_MSG_CHECKING([for C[#] compiler])
   HAVE_CSHARPCOMP=1
   pushdef([AC_MSG_CHECKING],[:])dnl
-  pushdef([AC_CHECKING],[:])dnl
+  pushdef([AS_MESSAGE([checking ...])],[:])dnl
   pushdef([AC_MSG_RESULT],[:])dnl
   AC_CHECK_PROG([HAVE_MCS_IN_PATH], [mcs], [yes])
   AC_CHECK_PROG([HAVE_CSC_IN_PATH], [csc], [yes])
   popdef([AC_MSG_RESULT])dnl
-  popdef([AC_CHECKING])dnl
+  popdef([AS_MESSAGE([checking ...])])dnl
   popdef([AC_MSG_CHECKING])dnl
   for impl in "$CSHARP_CHOICE" mono sscli no; do
 case "$impl" in
diff --git a/m4/csharpexec.m4 b/m4/csharpexec.m4
index 5158a1a58..4269e9232 100644
--- a/m4/csharpexec.m4
+++ b/m4/csharpexec.m4
@@ -21,12 +21,12 @@ AC_DEFUN([gt_CSHARPEXEC],
 ], MONO_PATH_SEPARATOR=';', MONO_PATH_SEPARATOR=':')
   HAVE_CSHARPEXEC=1
   pushdef([AC_MSG_CHECKING],[:])dnl
-  pushdef([AC_CHECKING],[:])dnl
+  pushdef([AS_MESSAGE([checking ...])],[:])dnl
   pushdef([AC_MSG_RESULT],[:])dnl
   AC_CHECK_PROG([HAVE_MONO_IN_PATH], [mono], [yes])
   AC_CHECK_PROG([HAVE_CLIX_IN_PATH], [clix], [yes])
   popdef([AC_MSG_RESULT])dnl
-  popdef([AC_CHECKING])dnl
+  popdef([AS_MESSAGE([checking ...])])dnl
   popdef([AC_MSG_CHECKING])dnl
   for impl in "$CSHARP_CHOICE" mono no; do
 case "$impl" in
diff --git a/m4/exponentd.m4 b/m4/exponentd.m4
index cca879782..457af3f94 100644
--- a/m4/exponentd.m4
+++ b/m4/exponentd.m4
@@ -91,12 +91,14 @@ int main ()
 [
   pushdef([AC_MSG_CHECKING],[:])dnl
   pushdef([AC_MSG_RESULT],[:])dnl
-  pushdef([AC_MSG_RESULT_UNQUOTED],[:])dnl
+  pushdef([_AS_ECHO_UNQUOTED([$as_me:${as_lineno-$LINENO}: result: 
],[AS_MESSAGE_LOG_FD])
+_AS_ECHO_UNQUOTED([])],[:])dnl
   AC_C_BIGENDIAN(
 [gl_cv_cc_double_expbit0="word 0 bit 20"],
 [gl_cv_cc_double_expbit0="word 1 bit 20"],
 [gl_cv_cc_double_expbit0="unknown"])
-  popdef([AC_MSG_RESULT_UNQUOTED])dnl
+  popdef([_AS_ECHO_UNQUOTED([$as_me:${as_lineno-$LINENO}: result: 
],[AS_MESSAGE_LOG_FD])
+_AS_ECHO_UNQUOTED([])])dnl
   popdef([AC_MSG_RESULT])dnl
   popdef([AC_MSG_CHECKING])dnl
 ])
diff --git a/m4/gethostname.m4 b/m4/gethostname.m4
index 1d05b2a60..1d5c0d450 100644
--- a/m4/gethostname.m4
+++ b/m4/gethostname.m4
@@ -74,7 +74,7 @@ lucky
 dnl mingw.
 gl_cv_decl_HOST_NAME_MAX=256
   else
-_AC_COMPUTE_INT([MAXHOSTNAMELEN], [gl_cv_decl_HOST_NAME_MAX], [
+AC_COMPUTE_INT([gl_cv_decl_HOST_NAME_MAX],[MAXHOSTNAMELEN],[
 #include 
 #if HAVE_SYS_PARAM_H
 # include 
@@ -85,8 +85,7 @@ lucky
 #if HAVE_NETDB_H
 # include 
 #endif
-],
-  [dnl The system does not define MAXHOSTNAMELEN in any of the common
+],[dnl The system does not define MAXHOSTNAMELEN in any of the common
dnl headers. Use a safe fallback.
gl_cv_decl_HOST_NAME_MAX=256
   ])
diff --git a/m4/javacomp.m4 b/m4/javacomp.m4
index b5eea0795..9c8ecc1f4 100644
--- a/m4/javacomp.m4
+++ b/m4/javacomp.m4
@@ -495,13 +495,13 @@ changequote([,])dnl
   fi
   if test -z "$HAVE_JAVACOMP"; then
 pushdef([AC_MSG_CHECKING],[:])dnl
-pushdef([AC_CHECKING],[:])dnl
+pushdef([AS_MESSAGE([checking ...])],[:])dnl
 pushdef([AC_MSG_RESULT],[:])dnl
 AC_CHECK_PROG([HAVE_GCJ_IN_PATH], [gcj], [yes])
 AC_CHECK_PROG([HAVE_JAVAC_IN_PATH], [javac], [yes])
 AC_CHECK_P