Re: gnulib-tool sh+py mismatch when updating Emacs

[Paul Eggert]

On 2024-05-04 16:03, Collin Funk wrote:


I noticed in your emacs/output.0 it says "Generated by GNU Autoconf 2.71".

I was using Autoconf 2.72 on my system.


Ah, I am using Autoconf 2.71, which is what's in /usr/bin/autoconf on 
Fedora 40. It comes from the autoconf-2.71-10.fc40.noarch package, which 
is the current version for Fedora 40.


Occasionlly I use bleeding-edge autoconf (even past 2.72) but that's 
only to test Autoconf, not other packages.




-diff -r $diff_options --exclude=__pycache__ -q . "$tmp" >/dev/null ||
+diff -r $diff_options --exclude=__pycache__ --exclude=autom4te.cache -q . 
"$tmp" >/dev/null ||


Looks good to me, thanks. (I didn't bother testing it on my old slow 
machine.)

Re: gnulib-tool.py: Fix an undefined function name.

[Collin Funk]

On 5/4/24 8:17 AM, Bruno Haible wrote:
>> For CI, I am thinking it is best to ignore the typing warnings for the
>> time being. Since gnulib-tool.py needs a bit more refactoring for that
>> to be helpful. It was written before typing was a thing. :)
> 
> Hmm. Is that refactoring, that you are imagining, intrusive? If it's
> intrusive, then maybe we should leave it alone. You already introduced
> types in the function signatures and in the fields of classes; what else
> is missing?

Depends on the specific warning being addressed. :)

There are more trivial cases like GLMakeFileTable which stores
information as a dictionary. We have a table of makefile variables:

 table: list[dict[str, str | bool]]
 # Each 'dict' has the following 'key': type.
 { 'dir': str, 'var': str, 'dotfirst': bool }

Using a value from a key at this will lead to a type error unless you
"narrow" the union [1]:

   if type(makefiletable['dir']) is str:
  makefiletable.split('/') # Would not work on 'bool'.

Me and you can tell the use of makefiletable['dir'] will return a str,
but the type checker can't. Using a class like this:

class GLMakeVar:
dir: str
var: str
dotfirst: bool

Would be type-checkable and would probably be easier for new
contributors to understand, in my opinion.

Other cases are less trivial and bring up situations which may cause
errors. For example there are many warnings about "list[GLModule]"
actually being "list[GLModule | None]".

This is caused by functions like GLModule.getDependenciesWithoutConditions():

 def getDependenciesWithoutConditions(self) -> list[GLModule | None]:
 # Read the "Depends-on" and find each module name.
 # Run self.modulesystem.find() to get the GLModule objects.
 # Return the list of dependencies.

Remember that modulesystem.find() returns None with a warning (or
error depending on arguments) if the module cannot be found. Therefore,
this function should probably filter out None in the return value.

>> I'm not too familiar with the GitLab CI stuff. Would a Makefile in
>> gnulib/pygnulib with some checks work?
> 
> Yes, a Makefile in pygnulib/ would be sufficient. The top-level Makefile then
> may invoke it.

Sounds good. I'll have a look at getting started with something in a bit.

[1] 
https://mypy.readthedocs.io/en/stable/type_narrowing.html#type-narrowing-expressions

Collin

Re: gnulib-tool sh+py mismatch when updating Emacs

[Collin Funk]

Hi Paul,

On 5/4/24 3:02 PM, Paul Eggert wrote:
> On 2024-05-04 13:45, Collin Funk wrote:
>> I can't reproduce this (using Fedora 40).
> 
> That's odd, as I just now reproduced it on Fedora 40 x86-64 using the 
> following from-scratch recipe:

I noticed in your emacs/output.0 it says "Generated by GNU Autoconf 2.71".

I was using Autoconf 2.72 on my system. Using my Emacs directory and
Autoconf ./configure'd from the v2.71 tag:

$ LC_ALL=en_US.utf8 GNULIB_TOOL_IMPL=sh+py admin/merge-gnulib
../gnulib/gnulib-tool: *** gnulib-tool.py produced different files than 
gnulib-tool.sh! Compare /home/collin/.local/src/emacs and 
/home/collin/.local/src/glpyR1z7xq.
../gnulib/gnulib-tool: *** Stop.
$ diff -ru /home/collin/.local/src/emacs /home/collin/.local/src/glpyR1z7xq
diff -ru /home/collin/.local/src/emacs/autom4te.cache/requests 
/home/collin/.local/src/glpyR1z7xq/autom4te.cache/requests
--- /home/collin/.local/src/emacs/autom4te.cache/requests   2024-05-04 
15:56:55.496035451 -0700
+++ /home/collin/.local/src/glpyR1z7xq/autom4te.cache/requests  
2024-05-04 15:56:25.973984808 -0700
  @@ -15,69 +15,69 @@
   'configure.ac'
 ],
 {
  -'AM_PROG_CC_C_O' => 1,
  -'AM_PROG_AR' => 1,
  -'AC_CANONICAL_BUILD' => 1,

I'm assuming sometime between 2.71 and 2.72 that file started getting
sorted when it previously didn't?

I think this change should be the proper fix:

diff --git a/gnulib-tool b/gnulib-tool
index 56c4473318..789fe916a8 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -209,7 +209,7 @@ case "$GNULIB_TOOL_IMPL" in
 else
   diff_options=
 fi
-diff -r $diff_options --exclude=__pycache__ -q . "$tmp" >/dev/null ||
+diff -r $diff_options --exclude=__pycache__ --exclude=autom4te.cache 
-q . "$tmp" >/dev/null ||
   func_fatal_error "gnulib-tool.py produced different files than 
gnulib-tool.sh! Compare `pwd` and $tmp."
 # Compare the two outputs.
 diff -q "$tmp-sh-out" "$tmp-py-out" >/dev/null ||

Can you confirm before I apply it?

Collin

Re: gnulib-tool sh+py mismatch when updating Emacs

[Collin Funk]

On 5/4/24 3:02 PM, Paul Eggert wrote:
> The attached build log shows what I got. The last 'grep' command shows the 
> bug, as lib/gnulib.mk.in uses HAVE_OFF64_T without defining it.
> 
> This invocation of admin/merge-gnulib uses a nearly-empty environment and an 
> empty home directory, to lessen any local changes I might have.
> 
> If it helps to debug, you can get a tarball of all the resulting Emacs 
> directory (sans .git subdirectory) temporarily, here:

Thanks for the details!

I'll have a look at it. I downloaded the tarball so no worries here if
you remove it.

Collin

Re: header file substitutes

[Collin Funk]

On 5/4/24 2:30 PM, Paul Eggert wrote:
>> But, I think the next POSIX revision has  like Glibc which I
>> prefer.
> 
> If the past is any guide, an advantage of the C23 version is that in
> the long run it is likely to be more portable. POSIX is an extension
> of C.

True. But I don't think C23 has the conversion macros:

/* big endian 32 to host.  */
uint32_t be32toh (uint32_t);
/* little endian 32 to host.  */
uint32_t le32toh (uint32_t);

like the next POSIX revision has and I am a fan of.

I assume the addition of  was inspired by glibc's
implementation. glibc seems to have extended  from the
BSDs by adding those conversion macros.

Since  seems resonably portable, that could do the bulk
of the work for non glibc systems.

GCC even has these:

$ echo | gcc -dM -E - | grep 'ENDIAN'
#define __ORDER_LITTLE_ENDIAN__ 1234
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __ORDER_PDP_ENDIAN__ 3412
#define __ORDER_BIG_ENDIAN__ 4321
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__

Collin

Re: gnulib-tool sh+py mismatch when updating Emacs

[Paul Eggert]

On 2024-05-04 13:45, Collin Funk wrote:

I can't reproduce this (using Fedora 40).


That's odd, as I just now reproduced it on Fedora 40 x86-64 using the 
following from-scratch recipe:


  mkdir new empty 

  empty_home=$PWD/empty 

  cd new 

  git clone git://git.sv.gnu.org/emacs.git 

  (cd emacs && git checkout fd859fbea2e9d13e76db1c5295d9ddd1c5955d83) 

  git clone git://git.sv.gnu.org/gnulib.git 

  (cd gnulib && git checkout fde88b711c9b1df5b142444ac7b0bc2aa8892d3a) 

  cd emacs 

  env -i HOME="$empty_home" PATH=/usr/bin admin/merge-gnulib 



The attached build log shows what I got. The last 'grep' command shows 
the bug, as lib/gnulib.mk.in uses HAVE_OFF64_T without defining it.


This invocation of admin/merge-gnulib uses a nearly-empty environment 
and an empty home directory, to lessen any local changes I might have.


If it helps to debug, you can get a tarball of all the resulting Emacs 
directory (sans .git subdirectory) temporarily, here:


https://www.cs.ucla.edu/~eggert/emacs.tgz$ mkdir new empty
$ empty_home=$PWD/empty
$ cd new
$ git clone git://git.sv.gnu.org/emacs.git
Cloning into 'emacs'...
remote: Counting objects: 1102975, done.
remote: Compressing objects: 100% (198635/198635), done.
remote: Total 1102975 (delta 903465), reused 1098398 (delta 899028)
Receiving objects: 100% (1102975/1102975), 413.08 MiB | 21.93 MiB/s, done.
Resolving deltas: 100% (903465/903465), done.
Updating files: 100% (5266/5266), done.
$ (cd emacs && git checkout fd859fbea2e9d13e76db1c5295d9ddd1c5955d83)
Note: switching to 'fd859fbea2e9d13e76db1c5295d9ddd1c5955d83'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c 

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at fd859fbea2e Allow `letrec` binding without init expression
$ git clone git://git.sv.gnu.org/gnulib.git
Cloning into 'gnulib'...
remote: Counting objects: 295108, done.
remote: Compressing objects: 100% (35985/35985), done.
remote: Total 295108 (delta 261344), reused 292549 (delta 259017)
Receiving objects: 100% (295108/295108), 74.69 MiB | 11.59 MiB/s, done.
Resolving deltas: 100% (261344/261344), done.
$ (cd gnulib && git checkout fde88b711c9b1df5b142444ac7b0bc2aa8892d3a)
Note: switching to 'fde88b711c9b1df5b142444ac7b0bc2aa8892d3a'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c 

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at fde88b711c readutmp, boot-time: Work around a Cygwin 3.5.3 bug.
$ cd emacs
$ env -i HOME="$empty_home" PATH=/usr/bin admin/merge-gnulib
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65) ... ok
Your system has the required tools.
Building aclocal.m4 ...
Running 'autoreconf -fi -I m4' ...
Building 'aclocal.m4' in exec ...
Running 'autoreconf -fi' in exec ...
Configuring local git repository...
'.git/config' -> '.git/config.~1~'
git config transfer.fsckObjects 'true'
git config diff.cpp.xfuncname '!^[ 
\t]*[A-Za-z_][A-Za-z_0-9]*:[[:space:]]*($|/[/*])
^((::[[:space:]]*)?[A-Za-z_][A-Za-z_0-9]*[[:space:]]*\(.*)$
^((#define[[:space:]]|DEFUN).*)$'
git config diff.elisp.xfuncname 
'^\([^[:space:]]*def[^[:space:]]+[[:space:]]+([^()[:space:]]+)'
git config diff.m4.xfuncname '^((m4_)?define|A._DEFUN(_ONCE)?)\([^),]*'
git config diff.make.xfuncname 
'^([$.[:alnum:]_].*:|[[:alnum:]_]+[[:space:]]*([*:+]?[:?]?|!?)=|define .*)'
git config diff.shell.xfuncname 
'^([[:space:]]*[[:alpha:]_][[:alnum:]_]*[[:space:]]*\(\)|[[:alpha:]_][[:alnum:]_]*=)'
git config diff.texinfo.xfuncname '^@node[[:space:]]+([^,[:space:]][^,]+)'
Installing git hooks...
'build-aux/git-hooks/commit-msg' -> '.git/hooks/commit-msg'
'build-aux/git-hooks/pre-commit' -> '.git/hooks/pre-commit'
'build-aux/git-hooks/prepare-commit-msg' -> '.git/hooks/prepare-commit-msg'
'build-aux/git-hooks/post-commit' -> '.git/hooks/post-commit'
'build-aux/git-hooks/pre-push' -> '.git/hooks/pre-push'
'build-aux/git-hooks/commit-msg-files.awk' -> '.git/hooks/commit-msg-files.awk'
'.git/hooks/applypatch-msg.sample' -> '.git/hooks/applypatch-msg'
'.git/hooks/pre-applypatch.sample' -> '.git/hooks/pre-applypatch'
You can now run './configure'.
Module list with included 

Re: header file substitutes

[Paul Eggert]

On 2024-05-04 13:54, Collin Funk wrote:

IIRC in  there is:

  #define __STDC_ENDIAN_LITTLE__ /* Unique constant */
  #define __STDC_ENDIAN_BIG__ /* Unique constant */
  #define __STDC_ENDIAN_NATIVE__ /* __STDC_ENDIAN_LITTLE__ or 
__STDC_ENDIAN_BIG__ */

probably with values taken from Glibc's  or BSD's
.

But, I think the next POSIX revision has  like Glibc which I
prefer.


If the past is any guide, an advantage of the C23 version is that in the 
long run it is likely to be more portable. POSIX is an extension of C.

Re: header file substitutes

[Collin Funk]

Hi Bruno,

On 5/4/24 7:52 AM, Bruno Haible wrote:
> And we're not yet done with the header file substitutes. One is still missing
> from Gnulib:  [1]. The only thing that has changed since last year
> is that glibc 2.39 now has this header file [2]. But who will want to use it,
> as long as it's not portable?

C23 and the upcoming POSIX release seem to have a lot of useful stuff.

IIRC in  there is:

 #define __STDC_ENDIAN_LITTLE__ /* Unique constant */
 #define __STDC_ENDIAN_BIG__ /* Unique constant */
 #define __STDC_ENDIAN_NATIVE__ /* __STDC_ENDIAN_LITTLE__ or 
__STDC_ENDIAN_BIG__ */

probably with values taken from Glibc's  or BSD's
.

But, I think the next POSIX revision has  like Glibc which I
prefer. So these defines:

#define LITTLE_ENDIAN /* Unique constant */
#define BIG_ENDIAN /* Unique constant */
#define BYTE_ORDER /* LITTLE_ENDIAN or BIG_ENDIAN */

plus functions or macros:

 uint16_t be16toh (uint16_t);
 uint16_t htobe16 (uint16_t);

I could try to work on that if it seems useful to anyone else.

Collin

Re: gnulib-tool sh+py mismatch when updating Emacs

[Collin Funk]

Hi Paul,

On 5/4/24 9:03 AM, Paul Eggert wrote:
> Some sort of locale problem? The issue seems to be benign, except that it's a 
> false positive when testing.
> 
> To reproduce it, clone current gnulib (commit 
> fde88b711c9b1df5b142444ac7b0bc2aa8892d3a) and current emacs (commit 
> fd859fbea2e9d13e76db1c5295d9ddd1c5955d83) under the current directory, and 
> then run:

I can't reproduce this (using Fedora 40).

For what it's worth I used Emacs admin/merge-gnulib frequently when I
started working on gnulib-tool.py. Bruno then added it as a test case
in maint-tools. So I can almost guarantee it is safe to use there.

Could the autogen.sh script be messing it up perhaps? Or old build
artifacts? My Emacs directory was clean during run.

Collin

Re: gnulib-tool problem with off64_t and Emacs

[Paul Eggert]

On 2024-05-04 10:43, Bruno Haible wrote:

but there was no "HAVE_OFF64_T = @HAVE_OFF64_T@" line

Strange. The AC_SUBST([HAVE_OFF64_T]) in m4/off64_t.m4 should produce
such a line.


My thought as well.



The fragile hackery is only needed because you've hit a bug


Yes. To be fair it could well be an Emacs bug. Whatever it is, it's 
currently a hassle.




Do you see the 'sys_types' module enabled? Or maybe it is only
conditionally enabled?


Yes, there is a sys_types section in lib/gnulib.mk.in, and yes, it is 
conditionally enabled. It's preceded by this:


  ## begin gnulib module sys_types
  ifeq (,$(OMIT_GNULIB_MODULE_sys_types))

The ifeq succeeds and sys/types.h is being built.

I can reproduce the problem in fresh emacs and gnulib checkout siblings 
(same commits as before) by running the following in the emacs 
directory. This invokes gnulib-tool directly instead of indirectly via 
admin/merge-gnulib:


  ../gnulib/gnulib-tool --dir= --conditional-dependencies --import 
--no-changelog --no-vc-files --gnu-make --makefile-name=gnulib.mk.in 
--avoid=access --avoid=btowc --avoid=chmod --avoid=close 
--avoid=crypto/af_alg --avoid=dup --avoid=fchdir --avoid=fstat 
--avoid=iswblank --avoid=iswctype --avoid=iswdigit --avoid=iswxdigit 
--avoid=langinfo --avoid=localename-unsafe-limited --avoid=lock 
--avoid=mbrtowc --avoid=mbsinit --avoid=memchr --avoid=mkdir 
--avoid=msvc-inval --avoid=msvc-nothrow --avoid=nl_langinfo 
--avoid=openat-die --avoid=opendir --avoid=pthread-h --avoid=raise 
--avoid=save-cwd --avoid=select --avoid=setenv --avoid=sigprocmask 
--avoid=stat --avoid=stdarg --avoid=threadlib --avoid=tzset 
--avoid=unsetenv --avoid=utime --avoid=utime-h --avoid=wchar 
--avoid=wcrtomb --avoid=wctype --avoid=wctype-h alignasof alloca-opt 
binary-io boot-time byteswap c-ctype c-strcase canonicalize-lgpl 
careadlinkat close-stream copy-file-range count-leading-zeros 
count-one-bits count-trailing-zeros crypto/md5 crypto/md5-buffer 
crypto/sha1-buffer crypto/sha256-buffer crypto/sha512-buffer d-type 
diffseq double-slash-root dtoastr dtotimespec dup2 environ execinfo 
faccessat fchmodat fcntl fcntl-h fdopendir file-has-acl filemode 
filename filevercmp flexmember fpieee free-posix fstatat fsusage fsync 
futimens getline getloadavg getopt-gnu getrandom gettime gettimeofday 
gitlog-to-changelog ieee754-h ignore-value intprops largefile libgmp 
lstat manywarnings memmem-simple mempcpy memrchr memset_explicit minmax 
mkostemp mktime nanosleep nproc nstrftime pathmax pipe2 pselect 
pthread_sigmask qcopy-acl readlink readlinkat regex sig2str sigdescr_np 
socklen stat-time std-gnu11 stdbool stdckdint stddef stdio stpcpy 
strnlen strnlen strtoimax symlink sys_stat sys_time tempname time-h 
time_r time_rz timegm timer-time timespec-add timespec-sub 
update-copyright unlocked-io utimensat vla warnings year2038

Re: gnulib-tool problem with off64_t and Emacs

[Bruno Haible]

Hi Paul,

> but there was no "HAVE_OFF64_T = @HAVE_OFF64_T@" line

Strange. The AC_SUBST([HAVE_OFF64_T]) in m4/off64_t.m4 should produce
such a line.

Do you see the 'sys_types' module enabled? Or maybe it is only
conditionally enabled?

> It strikes me that apps like Emacs and coreutils don't need (and 
> shouldn't use) off64_t which was merely intended as a transitional type 
> in the 1990s, so there should be some way for them to disable the 
> off64_t business

off64_t is defined on glibc in , , ,
, and since one of Gnulib's principles is to provide a glibc-like
API portably, we now ensure that off64_t is defined in these 4 header files
— regardless whether the future z*printf functions in  will be
requested by the package.

I hope that coreutils will make good use of the z*printf functions once
they exist.

> without the abovementioned sort of fragile hackery.

The fragile hackery is only needed because you've hit a bug, that I haven't
had time to investigate so far.

Bruno

gnulib-tool problem with off64_t and Emacs

[Paul Eggert]

I had problems updating Emacs to use gnulib commit 
fde88b711c9b1df5b142444ac7b0bc2aa8892d3a along with emacs commit 
fd859fbea2e9d13e76db1c5295d9ddd1c5955d83 (these are the same commits as 
I mentioned earlier today). I reproduced it like this:


cd emacs
admin/merge-gnulib

The resulting lib/gnulib.mk.in had a line:

  -e 's|@''HAVE_OFF64_T''@|$(HAVE_OFF64_T)|g' \

but there was no "HAVE_OFF64_T = @HAVE_OFF64_T@" line, and this caused 
the "#if !@HAVE_OFF64_T@" line in lib/sys_types.in.h to become "#if !" 
in lib/sys/types.h which is a syntax error in C.


I plan to work around the problem in Emacs by changing 
admin/merge-gnulib to remove m4/off64_t.m4, and putting the following 
into configure.ac instead:


  # Emacs does not need to check for off64_t. 

  AC_DEFUN([gl_TYPE_OFF64_T], 

[HAVE_OFF64_T=1 


 AC_SUBST([HAVE_OFF64_T])])

This is of course fragile.

It strikes me that apps like Emacs and coreutils don't need (and 
shouldn't use) off64_t which was merely intended as a transitional type 
in the 1990s, so there should be some way for them to disable the 
off64_t business without the abovementioned sort of fragile hackery. A 
documented way to disable off64_t discovery should simplify 
configuration and make it less likely to run into glitches such as the 
one I ran into with Emacs.

Re: [PATCH] getdtablesize: support systems without getrlimit like WASI

[Yuta Saito]

Hi Bruno,

Thank you for taking a look at this.

>   - How do _you_ compile and run a program in this environment?
> The same way as in [2], or do you do it differently?

Yes, I use wasi-sdk[1] as described in the document. My current motivation
is porting
libidn to WebAssembly and I needed to apply this patch to make it work.
To be precise, I use the following configuration to cross-build libidn with
this Gnulib patch.

```
$ ../libidn/configure --host wasm32-wasi --enable-cross-guesses=risky
--disable-doc \
  CFLAGS='-D_WASI_EMULATED_PROCESS_CLOCKS -fPIC'
LDFLAGS='-lwasi-emulated-process-clocks' \
  RANLIB=$WASI_SDK_PATH/bin/llvm-ranlib AR=$WASI_SDK_PATH/bin/llvm-ar
CC=$WASI_SDK_PATH/bin/clang
```

>  - [1] promises that the programs "may run anywhere—from browsers to
clouds
>to embedded devices". How do you run such a program in a browser?
>How do you run it in the "cloud"? How do you run it on a memory-
>restricted computer?

First of all, WebAssembly is a specification of a set of virtual
instruction set and program format.
It's quite similar to JVM, but it's enough low-level to be able to lower
LLVM IR to the program format.

Recent major browsers have implemented execution engines for WebAssembly
alongside their JavaScript engines.
Those engines typically validate the given program and translate the
virtual instruction sequence into native one _or_
just interpret it. The program effectively has isolated memory space from
the host environment and also needs to have
explicit verifiable metadata to interact with the host system[2].
Therefore, browsers can run such programs without certain
class of security issues.

The spec of WebAssembly does not mention the existence of "browser" at all,
and its runtime can be implemented
outside of browsers. One of the major implementations of such standalone
runtime is wasmtime[3], and also there is a
smaller runtime implementation for restricted environments[4].

WASI is an interface specification between host system and guest program
and we have a libc implementation on the top of
it based on musl[4].

You can see more design rationales at https://github.com/WebAssembly/design

The current major limitations on the WASI and WebAssembly stack are:
1. Control flow is restricted to be _structured_, so setjmp/longjmp cannot
be implemented easily.
   (We have a way based on CPS transformation but it's quite expensive)
2. WASI still defines quite a small space of interfaces compared with POSIX
system calls for now.
It has a basic file system, environment variable, socket, etc..., but
still doesn't have something corresponding to getrlimit or
flock, or something like that.

Given that our libc implementation is based on musl and it's
already supported in this project, I think the work required to port
Gnulib to WASI/WebAssembly would be basically providing stub
implementations for functions that cannot be implemented yet
due to a lack of the system interface spec.

Let me know if you need more information :)

Thanks,
Yuta

[1] https://github.com/WebAssembly/wasi-sdk
[2] https://github.com/WebAssembly/design/blob/main/Modules.md#imports
[3] https://github.com/bytecodealliance/wasmtime
[4] https://github.com/bytecodealliance/wasm-micro-runtime
[5] https://github.com/WebAssembly/wasi-sdk


2024年5月5日(日) 0:01 Bruno Haible :

> Hi Saito-san,
>
> > * lib/getdtablesize.c (getdtablesize): Return INT_MAX if getrlimit
> >   is not available. This is the case for WASI, WebAssembly System
> >   Interface.
>
> Can you please tell us more about this compilation environment?
>
> I want to evaluate how much effort Gnulib should put into supporting this
> environment.
>
> I see [1] and [2], but
>
>   - How do _you_ compile and run a program in this environment?
> The same way as in [2], or do you do it differently?
>
>   - [1] promises that the programs "may run anywhere—from browsers to
> clouds
> to embedded devices". How do you run such a program in a browser?
> How do you run it in the "cloud"? How do you run it on a memory-
> restricted computer?
>
> Bruno
>
> [1] https://wasi.dev/
> [2]
> https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-tutorial.md
>
>
>
>

Re: Problem with boot-time on Cygwin

[Ken Brown]

On 5/4/2024 10:36 AM, Bruno Haible wrote:

I'm committing this workaround.


Thanks! I've tested on Cygwin 3.5.3 and on the latest test release, and 
I can confirm that the problem is fixed.


Ken

Re: header file substitutes

[Paul Eggert]

On 2024-05-04 07:52, Bruno Haible wrote:

But who will want to use it,
as long as it's not portable?


I'm working on a Gnulib  that will work on pre-C23 platforms.

Eventually this should replace Gnulib's count-leading-zeros, 
count-trailing-zeros, and count-one-bits modules, which should be marked 
as obsolescent once we have a standard (and nicer) way to get that 
information.

gnulib-tool sh+py mismatch when updating Emacs

[Paul Eggert]

Some sort of locale problem? The issue seems to be benign, except that 
it's a false positive when testing.


To reproduce it, clone current gnulib (commit 
fde88b711c9b1df5b142444ac7b0bc2aa8892d3a) and current emacs (commit 
fd859fbea2e9d13e76db1c5295d9ddd1c5955d83) under the current directory, 
and then run:


cd emacs
LC_ALL=en_US.utf8 GNULIB_TOOL_IMPL=sh+py admin/merge-gnulib

Here's what happens on Ubuntu 23.10 x86-64:

$ GNULIB_TOOL_IMPL=sh+py admin/merge-gnulib
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65) ... ok
Your system has the required tools.
Building aclocal.m4 ...
Running 'autoreconf -fi -I m4' ...
Building 'aclocal.m4' in exec ...
Running 'autoreconf -fi' in exec ...
Configuring local git repository...
'.git/config' -> '.git/config.~1~'
git config transfer.fsckObjects 'true'
git config diff.cpp.xfuncname '!^[ 
]*[A-Za-z_][A-Za-z_0-9]*:[[:space:]]*($|/[/*])

^((::[[:space:]]*)?[A-Za-z_][A-Za-z_0-9]*[[:space:]]*\(.*)$
^((#define[[:space:]]|DEFUN).*)$'
git config diff.elisp.xfuncname 
'^\([^[:space:]]*def[^[:space:]]+[[:space:]]+([^()[:space:]]+)'

git config diff.m4.xfuncname '^((m4_)?define|A._DEFUN(_ONCE)?)\([^),]*'
git config diff.make.xfuncname 
'^([$.[:alnum:]_].*:|[[:alnum:]_]+[[:space:]]*([*:+]?[:?]?|!?)=|define .*)'
git config diff.shell.xfuncname 
'^([[:space:]]*[[:alpha:]_][[:alnum:]_]*[[:space:]]*\(\)|[[:alpha:]_][[:alnum:]_]*=)'

git config diff.texinfo.xfuncname '^@node[[:space:]]+([^,[:space:]][^,]+)'
Installing git hooks...
'build-aux/git-hooks/commit-msg' -> '.git/hooks/commit-msg'
'build-aux/git-hooks/pre-commit' -> '.git/hooks/pre-commit'
'build-aux/git-hooks/prepare-commit-msg' -> '.git/hooks/prepare-commit-msg'
'build-aux/git-hooks/post-commit' -> '.git/hooks/post-commit'
'build-aux/git-hooks/pre-push' -> '.git/hooks/pre-push'
'build-aux/git-hooks/commit-msg-files.awk' -> 
'.git/hooks/commit-msg-files.awk'

'.git/hooks/applypatch-msg.sample' -> '.git/hooks/applypatch-msg'
'.git/hooks/pre-applypatch.sample' -> '.git/hooks/pre-applypatch'
You can now run './configure'.
../gnulib/gnulib-tool: *** gnulib-tool.py produced different files than 
gnulib-tool.sh! Compare /home/eggert/src/gnu/d/emacs and 
/home/eggert/src/gnu/d/glpyD3FBYw.

../gnulib/gnulib-tool: *** Stop.


The difference is between autom4te.cache/requests and 
../glpyD3FBYw/autom4te.cache/requests. They're sorted differently:


$ diff -ur ./autom4te.cache/requests ../glpyD3FBYw/autom4te.cache/requests
--- ./autom4te.cache/requests   2024-05-04 08:47:45.397566582 -0700
+++ ../glpyD3FBYw/autom4te.cache/requests	2024-05-04 08:46:41.307448492 
-0700

@@ -16,69 +16,69 @@
 'configure.ac'
   ],
   {
-'_LT_AC_TAGCONFIG' => 1,
-'LT_INIT' => 1,
-'AM_XGETTEXT_OPTION' => 1,
-'m4_sinclude' => 1,
-'_AM_COND_IF' => 1,
-'AM_PROG_MOC' => 1,
+'AM_GNU_GETTEXT' => 1,
+'AC_CANONICAL_BUILD' => 1,
 'AC_CONFIG_AUX_DIR' => 1,
+'AC_PROG_LIBTOOL' => 1,
+'m4_include' => 1,
+'AM_CONDITIONAL' => 1,
+'AM_PROG_MKDIR_P' => 1,
+'_AM_SUBST_NOTMAKE' => 1,
+'AM_EXTRA_RECURSIVE_TARGETS' => 1,
+'AC_DEFINE_TRACE_LITERAL' => 1,
+'AC_FC_FREEFORM' => 1,
+'m4_pattern_forbid' => 1,
+'AM_INIT_AUTOMAKE' => 1,
 'LT_CONFIG_LTDL_DIR' => 1,
 'AC_CANONICAL_SYSTEM' => 1,
-'AC_CANONICAL_BUILD' => 1,
-'_AM_COND_ELSE' => 1,
+'AC_FC_PP_DEFINE' => 1,
+'LT_INIT' => 1,
+'AC_FC_SRCEXT' => 1,
+'AC_LIBSOURCE' => 1,
 'AC_CONFIG_LIBOBJ_DIR' => 1,
-'AM_PROG_CXX_C_O' => 1,
+'AM_PROG_FC_C_O' => 1,
 'AC_CONFIG_HEADERS' => 1,
-'AM_GNU_GETTEXT' => 1,
+'_AM_COND_ELSE' => 1,
+'AC_CONFIG_SUBDIRS' => 1,
+'AM_PROG_CXX_C_O' => 1,
+'m4_sinclude' => 1,
+'AM_MAKEFILE_INCLUDE' => 1,
 'AM_PROG_LIBTOOL' => 1,
+'AC_INIT' => 1,
+'include' => 1,
+'_AM_COND_ENDIF' => 1,
+'AM_POT_TOOLS' => 1,
+'AM_MAINTAINER_MODE' => 1,
+'_LT_AC_TAGCONFIG' => 1,
+'_AM_MAKEFILE_INCLUDE' 

Re: gnulib-tool.py: Fix an undefined function name.

[Bruno Haible]

Collin Funk wrote:
> For CI, I am thinking it is best to ignore the typing warnings for the
> time being. Since gnulib-tool.py needs a bit more refactoring for that
> to be helpful. It was written before typing was a thing. :)

Hmm. Is that refactoring, that you are imagining, intrusive? If it's
intrusive, then maybe we should leave it alone. You already introduced
types in the function signatures and in the fields of classes; what else
is missing?

> I'm removing the 'lines_to_multiline' import from GLImport.py. We
> could have a few checks for guaranteed runtime failures like this:
> 
>   $ mypy GLImport.py | grep 'name-defined'
>   GLImport.py:722: error: Name "lines_to_multiline" is not defined  
> [name-defined]
>   GLImport.py:736: error: Name "lines_to_multiline" is not defined  
> [name-defined]
>   GLImport.py:1143: error: Name "lines_to_multiline" is not defined  
> [name-defined]
>   GLImport.py:1326: error: Name "lines_to_multiline" is not defined  
> [name-defined]
>   GLImport.py:1327: error: Name "lines_to_multiline" is not defined  
> [name-defined]
>   GLImport.py:1328: error: Name "lines_to_multiline" is not defined  
> [name-defined]

Sounds like a reasonable way to automate this check...

> I'm not too familiar with the GitLab CI stuff. Would a Makefile in
> gnulib/pygnulib with some checks work?

Yes, a Makefile in pygnulib/ would be sufficient. The top-level Makefile then
may invoke it.

Bruno

Re: [PATCH] getdtablesize: support systems without getrlimit like WASI

[Bruno Haible]

Hi Saito-san,

> * lib/getdtablesize.c (getdtablesize): Return INT_MAX if getrlimit
>   is not available. This is the case for WASI, WebAssembly System
>   Interface.

Can you please tell us more about this compilation environment?

I want to evaluate how much effort Gnulib should put into supporting this
environment.

I see [1] and [2], but

  - How do _you_ compile and run a program in this environment?
The same way as in [2], or do you do it differently?

  - [1] promises that the programs "may run anywhere—from browsers to clouds
to embedded devices". How do you run such a program in a browser?
How do you run it in the "cloud"? How do you run it on a memory-
restricted computer?

Bruno

[1] https://wasi.dev/
[2] https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-tutorial.md

Re: header file substitutes

[Bruno Haible]

Collin Funk wrote:
> I didn't realize the header replacements started this early. I'm a big
> fan of them.

And we're not yet done with the header file substitutes. One is still missing
from Gnulib:  [1]. The only thing that has changed since last year
is that glibc 2.39 now has this header file [2]. But who will want to use it,
as long as it's not portable?

Bruno

[1] https://lists.gnu.org/archive/html/bug-gnulib/2023-03/msg00142.html
[2] https://lists.gnu.org/archive/html/info-gnu/2024-01/msg00017.html

Re: Problem with boot-time on Cygwin

[Bruno Haible]

Ken Brown wrote:
> I don't know what's going on, but it seems clear that pagefile.sys can't 
> be used reliably on Cygwin to get a boot time.  I wonder if 
> get_boot_time should simply bail out on Cygwin and always give a boot 
> time of 0.  Or do you have a better idea?

I'm committing this workaround.


2024-05-04  Bruno Haible  

readutmp, boot-time: Work around a Cygwin 3.5.3 bug.
Reported by Ken Brown  in
.
* lib/boot-time-aux.h (get_windows_boot_time): On Cygwin, ignore
pagefile.sys if it appears to be a directory, and use another file as
a fallback.

diff --git a/lib/boot-time-aux.h b/lib/boot-time-aux.h
index a94cdb3f30..7f8c5405e4 100644
--- a/lib/boot-time-aux.h
+++ b/lib/boot-time-aux.h
@@ -304,19 +304,35 @@ get_windows_boot_time (struct timespec *p_boot_time)
  Instead, on Windows, the boot time can be retrieved by looking at the
  time stamp of a file that (normally) gets touched only during the boot
  process, namely C:\pagefile.sys.  */
-  const char * const boot_touched_file =
-#if defined __CYGWIN__ && !defined _WIN32
-/* It is more portable to use /proc/cygdrive/c than /cygdrive/c.  */
-"/proc/cygdrive/c/pagefile.sys"
-#else
-"C:\\pagefile.sys"
-#endif
-;
-  struct stat statbuf;
-  if (stat (boot_touched_file, ) >= 0)
+  const char * const boot_touched_files[] =
 {
-  *p_boot_time = get_stat_mtime ();
-  return 0;
+  #if defined __CYGWIN__ && !defined _WIN32
+  /* It is more portable to use /proc/cygdrive/c than /cygdrive/c.  */
+  "/proc/cygdrive/c/pagefile.sys",
+  /* A fallback, working around a Cygwin 3.5.3 bug.  It has a modification
+ time about 1.5 minutes after the last boot; but that's better than
+ nothing.  */
+  
"/proc/cygdrive/c/ProgramData/Microsoft/Windows/DeviceMetadataCache/dmrc.idx"
+  #else
+  "C:\\pagefile.sys"
+  #endif
+};
+  for (idx_t i = 0; i < SIZEOF (boot_touched_files); i++)
+{
+  const char *filename = boot_touched_files[i];
+  struct stat statbuf;
+  if (stat (filename, ) >= 0)
+{
+# if defined __CYGWIN__ && !defined _WIN32
+  /* Work around a Cygwin 3.5.3 bug.
+   */
+  if (!S_ISDIR (statbuf.st_mode))
+# endif
+{
+  *p_boot_time = get_stat_mtime ();
+  return 0;
+}
+}
 }
   return -1;
 }

[PATCH] getdtablesize: support systems without getrlimit like WASI

[Yuta Saito]

* lib/getdtablesize.c (getdtablesize): Return INT_MAX if getrlimit
  is not available. This is the case for WASI, WebAssembly System
  Interface.
* m4/getdtablesize.m4 (gl_PREREQ_GETDTABLESIZE): Check for getrlimit.
---
 ChangeLog   | 8 
 lib/getdtablesize.c | 2 ++
 m4/getdtablesize.m4 | 6 --
 3 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 96b29b7fc4..1ef678a316 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2024-05-04 Yuta Saito 
+
+   getdtablesize: support systems without getrlimit like WASI
+   * lib/getdtablesize.c (getdtablesize): Return INT_MAX if getrlimit
+   is not available. This is the case for WASI, WebAssembly System
+   Interface.
+   * m4/getdtablesize.m4 (gl_PREREQ_GETDTABLESIZE): Check for getrlimit.
+
 2024-05-03  Collin Funk  
 
gnulib-tool.py: Fix an undefined function name.
diff --git a/lib/getdtablesize.c b/lib/getdtablesize.c
index 762c100b38..fca937a39a 100644
--- a/lib/getdtablesize.c
+++ b/lib/getdtablesize.c
@@ -109,6 +109,7 @@ getdtablesize (void)
 int
 getdtablesize (void)
 {
+# if HAVE_GETRLIMIT
   struct rlimit lim;
 
   if (getrlimit (RLIMIT_NOFILE, ) == 0
@@ -117,6 +118,7 @@ getdtablesize (void)
   && lim.rlim_cur != RLIM_SAVED_CUR
   && lim.rlim_cur != RLIM_SAVED_MAX)
 return lim.rlim_cur;
+# endif
 
   return INT_MAX;
 }
diff --git a/m4/getdtablesize.m4 b/m4/getdtablesize.m4
index aaefe9b298..7059eb933e 100644
--- a/m4/getdtablesize.m4
+++ b/m4/getdtablesize.m4
@@ -1,5 +1,5 @@
 # getdtablesize.m4
-# serial 8
+# serial 9
 dnl Copyright (C) 2008-2024 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -61,4 +61,6 @@ AC_DEFUN([gl_FUNC_GETDTABLESIZE],
 ])
 
 # Prerequisites of lib/getdtablesize.c.
-AC_DEFUN([gl_PREREQ_GETDTABLESIZE], [:])
+AC_DEFUN([gl_PREREQ_GETDTABLESIZE], [
+  AC_CHECK_FUNCS_ONCE([getrlimit])
+])
-- 
2.43.2

Re: gnulib-tool.py: Fix an undefined function name.

[Collin Funk]

On 5/4/24 3:34 AM, Bruno Haible wrote:
> Yes. We cannot change the past git log, but this is one of the situations
> where changing a ChangeLog entry is adequate. Other such situations are
> obvious typos and forgotten changes.

I've pushed the attached patch adding the date to the ChangeLog entry.

>>> In this case, maybe some Python style checker would have produced a
>>> warning, right? Would it be possible to add a Makefile rule that checks
>>> for the absense of such warnings, and then use this Makefile rule in the
>>> continuous integration that runs at least once a week?
>>
>> Hmm, I like that idea. I would have to find what tool works best for
>> that.
> 
> Yes, please.

Right now I am leaning towards mypy. It seems to be the best at
detecting errors/potential errors.

For CI, I am thinking it is best to ignore the typing warnings for the
time being. Since gnulib-tool.py needs a bit more refactoring for that
to be helpful. It was written before typing was a thing. :)

mypy has the following options:

 mypy --disable-error-code NAME
 mypy --enable-error-code NAME

but there is a long list of error names and I'm not sure how
compatible they are between versions.

I'm removing the 'lines_to_multiline' import from GLImport.py. We
could have a few checks for guaranteed runtime failures like this:

  $ mypy GLImport.py | grep 'name-defined'
  GLImport.py:722: error: Name "lines_to_multiline" is not defined  
[name-defined]
  GLImport.py:736: error: Name "lines_to_multiline" is not defined  
[name-defined]
  GLImport.py:1143: error: Name "lines_to_multiline" is not defined  
[name-defined]
  GLImport.py:1326: error: Name "lines_to_multiline" is not defined  
[name-defined]
  GLImport.py:1327: error: Name "lines_to_multiline" is not defined  
[name-defined]
  GLImport.py:1328: error: Name "lines_to_multiline" is not defined  
[name-defined]

I'm not too familiar with the GitLab CI stuff. Would a Makefile in
gnulib/pygnulib with some checks work? Or did you have a different idea?

I think mypy should be packed in Debian and Fedora as 'python3-mypy'.
Another method that works for Python software is pipx [1]:

$ pipx install mypy
$ readlink $HOME/.local/bin/mypy
/home/collin/.local/share/pipx/venvs/mypy/bin/mypy

Sometimes it helps for distributions with old packages. It uses a
virtual environment so you don't have to worry about breaking things
too.

[1] https://pipx.pypa.io/stable/

CollinFrom 284c0ad51e1caba277616c76c6cd14774d92e19c Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Sat, 4 May 2024 05:17:45 -0700
Subject: [PATCH] Fix ChangeLog entry for previous commit.

---
 ChangeLog | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index 96b29b7fc4..6b56b35d8e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,6 +1,6 @@
 2024-05-03  Collin Funk  
 
-	gnulib-tool.py: Fix an undefined function name.
+	gnulib-tool.py: Fix an undefined function name (regression 2024-05-02).
 	* pygnulib/main.py (main_with_exception_handling): Use the tempfile
 	module prefix when calling mkdtemp(). Use the 'glpy' prefix for the
 	temporary directory that exists for the entirety of the program.
-- 
2.44.0

Re: gnulib-tool.py: Fix an undefined function name.

[Bruno Haible]

Hi Collin,

> > Can you please add, to the first line of the ChangeLog entry, a note
> > that tells us when the regression was introduced? Such as
> >(regression 2024-xx-xy)
> > or
> >(regr. yesterday)
> ...
> I've never modified a ChangeLog entry before. Is the proper way to
> just modify it and commit with a one line summary?

Yes. We cannot change the past git log, but this is one of the situations
where changing a ChangeLog entry is adequate. Other such situations are
obvious typos and forgotten changes.

> > In this case, maybe some Python style checker would have produced a
> > warning, right? Would it be possible to add a Makefile rule that checks
> > for the absense of such warnings, and then use this Makefile rule in the
> > continuous integration that runs at least once a week?
> 
> Hmm, I like that idea. I would have to find what tool works best for
> that.

Yes, please.

Bruno

Re: doc: Add appendix about Gnulib history

[Bruno Haible]

Hi Collin,

> > doc: Add appendix about Gnulib history.
> > * doc/gnulib-history.texi: New file.
> > * doc/gnulib.texi: Include it.
> 
> Interesting section, thanks.

I wrote it also as an answer to misguided statements that
"gnulib is mainly about porting to non-Linux platforms" [1].

Bruno

[1] 
https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/YWMNOEJ34Q7QLBWQAB5TM6A2SVJFU4RV/

Re: doc: Add appendix about Gnulib history

[Collin Funk]

On 5/3/24 11:32 PM, Paul Eggert wrote:
> Not quite, because it doesn't check for padding bits.

Sorry, I should have clarified excluding that part...

> Also, your code assumes that preprocessor arithmetic works the same as
> runtime arithmetic. Although C23 guarantees this (because it
> guarantees two's complement) I don't think C17 and earlier do. If
> memory serves, they even allow mixed-runtime systems, where arithmetic
> is sometimes ones' complement and sometimes two's.

Ah, interesting. Thanks for the explanation.

Collin

Re: doc: Add appendix about Gnulib history

[Paul Eggert]

On 2024-05-03 17:14, Collin Funk wrote:

Also about this comment in intprops-internal.h:

 /* This include file assumes that signed types are two's complement without
padding bits; the above macros have undefined behavior otherwise.
If this is a problem for you, please let us know how to fix it for your 
host.
This assumption is tested by the intprops-tests module.  */

If you want to check this at compile time I think this should work:


Not quite, because it doesn't check for padding bits. Also, your code 
assumes that preprocessor arithmetic works the same as runtime 
arithmetic. Although C23 guarantees this (because it guarantees two's 
complement) I don't think C17 and earlier do. If memory serves, they 
even allow mixed-runtime systems, where arithmetic is sometimes ones' 
complement and sometimes two's. (Almost nobody cares about this 
theoretical possibility of course, which is why the Gnulib manual says 
not to worry about it.)