Re: Hurd Manual Structure/tooling

[Samuel Thibault]

Hello,

Alexander Ziaee, le jeu. 06 juin 2024 01:25:29 +, a ecrit:
> A. Would anyone be willing to share the sections of the Hurd manual? E.g (on 
> BSD). 1: user utilities, 2: syscalls, 3: libraries, 4: kernel interfaces, etc.

The sections are just like the GNU/Linux manual pages. "syscall" is thus
a somehow odd naming but from the application point of view that's fine.

> B. Are you guys using mandoc or groff or something else?

For the hurd documentation we use texinfo.

> Does your stack play nicely with mdoc(7) pages?

I don't know mdoc.

> I'm working on cleaning up manuals and especially apropos results across the 
> FreeBSD stack. Currently, xf86-input-keyboard is used by Hurd as well as us, 
> and I'd like to make sure that my changes won't cause any regressions for you.

Thanks for caring :)

Samuel

Bug#1072337: gst-plugins-good1.0: Missing build-dep for non-linux

[Samuel Thibault]

Source: gst-plugins-good1.0
Version: 1.24.4-1
Severity: important
Tags: patch
User: debian-hurd@lists.debian.org
Usertag: hurd

Hello,

Some dependencies are missing for gst-plugins-good1.0 on non-linux:

https://buildd.debian.org/status/fetch.php?pkg=gst-plugins-good1.0=hurd-i386=1.24.4-1=1717159633=0
../ext/adaptivedemux2/meson.build:104:6: ERROR: Problem encountered: 
adaptivedemux2: Either libsoup2 or libsoup3 is needed

https://buildd.debian.org/status/fetch.php?pkg=gst-plugins-good1.0=hurd-i386=1.24.4-1=1717233674=0
../ext/qt6/meson.build:64:6: ERROR: Program 'qsb-qt6 qsb' not found or not 
executable

the attached patch fixes them, could you apply it?

Thanks,
Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'stable'), 
(500, 'oldstable'), (1, 'experimental-debug'), (1, 'buildd-experimental'), (1, 
'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.9.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
--- debian/control.orig 2024-06-01 11:28:00.933813029 +0200
+++ debian/control  2024-06-01 11:30:01.364309857 +0200
@@ -27,6 +27,7 @@
libaa1-dev (>= 1.4p5),
libflac-dev (>= 1.1.4),
libdv4-dev | libdv-dev,
+   libsoup-3.0-dev [!linux-any],
libxdamage-dev,
libxext-dev,
libxfixes-dev,
@@ -51,7 +52,7 @@
qttools5-dev [amd64 arm64 armel armhf i386 mips64el ppc64el 
riscv64 s390x hurd-i386 powerpc ppc64 sparc64],
qt6-base-private-dev [amd64 arm64 armel armhf mips64el ppc64el 
riscv64 s390x hurd-i386 powerpc ppc64 sparc64],
qt6-declarative-dev [amd64 arm64 armel armhf mips64el ppc64el 
riscv64 s390x hurd-i386 powerpc ppc64 sparc64],
-   qt6-shader-baker [amd64 arm64 armel armhf mips64el ppc64el 
riscv64 s390x powerpc ppc64 sparc64],
+   qt6-shader-baker [amd64 arm64 armel armhf mips64el ppc64el 
riscv64 s390x hurd-i386 powerpc ppc64 sparc64],
qt6-tools-dev [amd64 arm64 armel armhf mips64el ppc64el riscv64 
s390x hurd-i386 powerpc ppc64 sparc64],
qt6-wayland-dev [amd64 arm64 armel armhf mips64el ppc64el 
riscv64 s390x powerpc ppc64 sparc64],
libqt5x11extras5-dev [amd64 arm64 armel armhf i386 mips64el 
ppc64el riscv64 s390x hurd-i386 powerpc ppc64 sparc64],

Bug#1072224: gst-plugins-base1.0: FTBFS on hurd-any

[Samuel Thibault]

Source: gst-plugins-base1.0
Version: 1.24.4-1
Severity: important
Tags: patch ftbfs
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

The attached patch fixes the build of gst-plugins-base1.0 on hurd-any,
could you apply it?

Thanks,
Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'stable'), 
(500, 'oldstable'), (1, 'experimental-debug'), (1, 'buildd-experimental'), (1, 
'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.9.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
--- debian/rules.original   2024-05-30 17:12:55.0 +
+++ debian/rules2024-05-30 17:32:59.0 +
@@ -33,6 +33,7 @@
 
 ifeq ($(DEB_HOST_ARCH_OS),hurd)
 conf_flags += -Dcdparanoia=disabled
+conf_flags += -Ddrm=disabled
 endif
 
 ifneq ($(DEB_HOST_ARCH_OS),linux)
--- debian/libgstreamer-gl1.0-0.symbols.orig2024-05-30 17:31:05.0 
+
+++ debian/libgstreamer-gl1.0-0.symbols 2024-05-30 17:31:32.0 +
@@ -36,7 +36,7 @@
  (arch=linux-any kfreebsd-any)gst_egl_image_from_dmabuf@Base 1.14.0
  (arch=linux-any kfreebsd-any)gst_egl_image_from_dmabuf_direct@Base 1.16.0
  (arch=linux-any kfreebsd-any)gst_egl_image_from_dmabuf_direct_target@Base 
1.18.0
- gst_egl_image_from_dmabuf_direct_target_with_dma_drm@Base 1.23.1
+ (arch=linux-any 
kfreebsd-any)gst_egl_image_from_dmabuf_direct_target_with_dma_drm@Base 1.23.1
  gst_egl_image_from_texture@Base 1.14.0
  gst_egl_image_get_image@Base 1.14.0
  gst_egl_image_get_type@Base 1.14.0

Re: rustc_1.72.1+dfsg1-1+hurd.1_multi.changes ACCEPTED

[Samuel Thibault]

Samuel Thibault, le mer. 29 mai 2024 11:09:28 +0200, a ecrit:
> (will become available in the unreleased archive within 6h)

It's available. The buildds have started building the hundreds of
packages that were waiting.

Samuel

Re: rustc_1.72.1+dfsg1-1+hurd.1_multi.changes ACCEPTED

[Samuel Thibault]

Hello,

It was becoming more and more a concern (gstreamer build-depends on
rustc nowadays). At last, the rustc compiler becomes available :D

Thanks to Vedant's GSoC work last summer, and then waiting for Debian to
catch with upstream releases, eventually we have rustc!

(will become available in the unreleased archive within 6h)

Samuel

Re: migration: ntpdate to rdate

[Samuel Thibault]

Hello,

Martin-Éric Racine, le sam. 25 mai 2024 12:39:20 +0300, a ecrit:
> la 25. toukok. 2024 klo 10.17 Martin-Éric Racine
> (martin-eric.rac...@iki.fi) kirjoitti:
> > I cannot help but notice that the Hurd port still depends on 'ntpdate'
> > to sync its clock upon bootup. The key problem is that Debian has
> > migrated to 'src:ntpsec' which made 'bin:ntpdate' a transitional
> > package. However, 'rdate' seemingly has been ported to Hurd. Assuming
> > that it is verified to work, it could be usefull to migrate the Hurd
> > port to it.
> 
> As far as I can tell, on Hurd i386, using 'rdate' without any option
> does nothing. The command just sits there and wait.
> 
> $ sudo rdate -v pool.ntp.org
> 
> However, if I use the -n option for SNTP, it returns something:
> 
> $ sudo rdate -n -v pool.ntp.org
> Sat May 25 12:35:42 EEST 2024
> rdate: adjust local clock by -0.005979 seconds

According to the package description,

“By default, rdate uses the RFC 868 TCP protocol.”

so it apparently indeed needs to be told to use ntp, when targetting an
ntp server.

> Unless I'm mistaken, this means that Hurd should be able to use a
> wrapper script similar to 'ntpdate-debian' to achieve the same result
> as the old reference 'ntpdate' Hurd has forked.

Indeed.

> It should be fairly trivial to implement and submit to the 'rdate'
> maintainer for inclusion, at which point Hurd's old NTP fork could be
> put to rest.

Help welcome ;)

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le mar. 21 mai 2024 11:50:25 +0200, a ecrit:
> Where I should've looked is:
> 
> http://ftp.ports.debian.org/debian-ports/pool-hurd-i386/main/p/pam/
> 
> Or I should've looked in the hurd-i386 Packages file instead of checking the
> pool directory. So what I did instead was to build pam with the patches from
> #1029097 on the exodar porter box resulting in this debdiff:
> 
> https://paste.debian.net/hidden/84c2d298/
> 
> Are the sources for pam that got uploaded to unreleased available somewhere?

It's in http://ftp.ports.debian.org/debian-ports/pool-hurd-i386/main/p/pam/

> I only see an empty (40 byte) Sources.gz in
> http://ftp.ports.debian.org/debian-ports/dists/unreleased/main/source/

Yes, that's a missing feature in the mini-dak setup of debian-ports.

> In any case, things go much further now. The next problem is some missing
> DPKG_ROOT support in the hurd maintainer script. I opened a merge request 
> here:
> 
> https://salsa.debian.org/hurd-team/hurd/-/merge_requests/1

Thanks!

> There were no forks and no merge requests on that repo yet so I hope I did
> everything correctly. :D

Congrats for the first MR :)

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le mer. 10 avril 2024 12:15:33 +0200, a ecrit:
> It seems that new versions of login, passwd and util-linux require a more
> recent version of pam than was last buildable:
> 
> 
> The following packages have unmet dependencies:
>  login : PreDepends: libpam-runtime but it is not going to be installed
>  PreDepends: libpam-modules but it is not installable
>  passwd : Depends: libpam0g (>= 0.99.7.1) but it is not installable
>   Depends: libpam-modules but it is not installable
>  util-linux : PreDepends: libpam0g (>= 0.99.7.1) but it is not installable
> E: Unable to correct problems, you have held broken packages.

The available version (1.5.3-6+hurd.1) is largely enough for 0.99.7.1, I
don't think it's a question of "newer login, passwd and util-linux".

Note the subtle difference between "is not going to be installed" and
"is not installable". The latter means there is no available version at
all, while libpam-modules=1.5.3-6+hurd.1 is installable, from

deb http://ftp.ports.debian.org/debian-ports/ unreleased main

Samuel

Re: smp i386 kernels

[Samuel Thibault]

Maite Gamper, le dim. 19 mai 2024 16:41:41 +0200, a ecrit:
> But I've noticed that "gnumach-image-1.8-486-smp"
> seems to contain a uniprocessor kernel in sid,  the
> files seem to be identical.

Oops, there must have been some mistake in the build scripts, I'll have
a look.

> I've heard that mach doesn't really support smp, is that still
> correct?

Rumours are never to be taken seriously. Always refer to the reference.

https://darnassus.sceen.net/~hurd-web/faq/smp/

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le mer. 10 avril 2024 10:09:02 +0200, a ecrit:
> So I tried the whole thing again and we got a new blocker:
> 
> https://buildd.debian.org/status/package.php?p=fakeroot

I'm not sure why it's a blocker? Is the previous fakeroot version not
enough?

> Do you see an easy way to fix fakeroot on hurd-i386

If it was easy it would be fixed already :)

> or should I just disable fakeroot mode?

Note that there is the fakeroot-hurd alternative which is actually much
more lightweight and much less hacky than fakeroot.

> Or maybe I should switch my tests from hurd-i386 to hurd-amd64?

You'll probably miss a lot of packages on hurd-amd64.

Samuel

smp and pae kernels

[Samuel Thibault]

Hello

In the latest gnumach upload, I have added smp and pae variants of the
kernel. pae should be relatively fine and allow to access more than 3G
memory. smp is completely experimental, and needs fixing here and there,
it seems like irq routing, notably, needs fixes, but at least people can
easily try.

Samuel

Re: ntpdate : Depends: ntpsec-ntpdate but it is not installable

[Samuel Thibault]

Hello,

Jose Luis Alarcon Sanchez, le dim. 24 mars 2024 13:31:22 +0100, a ecrit:
> What's the situation with the package ntpsec-ntpdate?. Is there a fix
> possible?. I volunteer, if i am able. Waiting for instructions to follow.

Fixing the build to at least support the step adjustment should be
relativement easy, yes. The slew adjustment would need adding some
kernel support for implementing adjtimex.

Samuel

Re: Hard Disk size limits on GNU Hurd

[Samuel Thibault]

Hello,

Tanguy LE CARROUR, le ven. 22 mars 2024 13:33:59 +0100, a ecrit:
> Quoting Samuel Thibault (2024-03-22 12:37:33)
> > Jose Luis Alarcon Sanchez, le ven. 22 mars 2024 12:31:04 +0100, a ecrit:
> > > Several years ago there was a limit in the size of the hard disks that 
> > > Hurd
> > > could manage. If i'm not wrong, i think it was 5 Gb. or so.
> > > 
> > > Is still this limit working right now?, or can i create a disk image (for
> > > kvm) with the size i decide (20 Gb. let's say)?.
> > 
> > Please read the faq ;)
> > 
> > https://darnassus.sceen.net/~hurd-web/faq/
> > https://darnassus.sceen.net/~hurd-web/faq/2_gib_partition_limit/
> 
> It was working fine until I ran short of disk space and increased it to
> 20Gb.

Did you really increase the filesystem size as well?

> Now, it randomly crashes with error messages like:
> 
> ```
> ext2fs: BUG: unexpected fault on disk image (10, 0x8ffc000) in
> [0xB8222000,0x18222000) eip 0x8052224 err 0xa
> ```
> 
> or:
> 
> ```
> ext2fs: disk-pager.c:109: fault_handler: Assertion ’err’ failed.
> ```

This looks to me like ENOSPC messages.

Saumel

Re: Hard Disk size limits on GNU Hurd

[Samuel Thibault]

Hello,

Jose Luis Alarcon Sanchez, le ven. 22 mars 2024 12:31:04 +0100, a ecrit:
> Several years ago there was a limit in the size of the hard disks that Hurd
> could manage. If i'm not wrong, i think it was 5 Gb. or so.
> 
> Is still this limit working right now?, or can i create a disk image (for
> kvm) with the size i decide (20 Gb. let's say)?.

Please read the faq ;)

https://darnassus.sceen.net/~hurd-web/faq/

https://darnassus.sceen.net/~hurd-web/faq/2_gib_partition_limit/

Samuel

Re: How connect Translator /hurd/ftpfs with a non anonimous ftp account with user and passwd

[Samuel Thibault]

Jose Luis Alarcon Sanchez, le dim. 17 mars 2024 20:13:13 +0100, a ecrit:
> On Sun, Mar 17, 2024 at 06:48:17PM +0100, Samuel Thibault wrote:
> > Hello,
> >
> > Jose Luis Alarcon Sanchez, le dim. 17 mars 2024 18:34:29 +0100, a ecrit:
> > > The 'test' work exactly as the text say, and i did ls on the GNU FTP, but 
> > > i want to introduce here my
> > > question: Can this be done also with a ftp server that requires an user 
> > > name
> > > and password?. I have my account with user name and password and do well
> > > "traditional ftp". Can i get too have this Transparent FTP that the
> > > marvelous concept of the Translator allows?.
> >
> > See /hurd/ftpfs --help:
> >
> > SERVER can be a hostname, in which case anonymous ftp is used, or may
> > include a user and password like `USER:PASSWORD@HOST' (the `:PASSWORD'
> > part is optional).
> >
> > So in theory you can pass user+password. But that will then show up in
> > ls, so you don't actually want this. Something would need to be added to
> > hostmux and/or ftpfs to support this in a safe way.
> >
> 
> My sequence of commands:
> 
>  $ settrans -c ftp: /hurd/hostmux /hurd/ftpfs /
>  $ ls ftp://myuser:mypas...@servername.net/
> ls: cannot access 'ftp://myuser:mypas...@servername.net/': Translator died

This does work for me:

$ settrans -c ftp: /hurd/hostmux /hurd/ftpfs /
$ ls ftp://anonymous:anonym...@ftp.gnu.org/
CRYPTO.README  find.txt.gznon-gnu  tmp
MISSING-FILES  gnuold-gnu  tree.json.gz
MISSING-FILES.README   gnu+linux-distros  pub  video
README ls-lrRt.txt.gz savannah welcome.msg
before-2003-08-01.md5sums.asc  mirrorsthird-party

>  $ ftp servername.net
> 
> The part 'ftp://' is not needed. Can this be important?

calling ftp already tells you're using the ftp protocol. Above, "ftp://;
is merely to use the ftp:/ directory that you have set up.

Samuel

Re: How connect Translator /hurd/ftpfs with a non anonimous ftp account with user and passwd

[Samuel Thibault]

Alexandre Garreau-Capitanio, le dim. 17 mars 2024 19:29:56 +0100, a ecrit:
> On 17/03/2024 18:48, Samuel Thibault wrote:
> > SERVER can be a hostname, in which case anonymous ftp is used, or may
> > include a user and password like `USER:PASSWORD@HOST' (the `:PASSWORD'
> > part is optional).
> > 
> > So in theory you can pass user+password. But that will then show up in
> > ls, so you don't actually want this. Something would need to be added to
> > hostmux and/or ftpfs to support this in a safe way.
> 
> .authrc(.gpg) support?

If you are setting up an ftpfs mount for yourself, yes. But for
globally-visible mounts, you'd need to put somewhere an .authrc file for
the uid that you used to set the mount.

Samuel

Re: How connect Translator /hurd/ftpfs with a non anonimous ftp account with user and passwd

[Samuel Thibault]

Hello,

Jose Luis Alarcon Sanchez, le dim. 17 mars 2024 18:34:29 +0100, a ecrit:
> The 'test' work exactly as the text say, and i did ls on the GNU FTP, but i 
> want to introduce here my
> question: Can this be done also with a ftp server that requires an user name
> and password?. I have my account with user name and password and do well
> "traditional ftp". Can i get too have this Transparent FTP that the
> marvelous concept of the Translator allows?.

See /hurd/ftpfs --help:

SERVER can be a hostname, in which case anonymous ftp is used, or may
include a user and password like `USER:PASSWORD@HOST' (the `:PASSWORD'
part is optional).

So in theory you can pass user+password. But that will then show up in
ls, so you don't actually want this. Something would need to be added to
hostmux and/or ftpfs to support this in a safe way.

Samuel

Bug#1066932: gcc-14: Please enable m2 on hurd-any

[Samuel Thibault]

Source: gcc-14
Version: 14-20240303-1
Severity: normal
Tags: patch
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

Now that upstream has fixed the m2 portability (available in the
20240303 snapshot), could you enable m2 on hurd-any, as the attached
patch does?

Thanks,
Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.7.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.
diff --git a/debian/rules.defs b/debian/rules.defs
index 928d4e56..44f3f3a0 100644
--- a/debian/rules.defs
+++ b/debian/rules.defs
@@ -1195,7 +1195,7 @@ ifneq ($(with_base_only),yes)
   endif
 endif
 m2_no_cpus = loong64 powerpc ppc64 sh4
-m2_no_systems = gnu
+m2_no_systems = 
 ifneq (,$(filter $(DEB_TARGET_ARCH_CPU),$(m2_no_cpus)))
   with_m2 := disabled for cpu $(DEB_TARGET_ARCH_CPU)
 endif

Bug#1065456: python3.11: Please don't enable dtrace on hurd

[Samuel Thibault]

Package: python3.11
Version: 3.11.8-1
Severity: important
Tags: patch
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

python3.11 currently fails to build on hurd-any because debian/rules
enables dtrace, which is not available on hurd-any.

Could you apply the attached patch that fixes this, on python3.11 as
well as on python3.12?

Thanks,
Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.7.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages python3.11 depends on:
ii  libpython3.11-stdlib  3.11.8-1
ii  media-types   10.1.0
ii  mime-support  3.66
ii  python3.11-minimal3.11.8-1

Versions of packages python3.11 recommends:
ii  ca-certificates  20240203

Versions of packages python3.11 suggests:
ii  binutils 2.42-3
pn  python3.11-doc   
ii  python3.11-venv  3.11.8-1

-- no debconf information

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.
--- debian/rules.orig   2024-03-03 10:23:40.0 +0100
+++ debian/rules2024-03-04 23:54:06.808627222 +0100
@@ -441,7 +441,6 @@
--with-computed-gotos \
--without-ensurepip \
--with-system-expat \
-   --with-dtrace \
--with-ssl-default-suites=openssl \
--with-wheel-pkg-dir=/usr/share/python-wheels/ \
--with-ssl-default-suites=openssl \
@@ -453,6 +452,10 @@
   common_configure_args += --with-system-ffi
 endif
 
+ifeq (,$(filter $(DEB_HOST_ARCH_OS), hurd))
+  common_configure_args += --with-dtrace
+endif
+
 ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE))
   common_configure_args += --host=$(DEB_HOST_GNU_TYPE) 
--build=$(DEB_BUILD_GNU_TYPE)
   common_configure_args += --with-build-python

time_t-related transition coming

[Samuel Thibault]

Hello,

Just to warn: the upload of the t64 packages is about to come into
Debian, that'll be a very big transition, so in the meanwhile the sid
distribution will be essentially uninstallable and non-upgradable.
Please bear with us while the world gets rebuilt ;)

Samuel

Bug#1063643: gcc-14: Fix disabling go and m2 according to OS

[Samuel Thibault]

Source: gcc-14
Version: 14-20240201-3
Severity: important
Tags: patch
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

There was a typo in rules.defs concerning go_no_systems and
m2_no_systems: they are still compared against DEB_TARGET_ARCH_OS,
while their content is gnu-system-like (kfreebsd-gnu, gnu), and
indeed all other foo_no_systems variables are compared against
DEB_TARGET_GNU_SYSTEM.

This was making the hurd-i386 build get stuck while building m2, the
attached patch fixes it.

Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.7.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.
diff --git a/debian/rules.defs b/debian/rules.defs
index 2810b233..1bb4b96e 100644
--- a/debian/rules.defs
+++ b/debian/rules.defs
@@ -989,7 +989,7 @@ endif
 ifneq (,$(filter $(DEB_TARGET_ARCH),$(go_no_cpus)))
   with_go := disabled for arch $(DEB_TARGET_ARCH)
 endif
-ifneq (,$(findstring $(DEB_TARGET_ARCH_OS),$(go_no_systems)))
+ifneq (,$(findstring $(DEB_TARGET_GNU_SYSTEM),$(go_no_systems)))
   with_go := disabled for system $(DEB_TARGET_GNU_SYSTEM)
 endif
 ifeq ($(go_no_cross)-$(DEB_CROSS),yes-yes)
@@ -1189,7 +1189,7 @@ n2_no_systems = gnu
 ifneq (,$(filter $(DEB_TARGET_ARCH_CPU),$(m2_no_cpus)))
   with_m2 := disabled for cpu $(DEB_TARGET_ARCH_CPU)
 endif
-ifneq (,$(findstring $(DEB_TARGET_ARCH_OS),$(m2_no_systems)))
+ifneq (,$(findstring $(DEB_TARGET_GNU_SYSTEM),$(m2_no_systems)))
   with_m2 := disabled for system $(DEB_TARGET_GNU_SYSTEM)
 endif
 ifeq ($(m2_no_cross)-$(DEB_CROSS),yes-yes)

Bug#1063642: gcc-13: Fix disabling go and m2 according to OS

[Samuel Thibault]

Package: gcc-13
Version: 13.2.0-13
Severity: important
Tags: patch
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

There was a typo in rules.defs concerning go_no_systems and
m2_no_systems: they are still compared against DEB_TARGET_ARCH_OS,
while their content is gnu-system-like (kfreebsd-gnu, gnu), and
indeed all other foo_no_systems variables are compared against
DEB_TARGET_GNU_SYSTEM.

This was making the hurd-i386 build get stuck while building m2, the
attached patch fixes it.

Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.7.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages gcc-13 depends on:
ii  binutils 2.42-2
ii  gcc-13-base  13.2.0-13
ii  gcc-13-x86-64-linux-gnu  13.2.0-13

Versions of packages gcc-13 recommends:
ii  libc6-dev  2.37-15~deb13u1

Versions of packages gcc-13 suggests:
ii  gcc-13-doc   13.2.0-1
pn  gcc-13-locales   
ii  gcc-13-multilib  13.2.0-13

-- no debconf information

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.
diff --git a/debian/rules.defs b/debian/rules.defs
index 8638be92..4fa62c62 100644
--- a/debian/rules.defs
+++ b/debian/rules.defs
@@ -1019,41 +1019,41 @@ endif
 
 go_no_cpus := arc avr hppa loong64
 go_no_cpus += m68k # See PR 79281 / PR 83314
 go_no_systems := kfreebsd-gnu
 ifneq (,$(filter $(distrelease),precise))
   go_no_cpus  = armhf
   go_no_archs = armhf
 endif
 # Debian #969221
 go_no_cpus  += sh4
 go_no_archs += sh4
 
 ifneq ($(with_base_only),yes)
   ifneq ($(separate_lang),yes)
 with_go := yes
   endif
 endif
 ifneq (,$(filter $(DEB_TARGET_ARCH_CPU),$(go_no_cpus)))
   with_go := disabled for cpu $(DEB_TARGET_ARCH_CPU)
 endif
-ifneq (,$(findstring $(DEB_TARGET_ARCH_OS),$(go_no_systems)))
+ifneq (,$(findstring $(DEB_TARGET_GNU_SYSTEM),$(go_no_systems)))
   with_go := disabled for system $(DEB_TARGET_GNU_SYSTEM)
 endif
 ifneq (,$(filter $(DEB_TARGET_ARCH),$(go_no_archs)))
   with_go := disabled for architecture $(DEB_TARGET_ARCH)
 endif
 ifeq ($(go_no_cross)-$(DEB_CROSS),yes-yes)
   with_go := disabled for cross compiler package
 endif
 ifeq ($(DEB_STAGE)-$(filter libgo, $(with_rtlibs)),rtlibs-)
   with_go := disabled for rtlibs stage
 endif
 with_go := $(call envfilt, go, , , $(with_go))
 
 # Build all packages needed for Go development
 ifneq (,$(findstring gcc, $(PKGSOURCE)))
   ifeq ($(with_go),yes)
 ifeq ($(with_dev),yes)
   with_godev := yes
 endif
 with_libgo := yes
@@ -1262,41 +1262,41 @@ endif
 with_objcxx := $(call envfilt, obj-c++, , c++ objc, $(with_objcxx))
 
 ifeq ($(with_objcxx),yes)
   enabled_languages += obj-c++
 endif
 
 # Modula-2 ---
 m2_no_cross := yes
 m2_no_cross := no
 
 ifneq ($(with_base_only),yes)
   ifneq ($(separate_lang),yes)
 with_m2 := yes
   endif
 endif
 m2_no_cpus = loong64 powerpc ppc64 sh4
 m2_no_systems = gnu kfreebsd-gnu
 ifneq (,$(filter $(DEB_TARGET_ARCH_CPU),$(m2_no_cpus)))
 with_m2 := disabled for cpu $(DEB_TARGET_ARCH_CPU)
 endif
-ifneq (,$(findstring $(DEB_TARGET_ARCH_OS),$(m2_no_systems)))
+ifneq (,$(findstring $(DEB_TARGET_GNU_SYSTEM),$(m2_no_systems)))
   with_m2 := disabled for system $(DEB_TARGET_GNU_SYSTEM)
 endif
 ifeq ($(m2_no_cross)-$(DEB_CROSS),yes-yes)
   with_m2 := disabled for cross compiler package
 endif
 ifeq ($(DEB_STAGE)-$(filter libgm2, $(with_rtlibs)),rtlibs-)
   with_m2 := disabled for rtlibs stage
 endif
 ifneq (,$(filter $(distrelease),precise))
   with_m2 := disabled for $(distrelease) backport
 endif
 #with_m2 := disabled for GCC 13
 
 with_m2 := $(call envfilt, m2, , , $(with_m2))
 
 # Build all packages needed for Modula-2 development
 ifeq ($(with_m2),yes)
   ifeq ($(with_dev),yes)
 with_m2dev := yes
   endif

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le jeu. 08 févr. 2024 08:12:51 +0100, a ecrit:
> So this is missing entries for /etc/hurd/runsystem.sysv. The hurd.postinst 
> only
> sets up the entry for runsystem.gnu at priority 5. So I looked at the
> initscripts.postinst from sysvinit and that one also calls "uname" :D

Ah, ok, that's it indeed.

> > > If I do that I get:
> > > 
> > > /usr/libexec/runsystem.hurd: 117: Pipe call failed
> > 
> > You are probably also missing /servers/socket/1
> 
> No, I have that. Check here:
> 
> $ curl --silent https://people.debian.org/~josch/hurd.tar.xz | unxz | tar tv 
> | grep /servers/socket/
> drwxr-xr-x root/root 0 2024-02-07 17:12 ./servers/socket/
> -rw-r--r-- root/root 0 2024-02-07 17:12 ./servers/socket/1

Actually, checking runsystem.{hurd,sysv} again, it's the converse: they
configure /servers/socket/1 only if it doesn't exist at all. So images
creators are currently expected to either configure it completely, or
not create it at all.

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le jeu. 08 févr. 2024 10:04:51 +0100, a ecrit:
> On 2024-02-08 00:13, Samuel Thibault wrote:
> > Johannes Schauer Marin Rodrigues, le jeu. 08 févr. 2024 00:04:25 +0100,
> > a ecrit:
> > > I'm probably missing more customizations to make this work. Where
> > > else other
> > > than in debootstrap should I look? Maybe the Debian installer is doing
> > > something funny? Maybe there is a hurd-specific udeb that does some
> > > setup?
> > 
> > There shouldn't be much more left than the /servers/socket/1 piece.
> 
> Maybe I found the missing bit. In debootstrap, there is:
> 
> in_target /usr/lib/hurd/setup-translators -k

This cannot be done on Linux.

> settrans -a "$TARGET/dev" /hurd/firmlink /dev
> settrans -a "$TARGET/servers" /hurd/firmlink /servers
> settrans -a "$TARGET/proc" /hurd/firmlink /proc

This is only useful when running as chroot (think of it as the bind
mount that you'd do in Linux)

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le jeu. 08 févr. 2024 00:04:25 +0100, a ecrit:
> Quoting Samuel Thibault (2024-02-07 23:32:34)
> > > So I manually created the empty files /servers/exec, /servers/startup and
> > > /dev/console as it is done by debootstrap here:
> > > 
> > > https://sources.debian.org/src/debootstrap/1.0.134/functions/?hl=1304#L1304
> > 
> > That's needed, yes.
> 
> Could/should those be created by a postinst maintainer script of a package in
> the essential set? Maybe by the hurd package?

It used to be set by scripts but we can probably make the hurd postinst
create them, yes. But is the hurd postinst actually run in your case?

> > > /usr/libexec/runsystem.hurd: line 129: /usr/libexec/rc: No such file
> > 
> > Not sure how your system looks like exactly. One issue we have is that
> > the debian-kosher way to run things is not the same as the hurd upstream
> > way to run things. Normally what happens is:
> > 
> > startup/startup.c's `tries' array starts with LIBEXECDIR "/runsystem",
> > i.e. /usr/libexec/runsystem, which symlinks to /etc/hurd/runsystem,
> > which symlinks to /etc/alternatives/runsystem, which symlinks to
> > /etc/hurd/runsystem.sysv, which doesn't look at /usr/libexec/rc at all.
> > All of this is supposed to be shipped by the hurd package, either from
> > the tarball or as an alternative, not sure why (I guess) your alternative is
> > not being set?
> 
> The symlink chain is this one:
> 
> /usr/libexec/runsystem -> /etc/hurd/runsystem -> /alternatives/runsystem -> 
> /etc/hurd/runsystem.gnu
> 
> Should that last one be linking to /etc/hurd/runsystem.sysv?

Yes, I don't see why it's not doing that, isn't the alternative like
this?

  SelectionPath  Priority   Status

* 0/etc/hurd/runsystem.sysv   10auto mode
  1/etc/hurd/runsystem.gnu5 manual mode
  2/etc/hurd/runsystem.sysv   10manual mode

> If I do that I get:
> 
> /usr/libexec/runsystem.hurd: 117: Pipe call failed

You are probably also missing /servers/socket/1

> My final goal is to have debvm-run (which is just a wrapper around mmdebstrap)
> create a disk image that can be run with debvm-run (which is just a wrapper
> around qemu). I think it would be really cool if in Hurd-related bug reports
> one could just say "to reproduce this hurd problem locally, just run this".

Yes, clearly (though we have already been providing hurd qemu images for
a long time without that many people actually using the recommended
qemu-based way to start them...)

> I'm probably missing more customizations to make this work. Where else other
> than in debootstrap should I look? Maybe the Debian installer is doing
> something funny? Maybe there is a hurd-specific udeb that does some setup?

There shouldn't be much more left than the /servers/socket/1 piece.

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le mer. 07 févr. 2024 18:07:21 +0100, a ecrit:
> if [ "$(uname -s)" = "Linux" ]; then
>   update-alternatives --install /usr/bin/pager pager /bin/more 50 \
>   --slave /usr/share/man/man1/pager.1.gz pager.1.gz \
>   /usr/share/man/man1/more.1.gz
> fi
> 
> And in unshare mode, uname -s prints "Linux" because I'm running this on 
> linux.
> Do you happen to know what this conditional is for on non-linux systems?

util-linux doesn't seem to ship /bin/more on non-linux.  Upstream indeed
added UL_REQUIRES_LINUX([more]) since the addition of using signalfd,
which is Linuxish.

> So I hacked around that by replacing /bin/uname with a shell script that 
> prints
> something that is not "Linux". And then it works! I'm now stuck elsewhere. 
> When
> putting everything into a disk image and attempting to boot it, I get the
> problem that was discussed here:
> 
> https://lists.debian.org/debian-hurd/2007/09/msg00073.html
> 
> So I manually created the empty files /servers/exec, /servers/startup and
> /dev/console as it is done by debootstrap here:
> 
> https://sources.debian.org/src/debootstrap/1.0.134/functions/?hl=1304#L1304

That's needed, yes.

> The next error I'm getting is:
> 
> /usr/libexec/console-run: /dev/console: Not a terminal

That one is just a warning.

> /usr/libexec/runsystem.hurd: line 129: /usr/libexec/rc: No such file

Not sure how your system looks like exactly. One issue we have is that
the debian-kosher way to run things is not the same as the hurd upstream
way to run things. Normally what happens is:

startup/startup.c's `tries' array starts with LIBEXECDIR "/runsystem",
i.e. /usr/libexec/runsystem, which symlinks to /etc/hurd/runsystem,
which symlinks to /etc/alternatives/runsystem, which symlinks to
/etc/hurd/runsystem.sysv, which doesn't look at /usr/libexec/rc at all.
All of this is supposed to be shipped by the hurd package, either from
the tarball or as an alternative, not sure why (I guess) your
alternative is not being set?

Samuel

Re: hurd-amd64 is missing from wanna-build graphs

[Samuel Thibault]

Amos Jeffries, le jeu. 01 févr. 2024 20:46:27 +1300, a ecrit:
> On 1/02/24 11:07, Samuel Thibault wrote:
> > marius, le mer. 31 janv. 2024 19:05:07 +, a ecrit:
> > > I noticed that hurd-amd64 is mssing from wanna-build graphs, is this 
> > > something that we want? If so i can propose a patch to the wb team
> > 
> > Since the hurd-amd64 buildd is not unleashed yet, it won't be useful
> > anyway :) better wait for the buildd to actually have started, rather
> > than getting bad press because nothing seems to be happening.
> 
> On that topic, how are things going with the buildd?
>  Is there a TODO list we can track and try to assist with?

This is just waiting for fixing the shell output pipe replacement
issue:

https://lists.gnu.org/archive/html/bug-hurd/2023-10/msg00062.html

Samuel

Re: hurd-amd64 is missing from wanna-build graphs

[Samuel Thibault]

Hello,

marius, le mer. 31 janv. 2024 19:05:07 +, a ecrit:
> I noticed that hurd-amd64 is mssing from wanna-build graphs, is this 
> something that we want? If so i can propose a patch to the wb team

Since the hurd-amd64 buildd is not unleashed yet, it won't be useful
anyway :) better wait for the buildd to actually have started, rather
than getting bad press because nothing seems to be happening.

Samuel

Re: Give-back request davix 0.8.5-1+b1 gnu-hurd

[Samuel Thibault]

Hello,

Mattias Ellert, le lun. 29 janv. 2024 08:53:44 +0100, a ecrit:
> The build of davix 0.8.5-1+b1 failed on hurd-i386 due to bug 1061610,
> which is a bug in debhelper 13.12. Could you retry the build with
> debhelper 13.13?
> 
> https://buildd.debian.org/status/package.php?p=davix
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1061610

Done so, thanks for the notice!

Samuel

Re: dhcpcd: FTBFS on Hurd

[Samuel Thibault]

Joshua Branson, le mer. 17 janv. 2024 12:09:30 -0500, a ecrit:
> Also may I ask why Debian is switching to dhcpcd?  Just curious.

Because ISC is not maintaining its dhcp software any more.

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le jeu. 11 janv. 2024 00:12:09 +0100, a ecrit:
> The util-linux problem is no surprise because less fails to install when
> investigating that issue I noticed the version of less is 487 which is the
> version from old-old-stable. Is that plausible?

It has been failing to build for a long time already and is still
waiting for somebody to contribute a fix, see
https://buildd.debian.org/status/logs.php?pkg=less=hurd-i386

Samuel

Re: 64bit startup

[Samuel Thibault]

Luca, le sam. 06 janv. 2024 00:42:35 +0100, a ecrit:
> Il 05/01/24 19:12, Sergey Bugaev ha scritto:
> > /servers/crash-dump-core crashes on the memset () call in
> > hurd:exec/elfcore.c:fetch_thread_fpregset (); the (*fpregs) pointer is
> > NULL. The caller passes fpregs = _fpreg, where note.data
> > is of type struct elf_lwpstatus, defined in hurd:include/sys/procfs.h,
> > whose pr_fpreg field is of type prfpregset_t, which is a typedef to
> > fpregset_t, which was an actual struct on i386, but is a pointer on
> > x86_64. This would've been easier to debug if I had debuginfo :)
> 
> I had this small patch applied that apparently is enough for me to have some
> kind of core dump, I'm not sure if it's a good solution:

You probably rather want to fix fetch_thread_fpregset, so as to properly
put the floating state into pr_fpreg.

This probably needs to actually copy over explicit fields, but that's
what we need anyway.

> diff --git a/exec/elfcore.c b/exec/elfcore.c
> index c6aa2bc8b..405fa8e0c 100644
> --- a/exec/elfcore.c
> +++ b/exec/elfcore.c
> @@ -544,6 +544,11 @@ dump_core (task_t task, file_t file, off_t corelimit,
>note.data.pr_info.si_code = sigcode;
>note.data.pr_info.si_errno = sigerror;
> 
> +#ifdef __x86_64__
> +  struct _libc_fpstate fpstate;
> +  memset(, 0, sizeof(fpstate));
> +  note.data.pr_fpreg = 
> +#endif
>fetch_thread_regset (threads[i], _reg);
>fetch_thread_fpregset (threads[i], _fpreg);
> 
> 
> HTH
> Luca

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le ven. 05 janv. 2024 21:12:48 +0300, a ecrit:
> Also I can't help but notice that the hurd package (i.e the translator
> binaries) is still not being built as PIE,

This is not actually specific to 64bit. This was set explicitly in 2016
in debian/rules, tests welcome to check whether building the package
without it works now.

Samuel

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le ven. 05 janv. 2024 21:12:48 +0300, a ecrit:
> I'm not seeing hurd-dbg / hurd-libs-dbg packages in your repo.

Yes, my repo is built from the rebootstrap scripts, which drop debug
etc. only for creating a booting system.

For proper packages, use the usual deb.debian.org debian-ports mirror.

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le jeu. 04 janv. 2024 08:57:51 +0100, a ecrit:
> Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> > perhaps I need to try two of them in parallel and some I/O-heavy
> > workload in the background, as you're saying.
> 
> Yes, that's needed to raise the probability of the bug.
> 
> > Could it be that the two strings are actually different (something
> > being wrong with pipes perhaps)?
> 
> I tried
> 
> A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; done 
> ; echo $A
> 
> The output is empty. But yes, that could be some missing flush or such
> in pipes.

It happens with dash too.

Samuel

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le ven. 05 janv. 2024 12:06:13 +0300, a ecrit:
> > I use
> >
> > while true ; do apt install --reinstall wdiff ; done
> 
> That did it! I can now reliably reproduce this.
> 
> (I assume you don't mind my box constantly banging on your repo.)

It's people.debian.org, it's meant for this :)

You can probably also pass to apt an option to keep the donwloaded .deb

> > got acd :)
> 
> In the six times that I've reproduced it so far, I got "acd" in all cases. 
> Hmmm.

"interesting" :)

Samuel

Re: 64bit startup

[Samuel Thibault]

Hello,

Sergey Bugaev, le jeu. 04 janv. 2024 22:21:11 +0300, a ecrit:
> On Thu, Jan 4, 2024 at 10:57 AM Samuel Thibault  
> wrote:
> > Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> > > perhaps I need to try two of them in parallel and some I/O-heavy
> > > workload in the background, as you're saying.
> >
> > Yes, that's needed to raise the probability of the bug.
> 
> I'm still unable to reproduce this, it's been running for 10+ hours at
> this point. That's two copies of it, and unrelated activity in the
> background.

Which kind of activity? I use 

while true ; do apt install --reinstall wdiff ; done

> > > Could it be that the two strings are actually different (something
> > > being wrong with pipes perhaps)?
> >
> > I tried
> >
> > A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; 
> > done ; echo $A
> >
> > The output is empty. But yes, that could be some missing flush or such
> > in pipes.
> 
> Try
> 
> A=abcd ; time while /usr/bin/\[ "$A" = abcd ] ; do A="$(echo -n `echo
> a``echo b`)$(echo -n `echo c``echo d`)" ; done ; echo $A
> 
> perhaps?

got acd :)

I'll be testing with dash over the next hours.

Samuel

Re: Bug#1059986: dpkg: Please add hurd-arm64 case

[Samuel Thibault]

Hello,

Guillem Jover, le jeu. 04 janv. 2024 20:23:02 +0100, a ecrit:
> but even though I've seen already some toolchain patches flying by,
> AFAIUI there's still no GNU Mach support, so I think I'd prefer to
> wait until that materializes,

Ok.

> as per the FAQ entry on new ports. I don't think this would block
> anything right now anyway, no?

I've hacked by chroot to add the arch to be able to build packages
already ;)

I was mostly anticipating this piece of the port which is needed for
proper set up of most of the rest :)

Samuel

Bug#1059986: dpkg: Please add hurd-arm64 case

[Samuel Thibault]

Package: dpkg
Version: 1.22.2
Severity: normal
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

aarch64-gnu support is coming too :)

Could you add a hurd-amd64 case in dpkg?

Thanks,
Samuel

-- Package-specific info:
This system uses merged-usr-via-aliased-dirs, going behind dpkg's
back, breaking its core assumptions. This can cause silent file
overwrites and disappearances, and its general tools misbehavior.
See .

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.6.0 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dpkg depends on:
ii  libbz2-1.0   1.0.8-5+b1
ii  libc62.37-13
ii  liblzma5 5.4.5-0.1
ii  libmd0   1.1.0-1
ii  libselinux1  3.5-1+b1
ii  libzstd1 1.5.5+dfsg2-2
ii  tar  1.34+dfsg-1.3
ii  zlib1g   1:1.3.dfsg-3

dpkg recommends no packages.

Versions of packages dpkg suggests:
ii  apt2.7.6
pn  debsig-verify  

-- Configuration Files:
/etc/logrotate.d/dpkg changed [not included]

-- no debconf information

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> perhaps I need to try two of them in parallel and some I/O-heavy
> workload in the background, as you're saying.

Yes, that's needed to raise the probability of the bug.

> Could it be that the two strings are actually different (something
> being wrong with pipes perhaps)?

I tried

A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; done ; 
echo $A

The output is empty. But yes, that could be some missing flush or such
in pipes.

Samuel

Re: 64bit startup

[Samuel Thibault]

Luca, le mer. 03 janv. 2024 20:07:00 +0100, a ecrit:
> Il 03/01/24 09:17, Sergey Bugaev ha scritto:
> > How are you running it? Should I still be using a ramdisk image and
> > not rumpdisk?
> 
> Recently I've been installing hurd-amd64 on another disk of my hurd-i386 vm
> and booting from that. Basically I prepare the disk with debootstrap
> --foreign, then I reuse the i386 grub install to boot the 64 bit kernel with
> a custom entry, then run the --second stage, configure login, fstab and
> network and reboot. I can give you the exact commands and setup I'm using if
> you want (I need to reinstall it anyway due to latest changes),

It could be useful to merge information into the wiki page.

Samuel

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le mer. 03 janv. 2024 11:17:53 +0300, a ecrit:
> I guess this is where I ask (consistent with the subject line) about
> how I would run the x86_64 system (to reproduce & debug this).

You probably want to start with the pre-built images I have linked from
the wiki page.

> I've tried debootstrapping from
> https://people.debian.org/~sthibault/tmp/hurd-amd64 as the wiki page
> says; but that doesn't proceed beyond the rumpdisk. Rumpdisk just sits
> there, slowly spitting out logs;

Does it detect disks? What qemu parameters are you using?

I'm using a mere 

kvm -M q35 -drive file=disk-amd64.img -m 1

Samuel

Re: 64bit startup

[Samuel Thibault]

Hello,

I'm still stuck without being able to start packages building for
hurd-amd64 due to this unreliability.

Sergey Bugaev, le mar. 31 oct. 2023 10:09:17 +0300, a ecrit:
> On Mon, Oct 30, 2023 at 1:27 AM Samuel Thibault  
> wrote:
> > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> >
> > by running two of them in parallel, along with an apt install loop in
> > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > 3...)
> 
> This could be [0], considering [ is a Bash built-in and not /bin/[, so
> it's Bash that both compares strings and receives SIGCHLDs.
> 
> [0]: https://lists.gnu.org/archive/html/bug-hurd/2023-06/msg00105.html

I tried

time while /usr/bin/\[ "$(echo -n `echo a` )" = a ] ; do : ; done

with the same result.

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le sam. 25 nov. 2023 00:17:35 +0100, a ecrit:
> Quoting Matthias Geiger (2023-11-24 23:56:11)
> > I see. I was merely wondering if there was an easy way to test hurd builds
> > locally without having to run a VM,
> 
> If you want to compile software, then ideally, no binaries of the host
> architecture (the architecture you are building for) are being run. So if you
> just want to build hurd software (and not run it) then all you need is "just" 
> a
> cross compiler for hurd (should be named gcc-i686-gnu) but I'm not aware of
> that being worked on right now?

Indeed. The rebootstrap scripts build one, though.

Samuel

Re: Patch series to avoid message resizing for x86_64 (v2)

[Samuel Thibault]

Hello,

Samuel Thibault, le dim. 17 déc. 2023 23:53:56 +0100, a ecrit:
> Flavio Cruz, le jeu. 14 déc. 2023 01:02:27 -0500, a ecrit:
> > Sending the updated patch series with the warnings fixed. The only
> > difference is in the glibc patch (added a cast when calling
> > clean_inlined_ports) and the MiG patch, which required a 3 line change
> > to add appropriate casts from mach_port_name_inlined_t* to mach_port_t*
> > in server.c.
> 
> Thanks for the update!
> 
> I confirm it now builds fine with -Werror, I have pushed the whole
> series.

I have uploaded the updated hurd-amd64 packages.
Since this is a completely incompatible change, people will probably
want to just re-generate their hurd-amd64 system if they already have
one. If you really don't want to re-generate, you can mount your
hurd-amd64 filesystem into another OS, and run:

cd path/to/hurd-amd64
for i in ~/hurd*_hurd-amd64.deb ~/libc*3.28*_hurd-amd64.deb ; do
  ar p $i data.tar.xz | tar xJh
done

tar's h option is very important, to avoid overwriting the lib, bin and
sbin symlinks.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Wojciech (Voitek) Aniszewski, le jeu. 07 déc. 2023 14:23:35 +0100, a ecrit:
> wojciech@saiph:~$ showtrans /servers/shutdown
> /hurd/shutdown
> wojciech@saiph:~$ showtrans /servers/acpi
> /hurd/acpi
> wojciech@saiph:~$ showtrans ls /servers/acpi/tables
> APIC   BOOT  FACP  MCFG  SSDT  SSDT  SSDT  TCPA
> 'ASF!' ECDT  HPET  SLIC  SSDT  SSDT  SSDT
> 
> dunno what I'm looking at unfortunately,

To tell the real truth, I don't know what these tables are either.

> docs needed.

ACPI is documented, that's completely independent from the Hurd.

That being said, start from the start: the shutdown trigger is performed
by the shutdown translator, in shutdown/shutdown.c, which calls into the
acpi translator. Then it's a matter of following the function calls, put
mach_print() calls along the way, and check what is actually happening.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le jeu. 07 déc. 2023 11:31:44 +0200, a ecrit:
> On Thu, Dec 7, 2023 at 11:26 AM Samuel Thibault  wrote:
> >
> > Martin-Éric Racine, le jeu. 07 déc. 2023 08:16:34 +0200, a ecrit:
> > > On Thu, Dec 7, 2023 at 2:13 AM Samuel Thibault  
> > > wrote:
> > > > Martin-Éric Racine, le mar. 05 déc. 2023 10:48:57 +0200, a ecrit:
> > > > > On Mon, Dec 4, 2023 at 1:48 PM Samuel Thibault  
> > > > > wrote:
> > > > > > Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> > > > > > > ACPI support. I noticed during bootup that an ACPI server is 
> > > > > > > launched,
> > > > > > > but issuing "exec sudo poweroff" merely halts the system; it 
> > > > > > > doesn't
> > > > > > > send an ACPI poweroff at the end of the shutdown process.
> > > > > > >
> > > > > > > Is there any way to enable this or is ACPI poweroff merely not
> > > > > > > supported by Hurd?
> > > > > >
> > > > > > It *is* supported and works for me. There is nothing particular to 
> > > > > > do to
> > > > > > get it.
> > > > > >
> > > > > > Is the acpi translator perhaps dying at some point?
> > > > > >
> > > > > > Are you running hurd-i386 or hurd-amd64?
> > > > >
> > > > > As far as I can tell, pci-arbiter succesfully launches acpi on bootup
> > > > > and terminates it during shutdown.
> > > >
> > > > Just to make sure, does
> > > >
> > > > showtrans /servers/shutdown
> > > >
> > > > tell you /hurd/shutdown? and
> > > >
> > > > showtrans /servers/acpi
> > > >
> > > > tell you /hurd/acpi? and /servers/acpi/tables/ contains some tables?
> > >
> > > [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ showtrans /servers/shutdown
> > > /hurd/shutdown
> > > [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ showtrans /servers/acpi
> > > /hurd/acpi
> > > [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ ls /servers/acpi/tables/
> > > APIC  FACP  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT
> > > [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$
> >
> > Ok, then I don't any immediate idea, this needs investigation on your
> > system.
> 
> Possibly.  The thing is, given how Hurd remains a sketchily documented
> OS,

?? There are plenty of wiki pages, and the whole source code is just
there to be looked at. The shutdown translator is 160 lines long and
will tell you which RPC it's trying to do to shut down the machine. From
there you have everything.

Read the source, Luke.

This was said to people 20-30 years ago when the free software movement
deployed, this is just exactly the same now, for exactly the same
reasons.

> I wouldn't remotely know where to look or using what tools.

mach_print("foobar\n"); is a very powerful tool, for a start, to know
what is actually happening.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le jeu. 07 déc. 2023 08:16:34 +0200, a ecrit:
> On Thu, Dec 7, 2023 at 2:13 AM Samuel Thibault  wrote:
> > Martin-Éric Racine, le mar. 05 déc. 2023 10:48:57 +0200, a ecrit:
> > > On Mon, Dec 4, 2023 at 1:48 PM Samuel Thibault  
> > > wrote:
> > > > Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> > > > > ACPI support. I noticed during bootup that an ACPI server is launched,
> > > > > but issuing "exec sudo poweroff" merely halts the system; it doesn't
> > > > > send an ACPI poweroff at the end of the shutdown process.
> > > > >
> > > > > Is there any way to enable this or is ACPI poweroff merely not
> > > > > supported by Hurd?
> > > >
> > > > It *is* supported and works for me. There is nothing particular to do to
> > > > get it.
> > > >
> > > > Is the acpi translator perhaps dying at some point?
> > > >
> > > > Are you running hurd-i386 or hurd-amd64?
> > >
> > > As far as I can tell, pci-arbiter succesfully launches acpi on bootup
> > > and terminates it during shutdown.
> >
> > Just to make sure, does
> >
> > showtrans /servers/shutdown
> >
> > tell you /hurd/shutdown? and
> >
> > showtrans /servers/acpi
> >
> > tell you /hurd/acpi? and /servers/acpi/tables/ contains some tables?
> 
> [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ showtrans /servers/shutdown
> /hurd/shutdown
> [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ showtrans /servers/acpi
> /hurd/acpi
> [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$ ls /servers/acpi/tables/
> APIC  FACP  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT  SSDT
> [2023-12-07 08:14](HURD i386)perkelix@pxeth:~$

Ok, then I don't any immediate idea, this needs investigation on your
system.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le mar. 05 déc. 2023 10:48:57 +0200, a ecrit:
> On Mon, Dec 4, 2023 at 1:48 PM Samuel Thibault  wrote:
> > Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> > > ACPI support. I noticed during bootup that an ACPI server is launched,
> > > but issuing "exec sudo poweroff" merely halts the system; it doesn't
> > > send an ACPI poweroff at the end of the shutdown process.
> > >
> > > Is there any way to enable this or is ACPI poweroff merely not
> > > supported by Hurd?
> >
> > It *is* supported and works for me. There is nothing particular to do to
> > get it.
> >
> > Is the acpi translator perhaps dying at some point?
> >
> > Are you running hurd-i386 or hurd-amd64?
> 
> As far as I can tell, pci-arbiter succesfully launches acpi on bootup
> and terminates it during shutdown.

Just to make sure, does

showtrans /servers/shutdown

tell you /hurd/shutdown? and

showtrans /servers/acpi

tell you /hurd/acpi? and /servers/acpi/tables/ contains some tables?

Samuel

Re: Bug#1057634: /sbin/hwclock: unrecognized option '--rtc=/dev/rtc0'

[Samuel Thibault]

Hello,

Thorsten Glaser, le mer. 06 déc. 2023 12:16:27 +0100, a ecrit:
> On Wed, 6 Dec 2023, Chris Hofstaedtler wrote:
> >I have zero knowledge about hurd, but it looks like[1] hwclock is built
> >with CMOS support on hurd.  So maybe it could work?
> 
> Maybe it just needs a different config?
> 
> >Given this I'd imagine nobody has ever used the hwclock initscripts
> >on hurd before, and maybe they shouldn't exist there? I'd hope hurd
> >wouldn't rely on userspace to do direct hardware access to set the
> >time.
> 
> AIUI Hurd is a microkernel so everything is done in userspace?

Yes, we'd rather set a translator on /dev/rtc, that does the ISA access
for everybody, instead of hwclock doing it (and possibly stepping on top
of another hwclock call). Contribution welcome ;)

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le lun. 04 déc. 2023 14:30:55 +0200, a ecrit:
> On Mon, Dec 4, 2023 at 2:14 PM Samuel Thibault  wrote:
> >
> > Martin-Éric Racine, le lun. 04 déc. 2023 14:08:05 +0200, a ecrit:
> > > > Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> > > > > ACPI support. I noticed during bootup that an ACPI server is launched,
> > > > > but issuing "exec sudo poweroff" merely halts the system; it doesn't
> > > > > send an ACPI poweroff at the end of the shutdown process.
> > > > >
> > > > > Is there any way to enable this or is ACPI poweroff merely not
> > > > > supported by Hurd?
> > > >
> > > > It *is* supported and works for me. There is nothing particular to do to
> > > > get it.
> > >
> > > Interesting.
> > >
> > > > Is the acpi translator perhaps dying at some point?
> > >
> > > What keyword am I supposed to grep in syslog?
> >
> > syslog probably doesn't notice that. But that probably shows up on the
> > mach console.
> 
> Speaking of which: bug #1057397.

(inetutils-syslogd should be working fine)

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le lun. 04 déc. 2023 14:44:54 +0200, a ecrit:
> On Mon, Dec 4, 2023 at 2:40 PM Samuel Thibault  wrote:
> > Martin-Éric Racine, le lun. 04 déc. 2023 14:30:55 +0200, a ecrit:
> > > As for the console, it doesn't show much since it's too busy clearing
> > > the screen while changing the font
> >
> > You can disable the hurd console in /etc/defaults/hurd-console.
> 
> Which then prevents loading the correct keyboard map.

Alternatively, you can put the gnumach console on the com0 serial port.

> > > Speaking of the console, logging in via that doesn't set LANG or
> > > LANGUAGE, unless I missed something?
> >
> > It does set it according to your Debian configuration.
> 
> Nope. /etc/default/locale is correct, but 'locale' shows LANG and
> LANGUAGE unset, while everything else in LC_* has POSIX.

Ok, it does work through ssh but not on the Hurd console (and not via
inetutils-telnetd either). Contribution welcome to fix it.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le lun. 04 déc. 2023 14:30:55 +0200, a ecrit:
> As for the console, it doesn't show much since it's too busy clearing
> the screen while changing the font

You can disable the hurd console in /etc/defaults/hurd-console.

> Speaking of the console, logging in via that doesn't set LANG or
> LANGUAGE, unless I missed something?

It does set it according to your Debian configuration.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Martin-Éric Racine, le lun. 04 déc. 2023 14:08:05 +0200, a ecrit:
> > Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> > > ACPI support. I noticed during bootup that an ACPI server is launched,
> > > but issuing "exec sudo poweroff" merely halts the system; it doesn't
> > > send an ACPI poweroff at the end of the shutdown process.
> > >
> > > Is there any way to enable this or is ACPI poweroff merely not
> > > supported by Hurd?
> >
> > It *is* supported and works for me. There is nothing particular to do to
> > get it.
> 
> Interesting.
> 
> > Is the acpi translator perhaps dying at some point?
> 
> What keyword am I supposed to grep in syslog?

syslog probably doesn't notice that. But that probably shows up on the
mach console.

Samuel

Re: poweroff support on Hurd?

[Samuel Thibault]

Hello,

Martin-Éric Racine, le lun. 04 déc. 2023 12:16:59 +0200, a ecrit:
> ACPI support. I noticed during bootup that an ACPI server is launched,
> but issuing "exec sudo poweroff" merely halts the system; it doesn't
> send an ACPI poweroff at the end of the shutdown process.
> 
> Is there any way to enable this or is ACPI poweroff merely not
> supported by Hurd?

It *is* supported and works for me. There is nothing particular to do to
get it.

Is the acpi translator perhaps dying at some point?

Are you running hurd-i386 or hurd-amd64?

Samuel

Re: keyboard configuration fails

[Samuel Thibault]

Hello,

Martin-Éric Racine, le jeu. 30 nov. 2023 12:23:47 +0200, a ecrit:
> Damn lists.debian's Reply-To... Please keep me in CC.

Ah, sorry, it's not the list's fault, but I told my mailer to avoid Cc
because most people are already subscribed to debian development lists.

> but the Hurd console does (which should work after a reboot).

(or just restarting the hurd-console service)

> This
> apparently fails, too, since debconf data for keyboard-configuration
> (correct for a default Finnish keyboard map) and what I get on the
> console (symbols where the US keyboard has them) don't match.

I just quickly tried within the installer, and things does work as far
as I can tell, for instance the key on the right of 'p' produces 'å'.

I tried to reconfigure keyboard-configuration to enable the Finnish
layout, and after hurd-console restart I did get the same.

Remember: in bug reports, always describe precisely what you did, what
you obtained, and what you expected. Otherwise we have a mail ping-pong
only to manage to find out what actually is going wrongly in your case,
thus a loss of time for everybody.

Samuel

Re: keyboard configuration fails

[Samuel Thibault]

Martin-Éric Racine, le mer. 29 nov. 2023 11:26:07 +0200, a ecrit:
> On Wed, Nov 29, 2023 at 11:22 AM Samuel Thibault  wrote:
> >
> > Martin-Éric Racine, le mer. 29 nov. 2023 01:55:23 +0200, a ecrit:
> > > On Tue, Nov 28, 2023 at 11:13 PM Samuel Thibault  
> > > wrote:
> > > > Martin-Éric Racine, le dim. 26 nov. 2023 16:19:58 +0200, a ecrit:
> > > > > The installer failed to configure the keyboard, which shows with
> > > > > incorrect symbols when typing in my user name at account creation.
> > > >
> > > > Which item did you choose in the boot menu?
> > > >
> > > > Which layout did you select?
> > >
> > > Text install.
> >
> > Text install is the pure gnumach console, which doesn't support keyboard
> > layouts.
> 
> Wait... the console doesn't support keyboard maps AT ALL?!

The Mach console, indeed (actually there is some support for it but it's
quite limited in terms of capacities, and thus we don't use it).

The Hurd console (pseudo-graphical) does support it, and uses the xkb
data.

Samuel

Re: keyboard configuration fails

[Samuel Thibault]

Martin-Éric Racine, le mer. 29 nov. 2023 01:55:23 +0200, a ecrit:
> On Tue, Nov 28, 2023 at 11:13 PM Samuel Thibault  wrote:
> > Martin-Éric Racine, le dim. 26 nov. 2023 16:19:58 +0200, a ecrit:
> > > The installer failed to configure the keyboard, which shows with
> > > incorrect symbols when typing in my user name at account creation.
> >
> > Which item did you choose in the boot menu?
> >
> > Which layout did you select?
> 
> Text install.

Text install is the pure gnumach console, which doesn't support keyboard
layouts.

Samuel

Re: keyboard configuration fails

[Samuel Thibault]

Hello,

Martin-Éric Racine, le dim. 26 nov. 2023 16:19:58 +0200, a ecrit:
> The installer failed to configure the keyboard, which shows with
> incorrect symbols when typing in my user name at account creation.

Which item did you choose in the boot menu?

Which layout did you select?

Samuel

Bug#1057004: gcc-13: hurd-amd64 support

[Samuel Thibault]

Package: gcc-13
Version: 13.2.0-7
Severity: important
Tags: patch
X-Debbugs-Cc: debian-hurd@lists.debian.org
User: debian-hurd@lists.debian.org
Usertags: hurd

Hello,

The attached patch adds support for hurd-amd64, could you apply it?

The part in hurd-amd64.diff comes from the upstream master commits,
which will thus be included in gcc 14.

The parts in hurd-multiarch.diff and hurd-multilib-multiarch.diff are
for long term, like gcc-multiarch.diff and gcc-multilib-multiarch.diff
for other archs.

Thanks,
Samuel

-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unreleased'), 
(500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 
'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 
'oldoldstable-proposed-updates'), (500, 'oldoldstable'), (500, 
'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 
'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, arm64

Kernel: Linux 6.5.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages gcc-13 depends on:
ii  binutils   2.41-7
ii  cpp-13 13.2.0-7
ii  gcc-13-base13.2.0-7
ii  libc6  2.37-12+b1
ii  libcc1-0   13.2.0-7
ii  libgcc-13-dev  13.2.0-7
ii  libgcc-s1  13.2.0-7
ii  libgmp10   2:6.3.0+dfsg-2
ii  libisl23   0.26-3
ii  libmpc31.3.1-1
ii  libmpfr6   4.2.1-1
ii  libstdc++6 13.2.0-7
ii  libzstd1   1.5.5+dfsg2-2
ii  zlib1g 1:1.2.13.dfsg-3

Versions of packages gcc-13 recommends:
ii  libc6-dev  2.37-12+b1

Versions of packages gcc-13 suggests:
ii  gcc-13-doc   13.2.0-1
pn  gcc-13-locales   
ii  gcc-13-multilib  13.2.0-7

-- no debconf information

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.
diff --git a/debian/patches/hurd-amd64.diff b/debian/patches/hurd-amd64.diff
new file mode 100644
index 000..e7288ea
--- /dev/null
+++ b/debian/patches/hurd-amd64.diff
@@ -0,0 +1,127 @@
+commit 5707e9db9c398d311defc80c5b7822c9a07ead60
+Author: Samuel Thibault 
+Date:   Sat May 6 13:50:36 2023 +0200
+
+hurd: Add multilib paths for gnu-x86_64
+
+We need the multilib paths in gcc to find e.g. glibc crt files on
+Debian.  This is essentially based on t-linux64 version.
+
+gcc/ChangeLog:
+
+* config/i386/t-gnu64: New file.
+* config.gcc [x86_64-*-gnu*]: Add i386/t-gnu64 to
+tmake_file.
+
+commit c768917402d4cba69a92c737e56e177f5b8ab0df
+Author: Samuel Thibault 
+Date:   Sat May 6 13:55:44 2023 +0200
+
+hurd: Ad default-pie and static-pie support
+
+This fixes the Hurd spec in the default-pie case, and adds static-pie
+support.
+
+gcc/ChangeLog:
+
+* config/i386/gnu.h: Use PIE_SPEC, add static-pie case.
+* config/i386/gnu64.h: Use PIE_SPEC, add static-pie case.
+
+diff --git a/src/gcc/config.gcc b/src/gcc/config.gcc
+index 3000379cafc..e62849c1230 100644
+--- a/src/gcc/config.gcc
 b/src/gcc/config.gcc
+@@ -5973,6 +5973,9 @@ case ${target} in
+   visium-*-*)
+   target_cpu_default2="TARGET_CPU_$with_cpu"
+   ;;
++  x86_64-*-gnu*)
++  tmake_file="$tmake_file i386/t-gnu64"
++  ;;
+ esac
+ 
+ t=
+diff --git a/src/gcc/config/i386/t-gnu64 b/src/gcc/config/i386/t-gnu64
+new file mode 100644
+index 000..23ee6823d65
+--- /dev/null
 b/src/gcc/config/i386/t-gnu64
+@@ -0,0 +1,38 @@
++# Copyright (C) 2002-2023 Free Software Foundation, Inc.
++#
++# This file is part of GCC.
++#
++# GCC is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 3, or (at your option)
++# any later version.
++#
++# GCC is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with GCC; see the file COPYING3.  If not see
++# <http://www.gnu.org/licenses/>.
++
++# On Debian, Ubuntu and other derivative distributions, the 32bit libraries
++# are found in /lib32 and /usr/lib32, /lib64 and /usr/lib64 are symlinks to
++# /lib and /usr/lib, while other distributions install libraries into /lib64
++# and /usr/lib64.  The LSB does not enforce the use of /lib64 and /usr/lib64,
++# it d

Re: Source for grub 2.06-13+hurd.2 ?

[Samuel Thibault]

Hello,

Martin-Éric Racine, le lun. 27 nov. 2023 09:35:19 +0200, a ecrit:
> The Hurd kernel in unstable wants a GRUB version higher than
> 2.06-13+hurd.2 yet neither unstable or experimental have that. Where
> can I find it?

It is available in snapshot:
http://snapshot.debian.org/package/grub2/
http://snapshot.debian.org/package/grub2/2.06-13%2Bhurd.2/
http://snapshot.debian.org/archive/debian-ports/20230701T202046Z/pool-hurd-i386/main/g/grub2/grub2_2.06-13%2Bhurd.2.dsc

Samuel

Re: proc leaking

[Samuel Thibault]

Hello,

Samuel Thibault, le mer. 01 nov. 2023 16:06:57 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 15:35:00 +0100, a ecrit:
> > Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> > > Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > > > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > > > increasing).
> > > > 
> > > > It seems 64bit-specific: the program below makes proc leak memory, 100
> > > > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > > > properly parse messages to be destroyed, so that in the error case the
> > > > server leaks non-inline data? Flavio, perhaps you have an idea?
> > > 
> > > I don't think we have the kernel-to-user equivalent for
> > > adjust_msg_type_size? So that we end up pushing twice too much data to
> > > userland for port arrays?
> > 
> > I found and fixed the allocation issue in the kernel.
> 
> It seems proc is still leaking, but on the heap this time. This is not
> 64bit-specific, the same simple reproducer triggers it:
> 
> while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> 
> or more simply:
> 
> while true ; do echo $(echo -n $(echo a)) > /dev/null ; done

I found the issue, it's because of the quiescence support in libports,
which assumes that all threads will sooner or later go through a
quiescent state (because it finished processing a message). But that's
not true, proc doesn't set a thread timeout, and thus some threads can
stay indefinitely stuck in receiving messages. And thus the deferred
dereferencing used by ports_destroy_right never gets achieved.

I'll push a fix.

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le dim. 19 nov. 2023 11:57:07 +0100, a ecrit:
> The command above and the advice to extract ext2fs.static and exec.static
> should be put on some wiki page, I think.

It's the eternal problem with documentation: sometimes it doesn't even
exist, sometimes people don't find it. I have now added more keywords.

https://darnassus.sceen.net/~hurd-web/hurd/running/qemu/#index8h1

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Johannes Schauer Marin Rodrigues, le dim. 19 nov. 2023 11:57:07 +0100, a ecrit:
> Would you also have handy what argument I'd have to pass to connect the serial
> terminal with the login tty to QEMU's serial device so that I can run this
> without graphic mode?

You can add in -append: console=com0

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le dim. 19 nov. 2023 07:45:16 +0100, a ecrit:
> So I downloaded
> https://cdimage.debian.org/cdimage/ports/stable/hurd-i386/debian-hurd.img.tar.gz
> and extracted /boot/gnumach-1.8-486.gz from it. I tried the following but it
> seems to get stuck and only prints "Booting from ROM...":
> 
> qemu-system-i386 \
> -m 1G \
> -kernel gnumach-1.8-486.gz \
> -append root=device:hd0s2 \
> -drive file=debian-hurd-20210812.img,format=raw
> 
> What am I missing so that I can boot Hurd using qemu without grub?

Various things :)

- qemu doesn't seem to properly detect gzipped files, so you need to
  gunzip gnumach first

- apparently, booting gnumach with qemu-system-i386 broke at some point
  without -enable-kvm, so it is needed (and recommended anyway for
  speed), or qemu-system-x86_64

- gnumach itself doesn't contain any ext2fs driver or anything to load
  binaries, so this needs to be loaded like grub would. You can fetch
  /hurd/ext2fs.static and /hurd/exec.static from the image (e.g. into
  /tmp), and add:

--initrd '/tmp/ext2fs.static --multiboot-command-line=${kernel-command-line} 
--host-priv-port=${host-port} --device-master-port=${device-port} 
--exec-server-task=${exec-task} -T typed ${root} $(task-create) 
$(task-resume),/tmp/exec.static $(exec-task=task-create)'

So in the end this works for me:

qemu-system-i386 -enable-kvm -m 1G \
-kernel /tmp/gnumach-1.8-486 \
-append root=device:hd0s2 \
--initrd '/tmp/ext2fs.static 
--multiboot-command-line=${kernel-command-line} --host-priv-port=${host-port} 
--device-master-port=${device-port} --exec-server-task=${exec-task} -T typed 
${root} $(task-create) $(task-resume),/tmp/exec.static 
$(exec-task=task-create)' \
-drive file=debian-hurd-20210812.img,format=raw

Samuel

Re: creating Hurd chroots on Linux using DPKG_ROOT chrootless mode

[Samuel Thibault]

Hello,

Johannes Schauer Marin Rodrigues, le sam. 18 nov. 2023 11:04:06 +0100, a ecrit:
> But even with those patches there is another blocker. The hurd package throws
> the following error in its preinst maintainer script:
> 
> ERROR: This version of the GNU Hurd requires kernel version
> 1.8+git20210923 or later.
> Please upgrade your kernel and reboot before installing hurd.
> 
> So the hurd package must be patched so that this check is not done if
> $DPKG_ROOT is a non-empty string. Can you take care of preparing that patch 
> and
> testing it?

Oh, indeed, that's an OS-specific thing that shouldn't be done on
cross-execution :)

I have commited the attached (untested) patch to the debian hurd repo.

Samuel
commit 8692df6707fd5d8e2533dd90bc740d422b02584f
Author: Samuel Thibault 
Date:   Sat Nov 18 11:30:35 2023 +0100

 hurd.preinst: Disable kernel version check when $DPKG_ROOT is not empty

diff --git a/debian/hurd.preinst b/debian/hurd.preinst
index 68e577355..5ac0a9adf 100644
--- a/debian/hurd.preinst
+++ b/debian/hurd.preinst
@@ -24,9 +24,10 @@ kernel_ver_git=${kernel_ver_git%%-*}
 kernel_ver=${kernel_ver%+git*}
 kernel_ver_min=1.8
 kernel_ver_git_min=20210923
-if kernel_compare_versions "$kernel_ver" lt $kernel_ver_min || \
- ( kernel_compare_versions "$kernel_ver" eq $kernel_ver_min && \
-[ "$kernel_ver_git" -lt $kernel_ver_git_min ] )
+if [ -z "$DPKG_ROOT" ] && \
+ ( kernel_compare_versions "$kernel_ver" lt $kernel_ver_min || \
+   ( kernel_compare_versions "$kernel_ver" eq $kernel_ver_min && \
+   [ "$kernel_ver_git" -lt $kernel_ver_git_min ] ) )
 then
 echo "ERROR: This version of the GNU Hurd requires kernel version"
 echo "$kernel_ver_min+git$kernel_ver_git_min or later."

Re: 64bit startup

[Samuel Thibault]

Hello,

Flávio Cruz, le dim. 05 nov. 2023 23:17:49 -0500, a ecrit:
> On Tue, Oct 31, 2023 at 9:14 PM Samuel Thibault <[1]samuel.thiba...@gnu.org>
> wrote:
> 
> > Realizing only now by reading the __mach_msg_destroy assembly...
> >
> >     unsigned int        msgt_inline : 1,
> >                         msgt_longform : 1,
> >                         msgt_deallocate : 1,
> >                         msgt_name : 8,
> >                         msgt_size : 16,
> >                         msgt_unused : 5;
> >
> > This field ordering makes reading them overly complex... It'll be a pain
> > to rebootstrap, but perhaps we do want to put msgt_size and msgt_name
> > first?
> 
> I only moved msgt_size to the end of the struct

Ok, so we can as well align fields to make bit mangling simpler.

> However, I wonder if it would be simpler to just ask the user land to
> pass port names using the following struct:
> 
> #ifdef KERNEL
> union mach_rpc_port {
>    mach_port_name_t name;
>    mach_port_t kernel_port;
> };
> #else
> struct mach_rpc_port {
>    mach_port_name_t name;
>    int unused;
> };
> #endif
> 
> It would make the kernel simpler since no message resizing is
> necessary

I was thinking about this suggestion today, and I think that'll be
better for the long run indeed. There are questions about holes being
uninitialized, but:

> and most of the code using this would be MiG generated.

indeed.

I'm almost done with the ground set of Debian packages. Will wait until
this is settled before building the hurd-amd64 Debian world :)

Samuel

Re: libssh2 builds on Hurd

[Samuel Thibault]

Hello,

Amos Jeffries, le lun. 06 nov. 2023 23:34:43 +1300, a ecrit:
> On hurd-amd64 there is a successful build of the same code with no sign of
> MAXPATHLEN being problematic.
> 
> AFAIK there has been no change to make that macro meaningful in Hurd. So
> what gives?

It was built with nocheck, to be able to bootstrap the build-dep
entanglements :)

When I'll be done with bootstrapping loops, I'll reschedule bin-NMUs so
that these failures get back coherent with hurd-i386.

Samuel

Re: Clang stack alignment on i386-gnu

[Samuel Thibault]

Sergey Bugaev, le sam. 04 nov. 2023 20:25:13 +0300, a ecrit:
> This piece of code in Clang appears to be the culprit,
> but note that I'm entirely unfamiliar with the LLVM code base:
> 
>   // Stack alignment is 16 bytes on Darwin, Linux, kFreeBSD, NaCl, and for all
>   // 64-bit targets.  On Solaris (32-bit), stack alignment is 4 bytes
>   // following the i386 psABI, while on Illumos it is always 16 bytes.
>   if (StackAlignOverride)
> stackAlignment = *StackAlignOverride;
>   else if (isTargetDarwin() || isTargetLinux() || isTargetKFreeBSD() ||
>isTargetNaCl() || Is64Bit)
> stackAlignment = Align(16);

That's a very probable culprit indeed. You can add an isTargetHurd
definition to llvm/lib/Target/X86/X86Subtarget.h, that calls isOSHurd(),
and add it to the if.

Samuel

Re: proc leaking

[Samuel Thibault]

Sergey Bugaev, le mer. 01 nov. 2023 23:18:01 +0300, a ecrit:
> On Wed, Nov 1, 2023 at 9:17 PM Samuel Thibault  
> wrote:
> > I tracked it a bit, it seems that libport is not always cleaning
> > structures from the proc class. Below is the tracing that we get for
> > instance with the while loop above. Alloc is the allocation of pi, free
> > is the freeing from the point of view of the proc server, and clean is
> > the actual cleanup done by libports.
> >
> > Could it be that proc is overflown with dead port notifications? That's
> > not many procs, but still. Maybe Sergey has an idea?
> 
> I don't think I understood what "freeing from the point of view of the
> proc server" and "actual cleanup done by libports" mean (for one
> thing, proc_class's clean_routine is NULL).

>From the point of view of the proc server = complete_exit was called
Actual cleanup = clean_routing called (I added one just to track it).

The difference between the two is probably some remaining port
reference for whatever reason.

> Perhaps you could post this as a patch to clear things up, and also
> for me to try and reproduce (& debug) this?

Here it is.

Samuel
diff --git a/proc/main.c b/proc/main.c
index 747646ef..8a74c14d 100644
--- a/proc/main.c
+++ b/proc/main.c
@@ -165,6 +165,20 @@ parse_opt (int key, char *arg, struct argp_state *state)
   return 0;
 }
 
+extern long proc_alloc;
+extern long proc_clean;
+extern long proc_free;
+static void clean_proc(void *p)
+{
+   proc_clean++;
+  if ((proc_clean % 100) == 0)
+  {
+ char s[128];
+ snprintf(s, sizeof(s), "proc: alloc %lu free %lu clean %lu\n", 
proc_alloc, proc_free, proc_clean);
+ mach_print(s);
+  }
+}
+
 int
 main (int argc, char **argv, char **envp)
 {
@@ -185,7 +199,7 @@ main (int argc, char **argv, char **envp)
 error (2, 0, "proc server can only be run by startup during boot");
 
   proc_bucket = ports_create_bucket ();
-  proc_class = ports_create_class (0, 0);
+  proc_class = ports_create_class (clean_proc, 0);
   generic_port_class = ports_create_class (0, 0);
   exc_class = ports_create_class (exc_clean, 0);
   ports_create_port (generic_port_class, proc_bucket,
diff --git a/proc/mgt.c b/proc/mgt.c
index f87daeb3..fa19e8d7 100644
--- a/proc/mgt.c
+++ b/proc/mgt.c
@@ -43,6 +43,7 @@
 #include "proc_exc_U.h"
 #include "task_notify_S.h"
 #include 
+#include 
 
 extern mach_msg_return_t
 mach_msg_receive (mach_msg_header_t *msg);
@@ -794,6 +795,10 @@ S_proc_getallpids (struct proc *p,
   return 0;
 }
 
+
+long proc_alloc;
+long proc_clean;
+long proc_free;
 /* Create a process for TASK, which is not otherwise known to us.
The PID/parentage/job-control fields are not yet filled in,
and the proc is not entered into any hash table.  */
@@ -810,6 +815,13 @@ allocate_proc (task_t task)
   err = ports_create_port (proc_class, proc_bucket, sizeof (struct proc), );
   if (err)
 return NULL;
+  proc_alloc++;
+  if ((proc_alloc % 100) == 0)
+  {
+ char s[128];
+ snprintf(s, sizeof(s), "proc: alloc %lu free %lu clean %lu\n", 
proc_alloc, proc_free, proc_clean);
+ mach_print(s);
+  }
 
   memset (>p_pi + 1, 0, sizeof *p - sizeof p->p_pi);
   p->p_task = task;
@@ -1158,6 +1170,13 @@ complete_exit (struct proc *p)
  will shortly vanish (because we are p_dead, those routines do
  nothing).  */
   ports_port_deref (p);
+  proc_free++;
+  if ((proc_free % 100) == 0)
+  {
+ char s[128];
+ snprintf(s, sizeof(s), "proc: alloc %lu free %lu clean %lu\n", 
proc_alloc, proc_free, proc_clean);
+ mach_print(s);
+  }
 }
 
 /* Get the list of all tasks from the kernel and start adding them.

Re: proc leaking

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 16:06:57 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 15:35:00 +0100, a ecrit:
> > Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> > > Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > > > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > > > increasing).
> > > > 
> > > > It seems 64bit-specific: the program below makes proc leak memory, 100
> > > > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > > > properly parse messages to be destroyed, so that in the error case the
> > > > server leaks non-inline data? Flavio, perhaps you have an idea?
> > > 
> > > I don't think we have the kernel-to-user equivalent for
> > > adjust_msg_type_size? So that we end up pushing twice too much data to
> > > userland for port arrays?
> > 
> > I found and fixed the allocation issue in the kernel.
> 
> It seems proc is still leaking, but on the heap this time. This is not
> 64bit-specific, the same simple reproducer triggers it:
> 
> while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> 
> or more simply:
> 
> while true ; do echo $(echo -n $(echo a)) > /dev/null ; done

I tracked it a bit, it seems that libport is not always cleaning
structures from the proc class. Below is the tracing that we get for
instance with the while loop above. Alloc is the allocation of pi, free
is the freeing from the point of view of the proc server, and clean is
the actual cleanup done by libports. I tell proc to print them whenever
one of them crosses a hundred boundary:

proc: alloc 651 free 600 clean 520
proc: alloc 700 free 648 clean 568
proc: alloc 731 free 679 clean 600
proc: alloc 751 free 700 clean 620
proc: alloc 800 free 748 clean 668
proc: alloc 831 free 779 clean 700
proc: alloc 851 free 800 clean 720
proc: alloc 900 free 848 clean 768
proc: alloc 931 free 879 clean 800
proc: alloc 951 free 900 clean 820
proc: alloc 1000 free 948 clean 868
proc: alloc 1031 free 979 clean 900
proc: alloc 1051 free 1000 clean 920
proc: alloc 1100 free 1048 clean 968
[...]
proc: alloc 2251 free 2200 clean 2120
proc: alloc 2300 free 2248 clean 2168
proc: alloc 2331 free 2279 clean 2200
proc: alloc 2351 free 2300 clean 2220
proc: alloc 2400 free 2348 clean 2268
proc: alloc 2431 free 2379 clean 2300
proc: alloc 2451 free 2400 clean 2320
proc: alloc 2500 free 2448 clean 2368
proc: alloc 2551 free 2500 clean 2368
proc: alloc 2600 free 2548 clean 2368
proc: alloc 2651 free 2600 clean 2368
[...]
proc: alloc 3400 free 3348 clean 2368
proc: alloc 3451 free 3400 clean 2368
proc: alloc 3500 free 3448 clean 2368
proc: alloc 3551 free 3500 clean 2368
proc: alloc 3600 free 3548 clean 2368

I.e. after a few seconds point the cleaning stops. I stopped the loop
there, waited a few seconds, and restarted it again, and got:

proc: alloc 3649 free 3597 clean 2400
proc: alloc 3651 free 3600 clean 2402
proc: alloc 3700 free 3648 clean 2450
proc: alloc 3749 free 3697 clean 2500
proc: alloc 3751 free 3700 clean 2502
proc: alloc 3800 free 3748 clean 2550
proc: alloc 3849 free 3797 clean 2600
proc: alloc 3851 free 3800 clean 2602
proc: alloc 3900 free 3848 clean 2650
proc: alloc 3949 free 3897 clean 2700
proc: alloc 3951 free 3900 clean 2702
proc: alloc 4000 free 3948 clean 2750

i.e. it restarts cleaning properly, but after some time the cleaning
stops again. Also, if I restart too quickly, the cleaning doesn't start
again. So it looks like the cleaning work somehow gets jammed.

Could it be that proc is overflown with dead port notifications? That's
not many procs, but still. Maybe Sergey has an idea?

Samuel

Re: proc leaking

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 15:35:00 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> > Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > > increasing).
> > > 
> > > It seems 64bit-specific: the program below makes proc leak memory, 100
> > > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > > properly parse messages to be destroyed, so that in the error case the
> > > server leaks non-inline data? Flavio, perhaps you have an idea?
> > 
> > I don't think we have the kernel-to-user equivalent for
> > adjust_msg_type_size? So that we end up pushing twice too much data to
> > userland for port arrays?
> 
> I found and fixed the allocation issue in the kernel.

It seems proc is still leaking, but on the heap this time. This is not
64bit-specific, the same simple reproducer triggers it:

while [  "$(echo -n `echo a` )" = a ] ; do : ; done

or more simply:

while true ; do echo $(echo -n $(echo a)) > /dev/null ; done

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 15:35:00 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> > Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > > increasing).
> > > 
> > > It seems 64bit-specific: the program below makes proc leak memory, 100
> > > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > > properly parse messages to be destroyed, so that in the error case the
> > > server leaks non-inline data? Flavio, perhaps you have an idea?
> > 
> > I don't think we have the kernel-to-user equivalent for
> > adjust_msg_type_size? So that we end up pushing twice too much data to
> > userland for port arrays?
> 
> I found and fixed the allocation issue in the kernel. We however still
> probably need some adjust_msg_type_size in copyoutmsg, otherwise
> userland will see a 64bit size for ports?

Ah, it's already done within copyout_unpack_msg_type

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > increasing).
> > 
> > It seems 64bit-specific: the program below makes proc leak memory, 100
> > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > properly parse messages to be destroyed, so that in the error case the
> > server leaks non-inline data? Flavio, perhaps you have an idea?
> 
> I don't think we have the kernel-to-user equivalent for
> adjust_msg_type_size? So that we end up pushing twice too much data to
> userland for port arrays?

I found and fixed the allocation issue in the kernel. We however still
probably need some adjust_msg_type_size in copyoutmsg, otherwise
userland will see a 64bit size for ports?

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > > 
> > > > For now, I could reproduce with
> > > > 
> > > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > > 
> > > > by running two of them in parallel, along with an apt install loop in
> > > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > > 3...)
> > > 
> > > It seems to happen more often when running inside a chroot (possibly
> > > because of the intermediate firmlink redirection?), and possibly
> > > eatmydata also makes it more frequent.
> > 
> > (it looks like there are memory leaks in proc, its vminfo keeps
> > increasing).
> 
> It seems 64bit-specific: the program below makes proc leak memory, 100
> vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> properly parse messages to be destroyed, so that in the error case the
> server leaks non-inline data? Flavio, perhaps you have an idea?

I don't think we have the kernel-to-user equivalent for
adjust_msg_type_size? So that we end up pushing twice too much data to
userland for port arrays?

> #include 
> #include 
> 
> #define N 1024
> int main(void) {
>   mach_port_t port = getproc();
>   mach_port_t ports[N];
>   int ints[N];
>   for (int i = 0; i < N; i++) {
>   ports[i] = MACH_PORT_DEAD;
>   }
>   for (int i = 0; i < 100; i++) {
>   int ret = proc_setexecdata(port, ports, 
> MACH_MSG_TYPE_COPY_SEND, N, ints, N);
>   if (ret) {
>   errno = ret;
>   perror("setexecdata");
>   }
>   }
>   return 0;
> }

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > > 
> > > > For now, I could reproduce with
> > > > 
> > > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > > 
> > > > by running two of them in parallel, along with an apt install loop in
> > > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > > 3...)
> > > 
> > > It seems to happen more often when running inside a chroot (possibly
> > > because of the intermediate firmlink redirection?), and possibly
> > > eatmydata also makes it more frequent.
> > 
> > (it looks like there are memory leaks in proc, its vminfo keeps
> > increasing).
> 
> It seems 64bit-specific: the program below makes proc leak memory, 100
> vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> properly parse messages to be destroyed, so that in the error case the
> server leaks non-inline data? Flavio, perhaps you have an idea?

Realizing only now by reading the __mach_msg_destroy assembly...

unsigned intmsgt_inline : 1,
msgt_longform : 1,
msgt_deallocate : 1,
msgt_name : 8,
msgt_size : 16,
msgt_unused : 5;

This field ordering makes reading them overly complex... It'll be a pain
to rebootstrap, but perhaps we do want to put msgt_size and msgt_name
first?

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > 
> > > For now, I could reproduce with
> > > 
> > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > 
> > > by running two of them in parallel, along with an apt install loop in
> > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > 3...)
> > 
> > It seems to happen more often when running inside a chroot (possibly
> > because of the intermediate firmlink redirection?), and possibly
> > eatmydata also makes it more frequent.
> 
> (it looks like there are memory leaks in proc, its vminfo keeps
> increasing).

It seems 64bit-specific: the program below makes proc leak memory, 100
vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
properly parse messages to be destroyed, so that in the error case the
server leaks non-inline data? Flavio, perhaps you have an idea?

Samuel


#include 
#include 

#define N 1024
int main(void) {
mach_port_t port = getproc();
mach_port_t ports[N];
int ints[N];
for (int i = 0; i < N; i++) {
ports[i] = MACH_PORT_DEAD;
}
for (int i = 0; i < 100; i++) {
int ret = proc_setexecdata(port, ports, 
MACH_MSG_TYPE_COPY_SEND, N, ints, N);
if (ret) {
errno = ret;
perror("setexecdata");
}
}
return 0;
}

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > 
> > For now, I could reproduce with
> > 
> > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > 
> > by running two of them in parallel, along with an apt install loop in
> > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > 3...)
> 
> It seems to happen more often when running inside a chroot (possibly
> because of the intermediate firmlink redirection?), and possibly
> eatmydata also makes it more frequent.

(it looks like there are memory leaks in proc, its vminfo keeps
increasing).

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> 
> For now, I could reproduce with
> 
> time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> 
> by running two of them in parallel, along with an apt install loop in
> parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> 3...)

It seems to happen more often when running inside a chroot (possibly
because of the intermediate firmlink redirection?), and possibly
eatmydata also makes it more frequent.

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 's/s-gox/gox/' 
> ` )" = internal/reflectlite.gox ] ; do : ; done

For now, I could reproduce with

time while [  "$(echo -n `echo a` )" = a ] ; do : ; done

by running two of them in parallel, along with an apt install loop in
parallel. It takes a few hours to reproduce (sometimes 1, sometimes
3...)

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le ven. 27 oct. 2023 00:42:06 +0200, a ecrit:
> Samuel Thibault, le mer. 25 oct. 2023 14:55:36 +0200, a ecrit:
> > Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> > > jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > > > October 25, 2023 3:43 AM, "Samuel Thibault"  
> > > > wrote:
> > > > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > > > 
> > > > >> Or maybe GCC is partly at fault for the
> > > > >> Hurd's X86_64 building troubles?
> > > > > 
> > > > > It's not at all. Nor is libtool.
> > > > > 
> > > > > I occasionally had issues in ./configure, too.
> > > > > 
> > > > > You'll say that's "yeah, it's all about auto-crap". No.
> > > > > 
> > > > > It's *very* most probably about bash, simply.
> > > > 
> > > > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> > > 
> > > It'd really better be short-term, because currently we cannot really
> > > trust the built packages: what if due to shell script misbehavior
> > > ./configure misdetects features, forgets enabling some support
> > > etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> > > down.
> > 
> > Today's gcc attempt:
> > 
> > Comparing stages 2 and 3
> > Bootstrap comparison failure!
> > libbacktrace/.libs/sort.o differs
> 
> Here is a good example:
> 
> f="internal/reflectlite.o"; if test ! -f $f; then 
> f="internal/.libs/reflectlite.o"; fi; x86_64-gnu-objcopy -j .go_export $f 
> internal/reflectlite.s-gox.tmp; /bin/bash ../../../src/libgo/mvifdiff.sh 
> internal/reflectlite.s-gox.tmp `echo internal/reflectlite.s-gox | sed -e 
> 's/s-gox/gox/'`
> mv: cannot move 'internal/reflectlite.s-gox.tmp' to '': No such file or 
> directory
> 
> It looks like the echo|sed part didn't work. (or command parameter
> passing to mvifdiff.sh, but I doubt that)

Indeed, 

while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 's/s-gox/gox/' ` 
)" = internal/reflectlite.gox ] ; do : ; done

does stop.

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 25 oct. 2023 00:04:33 +0200, a ecrit:
> Building packages is not very stable. I have been trying to build
> gcc-13 for a couple of weeks, without success so far. There are various
> failures, most often odd errors in the libtool script, which are a sign
> that the system itself is not behaving correctly. A way to reproduce
> the issue is to just repeatedly build a package that is using libtool,
> sooner or later that will fail very oddly.
> 
> This means that while the buildd will be ready, I'm really not at ease
> with letting it start, knowing that it can behave erratically.

Actually, until gcc-13 actually builds, nothing else can build since
libc depends on libgcc.

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 25 oct. 2023 14:55:36 +0200, a ecrit:
> Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> > jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > > October 25, 2023 3:43 AM, "Samuel Thibault"  
> > > wrote:
> > > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > > 
> > > >> Or maybe GCC is partly at fault for the
> > > >> Hurd's X86_64 building troubles?
> > > > 
> > > > It's not at all. Nor is libtool.
> > > > 
> > > > I occasionally had issues in ./configure, too.
> > > > 
> > > > You'll say that's "yeah, it's all about auto-crap". No.
> > > > 
> > > > It's *very* most probably about bash, simply.
> > > 
> > > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> > 
> > It'd really better be short-term, because currently we cannot really
> > trust the built packages: what if due to shell script misbehavior
> > ./configure misdetects features, forgets enabling some support
> > etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> > down.
> 
> Today's gcc attempt:
> 
> Comparing stages 2 and 3
> Bootstrap comparison failure!
> libbacktrace/.libs/sort.o differs

Here is a good example:

f="internal/reflectlite.o"; if test ! -f $f; then 
f="internal/.libs/reflectlite.o"; fi; x86_64-gnu-objcopy -j .go_export $f 
internal/reflectlite.s-gox.tmp; /bin/bash ../../../src/libgo/mvifdiff.sh 
internal/reflectlite.s-gox.tmp `echo internal/reflectlite.s-gox | sed -e 
's/s-gox/gox/'`
mv: cannot move 'internal/reflectlite.s-gox.tmp' to '': No such file or 
directory

It looks like the echo|sed part didn't work. (or command parameter
passing to mvifdiff.sh, but I doubt that)

Samuel

Re: 64bit startup

[Samuel Thibault]

Sergey Bugaev, le mer. 25 oct. 2023 16:29:29 +0300, a ecrit:
> On Wed, Oct 25, 2023 at 2:52 PM  wrote:
> >
> > October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> >
> > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > >
> > >> Or maybe GCC is partly at fault for the
> > >> Hurd's X86_64 building troubles?
> > >
> > > It's not at all. Nor is libtool.
> > >
> > > I occasionally had issues in ./configure, too.
> > >
> > > You'll say that's "yeah, it's all about auto-crap". No.
> > >
> > > It's *very* most probably about bash, simply.
> > >
> > > Samuel
> >
> > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> >
> > Could we change the default shell on X86_64 Debian Hurd in the meantime,
> > as a temporary solution?
> 
> I would rather ask, would it not be possible to set up a continuous
> build server (buildd? I know next to nothing about the Debian infra)
> that itself runs on a more stable architecture (amd64, or hurd-i386)
> and cross-compiles the packages?

Cross-compiling *very* often produces slightly bogus packages. They are
enough to bootstrap something you can build upon, but you cannot hope
more.

We do have cross-compiling boot strap set up on
https://jenkins.debian.net/view/rebootstrap/
but we don't want to upload the result, since when cross-compiling
there are various ./configure tests that you cannot run (execution-time
results).

Samuel

Re: 64bit startup

[Samuel Thibault]

Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > 
> > >> Or maybe GCC is partly at fault for the
> > >> Hurd's X86_64 building troubles?
> > > 
> > > It's not at all. Nor is libtool.
> > > 
> > > I occasionally had issues in ./configure, too.
> > > 
> > > You'll say that's "yeah, it's all about auto-crap". No.
> > > 
> > > It's *very* most probably about bash, simply.
> > 
> > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> 
> It'd really better be short-term, because currently we cannot really
> trust the built packages: what if due to shell script misbehavior
> ./configure misdetects features, forgets enabling some support
> etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> down.

Today's gcc attempt:

Comparing stages 2 and 3
Bootstrap comparison failure!
libbacktrace/.libs/sort.o differs

Samuel

Re: 64bit startup

[Samuel Thibault]

jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > 
> >> Or maybe GCC is partly at fault for the
> >> Hurd's X86_64 building troubles?
> > 
> > It's not at all. Nor is libtool.
> > 
> > I occasionally had issues in ./configure, too.
> > 
> > You'll say that's "yeah, it's all about auto-crap". No.
> > 
> > It's *very* most probably about bash, simply.
> 
> Hmmm. I guess in the long-term then, the bash issues should be fixed.

It'd really better be short-term, because currently we cannot really
trust the built packages: what if due to shell script misbehavior
./configure misdetects features, forgets enabling some support
etc. That'd lead to subtle incompatibilities that'll be hard to hunt
down.

> Could we change the default shell on X86_64 Debian Hurd in the meantime, 
> as a temporary solution?

Don't take me wrong: I'm not saying the concern is *because* of bash,
but *concerning* bash. Another shell could very well just face exactly
the same concerns.

And no, we cannot just switch it: libtool uses the bash features, so we
have to fix the behavior of the system for bash (and /bin/sh already
points to dash)

Samuel

Re: 64bit startup

[Samuel Thibault]

Jeffrey Walton, le mar. 24 oct. 2023 22:00:54 -0400, a ecrit:
> On Tue, Oct 24, 2023 at 9:56 PM Jessica Clarke  wrote:
> >
> > On 25 Oct 2023, at 02:40, Jeffrey Walton  wrote:
> > >
> > > On Tue, Oct 24, 2023 at 9:33 PM Jessica Clarke  wrote:
> > >>
> > >> On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
> > >>>
> > >>> On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault 
> > >>>  wrote:
> > >>>>
> > >>>> Some update on the 64bit port:
> > >>>>
> > >>>> - The debian-ports archive now has enough packages to bootstrap a
> > >>>> chroot.
> > >>>> - A 64bit debian buildd is getting set up, not much work is left there.
> > >>>> - The hurd-amd64 wanna-build infrastructure is to be set up in the
> > >>>> coming days.
> > >>>
> > >>> Congrats
> > >>>
> > >>>> *but*
> > >>>>
> > >>>> Building packages is not very stable. I have been trying to build
> > >>>> gcc-13 for a couple of weeks, without success so far. There are various
> > >>>> failures, most often odd errors in the libtool script, which are a sign
> > >>>> that the system itself is not behaving correctly. A way to reproduce
> > >>>> the issue is to just repeatedly build a package that is using libtool,
> > >>>> sooner or later that will fail very oddly.
> > >>>
> > >>> lol... <https://harmful.cat-v.org/software/GCC> and
> > >>
> > >> Yeah can we not spread this kind of vile rhetoric here? Regardless of
> > >> how much truth is in that, and whether it holds today, that kind of
> > >> language isn’t something we should be celebrating and encouraging
> > >> others to read. Let’s keep things more civil and on topic.
> > >
> > > My apologies for offending your delicate sensibilities.
> >
> > 1. I did not say I was offended. I said it was vile rhetoric. It does
> >not personally offend me, but that does not mean I want to see it
> >being circulated on these kinds of mailing lists.
> 
> Your overreaction. It looks like the stuff I would expect to see on
> social media, like one of those binary confused persons crying someone
> is perpetrating a hate crime because the wrong pronoun (subject?) was
> used.

*You* are overracting here.

Samuel

Re: 64bit startup

[Samuel Thibault]

jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> >> lol...  and
> > 
> > Exactly. So what?
> > 
> > Some folks have witty humor and appreciate the nostalgia. Others don't.
> > 
> > If you don't like my posts, then plonk me.
> > 
> > Jeff
> 
> I followed the link, and I did think it was a little funny. Perhaps
> it was a little off topic. Or maybe GCC is partly at fault for the
> Hurd's X86_64 building troubles?

It's not at all. Nor is libtool.

I occasionally had issues in ./configure, too.

You'll say that's "yeah, it's all about auto-crap". No.

It's *very* most probably about bash, simply.

(and while the gcc page makes sense (all very complex software have
a lot of bugs, so compilers have, but the message has to be terribly
moderated, considering the huge amount of tests that gcc receives), the
auto-hell page doesn't: nowhere was it ever suggested that it's normal
for software to run several ./configure instances).

So, yes, it was essentially off-topic.

Samuel

Re: 64bit startup

[Samuel Thibault]

Hello,

Some update on the 64bit port:

- The debian-ports archive now has enough packages to bootstrap a
chroot.
- A 64bit debian buildd is getting set up, not much work is left there.
- The hurd-amd64 wanna-build infrastructure is to be set up in the
coming days.

*but*

Building packages is not very stable. I have been trying to build
gcc-13 for a couple of weeks, without success so far. There are various
failures, most often odd errors in the libtool script, which are a sign
that the system itself is not behaving correctly. A way to reproduce
the issue is to just repeatedly build a package that is using libtool,
sooner or later that will fail very oddly.

This means that while the buildd will be ready, I'm really not at ease
with letting it start, knowing that it can behave erratically. When I
built the initial set of packages for debian-ports (~100 packages), I
got something like 5-10 such failures, that's quite high of a rate :/

Samuel

Re: Adding hurd-amd64 on debian-ports?

[Samuel Thibault]

Hello,

Aurelien Jarno, le jeu. 11 mai 2023 21:49:07 +0200, a ecrit:
> On 2023-05-02 22:01, Samuel Thibault wrote:
> > Could you create a debian-ports repo for hurd-amd64?
> > 
> > Yes, that is coming out :)
> > 
> > For now we have not tested a whole debian system running, but we do have
> > commands starting to run and the basic toolchain building, and having an
> > archive at hand would be convenient to be able to have built binaries
> > available, as well as an easy way to create chroots with debootstrap or
> > crosshurd.
> 
> This should be done now, sorry for the delay.

No worry! It took us some time to check that we are fine with the ABI
(and fixed a coupe of things).

> Upload rights should be the same as for hurd-i386.

The first upload apparently went fine, thanks!

Samuel

Re: 64bit startup

[Samuel Thibault]

Hello,

Good news!  It seems I fixed the bug that was making my 64bit VM
crashing quite often. The problem was that when receiving a message from
userland, ipc_kmsg_get would allocate a kernel buffer with the same size
as the userland message. But since we may expand the 32bit port names
into 64bit port addresses, this is not large enough.

I'm almost finished with building the base set of debian packages,
required to build & upload packages on the debian-ports archive (just
missing cmake that exposes some bootstrap bugs).

Before building the hurd-amd64 world, are we sure we are all set with
the ABI details?  It'll be way harder to fix later on.

Samuel

Re: jemalloc's Hurd patch

[Samuel Thibault]

Faidon Liambotis, le mar. 12 sept. 2023 19:03:30 +0300, a ecrit:
> > I have uploaded version 2.37-8~0 on debian-ports's unreleased, so it's
> > available on buildds as well. I have upgraded exodar.
> 
> I just uploaded jemalloc 5.3.0-2 containing the updated Hurd patch. I
> first tried a test build in an schroot on exodar, and it seemed to build
> successfully. (Binaries are still in my ~)

\o/

Thanks,
Samuel

Re: expect eats all the ptys

[Samuel Thibault]

Hello,

Craig Small, le ven. 08 sept. 2023 16:26:35 +1000, a ecrit:
> In other news, procps 4.0.4 should work on Hurd. I removed a PATH_MAX and 
> fixed
> the library to not use /proc/sys for Linux version.

Great :D

> I'm actually going to look at the procfs first, being the procps/psmisc 
> author,
> to see if there is anything important there.
> Wouldn't that be a translator?

Yes, it is, it's in hurd/procfs/

Samuel

Re: expect eats all the ptys

[Samuel Thibault]

Samuel Thibault, le jeu. 07 sept. 2023 16:55:33 +0200, a ecrit:
> Craig Small, le jeu. 07 sept. 2023 22:25:37 +1000, a ecrit:
> > spawn creates a new process and links the ptys, wait waits for the process 
> > to
> > die, if you change the "spawn true" to "spawn sleep 2" you'll see the script
> > slow down.
> > 
> > The script completes, ps shows no lurking scripts or true programs and yet 
> > the
> > problem exists.
> 
> Yes, but apparently expect leaks some fd. When running portinfo -v on
> it while starting "spawn true" and "wait" by hand, we see fd(6), fd(7),
> etc. appearing. So at least this is leaking, and probably very worth
> checking.

That also happens on Linux: type spawn true / wait several times and you
will see /dev/pts/ populating. The test passes on Linux only because its
ptmx interface allows for unbound ptys.

(yes, we should add ptmx support in the Hurd, help welcome!)

Samuel

Re: expect eats all the ptys

[Samuel Thibault]

Hello,

Craig Small, le jeu. 07 sept. 2023 22:25:37 +1000, a ecrit:
> spawn creates a new process and links the ptys, wait waits for the process to
> die, if you change the "spawn true" to "spawn sleep 2" you'll see the script
> slow down.
> 
> The script completes, ps shows no lurking scripts or true programs and yet the
> problem exists.

Yes, but apparently expect leaks some fd. When running portinfo -v on
it while starting "spawn true" and "wait" by hand, we see fd(6), fd(7),
etc. appearing. So at least this is leaking, and probably very worth
checking.

Samuel

Re: jemalloc's Hurd patch

[Samuel Thibault]

Hello,

Samuel Thibault, le ven. 18 août 2023 16:10:39 +0200, a ecrit:
> Faidon Liambotis, le ven. 18 août 2023 06:42:48 +0300, a ecrit:
> > On Mon, Aug 07, 2023 at 11:02:14AM +0200, Samuel Thibault wrote:
> > > I had work pending to fix that, I have now done it, and will commit it.
> > > With that done, the testsuite does pass indeed.
> > 
> > Thanks Samuel. Did you end up commiting that work?
> 
> That work is within glibc, to fix the chicken-and-egg issue between
> pthread and malloc(). It's commited in upstream glibc and the Debian
> package, and will be included in the next upload.

I have uploaded version 2.37-8~0 on debian-ports's unreleased, so it's
available on buildds as well. I have upgraded exodar.

Samuel

Re: More memory with PAE

[Samuel Thibault]

Samuel Thibault, le mer. 30 août 2023 01:36:57 +0200, a ecrit:
> Concerning rumpdisk, I had to fix it to allow PAE, so don't run a PAE
> kernel without installing the hurd package >= 1:0.9.git20230520-4+b1.

(which is not available yet, it'll get built once rumpkernel gets
installed on debian-ports' unreleased).

Samuel

More memory with PAE

[Samuel Thibault]

Hello,

FI, apparently I managed to fix the last bugs concerning PAE execution,
it works fine on my box, thus opening the path for >4GiB memory
even on i386.

Concerning rumpdisk, I had to fix it to allow PAE, so don't run a PAE
kernel without installing the hurd package >= 1:0.9.git20230520-4+b1.

Samuel

Re: jemalloc's Hurd patch

[Samuel Thibault]

Hello,

Faidon Liambotis, le ven. 18 août 2023 06:42:48 +0300, a ecrit:
> On Mon, Aug 07, 2023 at 11:02:14AM +0200, Samuel Thibault wrote:
> > I had work pending to fix that, I have now done it, and will commit it.
> > With that done, the testsuite does pass indeed.
> 
> Thanks Samuel. Did you end up commiting that work?

That work is within glibc, to fix the chicken-and-egg issue between
pthread and malloc(). It's commited in upstream glibc and the Debian
package, and will be included in the next upload.

Samuel

Re: jemalloc's Hurd patch

[Samuel Thibault]

Hello,

Faidon Liambotis, le lun. 17 juil. 2023 00:24:42 +0300, a ecrit:
> On Tue, May 16, 2023 at 06:31:58PM +0300, Faidon Liambotis wrote:
> > On Tue, May 16, 2023 at 05:22:44PM +0200, Samuel Thibault wrote:
> > > For information, I have completed and submitted jemalloc's hurd.patch to
> > > https://github.com/jemalloc/jemalloc/pull/2443
> > 
> > That's awesome, thank you! I'll include it in the next upload, which
> > will be after the bookworm release at this point.
> 
> I got around to testing this, and I tried a test build with the updated
> patch, in a Hurd QEMU (20230608 nightly, dist-upgraded to today's
> unstable).
> 
> The aligned_alloc test fails, with "Killed". gdb says "Thread 4 received
> signal ?, Unknown signal." and has a backtrace involving...
>   __GI___pthread_mutex_lock ->
>   __GI___pthread_key_create ->
>   tsd_boot0 ->
>   je_malloc_tsd_boot0 ->
>   malloc_init_hard ->
>   malloc_init ->
>   ...
> 
> Have you ("make check" and/or dpkg-buildpackage) tested this change? Are
> you able to reproduce?

Never cut a backtrace :)

The full backtrace is:

#0  __GI___pthread_mutex_lock (mtxp=0x11181e0 <__pthread_key_lock>) at 
../sysdeps/mach/hurd/htl/pt-mutex-lock.c:41
#1  0x0110ab1c in __GI___pthread_key_create (key=0x10f8e84 , 
destructor=0x10c7140 ) at ../sysdeps/htl/pt-key-create.c:42
#2  0x010c7673 in tsd_boot0 () at include/jemalloc/internal/tsd_tls.h:15
#3  je_malloc_tsd_boot0 () at src/tsd.c:459
#4  0x01052766 in malloc_init_hard () at src/jemalloc.c:2160
#5  0x0105563d in malloc_init () at src/jemalloc.c:301
#6  imalloc_init_check (dopts=, sopts=) 
at src/jemalloc.c:2643
#7  imalloc (dopts=, sopts=) at 
src/jemalloc.c:2674
#8  je_malloc_default (size=716) at src/jemalloc.c:2707
#9  0x01055ca0 in imalloc_fastpath (fallback_alloc=0x10550f0 
, size=) at 
include/jemalloc/internal/jemalloc_internal_inlines_c.h:277
#10 0x0110b3d5 in __pthread_alloc (pthread=0x1035c30) at ./htl/pt-alloc.c:114
#11 0x0110babc in __pthread_create_internal (thread=0x1035c78, attr=0x1035c7c, 
start_routine=0x0, arg=0x0) at ./htl/pt-create.c:114
#12 0x01110f2b in _init_routine (stack=) at 
../sysdeps/mach/hurd/htl/pt-sysdep.c:64
#13 0x6292 in call_init (env=0x1035d6c, argv=, argc=1, 
l=) at ./elf/dl-init.c:70
#14 call_init (l=, argc=1, argv=, env=0x1035d6c) 
at ./elf/dl-init.c:26
#15 0x63ff in _dl_init (main_map=0x34a30, argc=1, argv=0x1035d64, 
env=0x1035d6c) at ./elf/dl-init.c:84
#16 0x0001c0d0 in _dl_start_user () from /lib/ld.so

I.e. while libpthread is initializing, it allocates a thread structure,
thus triggering the jemalloc initialization, which makes libpthread
calls, but libpthread is not finished initializing.

I had work pending to fix that, I have now done it, and will commit it.
With that done, the testsuite does pass indeed.

Samuel