Re: 64bit startup

2024-01-07 Thread Joshua Branson
Luca  writes:

> Hi Sergey,
>
> Il 03/01/24 09:17, Sergey Bugaev ha scritto:
>> How are you running it? Should I still be using a ramdisk image and
>> not rumpdisk?
>
> Recently I've been installing hurd-amd64 on another disk of my
> hurd-i386 vm and booting from that. Basically I prepare the disk with
> debootstrap --foreign, then I reuse the i386 grub install to boot the
> 64 bit kernel with a custom entry, then run the --second stage,
> configure login, fstab and network and reboot. I can give you the
> exact commands and setup I'm using if you want (I need to reinstall it
> anyway due to latest changes),

If you do send an email detailing how to install hurd-amd64, please CC
me, and I will edit the wiki.

>
> I'm currently using qemu via virt-manager, mostly with the default
> configuration for an x86_64 vm; that means a virtual SATA disk
> controller and Q35 chipset.
>
> The only issue I see is that sometimes at shutdown rumpdisk hangs and
> I can't halt the system, however this seems the same with hurd-i686
> and it doesn't happen if I force the shutdown with halt-hurd (or
> reboot-hurd). I didn't had a deeper look at this so far, but I don't
> have issues booting or connecting via ssh. I didn't try heavy builds,
> so probably that's why I don't see Samuel's issue, but I've been
> building gdb and the hurd itself (for a fix for the crash server that
> I have in my queue).
>
>
> Luca
>

-- 

Joshua Branson
Sent from the Hurd



Re: 64bit startup

2024-01-06 Thread Sergey Bugaev
On Sat, Jan 6, 2024 at 2:42 AM Luca  wrote:
> I had this small patch applied that apparently is enough for me to have
> some kind of core dump, I'm not sure if it's a good solution:

> +#ifdef __x86_64__
> +  struct _libc_fpstate fpstate;
> +  memset(&fpstate, 0, sizeof(fpstate));
> +  note.data.pr_fpreg = &fpstate;
> +#endif
> fetch_thread_regset (threads[i], ¬e.data.pr_reg);
> fetch_thread_fpregset (threads[i], ¬e.data.pr_fpreg);

Well, that should surely prevent the crash, but so will commenting out
the fetch_thread_fpregset call completely (perhaps this is what we
should actually do for the time being).

note.data is what we're writing out into the ELF core dump file; it
doesn't make any sense for it to be a pointer to a stack-allocated
variable in the dumper's address space. It doesn't make much sense for
it to be a pointer at all.

We need to figure out what kind of note GDB expects there to be, and
write that out. It doesn't necessarily have to match our definition of
fpregset_t; in fact I think it'd be better if builds of GDB without
any explicit support for x86_64-gnu would be able to read our core
files. And this means copying what Linux does, instead of apparently
Solaris, as the current elfcore tries to. Doing this might fix issues
with reading core files on i686-gnu, too.

> sure, I've started looking into it,

\o/

> but it will take a while before I
> can run something in userspace.

Sure, I don't expect you to produce a working kernel overnight :)

I kind of want to participate in kernel-side hacking too, but I don't
nearly have the understanding required.

How does initial boot protocol work, for example? I mean, where and
how control is transferred, what gets passed in registers, that kind
of thing. I've found many detailed explanations of board-specific boot
details (e.g. for Raspberry Pi 3), and [0] has a nice explanation of
how to make U-Boot load a custom kernel instead of Linux. I haven't
been able to find much info on GRUB; it sounds like Multiboot still
exists on AArch64, but it's unclear exactly how it works.

[0]: https://krinkinmu.github.io/2023/08/21/how-u-boot-loads-linux-kernel.html

Then, we're supposed to parse & use the device tree, aren't we? Do I
understand it right that both Mach and userland would have to deal
with the device tree? — in case of Mach, this would be needed to get
info on CPU cores, RAM, and serial / console / UART, right?

Do the questions at least make sense? Or am I talking nonsense? Can
you recommend any useful resources?

Are you targeting (or: do you think it's realistic to target) many
platforms / boards in a generic manner, or would we have to have
platform-specific gnumach builds? Does it make sense to start with
some specific platform (qemu "virt"?), and expand from there, or is it
better to build in the genericity from the start?

Sergey



Re: 64bit startup

2024-01-05 Thread Samuel Thibault
Luca, le sam. 06 janv. 2024 00:42:35 +0100, a ecrit:
> Il 05/01/24 19:12, Sergey Bugaev ha scritto:
> > /servers/crash-dump-core crashes on the memset () call in
> > hurd:exec/elfcore.c:fetch_thread_fpregset (); the (*fpregs) pointer is
> > NULL. The caller passes fpregs = ¬e.data.pr_fpreg, where note.data
> > is of type struct elf_lwpstatus, defined in hurd:include/sys/procfs.h,
> > whose pr_fpreg field is of type prfpregset_t, which is a typedef to
> > fpregset_t, which was an actual struct on i386, but is a pointer on
> > x86_64. This would've been easier to debug if I had debuginfo :)
> 
> I had this small patch applied that apparently is enough for me to have some
> kind of core dump, I'm not sure if it's a good solution:

You probably rather want to fix fetch_thread_fpregset, so as to properly
put the floating state into pr_fpreg.

This probably needs to actually copy over explicit fields, but that's
what we need anyway.

> diff --git a/exec/elfcore.c b/exec/elfcore.c
> index c6aa2bc8b..405fa8e0c 100644
> --- a/exec/elfcore.c
> +++ b/exec/elfcore.c
> @@ -544,6 +544,11 @@ dump_core (task_t task, file_t file, off_t corelimit,
>note.data.pr_info.si_code = sigcode;
>note.data.pr_info.si_errno = sigerror;
> 
> +#ifdef __x86_64__
> +  struct _libc_fpstate fpstate;
> +  memset(&fpstate, 0, sizeof(fpstate));
> +  note.data.pr_fpreg = &fpstate;
> +#endif
>fetch_thread_regset (threads[i], ¬e.data.pr_reg);
>fetch_thread_fpregset (threads[i], ¬e.data.pr_fpreg);
> 
> 
> HTH
> Luca



Re: 64bit startup

2024-01-05 Thread Luca

Il 04/01/24 10:55, Sergey Bugaev ha scritto:

P.S. I have posted all of my patches, so if you're interested in
hacking on aarch64-gnu Mach, you should be able to build the full
toolchain now.


sure, I've started looking into it, but it will take a while before I 
can run something in userspace. I'm working on top of your gnumach patch 
for now.



Luca



Re: 64bit startup

2024-01-05 Thread Luca

Il 05/01/24 19:12, Sergey Bugaev ha scritto:

/servers/crash-dump-core crashes on the memset () call in
hurd:exec/elfcore.c:fetch_thread_fpregset (); the (*fpregs) pointer is
NULL. The caller passes fpregs = ¬e.data.pr_fpreg, where note.data
is of type struct elf_lwpstatus, defined in hurd:include/sys/procfs.h,
whose pr_fpreg field is of type prfpregset_t, which is a typedef to
fpregset_t, which was an actual struct on i386, but is a pointer on
x86_64. This would've been easier to debug if I had debuginfo :)


I had this small patch applied that apparently is enough for me to have 
some kind of core dump, I'm not sure if it's a good solution:


diff --git a/exec/elfcore.c b/exec/elfcore.c
index c6aa2bc8b..405fa8e0c 100644
--- a/exec/elfcore.c
+++ b/exec/elfcore.c
@@ -544,6 +544,11 @@ dump_core (task_t task, file_t file, off_t corelimit,
   note.data.pr_info.si_code = sigcode;
   note.data.pr_info.si_errno = sigerror;

+#ifdef __x86_64__
+  struct _libc_fpstate fpstate;
+  memset(&fpstate, 0, sizeof(fpstate));
+  note.data.pr_fpreg = &fpstate;
+#endif
   fetch_thread_regset (threads[i], ¬e.data.pr_reg);
   fetch_thread_fpregset (threads[i], ¬e.data.pr_fpreg);


HTH
Luca




Re: 64bit startup

2024-01-05 Thread Samuel Thibault
Sergey Bugaev, le ven. 05 janv. 2024 21:12:48 +0300, a ecrit:
> Also I can't help but notice that the hurd package (i.e the translator
> binaries) is still not being built as PIE,

This is not actually specific to 64bit. This was set explicitly in 2016
in debian/rules, tests welcome to check whether building the package
without it works now.

Samuel



Re: 64bit startup

2024-01-05 Thread Samuel Thibault
Sergey Bugaev, le ven. 05 janv. 2024 21:12:48 +0300, a ecrit:
> I'm not seeing hurd-dbg / hurd-libs-dbg packages in your repo.

Yes, my repo is built from the rebootstrap scripts, which drop debug
etc. only for creating a booting system.

For proper packages, use the usual deb.debian.org debian-ports mirror.

Samuel



Re: 64bit startup

2024-01-05 Thread Sergey Bugaev
I'm not seeing hurd-dbg / hurd-libs-dbg packages in your repo. Could
you please either teach me where to look, or if they're indeed
missing, upload them?

Also I can't help but notice that the hurd package (i.e the translator
binaries) is still not being built as PIE, unlike basically all the
other binaries. This actually helps with debugging for now, but please
remember to ensure it does end up as PIE when the system is ready for
broader use.

/servers/crash-dump-core crashes on the memset () call in
hurd:exec/elfcore.c:fetch_thread_fpregset (); the (*fpregs) pointer is
NULL. The caller passes fpregs = ¬e.data.pr_fpreg, where note.data
is of type struct elf_lwpstatus, defined in hurd:include/sys/procfs.h,
whose pr_fpreg field is of type prfpregset_t, which is a typedef to
fpregset_t, which was an actual struct on i386, but is a pointer on
x86_64. This would've been easier to debug if I had debuginfo :)

Sergey



Re: 64bit startup

2024-01-05 Thread Samuel Thibault
Samuel Thibault, le jeu. 04 janv. 2024 08:57:51 +0100, a ecrit:
> Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> > perhaps I need to try two of them in parallel and some I/O-heavy
> > workload in the background, as you're saying.
> 
> Yes, that's needed to raise the probability of the bug.
> 
> > Could it be that the two strings are actually different (something
> > being wrong with pipes perhaps)?
> 
> I tried
> 
> A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; done 
> ; echo $A
> 
> The output is empty. But yes, that could be some missing flush or such
> in pipes.

It happens with dash too.

Samuel



Re: 64bit startup

2024-01-05 Thread Samuel Thibault
Sergey Bugaev, le ven. 05 janv. 2024 12:06:13 +0300, a ecrit:
> > I use
> >
> > while true ; do apt install --reinstall wdiff ; done
> 
> That did it! I can now reliably reproduce this.
> 
> (I assume you don't mind my box constantly banging on your repo.)

It's people.debian.org, it's meant for this :)

You can probably also pass to apt an option to keep the donwloaded .deb

> > got acd :)
> 
> In the six times that I've reproduced it so far, I got "acd" in all cases. 
> Hmmm.

"interesting" :)

Samuel



Re: 64bit startup

2024-01-05 Thread Sergey Bugaev
On Fri, Jan 5, 2024 at 12:52 AM Samuel Thibault  wrote:
> Which kind of activity?

I just had a loop spawning /bin/true — this should've triggered it
assuming it was related to some state getting corrupted on
context-switching.

> I use
>
> while true ; do apt install --reinstall wdiff ; done

That did it! I can now reliably reproduce this.

(I assume you don't mind my box constantly banging on your repo.)

> got acd :)

In the six times that I've reproduced it so far, I got "acd" in all cases. Hmmm.

Sergey



Re: 64bit startup

2024-01-04 Thread Samuel Thibault
Hello,

Sergey Bugaev, le jeu. 04 janv. 2024 22:21:11 +0300, a ecrit:
> On Thu, Jan 4, 2024 at 10:57 AM Samuel Thibault  
> wrote:
> > Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> > > perhaps I need to try two of them in parallel and some I/O-heavy
> > > workload in the background, as you're saying.
> >
> > Yes, that's needed to raise the probability of the bug.
> 
> I'm still unable to reproduce this, it's been running for 10+ hours at
> this point. That's two copies of it, and unrelated activity in the
> background.

Which kind of activity? I use 

while true ; do apt install --reinstall wdiff ; done

> > > Could it be that the two strings are actually different (something
> > > being wrong with pipes perhaps)?
> >
> > I tried
> >
> > A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; 
> > done ; echo $A
> >
> > The output is empty. But yes, that could be some missing flush or such
> > in pipes.
> 
> Try
> 
> A=abcd ; time while /usr/bin/\[ "$A" = abcd ] ; do A="$(echo -n `echo
> a``echo b`)$(echo -n `echo c``echo d`)" ; done ; echo $A
> 
> perhaps?

got acd :)

I'll be testing with dash over the next hours.

Samuel



Re: 64bit startup

2024-01-04 Thread Sergey Bugaev
On Thu, Jan 4, 2024 at 10:57 AM Samuel Thibault  wrote:
>
> Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> > perhaps I need to try two of them in parallel and some I/O-heavy
> > workload in the background, as you're saying.
>
> Yes, that's needed to raise the probability of the bug.

I'm still unable to reproduce this, it's been running for 10+ hours at
this point. That's two copies of it, and unrelated activity in the
background.

> > Could it be that the two strings are actually different (something
> > being wrong with pipes perhaps)?
>
> I tried
>
> A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; done 
> ; echo $A
>
> The output is empty. But yes, that could be some missing flush or such
> in pipes.

Try

A=abcd ; time while /usr/bin/\[ "$A" = abcd ] ; do A="$(echo -n `echo
a``echo b`)$(echo -n `echo c``echo d`)" ; done ; echo $A

perhaps?

Sergey



Re: 64bit startup

2024-01-04 Thread Sergey Bugaev
On Wed, Jan 3, 2024 at 10:07 PM Luca  wrote:
> Hi Sergey,

Hi,

> Recently I've been installing hurd-amd64 on another disk of my hurd-i386
> vm and booting from that. Basically I prepare the disk with debootstrap
> --foreign, then I reuse the i386 grub install to boot the 64 bit kernel
> with a custom entry,

That (debootstrap + reusing existing GRUB from the i686 installation)
is what I was doing, yes, in one of the two setups that I've tried. On
the other (on a different host) I was doing grub2-install myself. In
both cases I got the same result, with GRUB working fine, but then
rumpdisk apparently misbehaving.

I could reproduce this if we want to debug it further, but Samuel's
image works great for now.

> then run the --second stage, configure login, fstab
> and network and reboot. I can give you the exact commands and setup I'm
> using if you want (I need to reinstall it anyway due to latest changes),
>
> I'm currently using qemu via virt-manager, mostly with the default
> configuration for an x86_64 vm; that means a virtual SATA disk
> controller and Q35 chipset.

Yes, I'd like to use libvirt eventually too, like I'm doing for my
i686 Hurd VM. But I need greater control over how I invoke QEMU for
now.

Sergey

P.S. I have posted all of my patches, so if you're interested in
hacking on aarch64-gnu Mach, you should be able to build the full
toolchain now.



Re: 64bit startup

2024-01-03 Thread Samuel Thibault
Sergey Bugaev, le mer. 03 janv. 2024 21:56:54 +0300, a ecrit:
> perhaps I need to try two of them in parallel and some I/O-heavy
> workload in the background, as you're saying.

Yes, that's needed to raise the probability of the bug.

> Could it be that the two strings are actually different (something
> being wrong with pipes perhaps)?

I tried

A=a ; time while /usr/bin/\[ "$A" = a ] ; do A="$(echo -n `echo a` )" ; done ; 
echo $A

The output is empty. But yes, that could be some missing flush or such
in pipes.

Samuel



Re: 64bit startup

2024-01-03 Thread Samuel Thibault
Luca, le mer. 03 janv. 2024 20:07:00 +0100, a ecrit:
> Il 03/01/24 09:17, Sergey Bugaev ha scritto:
> > How are you running it? Should I still be using a ramdisk image and
> > not rumpdisk?
> 
> Recently I've been installing hurd-amd64 on another disk of my hurd-i386 vm
> and booting from that. Basically I prepare the disk with debootstrap
> --foreign, then I reuse the i386 grub install to boot the 64 bit kernel with
> a custom entry, then run the --second stage, configure login, fstab and
> network and reboot. I can give you the exact commands and setup I'm using if
> you want (I need to reinstall it anyway due to latest changes),

It could be useful to merge information into the wiki page.

Samuel



Re: 64bit startup

2024-01-03 Thread Luca

Hi Sergey,

Il 03/01/24 09:17, Sergey Bugaev ha scritto:

How are you running it? Should I still be using a ramdisk image and
not rumpdisk?


Recently I've been installing hurd-amd64 on another disk of my hurd-i386 
vm and booting from that. Basically I prepare the disk with debootstrap 
--foreign, then I reuse the i386 grub install to boot the 64 bit kernel 
with a custom entry, then run the --second stage, configure login, fstab 
and network and reboot. I can give you the exact commands and setup I'm 
using if you want (I need to reinstall it anyway due to latest changes),


I'm currently using qemu via virt-manager, mostly with the default 
configuration for an x86_64 vm; that means a virtual SATA disk 
controller and Q35 chipset.


The only issue I see is that sometimes at shutdown rumpdisk hangs and I 
can't halt the system, however this seems the same with hurd-i686 and it 
doesn't happen if I force the shutdown with halt-hurd (or reboot-hurd). 
I didn't had a deeper look at this so far, but I don't have issues 
booting or connecting via ssh. I didn't try heavy builds, so probably 
that's why I don't see Samuel's issue, but I've been building gdb and 
the hurd itself (for a fix for the crash server that I have in my queue).



Luca



Re: 64bit startup

2024-01-03 Thread Sergey Bugaev
On Wed, Jan 3, 2024 at 11:27 AM Samuel Thibault  wrote:
> Sergey Bugaev, le mer. 03 janv. 2024 11:17:53 +0300, a ecrit:
> > I guess this is where I ask (consistent with the subject line) about
> > how I would run the x86_64 system (to reproduce & debug this).
>
> You probably want to start with the pre-built images I have linked from
> the wiki page.

Ah... I have been reading the wrong version of the wiki page again. It
doesn't help that there are many of them [0][1][2][3].

[0] https://www.gnu.org/software/hurd/faq/64-bit.html
[1] https://www.gnu.org/software/hurd/open_issues/64-bit_port.html
[2] https://darnassus.sceen.net/~hurd-web/faq/64-bit/
[3] https://darnassus.sceen.net/~hurd-web/open_issues/64-bit_port/

But your disk image works *great*! \o/ I don't know what is different
compared to what I was trying, but yours just works.

I haven't been able to reproduce your bug in a few hours of testing;
perhaps I need to try two of them in parallel and some I/O-heavy
workload in the background, as you're saying. Even if I do manage to
reproduce this, I don't immediately know how to debug it; maybe we
could try to use qemu's record/replay functionality to debug it
backwards from where we can detect it? (I have found the rr(1) tool
*immensely* useful for debugging GTK issues.)

Could it be that the two strings are actually different (something
being wrong with pipes perhaps)?

Sergey



Re: 64bit startup

2024-01-03 Thread Samuel Thibault
Sergey Bugaev, le mer. 03 janv. 2024 11:17:53 +0300, a ecrit:
> I guess this is where I ask (consistent with the subject line) about
> how I would run the x86_64 system (to reproduce & debug this).

You probably want to start with the pre-built images I have linked from
the wiki page.

> I've tried debootstrapping from
> https://people.debian.org/~sthibault/tmp/hurd-amd64 as the wiki page
> says; but that doesn't proceed beyond the rumpdisk. Rumpdisk just sits
> there, slowly spitting out logs;

Does it detect disks? What qemu parameters are you using?

I'm using a mere 

kvm -M q35 -drive file=disk-amd64.img -m 1

Samuel



Re: 64bit startup

2024-01-03 Thread Sergey Bugaev
Hello,

I guess this is where I ask (consistent with the subject line) about
how I would run the x86_64 system (to reproduce & debug this).

I've tried debootstrapping from
https://people.debian.org/~sthibault/tmp/hurd-amd64 as the wiki page
says; but that doesn't proceed beyond the rumpdisk. Rumpdisk just sits
there, slowly spitting out logs; ext2fs gives up waiting for it after
several minutes. (I can attach more logs/details if needed.) hurd-i386
w/ rumpdisk boots fine on the same (virtual) hardware.

How are you running it? Should I still be using a ramdisk image and
not rumpdisk?

Sergey



Re: 64bit startup

2024-01-02 Thread Samuel Thibault
Hello,

I'm still stuck without being able to start packages building for
hurd-amd64 due to this unreliability.

Sergey Bugaev, le mar. 31 oct. 2023 10:09:17 +0300, a ecrit:
> On Mon, Oct 30, 2023 at 1:27 AM Samuel Thibault  
> wrote:
> > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> >
> > by running two of them in parallel, along with an apt install loop in
> > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > 3...)
> 
> This could be [0], considering [ is a Bash built-in and not /bin/[, so
> it's Bash that both compares strings and receives SIGCHLDs.
> 
> [0]: https://lists.gnu.org/archive/html/bug-hurd/2023-06/msg00105.html

I tried

time while /usr/bin/\[ "$(echo -n `echo a` )" = a ] ; do : ; done

with the same result.

Samuel



Re: 64bit startup

2023-11-06 Thread Samuel Thibault
Hello,

Flávio Cruz, le dim. 05 nov. 2023 23:17:49 -0500, a ecrit:
> On Tue, Oct 31, 2023 at 9:14 PM Samuel Thibault <[1]samuel.thiba...@gnu.org>
> wrote:
> 
> > Realizing only now by reading the __mach_msg_destroy assembly...
> >
> >     unsigned int        msgt_inline : 1,
> >                         msgt_longform : 1,
> >                         msgt_deallocate : 1,
> >                         msgt_name : 8,
> >                         msgt_size : 16,
> >                         msgt_unused : 5;
> >
> > This field ordering makes reading them overly complex... It'll be a pain
> > to rebootstrap, but perhaps we do want to put msgt_size and msgt_name
> > first?
> 
> I only moved msgt_size to the end of the struct

Ok, so we can as well align fields to make bit mangling simpler.

> However, I wonder if it would be simpler to just ask the user land to
> pass port names using the following struct:
> 
> #ifdef KERNEL
> union mach_rpc_port {
>    mach_port_name_t name;
>    mach_port_t kernel_port;
> };
> #else
> struct mach_rpc_port {
>    mach_port_name_t name;
>    int unused;
> };
> #endif
> 
> It would make the kernel simpler since no message resizing is
> necessary

I was thinking about this suggestion today, and I think that'll be
better for the long run indeed. There are questions about holes being
uninitialized, but:

> and most of the code using this would be MiG generated.

indeed.

I'm almost done with the ground set of Debian packages. Will wait until
this is settled before building the hurd-amd64 Debian world :)

Samuel



Re: 64bit startup

2023-11-05 Thread Flávio Cruz
Hi Samuel

On Tue, Oct 31, 2023 at 9:14 PM Samuel Thibault 
wrote:

> Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > > > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e
> 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > > >
> > > > > For now, I could reproduce with
> > > > >
> > > > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > > >
> > > > > by running two of them in parallel, along with an apt install loop
> in
> > > > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > > > 3...)
> > > >
> > > > It seems to happen more often when running inside a chroot (possibly
> > > > because of the intermediate firmlink redirection?), and possibly
> > > > eatmydata also makes it more frequent.
> > >
> > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > increasing).
> >
> > It seems 64bit-specific: the program below makes proc leak memory, 100
> > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > properly parse messages to be destroyed, so that in the error case the
> > server leaks non-inline data? Flavio, perhaps you have an idea?
>
> Realizing only now by reading the __mach_msg_destroy assembly...
>
> unsigned intmsgt_inline : 1,
> msgt_longform : 1,
> msgt_deallocate : 1,
> msgt_name : 8,
> msgt_size : 16,
> msgt_unused : 5;
>
> This field ordering makes reading them overly complex... It'll be a pain
> to rebootstrap, but perhaps we do want to put msgt_size and msgt_name
> first?
>

I only moved msgt_size to the end of the struct since a small number of
(deprecated) RPC types
require msgt_size to be 2 bytes long and those can be shortened to be under
1 byte.
We could end up with a much larger contiguous msgt_unused that could be
used for
other things in the future.

In relation to other changes we have to do for finishing the ABI... I think
we have a reasonable
ABI now. However, I wonder if it would be simpler to just ask the user land
to pass port names using
the following struct:

#ifdef KERNEL
union mach_rpc_port {
   mach_port_name_t name;
   mach_port_t kernel_port;
};
#else
struct mach_rpc_port {
   mach_port_name_t name;
   int unused;
};
#endif

It would make the kernel simpler since no message resizing is necessary and
most of the code using this
would be MiG generated.

Flavio


> Samuel
>


Re: 64bit startup

2023-11-01 Thread Samuel Thibault
Samuel Thibault, le mer. 01 nov. 2023 15:35:00 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> > Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > > increasing).
> > > 
> > > It seems 64bit-specific: the program below makes proc leak memory, 100
> > > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > > properly parse messages to be destroyed, so that in the error case the
> > > server leaks non-inline data? Flavio, perhaps you have an idea?
> > 
> > I don't think we have the kernel-to-user equivalent for
> > adjust_msg_type_size? So that we end up pushing twice too much data to
> > userland for port arrays?
> 
> I found and fixed the allocation issue in the kernel. We however still
> probably need some adjust_msg_type_size in copyoutmsg, otherwise
> userland will see a 64bit size for ports?

Ah, it's already done within copyout_unpack_msg_type

Samuel



Re: 64bit startup

2023-11-01 Thread Samuel Thibault
Samuel Thibault, le mer. 01 nov. 2023 13:14:17 +0100, a ecrit:
> Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> > Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > > (it looks like there are memory leaks in proc, its vminfo keeps
> > > increasing).
> > 
> > It seems 64bit-specific: the program below makes proc leak memory, 100
> > vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> > properly parse messages to be destroyed, so that in the error case the
> > server leaks non-inline data? Flavio, perhaps you have an idea?
> 
> I don't think we have the kernel-to-user equivalent for
> adjust_msg_type_size? So that we end up pushing twice too much data to
> userland for port arrays?

I found and fixed the allocation issue in the kernel. We however still
probably need some adjust_msg_type_size in copyoutmsg, otherwise
userland will see a 64bit size for ports?

Samuel



Re: 64bit startup

2023-11-01 Thread Samuel Thibault
Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > > 
> > > > For now, I could reproduce with
> > > > 
> > > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > > 
> > > > by running two of them in parallel, along with an apt install loop in
> > > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > > 3...)
> > > 
> > > It seems to happen more often when running inside a chroot (possibly
> > > because of the intermediate firmlink redirection?), and possibly
> > > eatmydata also makes it more frequent.
> > 
> > (it looks like there are memory leaks in proc, its vminfo keeps
> > increasing).
> 
> It seems 64bit-specific: the program below makes proc leak memory, 100
> vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> properly parse messages to be destroyed, so that in the error case the
> server leaks non-inline data? Flavio, perhaps you have an idea?

I don't think we have the kernel-to-user equivalent for
adjust_msg_type_size? So that we end up pushing twice too much data to
userland for port arrays?

> #include 
> #include 
> 
> #define N 1024
> int main(void) {
>   mach_port_t port = getproc();
>   mach_port_t ports[N];
>   int ints[N];
>   for (int i = 0; i < N; i++) {
>   ports[i] = MACH_PORT_DEAD;
>   }
>   for (int i = 0; i < 100; i++) {
>   int ret = proc_setexecdata(port, ports, 
> MACH_MSG_TYPE_COPY_SEND, N, ints, N);
>   if (ret) {
>   errno = ret;
>   perror("setexecdata");
>   }
>   }
>   return 0;
> }



Re: 64bit startup

2023-10-31 Thread Samuel Thibault
Samuel Thibault, le mer. 01 nov. 2023 01:50:40 +0100, a ecrit:
> Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> > Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > > 
> > > > For now, I could reproduce with
> > > > 
> > > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > > 
> > > > by running two of them in parallel, along with an apt install loop in
> > > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > > 3...)
> > > 
> > > It seems to happen more often when running inside a chroot (possibly
> > > because of the intermediate firmlink redirection?), and possibly
> > > eatmydata also makes it more frequent.
> > 
> > (it looks like there are memory leaks in proc, its vminfo keeps
> > increasing).
> 
> It seems 64bit-specific: the program below makes proc leak memory, 100
> vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
> properly parse messages to be destroyed, so that in the error case the
> server leaks non-inline data? Flavio, perhaps you have an idea?

Realizing only now by reading the __mach_msg_destroy assembly...

unsigned intmsgt_inline : 1,
msgt_longform : 1,
msgt_deallocate : 1,
msgt_name : 8,
msgt_size : 16,
msgt_unused : 5;

This field ordering makes reading them overly complex... It'll be a pain
to rebootstrap, but perhaps we do want to put msgt_size and msgt_name
first?

Samuel



Re: 64bit startup

2023-10-31 Thread Samuel Thibault
Samuel Thibault, le mar. 31 oct. 2023 04:40:43 +0100, a ecrit:
> Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> > Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > > 
> > > For now, I could reproduce with
> > > 
> > > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > > 
> > > by running two of them in parallel, along with an apt install loop in
> > > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > > 3...)
> > 
> > It seems to happen more often when running inside a chroot (possibly
> > because of the intermediate firmlink redirection?), and possibly
> > eatmydata also makes it more frequent.
> 
> (it looks like there are memory leaks in proc, its vminfo keeps
> increasing).

It seems 64bit-specific: the program below makes proc leak memory, 100
vminfo lines at a time. Possibly __mach_msg_destroy doesn't actually
properly parse messages to be destroyed, so that in the error case the
server leaks non-inline data? Flavio, perhaps you have an idea?

Samuel


#include 
#include 

#define N 1024
int main(void) {
mach_port_t port = getproc();
mach_port_t ports[N];
int ints[N];
for (int i = 0; i < N; i++) {
ports[i] = MACH_PORT_DEAD;
}
for (int i = 0; i < 100; i++) {
int ret = proc_setexecdata(port, ports, 
MACH_MSG_TYPE_COPY_SEND, N, ints, N);
if (ret) {
errno = ret;
perror("setexecdata");
}
}
return 0;
}



Re: 64bit startup

2023-10-31 Thread Sergey Bugaev
On Mon, Oct 30, 2023 at 1:27 AM Samuel Thibault  wrote:
> Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
>
> For now, I could reproduce with
>
> time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
>
> by running two of them in parallel, along with an apt install loop in
> parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> 3...)

This could be [0], considering [ is a Bash built-in and not /bin/[, so
it's Bash that both compares strings and receives SIGCHLDs.

[0]: https://lists.gnu.org/archive/html/bug-hurd/2023-06/msg00105.html

Sergey



Re: 64bit startup

2023-10-30 Thread Samuel Thibault
Samuel Thibault, le lun. 30 oct. 2023 18:35:03 +0100, a ecrit:
> Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> > Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> > 
> > For now, I could reproduce with
> > 
> > time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> > 
> > by running two of them in parallel, along with an apt install loop in
> > parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> > 3...)
> 
> It seems to happen more often when running inside a chroot (possibly
> because of the intermediate firmlink redirection?), and possibly
> eatmydata also makes it more frequent.

(it looks like there are memory leaks in proc, its vminfo keeps
increasing).

Samuel



Re: 64bit startup

2023-10-30 Thread Samuel Thibault
Samuel Thibault, le dim. 29 oct. 2023 23:27:22 +0100, a ecrit:
> Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> > while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 
> > 's/s-gox/gox/' ` )" = internal/reflectlite.gox ] ; do : ; done
> 
> For now, I could reproduce with
> 
> time while [  "$(echo -n `echo a` )" = a ] ; do : ; done
> 
> by running two of them in parallel, along with an apt install loop in
> parallel. It takes a few hours to reproduce (sometimes 1, sometimes
> 3...)

It seems to happen more often when running inside a chroot (possibly
because of the intermediate firmlink redirection?), and possibly
eatmydata also makes it more frequent.

Samuel



Re: 64bit startup

2023-10-29 Thread Samuel Thibault
Samuel Thibault, le ven. 27 oct. 2023 08:48:19 +0200, a ecrit:
> while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 's/s-gox/gox/' 
> ` )" = internal/reflectlite.gox ] ; do : ; done

For now, I could reproduce with

time while [  "$(echo -n `echo a` )" = a ] ; do : ; done

by running two of them in parallel, along with an apt install loop in
parallel. It takes a few hours to reproduce (sometimes 1, sometimes
3...)

Samuel



Re: 64bit startup

2023-10-26 Thread Samuel Thibault
Samuel Thibault, le ven. 27 oct. 2023 00:42:06 +0200, a ecrit:
> Samuel Thibault, le mer. 25 oct. 2023 14:55:36 +0200, a ecrit:
> > Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> > > jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > > > October 25, 2023 3:43 AM, "Samuel Thibault"  
> > > > wrote:
> > > > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > > > 
> > > > >> Or maybe GCC is partly at fault for the
> > > > >> Hurd's X86_64 building troubles?
> > > > > 
> > > > > It's not at all. Nor is libtool.
> > > > > 
> > > > > I occasionally had issues in ./configure, too.
> > > > > 
> > > > > You'll say that's "yeah, it's all about auto-crap". No.
> > > > > 
> > > > > It's *very* most probably about bash, simply.
> > > > 
> > > > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> > > 
> > > It'd really better be short-term, because currently we cannot really
> > > trust the built packages: what if due to shell script misbehavior
> > > ./configure misdetects features, forgets enabling some support
> > > etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> > > down.
> > 
> > Today's gcc attempt:
> > 
> > Comparing stages 2 and 3
> > Bootstrap comparison failure!
> > libbacktrace/.libs/sort.o differs
> 
> Here is a good example:
> 
> f="internal/reflectlite.o"; if test ! -f $f; then 
> f="internal/.libs/reflectlite.o"; fi; x86_64-gnu-objcopy -j .go_export $f 
> internal/reflectlite.s-gox.tmp; /bin/bash ../../../src/libgo/mvifdiff.sh 
> internal/reflectlite.s-gox.tmp `echo internal/reflectlite.s-gox | sed -e 
> 's/s-gox/gox/'`
> mv: cannot move 'internal/reflectlite.s-gox.tmp' to '': No such file or 
> directory
> 
> It looks like the echo|sed part didn't work. (or command parameter
> passing to mvifdiff.sh, but I doubt that)

Indeed, 

while [  "$(echo -n `echo  internal/reflectlite.s-gox | sed -e 's/s-gox/gox/' ` 
)" = internal/reflectlite.gox ] ; do : ; done

does stop.

Samuel



Re: 64bit startup

2023-10-26 Thread Damien Zammit
Please check the locore.S on 64 bit. I think the int stack checks may not be 
pointing to the right location. I remember making some changes a long time ago 
without updating 64 bit because i had no way to test.

Damien

Sent from ProtonMail mobile

 Original Message 
On 27 Oct 2023, 10:05 am, Samuel Thibault wrote:

> Samuel Thibault, le mer. 25 oct. 2023 00:04:33 +0200, a ecrit:
>> Building packages is not very stable. I have been trying to build
>> gcc-13 for a couple of weeks, without success so far. There are various
>> failures, most often odd errors in the libtool script, which are a sign
>> that the system itself is not behaving correctly. A way to reproduce
>> the issue is to just repeatedly build a package that is using libtool,
>> sooner or later that will fail very oddly.
>>
>> This means that while the buildd will be ready, I'm really not at ease
>> with letting it start, knowing that it can behave erratically.
>
> Actually, until gcc-13 actually builds, nothing else can build since
> libc depends on libgcc.
>
> Samuel

Re: 64bit startup

2023-10-26 Thread Samuel Thibault
Samuel Thibault, le mer. 25 oct. 2023 00:04:33 +0200, a ecrit:
> Building packages is not very stable. I have been trying to build
> gcc-13 for a couple of weeks, without success so far. There are various
> failures, most often odd errors in the libtool script, which are a sign
> that the system itself is not behaving correctly. A way to reproduce
> the issue is to just repeatedly build a package that is using libtool,
> sooner or later that will fail very oddly.
> 
> This means that while the buildd will be ready, I'm really not at ease
> with letting it start, knowing that it can behave erratically.

Actually, until gcc-13 actually builds, nothing else can build since
libc depends on libgcc.

Samuel



Re: 64bit startup

2023-10-26 Thread Samuel Thibault
Samuel Thibault, le mer. 25 oct. 2023 14:55:36 +0200, a ecrit:
> Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> > jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > > October 25, 2023 3:43 AM, "Samuel Thibault"  
> > > wrote:
> > > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > > 
> > > >> Or maybe GCC is partly at fault for the
> > > >> Hurd's X86_64 building troubles?
> > > > 
> > > > It's not at all. Nor is libtool.
> > > > 
> > > > I occasionally had issues in ./configure, too.
> > > > 
> > > > You'll say that's "yeah, it's all about auto-crap". No.
> > > > 
> > > > It's *very* most probably about bash, simply.
> > > 
> > > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> > 
> > It'd really better be short-term, because currently we cannot really
> > trust the built packages: what if due to shell script misbehavior
> > ./configure misdetects features, forgets enabling some support
> > etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> > down.
> 
> Today's gcc attempt:
> 
> Comparing stages 2 and 3
> Bootstrap comparison failure!
> libbacktrace/.libs/sort.o differs

Here is a good example:

f="internal/reflectlite.o"; if test ! -f $f; then 
f="internal/.libs/reflectlite.o"; fi; x86_64-gnu-objcopy -j .go_export $f 
internal/reflectlite.s-gox.tmp; /bin/bash ../../../src/libgo/mvifdiff.sh 
internal/reflectlite.s-gox.tmp `echo internal/reflectlite.s-gox | sed -e 
's/s-gox/gox/'`
mv: cannot move 'internal/reflectlite.s-gox.tmp' to '': No such file or 
directory

It looks like the echo|sed part didn't work. (or command parameter
passing to mvifdiff.sh, but I doubt that)

Samuel



Re: 64bit startup

2023-10-25 Thread Samuel Thibault
Sergey Bugaev, le mer. 25 oct. 2023 16:29:29 +0300, a ecrit:
> On Wed, Oct 25, 2023 at 2:52 PM  wrote:
> >
> > October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> >
> > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > >
> > >> Or maybe GCC is partly at fault for the
> > >> Hurd's X86_64 building troubles?
> > >
> > > It's not at all. Nor is libtool.
> > >
> > > I occasionally had issues in ./configure, too.
> > >
> > > You'll say that's "yeah, it's all about auto-crap". No.
> > >
> > > It's *very* most probably about bash, simply.
> > >
> > > Samuel
> >
> > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> >
> > Could we change the default shell on X86_64 Debian Hurd in the meantime,
> > as a temporary solution?
> 
> I would rather ask, would it not be possible to set up a continuous
> build server (buildd? I know next to nothing about the Debian infra)
> that itself runs on a more stable architecture (amd64, or hurd-i386)
> and cross-compiles the packages?

Cross-compiling *very* often produces slightly bogus packages. They are
enough to bootstrap something you can build upon, but you cannot hope
more.

We do have cross-compiling boot strap set up on
https://jenkins.debian.net/view/rebootstrap/
but we don't want to upload the result, since when cross-compiling
there are various ./configure tests that you cannot run (execution-time
results).

Samuel



Re: 64bit startup

2023-10-25 Thread Sergey Bugaev
On Wed, Oct 25, 2023 at 2:52 PM  wrote:
>
> October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
>
> > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> >
> >> Or maybe GCC is partly at fault for the
> >> Hurd's X86_64 building troubles?
> >
> > It's not at all. Nor is libtool.
> >
> > I occasionally had issues in ./configure, too.
> >
> > You'll say that's "yeah, it's all about auto-crap". No.
> >
> > It's *very* most probably about bash, simply.
> >
> > Samuel
>
> Hmmm. I guess in the long-term then, the bash issues should be fixed.
>
> Could we change the default shell on X86_64 Debian Hurd in the meantime,
> as a temporary solution?

I would rather ask, would it not be possible to set up a continuous
build server (buildd? I know next to nothing about the Debian infra)
that itself runs on a more stable architecture (amd64, or hurd-i386)
and cross-compiles the packages?

Sergey



Re: 64bit startup

2023-10-25 Thread Samuel Thibault
Samuel Thibault, le mer. 25 oct. 2023 14:05:35 +0200, a ecrit:
> jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> > October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> > > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > > 
> > >> Or maybe GCC is partly at fault for the
> > >> Hurd's X86_64 building troubles?
> > > 
> > > It's not at all. Nor is libtool.
> > > 
> > > I occasionally had issues in ./configure, too.
> > > 
> > > You'll say that's "yeah, it's all about auto-crap". No.
> > > 
> > > It's *very* most probably about bash, simply.
> > 
> > Hmmm. I guess in the long-term then, the bash issues should be fixed.
> 
> It'd really better be short-term, because currently we cannot really
> trust the built packages: what if due to shell script misbehavior
> ./configure misdetects features, forgets enabling some support
> etc. That'd lead to subtle incompatibilities that'll be hard to hunt
> down.

Today's gcc attempt:

Comparing stages 2 and 3
Bootstrap comparison failure!
libbacktrace/.libs/sort.o differs

Samuel



Re: 64bit startup

2023-10-25 Thread Samuel Thibault
jbra...@dismail.de, le mer. 25 oct. 2023 11:52:02 +, a ecrit:
> October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:
> > jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> > 
> >> Or maybe GCC is partly at fault for the
> >> Hurd's X86_64 building troubles?
> > 
> > It's not at all. Nor is libtool.
> > 
> > I occasionally had issues in ./configure, too.
> > 
> > You'll say that's "yeah, it's all about auto-crap". No.
> > 
> > It's *very* most probably about bash, simply.
> 
> Hmmm. I guess in the long-term then, the bash issues should be fixed.

It'd really better be short-term, because currently we cannot really
trust the built packages: what if due to shell script misbehavior
./configure misdetects features, forgets enabling some support
etc. That'd lead to subtle incompatibilities that'll be hard to hunt
down.

> Could we change the default shell on X86_64 Debian Hurd in the meantime, 
> as a temporary solution?

Don't take me wrong: I'm not saying the concern is *because* of bash,
but *concerning* bash. Another shell could very well just face exactly
the same concerns.

And no, we cannot just switch it: libtool uses the bash features, so we
have to fix the behavior of the system for bash (and /bin/sh already
points to dash)

Samuel



Re: 64bit startup

2023-10-25 Thread jbranso
October 25, 2023 3:43 AM, "Samuel Thibault"  wrote:

> jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> 
>> Or maybe GCC is partly at fault for the
>> Hurd's X86_64 building troubles?
> 
> It's not at all. Nor is libtool.
> 
> I occasionally had issues in ./configure, too.
> 
> You'll say that's "yeah, it's all about auto-crap". No.
> 
> It's *very* most probably about bash, simply.
> 
> Samuel

Hmmm. I guess in the long-term then, the bash issues should be fixed.

Could we change the default shell on X86_64 Debian Hurd in the meantime, 
as a temporary solution? Or is that a silly suggestion? I assume the ksh
shell is simplier, mainly because that is the default shell in OpenBSD. 
Also, this is coming from someone who has yet to try the 64 bit Hurd. I
should really fire up qemu and give it a try.

Joshua



Re: 64bit startup

2023-10-25 Thread Martin Steigerwald
Hi Samuel, hi,

Samuel Thibault - 25.10.23, 00:04:33 CEST:
> Some update on the 64bit port:
> 
> - The debian-ports archive now has enough packages to bootstrap a
> chroot.
> - A 64bit debian buildd is getting set up, not much work is left there.
> - The hurd-amd64 wanna-build infrastructure is to be set up in the
> coming days.

Congratulations! This is a great achievement. I appreciate it.

> *but*

Even though there are still challenges ahead, this is a great 
accomplishment.

Thank you.

Best,
-- 
Martin





Re: 64bit startup

2023-10-25 Thread Samuel Thibault
Jeffrey Walton, le mar. 24 oct. 2023 22:00:54 -0400, a ecrit:
> On Tue, Oct 24, 2023 at 9:56 PM Jessica Clarke  wrote:
> >
> > On 25 Oct 2023, at 02:40, Jeffrey Walton  wrote:
> > >
> > > On Tue, Oct 24, 2023 at 9:33 PM Jessica Clarke  wrote:
> > >>
> > >> On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
> > >>>
> > >>> On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault 
> > >>>  wrote:
> > 
> >  Some update on the 64bit port:
> > 
> >  - The debian-ports archive now has enough packages to bootstrap a
> >  chroot.
> >  - A 64bit debian buildd is getting set up, not much work is left there.
> >  - The hurd-amd64 wanna-build infrastructure is to be set up in the
> >  coming days.
> > >>>
> > >>> Congrats
> > >>>
> >  *but*
> > 
> >  Building packages is not very stable. I have been trying to build
> >  gcc-13 for a couple of weeks, without success so far. There are various
> >  failures, most often odd errors in the libtool script, which are a sign
> >  that the system itself is not behaving correctly. A way to reproduce
> >  the issue is to just repeatedly build a package that is using libtool,
> >  sooner or later that will fail very oddly.
> > >>>
> > >>> lol...  and
> > >>
> > >> Yeah can we not spread this kind of vile rhetoric here? Regardless of
> > >> how much truth is in that, and whether it holds today, that kind of
> > >> language isn’t something we should be celebrating and encouraging
> > >> others to read. Let’s keep things more civil and on topic.
> > >
> > > My apologies for offending your delicate sensibilities.
> >
> > 1. I did not say I was offended. I said it was vile rhetoric. It does
> >not personally offend me, but that does not mean I want to see it
> >being circulated on these kinds of mailing lists.
> 
> Your overreaction. It looks like the stuff I would expect to see on
> social media, like one of those binary confused persons crying someone
> is perpetrating a hate crime because the wrong pronoun (subject?) was
> used.

*You* are overracting here.

Samuel



Re: 64bit startup

2023-10-25 Thread Samuel Thibault
jbra...@dismail.de, le mer. 25 oct. 2023 03:40:16 +, a ecrit:
> >> lol...  and
> > 
> > Exactly. So what?
> > 
> > Some folks have witty humor and appreciate the nostalgia. Others don't.
> > 
> > If you don't like my posts, then plonk me.
> > 
> > Jeff
> 
> I followed the link, and I did think it was a little funny. Perhaps
> it was a little off topic. Or maybe GCC is partly at fault for the
> Hurd's X86_64 building troubles?

It's not at all. Nor is libtool.

I occasionally had issues in ./configure, too.

You'll say that's "yeah, it's all about auto-crap". No.

It's *very* most probably about bash, simply.

(and while the gcc page makes sense (all very complex software have
a lot of bugs, so compilers have, but the message has to be terribly
moderated, considering the huge amount of tests that gcc receives), the
auto-hell page doesn't: nowhere was it ever suggested that it's normal
for software to run several ./configure instances).

So, yes, it was essentially off-topic.

Samuel



Re: 64bit startup

2023-10-24 Thread jbranso
>> lol...  and
> 
> Exactly. So what?
> 
> Some folks have witty humor and appreciate the nostalgia. Others don't.
> 
> If you don't like my posts, then plonk me.
> 
> Jeff

I followed the link, and I did think it was a little funny. Perhaps
it was a little off topic. Or maybe GCC is partly at fault for the
Hurd's X86_64 building troubles?

But can we all remember that we are friends here? We all want the
Hurd to succeed right?

Thanks,

Joshua



Re: 64bit startup

2023-10-24 Thread Jeffrey Walton
On Tue, Oct 24, 2023 at 9:56 PM Jessica Clarke  wrote:
>
> On 25 Oct 2023, at 02:40, Jeffrey Walton  wrote:
> >
> > On Tue, Oct 24, 2023 at 9:33 PM Jessica Clarke  wrote:
> >>
> >> On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
> >>>
> >>> On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault  
> >>> wrote:
> 
>  Some update on the 64bit port:
> 
>  - The debian-ports archive now has enough packages to bootstrap a
>  chroot.
>  - A 64bit debian buildd is getting set up, not much work is left there.
>  - The hurd-amd64 wanna-build infrastructure is to be set up in the
>  coming days.
> >>>
> >>> Congrats
> >>>
>  *but*
> 
>  Building packages is not very stable. I have been trying to build
>  gcc-13 for a couple of weeks, without success so far. There are various
>  failures, most often odd errors in the libtool script, which are a sign
>  that the system itself is not behaving correctly. A way to reproduce
>  the issue is to just repeatedly build a package that is using libtool,
>  sooner or later that will fail very oddly.
> >>>
> >>> lol...  and
> >>
> >> Yeah can we not spread this kind of vile rhetoric here? Regardless of
> >> how much truth is in that, and whether it holds today, that kind of
> >> language isn’t something we should be celebrating and encouraging
> >> others to read. Let’s keep things more civil and on topic.
> >
> > My apologies for offending your delicate sensibilities.
>
> 1. I did not say I was offended. I said it was vile rhetoric. It does
>not personally offend me, but that does not mean I want to see it
>being circulated on these kinds of mailing lists.

Your overreaction. It looks like the stuff I would expect to see on
social media, like one of those binary confused persons crying someone
is perpetrating a hate crime because the wrong pronoun (subject?) was
used.

> 2. Even if it did, so what? Per the Debian Mailing Lists’ Code of
>Conduct, a superset of Debian’s. Off-topic and unwelcoming content is
>against those. Even if it wasn’t explicitly written down as a rule,
>though, the decent response to “don’t send unwelcoming content” isn’t
>“I’m sorry you’re so sensitive” but “sorry, I won’t do it again”. So
>kindly act decently or don’t contribute.

Exactly. So what?

Some folks have witty humor and appreciate the nostalgia. Others don't.

If you don't like my posts, then plonk me.

Jeff



Re: 64bit startup

2023-10-24 Thread Jessica Clarke
On 25 Oct 2023, at 02:40, Jeffrey Walton  wrote:
> 
> On Tue, Oct 24, 2023 at 9:33 PM Jessica Clarke  wrote:
>> 
>> On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
>>> 
>>> On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault  
>>> wrote:
 
 Some update on the 64bit port:
 
 - The debian-ports archive now has enough packages to bootstrap a
 chroot.
 - A 64bit debian buildd is getting set up, not much work is left there.
 - The hurd-amd64 wanna-build infrastructure is to be set up in the
 coming days.
>>> 
>>> Congrats
>>> 
 *but*
 
 Building packages is not very stable. I have been trying to build
 gcc-13 for a couple of weeks, without success so far. There are various
 failures, most often odd errors in the libtool script, which are a sign
 that the system itself is not behaving correctly. A way to reproduce
 the issue is to just repeatedly build a package that is using libtool,
 sooner or later that will fail very oddly.
>>> 
>>> lol...  and
>> 
>> Yeah can we not spread this kind of vile rhetoric here? Regardless of
>> how much truth is in that, and whether it holds today, that kind of
>> language isn’t something we should be celebrating and encouraging
>> others to read. Let’s keep things more civil and on topic.
> 
> My apologies for offending your delicate sensibilities.

1. I did not say I was offended. I said it was vile rhetoric. It does
   not personally offend me, but that does not mean I want to see it
   being circulated on these kinds of mailing lists.

2. Even if it did, so what? Per the Debian Mailing Lists’ Code of
   Conduct, a superset of Debian’s. Off-topic and unwelcoming content is
   against those. Even if it wasn’t explicitly written down as a rule,
   though, the decent response to “don’t send unwelcoming content” isn’t
   “I’m sorry you’re so sensitive” but “sorry, I won’t do it again”. So
   kindly act decently or don’t contribute.

Jess




Re: 64bit startup

2023-10-24 Thread Jeffrey Walton
On Tue, Oct 24, 2023 at 9:33 PM Jessica Clarke  wrote:
>
> On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
> >
> > On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault  
> > wrote:
> >>
> >> Some update on the 64bit port:
> >>
> >> - The debian-ports archive now has enough packages to bootstrap a
> >> chroot.
> >> - A 64bit debian buildd is getting set up, not much work is left there.
> >> - The hurd-amd64 wanna-build infrastructure is to be set up in the
> >> coming days.
> >
> > Congrats
> >
> >> *but*
> >>
> >> Building packages is not very stable. I have been trying to build
> >> gcc-13 for a couple of weeks, without success so far. There are various
> >> failures, most often odd errors in the libtool script, which are a sign
> >> that the system itself is not behaving correctly. A way to reproduce
> >> the issue is to just repeatedly build a package that is using libtool,
> >> sooner or later that will fail very oddly.
> >
> > lol...  and
>
> Yeah can we not spread this kind of vile rhetoric here? Regardless of
> how much truth is in that, and whether it holds today, that kind of
> language isn’t something we should be celebrating and encouraging
> others to read. Let’s keep things more civil and on topic.

My apologies for offending your delicate sensibilities.

Jeff



Re: 64bit startup

2023-10-24 Thread Jessica Clarke
On 25 Oct 2023, at 02:26, Jeffrey Walton  wrote:
> 
> On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault  
> wrote:
>> 
>> Some update on the 64bit port:
>> 
>> - The debian-ports archive now has enough packages to bootstrap a
>> chroot.
>> - A 64bit debian buildd is getting set up, not much work is left there.
>> - The hurd-amd64 wanna-build infrastructure is to be set up in the
>> coming days.
> 
> Congrats
> 
>> *but*
>> 
>> Building packages is not very stable. I have been trying to build
>> gcc-13 for a couple of weeks, without success so far. There are various
>> failures, most often odd errors in the libtool script, which are a sign
>> that the system itself is not behaving correctly. A way to reproduce
>> the issue is to just repeatedly build a package that is using libtool,
>> sooner or later that will fail very oddly.
> 
> lol...  and

Yeah can we not spread this kind of vile rhetoric here? Regardless of
how much truth is in that, and whether it holds today, that kind of
language isn’t something we should be celebrating and encouraging
others to read. Let’s keep things more civil and on topic.

Jess

> .
> 
> Jeff
> 




Re: 64bit startup

2023-10-24 Thread Jeffrey Walton
On Tue, Oct 24, 2023 at 6:21 PM Samuel Thibault  wrote:
>
> Some update on the 64bit port:
>
> - The debian-ports archive now has enough packages to bootstrap a
> chroot.
> - A 64bit debian buildd is getting set up, not much work is left there.
> - The hurd-amd64 wanna-build infrastructure is to be set up in the
> coming days.

Congrats

> *but*
>
> Building packages is not very stable. I have been trying to build
> gcc-13 for a couple of weeks, without success so far. There are various
> failures, most often odd errors in the libtool script, which are a sign
> that the system itself is not behaving correctly. A way to reproduce
> the issue is to just repeatedly build a package that is using libtool,
> sooner or later that will fail very oddly.

lol...  and
.

Jeff



Re: 64bit startup

2023-10-24 Thread Samuel Thibault
Hello,

Some update on the 64bit port:

- The debian-ports archive now has enough packages to bootstrap a
chroot.
- A 64bit debian buildd is getting set up, not much work is left there.
- The hurd-amd64 wanna-build infrastructure is to be set up in the
coming days.

*but*

Building packages is not very stable. I have been trying to build
gcc-13 for a couple of weeks, without success so far. There are various
failures, most often odd errors in the libtool script, which are a sign
that the system itself is not behaving correctly. A way to reproduce
the issue is to just repeatedly build a package that is using libtool,
sooner or later that will fail very oddly.

This means that while the buildd will be ready, I'm really not at ease
with letting it start, knowing that it can behave erratically. When I
built the initial set of packages for debian-ports (~100 packages), I
got something like 5-10 such failures, that's quite high of a rate :/

Samuel



Re: 64bit startup

2023-10-01 Thread Samuel Thibault
Hello,

Good news!  It seems I fixed the bug that was making my 64bit VM
crashing quite often. The problem was that when receiving a message from
userland, ipc_kmsg_get would allocate a kernel buffer with the same size
as the userland message. But since we may expand the 32bit port names
into 64bit port addresses, this is not large enough.

I'm almost finished with building the base set of debian packages,
required to build & upload packages on the debian-ports archive (just
missing cmake that exposes some bootstrap bugs).

Before building the hurd-amd64 world, are we sure we are all set with
the ABI details?  It'll be way harder to fix later on.

Samuel



Re: 64bit startup

2023-08-15 Thread Samuel Thibault
Samuel Thibault, le sam. 12 août 2023 17:37:06 +0200, a ecrit:
> BTW, git is now available and seems to be working fine, I could clone
> the upstream glibc repository for instance.

The testsuite runs quite fine. Of course most xfails on hurd-i386 are
also xfails on hurd-amd64 :)

I have put below the list of xfails for hurd-amd64, most of them are
probably worth looking at, notably the default FPU config (to be fixed
inside gnumach), the context support, and the backtrace support, which
we will want anyway, and then the bug fixes, to be on part with
hurd-i386 in terms of bugs :)

Samuel

# TODO: fix default FPU config
test-xfail-test-fenv = yes
test-xfail-test-float64x-acos = yes
test-xfail-test-float64x-log10 = yes
test-xfail-test-float64x-log2 = yes
test-xfail-test-float64x-y0 = yes
test-xfail-test-float64x-y1 = yes
test-xfail-test-ldouble-acos = yes
test-xfail-test-ldouble-log10 = yes
test-xfail-test-ldouble-log2 = yes
test-xfail-test-ldouble-y0 = yes
test-xfail-test-ldouble-y1 = yes

# TODO context support
test-xfail-bug-getcontext = yes
test-xfail-tst-setcontext2 = yes
test-xfail-tst-setcontext4 = yes
test-xfail-tst-setcontext5 = yes
test-xfail-tst-setcontext6 = yes
test-xfail-tst-setcontext7 = yes
test-xfail-tst-setcontext8 = yes
test-xfail-tst-setcontext9 = yes
test-xfail-tst-swapcontext1 = yes
test-xfail-tst-xbzero-opt = yes

# Bus error
test-xfail-test-bz22786 = yes

# memory leak
test-xfail-tst-vfprintf-width-prec-mem = yes
test-xfail-tst-vfprintf-width-prec = yes

# timeout
test-xfail-tst-basic7 = yes

# timeout
test-xfail-tst-malloc-too-large = yes
test-xfail-tst-malloc-too-large-malloc-check = yes
test-xfail-tst-malloc-too-large-hugetlb1 = yes
test-xfail-tst-malloc-too-large-hugetlb2 = yes

# Bus error
test-xfail-bug18240 = yes

# cmsg bug
test-xfail-tst-cmsghdr = yes

# TODO support (for signals I guess)
test-xfail-tst-backtrace4 = yes
test-xfail-tst-backtrace5 = yes
test-xfail-tst-backtrace6 = yes

test-xfail-tst-dlopen-nodelete-reloc = yes
test-xfail-tst-platform-1 = yes
test-xfail-tst-audit4 = yes
test-xfail-tst-audit5 = yes
test-xfail-tst-audit6 = yes
test-xfail-tst-audit7 = yes
test-xfail-tst-audit10 = yes



Re: 64bit startup

2023-08-15 Thread Samuel Thibault
For information, core dumping seems to be broken, so better disable it
with 

rm -f /servers/crash
ln -s crash-kill /servers/crash

otherwise you get hangs or worse ;)

Samuel



Re: 64bit startup

2023-08-15 Thread Guy-Fleury Iteriteka
On August 15, 2023 4:11:08 PM GMT+02:00, jbra...@dismail.de wrote:
>August 15, 2023 12:36 AM, "Guy-Fleury Iteriteka"  wrote:
>
>> On August 14, 2023 10:48:57 PM GMT+02:00, Samuel Thibault 
>>  wrote:
>> 
>>> Samuel Thibault, le sam. 12 août 2023 17:37:06 +0200, a ecrit:
>> 
>> Thats great,thanks. On irc you mention building python too cause many 
>> problems. Today i am going to
>> search a good computer so that i learn programming in hurd
>>> Samuel
>> 
>
>I've got the Hurd running pretty well on a T43.   But that is only a 32-bit 
>single core machine.
>
>It also runs really well in qemu.  Many Hurd developers run the Hurd in qemu.  
>:)

I know that Yes. My current compter have an issue where some key are not 
working so it's not pratical for programming. 
>




Re: 64bit startup

2023-08-15 Thread jbranso
August 15, 2023 12:36 AM, "Guy-Fleury Iteriteka"  wrote:

> On August 14, 2023 10:48:57 PM GMT+02:00, Samuel Thibault 
>  wrote:
> 
>> Samuel Thibault, le sam. 12 août 2023 17:37:06 +0200, a ecrit:
> 
> Thats great,thanks. On irc you mention building python too cause many 
> problems. Today i am going to
> search a good computer so that i learn programming in hurd
>> Samuel
> 

I've got the Hurd running pretty well on a T43.   But that is only a 32-bit 
single core machine.

It also runs really well in qemu.  Many Hurd developers run the Hurd in qemu.  
:)



Re: 64bit startup

2023-08-14 Thread Guy-Fleury Iteriteka
On August 14, 2023 10:48:57 PM GMT+02:00, Samuel Thibault 
 wrote:
>Samuel Thibault, le sam. 12 août 2023 17:37:06 +0200, a ecrit:
>> BTW, git is now available and seems to be working fine, I could clone
>> the upstream glibc repository for instance.
>
>I had a lot of troubles building glibc. We had a nasty bug in
>pmap_remove and pmap_protect which were bringing mayhem, now fixed,
>things now work *WAY* better :D
>
Thats great,thanks. On irc you mention building python too cause many problems. 
 Today i am going to search a good computer so that i learn programming in hurd
>Samuel
>

Hello,



Re: 64bit startup

2023-08-14 Thread Samuel Thibault
Samuel Thibault, le sam. 12 août 2023 17:37:06 +0200, a ecrit:
> BTW, git is now available and seems to be working fine, I could clone
> the upstream glibc repository for instance.

I had a lot of troubles building glibc. We had a nasty bug in
pmap_remove and pmap_protect which were bringing mayhem, now fixed,
things now work *WAY* better :D

Samuel



Re: 64bit startup

2023-08-12 Thread Samuel Thibault
BTW, git is now available and seems to be working fine, I could clone
the upstream glibc repository for instance.

Samuel



Re: 64bit startup

2023-08-12 Thread Samuel Thibault
Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> The issue I'm still seeing now is that I cannot get to install some
> packages, such as libicu72. When I pass it as extra package to
> debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> things halt.

I found the issue. pmap_protect wasn't updated for sparse pde array.
Whenever ext2fs would ping memory beyond a pde limit, things would go
mayhem.

And since nobody had yet fixed the debugging tools for x86_64
completely, the symptom was terribly far from the real issue.

In the end I eventually forged simple_lock_irq helpers to possibly fix
some bugs, at least it'll make maintenance of spl for simple_locks
simpler. I'll push that soon.

Generally, please do not underestimate the vast benefit of fixing our
debugging tools.

Concerning the x86_64 port, we very probably want to port gdb ASAP, to
be able to e.g. fix the glibc testsuite. For instance, I see that perl's
configure hangs at some point, apparently due to a buffering issue in
the "<< EOF" processing of the shell.

Samuel



Re: 64bit startup

2023-08-10 Thread Samuel Thibault
Luca, le jeu. 10 août 2023 21:58:49 +0200, a ecrit:
> By the way, if I try to add the repository key with
> 
> apt-key adv --recv-keys --keyserver keyring.debian.org
> 900CB024B67931D40F82304BD0178C767D069EE6
> 
> it fails with
> 
> E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them is
> required for this operation
> 
> Were there any build failures with gnupg or was it just left out?

I just didn't include them in the image, to make it lighter. You can
install them, I had just forgotten to add the gnupg and gnupg2 arch:all
packages to the repo, now done so.

> (I see other gnupg-related packages built successfully, e.g. gpgv). I
> can bypass authentication anyway for now with --allow-unauthenticated
> and --allow-insecure-repositories.

You can also modify sources.list to add [trusted=yes]

Samuel



Re: 64bit startup

2023-08-10 Thread Luca

Il 09/08/23 19:57, Samuel Thibault ha scritto:

Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:

I continued stabbing at the network issue, it was just a size alignment
problem, so that depending on their sizes, half of the network packets
would be discarded by mach_msg. Hurd-amd64 packages are getting rebuilt
to contain the fix.


They're rebuilt, the image as well.

Along the way, I fixed the hurd package getting built with pie enabled,
that was a missing commit to be uploaded to dpkg.

I'll also try to build with nostrip. In case it's not too big that'll
make debugging even easier.


Thanks! now I am able to set up networking so I can install some more 
stuff, I can also ssh from my host to the 64-bit hurd :D


By the way, if I try to add the repository key with

apt-key adv --recv-keys --keyserver keyring.debian.org 
900CB024B67931D40F82304BD0178C767D069EE6


it fails with

E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them 
is required for this operation


Were there any build failures with gnupg or was it just left out? (I see 
other gnupg-related packages built successfully, e.g. gpgv). I can 
bypass authentication anyway for now with --allow-unauthenticated and 
--allow-insecure-repositories.



Luca



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> The issue I'm still seeing now is that I cannot get to install some
> packages, such as libicu72. When I pass it as extra package to
> debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> things halt.

It seems that the most common case is that rumpdisk gets stuck. I guess
we have e.g. a lost interrupt which makes rumpdisk endlessly wait for
termination of a write. We can avoid this issue by using a ramdisk.


Another case seems like a four-players issue.

- one ext2fs thread is doing diskfs_S_file_syncfs, it's stuck
  inside the helper calling fsys_syncfs on some other translator

  and it holds the translator_ihash_lock lock

- one ext2fs thread is trying to call fshelp_set_active_translator from
  dir_lookup. It is stuck on trying to acquire translator_ihash_lock,
  and it holds the np.

- one ext2fs thread doing the diskfs_sync_everything, stuck in
  diskfs_node_iterate trying to lock a node, while holding the
  nodecache_lock.

So in the end nodecache_lock is stuck just because some translator is
stuck on fsys_syncfs.

It's unfortunate that we had never seen that before and it's appearing
now at the worst time for the hurd-amd64 bootstrap :)

We can avoid this issue by using eatmydata, to prevent dpkg from running
syncs.

Samuel



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Samuel Thibault, le mer. 09 août 2023 22:58:20 +0200, a ecrit:
> Samuel Thibault, le mar. 08 août 2023 23:28:02 +0200, a ecrit:
> > Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> > > The issue I'm still seeing now is that I cannot get to install some
> > > packages, such as libicu72. When I pass it as extra package to
> > > debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> > > things halt.
> > 
> > BTW, to get rumpdisk out of the equation, it would be useful to work on
> > moving the kernel map below 0x8000, so we do not get any
> > size constraint on it, and thus are able to load initrds of whatever
> > size, provided that we have the memory.
> 
> BTW, using gunzip:device:rd0 and module --nounzip in grub allows to get
> grub to load the gz version and let ext2fs unzip it into memory, so the
> memory pressure is less of a concern, and one can create a large sparse
> image, that'll be all in memory without making the initrd.gz large.

Also, that allows the ramdisk pages to go to high-memory rather than
stay stuck below 4G where grub could load it.

Samuel



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Samuel Thibault, le mar. 08 août 2023 23:28:02 +0200, a ecrit:
> Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> > The issue I'm still seeing now is that I cannot get to install some
> > packages, such as libicu72. When I pass it as extra package to
> > debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> > things halt.
> 
> BTW, to get rumpdisk out of the equation, it would be useful to work on
> moving the kernel map below 0x8000, so we do not get any
> size constraint on it, and thus are able to load initrds of whatever
> size, provided that we have the memory.

BTW, using gunzip:device:rd0 and module --nounzip in grub allows to get
grub to load the gz version and let ext2fs unzip it into memory, so the
memory pressure is less of a concern, and one can create a large sparse
image, that'll be all in memory without making the initrd.gz large.

Samuel



gnumach kernel memory map (was: Re: 64bit startup)

2023-08-09 Thread Luca

Il 09/08/23 11:20, Samuel Thibault ha scritto:

Sergey Bugaev, le mer. 09 août 2023 12:12:29 +0300, a ecrit:

It should just reuse whatever memory the bootloader has already loaded
the module into, no need to copy it out anywhere.


(my guess is that it's not so simple because possibly we somehow get rid
of the memory allocated for modules, otherwise we'd get stuck with that
piece of memory area that we cannot reuse for other system memory uses)


From what I understand so far it's a bit tricky, especially in the 
early boot phase. The current kernel map on x86_64 is very very simple, 
and inherits some constraints from the 32-bit map for simplicity. It's 
also partly limited by the simple bootstrapping procedure in boothdr.S. 
Some issues that I found so far are e.g.:
* decouple the bootstrap map created in boothdr.S from the runtime map 
and initalize it properly in pmap_bootstarap() (see also _phystokv vs 
phystokv). There is some memory allocated in the early C code that might 
need to be remapped (e.g the kernel page table map and the biosmem heap)
* the kernel .text/.bss/.data etc seem to be the only parts required to 
be in the upper 2G, due to mcmodel=kernel. Everything else could be 
anywhere.
* highmem is somehow limited by VM_KERNEL_MAP_SIZE, which itself is 
currently limited by the 2G range
* in some places it seems there is the assumption that memory is 
directly mapped (i.e. with a simple offset between virtual/physical 
address), I'm not sure how pervasive this assumption is and how 
difficult it would be to remove it where it's not needed.


Some time ago I started a first step in this direction, attempting to 
move down the kernel map so the highmem/directmap regions could be made 
much bigger, and still keeping the static data in the upper 2G. I 
uploaded it here [0] but it's really just a first step.


My idea here was to have as a first step the various 
dma/directnap/highmem regions between -4G and -2G and still contiguous, 
and then try to leave some red zone between them and make the regions 
bigger, once the limits could be freely moved.


For the bootstrap modules, it might be enough to explicitly reserve the 
memory in pmap_bootstrap(), although that would consume "low" memory. 
Once we have bigger memory regions, it shouldn't be too hard to move the 
bootstrap module somewhere else.


Any thoughts? I have probably overlooked some other issues, so please 
correct me if I'm wrong.


Luca


[0] 
https://gitlab.com/luckyd/gnumach/-/commit/f0564e8ed63b82b4e8b0342821c37f2461625438




Re: 64bit startup

2023-08-09 Thread Sergey Bugaev
On Wed, Aug 9, 2023 at 8:57 PM Samuel Thibault  wrote:
>
> Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> > I continued stabbing at the network issue, it was just a size alignment
> > problem, so that depending on their sizes, half of the network packets
> > would be discarded by mach_msg. Hurd-amd64 packages are getting rebuilt
> > to contain the fix.
>
> They're rebuilt, the image as well.
>
> Along the way, I fixed the hurd package getting built with pie enabled,
> that was a missing commit to be uploaded to dpkg.
>
> I'll also try to build with nostrip. In case it's not too big that'll
> make debugging even easier.

Awesome thanks, I'll check it out.

FWIW, I hacked up a basic version of vmobjects-as-ramdisk-repalcement;
and as a side effect done some serious surgery to how VM objects'
pagers are created/initialized. It all looks pretty nice, but it
doesn't yet boot :|

Sergey



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> I continued stabbing at the network issue, it was just a size alignment
> problem, so that depending on their sizes, half of the network packets
> would be discarded by mach_msg. Hurd-amd64 packages are getting rebuilt
> to contain the fix.

They're rebuilt, the image as well.

Along the way, I fixed the hurd package getting built with pie enabled,
that was a missing commit to be uploaded to dpkg.

I'll also try to build with nostrip. In case it's not too big that'll
make debugging even easier.

Samuel



Re: 64bit startup

2023-08-09 Thread Sergey Bugaev
On Wed, Aug 9, 2023 at 12:20 PM Samuel Thibault  wrote:
> Sergey Bugaev, le mer. 09 août 2023 12:12:29 +0300, a ecrit:
> > It should just reuse whatever memory the bootloader has already loaded
> > the module into, no need to copy it out anywhere.
>
> (my guess is that it's not so simple because possibly we somehow get rid
> of the memory allocated for modules, otherwise we'd get stuck with that
> piece of memory area that we cannot reuse for other system memory uses)

We do that yeah (see free_bootstrap_pages in kern/bootstrap.h);

there should be a way to turn that off. For now I'm thinking I'll just
set mod_end = mod_start = 0 for the bootstrap module that I take pages
from; later we may do something more advanced like making each
bootscript command free its module pages if it doesn't end up using
them, as opposed to freeing all the modules later in single loop. I'd
expect all pages to basically always get used, so we'd never actually
have to free them at boot time.

Sergey



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Sergey Bugaev, le mer. 09 août 2023 12:12:29 +0300, a ecrit:
> It should just reuse whatever memory the bootloader has already loaded
> the module into, no need to copy it out anywhere.

(my guess is that it's not so simple because possibly we somehow get rid
of the memory allocated for modules, otherwise we'd get stuck with that
piece of memory area that we cannot reuse for other system memory uses)

Samuel



Re: 64bit startup

2023-08-09 Thread Samuel Thibault
Sergey Bugaev, le mer. 09 août 2023 12:12:29 +0300, a ecrit:
> On Wed, Aug 9, 2023 at 12:29 AM Samuel Thibault  
> wrote:
> > BTW, to get rumpdisk out of the equation, it would be useful to work on
> > moving the kernel map below 0x8000, so we do not get any
> > size constraint on it, and thus are able to load initrds of whatever
> > size, provided that we have the memory.
> 
> Does ramdisk use kmalloc memory? That's horrifying.
> 
> It should

Help welcome!

Samuel



Re: 64bit startup

2023-08-09 Thread Sergey Bugaev
On Wed, Aug 9, 2023 at 12:29 AM Samuel Thibault  wrote:
> BTW, to get rumpdisk out of the equation, it would be useful to work on
> moving the kernel map below 0x8000, so we do not get any
> size constraint on it, and thus are able to load initrds of whatever
> size, provided that we have the memory.

Does ramdisk use kmalloc memory? That's horrifying.

It should just reuse whatever memory the bootloader has already loaded
the module into, no need to copy it out anywhere. As we discussed
before, this goes really nicely with VM objects: it should just create
an internal VM object out of those pages and expose its pager port to
the bootstrap tasks which can then map it using libstore's memobj
backend. The data / VM object's size is only limited by available RAM
then, you could have multi-GB-sized ramdisks if you want to.

I'll look into implementing this then.

We could also attempt to do the same when exec'ing ELFs: the data is
already loaded into the physical pages by the boot loader, no need to
allocate more pages, make a copy, and then free the original ones.

Sergey



Re: 64bit startup

2023-08-08 Thread Samuel Thibault
Samuel Thibault, le mar. 08 août 2023 19:15:50 +0200, a ecrit:
> The issue I'm still seeing now is that I cannot get to install some
> packages, such as libicu72. When I pass it as extra package to
> debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> things halt.

BTW, to get rumpdisk out of the equation, it would be useful to work on
moving the kernel map below 0x8000, so we do not get any
size constraint on it, and thus are able to load initrds of whatever
size, provided that we have the memory.

Samuel



Re: 64bit startup

2023-08-08 Thread Sergey Bugaev
On Tue, Aug 8, 2023 at 9:16 PM Samuel Thibault  wrote:
> I have already upgraded the source being used to 2.38. To keep things
> simple, we will bootstrap from that, not from 2.36 or 2.37.

Ah cool, you're one step ahead of me :)

Sergey



Re: 64bit startup

2023-08-08 Thread Samuel Thibault
Sergey Bugaev, le mar. 08 août 2023 21:12:05 +0300, a ecrit:
> There's another issue we need to sort out: is it OK for Debian to ship
> glibc 2.36 whose symbols claim to be GLIBC_2.38?

I have already upgraded the source being used to 2.38. To keep things
simple, we will bootstrap from that, not from 2.36 or 2.37.

Samuel



Re: 64bit startup

2023-08-08 Thread Sergey Bugaev
Hello!

On Tue, Aug 8, 2023 at 8:16 PM Samuel Thibault  wrote:
> Hello,
>
> I continued stabbing at the network issue, it was just a size alignment
> problem, so that depending on their sizes, half of the network packets
> would be discarded by mach_msg. Hurd-amd64 packages are getting rebuilt
> to contain the fix.

Awesome!

> The issue I'm still seeing now is that I cannot get to install some
> packages, such as libicu72. When I pass it as extra package to
> debootstrap, it does get installed, but when I "dpkg -i" it by hand,
> things halt.
>
> And there is still the 64bit change proposed proposed by Flavio that
> needs to get reviewed.

There's another issue we need to sort out: is it OK for Debian to ship
glibc 2.36 whose symbols claim to be GLIBC_2.38? It doesn't look like
there's been any compat symbols (as in SHLIB_COMPAT / compat_symbol)
added in between, but programs built against actual glibc 2.38 link
against newer symbols that just aren't present in 2.36 (such as those
__isoc23 functions), and you get symbol errors at runtime, instead of
the usual error about the required version not being present. Could
there be other side effects?

Sergey



Re: 64bit startup

2023-08-08 Thread Samuel Thibault
Hello,

I continued stabbing at the network issue, it was just a size alignment
problem, so that depending on their sizes, half of the network packets
would be discarded by mach_msg. Hurd-amd64 packages are getting rebuilt
to contain the fix.

The issue I'm still seeing now is that I cannot get to install some
packages, such as libicu72. When I pass it as extra package to
debootstrap, it does get installed, but when I "dpkg -i" it by hand,
things halt.

And there is still the 64bit change proposed proposed by Flavio that
needs to get reviewed.

Samuel



Re: 64bit startup

2023-06-20 Thread Sergey Bugaev
On Tue, Jun 20, 2023 at 5:07 PM Sergey Bugaev  wrote:
> ...no it does not! -- and it's visible from the output I posted. ASLR
> works, the huge heap mapping works, but not the other two things.
> Investigating now.

I must have just used the wrong exec server accidentally. Here:

# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x0002319bd000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x0002361d5000)
/lib/ld-x86-64.so.1 (0x0001)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1 (0x00023465e000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3 (0x000235cd4000)
# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x004be7fe4000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x004bd3123000)
/lib/ld-x86-64.so.1 (0x0001)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1 (0x004bce5b9000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3 (0x004bcce5b000)
# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x00469d756000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x00467b83a000)
/lib/ld-x86-64.so.1 (0x0001)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1 (0x00467946)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3 (0x00467990e000)
# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x0183a517d000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x0183cd306000)
/lib/ld-x86-64.so.1 (0x0001)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1 (0x0183da6e3000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3 (0x0183d4995000)
# vminfo $$
 0[0x1] (prot=0)
0x1[0x10f000] (prot=RX, max_prot=RWX, mem_obj=11)
0x10010f000[0x4000] (prot=R, max_prot=RWX, mem_obj=11)
0x100113000[0x8000] (prot=RW, max_prot=RWX, mem_obj=11)
0x10011b000[0x1000] (prot=RW, max_prot=RWX, mem_obj=11)
0x10011c000[0x9000] (prot=RW, max_prot=RWX, mem_obj=4)
0x100125000[0x36000] (prot=RX, max_prot=RWX, mem_obj=10)
0x10015b000[0x2000] (prot=R, max_prot=RWX, mem_obj=10)
0x10015d000[0x1000] (prot=RW, max_prot=RWX, mem_obj=10)
0x10015e000[0x1000] (prot=RW, max_prot=RWX, mem_obj=16)
0x10015f000[0x1000] (prot=0, max_prot=RWX, offs=0x1000)
0x10016[0x100] (prot=RWX, mem_obj=17)
0x2000[0x21000] (prot=RWX, mem_obj=18)
0x20021000[0x21000] (prot=RWX, mem_obj=19)
0x20042000[0x7fbe000] (prot=0, max_prot=RWX, offs=0x42000)
0x70decf7a5000[0x1000] (prot=RW, max_prot=RWX, mem_obj=20)
0x7d938a35[0x1000] (prot=RW, max_prot=RWX, mem_obj=22)
0x7d938a38d000[0x2e000] (prot=RX, max_prot=RWX, mem_obj=23)
0x7d938a3bb000[0x1000] (prot=R, max_prot=RWX, mem_obj=23)
0x7d938a3bc000[0x1000] (prot=RW, max_prot=RWX, mem_obj=23)
0x7d938a3bd000[0x1000] (prot=0, max_prot=RWX)
0x7d938a3be000[0x8000] (prot=RW, max_prot=RWX, mem_obj=24)
0x7d938a3cf000[0x14000] (prot=RX, max_prot=RWX, mem_obj=25)
0x7d938a3e3000[0x1000] (prot=R, max_prot=RWX, mem_obj=25)
0x7d938a3e4000[0x1000] (prot=RW, max_prot=RWX, mem_obj=25)
0x7d938a3e5000[0x2000] (prot=RW, max_prot=RWX, mem_obj=26)
0x7d938a3e7000[0x3000] (prot=RW, max_prot=RWX, mem_obj=27)
0x7d938a3ea000[0x3000] (prot=RW, max_prot=RWX, mem_obj=28)
0x7d938a3ed000[0x2000] (prot=RW, max_prot=RWX, mem_obj=29)
0x7d938a3ef000[0x2000] (prot=RW, max_prot=RWX, mem_obj=30)
0x7d938a3f1000[0x233000] (prot=RX, max_prot=RWX, mem_obj=31)
0x7d938a624000[0x4000] (prot=R, max_prot=RWX, mem_obj=31)
0x7d938a628000[0x3000] (prot=RW, max_prot=RWX, mem_obj=31)
0x7d938a62b000[0x5000] (prot=RW, max_prot=RWX, mem_obj=32)
0x7d938b563000[0x2c000] (prot=RX, max_prot=RWX, mem_obj=33)
0x7d938b58f000[0x4000] (prot=R, max_prot=RWX, mem_obj=33)
0x7d938b593000[0x1000] (prot=RW, max_prot=RWX, mem_obj=33, offs=0x4000)
0x7d938b7a3000[0x2000] (prot=RW, max_prot=RWX, mem_obj=34)



Re: 64bit startup

2023-06-20 Thread Sergey Bugaev
On Tue, Jun 20, 2023 at 4:55 PM Sergey Bugaev  wrote:
> Works beautifully, thank you :D

...no it does not! -- and it's visible from the output I posted. ASLR
works, the huge heap mapping works, but not the other two things.
Investigating now.

Sergey



Re: 64bit startup

2023-06-20 Thread Sergey Bugaev
On Tue, Jun 20, 2023 at 4:00 PM Samuel Thibault  wrote:
> (done so)

Works beautifully, thank you :D

root@hurd:/# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x01dc)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x0100e000)
/lib/ld-x86-64.so.1 (0x0800)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1
(0x01e73000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3
(0x01e31000)
root@hurd:/# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x03785000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x02ecb000)
/lib/ld-x86-64.so.1 (0x0800)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1
(0x037d3000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3
(0x03713000)
root@hurd:/# ldd /bin/bash
libtinfo.so.6 => /lib/x86_64-gnu/libtinfo.so.6 (0x03c2b000)
libc.so.0.3 => /lib/x86_64-gnu/libc.so.0.3 (0x034f9000)
/lib/ld-x86-64.so.1 (0x0800)
libmachuser.so.1 => /lib/x86_64-gnu/libmachuser.so.1
(0x03f15000)
libhurduser.so.0.3 => /lib/x86_64-gnu/libhurduser.so.0.3
(0x034b6000)
root@hurd:/# vminfo $$
 0[0x1000] (prot=0)
0x1000[0x36000] (prot=RX, max_prot=RWX, mem_obj=12)
   0x37000[0x2000] (prot=R, max_prot=RWX, mem_obj=12)
   0x39000[0x1000] (prot=RW, max_prot=RWX, mem_obj=12)
   0x3a000[0x1000] (prot=RW, max_prot=RWX, mem_obj=10)
   0x3b000[0x1000] (prot=0, max_prot=RWX, offs=0x1000)
   0x3c000[0x100] (prot=RWX, mem_obj=16)
 0x5de[0x1000] (prot=RW, max_prot=RWX, mem_obj=18)
 0x732d000[0x1000] (prot=RW, max_prot=RWX, mem_obj=18, offs=0x1000)
 0x733[0x2000] (prot=RW, max_prot=RWX, mem_obj=19)
 0x7332000[0x2000] (prot=R, max_prot=RWX, mem_obj=20)
 0x7334000[0x5000] (prot=RW, max_prot=RWX, mem_obj=22)
 0x7339000[0x1000] (prot=0, max_prot=RWX)
 0x733a000[0x9000] (prot=RW, max_prot=RWX, mem_obj=23)
 0x764a000[0x233000] (prot=RX, max_prot=RWX, mem_obj=24)
 0x787d000[0x4000] (prot=R, max_prot=RWX, mem_obj=24)
 0x7881000[0x3000] (prot=RW, max_prot=RWX, mem_obj=24)
 0x7884000[0x5000] (prot=RW, max_prot=RWX, mem_obj=25)
 0x791b000[0x2c000] (prot=RX, max_prot=RWX, mem_obj=26)
 0x7947000[0x4000] (prot=R, max_prot=RWX, mem_obj=26)
 0x794b000[0x1000] (prot=RW, max_prot=RWX, mem_obj=26, offs=0x4000)
 0x794c000[0x2e000] (prot=RX, max_prot=RWX, mem_obj=27)
 0x797a000[0x1000] (prot=R, max_prot=RWX, mem_obj=27)
 0x797b000[0x1000] (prot=RW, max_prot=RWX, mem_obj=27)
 0x797c000[0x4000] (prot=RW, max_prot=RWX, mem_obj=28)
 0x798[0x14000] (prot=RX, max_prot=RWX, mem_obj=29)
 0x7994000[0x1000] (prot=R, max_prot=RWX, mem_obj=29)
 0x7995000[0x1000] (prot=RW, max_prot=RWX, mem_obj=29)
 0x7996000[0x2000] (prot=RW, max_prot=RWX, mem_obj=30)
 0x803b000[0x10f000] (prot=RX, max_prot=RWX, mem_obj=31)
 0x814a000[0x4000] (prot=R, max_prot=RWX, mem_obj=31)
 0x814e000[0x8000] (prot=RW, max_prot=RWX, mem_obj=31)
 0x8156000[0x1000] (prot=RW, max_prot=RWX, mem_obj=31)
 0x8157000[0x9000] (prot=RW, max_prot=RWX, mem_obj=32)
0x2000[0x21000] (prot=RWX, mem_obj=33)
0x20021000[0x21000] (prot=RWX, mem_obj=34)
0x20042000[0x21000] (prot=RWX, mem_obj=35)
0x20063000[0x7f9d000] (prot=0, max_prot=RWX, offs=0x63000)



Re: 64bit startup

2023-06-20 Thread Samuel Thibault
Sergey Bugaev, le mar. 20 juin 2023 16:14:33 +0300, a ecrit:
> On Tue, Jun 20, 2023 at 3:59 PM Samuel Thibault  
> wrote:
> > € file hurd/pfinet
> > hurd/pfinet: ELF 64-bit LSB pie executable
> 
> But non-PIE is definitely the case for hurd-i386,
> 
> and only for binaries from the hurd and netdde packages, it appears.

Indeed, I don't know why, it's probably worth checking why, the gcc
default really is pie.

Samuel



Re: 64bit startup

2023-06-20 Thread Sergey Bugaev
On Tue, Jun 20, 2023 at 3:59 PM Samuel Thibault  wrote:
> Concerning dhcp, isc-dhcp currently FTBFS.

Oh :(

> € file hurd/pfinet
> hurd/pfinet: ELF 64-bit LSB pie executable

I must have been looking in the wrong place then, good.

But non-PIE is definitely the case for hurd-i386, and only for
binaries from the hurd and netdde packages, it appears.
hurd-recommended is PIE.

Sergey



Re: 64bit startup

2023-06-20 Thread Samuel Thibault
Samuel Thibault, le mar. 20 juin 2023 14:59:55 +0200, a ecrit:
> Sergey Bugaev, le mar. 20 juin 2023 13:42:03 +0300, a ecrit:
> > Could you please (when you have time, no rush) make another image
> > without the vm_param.h hack?
> 
> Ah, yes. That's terribly easy thanks to the rebootstrap script :)

(done so)



Re: 64bit startup

2023-06-20 Thread Samuel Thibault
Sergey Bugaev, le mar. 20 juin 2023 13:42:03 +0300, a ecrit:
> Could you please (when you have time, no rush) make another image
> without the vm_param.h hack?

Ah, yes. That's terribly easy thanks to the rebootstrap script :)

> Ideally, with curl and netcat and a DHCP client included. (Curl used
> to be present in the apt packages cache, but seems to have disappeared
> from the last image.)

Indeed, since we are quite limited in terms of initrd size I had dropped
a bit, but I included them again.

Concerning dhcp, isc-dhcp currently FTBFS.

> Also: any reason the Hurd executables are not compiled as PIE?

?

€ file bin/bash
bin/bash: ELF 64-bit LSB pie executable
[...]
€ file hurd/pfinet
hurd/pfinet: ELF 64-bit LSB pie executable
[...]

Samuel



Re: 64bit startup

2023-06-20 Thread Sergey Bugaev
Could you please (when you have time, no rush) make another image
without the vm_param.h hack? Ideally, with curl and netcat and a DHCP
client included. (Curl used to be present in the apt packages cache,
but seems to have disappeared from the last image.)

Context: I have some changes here that add the inaccessible lower 4 GB
mapping, tweak ELF loading in the exec server (preallocate space
first, map at that address later), and add some kind of ASLR to
gnumach. It all kind of works, but BRK_START being in the lower 4 GB
messes things up as one might expect.

Also: any reason the Hurd executables are not compiled as PIE? It
seems that the rest of Debian is. We'd really prefer PIE for x86_64 at
least, otherwise the 4 GB mapping thing doesn't work.

Sergey



Re: 64bit startup

2023-06-10 Thread Samuel Thibault
Hello,

Sergey Bugaev, le ven. 09 juin 2023 20:49:44 +0300, a ecrit:
> Well, debootstrap does output lots of errors like
> 
> dpkg: dependency problems prevent configuration of openssh-server:
>  openssh-server depends on ucf; however:
>   Package ucf is not installed.
>  openssh-server depends on runit-helper (>= 2.14.0~); however:
>   Package runit-helper is not installed.

Ah, yes, I missed adding them to the pool-all repo. That's now fixed.
The initrd image now properly passes second-stage :)

I also fixed the installability of build-essential, so in terms of
Debian packages we should now have everything needed to set up a buildd
and build the rest of Debian. We now "just" lack fixing the ABI details.

> Nano works, I can write text! The network stack kind of works; I can
> 'curl http://example.com' and get *almost* the whole output, but it
> hangs towards the end there.

Yes, it seems something gets wrong in receiving packets, perhaps some
checksum bug for instance.

Samuel



Re: 64bit startup

2023-06-10 Thread Samuel Thibault
Samuel Thibault, le sam. 10 juin 2023 01:47:00 +0200, a ecrit:
> Sergey Bugaev, le ven. 09 juin 2023 20:49:44 +0300, a ecrit:
> > with the following hacky patch, I no longer see any crashes,
> 
> That does help indeed.
> 
> When I'm using rumpdisk I'm still observing crashes, however, more
> precisely:
> 
> Kernel General protection trap, eip 0x81011eee
> kernel: General protection (13), code=2000
> Stopped   at  0x81011eee: mov %ax,%es
> 0x81011eee(bfbd4618,bfff5230,be396d50,be396d90,8103705b)
> 
> That's:
> 
> 81011ee9 t _kret_popl_fs
> 81011ee9 t _kret_popl_gs
> 81011eed t _kret_popl_es
>0x81011eed <+0>:   pop%rax
>0x81011eee <+1>:   mov%eax,%es
> 81011ef0 t _kret_popl_ds
> 81011f12 t _kret_iret

Ah, it happens without rumpdisk, too.

%rax at that point can be 0x30, 0xa72006c8, 0x983b5fff,
0x103a260, etc.

Samuel



Re: 64bit startup

2023-06-10 Thread Luca

Hi,

Il 09/06/23 19:49, Sergey Bugaev ha scritto:

with the following hacky patch, I no longer see any crashes,
debootstrap --second-stage runs all the way and leaves me with an
almost full Debian GNU/Hurd x86_64 system \o/


I was able to avoid the issues with a different approach, basically 
commenting any write to ES/DS in x86_64/locore.S. In my case the 
corruption usually happens when handling the error of writing a bad 
value to ES/DS, and disabling this probably hides the corruption issue, 
but it's still a different issue that could be solved by just removing 
the useless segment handling.


It might be that the corruption itself is caused by a second trap/irq 
happening when the exit code from a first trap is executing (e.g. pop 
the last registers before iretq). This would enter the trap/irq handler 
using the same pcb stack, and probably some fsbase/gsbase push/pop is 
not done correctly in this case.



Luca



Re: 64bit startup

2023-06-09 Thread Samuel Thibault
Sergey Bugaev, le ven. 09 juin 2023 20:49:44 +0300, a ecrit:
> with the following hacky patch, I no longer see any crashes,

That does help indeed.

When I'm using rumpdisk I'm still observing crashes, however, more
precisely:

Kernel General protection trap, eip 0x81011eee
kernel: General protection (13), code=2000
Stopped at  0x81011eee: mov %ax,%es
0x81011eee(bfbd4618,bfff5230,be396d50,be396d90,8103705b)

That's:

81011ee9 t _kret_popl_fs
81011ee9 t _kret_popl_gs
81011eed t _kret_popl_es
   0x81011eed <+0>: pop%rax
   0x81011eee <+1>: mov%eax,%es
81011ef0 t _kret_popl_ds
81011f12 t _kret_iret

Samuel



Re: 64bit startup

2023-06-09 Thread Sergey Bugaev
Hello,

with the following hacky patch, I no longer see any crashes,
debootstrap --second-stage runs all the way and leaves me with an
almost full Debian GNU/Hurd x86_64 system \o/

Well, debootstrap does output lots of errors like

dpkg: dependency problems prevent configuration of openssh-server:
 openssh-server depends on ucf; however:
  Package ucf is not installed.
 openssh-server depends on runit-helper (>= 2.14.0~); however:
  Package runit-helper is not installed.

but that's an issue with the selection of packages and not with the x86_64 Hurd.

Nano works, I can write text! The network stack kind of works; I can
'curl http://example.com' and get *almost* the whole output, but it
hangs towards the end there.

Sergey

-- >8 --

diff --git a/i386/i386/debug_i386.c b/i386/i386/debug_i386.c
index b5465796..41d032e3 100644
--- a/i386/i386/debug_i386.c
+++ b/i386/i386/debug_i386.c
@@ -40,8 +40,6 @@ void dump_ss(const struct i386_saved_state *st)
  st->r8, st->r9, st->r10, st->r11);
  printf("R12 %016lx R13 %016lx R14 %016lx R15 %016lx\n",
  st->r12, st->r13, st->r14, st->r15);
- printf("FSBASE %016lx GSBASE %016lx\n",
- st->fsbase, st->gsbase);
  printf("RIP %016lx EFLAGS %08lx\n", st->eip, st->efl);
 #else
  printf("EAX %08lx EBX %08lx ECX %08lx EDX %08lx\n",
diff --git a/i386/i386/pcb.c b/i386/i386/pcb.c
index fb535709..3691a1f3 100644
--- a/i386/i386/pcb.c
+++ b/i386/i386/pcb.c
@@ -224,8 +224,8 @@ void switch_ktss(pcb_t pcb)
 #endif /* MACH_PV_DESCRIPTORS */

 #if defined(__x86_64__) && !defined(USER32)
- wrmsr(MSR_REG_FSBASE, pcb->iss.fsbase);
- wrmsr(MSR_REG_GSBASE, pcb->iss.gsbase);
+ wrmsr(MSR_REG_FSBASE, pcb->fsbase);
+ wrmsr(MSR_REG_GSBASE, pcb->gsbase);
 #endif

  db_load_context(pcb);
@@ -680,8 +680,8 @@ kern_return_t thread_setstatus(
 return KERN_INVALID_ARGUMENT;

 state = (struct i386_fsgs_base_state *) tstate;
-thread->pcb->iss.fsbase = state->fs_base;
-thread->pcb->iss.gsbase = state->gs_base;
+thread->pcb->fsbase = state->fs_base;
+thread->pcb->gsbase = state->gs_base;
 if (thread == current_thread()) {
 wrmsr(MSR_REG_FSBASE, state->fs_base);
 wrmsr(MSR_REG_GSBASE, state->gs_base);
@@ -872,8 +872,8 @@ kern_return_t thread_getstatus(
 return KERN_INVALID_ARGUMENT;

 state = (struct i386_fsgs_base_state *) tstate;
-state->fs_base = thread->pcb->iss.fsbase;
-state->gs_base = thread->pcb->iss.gsbase;
+state->fs_base = thread->pcb->fsbase;
+state->gs_base = thread->pcb->gsbase;
 *count = i386_FSGS_BASE_STATE_COUNT;
 break;
 }
diff --git a/i386/i386/thread.h b/i386/i386/thread.h
index b5fc5ffb..4f42c145 100644
--- a/i386/i386/thread.h
+++ b/i386/i386/thread.h
@@ -51,10 +51,7 @@
  */

 struct i386_saved_state {
-#ifdef __x86_64__
- unsigned long fsbase;
- unsigned long gsbase;
-#endif
+ unsigned long fake_bases[2];
  unsigned long gs;
  unsigned long fs;
  unsigned long es;
@@ -208,6 +205,10 @@ typedef struct pcb {
  unsigned long pad; /* ensure exception stack is aligned to 16 */
 #endif
  struct i386_saved_state iss;
+#ifdef __x86_64__
+ unsigned long fsbase;
+ unsigned long gsbase;
+#endif
  struct i386_machine_state ims;
  decl_simple_lock_data(, lock)
  unsigned short init_control; /* Initial FPU control to set */

-- >8 --

I have no name!@hurd:/# fsysopts / --writable
I have no name!@hurd:/# /debootstrap/debootstrap --second-stage
I: Installing core packages...
I: Unpacking required packages...
I: Unpacking base-files...
I: Unpacking base-passwd...
I: Unpacking bash...
I: Unpacking bsdutils...
I: Unpacking coreutils...
I: Unpacking dash...
I: Unpacking debconf...
I: Unpacking debianutils...
I: Unpacking diffutils...
I: Unpacking dpkg...
I: Unpacking e2fsprogs...
I: Unpacking findutils...
I: Unpacking gnumach-common...
I: Unpacking gnumach-image-1.8-486...
I: Unpacking grep...
I: Unpacking gzip...
I: Unpacking hostname...
I: Unpacking hurd...
I: Unpacking hurd-libs0.3:hurd-amd64...
I: Unpacking init-system-helpers...
I: Unpacking less...
I: Unpacking libacl1:hurd-amd64...
I: Unpacking libblkid1:hurd-amd64...
I: Unpacking libbsd0:hurd-amd64...
I: Unpacking libbz2-1.0:hurd-amd64...
I: Unpacking libc-bin...
I: Unpacking libc0.3:hurd-amd64...
I: Unpacking libcom-err2:hurd-amd64...
I: Unpacking libcrypt1:hurd-amd64...
I: Unpacking libdaemon0:hurd-amd64...
I: Unpacking libdb5.3:hurd-amd64...
I: Unpacking libdebconfclient0:hurd-amd64...
I: Unpacking libext2fs2:hurd-amd64...
I: Unpacking libgcrypt20:hurd-amd64...
I: Unpacking libgmp10:hurd-amd64...
I: Unpacking libgpg-error0:hurd-amd64...
I: Unpacking liblwip0:hurd-amd64...
I: Unpacking liblzma5:hurd-amd64...
I: Unpacking libmd0:hurd-amd64...
I: Unpacking libmo

fsbase getting clobbered (was: Re: 64bit startup)

2023-06-09 Thread Sergey Bugaev
On Wed, Jun 7, 2023 at 12:15 AM Sergey Bugaev  wrote:
> If my theory is correct, what we actually do wrong is assuming that
> the fsbase value located on the (PCB) stack beyond the %rsp will not
> spontaneously change; but in absence of a red zone that's just not
> true — anything on your current stack beyond the %rsp can
> spontaneously change to any other value at any moment.

Ping? Does the ^above^ make sense?

Other than for fsbase/gsbase that are x86_64-specific, how is this
*supposed* to work, on both i386 and x86_64, for the other members
of the i386/i386/thread.h:struct pcb that are "beyond" our stack
pointer when running on the PCB stack? Beyond, in this case, meaning
lower addresses, so specifically 'struct i386_interrupt_state iis[2];'
is in danger of being spontaneously overwritten. And then after
(before) that, there's some other memory, possibly another struct pcb
allocated in the pcb_cache. So we could be corrupting that too.

Again, how is this *supposed* to work? Possibly there's some mechanism
in place that prevents this that I'm not aware of? If not, then how
come this never causes any issues on i386?

Should we implement some protective measures? Perhaps figure out an
upper limit on the number of bytes that can get pushed should an
interrupt happen, and add an explicit buffer of that size before the
iss?

But then is there an upper limit? -- what if yet another interrupt
happens while we're processing the previous one, then another one and
so on? Perhaps we're supposed to turn interrupts off and then back on?

Maybe we can just turn interrupts off while we're on the PCB stack?
>From what I have found online, there is a difference between trap and
interrupt "gates" (whatever those are, I have no clue), and it's that
one of them automatically disables nested interrupts on entry and
reenables them on iret, a lot like a Unix signal gets masked off while
its handler runs (unless you specify SA_NODEFER). Can we make use of
this mechanism? Are we perhaps making use of it already? If so, how is
the corruption issue still happening?

Sergey



Re: 64bit startup

2023-06-06 Thread Samuel Thibault
Sergey Bugaev, le mar. 06 juin 2023 23:22:53 +0300, a ecrit:
> For one thing you seem to have rebuilt/updated the packages, but not
> the rootfs image, so now the debuginfo I download doesn't match the
> binaries in the image. Please update the image too!

Oh, right, sorry!

Now done so.

Samuel



Re: 64bit startup

2023-06-06 Thread Sergey Bugaev
On Tue, Jun 6, 2023, 23:56 Luca  wrote:
> yes this makes perfectly sense, I think I'm chasing the same bug currently, 
> or some variation of it. With some tracing I saw that this corruption seems 
> to happen when an irq (usually timer) fires when returning from a trap, 
> although not necessarily at one specific point.

Yes, a timer is also what I am suspecting; but OTOH the address is
very reproducible for me, it always happens at this specific place,
and that suggests that it might be something other than a timer.

> I still have to find exactly the reason, in my tests fsbase gets overwritten 
> either with a kernel address or 0x17, which might be a segment value. Btw in 
> locore.S the 64-bit-only segment handling code is doing way too much, e.g. 
> es/ds and such could be ignored, I guess simplifying this part may also solve 
> this issue.

That would be a nice mini-optimization (indeed I'm wondering why
!USER32 build still tracks all the segment stuff), but I don't see how
it would help?

If my theory is correct, what we actually do wrong is assuming that
the fsbase value located on the (PCB) stack beyond the %rsp will not
spontaneously change; but in absence of a red zone that's just not
true — anything on your current stack beyond the %rsp can
spontaneously change to any other value at any moment. So one way to
solve this would be keeping fsbase/gsbase someplace else, outside of
i386_saved_state / the PCB stack, especially given that it doesn't
need to be saved and restored like the other registers. The same place
fp and other kinds of state are kept in, perhaps.

Sergey



Re: 64bit startup

2023-06-06 Thread Luca
Il 6 giugno 2023 20:22:53 UTC, Sergey Bugaev  ha scritto:
>On Mon, Jun 5, 2023 at 3:03 PM Sergey Bugaev  wrote:
>> That is going to be much easier to debug than debootstrap, thank you!
>
>Unfortunately I'm facing some troubles :|
>
>For one thing you seem to have rebuilt/updated the packages, but not
>the rootfs image, so now the debuginfo I download doesn't match the
>binaries in the image. Please update the image too!
>
>Also, gdb seems to suddenly have some sort of trouble with
>interpreting the debuginfo files, specifically I'm trying ones from
>the 'hurd-dbsym' package (to be even more specific:
>/usr/lib/debug/.build-id/bf/dd0c0525d0ca383bd842796063345a2dd0c001.debug
>from that package, which corresponds to ext2fs.static -- but I've done
>a quick check and other files seem to behave the same way too). GDB
>loads regular symbols from them, but not the debuginfo, i.e. I can see
>what function is at which address, but not map addresses to source
>lines or access local variables or use types (tcbhead_t is the one I
>currently need most). I don't know enough about GDB and DWARF to
>diagnose exactly what's going on; readelf --debug-dump=info seems to
>dump the debuginfo just fine.
>
>Please try to reproduce this with your GDB (no Hurd system required),
>and if you have changed something recently about how debug files are
>generated, maybe that's what has broken it.
>
>So that all being said, here's one crash I am (and have been) seeing a
>lot: the crash at any sort of TCB access when fs_base suddenly turns
>out to be equal to the address of _kret_popl_ds. This makes no sense
>-- surely userspace would never set that, so it must be a gnumach bug.
>
>I've got a little theory of how something like that could happen:
>
>It is my understanding that "the PCB stack" (whatever that is) where
>locore.S pushes user's registers and thread->pcb->iss is really the
>exact same place, pushing registers onto that stack is exactly writing
>to the thread's i386_saved_state structure. The first four members of
>struct i386_saved_state are unsigned long fsbase, gsbase, gs, fs --
>and being the first members of the struct means they have the lowest
>addresses, i.e. are located at the top of the PCB stack.
>
>locore.S actually skips pushing or popping these four members:
>
>#define PUSH_FSGS   \
>subq$32,%rsp
>
>#define POP_FSGS\
>addq$32,%rsp
>
>This is because fs and gs we don't care about, and fsbase/gsbase of a
>thread state can only be changed by explicit thread_set_state calls
>and not by the thread itself, so, no need to rdmsr and push it, since
>the value is already saved in the PCB slot.
>
>However, *something* goes wrong and the fsbase slot gets overwritten
>with an unrelated value (_kret_popl_ds). The real %fs_base MSR keeps
>the proper value -- until we context-switch away from the thread and
>then back to it, at which point the bogus value gets loaded into
>%fs_base and then the userland tries to use it and faults.
>
>I don't know nearly enough about x86 interrupts/traps to say, but
>could it be that we get another interrupt/trap, while presumably
>executing _kret_popl_ds, and that causes the faulting %rip to be
>pushed onto the stack, but since we're at the PCB stack at that point
>it clobbers the stored fsbase? That doesn't cause issues for all the
>other registers because we have already popped their values off and
>won't be accessing them anymore; we'll push the new values the next
>time the thread enters the kernel -- though I guess it could show up
>in thread_get_state if you do that without stopping the thread on an
>SMP kernel.
>
>cc'ing Luca -- does what I'm saying make sense? could this happen? can
>you reproduce %fs_base getting set to _kret_popl_ds?
 
yes this makes perfectly sense, I think I'm chasing the same bug currently, or 
some variation of it. With some tracing I saw that this corruption seems to 
happen when an irq (usually timer) fires when returning from a trap, although 
not necessarily at one specific point.

I still have to find exactly the reason, in my tests fsbase gets overwritten 
either with a kernel address or 0x17, which might be a segment value. Btw in 
locore.S the 64-bit-only segment handling code is doing way too much, e.g. 
es/ds and such could be ignored, I guess simplifying this part may also solve 
this issue.


Luca
Hi Sergey,



Re: 64bit startup

2023-06-06 Thread Sergey Bugaev
On Mon, Jun 5, 2023 at 3:03 PM Sergey Bugaev  wrote:
> That is going to be much easier to debug than debootstrap, thank you!

Unfortunately I'm facing some troubles :|

For one thing you seem to have rebuilt/updated the packages, but not
the rootfs image, so now the debuginfo I download doesn't match the
binaries in the image. Please update the image too!

Also, gdb seems to suddenly have some sort of trouble with
interpreting the debuginfo files, specifically I'm trying ones from
the 'hurd-dbsym' package (to be even more specific:
/usr/lib/debug/.build-id/bf/dd0c0525d0ca383bd842796063345a2dd0c001.debug
from that package, which corresponds to ext2fs.static -- but I've done
a quick check and other files seem to behave the same way too). GDB
loads regular symbols from them, but not the debuginfo, i.e. I can see
what function is at which address, but not map addresses to source
lines or access local variables or use types (tcbhead_t is the one I
currently need most). I don't know enough about GDB and DWARF to
diagnose exactly what's going on; readelf --debug-dump=info seems to
dump the debuginfo just fine.

Please try to reproduce this with your GDB (no Hurd system required),
and if you have changed something recently about how debug files are
generated, maybe that's what has broken it.

So that all being said, here's one crash I am (and have been) seeing a
lot: the crash at any sort of TCB access when fs_base suddenly turns
out to be equal to the address of _kret_popl_ds. This makes no sense
-- surely userspace would never set that, so it must be a gnumach bug.

I've got a little theory of how something like that could happen:

It is my understanding that "the PCB stack" (whatever that is) where
locore.S pushes user's registers and thread->pcb->iss is really the
exact same place, pushing registers onto that stack is exactly writing
to the thread's i386_saved_state structure. The first four members of
struct i386_saved_state are unsigned long fsbase, gsbase, gs, fs --
and being the first members of the struct means they have the lowest
addresses, i.e. are located at the top of the PCB stack.

locore.S actually skips pushing or popping these four members:

#define PUSH_FSGS   \
subq$32,%rsp

#define POP_FSGS\
addq$32,%rsp

This is because fs and gs we don't care about, and fsbase/gsbase of a
thread state can only be changed by explicit thread_set_state calls
and not by the thread itself, so, no need to rdmsr and push it, since
the value is already saved in the PCB slot.

However, *something* goes wrong and the fsbase slot gets overwritten
with an unrelated value (_kret_popl_ds). The real %fs_base MSR keeps
the proper value -- until we context-switch away from the thread and
then back to it, at which point the bogus value gets loaded into
%fs_base and then the userland tries to use it and faults.

I don't know nearly enough about x86 interrupts/traps to say, but
could it be that we get another interrupt/trap, while presumably
executing _kret_popl_ds, and that causes the faulting %rip to be
pushed onto the stack, but since we're at the PCB stack at that point
it clobbers the stored fsbase? That doesn't cause issues for all the
other registers because we have already popped their values off and
won't be accessing them anymore; we'll push the new values the next
time the thread enters the kernel -- though I guess it could show up
in thread_get_state if you do that without stopping the thread on an
SMP kernel.

cc'ing Luca -- does what I'm saying make sense? could this happen? can
you reproduce %fs_base getting set to _kret_popl_ds?

Sergey



Re: 64bit startup

2023-06-05 Thread Sergey Bugaev
On Mon, Jun 5, 2023 at 2:05 PM Samuel Thibault  wrote:
> I can easily trigger crashes with
>
> fsysopts / --writable
> while true ; do ps-hurd  -feMj > /tmp/blip ; done

That is going to be much easier to debug than debootstrap, thank you!

Sergey



Re: 64bit startup

2023-06-05 Thread Samuel Thibault
Samuel Thibault, le dim. 04 juin 2023 19:46:25 +0200, a ecrit:
> Sergey Bugaev, le dim. 04 juin 2023 20:40:54 +0300, a ecrit:
> > On Sun, Jun 4, 2023 at 8:21 PM Samuel Thibault  
> > wrote:
> > > It seems so indeed. But it's not only the stack guard, but various TLS
> > > accesses.
> > 
> > What would be an easy way for me to reproduce this?
> 
> Basically all I'm trying to do is the second-stage:
> 
> fsysopts / --writable
> tail -f /debootstrap/debootstrap.log
> /debootstrap/debootstrap --second-stage

I can easily trigger crashes with

fsysopts / --writable
while true ; do ps-hurd  -feMj > /tmp/blip ; done

Samuel



Re: 64bit startup

2023-06-04 Thread Samuel Thibault
Sergey Bugaev, le dim. 04 juin 2023 20:40:54 +0300, a ecrit:
> On Sun, Jun 4, 2023 at 8:21 PM Samuel Thibault  
> wrote:
> > It seems so indeed. But it's not only the stack guard, but various TLS
> > accesses.
> 
> What would be an easy way for me to reproduce this?

Basically all I'm trying to do is the second-stage:

fsysopts / --writable
tail -f /debootstrap/debootstrap.log
/debootstrap/debootstrap --second-stage

Samuel



Re: 64bit startup

2023-06-04 Thread Sergey Bugaev
On Sun, Jun 4, 2023 at 8:21 PM Samuel Thibault  wrote:
> It seems so indeed. But it's not only the stack guard, but various TLS
> accesses.

What would be an easy way for me to reproduce this?

Sergey



Re: 64bit startup

2023-06-04 Thread Samuel Thibault
Hello,

Sergey Bugaev, le sam. 03 juin 2023 20:20:24 +0300, a ecrit:
> I'm attaching a patch that does that and also documents the pitfalls
> (and also removes an extra block scope / level of indentation). It's
> untested, but it builds and the generated code looks OK (sc_reply_port
> gets loaded straight into %rdx, to be passed into __sigreturn2). Please
> test!

It seems to be working fine indeed, I have applied it, thanks!

> The stack guard issue must be caused by something else.

It seems so indeed. But it's not only the stack guard, but various TLS
accesses.

Samuel



  1   2   >