Re: [PATCH] hurd: Fix using interposable hurd_thread_self

2023-05-19 Thread Samuel Thibault
Checked and applied, thanks!

Sergey Bugaev, le ven. 19 mai 2023 17:47:24 +0300, a ecrit:
> Create a private hidden __hurd_thread_self alias, and use that one.
> 
> Fixes 2f8ecb58a59eb82c43214d000842d99644a662d1
> "hurd: Fix x86_64 _hurd_tls_fork" and
> c7fcce38c83a2bb665ef5dc4981bf20c7e586123
> "hurd: Make sure to not use tcb->self"
> 
> Reported-by: Joseph Myers 
> Signed-off-by: Sergey Bugaev 
> ---
> 
> This should hopefully fix it (but please test!); I have only checked
> that it builds on x86_64-gnu, and that _Fork now calls
> __GI___hurd_thread_self without the PLT, and that libpthread.so only
> references __hurd_thread_self@GLIBC_PRIVATE.
> 
>  hurd/Versions   | 1 +
>  hurd/thread-self.c  | 5 -
>  sysdeps/hurd/include/hurd.h | 3 +++
>  sysdeps/mach/hurd/i386/htl/pt-setup.c   | 2 +-
>  sysdeps/mach/hurd/x86_64/htl/pt-setup.c | 2 +-
>  sysdeps/mach/hurd/x86_64/tls.h  | 7 +--
>  6 files changed, 15 insertions(+), 5 deletions(-)
> 
> diff --git a/hurd/Versions b/hurd/Versions
> index 3d8b412d..439e8abf 100644
> --- a/hurd/Versions
> +++ b/hurd/Versions
> @@ -156,6 +156,7 @@ libc {
>  __libc_open; __libc_close;
>  
>  # Used by libpthread.
> +__hurd_thread_self;
>  _hurd_sigstate_set_global_rcv;
>  _hurd_sigstate_lock;
>  _hurd_sigstate_pending;
> diff --git a/hurd/thread-self.c b/hurd/thread-self.c
> index f3718165..af013503 100644
> --- a/hurd/thread-self.c
> +++ b/hurd/thread-self.c
> @@ -19,7 +19,10 @@
>  #include 
>  
>  thread_t
> -hurd_thread_self (void)
> +__hurd_thread_self (void)
>  {
>return _hurd_self_sigstate ()->thread;
>  }
> +
> +libc_hidden_def (__hurd_thread_self)
> +weak_alias (__hurd_thread_self, hurd_thread_self)
> diff --git a/sysdeps/hurd/include/hurd.h b/sysdeps/hurd/include/hurd.h
> index 7da9af26..5d904e0e 100644
> --- a/sysdeps/hurd/include/hurd.h
> +++ b/sysdeps/hurd/include/hurd.h
> @@ -11,5 +11,8 @@ void _hurd_libc_proc_init (char **argv);
>  libc_hidden_proto (_hurd_exec_paths)
>  libc_hidden_proto (_hurd_init)
>  libc_hidden_proto (_hurd_libc_proc_init)
> +
> +extern thread_t __hurd_thread_self (void);
> +libc_hidden_proto (__hurd_thread_self)
>  #endif
>  #endif
> diff --git a/sysdeps/mach/hurd/i386/htl/pt-setup.c 
> b/sysdeps/mach/hurd/i386/htl/pt-setup.c
> index ba108b96..27e5c98b 100644
> --- a/sysdeps/mach/hurd/i386/htl/pt-setup.c
> +++ b/sysdeps/mach/hurd/i386/htl/pt-setup.c
> @@ -79,7 +79,7 @@ __pthread_setup (struct __pthread *thread,
>  {
>error_t err;
>  
> -  if (thread->kernel_thread == hurd_thread_self ())
> +  if (thread->kernel_thread == __hurd_thread_self ())
>  /* Fix up the TCB for the main thread.  The C library has already
> installed a TCB, which we want to keep using.  This TCB must not
> be freed so don't register it in the thread structure.  On the
> diff --git a/sysdeps/mach/hurd/x86_64/htl/pt-setup.c 
> b/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
> index 7dc62912..8eebaf97 100644
> --- a/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
> +++ b/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
> @@ -56,7 +56,7 @@ __pthread_setup (struct __pthread *thread,
>struct i386_thread_state state;
>struct i386_fsgs_base_state fsgs_state;
>  
> -  if (thread->kernel_thread == hurd_thread_self ())
> +  if (thread->kernel_thread == __hurd_thread_self ())
>  /* Fix up the TCB for the main thread.  The C library has already
> installed a TCB, which we want to keep using.  This TCB must not
> be freed so don't register it in the thread structure.  On the
> diff --git a/sysdeps/mach/hurd/x86_64/tls.h b/sysdeps/mach/hurd/x86_64/tls.h
> index 468f703f..a5a49a47 100644
> --- a/sysdeps/mach/hurd/x86_64/tls.h
> +++ b/sysdeps/mach/hurd/x86_64/tls.h
> @@ -132,6 +132,10 @@ THREAD_TCB (thread_t thread,
>((descr)->pointer_guard\
> = THREAD_GETMEM (THREAD_SELF, pointer_guard))
>  
> +/* From hurd.h, reproduced here to avoid a circular include.  */
> +extern thread_t __hurd_thread_self (void);
> +libc_hidden_proto (__hurd_thread_self)
> +
>  /* Set up TLS in the new thread of a fork child, copying from the original.  
> */
>  static inline kern_return_t __attribute__ ((unused))
>  _hurd_tls_fork (thread_t child, thread_t orig,
> @@ -141,8 +145,7 @@ _hurd_tls_fork (thread_t child, thread_t orig,
>struct i386_fsgs_base_state state;
>mach_msg_type_number_t state_count = i386_FSGS_BASE_STATE_COUNT;
>  
> -  extern thread_t hurd_thread_self (void);
> -  if (orig != hurd_thread_self ())
> +  if (orig != __hurd_thread_self ())
>  {
>err = __thread_get_state (orig, i386_FSGS_BASE_STATE,
>  (thread_state_t) ,
> -- 
> 2.40.1
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.



Re: [PATCH] hurd: Fix __TIMESIZE on x86_64

2023-05-19 Thread Samuel Thibault
Applied, thanks!

Sergey Bugaev, le ven. 19 mai 2023 20:15:16 +0300, a ecrit:
> We had sizeof (time_t) == 8, but __TIMESIZE == 32.
> 
> Signed-off-by: Sergey Bugaev 
> ---
> 
> ...oops.
> 
> I have not tested whether this breaks anything (other then checking
> that it still builds), but it's kind of amazing that it had worked at
> all before this,

There aren't that many users of time_t actually.

> and there's a higher chance of this fixing something
> than breaking something.
> 
> Unfortunately I believe this means that all binaries built for
> x86_64-gnu so far will have to be rebuilt.

It's not a problem on my side, I'm still always reboostraping everything
everytime. Until we have made a careful review of the types we expose,
I'm not considering the ABI as stable.

Samuel

> Fortunately I don't think
> anybody except for myself (and Samuel himself) had installed them.
> 
> Discovered while attempting to hack on fcntl{,64,_time64}.
> 
>  sysdeps/mach/clock_gettime.c| 2 ++
>  sysdeps/mach/hurd/bits/timesize.h   | 4 +++-
>  sysdeps/mach/hurd/x86_64/libanl.abilist | 1 -
>  3 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/sysdeps/mach/clock_gettime.c b/sysdeps/mach/clock_gettime.c
> index be775ed2..9124a32e 100644
> --- a/sysdeps/mach/clock_gettime.c
> +++ b/sysdeps/mach/clock_gettime.c
> @@ -111,6 +111,7 @@ strong_alias (__clock_gettime, __clock_gettime_2);
>  compat_symbol (libc, __clock_gettime_2, clock_gettime, GLIBC_2_2);
>  #endif
>  
> +#if __TIMESIZE != 64
>  int
>  __clock_gettime64 (clockid_t clock_id, struct __timespec64 *ts64)
>  {
> @@ -124,3 +125,4 @@ __clock_gettime64 (clockid_t clock_id, struct 
> __timespec64 *ts64)
>return ret;
>  }
>  libc_hidden_def (__clock_gettime64)
> +#endif
> diff --git a/sysdeps/mach/hurd/bits/timesize.h 
> b/sysdeps/mach/hurd/bits/timesize.h
> index 900a0a23..981452c0 100644
> --- a/sysdeps/mach/hurd/bits/timesize.h
> +++ b/sysdeps/mach/hurd/bits/timesize.h
> @@ -16,5 +16,7 @@
> License along with the GNU C Library; if not, see
> .  */
>  
> +#include 
> +
>  /* Size in bits of the 'time_t' type of the default ABI.  */
> -#define __TIMESIZE   32
> +#define __TIMESIZE   __WORDSIZE
> diff --git a/sysdeps/mach/hurd/x86_64/libanl.abilist 
> b/sysdeps/mach/hurd/x86_64/libanl.abilist
> index c0a6a009..baad6de2 100644
> --- a/sysdeps/mach/hurd/x86_64/libanl.abilist
> +++ b/sysdeps/mach/hurd/x86_64/libanl.abilist
> @@ -1,4 +1,3 @@
>  GLIBC_2.38 gai_cancel F
>  GLIBC_2.38 gai_error F
> -GLIBC_2.38 gai_suspend F
>  GLIBC_2.38 getaddrinfo_a F
> -- 
> 2.40.1
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.



[PATCH] hurd: Fix __TIMESIZE on x86_64

2023-05-19 Thread Sergey Bugaev
We had sizeof (time_t) == 8, but __TIMESIZE == 32.

Signed-off-by: Sergey Bugaev 
---

...oops.

I have not tested whether this breaks anything (other then checking
that it still builds), but it's kind of amazing that it had worked at
all before this, and there's a higher chance of this fixing something
than breaking something.

Unfortunately I believe this means that all binaries built for
x86_64-gnu so far will have to be rebuilt. Fortunately I don't think
anybody except for myself (and Samuel himself) had installed them.

Discovered while attempting to hack on fcntl{,64,_time64}.

 sysdeps/mach/clock_gettime.c| 2 ++
 sysdeps/mach/hurd/bits/timesize.h   | 4 +++-
 sysdeps/mach/hurd/x86_64/libanl.abilist | 1 -
 3 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/sysdeps/mach/clock_gettime.c b/sysdeps/mach/clock_gettime.c
index be775ed2..9124a32e 100644
--- a/sysdeps/mach/clock_gettime.c
+++ b/sysdeps/mach/clock_gettime.c
@@ -111,6 +111,7 @@ strong_alias (__clock_gettime, __clock_gettime_2);
 compat_symbol (libc, __clock_gettime_2, clock_gettime, GLIBC_2_2);
 #endif
 
+#if __TIMESIZE != 64
 int
 __clock_gettime64 (clockid_t clock_id, struct __timespec64 *ts64)
 {
@@ -124,3 +125,4 @@ __clock_gettime64 (clockid_t clock_id, struct __timespec64 
*ts64)
   return ret;
 }
 libc_hidden_def (__clock_gettime64)
+#endif
diff --git a/sysdeps/mach/hurd/bits/timesize.h 
b/sysdeps/mach/hurd/bits/timesize.h
index 900a0a23..981452c0 100644
--- a/sysdeps/mach/hurd/bits/timesize.h
+++ b/sysdeps/mach/hurd/bits/timesize.h
@@ -16,5 +16,7 @@
License along with the GNU C Library; if not, see
.  */
 
+#include 
+
 /* Size in bits of the 'time_t' type of the default ABI.  */
-#define __TIMESIZE 32
+#define __TIMESIZE __WORDSIZE
diff --git a/sysdeps/mach/hurd/x86_64/libanl.abilist 
b/sysdeps/mach/hurd/x86_64/libanl.abilist
index c0a6a009..baad6de2 100644
--- a/sysdeps/mach/hurd/x86_64/libanl.abilist
+++ b/sysdeps/mach/hurd/x86_64/libanl.abilist
@@ -1,4 +1,3 @@
 GLIBC_2.38 gai_cancel F
 GLIBC_2.38 gai_error F
-GLIBC_2.38 gai_suspend F
 GLIBC_2.38 getaddrinfo_a F
-- 
2.40.1




Re: [RFC PATCH 06/10] hurd: Make sure to not use tcb->self

2023-05-19 Thread Joseph Myers
On Fri, 19 May 2023, Sergey Bugaev via Libc-alpha wrote:

> 'foo' is a public symbol, to be used by the user, and also
> interposable by the user. Always called via PLT (except from inside
> the user's code when redefined inside the executable, in which case
> the compiler/linker will see that no PLT is needed), and should not be
> called from inside glibc.

Should not be called from within glibc via the PLT within the same 
library.

It's OK for foo to be called from bar if foo is part of all the standards 
that contain bar.  But in that case, if the call is from the same library, 
*_hidden_def / *_hidden_proto should be used to avoid calling via the PLT.

If foo is not part of all the standards that contain bar, then glibc code 
for bar should use __foo instead to avoid namespace issues, especially for 
static linking.

If __foo is needed by executables, *_nonshared.a or other shared 
libraries, then it also needs to be exported from the library it's in (in 
which case PLT avoidance is also relevant for __foo, so *_hidden_* should 
be used for __foo).

If __foo is only used within a single library and doesn't need to be 
exported, it's OK to declare it directly with attribute_hidden rather than 
using *_hidden_* to create __GI___foo as an alias.  (In this case, if you 
forget to use attribute_hidden, you won't get test failures - the symbol 
in fact won't get exported, because only symbols explicitly listed in the 
version maps get exported, and so it won't get a PLT entry.  But in some 
cases, code generation is more efficient if the compiler knows that the 
symbol is hidden.  So when you're calling an unexported symbol in the same 
library, it's still desirable for it to be declared as hidden in a way 
visible to the compiler.)  The more complicated hidden_proto / hidden_def 
approach also works for unexported symbols used within a single library, 
it's just more complicated than necessary in that case.

-- 
Joseph S. Myers
jos...@codesourcery.com



Booted Debian with noide => rumpdisk! [WAS Re: Guix hurd with rumpdisk boots! [WAS Re: Some progress, Guix rumpdisk still crashes...]]

2023-05-19 Thread Janneke Nieuwenhuizen
Janneke Nieuwenhuizen writes:

> Sergey Bugaev writes:
>
>> On Fri, May 19, 2023 at 1:20 PM Janneke Nieuwenhuizen  
>> wrote:
[..]
>> Using rumpdisk on Debian should be a matter of changing
>> part:1:device:hd0 to part:1:device:wd0 (why part:2?), and adding
>> noide. (Unless I'm misremembering, of course, and note I'm not at all
>> qualified to talk about any of this).
>
> Hmm, the image above already has wd0 (part:2:device:wd0)...but it does
> not use rump, now it really gets confusing...

Okay, so rumpdisk "just works"(TM) on latest Debian.

After discussing on IRC

https://logs.guix.gnu.org/hurd/2023-05-18.log#091300

using


https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd-20220824.img

I upgraded to lastest greatest:

--8<---cut here---start->8---
wget 
http://ftp.de.debian.org/debian/pool/main/d/debian-ports-archive-keyring/debian-ports-archive-keyring_2023.02.01_all.deb
dpkg -i debian-ports-archive-keyring_2023.02.01_all.deb
apt-get update
apt-get dist-upgrade
--8<---cut here---end--->8---

changed /boot/grub/grub.cfg

--8<---cut here---start->8---
#   multiboot   /boot/gnumach-1.8-486.gz root=part:2:device:hd0 
console=com0
multiboot   /boot/gnumach-1.8-486.gz root=part:2:device:wd0 
console=com0 noide
--8<---cut here---end--->8---

and /etc/fstab

--8<---cut here---start->8---
#/dev/hd0s2  /   ext2defaults0   1
/dev/wd0s2  /   ext2defaults0   1
#/dev/hd0s1  noneswapsw  0   0
/dev/wd0s1  noneswapsw  0   0
#/dev/hd2/media/cdrom0   iso9660 noauto  0   0
/dev/wd2/media/cdrom0   iso9660 noauto  0   0
--8<---cut here---end--->8---

I got:

Checking root file system.../dev/wd0s2: clean, 42057/259072 files, 
416312/1035776 blocks

full log attached.

Now to replicate this on Guix...I already notice that this setup uses
acpi, which is broken an has been disabled in the latest rumpkernel
build...

Thanks for all the help!

Greetings,
Janneke



upgraded-noide.log
Description: Binary data

-- 
Janneke Nieuwenhuizen   | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com


[PATCH] hurd: Fix using interposable hurd_thread_self

2023-05-19 Thread Sergey Bugaev
Create a private hidden __hurd_thread_self alias, and use that one.

Fixes 2f8ecb58a59eb82c43214d000842d99644a662d1
"hurd: Fix x86_64 _hurd_tls_fork" and
c7fcce38c83a2bb665ef5dc4981bf20c7e586123
"hurd: Make sure to not use tcb->self"

Reported-by: Joseph Myers 
Signed-off-by: Sergey Bugaev 
---

This should hopefully fix it (but please test!); I have only checked
that it builds on x86_64-gnu, and that _Fork now calls
__GI___hurd_thread_self without the PLT, and that libpthread.so only
references __hurd_thread_self@GLIBC_PRIVATE.

 hurd/Versions   | 1 +
 hurd/thread-self.c  | 5 -
 sysdeps/hurd/include/hurd.h | 3 +++
 sysdeps/mach/hurd/i386/htl/pt-setup.c   | 2 +-
 sysdeps/mach/hurd/x86_64/htl/pt-setup.c | 2 +-
 sysdeps/mach/hurd/x86_64/tls.h  | 7 +--
 6 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/hurd/Versions b/hurd/Versions
index 3d8b412d..439e8abf 100644
--- a/hurd/Versions
+++ b/hurd/Versions
@@ -156,6 +156,7 @@ libc {
 __libc_open; __libc_close;
 
 # Used by libpthread.
+__hurd_thread_self;
 _hurd_sigstate_set_global_rcv;
 _hurd_sigstate_lock;
 _hurd_sigstate_pending;
diff --git a/hurd/thread-self.c b/hurd/thread-self.c
index f3718165..af013503 100644
--- a/hurd/thread-self.c
+++ b/hurd/thread-self.c
@@ -19,7 +19,10 @@
 #include 
 
 thread_t
-hurd_thread_self (void)
+__hurd_thread_self (void)
 {
   return _hurd_self_sigstate ()->thread;
 }
+
+libc_hidden_def (__hurd_thread_self)
+weak_alias (__hurd_thread_self, hurd_thread_self)
diff --git a/sysdeps/hurd/include/hurd.h b/sysdeps/hurd/include/hurd.h
index 7da9af26..5d904e0e 100644
--- a/sysdeps/hurd/include/hurd.h
+++ b/sysdeps/hurd/include/hurd.h
@@ -11,5 +11,8 @@ void _hurd_libc_proc_init (char **argv);
 libc_hidden_proto (_hurd_exec_paths)
 libc_hidden_proto (_hurd_init)
 libc_hidden_proto (_hurd_libc_proc_init)
+
+extern thread_t __hurd_thread_self (void);
+libc_hidden_proto (__hurd_thread_self)
 #endif
 #endif
diff --git a/sysdeps/mach/hurd/i386/htl/pt-setup.c 
b/sysdeps/mach/hurd/i386/htl/pt-setup.c
index ba108b96..27e5c98b 100644
--- a/sysdeps/mach/hurd/i386/htl/pt-setup.c
+++ b/sysdeps/mach/hurd/i386/htl/pt-setup.c
@@ -79,7 +79,7 @@ __pthread_setup (struct __pthread *thread,
 {
   error_t err;
 
-  if (thread->kernel_thread == hurd_thread_self ())
+  if (thread->kernel_thread == __hurd_thread_self ())
 /* Fix up the TCB for the main thread.  The C library has already
installed a TCB, which we want to keep using.  This TCB must not
be freed so don't register it in the thread structure.  On the
diff --git a/sysdeps/mach/hurd/x86_64/htl/pt-setup.c 
b/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
index 7dc62912..8eebaf97 100644
--- a/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
+++ b/sysdeps/mach/hurd/x86_64/htl/pt-setup.c
@@ -56,7 +56,7 @@ __pthread_setup (struct __pthread *thread,
   struct i386_thread_state state;
   struct i386_fsgs_base_state fsgs_state;
 
-  if (thread->kernel_thread == hurd_thread_self ())
+  if (thread->kernel_thread == __hurd_thread_self ())
 /* Fix up the TCB for the main thread.  The C library has already
installed a TCB, which we want to keep using.  This TCB must not
be freed so don't register it in the thread structure.  On the
diff --git a/sysdeps/mach/hurd/x86_64/tls.h b/sysdeps/mach/hurd/x86_64/tls.h
index 468f703f..a5a49a47 100644
--- a/sysdeps/mach/hurd/x86_64/tls.h
+++ b/sysdeps/mach/hurd/x86_64/tls.h
@@ -132,6 +132,10 @@ THREAD_TCB (thread_t thread,
   ((descr)->pointer_guard  \
= THREAD_GETMEM (THREAD_SELF, pointer_guard))
 
+/* From hurd.h, reproduced here to avoid a circular include.  */
+extern thread_t __hurd_thread_self (void);
+libc_hidden_proto (__hurd_thread_self)
+
 /* Set up TLS in the new thread of a fork child, copying from the original.  */
 static inline kern_return_t __attribute__ ((unused))
 _hurd_tls_fork (thread_t child, thread_t orig,
@@ -141,8 +145,7 @@ _hurd_tls_fork (thread_t child, thread_t orig,
   struct i386_fsgs_base_state state;
   mach_msg_type_number_t state_count = i386_FSGS_BASE_STATE_COUNT;
 
-  extern thread_t hurd_thread_self (void);
-  if (orig != hurd_thread_self ())
+  if (orig != __hurd_thread_self ())
 {
   err = __thread_get_state (orig, i386_FSGS_BASE_STATE,
 (thread_state_t) ,
-- 
2.40.1




Re: [PATCH] faq/64-bit.mdwn: added up to date 64-bit porting info open_issues/64-bit_port.mdwn: added up to date 64-bit porting info

2023-05-19 Thread Joshua Branson
Svante Signell  writes:

> On Sat, 2023-05-13 at 19:01 +, jbra...@dismail.de wrote:
>> May 13, 2023 1:46 PM, "Sergey Bugaev"  wrote:
>> 
>> > On Sat, May 13, 2023 at 7:38 PM jbra...@dismail.de 
>> > wrote:
>> > 
>> > > +Hurd developers are adding 64-bit support, and they plan to drop the
>> > > +32-bit support, once the 64-bit support is deemed stable.
>> > 
>> > Is this really the plan? :(
>> > 
>> > Has this been discussed anywhere?
>> 
>> It is entirely possible that I made up the plan to drop 32-bit support.  
>> I thought I had seem someone say that was the plan...
>
> Why dropping 32-bit support? It does not make sense.

We are not dropping 64-bit support.  I assumed we were. I was
incorrect.  :)   Check the hurd wiki faq "64 bit version".  Samuel put
some up to date info there.

>
> Thanks! 
>

-- 

Joshua Branson
Sent from the Hurd



Re: [RFC PATCH 0/2] On ldconfig and ld.so.cache

2023-05-19 Thread Sergey Bugaev
Hello,

On Fri, May 19, 2023 at 3:30 PM Florian Weimer  wrote:
> Debian hasn't upstreamed there multi-arch path layouts.  We could
> implement multi-arch ldconfig in /etc/ld.so.cache, but with the paths
> that Debian currently uses, it's not easy because there's no automated
> way ldconfig can recognize the relevant subdirectories.  There's no
> intermediate directly like “…/glibc-hwcaps/…” that could serve as a
> marker.  I suppose we could look for subdirectories containing libc.so.6
> files and its variants as an approximation …

Since you're taking my little rant seriously: I'm not arguing that
multi-arch path layouts should be upstreamed or that ld.so should
support that natively likely it does hwcaps. What I'm saying is:

* There is a clear need to configure custom (downstream- or installation-
  specific) paths where shared libraries are to be loaded from, other
  than just /lib and /lib64;

* /etc/ld.so.conf is the proper, well-supported, and popular way to do
  just that, and it is being used by many (if not all) major distros
  for this exact purpose -- in Debian's case, for enabling multiarch
  layout, among other things;

* with use_ldconfig=no, which is the upstream default for non-Linux,
  there does not seem to be a way to achieve that (without resorting to
  LD_LIBRARY_PATH or something like that).

So if use_ldconfig=no is to be kept, there should be another
recommended and supported way to configure search directories. Making
ld.so read ld.so.conf (parsing/globbing questions notwithstanding)
might be one such way.

Or maybe use_ldconfig should just get enabled everywhere :)

On Fri, May 19, 2023 at 2:52 PM Carlos O'Donell  wrote:
> Removing configuration options and making it simple to configure and use 
> glibc is great
> goal. I think that ldconfig should always be enabled and I don't see a 
> downside to making
> `use_ldconfig=yes` the default, but I think we'd have to check with Guix or 
> Nix to see if
> they have any custom changes there. It involves probably a slightly broader 
> distro
> discussion.

cc'ing Ludovic; but again, why would it be convenient for Nix/Guix to
have use_ldconfig enabled when building their distro with Linux but
not with the Hurd? Surely they'd rather prefer it to be consistent?

I don't think the reason for use_ldconfig=no default is about Guix...
but rather about the ideas that the original Hurd developers have had
for the design of the GNU system (see also: /usr vs /). That hasn't
really panned out; but I understand if the upstream defaults are still
kept according to those plans. Still, using use_ldconfig=yes with Hurd
needs to be better supported, since that's what Hurd distros actually
do in practice (I think -- well, at least Debian does), and if
use_ldconfig=no is to be kept supported, there has to be a way to
configure library search directories without ldconfig.

Sergey



Re: [RFC PATCH 0/2] On ldconfig and ld.so.cache

2023-05-19 Thread Florian Weimer
* Sergey Bugaev via Libc-alpha:

> Moreover, Debian GNU/Hurd, the primary Hurd-based distribution, has been
> shipping ld.so.cache on Hurd as a downstream patch [1] (note that more
> changes would be required for x86_64-gnu because of FLAG_X8664_LIB64).
> They don't really have a choice, it seems: with their "multiarch"
> filesystem layout, the shared libraries are to be located in
> /lib/i386-gnu/, but that is not a path that ld.so searches for libraries
> in! This is solved by use_ldconfig=yes, and putting /lib/i386-gnu/ into
> /etc/ld.so.conf.d/i386-gnu.conf, where it is then picked up by ldconfig,
> which finds all the libraries and bakes their paths into the cache, where
> ld.so then picks them up.
>
> [1]: 
> https://salsa.debian.org/glibc-team/glibc/-/raw/sid/debian/patches/hurd-i386/local-enable-ldconfig.diff
>
> This is also another thing that suprises me: how come ldconfig does read
> ld.so.conf, but ld.so does not? It's not an "ldconfig.conf" after all...
> How is one even supposed to configure library paths with use_ldconfig=no?

The cache isn't really a cache, it's required for correct operation.

Debian hasn't upstreamed there multi-arch path layouts.  We could
implement multi-arch ldconfig in /etc/ld.so.cache, but with the paths
that Debian currently uses, it's not easy because there's no automated
way ldconfig can recognize the relevant subdirectories.  There's no
intermediate directly like “…/glibc-hwcaps/…” that could serve as a
marker.  I suppose we could look for subdirectories containing libc.so.6
files and its variants as an approximation …

Multi-arch /etc/ld.so.cache needs some new on-disk data structures.

Thanks,
Florian




Re: [RFC PATCH 1/2] elf: Port ldconfig away from stack-allocated paths

2023-05-19 Thread Florian Weimer
* Sergey Bugaev via Libc-alpha:

> @@ -733,25 +723,11 @@ search_dir (const struct dir_entry *entry)
>+ 1, ".#prelink#.", sizeof (".#prelink#.") - 1) == 0)
>   continue;
>   }
> -  len += strlen (entry->path) + 2;
> -  if (len > file_name_len)
> - {
> -   file_name_len = len;
> -   file_name = alloca (file_name_len);
> -   if (!opt_chroot)
> - real_file_name = file_name;
> - }
> -  sprintf (file_name, "%s/%s", entry->path, direntry->d_name);
> +  asprintf (_name, "%s/%s", entry->path, direntry->d_name);
>if (opt_chroot != NULL)
> - {
> -   len = strlen (dir_name) + strlen (direntry->d_name) + 2;
> -   if (len > real_file_name_len)
> - {
> -   real_file_name_len = len;
> -   real_file_name = alloca (real_file_name_len);
> - }
> -   sprintf (real_file_name, "%s/%s", dir_name, direntry->d_name);
> - }
> + asprintf (_file_name, "%s/%s", dir_name, direntry->d_name);
> +  else
> + real_file_name = file_name;
>  
>struct stat lstat_buf;
>/* We optimize and try to do the lstat call only if needed.  */

Missing error checking?

I think we use x* functions such a xstrdup elsewhere in ldconfig, so you
could add xasprintf as well.

Thanks,
Florian




Re: [RFC PATCH 0/2] On ldconfig and ld.so.cache

2023-05-19 Thread Carlos O'Donell
On 5/17/23 14:54, Sergey Bugaev via Libc-alpha wrote:
> Hello,
> 
> having set up a very basic x86_64-gnu system to debug startup issues, I
> was surprised to discover that my self-built ld.so does not look for the
> shared libraries in /lib/x86_64-gnu/ (which is where Samuel Thibault's deb
> packages place them) at all. I then learned that ld.so.cache and ldconfig
> and ld.so.conf support is explicitly getting compiled out in the *-gnu
> configurations -- and that this is being done intentionally, since
> ldconfig is apparently considered to be a hack and a misfeature [0].
> 
> [0]: https://lists.debian.org/debian-hurd/2001/04/msg00179.html
> 
> (Might this be just due to misunderstanding of what ldconfig does? It's
> not only about creating symlinks, it's primarily about the ld.so.cache and
> ld.so.conf...)
> 
> 
> 
> This doesn't really make sense to me: surely whether ld.so.cache is a
> good idea or not is not tied to a particular kernel? Surely the kernel
> could not care less about things like the file dystem layout and how
> ld.so looks for shared libraries? Predicating ld.so.cache usage on whether
> the GNU system is running on the Linux or Hurd kernel sounds like creating
> pointless differences to me.

I agree that we should reduce differences here, and that the data files used by 
the
loader should be more-or-less the same across kernels.

Keep in mind that ld.so.cache is not actually a cache, but a data file that 
contains
the pre-processed contents of /etc/ld.so.conf* in a format that can be easily 
loaded
by the dynamic loader without having a parser (glob included) in the early 
startup
code. Deleting the cache can break a system, and perhaps this is what some 
developers
have objected to and called "a hack."

I filed a bug for this in 2017 to capture the issue:
ld.so's cache should live in /var/cache, and support cache deletion.
https://sourceware.org/bugzilla/show_bug.cgi?id=22359

> Moreover, Debian GNU/Hurd, the primary Hurd-based distribution, has been
> shipping ld.so.cache on Hurd as a downstream patch [1] (note that more
> changes would be required for x86_64-gnu because of FLAG_X8664_LIB64).
> They don't really have a choice, it seems: with their "multiarch"
> filesystem layout, the shared libraries are to be located in
> /lib/i386-gnu/, but that is not a path that ld.so searches for libraries
> in! This is solved by use_ldconfig=yes, and putting /lib/i386-gnu/ into
> /etc/ld.so.conf.d/i386-gnu.conf, where it is then picked up by ldconfig,
> which finds all the libraries and bakes their paths into the cache, where
> ld.so then picks them up.

>From 2018 we have this: 
Implement multiarch ldconfig
https://sourceware.org/bugzilla/show_bug.cgi?id=22825


> [1]: 
> https://salsa.debian.org/glibc-team/glibc/-/raw/sid/debian/patches/hurd-i386/local-enable-ldconfig.diff
> 
> This is also another thing that suprises me: how come ldconfig does read
> ld.so.conf, but ld.so does not? It's not an "ldconfig.conf" after all...
> How is one even supposed to configure library paths with use_ldconfig=no?

The file /etc/ld.so.cache is not a cache, it is a data file used by the dynamic 
loader.

The data file is generated by ldconfig as it parses /etc/ld.so.conf*.

The only other way to configure library paths with `use_ldconfig=` is to use 
LD_PRELOAD,
LD_LIBRARY_PATH, DT_RPATH, or DT_RUNPATH.


> 
> 
> Anyway, maybe there are valid technical reasons to not enable ldconfig by
> default; this is not a hill that I'm willing to die on. But wouldn't it be
> nicer if it was easier to enable ldconfig downstream if desired? To that
> end, I tweaked ldconfig.c to not use PATH_MAX (sadly, there are still
> PATH_MAX usages in other places, e.g. chroot_canon.c), and moved a couple
> of files to stop being specific to the Linux port. I hope that both of
> these changes are uncontroversial. With these changes it is possible to
> just patch in use_ldconfig=yes downstream (is desired, of course) and get
> an ld.so.cache-enabled glibc on both i386-gnu and x86_64-gnu.

Removing configuration options and making it simple to configure and use glibc 
is great
goal. I think that ldconfig should always be enabled and I don't see a downside 
to making
`use_ldconfig=yes` the default, but I think we'd have to check with Guix or Nix 
to see if
they have any custom changes there. It involves probably a slightly broader 
distro
discussion.

-- 
Cheers,
Carlos.




Re: Guix hurd with rumpdisk boots! [WAS Re: Some progress, Guix rumpdisk still crashes...]

2023-05-19 Thread Janneke Nieuwenhuizen
Sergey Bugaev writes:

> On Fri, May 19, 2023 at 1:20 PM Janneke Nieuwenhuizen  wrote:
>> Okay, yeah I tried
>>
[..]
> See, it's only seeing a single bootstrap module, treating
> hurd/exec.static and the rest as just further arguments to
> hurd/ext2fs.static. I believe you have to separate modules with a
> comma -- see how I've done it in my previous letter. Here's what man
> qemu says:
>
> -initrd "file1 arg=foo,file2"
> This syntax is only available with multiboot. Use file1 and file2 as
> modules and pass arg=foo as parameter to the first module.

Oops, yes I missed that.  New try:

--8<---cut here---start->8---
guix shell qemu -- qemu-system-i386 \
-m 4096 \
--enable-kvm \
--device rtl8139,netdev=net0 \
--netdev user,id=net0,hostfwd=tcp:0.0.0.0:11022-: \
--snapshot \
--no-reboot \
--device virtio-serial-pci \
--nographic \
--serial mon:stdio \
--hda debian-hurd-20220824.img \
--kernel gnumach-1.8-486 \
--append "root=part:2:device:wd0 console=com0" \
--initrd "hurd/ext2fs.static ex2fs \
--multiboot-command-line='\${kernel-command-line}' \
--host-priv-port='\${host-port}' \
--device-master-port='\${device-port}' \
--exec-server-task='\${exec-task}' \
--store-type=typed \
--x-xattr-translator-records \
'\${root}' \
'\$(task-create)' \
'\$(task-resume)' \
,hurd/exec.static exec \
'\$(exec-task=task-create)'"
--8<---cut here---end--->8---

gives

--8<---cut here---start->8---
module 0: hurd/ext2fs.static ex2fs 
--multiboot-command-line='${kernel-command-line}' 
--host-priv-port='${host-port}' --device-master-port='${device-port}' 
--exec-server-task='${exec-task}' --store-type=typed 
--x-xattr-translator-records '${root}' '$(task-create)' '$(task-resume)' 
module 1: hurd/exec.static exec '$(exec-task=task-create)'
2 multiboot modules
[hang]
--8<---cut here---end--->8---

>> When I use noide with
>>
>>  http://cdimage.debian.org/cdimage/ports/latest/hurd-i386/ 
>> debian-hurd-20220824.img
>>
>> like so
>>
>> multiboot /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 
>> noide
>>
>> I get
>>
>> ext2fs: part:2:device:wd0: No such device or address
>
> Wait, no, don't try that with an installer image (that's what
> "cdimage" is, right?).

No, it's not a cdimage (the name of the url is fu), this is a
pre-installed qemu qcow image.  The README says

To give Debian GNU/Hurd a try, it is probably easier to simply run the
preinstalled image, which is provided here: [...]

so using this instead of installing something myself ensures we are
looking an the same thing.  Is there any newer image that I could try?

> Install it properly first and boot the installed system. The
> installation image, as I understand from Samuel's explanations, does
> not actually access the drive/cdrom, it's located on a ramdisk that is
> loaded into RAM by GRUB.

Yeah, so what I've tried should really work, right?

>> see full log attached.  The root (disk) is already in the format that
>> rump expects, rigth?
>
> Not that I know anything about rump, but my understanding is it does
> not care about the format, it's ext2fs that does. rumpdisk simply
> exposes the device.

Ah, that makes sense, right.

>>  Is there anything else I'd need to do; I would
>> like to get this to work on Debian first!
>
> Using rumpdisk on Debian should be a matter of changing
> part:1:device:hd0 to part:1:device:wd0 (why part:2?), and adding
> noide. (Unless I'm misremembering, of course, and note I'm not at all
> qualified to talk about any of this).

Hmm, the image above already has wd0 (part:2:device:wd0)...but it does
not use rump, now it really gets confusing...

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen   | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com



Re: [RFC PATCH 2/2] x86: Make dl-cache.h and readelflib.c not Linux-specific

2023-05-19 Thread Carlos O'Donell
On 5/17/23 14:54, Sergey Bugaev via Libc-alpha wrote:
> These files could be useful to any port that wants to use ld.so.cache.

I agree, and this is likely cargo-cult across the targets.

At most we look at EI_CLASS, and EM_* machine, and FLAG_ELF_LIBC6 is
defined in sysdeps/generic/ldconfig.h (not Linux specific) and likewise
for the other flags.

LGTM.

Reviewed-by: Carlos O'Donell 
 
> Signed-off-by: Sergey Bugaev 
> ---
>  sysdeps/{unix/sysv/linux => }/x86/readelflib.c  | 0
>  sysdeps/{unix/sysv/linux => }/x86_64/dl-cache.h | 0
>  2 files changed, 0 insertions(+), 0 deletions(-)
>  rename sysdeps/{unix/sysv/linux => }/x86/readelflib.c (100%)
>  rename sysdeps/{unix/sysv/linux => }/x86_64/dl-cache.h (100%)
> 
> diff --git a/sysdeps/unix/sysv/linux/x86/readelflib.c 
> b/sysdeps/x86/readelflib.c
> similarity index 100%
> rename from sysdeps/unix/sysv/linux/x86/readelflib.c
> rename to sysdeps/x86/readelflib.c
> diff --git a/sysdeps/unix/sysv/linux/x86_64/dl-cache.h 
> b/sysdeps/x86_64/dl-cache.h
> similarity index 100%
> rename from sysdeps/unix/sysv/linux/x86_64/dl-cache.h
> rename to sysdeps/x86_64/dl-cache.h



-- 
Cheers,
Carlos.




Re: Guix hurd with rumpdisk boots! [WAS Re: Some progress, Guix rumpdisk still crashes...]

2023-05-19 Thread Sergey Bugaev
On Fri, May 19, 2023 at 1:20 PM Janneke Nieuwenhuizen  wrote:
> Okay, yeah I tried
>
> --8<---cut here---start->8---
> guix shell qemu -- qemu-system-i386 \
> -m 4096 \
> --enable-kvm\
> --device rtl8139,netdev=net0\
> --netdev user,id=net0,hostfwd=tcp:0.0.0.0:11022-:   \
> --snapshot  \
> --no-reboot \
> --device virtio-serial-pci  \
> --nographic \
> --serial mon:stdio  \
> --hda debian-hurd-20220824.img  \
> --kernel gnumach-1.8-486\
> --append "root=part:2:device:wd0 console=com0"  \
> --initrd "hurd/ext2fs.static ex2fs  \
>  --multiboot-command-line='\${kernel-command-line}' \
>  --host-priv-port='\${host-port}'   \
>  --device-master-port='\${device-port}' \
>  --exec-server-task='\${exec-task}' \
>  --store-type=typed \
>  --x-xattr-translator-records   \
>  '\${root}' \
>  '\$(task-create)'  \
>  '\$(task-resume)'  \
>   hurd/exec.static exec \
>  '\$(exec-task=task-create)'"
> --8<---cut here---end--->8---
>
> but that stops here
>
> --8<---cut here---start->8---
> module 0: hurd/ext2fs.static ex2fs  
> --multiboot-command-line='${kernel-command-line}'  
> --host-priv-port='${host-port}'
> --device-master-port='${device-port}'  
> --exec-server-task='${exec-task}'  --store-type=typed 
>  --x-xattr-translator-records 
>'${root}'  
> '$(task-create)'   '$(task-resume)'   
>  hurd/exec.static exec
>   '$(exec-task=task-create)'
> 1 multiboot modules
> --8<---cut here---end--->8---

See, it's only seeing a single bootstrap module, treating
hurd/exec.static and the rest as just further arguments to
hurd/ext2fs.static. I believe you have to separate modules with a
comma -- see how I've done it in my previous letter. Here's what man
qemu says:

-initrd "file1 arg=foo,file2"
This syntax is only available with multiboot. Use file1 and file2 as
modules and pass arg=foo as parameter to the first module.

> When I use noide with
>
>  
> http://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd-20220824.img
>
> like so
>
> multiboot /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 
> noide
>
> I get
>
> ext2fs: part:2:device:wd0: No such device or address

Wait, no, don't try that with an installer image (that's what
"cdimage" is, right?). Install it properly first and boot the
installed system. The installation image, as I understand from
Samuel's explanations, does not actually access the drive/cdrom, it's
located on a ramdisk that is loaded into RAM by GRUB.

> see full log attached.  The root (disk) is already in the format that
> rump expects, rigth?

Not that I know anything about rump, but my understanding is it does
not care about the format, it's ext2fs that does. rumpdisk simply
exposes the device.

>  Is there anything else I'd need to do; I would
> like to get this to work on Debian first!

Using rumpdisk on Debian should be a matter of changing
part:1:device:hd0 to part:1:device:wd0 (why part:2?), and adding
noide. (Unless I'm misremembering, of course, and note I'm not at all
qualified to talk about any of this).

Sergey



Re: Guix hurd with rumpdisk boots! [WAS Re: Some progress, Guix rumpdisk still crashes...]

2023-05-19 Thread Janneke Nieuwenhuizen
Sergey Bugaev writes:

Hi!

> On Thu, May 18, 2023 at 11:07 AM Janneke Nieuwenhuizen  
> wrote:
>> Now that was really a great help, thanks!
>>
>> Ah, I had no idea; this is so helpful.  Maybe a good idea to have this
>> on the website/wiki, right?
>
> Glad I was able to help :D
>
>> Is there a way to pass the "console=com0" argument from the QEMU command
>> line?  That would be nice!
>
> I don't think you can alter the GRUB script from QEMU cmdline, but
> note that on 32-bit x86 (i?86) you don't even technically need GRUB:
> QEMU itself can act as a multiboot bootloader. Something like the
> following should work:
>
> $ qemu-system-x86_64 -other-args -kernel /path/to/gnumach -append
> "console=com0 other kernel args" -initrd "/path/to/pci-arbiter.static
> pci-arbiter args,/path/to/rumpdisk.static rumpdisk
> args,/path/to/ext2fs.static ext2fs args"
>
> (but I've only tried that with a single bootstrap task).

Okay, yeah I tried

--8<---cut here---start->8---
guix shell qemu -- qemu-system-i386 \
-m 4096 \
--enable-kvm\
--device rtl8139,netdev=net0\
--netdev user,id=net0,hostfwd=tcp:0.0.0.0:11022-:   \
--snapshot  \
--no-reboot \
--device virtio-serial-pci  \
--nographic \
--serial mon:stdio  \
--hda debian-hurd-20220824.img  \
--kernel gnumach-1.8-486\
--append "root=part:2:device:wd0 console=com0"  \
--initrd "hurd/ext2fs.static ex2fs  \
 --multiboot-command-line='\${kernel-command-line}' \
 --host-priv-port='\${host-port}'   \
 --device-master-port='\${device-port}' \
 --exec-server-task='\${exec-task}' \
 --store-type=typed \
 --x-xattr-translator-records   \
 '\${root}' \
 '\$(task-create)'  \
 '\$(task-resume)'  \
  hurd/exec.static exec \
 '\$(exec-task=task-create)'"
--8<---cut here---end--->8---

but that stops here

--8<---cut here---start->8---
module 0: hurd/ext2fs.static ex2fs  
--multiboot-command-line='${kernel-command-line}'  
--host-priv-port='${host-port}'
--device-master-port='${device-port}'  
--exec-server-task='${exec-task}'  --store-type=typed   
   --x-xattr-translator-records 
   '${root}'  '$(task-create)'  
 '$(task-resume)'   
 hurd/exec.static exec  
'$(exec-task=task-create)'
1 multiboot modules
--8<---cut here---end--->8---

>> Just for fun, find the succesful log attached.
>
> But that... does not look like rumpdisk actually gets used? The
> in-kernel IDE drivers are enabled, as you can see here:

Ah, uh oh...

> pass "noide" on gnumach cmdline to disable them, or just compile them
> out. I don't see it in your rumpdisk output, but when run this way it
> typically discovers that the kernel is already driving IDE, and does
> nothing.

Okay, make sense.

> Then you're passing "hd0s1" to ext2fs as the device to open; that's
> again a reference to the Mach-implemented device (and partition). The
> rumpdisk drive is named 'wd0', and you also have to do partitions
> libstore-side, so: root=part:1:device:wd0

Thanks.

When I use noide with

 
http://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd-20220824.img

like so

multiboot /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 noide

I get

ext2fs: part:2:device:wd0: No such device or address

see full log attached.  The root (disk) is already in the format that
rump expects, rigth?  Is there anything else I'd need to do; I would
like to get this to work on Debian first!

Greetings,
Janneke



noide.log
Description: Binary data

-- 
Janneke Nieuwenhuizen   | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com


Re: [RFC PATCH 06/10] hurd: Make sure to not use tcb->self

2023-05-19 Thread Florian Weimer
* Sergey Bugaev:

> On Thu, May 18, 2023 at 11:16 PM Joseph Myers  wrote:
>> Strictly there are two separate issues: (a) calling an exported symbol
>> should be done via a hidden alias, to avoid going via the PLT, and (b)
>> functions in a standard namespace should not call names in the user's
>> namespace, which requires using a name such as __hurd_thread_self instead
>> (which should also be marked hidden to make the call optimally efficient).
>
> While we're talking about this, could you please say if my
> understanding below is correct (and correct me if not)?
>
> 'foo' is a public symbol, to be used by the user, and also
> interposable by the user. Always called via PLT (except from inside
> the user's code when redefined inside the executable, in which case
> the compiler/linker will see that no PLT is needed), and should not be
> called from inside glibc.
>
> '__foo' is a (public? private? semi-private?) symbol, the user code is
> not supposed to use it, but it's exported from libc.so for the benefit
> of other glibc code that ends up in different DSOs and still wants to
> call the original version even when 'foo' gets interposed. Invoked via
> PLT from other DSOs, not invoked from libc.so because of...

__foo may be exported or not.  We have some __ symbols that are used by
the implementation (so GCC etc.), and probably some that are expected to
be used by users.  Truely exported symbols have a GLIBC_2.y symbol
version, internal exported symbols (usually added for coordination
between different shared objects that make up glibc) have a
GLIBC_PRIVATE symbol version.

Some old internal symbols have GLIBC_2.0 or similar early versions
because GLIBC_PRIVATE did not exist then.

> '__GI___foo' is a private non-exported symbol created by the
> hidden_def/hidden_proto macro being used on '__foo', this is what the
> code in libc.so (or: whatever DSO the symbol is hidden_def'ed in)
> calls to avoid PLT jumps.

Correct.

> Q: why '__foo', why not use hidden_def/hidden_proto on the 'foo' directly?
> A: that would only work for code that ends up in libc.so (or rather,
> the same DSOs as 'foo'), but we still want other code to also call the
> non-interposed, original version of 'foo'

hidden_def/hidden_proto does not do anything for static linking, given
the macros are defined today.

It's of course possible to do all this quite differently.

Thanks,
Florian




Re: [PATCH] faq/64-bit.mdwn: added up to date 64-bit porting info open_issues/64-bit_port.mdwn: added up to date 64-bit porting info

2023-05-19 Thread Svante Signell
On Sat, 2023-05-13 at 19:01 +, jbra...@dismail.de wrote:
> May 13, 2023 1:46 PM, "Sergey Bugaev"  wrote:
> 
> > On Sat, May 13, 2023 at 7:38 PM jbra...@dismail.de 
> > wrote:
> > 
> > > +Hurd developers are adding 64-bit support, and they plan to drop the
> > > +32-bit support, once the 64-bit support is deemed stable.
> > 
> > Is this really the plan? :(
> > 
> > Has this been discussed anywhere?
> 
> It is entirely possible that I made up the plan to drop 32-bit support.  
> I thought I had seem someone say that was the plan...

Why dropping 32-bit support? It does not make sense.

Thanks! 



Re: [RFC PATCH 06/10] hurd: Make sure to not use tcb->self

2023-05-19 Thread Sergey Bugaev
On Thu, May 18, 2023 at 11:16 PM Joseph Myers  wrote:
> Strictly there are two separate issues: (a) calling an exported symbol
> should be done via a hidden alias, to avoid going via the PLT, and (b)
> functions in a standard namespace should not call names in the user's
> namespace, which requires using a name such as __hurd_thread_self instead
> (which should also be marked hidden to make the call optimally efficient).

While we're talking about this, could you please say if my
understanding below is correct (and correct me if not)?

'foo' is a public symbol, to be used by the user, and also
interposable by the user. Always called via PLT (except from inside
the user's code when redefined inside the executable, in which case
the compiler/linker will see that no PLT is needed), and should not be
called from inside glibc.

'__foo' is a (public? private? semi-private?) symbol, the user code is
not supposed to use it, but it's exported from libc.so for the benefit
of other glibc code that ends up in different DSOs and still wants to
call the original version even when 'foo' gets interposed. Invoked via
PLT from other DSOs, not invoked from libc.so because of...

'__GI___foo' is a private non-exported symbol created by the
hidden_def/hidden_proto macro being used on '__foo', this is what the
code in libc.so (or: whatever DSO the symbol is hidden_def'ed in)
calls to avoid PLT jumps.

Q: why '__foo', why not use hidden_def/hidden_proto on the 'foo' directly?
A: that would only work for code that ends up in libc.so (or rather,
the same DSOs as 'foo'), but we still want other code to also call the
non-interposed, original version of 'foo'

Is that ^^ correct? Should each and every function that is exported to
the user and also used inside libc's own code have '__foo' and
'__GI___foo' versions? What does 'GI' even stand for?

Thanks in advance,
Sergey



Re: [PATCH glibc] Stop checking if MiG supports retcode.

2023-05-19 Thread Sergey Bugaev
Hi,

On Fri, May 19, 2023 at 9:43 AM Flávio Cruz  wrote:
> I have made changes so that it does daily builds and I'm able to boot small 
> programs. However, I haven't had the time to boot programs built against 
> Glibc. How do you package and boot the static binaries using a ramdisk? I've 
> been reading the other threads about the Guix/rumpkernel so I might be able 
> to piece something together and try it this weekend.

You just put the entirety of the root filesystem (containing /usr,
/bin, /lib, /hurd, and so on) as an ext2 image into a *file* that you
place onto the actual drive (a CD disk in my case), and then you ask
GRUB to load the file from the drive into memory, tell gnumach to make
a ramdisk device out of it (you'll need to apply [0]), and tell ext2fs
to use that device. Here's the relevant piece of my grub config
script:

[0]: 
https://salsa.debian.org/hurd-team/gnumach/-/blob/master/debian/patches/50_initrd.patch

multiboot /boot/gnumach console=com0
module /boot/initrd.ext2 initrd.ext2 '$(ramdisk-create)'
module /sbin/ext2fs.static ext2fs
--multiboot-command-line='${kernel-command-line}' --readonly
--host-priv-port='${host-port}' --device-master-port='${device-port}'
--exec-server-task='${exec-task}' --kernel-task='${kernel-task}' -T
device rd0 '$(fs-task=task-create)' '$(prompt-task-resume)'
module /lib/ld.so.1 ld.so.1 /hurd/exec
--device-master-port='${device-port}' '$(exec-task=task-create)'
boot

(I should probably change it to not hardcode 'rd0', but whatever).
Note that /boot/gnumach, /boot/initrd.ext2, /sbin/ext2fs.static, and
/lib/ld.so.1 are all paths inside the CD image (those are going to be
loaded by GRUB), and /boot/initrd.ext2 is the ext2 filesystem image
containing the actual Hurd root. /hurd/exec however is already a path
inside the fs image -- this is where ld.so (not grub) is going to load
the exec server from. The only static binary here is ext2fs.static,
the rest are all dynamically linked.

Then in /libexec/console-run (inside the filesystem image), I have
written the following:

#! /bin/sh

settrans -ac /dev/mach-console /hurd/streamio console
exec <>/dev/mach-console >&0 2>&0
echo Hello from /bin/sh!
exec /bin/sh -i

(If you're going to do the same, don't forget to create the
/dev/mach-console node beforehand, since the fs is read-only.) I also
had to patch streamio a little to do the \r -> \n conversion like
glibc already does in devstream:

diff --git a/trans/streamio.c b/trans/streamio.c
index 272a002c..0af1aea3 100644
--- a/trans/streamio.c
+++ b/trans/streamio.c
@@ -500,6 +500,9 @@ trivfs_S_io_read (struct trivfs_protid *cred,
  cred->po->openmodes & O_NONBLOCK);
   pthread_mutex_unlock (_lock);
   *data_len = data_size;
+  for (size_t i = 0; i < data_size; i++)
+if ((*data)[i] == '\r')
+  (*data)[i] = '\n';
   return err;
 }

(maybe I should also add echoing of input characters in the same way,
which is also what glibc's devstream does -- otherwise currently I
don't see what I'm typing on the console).

Make sure to use the very latest glibc (Samuel has already pushed all
of my patches upstream!) + the BRK_START hack.

Sergey



Re: [PATCH glibc] Stop checking if MiG supports retcode.

2023-05-19 Thread Flávio Cruz
Hi Sergey

Sorry for the late reply, I have been busy with work and also traveling.

On Tue, May 16, 2023 at 8:41 AM Sergey Bugaev  wrote:

> On Tue, May 16, 2023 at 6:02 PM Flávio Cruz  wrote:
> > Yes, I meant this when I said "generate code to call the server reply
> routine
> > in case of errors".
>
> Ah, I see.
>
> > Take the example of the term translator, it forces TypeCheck to be 0 so
> that
> > it can still receive replies in the error case (otherwise BAD_TYPECHECK
> would fail).
> > MiG itself should generate an updated stub code that first checks the
> > return code and, in case of error, assume the message has the shape of
> > mig_reply_header_t. I believe term currently gets garbage values for all
> > other missing arguments.
>
> So *that* what type checking being optional is useful for! Makes
> sense, thank you.
>
> But let me emphasize once again that retcode needs special handling on
> both client and server sides, to produce and consume error messages
> properly.
>
> Sergey
>
> P.S. I too am very interested in whether you would be able to get
> something booting and working on x86_64 with your cross-hurd
> mini-distro. I see you have pushed the buildsystem changes to build
> for x86_64-gnu; have you tried to get it working?


I have made changes so that it does daily builds and I'm able to boot small
programs. However, I haven't had the time to boot programs built against
Glibc. How do you package and boot the static binaries using a ramdisk?
I've been reading the other threads about the Guix/rumpkernel so I might be
able to piece something together and try it this weekend.


>
> I'll soon post another series of glibc fixes, so maybe consider trying
> them out before they land upstream. You're also going to need [0]. And
> you have to use either ramdisk or rumpdisk; just passing device:hd0s1
> is not going to work without in-kernel IDE drivers.
>
> [0]: https://sourceware.org/pipermail/libc-alpha/2023-April/147734.html
>