Re: [RFC PATCH 0/2] On ldconfig and ld.so.cache

[Ludovic Courtès]

Hi,

Sergey Bugaev  skribis:

> On Fri, May 19, 2023 at 2:52 PM Carlos O'Donell  wrote:
>> Removing configuration options and making it simple to configure and use 
>> glibc is great
>> goal. I think that ldconfig should always be enabled and I don't see a 
>> downside to making
>> `use_ldconfig=yes` the default, but I think we'd have to check with Guix or 
>> Nix to see if
>> they have any custom changes there. It involves probably a slightly broader 
>> distro
>> discussion.

I missed the beginning of the discussion and I fear I’m slightly
changing topics :-), but to me we should work toward per-application
loader caches:

  https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/
  
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/glibc-dl-cache.patch?id=6d0571215d661d21cac2150ca45906e77a79a5fb

This is the one custom change we have in Guix.

(And IMO it should work the same on Linux and on the Hurd.)

Ludo’.

Pushing a MiG release tarball?

[Ludovic Courtès]

Hello!

Version 2.35 of glibc expects ‘const’-qualified parameters in
MiG-generated stubs.  Unfortunately, that feature is not is MiG 1.8, the
latest release.

Would you consider tagging a new release and uploading a tarball to
ftp.gnu.org?

The reason I’m asking is that Guix will need such a tarball in its
dependency tree; building from a checkout would require adding
dependencies on Autotools, which would significantly complicate
bootstrapping.

TIA,
Ludo’.

Re: Apparent deadlock in processes interacting with /hurd/fifo

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> Ludovic Courtès, le ven. 25 nov. 2022 12:35:58 +0100, a ecrit:
>> Let’s assume you do this:
>> 
>>   mkfifo fifo
>>   rpctrace cat fifo
>> 
>> I think there’s at least one bug here: ‘dir_lookup’ should complete
>> immediately; it’s ‘io_read’ that should block.
>
> ? No. Open Group says about open():
>
>   ○ If O_NONBLOCK is clear, an open() for reading-only shall block the
> calling thread until a thread opens the file for writing. An 
> open()
> for writing-only shall block the calling thread until a thread
> opens the file for reading.
>
> (and Linux does that indeed)


Indeed, I had overlooked that.

So it seems that the reason ‘df /tmp/fifo’ worked with Coreutils 8.32 on
GNU/Linux is that there it would not try to open the file:

--8<---cut here---start->8---
newfstatat(AT_FDCWD, "/tmp/fifo", {st_mode=S_IFIFO|0644, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
read(3, "22 25 0:20 / /proc rw,relatime -"..., 1024) = 1024
read(3, "30 / /sys/fs/cgroup/cpuset rw,re"..., 1024) = 1024
read(3, "27308k,nr_inodes=406827,mode=700"..., 1024) = 50
read(3, "", 1024)   = 0
lseek(3, 0, SEEK_CUR)   = 2098
close(3)= 0
ioctl(1, TCGETS, {B38400 opost isig icanon -echo ...}) = 0
readlink("/tmp", 0x7ffcb94aabf0, 1023)  = -1 EINVAL (Invalid argument)
readlink("/tmp/fifo", 0x7ffcb94aabf0, 1023) = -1 EINVAL (Invalid argument)
newfstatat(AT_FDCWD, "/", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
uname({sysname="Linux", nodename="ribbon", ...}) = 0
statfs("/tmp/fifo", {f_type=EXT2_SUPER_MAGIC, f_bsize=4096, f_blocks=121739374, 
f_bfree=8850770, f_bavail=2660102, f_files=30932992, f_ffree=23669755, 
f_fsid={val=[0x627eb274, 0x360fd8c4]}, f_namelen=255, f_frsize=4096, 
f_flags=ST_VALID|ST_RELATIME}) = 0
--8<---cut here---end--->8---

Thanks for your feedback!

Ludo’.

Re: coreutils-8.32 test failure on i586-gnu

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> It seems that replying on the web interface didn't work, so replying
> again here by mail.

Weird.

> Ludovic Courtès, le mer. 26 oct. 2022 19:20:07 +0200, a ecrit:
>> ludo@childhurd ~$ mkfifo fifo
>> ludo@childhurd ~$ ls -l fifo
>> prw-r--r-- 1 ludo users 0 Oct 21 07:08 fifo
>> ludo@childhurd ~$ df ./fifo
>> df: Warning: cannot read table of mounted file systems: No such file or 
>> directory
>> ^C
>
> Using coreutils 9.1, I am not getting the issue any more:
>
> € mkfifo foo
> € ls -l foo
> prw-r--r-- 1 samy samy 0 Nov 26 11:47 foo
> € df ./foo
> Filesystem  Size  Used Avail Use% Mounted on
> -20G   18G  1.8G  91% /

Oh nice.  This may be a side effect of Coreutils commit
a3c04f8da14f0fe2a0561bf5562032b8ce5dafa9 (which made it in 9.0), which
opens the file with O_NONBLOCK.

>> Maybe /hurd/fifo needs to implement ‘trivfs_check_open_hook’ to avoid
>> actually opening?
>
> I don't think it will change the behavior? trivfs_S_dir_lookup will call
> trivfs_open after trivfs_check_open_hook anyway.

Indeed, that was misguided, sorry for the confusion.

Ludo’.

Re: Apparent deadlock in processes interacting with /hurd/fifo

[Ludovic Courtès]

Ludovic Courtès  skribis:

> I think there’s at least one bug here: ‘dir_lookup’ should complete
> immediately; it’s ‘io_read’ that should block.

This issue also breaks a Coreutils test:

  https://issues.guix.gnu.org/58803

Ludo’.

Apparent deadlock in processes interacting with /hurd/fifo

[Ludovic Courtès]

Hello,

Let’s assume you do this:

  mkfifo fifo
  rpctrace cat fifo

In another terminal, you find the PID of ‘cat’ and run “kill PID”,
twice.  The second ‘kill’ command hangs.  The ‘cat’ process, which was
initially stuck in dir_lookup("fifo"), eventually fails to service the
second kill request (‘msg_sig_post_request’):

--8<---cut here---start->8---
task111(pid14902)-> 4205 (227424 0 0);
  91<--123(pid14902)->dir_lookup ("fifo" 1 0) ...131
  118<--132(pid-1)->msg_get_init_port ( task111(pid14902) 4) ...115
task111(pid14902)-> 4205 (227424 0 0);
task111(pid14902)-> 4205 (227424 0 0);
task111(pid14902)-> 4205 (227424 0 0);
task111(pid14902)-> 3208 (pn{ 12} 0 1) = 0 
115... = 098<--127(pid14902)
  118<--132(pid-1)->msg_sig_post_request (15 0  task111(pid14902)) ...115
task111(pid14902)-> 4205 (227424 0 0);
task111(pid14902)-> 3206 (pn{  1}) = 0 
task111(pid14902)-> 4205 (227424 0 0);
  120<--126(pid14902)->proc_dostop_request ( thread128(pid14902)) = 0 
task111(pid14902)-> 4205 (227424 0 0);
115... = 0 
thread113(pid14902)-> 2064 () = 0 
thread113(pid14902)-> 2017 (5 17) = 0 {75 31 31 31 19451904 3 0 0 17007276 
1868982630 17007168 268451845 17302849 23 582 17007228 31}
task111(pid14902)-> 4205 (227424 0 0);
thread113(pid14902)-> 3808 (75) = 0 {-127074305 30405123}
task111(pid14902)-> 4205 (227424 0 0);
  91<--123(pid14902)->interrupt_operation () = 0 
  118<--132(pid-1)->msg_sig_post_request (15 0  task111(pid14902))
--8<---cut here---end--->8---

It eventually completes after several seconds:

--8<---cut here---start->8---
task111(pid14902)-> 4205 (227424 0 0);
  91<--123(pid14902)->interrupt_operation () = 0 
  118<--132(pid-1)->msg_sig_post_request (15 0  task111(pid14902)) ...115
task111(pid14902)-> 4205 (227424 0 0);
  120<--126(pid14902)->proc_mark_exit_request (15 0) = 0 
task111(pid14902)-> 4205 (227424 0 0);
task111(pid14902)-> 2008 () ...137
 115->   71 ();
137... = 0 
Child 14902 Terminated
--8<---cut here---end--->8---

Does that ring a bell?

I think there’s at least one bug here: ‘dir_lookup’ should complete
immediately; it’s ‘io_read’ that should block.

Thanks,
Ludo’.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi,

Ludovic Courtès  skribis:

> Of course, the thing boots just fine on that machine when using
> ‘exec.static’.

It’s frustrating I did not get to the bottom of it, but time passes, so
I pushed this workaround in Guix commit
3fb3bd3da530a5f82a169b1fa451474f9d90c3b6.

Ludo’.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi,

Ludovic Courtès  skribis:

> … so ‘exec_load’ is doing its job, it seems.

Turns out that may not be the case.

Here’s a *bad* mapping on the second ‘task_resume’ breakpoint (when
‘exec’ is about to start):

--8<---cut here---start->8---
  db> show all threads
  TASKTHREADS
0 gnumach (f5f7cf00): 7 threads:
0 (f5f7be18) .W..N. 0xc11dac04
1 (f5f7bcd0) R..O..(idle_thread_continue)
2 (f5f7bb88) .W.ON.(reaper_thread_continue) 0xc12015d4
3 (f5f7ba40) .W.ON.(swapin_thread_continue) 0xc11f8e2c
4 (f5f7b8f8) .W.ON.(sched_thread_continue) 0
5 (f5f7b7b0) .W.ON.(io_done_thread_continue) 0xc1201f74
6 (f5f7b668) .W.ON.(net_thread_continue) 0xc11db0a8
1 ext2fs (f5f7ce40): 6 threads:
0 (f5f7b520) RF
1 (f5f7b290) .W.O..(mach_msg_receive_continue) 0
2 (f5f7b148) .W.O..(mach_msg_receive_continue) 0
3 (f5f7b000) .W.O..(mach_msg_continue) 0
4 (f67d3e20) .W.O..(mach_msg_receive_continue) 0
5 (f67d3cd8) .W.O..(mach_msg_continue) 0
2 exec (f5f7cd80): (f5f7b3d8) ..SO..(thread_bootstrap_return)
   db> trace
  task_resume(f593e010,fb7d9010,f5f73e80,c106972a)
  ipc_kobject_server(f593e000,3,18,0)+0x1eb
  mach_msg_trap(b4c0,3,18,20,8)+0x1703
  >>>>> user space <<<<<
  db> x/tbx 0xcbc 0xf5f7b3d8

  no memory is assigned to address 0cbc
  0
  db> show map $map2
Map 0xf5f6ff30: name="exec", pmap=0xf5f71fa8,ref=1,nentries=5
  size=290816,resident:225280,wired=0
  version=13
 map entry 0xf625ec08: start=0x0, end=0x1000
 prot=1/7/copy, object=0x0, offset=0x0
 map entry 0xf625ebb0: start=0x1000, end=0x26000
 prot=5/7/copy, object=0xf5f6ad70, offset=0x0
  Object 0xf5f6ad70: size=0x25000, 1 references
  37 resident pages, 0 absent pages, 0 paging ops
   memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82780
   uninitialized,temporaryinternal,copy_strategy=0
   shadow=0x0 (offset=0x0),copy=0x0
 map entry 0xf625eb58: start=0x26000, end=0x34000
 prot=1/7/copy, object=0xf5f6ad20, offset=0x0
  Object 0xf5f6ad20: size=0xe000, 1 references
  14 resident pages, 0 absent pages, 0 paging ops
   memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82730
   uninitialized,temporaryinternal,copy_strategy=0
   shadow=0x0 (offset=0x0),copy=0x0
 map entry 0xf625eb00: start=0x34000, end=0x37000
 prot=3/7/copy, object=0xf5f6acd0, offset=0x0
  Object 0xf5f6acd0: size=0x3000, 1 references
  3 resident pages,--db_more--
--8<---cut here---end--->8---

Compare with what a “good” mapping looks like at that same moment:

--8<---cut here---start->8---
  start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1]Kernel Breakpoint 
trap,
   eip 0xc1030d5b
  Breakpoint at  task_resume: pushl   %ebp
  db> show all threads
  TASKTHREADS
0 gnumach (f5f7cf00): 7 threads:
0 (f5f7be18) .W..N. 0xc11dac04
1 (f5f7bcd0) R..O..(idle_thread_continue)
2 (f5f7bb88) .W.ON.(reaper_thread_continue) 0xc12015d4
3 (f5f7ba40) .W.ON.(swapin_thread_continue) 0xc11f8e2c
4 (f5f7b8f8) .W.ON.(sched_thread_continue) 0
5 (f5f7b7b0) .W.ON.(io_done_thread_continue) 0xc1201f74
6 (f5f7b668) .W.ON.(net_thread_continue) 0xc11db0a8
1 ext2fs (f5f7ce40): 6 threads:
0 (f5f7b520) RF
1 (f5f7b290) .W.O..(mach_msg_receive_continue) 0
2 (f5f7b148) .W.O..(mach_msg_receive_continue) 0
3 (f5f7b000) .W.O..(mach_msg_continue) 0
4 (f67d2e20) .W.O..(mach_msg_receive_continue) 0
5 (f67d2cd8) .W.O..(mach_msg_continue) 0
2 exec (f5f7cd80): (f5f7b3d8) ..SO..(thread_bootstrap_return)
  db> x/tbx 0xcbc 0xf5f7b3d8
  8
  db> show map $map2
  Map 0xf5f6ff30: name="exec", pmap=0xf5f71fa8,ref=1,nentries=5
  size=290816,resident:229376,wired=0
  version=14
   map entry 0xf625ec08: start=0x0, end=0x1000
   prot=1/7/copy, object=0xf5f6ad70, offset=0x0
Object 0xf5f6ad70: size=0x1000, 1 references
1 resident pages, 0 absent pages, 0 paging ops
 memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82780
 uninitialized,temporary  internal,copy_strategy=0
 shadow=0x0 (offset=0x0),copy=0x0
   map entry 0xf625ebb0: start=0x1000, end=0x26000
   prot=5/7/copy, object=0xf5f6ad20, offset=0x0
Object 0xf5f6ad20: size=0x25000, 1 references
37 resident pages, 0 absent pages, 0 paging ops
 memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82730
 uninitialized,temporary  internal,copy_strategy=0
 shadow=0x0 (offse

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Ludovic Courtès  skribis:

> Through a dichotomy I tried to see how far it goes.  The info I have so
> far is that ld.so errors out from elf/rtld.c:563 (line 565 is not
> reached):
>
> 558:  if (bootstrap_map.l_addr || ! 
> bootstrap_map.l_info[VALIDX(DT_GNU_PRELINKED)])
> 559:{
> 560:  /* Relocate ourselves so we can do normal function calls and
> 561: data access using the global offset table.  */
> 562:
> 563:  ELF_DYNAMIC_RELOCATE (_map, 0, 0, 0);
> 564:}
> 565:  bootstrap_map.l_relocated = 1;
> ...
> 578:  __rtld_malloc_init_stubs ();

Via brute force¹, I found that ‘__assert_fail’ is hit, with its first
argument in $eax being:

--8<---cut here---start->8---
db> x/c 0x28604,80  

ELF32_R_TYPE (reloc->r_info) == R_386_RELATIVE\000\000map->l_in 

fo[VERSYMIDX (DT_VERSYM)] != NULL\000\000Fatal glibc error: Too 

 many audit mo  

--8<---cut here---end--->8---

This comes from i386/dl-machine.h:

--8<---cut here---start->8---
auto inline void
__attribute ((always_inline))
elf_machine_rel_relative (Elf32_Addr l_addr, const Elf32_Rel *reloc,
  void *const reloc_addr_arg)
{
  Elf32_Addr *const reloc_addr = reloc_addr_arg;
  assert (ELF32_R_TYPE (reloc->r_info) == R_386_RELATIVE);
  *reloc_addr += l_addr;
}
--8<---cut here---end--->8---

How can we get there?  Looking at ‘_dl_start’, it could be that
‘elf_machine_load_address’ returns a bogus value and we end up reading
wrong ELF data?  Or it could be memory corruption somewhere.  Or…?

Thing is, it’s not fully deterministic (happens 9 times out of 10 with
KVM, never happens without KVM).

Ideas?  :-)

Ludo’.

¹ Building with ‘-fno-optimize-sibling-calls’ didn’t help get nicer
  backtraces, but that’s prolly because all that early relocation code
  is inlined.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi!

Ludovic Courtès  skribis:

> $ addr2line -e  
> /gnu/store/m8afvcgwmrfhvjpd7b0xllk8vv5isd6j-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1
>  0x1000 0x11627 0x11bb
> ??:0
> /tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/glibc-2.33/elf/dl-misc.c:333
> :?
>
>
> That’s ‘_dl_fatal_printf’ calling ‘_exit’; it’s trying to tell us
> something.
>
> I’ll try and rebuild the system with the debugging patches at
> <https://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00038.html>, to
> get early ld.so output, for lack of a better solution…

I tried adapted the patches above and tried them, but it seems that
‘_dl_sysdep_start’ isn’t even reached.  For example, I set a breakpoint
on ‘mach_task_self’ (called from ‘__mach_init’, called from
‘_dl_sysdep_start’), but that’s never reached (I’m assuming ‘break/tu’
is reliable, is it?).

The user-space backtrace upon trap remains unhelpful:

--8<---cut here---start->8---
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1]Kernel Breakpoint 
trap,
 eip 0xc1030d5b 

Breakpoint at  task_resume: pushl   %ebp

db> debug traps /on 

db> b task_terminate

set breakpoint #2   

db> c   

Kernel Debug trap trap, eip 0xc1030d5b  

 execkernel: Page fault (14), code=6

Stopped at  0x1000: pushl   0x4(%ebx)   

>>>>> user space <<<<<  
>>>>> 
0x1000(bf24,0,0,1160b,0)

0x11627(bf9c,0,0,0,2)   

0x11bb()

--8<---cut here---end--->8---

… where:

--8<---cut here---start->8---
$ addr2line -e 
/gnu/store/4p1kab1c4h7h3kvgcm1hbjja4y5k9x4p-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1
 0x11627 0x11bb
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/glibc-2.33/elf/dl-misc.c:333
:?
$ objdump -S 
/gnu/store/4p1kab1c4h7h3kvgcm1hbjja4y5k9x4p-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1
 --start-address=0x11b0 |head -40

/gnu/store/4p1kab1c4h7h3kvgcm1hbjja4y5k9x4p-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1:
 file format elf32-i386


Disassembly of section .text:

11b0 <_start>:
11b0:   89 e0   mov%esp,%eax
11b2:   83 ec 0csub$0xc,%esp
11b5:   50  push   %eax
11b6:   e8 b5 0a 00 00  call   1c70 <_dl_start>
11bb:   83 c4 10add$0x10,%esp
--8<---cut here---end--->8---

So it would seem that ‘_dl_start’ is called and somehow then a tail-call
to ‘_dl_fatal_printf’ is made.

Through a dichotomy I tried to see how far it goes.  The info I have so
far is that ld.so errors out from elf/rtld.c:563 (line 565 is not
reached):

--8<---cut here---start->8---
558:  if (bootstrap_map.l_addr || ! 
bootstrap_map.l_info[VALIDX(DT_GNU_PRELINKED)])
559:{
560:  /* Relocate ourselves so we can do normal function calls and
561: data access using the global offset table.  */
562:
563:  ELF_DYNAMIC_RELOCATE (_map, 0, 0, 0);
564:}
565:  bootstrap_map.l_relocated = 1;
...
578:  __rtld_malloc_init_stubs ();
--8<---cut here---end--->8---

It’s hard to be more precise because ELF_DYNAMIC_RELOCATE is a macro
that expands to quite a lot of code.

I don’t see the code path that would lead to a ‘_dl_fatal_printf’ call
though.

Ideas?  :-)

Ludo’.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi Samuel,

Samuel Thibault  skribis:

> About the backtrace:
>
>> user space <
> 0x1000(bf24,0,0,1160b,0)
> 0x11627(bf9c,0,0,0,2)
> 0x11bb()
>
> That is quite surprising actually: in my ld.so there is nothing useful
> at 0x1000. Perhaps you can check what 0x11627 is all about?

Sure:

--8<---cut here---start->8---
$ addr2line -e  
/gnu/store/m8afvcgwmrfhvjpd7b0xllk8vv5isd6j-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1
 0x1000 0x11627 0x11bb
??:0
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/glibc-2.33/elf/dl-misc.c:333
:?
--8<---cut here---end--->8---

That’s ‘_dl_fatal_printf’ calling ‘_exit’; it’s trying to tell us
something.

I’ll try and rebuild the system with the debugging patches at
, to
get early ld.so output, for lack of a better solution…

>> Program Header:
>> LOAD off0x vaddr 0x paddr 0x align 2**12
>>  filesz 0x0dd8 memsz 0x0dd8 flags r--
>
> We don't have this section in the Debian glibc. It'd probably be useful
> to know what this is about.

Address 0 is for the ‘_begin’ symbol, passed by -Wl,-defsym:

--8<---cut here---start->8---
i586-pc-gnu-gcc   -nostdlib -nostartfiles -r -o 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/librtld.os 
'-Wl,-(' 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/dl-allobjs.os 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/rtld-libc.a -lgcc 
'-Wl,-)' \
  
-Wl,-Map,/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/librtld.os.map
i586-pc-gnu-gcc   -nostdlib -nostartfiles -shared -o 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/ld.so.new  
   \
  -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,defs   
\
  
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/librtld.os 
-Wl,--version-script=/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/ld.map
  \
  -Wl,-soname=ld.so.1   \
  -Wl,-defsym=_begin=0
i586-pc-gnu-readelf -s 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/ld.so.new \
  | gawk '($7 ~ /^UND(|EF)$/ && $1 != "0:" && $4 != "REGISTER") { print; p=1 } 
END { exit p != 0 }'
mv -f /tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/ld.so.new 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.33.drv-0/build/elf/ld.so
--8<---cut here---end--->8---

And indeed:

--8<---cut here---start->8---
$ objdump -t  
/gnu/store/m8afvcgwmrfhvjpd7b0xllk8vv5isd6j-glibc-cross-i586-pc-gnu-2.33/lib/ld.so.1|grep
 _begin
 l   *ABS*    _begin
--8<---cut here---end--->8---

That ‘-Wl,-defsym=_begin=0’ flag was removed in glibc commit
6f043e0ee7e477f50a44024ed0cb579d5e3f511d (April 2022).

On darnassus it’s different but then it’s Debian’s glibc 2.35, natively
built, so I don’t what conclusions can be drawn:

--8<---cut here---start->8---
ludo@darnassus:~$ /lib/ld.so.1 --version
ld.so (Debian GLIBC 2.35-1) stable release version 2.35.
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
ludo@darnassus:~$ objdump -x /lib/ld.so.1 |head -40

/lib/ld.so.1: file format elf32-i386
/lib/ld.so.1
architecture: i386, flags 0x0150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0001cc40

Program Header:
LOAD off0x vaddr 0x paddr 0x align 2**12
 filesz 0x00038494 memsz 0x00038494 flags r-x
LOAD off0x00038c00 vaddr 0x00039c00 paddr 0x00039c00 align 2**12
 filesz 0x1ca8 memsz 0x1e34 flags rw-
 DYNAMIC off0x00039f24 vaddr 0x0003af24 paddr 0x0003af24 align 2**2
 filesz 0x00b8 memsz 0x00b8 flags rw-
NOTE off0x0114 vaddr 0x0114 paddr 0x0114 align 2**2
 filesz 0x0024 memsz 0x0024 flags r--
--8<---cut here---end--->8---

Thanks for your feedback!

Ludo’.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi!

Samuel Thibault  skribis:

> Ludovic Courtès, le ven. 07 oct. 2022 00:10:15 +0200, a ecrit:

[...]

>> Of course, the thing boots just fine on that machine when using
>> ‘exec.static’.
>
> Uh. At least you have a workaround :)

Yup.  :-)

>> So the issue might be somewhere in ld.so, or triggered by ld.so.
>> Any debugging tricks here?
>
> maybe for a start check with 
>
> show map $map2
>
> what is actually mapped, whether it's just ld.so, or also exec, etc.

On the first trap (page fault) I see:

--8<---cut here---start->8---
db> show all tasks  

 ID TASK NAME [THREADS] 

  0 f5f7cf00 gnumach [7]

  1 f5f7ce40 ext2fs [6] 

  2 f5f7cd80 exec [1]   

db> show map $map2  

Map 0xf5f6ff30: name="exec", pmap=0xf5f71fa8,ref=1,nentries=5   

size=290816,resident:290816,wired=0 

version=14  

 map entry 0xf625ec08: start=0x0, end=0x1000

 prot=1/7/copy, object=0xf5f6a7d0, offset=0x0   

  Object 0xf5f6a7d0: size=0x1000, 1 references  

  1 resident pages, 0 absent pages, 0 paging ops

   memory object=0x0 (offset=0x0),control=0x0, name=0xf5938968  

   uninitialized,temporary  internal,copy_strategy=0

   shadow=0x0 (offset=0x0),copy=0x0 

 map entry 0xf625ebb0: start=0x1000, end=0x26000

 prot=5/7/copy, object=0xf5f6ad70, offset=0x0   

  Object 0xf5f6ad70: size=0x25000, 1 references 

  37 resident pages, 0 absent pages, 0 paging ops   

   memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82780  

   uninitialized,temporary  internal,copy_strategy=0

   shadow=0x0 (offset=0x0),copy=0x0 

 map entry 0xf625eb58: start=0x26000, end=0x34000   

 prot=1/7/copy, object=0xf5f6ad20, offset=0x0   

  Object 0xf5f6ad20: size=0xe000, 1 references  

  14 resident pages, 0 absent pages, 0 paging ops   

   memory object=0x0 (offset=0x0),control=0x0, name=0xf5f82730  

   uninitialized,temporary  internal,copy_strategy=0

   shadow=0x0 (offset=0x0),copy=0x0 

 map entry 0xf625eb00: start=0x34000, end=0x37000   

 prot=3/7/copy, object=0xf5f6acd0, offset=0x0   

  Object 0xf5f6acd0: size=0x3000, 1 references  

  3 resident pages, 0 absent pages, 0 paging ops

   memory object=0x0 (offset=0x0),control=0x0, name=0xf5f826e0  

   uninitialized,temporary  internal,copy_strategy=0

   shadow=0x0 (offset=0x0),copy=0x0 

 map entry 0xf625eaa8: start=0xbfff, end=0xc000 
 

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Samuel Thibault  skribis:

> Ludovic Courtès, le jeu. 06 oct. 2022 15:14:13 +0200, a ecrit:
>> such that it would stop on each trap, hopefully allowing me to see why
>> exec is not starting.
>
> Also, better use exec.static to have static addresses.

Thanks for the hint.

Of course, the thing boots just fine on that machine when using
‘exec.static’.

So the issue might be somewhere in ld.so, or triggered by ld.so.
Any debugging tricks here?

Ludo’.

Re: bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hi!

As suggested by Samuel on IRC, I did that early on in kdb:

  debug traps /on

such that it would stop on each trap, hopefully allowing me to see why
exec is not starting.

--8<---cut here---start->8---
module 0: ext2fs --multiboot-command-line=${kernel-command-line} 
--host-priv-por
t=${host-port} --device-master-port=${device-port} 
--exec-server-task=${exec-tas
k} --store-type=typed --x-xattr-translator-records ${root} $(task-create) 
$(task
-resume)

module 1: exec 
/gnu/store/99sqiayswrxxb80331pl7jxin18wv28b-hurd-0.9-1.91a5167/hu   
 
rd/exec $(exec-task=task-create)

2 multiboot modules 

task loaded: ext2fs --multiboot-command-line=root=device:hd0s1 
root=3367134b-cfb
d-1e90-2f38-dfd13367134b 
gnu.system=/gnu/store/m66ccpdzdbcd3k2fdvyaj8cgmk23lybn-
system gnu.load=/gnu/store/m66ccpdzdbcd3k2fdvyaj8cgmk23lybn-system/boot 
--host-p
riv-port=1 --device-master-port=2 --exec-server-task=3 --store-type=typed 
--x-xa
ttr-translator-records device:hd0s1 

task loaded: exec 
/gnu/store/99sqiayswrxxb80331pl7jxin18wv28b-hurd-0.9-1.91a5167  
  
/hurd/exec  



start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] execkernel: Page 
fault
 (14), code=6   

Stopped at  0x1000: pushl   0x4(%ebx)   

> user space <  
> 
0x1000(bf24,0,0,1160b,0)

0x11627(bf9c,0,0,0,2)   

0x11bb()

db> show all threads

TASKTHREADS 

  0 gnumach (f5f7cf00): 7 threads:  

  0 (f5f7be18) .W..N. 0xc11dac04

  1 (f5f7bcd0) R..O..(idle_thread_continue) 

  2 (f5f7bb88) .W.ON.(reaper_thread_continue) 0xc12015d4

  3 (f5f7ba40) .W.ON.(swapin_thread_continue) 0xc11f8e2c

  4 (f5f7b8f8) .W.ON.(sched_thread_continue) 0  

  5 (f5f7b7b0) .W.ON.(io_done_thread_continue) 0xc1201f74   

  6 (f5f7b668) .W.ON.(net_thread_continue) 0xc11db0a8   

  1 ext2fs (f5f7ce40): 6 threads:   

  0 (f5f7b520) .W.O.F(mach_msg_continue) 0  

  1 (f5f7b290) .W.O..(mach_msg_receive_continue) 0  

  2 (f5f7b148) .W.O..(mach_msg_receive_continue) 0  

  3 (f5f7b000) .W.O..(mach_msg_continue) 0  

  4 (f67d4e20) .W.O..(mach_msg_receive_continue) 0  

  5 (f67d4cd8) .W.O..(mach_msg_continue) 0  

  2 exec (f5f7cd80): (f5f7b3d8) R.  

--8<---cut here---end--->8---

Then lots of page faults with the same stack trace, seemingly endlessly:

--8<---cut here---start->8---
db> c   

kernel: Page fault (14), code=6 

Stopped at  0x1000: pushl   0x4(%ebx)   

> user space <  
> 
0x1000(bf24,0,0,1160b,0)  

bug#58320: Hurd VM fails to boot on AMD EPYC (kvm-amd)

[Ludovic Courtès]

Hello!

On AMD EPYC processors, as found on the build nodes of ci.guix.gnu.org,
childhurd VMs fail to boot when running with ‘qemu-system-i386
-enable-kvm’ (the kvm-amd Linux kernel module is used), with the Hurd
startup process hanging before /hurd/exec has been started:

--8<---cut here---start->8---
module 0: ext2fs --multiboot-command-line=${kernel-command-line} 
--host-priv-por
t=${host-port} --device-master-port=${device-port} 
--exec-server-task=${exec-tas
k} --store-type=typed --x-xattr-translator-records ${root} $(task-create) 
$(task
-resume)

module 1: exec 
/gnu/store/99sqiayswrxxb80331pl7jxin18wv28b-hurd-0.9-1.91a5167/hu   
 
rd/exec $(exec-task=task-create)

2 multiboot modules 

task loaded: ext2fs --multiboot-command-line=root=device:hd0s1 
root=3367134b-cfb
d-1e90-2f38-dfd13367134b 
gnu.system=/gnu/store/m66ccpdzdbcd3k2fdvyaj8cgmk23lybn-
system gnu.load=/gnu/store/m66ccpdzdbcd3k2fdvyaj8cgmk23lybn-system/boot 
--host-p
riv-port=1 --device-master-port=2 --exec-server-task=3 --store-type=typed 
--x-xa
ttr-translator-records device:hd0s1 

task loaded: exec 
/gnu/store/99sqiayswrxxb80331pl7jxin18wv28b-hurd-0.9-1.91a5167  
  
/hurd/exec  



start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec  

--8<---cut here---end--->8---

Kdb shows these two tasks:

--8<---cut here---start->8---
Stopped   at  mach
ine_idle+0x26:  nop 

machine_idle(0,c102f380,f5f7bcd0,c102e3fa)+0x26 

idle_thread_continue(f5f7ade0,36366000,f5f73868,f5f73890,0)+0x73

> user space <  
> 
db> show all threads

TASKTHREADS 

  0 gnumach (f5f7cf00): 7 threads:  

  0 (f5f7be18) .W..N. 0xc11dac04

  1 (f5f7bcd0) R.   

  2 (f5f7bb88) .W.ON.(reaper_thread_continue) 0xc12015d4

  3 (f5f7ba40) .W.ON.(swapin_thread_continue) 0xc11f8e2c

  4 (f5f7b8f8) .W.ON.(sched_thread_continue) 0  

  5 (f5f7b7b0) .W.ON.(io_done_thread_continue) 0xc1201f74   

  6 (f5f7b668) .W.ON.(net_thread_continue) 0xc11db0a8   

  1 ext2fs (f5f7ce40): 6 threads:   

  0 (f5f7b520) .W.O.F(mach_msg_continue) 0  

  1 (f5f7b290) .W.O.F(mach_msg_receive_continue) 0  

  2 (f5f7b148) .W.O..(mach_msg_receive_continue) 0  

  3 (f5f7b000) .W.O..(mach_msg_continue) 0  

  4 (f67d3e20) .W.O..(mach_msg_receive_continue) 0  

  5 (f67d3cd8) .W.O..(mach_msg_continue) 0  

db> trace/t 0xf5f7b520  

Continuation mach_msg_continue  

> user space <  
> 
0x80ccaec() 

--8<---cut here---end--->8---

For ext2fs.static, that just means thread 0 is here:

--8<---cut here---start->8---
$ addr2line -e 

Re: [VULN 0/4] Hurd vulnerability details

[Ludovic Courtès]

Hi Samuel, Sergey, & all,

Samuel Thibault  skribis:

> Ludovic Courtès, le mar. 09 nov. 2021 18:19:03 +0100, a ecrit:
>> Am I right that the fixes have not been applied yet in the upstream
>> repository?
>
> That's right. That's still waiting for the copyright assignment.

How about making it the first contribution without FSF copyright
assignment?

The delay we observe here might be an outlier in FSF assignment
processing times, but regardless, it’s a good illustration of what a
project can lose by waiting for this long.

Thanks,
Ludo’.

Re: [VULN 0/4] Hurd vulnerability details

[Ludovic Courtès]

Hello,

Samuel Thibault  skribis:

> Thanks a lot for this writing! That'll surely be an interesting read for
> whoever wants to look a bit at the details of how the Hurd works. And of
> course thanks for finding and fixing the vulnerabilities :)

Seconded.  It’s interesting both from a security perspective and as a
deep dive into Mach/Hurd internals; excellent work, Sergey!

Am I right that the fixes have not been applied yet in the upstream
repository?

Thanks,
Ludo’.

Re: Kernel crash with rtl8139 while restarting pfinet

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> Ludovic Courtès, le dim. 24 oct. 2021 15:47:36 +0200, a ecrit:
>> $ addr2line -e 
>> /gnu/store/acl9ffg0pjcj1hvzf8f5pz98xm0cqpps-gnumach-1.8-1.097f9cf/boot/gnumach
>>  0xc10b1f5d 0xc105c6da 0xc1046209 0xc1040d25 0xc104a7a5 0xc1022252 
>> 0xc1052ef9 0xc10071f8
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/./linux/dev/include/asm/string.h:209
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/linux/dev/arch/i386/kernel/irq.c:110
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/i386/i386at/interrupt.S:38
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/device/ds_routines.c:198
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/device/device.server.c:280
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/kern/ipc_kobject.c:178
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/ipc/mach_msg.c:1301
>> /tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/i386/i386/locore.S:1109
>
> That looks like the driver not properly cleaning its IRQ handler.
>
>> Maybe I should just be using netdde in the first place (better have
>> crashy code in user space :-))
>
> Yes, and also because we don't really plan to debug all that legacy
> code.

Yeah, understood.

Perhaps it would make sense to progressively reap most of the linux/
sub-directory in Mach, keep only a few drivers that may be convenient
when debugging things or playing in an emulator?

Ludo’.

Kernel crash with rtl8139 while restarting pfinet

[Ludovic Courtès]

Hello!

I stumbled upon a crash while running GNU/Hurd in QEMU with rtl8139
emulation along these lines:

  qemu-system-i386 -enable-kvm -m 1024 -hda 
/gnu/store/mkvai2a97w702yhayv66y62kd7r2j1ps-disk-image \
-snapshot "--device" "rtl8139,netdev=net0" --netdev user,id=net0

The crash is pretty reproducible:

  1. Start an active pfinet with, say:

   settrans -fac /servers/socket/2 /hurd/pfinet --interface eth0 \
 --ipv6 /servers/socket/26 --address 10.0.2.15 \
 --netmask 255.255.255.0

  2. Forcefully terminate pfinet:

   settrans -fg /servers/socket/2

  3. Restart it as in #1.

After a couple of seconds, I get a kernel debugger prompt:

--8<---cut here---start->8---
Kernel page fault at address 0xc100, eip = 0xc10b1f5d   
Kernel Page fault trap, eip 0xc10b1f5d  
kernel: Page fault (14), code=3 
Stopped at  0xc10b1f5d: repe movsl  (%esi),%es:(%edi)
0xc10b1f5d(b,c11b1a20,c11bdf84,c1029841,7)
0xc105c6da(b)
0xc1046209(f59c7d00,ecea5880,12,8,0,ecc99a38,46,f9249034)
0xc1040d25(f59c7d54,ecea5880,12,8,0,ecc99a38,46,f9249034,f52175c8)
0xc104a7a5(ed077010,f9249010,eef9cf60,c1015b3d,ed077000)
0xc1022252(ed077000,f59ce990,f59b8750,0)
0xc1052ef9(3005de8,3,38,28,1b)
0xc10071f8()
--8<---cut here---end--->8---

… which corresponds to this backtrace:

--8<---cut here---start->8---
$ addr2line -e 
/gnu/store/acl9ffg0pjcj1hvzf8f5pz98xm0cqpps-gnumach-1.8-1.097f9cf/boot/gnumach 
0xc10b1f5d 0xc105c6da 0xc1046209 0xc1040d25 0xc104a7a5 0xc1022252 0xc1052ef9 
0xc10071f8
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/./linux/dev/include/asm/string.h:209
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/linux/dev/arch/i386/kernel/irq.c:110
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/i386/i386at/interrupt.S:38
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/device/ds_routines.c:198
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/device/device.server.c:280
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/kern/ipc_kobject.c:178
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/ipc/mach_msg.c:1301
/tmp/guix-build-gnumach-1.8-1.097f9cf.drv-0/source/i386/i386/locore.S:1109
--8<---cut here---end--->8---

This points at this bit of the old in-kernel Linux drivers:

--8<---cut here---start->8---
/*
 * Generic interrupt handler for Linux devices.
 * Set up a fake `struct pt_regs' then call the real handler.
 */
static void
linux_intr (int irq)
{
  struct pt_regs regs;
  struct linux_action *action = *(irq_action + irq);
  unsigned long flags;

  kstat.interrupts[irq]++;
  intr_count++;

  save_flags (flags);
  if (action && (action->flags & SA_INTERRUPT))
cli ();

  while (action)
{
  action->handler (irq, action->dev_id, );
  action = action->next;   // <- irq.c:110
}

  restore_flags (flags);

  intr_count--;
}
--8<---cut here---end--->8---

Maybe I should just be using netdde in the first place (better have
crashy code in user space :-)), but reporting just in case it rings a
bell or something.

Thoughts?

Ludo’.

Re: Hurd Security vulnerabilities, please upgrade!

[Ludovic Courtès]

Hi Samuel,

Samuel Thibault  skribis:

> Ricardo Wurmus, le mar. 10 août 2021 17:52:34 +0200, a ecrit:
>> I’m a little unclear on what this means for distributions like Guix.  Should
>> we just update to the latest version from git?  Are there specific commits
>> we should use if it’s not just the latest?
>
> Since Sergey's copyright assignment is not complete yet, it's not
> commited yet, so you have to pick up the patches from the debian
> repository.

It would be interesting to consider dropping the copyright assignment
requirement for Hurd/Mach/MiG.  For what remains primarily a hobby
project, this looks to me like a hindrance more than anything else.

Ludo’.

Re: Broken stack traces on crashed programs

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> Ludovic Courtès, le mar. 17 nov. 2020 14:55:32 +0100, a ecrit:
>> Samuel Thibault  skribis:
>> 
>> > Ludovic Courtès, le mar. 17 nov. 2020 10:57:43 +0100, a ecrit:
>> >> I’ve noticed that I’d always get “broken” stack traces in GDB when (1)
>> >> attaching to a program suspended by /servers/crash-suspend, (2)
>> >> examining a core dump, or (3) spawning a program in GDB and examining it
>> >> after it’s received an unhandled signal like SIGILL.
>
> Ah, in all case you mean when receiving an unhandled signal?

Yes.

> I see that happen indeed.

Ah ha!

With a native GDB build¹ I observe the same phenomenon.

>> I get pretty back traces until the program gets an unhandled signal,
>> AFAICT.  This makes me think it could have something to do with how GDB
>> obtains thread state info for suspended threads.
>
> Probably, yes.

I looked around in gdb/*gnu-nat.c though I’m not quite sure what to look
for.

‘set debug gnu-nat on’ gives this right before the inferior, started
from GDB, gets an unhandled SIGILL:

--8<---cut here---start->8---
../../gdb-9.2/gdb/gnu-nat.c:953: {inf 8543 0x88062a0}: running...
../../gdb-9.2/gdb/gnu-nat.c:688: {inf 8543 0x88062a0}: clearing wait
../../gdb-9.2/gdb/gnu-nat.c:1535: {inf 8543 0x88062a0}: waiting for an event...
receiving objects   1% [  
]../../gdb-9.2/gdb/gnu-nat.c:1027: {inf 8543 0x88062a0}: fetching threads
../../gdb-9.2/gdb/gnu-nat.c:927: {inf 8543 0x88062a0}: updating suspend counts
../../gdb-9.2/gdb/gnu-nat.c:273: {proc 8543/-1 0x8806b10}: sc: 0 --> 1
../../gdb-9.2/gdb/gnu-nat.c:312: {proc 8543/-1 0x8806b10}: is suspended
../../gdb-9.2/gdb/gnu-nat.c:312: {proc 8543/4 0x88083d0}: is running
../../gdb-9.2/gdb/gnu-nat.c:312: {proc 8543/5 0x8da2db0}: is running
../../gdb-9.2/gdb/gnu-nat.c:312: {proc 8543/6 0x8d9d9d0}: is running
../../gdb-9.2/gdb/gnu-nat.c:312: {proc 8543/7 0x8ece5b0}: is running
../../gdb-9.2/gdb/gnu-nat.c:953: {inf 8543 0x88062a0}: not running...
../../gdb-9.2/gdb/gnu-nat.c:1564: {inf 8543 0x88062a0}: event: msgid = 24120
../../gdb-9.2/gdb/gnu-nat.c:1827: {inf 8543 0x88062a0}: err = 0, pid = 8543, 
status = 0x47f, sigcode = 0
../../gdb-9.2/gdb/gnu-nat.c:1842: {inf 8543 0x88062a0}: waits pending now: 0
../../gdb-9.2/gdb/gnu-nat.c:1863: {inf 8543 0x88062a0}: process has stopped 
itself
../../gdb-9.2/gdb/gnu-nat.c:1027: {inf 8543 0x88062a0}: fetching threads
../../gdb-9.2/gdb/gnu-nat.c:1653: {inf 8543 0x88062a0}: returning ptid = Thread 
8543.4, status->kind = stopped, signal = GDB_SIGNAL_ILL
../../gdb-9.2/gdb/gnu-nat.c:374: {proc 8543/4 0x88083d0}: updating state info
../../gdb-9.2/gdb/gnu-nat.c:356: {proc 8543/4 0x88083d0}: aborted
../../gdb-9.2/gdb/gnu-nat.c:389: {proc 8543/4 0x88083d0}: getting thread state
../../gdb-9.2/gdb/i386-gnu-nat.c:149: {proc 8543/4 0x88083d0}: fetching 
register eip

Thread 4 received signal SIGILL, Illegal instruction.
../../gdb-9.2/gdb/gnu-nat.c:2525: {inf 8543 0x88062a0}: writing 0x11270[1] <-- 
0x881d5d8
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x810be88[1] 
--> 0x280479f
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x810be88[1] 
--> 0x280479b
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x810be80[64] 
--> 0x8f0f710
0x0810be88 in ?? ()
(gdb) bt
#0  0x0810be88 in ?? ()
../../gdb-9.2/gdb/gnu-nat.c:374: {proc 8543/4 0x88083d0}: updating state info
../../gdb-9.2/gdb/i386-gnu-nat.c:149: {proc 8543/4 0x88083d0}: fetching 
register ebp
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x813b480[64] 
--> 0x95328f0
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x0[64] --> 
0x8f60410
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x0[1] --> 
0x280478c
#1  0x in ?? ()
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x813b4c0[64] 
--> 0x8f60410
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x0[64] --> 
0x960e1b0
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x1[1] --> 
0x28047cc
(gdb) thread 5
[Switching to thread 5 (Thread 8543.5)]
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}: reading 0x123582c[1] 
--> 0x280468f
#0  0x0123582c in mach_msg_trap () at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S:2
2   
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S: 
No such file or directory.
(gdb) bt
#0  0x0123582c in mach_msg_trap () at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S:2
../../gdb-9.2/gdb/gnu-nat.c:374: {proc 8543/5 0x8da2db0}: updating state info
../../gdb-9.2/gdb/i386-gnu-nat.c:149: {proc 8543/5 0x8da2db0}: fetching 
register esp
../../gdb-9.2/gdb/gnu-nat.c:2532: {inf 8543 0x88062a0}:

Re: Broken stack traces on crashed programs

[Ludovic Courtès]

Hi!

Samuel Thibault  skribis:

> Ludovic Courtès, le mar. 17 nov. 2020 10:57:43 +0100, a ecrit:
>> I’ve noticed that I’d always get “broken” stack traces in GDB when (1)
>> attaching to a program suspended by /servers/crash-suspend, (2)
>> examining a core dump, or (3) spawning a program in GDB and examining it
>> after it’s received an unhandled signal like SIGILL.
>> 
>> At best I can see the backtrace of the msg thread, the other ones are
>> all question-marky:
>
> Silly question, but still important to ask: did you build with -g?

Yes.

I get pretty back traces until the program gets an unhandled signal,
AFAICT.  This makes me think it could have something to do with how GDB
obtains thread state info for suspended threads.

> (meaning: no, I don't have such kind of issue with gdb 9.2 in debian)

(Same with GDB 10.1.)

It could be that we’re missing a libc patch that Debian has, or (more
likely) that we’re miscompiling something on the way (this is all
cross-compiled from x86_64-linux-gnu).

To be continued…

Ludo’.

Broken stack traces on crashed programs

[Ludovic Courtès]

Hello!

I’ve noticed that I’d always get “broken” stack traces in GDB when (1)
attaching to a program suspended by /servers/crash-suspend, (2)
examining a core dump, or (3) spawning a program in GDB and examining it
after it’s received an unhandled signal like SIGILL.

At best I can see the backtrace of the msg thread, the other ones are
all question-marky:

--8<---cut here---start->8---
(gdb) thread 1
[Switching to thread 1 (process 310)]
#0  0x080f08c0 in ?? ()
(gdb) bt
#0  0x080f08c0 in ?? ()
#1  0x in ?? ()
(gdb) thread 2
[Switching to thread 2 (process 1)]
#0  0x0159282c in mach_msg_trap () at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S:2
2   
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S: 
No such file or directory.
(gdb) bt
#0  0x0159282c in mach_msg_trap () at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach_msg_trap.S:2
#1  0x01592f2a in __GI___mach_msg (msg=0x2802aa0, option=3, send_size=96, 
rcv_size=32, rcv_name=109, timeout=0, notify=0) at msg.c:111
#2  0x017dc8ab in __crash_dump_task (crashserver=132, task=1, file=133, 
signo=11, sigcode=2, sigerror=2, exc=1, code=2, subcode=210986494, 
cttyid_port=102, cttyid_portPoly=19)
at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/hurd/RPC_crash_dump_task.c:254
#3  0x015b248c in write_corefile (detail=, signo=) at hurdsig.c:296
#4  post_signal (untraced=) at hurdsig.c:947
#5  0x015b274b in _hurd_internal_post_signal (ss=0x1800808, signo=11, 
detail=0x2802e5c, reply_port=0, reply_port_type=17, untraced=0) at 
hurdsig.c:1235
#6  0x015b3fc1 in _S_catch_exception_raise (port=96, thread=39, task=1, 
exception=1, code=2, subcode=210986494) at catch-exc.c:88
#7  0x017c09b4 in _Xexception_raise (InHeadP=0x2802f20, OutHeadP=0x2803f30) at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach/exc_server.c:155
#8  0x017c0a52 in _S_exc_server (InHeadP=0x2802f20, OutHeadP=0x2803f30) at 
/tmp/guix-build-glibc-cross-i586-pc-gnu-2.31.drv-0/build/mach/mach/exc_server.c:208
#9  0x015a7a09 in msgport_server (inp=0x2802f20, outp=0x2803f30) at 
msgportdemux.c:49
#10 0x015934c3 in __mach_msg_server_timeout (demux=0x15a79b0 , 
max_size=4096, rcv_name=96, option=0, timeout=0) at msgserver.c:108
#11 0x01593607 in __mach_msg_server (demux=0x15a79b0 , 
max_size=4096, rcv_name=96) at msgserver.c:195
#12 0x015a7a86 in _hurd_msgport_receive () at msgportdemux.c:67
#13 0x011eda50 in entry_point (self=0x804ac20, start_routine=0x15a7a30 
<_hurd_msgport_receive>, arg=0x0) at pt-create.c:62
#14 0x in ?? ()
--8<---cut here---end--->8---

(This is on Guix System with GDB 9.2.)

Does that ring a bell?

Thanks,
Ludo’.

Re: [PATCH] hurd: '_hurd_raise_signal' checks signal number is valid

[Ludovic Courtès]

Andreas Schwab  skribis:

> https://sourceware.org/git/?p=glibc.git;a=commit;h=785ec62dbd

Thanks, I guess I was looking at a stale checkout, apologies!

Ludo’.

[PATCH] hurd: '_hurd_raise_signal' checks signal number is valid

[Ludovic Courtès]

Previously, 'pthread_kill (pthread_self (), -1)' would wrongfully
succeed:

  https://lists.gnu.org/archive/html/guix-devel/2020-10/msg00152.html

Reported-by: Jan Nieuwenhuizen 
---
 hurd/hurd-raise.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/hurd/hurd-raise.c b/hurd/hurd-raise.c
index d53fd417d2..58b4ffcc83 100644
--- a/hurd/hurd-raise.c
+++ b/hurd/hurd-raise.c
@@ -28,6 +28,9 @@ int
 _hurd_raise_signal (struct hurd_sigstate *ss,
int signo, const struct hurd_signal_detail *detail)
 {
+  if (signo < 1 || signo >= _NSIG)
+return EINVAL;
+
   if (ss == NULL)
 {
   ss = _hurd_self_sigstate ();
-- 
2.28.0

Re: raise(-1) succeeds for programs linked against libpthread

[Ludovic Courtès]

Samuel Thibault  skribis:

> Ludovic Courtès, le mar. 13 oct. 2020 15:41:37 +0200, a ecrit:
>> ‘pthread_kill’ passes the signal number to ‘_hurd_raise_signal’, which
>> assumes it is valid:
> [...]
>> I suppose that before calling ‘sigaddset’, it should check whether SIGNO
>> is within bounds, along the lines of:
>> 
>>   if (signo < 2 || signo >= _NSIG)
>> return EINVAL;
>> 
>> Does that make sense?
>
> Probably, yes. Why excluding SIGHUP?

Oops, an oversight: I was looking at ‘signum-generic.h’ and the first
definition is SIGINT (2).  :-)

Ludo’.

raise(-1) succeeds for programs linked against libpthread

[Ludovic Courtès]

Hi!

(Cc: bug-hurd.)

Jan Nieuwenhuizen  skribis:

>>> #include 
>>>
>>> int
>>> main (void)
>>> {
>>>   if (!raise (-1))
>>> return 1;
>>>   
>>>   return 0;
>>> }
>>
>> I don’t know if it’s relevant here, but you should always use ‘-pthread’
>> both at compile time and link time:
>>
>>   gcc raise.c -pthread
>>
>> That typically defines a few macros that may or may not have an effect
>> on the code at hand.
>
> Ah...right.  Makes no difference, though:
>
> root@childhurd ~# guix environment --bootstrap --ad-hoc gcc-toolchain@7
> root@childhurd ~ [env]# gcc raise.c
> root@childhurd ~ [env]# ./a.out
> root@childhurd ~ [env]# echo $?
> 0
> root@childhurd ~ [env]# gcc raise.c -pthread
> root@childhurd ~ [env]# ./a.out
> User defined signal 2

Interesting!  In the second case, we’re using ‘__pthread_kill’ from
‘pt-kill.c’ (instead of ‘kill’).

The expected behavior is that ‘raise’ should return non-zero and EINVAL.

‘pthread_kill’ passes the signal number to ‘_hurd_raise_signal’, which
assumes it is valid:

--8<---cut here---start->8---
int
_hurd_raise_signal (struct hurd_sigstate *ss,
int signo, const struct hurd_signal_detail *detail)
{
  if (ss == NULL)
{
  ss = _hurd_self_sigstate ();
  __spin_lock (>lock);
}

  /* Mark SIGNO as pending to be delivered.  */
  __sigaddset (>pending, signo);
  ss->pending_data[signo] = *detail;
--8<---cut here---end--->8---

I suppose that before calling ‘sigaddset’, it should check whether SIGNO
is within bounds, along the lines of:

  if (signo < 2 || signo >= _NSIG)
return EINVAL;

Does that make sense?

Thanks,
Ludo’.

Re: exec hang at boot.

[Ludovic Courtès]

Hi Almudena,

Almudena Garcia  skribis:

> In Qemu, the boot looks fine, and the exec problems seem to have
> disappeared. I tested It using gnumach in SMP mode (--enable-cpus=2).

The new “childhurd” feature in Guix makes it easy (and safe) to test the
kernel in QEMU.  What would you suggest for us to test your branch
(branch or commit, configure flags, etc.)?

Thanks for the hard work!

Ludo’.

Re: [BLOG] A "Hello World" VM running the Hurd!

[Ludovic Courtès]

Hi Samuel,

Samuel Thibault  skribis:

> Jan Nieuwenhuizen, le mer. 08 avril 2020 22:02:25 +0200, a ecrit:
>> Using Guix we have cross-built a VM image for the Hurd.  Read more...
>> 
>> 
>> https://guix.gnu.org/blog/2020/a-hello-world-virtual-machine-running-the-hurd/
>
> It would be nice to have this in a CI, and report when it breaks, so we
> catch that as early as possible when pushing stuff upstream :)

Someone did that a while back:

  https://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00042.html

Time flies!  :-)

I agree that it would be nice to revive this, with a Guix setup this time.

Cheers,
Ludo’.

Re: readlink("/proc/self/exe") with glibc 2.28 is broken?

[Ludovic Courtès]

Samuel Thibault  skribis:

> Samuel Thibault, le dim. 16 déc. 2018 16:53:08 +0100, a ecrit:
>> Ludovic Courtès, le dim. 16 déc. 2018 16:44:54 +0100, a ecrit:
>> > What is the glibc test you were referring to?
>> 
>> make check
>
> I have now committed it.

Awesome, thank you!

Ludo’.

Re: readlink("/proc/self/exe") with glibc 2.28 is broken?

[Ludovic Courtès]

Hi Samuel,

Samuel Thibault  skribis:

> Ludovic Courtès, le sam. 15 déc. 2018 23:10:39 +0100, a ecrit:
>> Any particular reason why this patch hasn’t made it upstream?
>
> Most probably the answer is "nobody took the time to polish the patch
> and run the glibc test with it and say s⋅he did".

OK, understood.

FWIW I applied the two patches from ‘t/magic-pid’ in Guix and it works
well AFAICS:

  
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=92391eaf8df8fecbf6844c57ce5bcd3014eb5d28

What is the glibc test you were referring to?

Thank you,
Ludo’.

Re: readlink("/proc/self/exe") with glibc 2.28 is broken?

[Ludovic Courtès]

Ludovic Courtès  skribis:

>   137<--176(pid8674)->dir_lookup ("proc/self/exe" 65 0) = 0 1 "self/exe"
> 191<--190(pid8674)
>   191<--190(pid8674)->dir_lookup ("self/exe" 65 0) = 0 3 "pid/exe"  (null)

Looking more closely, I found that we’re missing FS_RETRY_MAGICAL
support for “pid” in hurd/lookup-retry.c, which the ‘t/magic-pid’ branch
on hurd/glibc.git provides.  I’ll integrate it in Guix.

Any particular reason why this patch hasn’t made it upstream?

Ludo’.

readlink("/proc/self/exe") with glibc 2.28 is broken?

[Ludovic Courtès]

Hello hacker herd!

The attached C code reads /proc/self/exe.  I tested it on darnassus in
two contexts:

  1. Natively compiled with the Debian toolchain available on darnassus;

  2. Cross-built with the GNU toolchain (which includes stock
 glibc 2.28) found in Guix, statically-linked.

#1 works as expected whereas for #2 readlink("/proc/self/exe") returns
EGRATUITOUS; #2 works for other symlinks though.

rpctrace shows that Guix’s libc doesn’t behave like Debian’s on
darnassus.  Namely, Guix libc 2.28 does this:

--8<---cut here---start->8---
  137<--176(pid8674)->dir_lookup ("proc/self/exe" 65 0) = 0 1 "self/exe"
191<--190(pid8674)
  191<--190(pid8674)->dir_lookup ("self/exe" 65 0) = 0 3 "pid/exe"  (null)
task163(pid8674)->mach_port_deallocate (pn{ 20}) = 0 
task163(pid8674)->mach_port_mod_refs (pn{  7} 0 1) = 0 
task163(pid8674)->mach_port_mod_refs (pn{ 19} 0 1) = 0 
  124<--174(pid8674)->io_get_openmodes () = 0 3
  124<--174(pid8674)->io_stat () = 0 {14 21405 0 0 0 1437067240 0 8397200 1 
1006 5 0 0 1544895840 0 1544895840 0 1544895840 0 512 8 0 0 0 136648208 3259 0 
0 1315456 3261 136650912 3262}
  189<--188(pid8674)->io_write ("readlink: Gratuitous error\n" -1)readlink: 
Gratuitous error
 = 0 27
--8<---cut here---end--->8---

… whereas Debian’s libc does:

--8<---cut here---start->8---
  137<--176(pid8678)->dir_lookup ("proc/self/exe" 65 0) = 0 1 "self/exe"
192<--191(pid8678)
task163(pid8678)->mach_port_mod_refs (pn{ 21} 0 1) = 0 
  192<--191(pid8678)->dir_lookup ("self/exe" 65 0) = 0 3 "pid/exe"  (null)
task163(pid8678)->mach_port_deallocate (pn{ 21}) = 0 
task163(pid8678)->mach_port_mod_refs (pn{ 21} 0 1) = 0 
  192<--191(pid8678)->dir_lookup ("8678/exe" 65 0) = 0 1 ""
194<--193(pid8678)
task163(pid8678)->mach_port_deallocate (pn{ 21}) = 0 
task163(pid8678)->mach_port_deallocate (pn{ 21}) = 0 
  194<--193(pid8678)->io_stat () = 0 {0 0 0 -395048269 0 0 0 41471 0 1006 0 18 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0}
  194<--193(pid8678)->io_read (0 256) = 0 "./cross-built.dynl"
task163(pid8678)->mach_port_deallocate (pn{ 22}) = 0 
  139<--173(pid8678)->io_stat () = 0 {14 21405 0 0 0 1437067240 0 8397200 1 
1006 5 0 0 1544895879 0 1544895879 0 1544895879 0 512 8 0 0 0 136648208 3259 0 
0 1315456 3261 136650912 3262}
  188<--166(pid8678)->io_write ("readlink ("/proc/self/exe") = 
"./cross-built.dynl"\n" -1)readlink ("/proc/self/exe") = "./cross-built.dynl"
 = 0 51
--8<---cut here---end--->8---

So Debian libc (presumably ‘file_name_lookup’) is doing more work, which
proves to be useful.  :-)

Am I missing anything obvious to you?  For example, are there patches
that Debian uses and that upstream 2.28 still lacks?

Thanks in advance!

Ludo’.

#define _GNU_SOURCE 1
#include 
#include 
#include 
#include 
#include 

int
main (int argc, char *argv[])
{
  ssize_t ret;
  char program[256] = { 0, };
  const char *arg = (argc > 1) ? argv[1] : "/proc/self/exe";

  ret = readlink (arg, program, sizeof program);
  if (ret < 0)
perror ("readlink");

  printf ("readlink (\"%s\") = \"%s\"\n", arg, program);

  return 0;
}

Re: Meta Learning the Hurd Stuff

[Ludovic Courtès]

Hi Joshua,

Joshua Branson  skribis:

> So yesterday, I decided to write a GNU/Hurd translator in gnu guile.

I think this is a worthy goal (Guile bindings would be lovely!), but
maybe not the easiest way to get started since there are currently no
Guile bindings to write Hurd translators.

For a start, I would recommend either writing a simple translator in C
with libtrivfs, or trying out the Common Lisp bindings that were written
a while back as part of GSoC: .

Hope this helps!

Ludo’.

Re: Fwd: The GNU C Library version 2.28 is now available

[Ludovic Courtès]

Hello!

Richard Braun  skribis:

> On Wed, Aug 01, 2018 at 09:53:59AM +0200, Samuel Thibault wrote:
>> NEWS for version 2.28
>> =
>> 
>> [...]
>> 
>> * Building and running on GNU/Hurd systems now works without out-of-tree
>>   patches.
>
> I didn't think it would actually happen. Great work :).

Indeed, a few months back it looked like something we wouldn’t see in
our lifetime.  ;-)

Thumbs up to everyone who contributed and in particular to Samuel and
Joseph for tirelessly going through all these patches!

Ludo’.

Re: About GCC-5.5.0

[Ludovic Courtès]

Hello Rene,

Rene  skribis:

> configure:6227: checking whether i586-pc-gnu-gcc implicitly enables 
> -fstack-protector
> configure:6244: i586-pc-gnu-gcc -c -g -O2  conftest.c >&5
> configure:6244: $? = 0
> libc_undefs=''
> configure:6263: error: unexpected symbols in test: 

This comes from this code in the configure.ac file of the glibc 2.23ish
package that’s used in Guix for GNU/Hurd (this code is absent in the
2.26ish glibc used for GNU/Linux):

--8<---cut here---start->8---
AC_CACHE_CHECK([whether $CC implicitly enables -fstack-protector],
   libc_cv_predef_stack_protector, [
AC_TRY_COMPILE([extern void foobar (char *);],
   [char large_array[2048]; foobar (large_array);], [
libc_undefs=`$NM -u conftest.o |
  LC_ALL=C $AWK '$1 == "U" { print $2 | "sort -u"; next } { exit(1) }' \
2>_MESSAGE_LOG_FD` || {
  AC_MSG_ERROR([confusing output from $NM -u])
}
echo >_MESSAGE_LOG_FD "libc_undefs='$libc_undefs'"
# On some architectures, there are architecture-specific undefined
# symbols (resolved by the linker), so filter out unknown symbols.
# This will fail to produce the correct result if the compiler
# defaults to -fstack-protector but this produces an undefined symbol
# other than __stack_chk_fail.  However, compilers like that have not
# been encountered in practice.
libc_undefs=`echo "$libc_undefs" | egrep '^(foobar|__stack_chk_fail)$'`
case "$libc_undefs" in
foobar) libc_cv_predef_stack_protector=no ;;
'__stack_chk_fail
foobar') libc_cv_predef_stack_protector=yes ;;
*) AC_MSG_ERROR([unexpected symbols in test: $libc_undefs]) ;;
esac],
   [AC_MSG_ERROR([test compilation failed])])
])
--8<---cut here---end--->8---

So it looks like the right-hand size of the ‘libc_undefs’ assignment
expands to the empty string.

Could you check the value of $NM in there?  Could it be that it got it
wrong?

HTH,
Ludo’.

Re: Add a new exec_exec_file_name RPC

[Ludovic Courtès]

Hello!

Samuel Thibault  skribis:

> For non-Debian distributions, you may want
> to pick up the hurd exec_filename_ patches from
> https://anonscm.debian.org/git/pkg-hurd/hurd.git/tree/debian/patches
> and the local-exec_filename.diff patch from
> https://anonscm.debian.org/cgit/pkg-glibc/glibc.git/tree/debian/patches/hurd-i386
> at least for some time to get backward compatibility with the previous
> experimental RPCs.

Are you planning to commit them upstream?  That’d be nice.

Also, now that you’re a glibc maintainer for the Hurd part, what would
you think of doing a “mass commit” (with a lightweight re-review
process) of all the patches that are in hurd/glibc.git?

Ludo’.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

rennes  skribis:

>   1) Manually run :
>   
> '/gnu/store/6fgz3s8fjva40hsdvs2hs0f5p4bw12jc-guile-static-stripped-2.2.2/bin/guile
>   
> --version'
>
>   Output:
>
> guile: warning: failed to install locale
> ..
> Uncaught exception:
> Throw to key misc-error with args ("primitive-load-path" "Unable to  
> find file ~S in load path" ("ice-9/boot-9") #f)Can\
> not exit gracefully when init is in progress; aborting.
> Aborted

This must come from the lack of /proc/self/exe.

Specifically, “guile-static-stripped” has this patch to make it
relocatable:

  
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch

This is necessary for the bootstrap Guile (see make-bootstrap.scm), and
was necessary in the initrd on GNU/Linux (I think we no longer need it
for the initrd).

At any rate, you may be able to boot the Hurd with the “real”,
dynamically-linked Guile, no?  That would solve the problem entirely.

HTH,
Ludo’.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

ren...@openmailbox.org skribis:

> This is the demo generated with Guix:
>
> https://github.com/methalo/boot-hurd
>
> The binary files were generated in Debian/Hurd and placed in an 'img' file.
>
> The command used to generate the binaries is:
>
> './pre-inst-env guix system init ~/light.scm /guix'
>
> To test Hurd, execute:
>
> 'sudo qemu-system-i386 -enable-kvm -m 1G -hda guixsdhurd.img -curses'

[...]

> https://ombx.io/ipoWt9uK

I just gave it a try, and woow!  :-)

It’s really nice to see that in action.

It lacks a couple of things such as the console server and client and
the pipe server, but tweaking this will be the funny part.  ;-)

Also, in GRUB, you currently load ext2fs.static and exec explicitly.
There’s now a /hurd/startup server that takes care of launching
/hurd/proc, /hurd/auth, and then passes control to /libexec/runsystem.
I suppose this is the preferred method, but hurd.texi doesn’t give the
exact GRUB commands.  Can anyone shed some light?

BTW, the image you posted is in “raw” format.  You would get a smaller
file by using the qcow2 format, which you can create with “qemu-img
create -f qcow2”.

Anyway, kudos, and keep up the good work!

Thank you,
Ludo’.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi Samuel,  :-)

Samuel Thibault <samuel.thiba...@gnu.org> skribis:

> Ludovic Courtès, on lun. 13 nov. 2017 11:42:01 +0100, wrote:
>> PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.
>
> Does our procfs have everything it needs already?

It needs /proc/PID/{exe,cwd,fd,maps,environ}:

  
https://git.savannah.gnu.org/cgit/guix.git/tree/nix/scripts/list-runtime-roots.in

That said, it’s quite optional: the worst that can happen if one of
these things is missing is that something can be reclaimed “prematurely”
from /gnu/store.  I say “prematurely” with quotes because users are
supposed to register “real” GC roots anyway, as with ‘guix package -i’
or ‘guix build --root’.

Thanks,
Ludo’.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

(Something went wrong with your message headers.)

ren...@openmailbox.org skribis:

>>> Finally I was able to start the Hurd with the binaries generated with the 
>>> guix package manager.
>> 
>> Woohoo!  Does that mean you were able to run packages cross-compiled
>> with Guix, or packages built natively with Guix?
>> 
>
> the packages were built on top of Debian/Hurd. At this moment I am copying 
> everything that is compiled into /gnu/store to the image, 
> and then create symbolic links so that grub, Mach and Hurd can locate the 
> binaries.

Neat.

>>> At the moment the image of Hurd I have built it manually and does not yet 
>>> have any functionality like GuixSD / Linux.
>> 
>> … or did you build a VM image of GNU/Hurd?
>> 
>
> Yes, I have built the VM image according to your notes [1].

Oh, awesome!  I can’t wait to fiddle with this image!

Please make sure to send us a recap of what needs to be done in Guix
proper to make this happen.  We’ll think about ways to incorporate those
changes.

Congrats!

Ludo’.

PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes!

ren...@openmailbox.org skribis:

> Finally I was able to start the Hurd with the binaries generated with the 
> guix package manager.

Woohoo!  Does that mean you were able to run packages cross-compiled
with Guix, or packages built natively with Guix?

> At the moment the image of Hurd I have built it manually and does not yet 
> have any functionality like GuixSD / Linux.

… or did you build a VM image of GNU/Hurd?

> Next goal:
>   * Generate the image file in the style of guix.
>   ** For this I have noticed that it is required to compile qemu for the 
> architecture i586-pc-gnu, at the moment I could not compile qemu in Hurd. I 
> have tried to disable some quemu features that come by default; however, I 
> have not yet achieved it.

Things like ‘guix system vm’ use QEMU because we need to start a VM
because on GNU/Linux one needs to be root to create a file system, mount
it, and populate it.

However, on GNU/Hurd, file systems can be mounted without being root.
So you could in fact skip QEMU altogether, though that’d require
tweaking in the VM-generation code (in (gnu build vm), etc.).

> Other pending things:
>   * Some packages fail during the compilation and testing phases.
>   * Other packages used by guix, such as lsof are not ported in Hurd yet.

We could probably get rid the lsof dependency.  What matters here is a
Linuxish /proc.

Anyway, congrats on this milestone!  :-)

Ludo’.

Re: Documentation severely outdate on gnu.org

[Ludovic Courtès]

Samuel Thibault <samuel.thiba...@gnu.org> skribis:

> Ludovic Courtès, on dim. 05 nov. 2017 18:37:06 +0100, wrote:
>> Samuel Thibault <samuel.thiba...@gnu.org> skribis:
>> > People find
>> > https://www.gnu.org/software/hurd/users-guide/using_gnuhurd.html#dir
>> > as documentation for the Hurd, which is just antique. Does anybody know
>> > how to update this?
>> 
>> See the bits about “Webpages repository” at
>> <https://savannah.gnu.org/cvs/?group=hurd>.  Essentially you need to
>> commit HTML pages to CVS (!) and they are eventually updated on gnu.org.
>> 
>> To generate Texinfo manual web pages similar to
>> <https://gnu.org/s/guix/manual>, you can use the gendocs.sh script from
>> Gnulib, which is quite convenient.
>
> Actually there is a Makefile to generate the files.
>
> However, it seems using_gnuhurd.texi is not the same as hurd.texi: there
> is a lot in common, but there are also large part which are not the
> same, so some merging would be needed...

I suppose you could remove traces of using_gnuhurd from the web pages
and publish hurd.texi instead?

Ludo’.

Re: Documentation severely outdate on gnu.org

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> People find
> https://www.gnu.org/software/hurd/users-guide/using_gnuhurd.html#dir
> as documentation for the Hurd, which is just antique. Does anybody know
> how to update this?

See the bits about “Webpages repository” at
.  Essentially you need to
commit HTML pages to CVS (!) and they are eventually updated on gnu.org.

To generate Texinfo manual web pages similar to
, you can use the gendocs.sh script from
Gnulib, which is quite convenient.

HTH,
Ludo’.

Re: guile 2.0.14 assertion failure

[Ludovic Courtès]

Howdy!

Samuel Thibault  skribis:

> Manolis Ragkousis, on mar. 14 mars 2017 20:22:44 +0200, wrote:
>> "scheme@(repl-server)> While reading expression:\nERROR: In procedure
>> fport_fill_input: Resource temporarily
>> unavailable\nscheme@(repl-server)> While reading expression:\nERROR: In
>> procedure fport_fill_input: Resource temporarily
>
> Having a look at the resources, it looks like the test is just looping
> eating more and more resources, until there is just none any more :)

Do you mean that the __pthread_create_internal assertion failure is a
consequence of this, or its cause?  (I would think the latter.)

Ludo’.

Re: Porting with Guix

[Ludovic Courtès]

ren...@openmailbox.org skribis:

> make[3]: Entering directory '/home/buzz/gits/D/bdwgc'
> PASS: cordtest
> hurd: Can't add reference on Mach thread

FWIW this comes from the Hurd signal handling code in glibc:

  err = __mach_port_mod_refs (__mach_task_self (), thread,
  MACH_PORT_RIGHT_SEND, 1);
  if (err)
__libc_fatal ("hurd: Can't add reference on Mach thread\n");

Ludo’.

Re: Porting with Guix

[Ludovic Courtès]

Ricardo Wurmus  skribis:

> ren...@openmailbox.org writes:
>
>> Now there are two errors related to the 'Check' phase of libgc and 
>> guile:
>> In libgc shows:
>
> […]
>
>>   building of 
>> `/gnu/store/y3icscjhkk7pa7dg21xqy46riysi36rn-libgc-7.6.0.drv' timed
>>   out after 3600 seconds of silence
>
> You can increase the timeout:

There may be a genuine deadlock though.  :-)

Ludo’.

Re: rpctrace man page

[Ludovic Courtès]

Hello!

Thanks for taking care of the manual!  Just a very very partial review…

"Brent W. Baccala"  skribis:

> +@node rpctrace
> +@section rpctrace

The convention would be to call this section “Invoking rpctrace”, so:

  @node Invoking rpctrace
  @section Invoking @command{rpctrace}

> +@example
> +rpctrace [-E var[=value]] [-i FILE] [-I DIR] [--nostdinc] [-o FILE] [-s 
> SIZE] command [args]

Perhaps simply:

@example
rpctrace @var{options} @var{command} @var{args}...
@end example

> +@item -E @var{var[=value]}

@item -E @var{var}[=@var{value}]

My .2¢,
Ludo’.

Re: Time for another round of releases

[Ludovic Courtès]

Samuel Thibault <samuel.thiba...@gnu.org> skribis:

> Ludovic Courtès, on Tue 04 Oct 2016 15:22:32 +0200, wrote:
>> > Yes, but our current glibc tree is based on glibc 2.22, as guix needs
>> > it.
>> 
>> FWIW Guix follows upstream glibc releases for GNU/Linux (we’re currently
>> in the process of switching from 2.23 to 2.24), and we’d prefer to have
>> the same version on both GNU/Linux and GNU/Hurd, modulo the
>> Hurd-specific patches.
>
> So it's now 2.23.  When Guix switched to 2.24 for Linux, please tell me,
> I'll update the tree to 2.24.

It’s already done in a branch that is being built (hopefully merged
within a week or so).

So you could go ahead and just let us know which tarballs or commits we
should be using for each version.  Is that fine with you?  Manolis,
WDYT?

Thanks!

Ludo’.

Re: Time for another round of releases

[Ludovic Courtès]

Hi,

Samuel Thibault  skribis:

> Svante Signell, on Tue 04 Oct 2016 12:27:12 +0200, wrote:
>> On Sun, 2016-10-02 at 18:54 +0200, Justus Winter wrote:
>> > it is October, therefore, it is time for a new set of releases :)
>> > 
>> > I'll be going over the changes and update the NEWS files as usual, but
>> > feel free to beat me to that.  Also, if anyone has some pet patches or
>> > fixes that would be nice to include, feel free to speak up or send
>> > patches.
>> > 
>> > Personally, I'd love to include Shengyu Zhang's work on xattrs, so I'm
>> > going to see whether I can weed out the remaining issues in time.  Also,
>> > I have a cleanup patch for Mach.
>> > 
>> > I'm afraid there haven't been any commits for MIG.  Does anyone have a
>> > patch for it?  If not, are we going to make an release anyway, to keep
>> > the version numbers in sync with GNU Mach?
>> 
>> Is it possible to make a release of Hurd until mlockall/munlockall is
>> implemented? From what I understand these functions are needed to build Hurd 
>> on
>> top of glibc2.24.
>
> Yes, but our current glibc tree is based on glibc 2.22, as guix needs
> it.

FWIW Guix follows upstream glibc releases for GNU/Linux (we’re currently
in the process of switching from 2.23 to 2.24), and we’d prefer to have
the same version on both GNU/Linux and GNU/Hurd, modulo the
Hurd-specific patches.

Ludo’.

Re: [GSoC] Porting GuixSD to GNU/Hurd Update

[Ludovic Courtès]

Hello Manolis,

Manolis Ragkousis  skribis:

> On Hurd though, because of the lack of mount(), the above could not
> work. But with the help of my libhurdutil library, which I wrote at the
> beginning of this project, I created 2 wrappers inside
> nix/libutil/call.(hh.cc) for mount() and umount2(), called nixMount()
> and nixUmount2(), and depending on the system the implementation
> changes. On Hurd I am using /hurd/firmlink to offer the same
> functionality to bind-mounts.
>
> It seems to work but I am testing it thoroughly so we don't have any
> unpleasant surprises in the future. We will have fully isolated builds
> on Hurd as soon as I am sure that my code works as expected and it's
> merged in Guix upstream.

Neat!  You did the right thing.

> Now on the GuixSD side, I will start on a big issue I had. If you check
> the daemon, and specifically nix/libstore/build.cc:2205-2228, you will
> see that when, for example, guix tries to create a 32 bit vm/image from
> a 64 bit machine, the daemon actually builds the 32 bit binaries on 32
> bit personality mode.

Indeed; hopefully we can address that.

> As a result it is not possible to build GuixSD/Hurd vm/images from
> Linux, for now. But this is not a big problem because we can do it from
> Guix running on Hurd :-). The approach I used, was to add a second hard
> drive on my Hurd vm, mount it, and try to directly deploy a GuixSD
> system on that drive.  Currently Guix can build most of the binaries for
> the system but it still needs work to actually boot into one. You can
> see the result of this work on the currect core-updates (and on my
> github repo for some hacky commits).

Woow!  We knew booting GuixSD on GNU/Hurd was an ambitious goal, so what
you describe is already impressive.

> I have also created a new module called (guix build syscalls-tools)
> which contains some of the code from (guix build syscalls) which will be
> used by a (guix build hurd) module, which will contain call wrappers for
> some Hurd libraries. This work is still in my github repo because it
> needs some work.

Nice stuff again.

> There was one more problem that appeared after we started using
> C_INCLUDE_PATH in our cross-builds. As it seems MiG needs glibc in order
> to be built.  That's why for now I patch cross-mig with the
> glibc-headers so I don't have to depend on the Linux ones. But I will
> talk more on this in a different email.
>
> I think that's enough for now. I avoid talking about things discussed in
> other mails, but if you want please feel free to ask me :-)
>
> To sum things up, we almost have build isolation working but GuixSD
> still needs some work to become bootable. From here on I will
> finish/test my guix-daemon code, and then I will continue on finishing
> the low-level call wrappers for the Hurd libraries, in order to get the
> bootable system. We are close :-).
>
> The repos which you can check any code not yet in upstream Guix or Hurd are:
> https://github.com/Phant0mas/Guix-on-Hurd
> https://github.com/Phant0mas/Hurd

Now we need to chew that, both on the Guix and Hurd sides.  Make sure to
ping us with sizable chunks of this code!  :-)

This is again an impressive piece of work, especially given its breadth:
From cross-compilation toolchain issues, to low-level Linux- and
Hurd-specific programming, and to high-level Scheme hacking.  Kudos!

Thank you Manolis!

Ludo’.


signature.asc
Description: PGP signature

Re: [PATCH hurd 2/2] libshouldbeinlibc: add safe port handling macros

[Ludovic Courtès]

Thomas Schwinge  skribis:

> The next thought then occurred to me: why not use a programming language
> that allows for defining additional types, powerful enough to model the
> desired semantics?  For example, if we'd compile the Hurd with a C++
> compiler (which, hopefully, will just work -- mostly), could we then
> create a mach_port_ta C++ class, wrapping the "primitive" Mach data type,
> and teach that one the Mach API semantics?

Indeed, for automatic variables, C++ destructors are the same thing as
the ‘cleanup’ attribute, so that would work too.

Ludo’.

Re: [PATCH hurd 2/2] libshouldbeinlibc: add safe port handling macros

[Ludovic Courtès]

Samuel Thibault <samuel.thiba...@gnu.org> skribis:

> Ludovic Courtès, on Sun 05 Jun 2016 21:53:35 +0200, wrote:
>> Justus Winter <jus...@gnupg.org> skribis:
>> > +#define Mach_port_check(NAME)   \
>> > +  void _Mach_port_check_##NAME(char *_unused[] __attribute__ ((unused))) \
>> > +  { \
>> > +  if (MACH_PORT_VALID (NAME))   \
>> > +__print_fail_backtrace (#NAME " leaked",\
>> > +__FILE__, __LINE__, "Port leak detector");  \
>> > +  } \
>> > +  char _Mach_port_check_x_##NAME[0] \
>> > +  __attribute__ ((unused, cleanup (_Mach_port_check_##NAME)))
>> 
>> I think writing a GCC plug-in that would automatically add a cleanup
>> handler to automatic variables of type ‘mach_port_t’ wouldn’t be
>> unreasonable.
>
> I don't enough to be sure, but isn't that the typical work of
> LocalitySanitizer, precisely?

Could be!  I’m not familiar with it.

Ludo’.

Re: [PATCH hurd 2/2] libshouldbeinlibc: add safe port handling macros

[Ludovic Courtès]

Justus Winter  skribis:

> +#define Mach_port_check(NAME)   \
> +  void _Mach_port_check_##NAME(char *_unused[] __attribute__ ((unused))) \
> +  { \
> +  if (MACH_PORT_VALID (NAME))   \
> +__print_fail_backtrace (#NAME " leaked",\
> +__FILE__, __LINE__, "Port leak detector");  \
> +  } \
> +  char _Mach_port_check_x_##NAME[0] \
> +  __attribute__ ((unused, cleanup (_Mach_port_check_##NAME)))

I think writing a GCC plug-in that would automatically add a cleanup
handler to automatic variables of type ‘mach_port_t’ wouldn’t be
unreasonable.

Ludo’.

Re: [GSoC] Porting GuixSD to GNU/Hurd Update 1

[Ludovic Courtès]

Hi Manolis!

Manolis Ragkousis  skribis:

> We already knew that the guix-daemon was not working properly on the
> Hurd.  I looked more into it and I found that if you do a build with
> Guix and it fails, and then you retry again, the next one will create a
> new build directory in /tmp/ as it should, but it will still use the
> first one.

Are you sure about this?  There’s a nice feature from a reproducibility
viewpoint, when using chroot builds, which is that the build directory
inside the chroot is always the same (always
“/tmp/guix-build-foobar.drv-0”), even if its name outside the chroot
needs to be different (say, “/my/tmp/guix-build-foobar.drv-12”).  See
Guix commit cb9601029ea164b86bdf997f7160d494c15d344b.

> I also found out that guix publish may sometimes crash if a client asks
> for a big substitute.  I will investigate this one as soon as possible.

If you can find a way to reproduce it, please send it to
bug-g...@gnu.org.

I saw on IRC that you publish GNU/Hurd binaries from your machine, and I
find it pretty cool!

> Now on the coding part, in Guix I updated the gnumach/mig/hurd packages
> to the latest version and worked on getting wip-hurd in a state which
> can eventually merge to master.  A part of wip-hurd is already on master
> and after this core-updates cycle is merged to master, we will be able
> to get the rest of wip-hurd merged as well. Package glibc/hurd will also
> be updated then as well.

Great, I’m willing to make progress on this front!

> On debian/hurd I am using Guix every day and in my github wip-hurd I
> have some patches which I need to clean up and apply to the Guix repo.
> These patches are for various packages of various importance.

Awesome.

> And on the Hurd side, I created a new library called libhurdutil and
> moved the settrans implementation there.  For implementation info please
> read this email[1]. I am currently improving that library after
> Guillem's and Justus' input. (Thank you :-))  On the Guix side I will
> just write a wrapper to use this call and we will be able to control
> translators really easily.  This way the non existing mount() is not a
> problem anymore :-).

Nice work!  Perhaps the library should be called “libhurdtrans” or
something like that to better reflect what it’s about?

It may be that the functionality that was extracted from the ‘settrans’
command needs be nicely “repackaged” to make for a nice library
interface (for instance, by removing global variables and by untangling
UI concerns from actual functionality), but I’ll leave it up to the Hurd
people to comment on it.  :-)

Thanks for the update!

Ludo’.

Re: A malware? and GPL infrigment?

[Ludovic Courtès]

Samuel Thibault  skribis:

> I have however been approached by clamav maintainers, who told me
> that one such malware got ported to GNU/Hurd! They apparently "just"
> rebuilt the malware using a GNU/Hurd cross tolchain (why not running
> the qemu image?! Beats me). They however had to patch the source a bit:
> apparently in previous versions it was using the PATH_MAX constant,
> whose 4096 magic number could be seen in the Linux binary. The authors
> changed that into the classical dynamic allocation loop. Actually the
> generated code looks very much like the "sample" loop linked from
> https://www.gnu.org/software/hurd/hurd/porting/guidelines.html : 

I think two lessons can be learned from that:

  1. Lack of PATH_MAX has been effective at protecting us from malware
 (and arguably goodware as well) up to now.

  2. The GNU Coding Standard’s stance on removing arbitrary limits is
 finally percolating, reaching even malware developers!

Anyway, thanks for the heads-up!

Ludo’.

Re: [GSoC] Porting GuixSD to GNU Hurd draft

[Ludovic Courtès]

Hi Justus,

Justus Winter <4win...@informatik.uni-hamburg.de> skribis:

> Quoting Ludovic Courtès (2016-03-23 14:40:38)
>> > 2. The Project
>> >
>> > The project consists of four main stages
>> >
>> > 1. Modify Guix so it will be able to create and mount the file-system 
>> > needed to boot into a system with Hurd at its core. 
>> > 2. Modify Guix so it can produce a working image, while isolating any 
>> > cases of Linux assumptions.
>> > 3. Successfully boot into one such system using GNU Shepherd with pid 1.
>> > 4. Modify the new Guix system to take advantage of Hurd specific 
>> > mechanisms.
>
> For me, 4. is the most important bit, so we can build packages in
> isolation.

I guess you mean isolation in guix-daemon.

>> This is more important, and already non-trivial work.  If libc provided
>> ‘mount’ with support for MS_BIND (which I think it should), it would
>> help a bit, but there’s still the problem of the other Linux name spaces
>> that are used (the CLONE_NEW* clone(2) flags.)
>> 
>> Thus it may make more sense to write this functionality in guix-daemon
>> using directly the Hurd interfaces.  Separate PID name spaces, UID name
>> spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
>> “just” a matter of giving the child process ports to separate proc,
>> auth, etc. translators.
>> 
>> In itself this is also a bit of work.  I wonder what the Hurd folks
>> think about priorities here?
>
> I'd go for specializing guix-daemon over trying too hard to implement
> Linux-compatible interfaces in the libc.

So this would become maybe half of the GSoC coding part maybe.  WDYT?

> I consider the filesystem-isolation part easy, UID isolation
> relatively easy, PID isolation is a bit tricky.  Currently one cannot
> simply start another proc server and expect it to work as it needs
> privileged kernel interfaces.  I created an RPC to allow nested proc
> servers for unprivileged subhurds.  That should do the trick, it is
> merely a matter of making it actually work I guess.

“Merely”, hmm…  :-)

So, let’s say PID isolation will be optional, and we can always adjust
later on.  Sounds good?

Manolis, make sure to read about how the various Hurd servers provides
these parts of POSIX personality: file systems, UIDs, PIDs, networking,
and so on.

As far as code integration code, I think we won’t bother syncing this
work with nix-daemon; guix-daemon has already diverged, and for instance
it does not have OS X sandbox support.

You’ll have to arrange to have the Hurd-specific bits in a separate
file, so that ‘#ifdef HURD’ are not scattered all over the place.

This is C(++) as you know.  WDYT, Manolis?

>> This in itself requires some thought: currently (guix system vm) relies
>> on QEMU’s ‘-kernel’ option to boot the kernel Linux.  For GNU/Hurd, we
>> instead need to boot a complete image with GRUB, like ‘guix system
>> vm-image’ does.  You’ll have to investigate how to port this.
>
> qemu can boot multiboot operating systems.

Great, didn’t know that.

>> > Deliver a working GuixSD system image with Hurd as the kernel.
>
> Hurd is not a kernel.

I think we use “kernel” to mean “the thing(s) that libc talks to.”

>> The main question is whether you should implement build isolation in
>> guix-daemon, in which case that would leave little time for the GuixSD
>> parts.  I think I would rather let you focus on the GuixSD stuff as you
>> wrote, but I’d like to hear what the Hurd folks think.
>
> I consider isolation more important.

OK.

So, Manolis, what about reframing the agenda such that porting
guix-daemon to GNU/Hurd comes first (I’d consider it roughly half of the
programming effort), followed by GuixSD stuff?

Thanks,
Ludo’.

Re: [GSoC] Porting GuixSD to GNU Hurd draft

[Ludovic Courtès]

Hello!

Manolis Ragkousis  skribis:

> Although I have uploaded and shared my draft to the GSoC website, I am
> also resending it to the lists as per Ludovic's instruction.

Yeah, the GSoC website makes it possible to provide a link to a text
file, so let’s do that instead of using their SaaSS.

> 2. The Project
>
> The project consists of four main stages
>
> 1. Modify Guix so it will be able to create and mount the file-system needed 
> to boot into a system with Hurd at its core. 
> 2. Modify Guix so it can produce a working image, while isolating any cases 
> of Linux assumptions.
> 3. Successfully boot into one such system using GNU Shepherd with pid 1.
> 4. Modify the new Guix system to take advantage of Hurd specific mechanisms.
>
>
> Currently the tools Guix uses to interact with the filesystem exist inside 
> the (guix build syscalls) module. 
> This module provides bindings to libc's syscall wrappers, which are only 
> available on a GNU/Linux system. 
> In order to offer the same functionality on a GNU/Hurd system we must first 
> write Guile bindings for the 
> relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs
> for example.

Note that technically the ‘file_set_translator’ function is in libc:

--8<---cut here---start->8---
scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link))
$1 = #
--8<---cut here---end--->8---

> This module will be called (guix build hurd). This allows us to
> re-implement the functionality of the 'settrans' command, as described
> here[1], in Scheme and use that in place of 'mount()', where
> applicable.

Good!  (We might as well call it (hurd …) in fact: (hurd fs) would
correspond to fs.defs, (hurd io) for io.defs, and so on.)

> Additionally, we need to make sure that all modules in (guix build *) and 
> (gnu build *) can offer the same 
> functionalities on a GNU/Hurd system.  For example (gnu build vm) relies on 
> QEMU's '-kernel' command-line 
> option, which is Linux-specific.  On the other hand, in the case of modules 
> like (gnu build linux-boot) or 
> (gnu build linux-initrd), which by design are Linux-specific, we will need to 
> provide a separate module with 
> equivalent functionality. (gnu system *) modules will require changes as 
> well, in order to be able to use 
> modifications that will happen in the (guix build *) and (gnu build *) 
> modules. 

I think it’s important to think about:

  1. How functionality equivalent to linux-{initrd,boot} will be
 implemented on the Hurd.

 In my experience the equivalent thing is simpler on the Hurd,
 esp. if relying on passive translators (see daemons/runsystem.sh in
 the Hurd), though here we’ll probably explicitly activate
 translators at boot time.

  2. How to structure GuixSD code in a way that abstracts the underlying
 OS kernel.

 For instance, we could have a (guix build os) module providing an
 API that works for both kernels, probably close to the Linux one,
 and that would dispatch to the right implementation, the Linux or
 Hurd one.

Both of these are quite a bit of design and implementation work that we
should not underestimate.

> Finally, once GuixSD is successfully ported, we can cater to the last stage 
> of taking advantage of Hurd specific 
> mechanisms. 
> This includes but is not limited to:
> 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a 
> GNU/Hurd system. Subhurds can offer 
> isolation similar to Linux containers as described here[3].

This is really super optional IMO.  This module is only used by ‘guix
environment --container’, which is an optional feature.

> 2)Modify the guix-daemon to run without root privileges by utilizing the Hurd 
> architecture. The daemon needs root 
> privileges in order to setup chrooted environments for the builds.  In the 
> Hurd this can be done by setting up a 
> translator as described here[4]. 

This is more important, and already non-trivial work.  If libc provided
‘mount’ with support for MS_BIND (which I think it should), it would
help a bit, but there’s still the problem of the other Linux name spaces
that are used (the CLONE_NEW* clone(2) flags.)

Thus it may make more sense to write this functionality in guix-daemon
using directly the Hurd interfaces.  Separate PID name spaces, UID name
spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
“just” a matter of giving the child process ports to separate proc,
auth, etc. translators.

In itself this is also a bit of work.  I wonder what the Hurd folks
think about priorities here?

> 3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]. 
> Stowfs creates a traditional Unix directory 
> structure from all the files in the individual package directories.

Fun but optional.  ;-)

> 3. Estimated Project Timeline
>
> Before March 31
> Finish merging the wip-hurd branch to 

Re: Coppyright assignment

[Ludovic Courtès]

Justus Winter <4win...@informatik.uni-hamburg.de> skribis:

> Quoting Ludovic Courtès (2016-03-04 13:49:09)
>> Justus Winter <4win...@informatik.uni-hamburg.de> skribis:
>> 
>> > I have learned that not all GNU projects require this.  For example,
>> > GnuPG uses DCO [0], and Guix doesn't even seem to require that.
>> >
>> > Maybe we can change this policy for Hurd, GNU Mach, and MIG too?
>> 
>> I general, we “cannot” (i.e., are invited not to) change GNU packages
>> from requiring-assignment to no-assignment policy (although I guess
>> GnuPG did exactly that…).
>
> So one can do that.

Everything can be done, but ultimately it’s a matter of reaching
consensus.

> I see it as an barrier for contributing.

Right, but whether it’s a good idea depends on a number of other
factors.  For Guix, copyright assignment would have been a hindrance
IMO, but what’s at stake is different than for libc or GCC, for
instance.

>> Hopefully paperwork is fast enough these days anyway.
>
> I don't know how it is nowadays (maybe someone can enlighten us?), but
> when I did that like three years ago the process took weeks and
> required quite a bit of nagging.

Use of PDFs has helped a bit:

  https://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html

as well as digital signatures in some cases:

  
https://www.fsf.org/blogs/licensing/fsf-to-begin-accepting-gpg-signed-assignments-from-the-u-s
  
https://www.fsf.org/blogs/licensing/fsf-to-begin-accepting-gpg-signatures-for-copyright-assignments-from-italy

Ludo’.

Re: Coppyright assignment

[Ludovic Courtès]

Justus Winter <4win...@informatik.uni-hamburg.de> skribis:

> I have learned that not all GNU projects require this.  For example,
> GnuPG uses DCO [0], and Guix doesn't even seem to require that.
>
> Maybe we can change this policy for Hurd, GNU Mach, and MIG too?

I general, we “cannot” (i.e., are invited not to) change GNU packages
from requiring-assignment to no-assignment policy (although I guess
GnuPG did exactly that…).

Hopefully paperwork is fast enough these days anyway.

Ludo’.

Re: GSoC ideas

[Ludovic Courtès]

Samuel Thibault <samuel.thiba...@gnu.org> skribis:

> Ludovic Courtès, on Wed 02 Mar 2016 11:09:28 +0100, wrote:
>> Manolis Ragkousis <manolis...@gmail.com> skribis:
>> > 1) Merge wip-hurd branch.
>> > 2) Make the daemon handle chroot builds on the Hurd.
>> > Note here that on the Hurd, one does not need to be root to achieve
>> > isolation, so I should change the daemon to use this capability.
>> 
>> I think an ideal situation would be if libc provides ‘mount’ and
>> ‘umount’, with MS_BIND implemented in terms of firmlinks.
>> 
>> I remember Roland was not thrilled with the idea of adding ‘mount’ and
>> ‘umount’, but it would clearly help porting.
>
> It can both help and hurt. Roland's concern is that we will probably not
> want to provide a mount() that behaves like Linux, because in some cases
> it may just not even make sense. So the "ported" software that would try
> to use it would just fail.

Do you have examples of GNU/Linux uses that would make no sense?

Other options include calling out to the ‘settrans’ command (inelegant
IMO), writing Guile bindings for some of the Hurd libraries, and/or some
sort of a MiG in Scheme (neat but takes some time!).

Ludo’.

Re: GSoC ideas

[Ludovic Courtès]

Hi!

I realize I hadn’t responded to Manolis.  Another +1 from me!

Some random comments:

Manolis Ragkousis  skribis:

> Before May:
>
> 1) Merge wip-hurd branch.
> 2) Make the daemon handle chroot builds on the Hurd.
> Note here that on the Hurd, one does not need to be root to achieve
> isolation, so I should change the daemon to use this capability.

I think an ideal situation would be if libc provides ‘mount’ and
‘umount’, with MS_BIND implemented in terms of firmlinks.

I remember Roland was not thrilled with the idea of adding ‘mount’ and
‘umount’, but it would clearly help porting.  I don’t know if this task
should be part of the GSoC project though, and you may need guidance
from the Hurd folks.  What do people think?

> 3) Instead of using the Linux syscall guile wrappers, I should modify
> Guix to use a more Hurdish way (i.e settrans) so later on we can handle
> translators and bootstrap a GNU/Hurd system.

Possibly, depending on whether libc provides ‘mount’ and ‘umount’, which
would make things simpler.

> End of GSoC:
> Have a working GuixSD with Hurd as the kernel.

\o/

Cheers,
Ludo’.

Re: active translators stdout/stderr

[Ludovic Courtès]

Richard Braun  skribis:

> On Sun, Feb 14, 2016 at 10:09:37PM +0100, Samuel Thibault wrote:
>> Hello,
>> 
>> We have various issues with the active translators' stdout/stderr:
>> 
>> - When mounting /proc from /etc/init.d/rc, stdout/stderr is closed, and
>>   thus as soon as e.g. mtab reports a warning, it gets SIGLOST and dies.
>> - When mounting the cdrom image from debian installer scripts, the
>>   script is not considered finished since the isofs translator is still
>>   running and possibly emitting output.
>> 
>> So, what should we do about stdout/stderr output?  Do we require their
>> output to be explicitly redirected by callers of mount and settrans?
>> In the mount case, people used to Linux will not think of doing it, so
>> perhaps we need it to do it for the user?
>
> I think we could have a default log server for that. We could make it
> a gsoc project. It should make a good entry project.

In the meantime I suppose /dev/null would make for a good log server,
and it’s presumably a simple fix?

Ludo’.

Re: Rolling new releases

[Ludovic Courtès]

Justus Winter <4win...@informatik.uni-hamburg.de> skribis:

> as agreed earlier, we're trying to produce two releases a year.  We
> released GNU Mach 1.5, GNU MIG 1.5, and GNU Hurd 0.6 in April, hence
> it is time for our next release :)

Don’t forget libc!  :-)

Ludo’.

Re: [GSoC update] Porting Guix to GNU/Hurd

[Ludovic Courtès]

Hi!

Manolis Ragkousis manolis...@gmail.com skribis:

 1) Guix can successfully cross-build any package for the Hurd and produce
 the bootstrap-tarballs to build packages with Guix natively on such a system.

 2) Guix can build the native final toolchain.

 3) Guix can build packages natively using the final toolchain.

 Even though most of the work is already present in the wip-hurd branch, there
 are still some patches to be reviewed/merged. This will happen asap.

 But nevertheless we can safely say we have ported Guix to Hurd. :-)

Congratulations on that!  That’s a great achievement, and I know there
were some bumps on the road, and the project covers several very
different pieces of software, so thumbs up!

 2) guix/build/syscalls (mount, umount, setns ) Those aren't available
 on Hurd, so
 building Guix would fail. It seems turning them into procedures is
 enough to work
 around this.

This module is only used by GuixSD anyway (not Guix itself), so no
problem.

In the interim, we should perhaps arrange to not build that module when,
say, ‘mount’ is missing (using an ‘AM_CONDITIONAL’.)

 4) Perl could not be build because of a problem with memmove in hurd's
 glibc. Together
 with Samuel and Justus we traced the problem to a not properly tested
 part of glibc.
 Fixed.

Pheew.  :-)

 5) glibc-intermediate could not find the gnumach/hurd headers because
 of unsetting the
 path during the build process. It seems that glibc's sysdeps/mach and
 sysdeps/mach/hurd
 configure weren't using the --with-headers variable to get the
 headers. Fixed that and sent
 the patch to bug-hurd :-).

OK.

 6) Binaries produced from gcc-boot0 had problems with their runpath
 and validate-runpath?
 was failing. It seems ld wasn't passing -rpath to them. Added an
 ld-wrapper, solved.

Interesting, will look at the patch hopefully soon.

 7) glibc-final's debug output refers to %glibc-bootstrap. I think this
 happens indirectly through
 the headers, currently solving that.

Would be nice to understand why that happens.  In the meantime you could
disable #:allowed-references altogether for that package when on
GNU/Hurd, if you want.

 Also, I am near finishing sys/mount.h for Hurd. Will finish it as soon
 as possible.

Great!

 I get the feeling I forgot some things so please remind/ask me
 anything. For the next two
 days I will wrap up patches and push them to wip-hurd. After that I
 will continue working on
 getting the full GuixSD experience on Hurd. And one of these days we
 will deploy guix on
 darnassus together with rbraunr. I will inform you when that happens.

Excellent.  Thanks to the Hurd and Guix people for giving you a hand!

Cheers,
Ludo’.

Re: Problem with natively built binaries on Hurd from Guix

[Ludovic Courtès]

Hi Manolis,

I just remembered how I addressed it when I cross-built the Hurd with
Nix: Given that libc.so is an ld script on GNU/Hurd, simply add
libhurduser.so and libmachuser.so in there, next to libc.so.0.3 (see
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/glibc/builder.sh#L34.)

I think it can be a relative file name (so literally “libhurduser.so”
would do.)

HTH!

Ludo’.

Re: Question with moving mount/umount logic in glibc

[Ludovic Courtès]

Roland McGrath rol...@hack.frob.com skribis:

 Frankly I think it would be better to keep these single-caller interfaces
 out of libc proper.  It's not really ideal that they are there for Linux
 either, but syscall stubs are less of an issue than real code.

While not ideal, I think it would greatly simplify porting to have libc
provide those functions regardless of the kernel.

Ludo’.

Re: Question with moving mount/umount logic in glibc

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 Sounds awesome.  One thing to be aware of (iirc) is that the
 mount/umount code depends on the fstab parser.  I'm not sure whether
 it is needed for the mount/umount(2) interface, or just for the
 command line frontend.  I bet the former, that means that you also
 have to move the fstab parser to the libc.

/etc/fstab handling is not part of the mount/umount functions (on Linux
mount and umount are syscall wrappers generated from syscalls.list.)

Ludo’.

[GSoC] Guix + Hurd continuation

[Ludovic Courtès]

Hello!

Now that Manolis managed to produce the bootstrap binaries (yay!), we
must plan for what’s next.

I think the first thing will be to try and start building things
natively, as noted in
https://www.gnu.org/software/guix/manual/html_node/Porting.html.  That
implies a few things:

  0. Run Guix’s ‘./configure --with-courage  make’ and see what
 happens.

  1. ‘guix-daemon’ must work correctly on GNU/Hurd.  This should be
 mostly the case, but it’s degraded compared to GNU/Linux because it
 will not use name spaces and bind mounts, so no chroot either (see
 
http://git.savannah.gnu.org/cgit/guix.git/tree/nix/libstore/build.cc#n54.)
 That’s OK for a start.  Maybe there are other issues I haven’t
 thought of.

  2. When building natively, surely you’ll find out that some packages
 do not build (PATH_MAX!), and that there are assumptions in
 hurd.scm and base.scm, such as the fact that GNU/Hurd is a
 cross-compilation target and not a native system.

  3. In parallel to that, I should review wip-hurd again and apply the
 patches to a new ‘core-updates’ branch.

  4. Instead of sitting idle watching build logs ;-), it Would Be Nice™
 to implement the ‘mount’ and ‘umount’ functions for GNU/Hurd in
 libc, with support for MS_BIND using /hurd/firmlink.  See
 http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/utils/mount.c#n213
 (the Hurd’s ‘mount’ command) for a good starting point.

 Of course libc hacking can be quite involved.  So ideally Samuel,
 Thomas, and others would give you detailed guidance and/or hack
 power.  What do people think?

 The reason I’m suggesting this is that it will be useful for the
 build daemon (see #1), and it will also facilitate porting of
 GuixSD afterwards (see
 
http://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/syscalls.scm#n144.)

Anything else?  Comments?

Ludo’.

Re: [GSoC] Guix + Hurd continuation

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

   2. When building natively, surely you’ll find out that some packages
  do not build (PATH_MAX!), and that there are assumptions in
  hurd.scm and base.scm, such as the fact that GNU/Hurd is a
  cross-compilation target and not a native system.

Another can of worms I forgot to mention is hurd/paths.h, which
assumes that translators live in /hurd, whereas we’ll rather have them
in /gnu/store/…/hurd.

The recipe for the ‘hurd’ package could change those file names in
hurd/paths.h.  However, translator file names such as “/hurd/symlink”
serve as shortcuts to avoid starting a translator for each symlink:
http://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/libdiskfs/file-set-trans.c#n111.

Thus, with /gnu/store file names for translators, a libc that uses
/gnu/store/XYZ/hurd/symlink as _HURD_SYMLINK (e.g., in
http://git.savannah.gnu.org/cgit/hurd/glibc.git/tree/sysdeps/mach/hurd/symlink.c)
would prevent the shortcut if it talks to a translator that expects
/gnu/store/ABC/hurd/symlink.

I think it would work anyway, but would end up starting one instance of
/hurd/symlink for each symlink, which is suboptimal.

Food for thought!

Ludo’.

Re: glibc build log (TLS-related issue)

[Ludovic Courtès]

Manolis Ragkousis manolis...@gmail.com skribis:

 On 24 June 2015 at 12:17, Thomas Schwinge tho...@codesourcery.com wrote:
 and the same command with the tarballs sources end up with

 http://paste.lisp.org/display/150437

 That looks like https://sourceware.org/PR15605,
 http://news.gmane.org/find-root.php?message_id=%3CE1VU3KH-00083u-04%40vasks.debian.org%3E,
 http://news.gmane.org/find-root.php?message_id=%3CE1VU9tp-0007UR-IS%40vasks.debian.org%3E.

 Applied unsubmitted-NO_HIDDEN.diff, in the tarball source, built with
 -enable-add-ons=libpthread --enable-pt_chown --disable-nscd
 CFLAGS=-O2 --host=i586-gnu --build=i586-gnu
 and I got a working static binary :-D

It would be nice to see if it’s the patch that’s “fixing the issue” or
if it’s using i586 rather than i686.

Thanks for investigating!

Ludo’.

Re: GSoC: Porting Guix to Hurd week 3+4 report.

[Ludovic Courtès]

Thomas Schwinge tho...@codesourcery.com skribis:

 On Thu, 04 Jun 2015 22:48:48 +0200, l...@gnu.org (Ludovic 
 =?utf-8?Q?Court=C3=A8s?=) wrote:
 Manolis Ragkousis manolis...@gmail.com skribis:

[...]

   autoreconf  ./configure --localstatedir=/var \
 --with-libgcrypt-prefix=/gnu/store/...  make

 (Not relevant right now, but why is the libgcrypt path not communicated
 via the environment variables set up with guix environment?  Is that just
 because there are no appropriate environment variables, or something
 else?)  Also, I wanted to note that I, very guixy, computed that path
 using:

 $ guix build libgcrypt
 warning: failed to install locale: Invalid argument
 /gnu/store/r16v30hlw2d6n232rm37p53qy8rpr7f2-libgcrypt-1.6.3
 /gnu/store/42ls5n7k6lai1k6xfa8v8wks7nn9g3gn-libgcrypt-1.6.3-debug

Yes, that’s fine.

 Next, I ran:

 $ ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K -c 8

  After it fails

 ..., and eventually reproduced the problem.  However, that took a series
 of steps that was much longer that I had anticipated:

It turns out that hydra.gnu.org is not (yet) serving pre-built binaries
for this branch, so you ended up rebuilding the world, including doing a
complete bootstrap of the distro (info (guix) Bootstrapping).

I realize this is terribly inconvenient, so apologies for that!

I hope we can soon merge the changes to the “core” packages in the
‘core-updates’ branch, and have ‘wip-hurd’ contain only very specific
patches.  That way, most of the things will have pre-built binaries
available on hydra.gnu.org.

 Assuming I need to patch the GCC sources, do I just do that in
 /tmp/nix-build-gcc-4.7.4.drv-0/gcc-4.7.4/, and then, to continue the
 build, just run the same guix build command again?  (I guess I'll also
 have to preserve my changes elsewhere, as that temporary working tree
 will be removed upon a successful build?)

The simplest way is to use the --with-source option of ‘guix build’,
which allows you to specify an alternate source tarball (info (guix)
Invoking guix build).

Now, from a discussion we had on IRC, I think the problem reported at
the beginning of this thread is fixed.  Manolis, can you confirm?

Also, I think the target should be 4.9 or 4.8, but definitely not 4.7.
See also
http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00052.html.

Thank you!

Ludo’.

Re: GSoC: Porting Guix to Hurd week 3+4 report.

[Ludovic Courtès]

Manolis Ragkousis manolis...@gmail.com skribis:

 Hey Thomas, thank you for looking into this.

 On 2 June 2015 at 18:55, Thomas Schwinge tho...@codesourcery.com wrote:
 Shame on me, but I've never actively used/built Guix before.  I do know
 about https://github.com/Phant0mas/Guix-on-Hurd, and that there must be
 a Guix manual existing -- but can you help me get started, please?
 (Pointers to specific parts of documentation are appreciated, of course.)

 So let's start with the easy one. Currently Guix can be used only of top of a
 GNU/Linux system. So with that in mind, clone the repo (the one from savannah
 git://git.savannah.gnu.org/guix.git )
 on one such system and make sure you have the dependencies mentioned at
 http://www.gnu.org/software/guix/manual/guix.html#Requirements installed.

Alternately, there’s a really easy way: grab the binary tarball and
follow the steps at http://www.gnu.org/software/guix/download/ (make
sure to check the ownership of /var and /root after unpacking it; the
extracting the current tarball will incorrectly change the owner.)

From there:

  git clone git://git.savannah.gnu.org/guix.git
  cd guix
  git checkout wip-hurd
  guix environment guix
  autoreconf  ./configure --localstatedir=/var \
--with-libgcrypt-prefix=/gnu/store/...  make

 After it fails go to /tmp/nix-build-gcc-4.7...  and there you can find
 the failed build. Everything will be there.
 As for the the logs, they can  be found with ls -ltr
 /usr/local/var/log/guix/drvs/*/*.

.. or “guix build gcc-4.7 --target=i686-pc-gnu --log-file”.

Ludo’.

Re: GSoC: Manolis to work on Guix port to GNU/Hurd

[Ludovic Courtès]

Samuel Thibault samuel.thiba...@gnu.org skribis:

 Ludovic Courtès, le Thu 07 May 2015 14:42:10 +0200, a écrit :
 It would be nice to tag commits that correspond to glibc X.Y plus
 Hurd-specific patches.

 I'm not sure what exactly you mean.  What I have done is tagging the
 commits in the master-glibc-2.21 branch.

Ah OK, perfect then.  I thought this was based on an arbitrary commit of
upstream glibc.

Thanks,
Ludo’.

Re: GSoC: Manolis to work on Guix port to GNU/Hurd

[Ludovic Courtès]

Samuel Thibault samuel.thiba...@gnu.org skribis:

 Ludovic Courtès, le Wed 29 Apr 2015 21:40:13 +0200, a écrit :
 The last missing bit upstream is a libc-for-hurd tarball.

 I have prepared a master-glibc branch in the libpthread repo.

Excellent!

 $ git clone git.savannah.gnu.org:/srv/git/hurd/glibc.git/
 $ cd glibc
 $ git checkout tschwinge/Roger_Whittaker
 $ git clone git.savannah.gnu.org:/srv/git/hurd/libpthread.git/
 $ cd libpthread
 $ git checkout master-glibc
 $ cd ..
 $ mkdir build
 $ cd build
 $ CFLAGS=-O2 ../configure --prefix=/ --enable-add-ons=libpthread 
 --enable-pt_chown --disable-nscd
 $ make
 $ make check -k

It would be nice to tag commits that correspond to glibc X.Y plus
Hurd-specific patches.

We could have, say, glibc-2.21-hurd1, similar to the version names used
by IceCat and Linux-libre.

Anyway, a big thank you!

Ludo’.

Re: GSoC: Manolis to work on Guix port to GNU/Hurd

[Ludovic Courtès]

Andreas Enge andr...@enge.fr skribis:

 On Thu, Apr 30, 2015 at 12:01:44AM +0300, Manolis Ragkousis wrote:
 I just wanted to add that I will keep a repo at github which I will upload 
 any
 (serious) local changes before I send them as patches so anyone can keep
 an eye on the current status of my work.

 I am not quite sure how this is supposed to make things easier. Why not simply
 use a branch in the guix repo (like the existing wip-hurd, or wip-wip-hurd for
 very experimental changes)? Then every commit of yours will be sent to our
 mailing list, and you develop directly in our infrastructure.

I think it’s fine if Manolis wants to have experiments in another repo.
The “final” code will indeed go to Savannah anyway, and Manolis is
already used to it.

Ludo’.

Re: Release process rolling new releases

[Ludovic Courtès]

Woohoo!  Congratulations to everyone involved!  :-)

Ludo’.

Re: Hurd development

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 Quoting Samuel Thibault (2015-01-16 12:08:07)
 Justus Winter, le Fri 16 Jan 2015 12:01:26 +0100, a écrit :

[...]

 Also, don't you have commit access to the debian packaging?  I guess
 that's one of the things we must fix.

 No I don't have access.

Better this way.  ;-)

Seriously, I think it’s saner to clearly separate Debian and upstream
again.  That upstream libc doesn’t work at all is in part due to the
fact that it’s so much easier to add patches to Debian than to go
through upstream review–and I’m not blaming anyone, I understand the
short-term benefits of doing that, but I think it’s harmful in the long
term.

Ludo’.

Re: Reinventing the Hurd server bootstrap

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 Quoting Ludovic Courtès (2014-12-27 22:22:34)

 Did you consider using a statically-linked Guile?  This is what we do in
 the Linux initrd for Guix, and it’s wonderful.  :-)
 
 If you take that route, we can make it easier to share code.  (And we
 can adjust Guile as needed, too.)

 Yes, I actually tried this before I went with TinySCHEME.  I didn't
 managed to get it to work, maybe because the program has to be totally
 stand-alone.  When bootshell starts, there is no filesystem, so one
 cannot load resources from it.

Oh right.  Guile 2.0 relies on the availability of a file system to load
its modules from.  I wonder if we could make some sort of an initrd or
multiboot module that would contain those files?

With Guile 2.2 we may be able to lump all the modules together in a big
ELF image, but that’s not for now.

 Keep up the good hack!

 I'm excited about it too.  For the first time, I feel like I can
 actually change the bootstrap process that has caused me so much pain
 in the past ,

Sounds like a big improvement, indeed!

Ludo’.

Re: Reinventing the Hurd server bootstrap

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 Wait, that sounds like serverboot, doesn't it ?

 It does.  I think it was a mistake to abandon it in the first place.
 Evidence for that: 1.  We now have two copies of the bootscript
 parser, once in the kernel, once in boot.  2.  We moved non-essential
 functionality (with lot's of string parsing) into the kernel.  3.  It
 makes strong assumptions about the platform, e.g. assumes a bootloader
 that can load modules like GRUB does with the multiboot protocol.

 But we don't use some made-up shell-like language, we use a Scheme
 interpreter.

 \o/

This is exciting!  ;-)

 Why is it awesome ?

 (define (bootstrap)
   (log Hurd server bootstrap:)
   (bind-root rootfs-task)
   [...]
   (log .\n))

 (catch (panic Hurd bootstrap failed:  last-exception \n)
(bootstrap))

We’ve been using this approach in Guix to bootstrap GNU/Linux: all the
code to load kernel modules, mount the root file system, switch to the
new root, etc. are written in Scheme.  And likewise, we get a REPL with
a debugger when something goes wrong.

 start bootshell: bootshell/TinySCHEME 1.41.

Did you consider using a statically-linked Guile?  This is what we do in
the Linux initrd for Guix, and it’s wonderful.  :-)

If you take that route, we can make it easier to share code.  (And we
can adjust Guile as needed, too.)

Keep up the good hack!

Ludo’.

Re: [PATCH hurd] libpager: use a fixed number of threads

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 Quoting Samuel Thibault (2014-11-02 13:52:16)
 AIUI, this does work because you had previously separated disk and file
 pagers?

 Yes.

  +  /* What follows is basically the second part of
  +  mach_msg_server_timeout.  */
 
 So you need to copy the copyright years :)

 That's not straight forward as the glibc uses year ranges (1993-2014
 in this case) and we do not.  I remember Thomas saying something like
 we could use ranges too, but that needs some setting up or something.

The convention is to just add something along these lines in ‘README’:

  See the LICENSE file for the specific terms that apply to Guile.  Note
  that for any copyright year range specified as - in this
  package, the range specifies every single year in that closed interval.

Ludo’.

Re: Hurd server introspection and tracing

[Ludovic Courtès]

Hi,

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 * I use Machs `Inherited Ports' mechanism to install a receive right
   at a well-known location to serve introspection requests.  This is
   by choice orthogonal to the usual mechanism used in the Hurd, as I
   want it to be as universal as possible (e.g. I want to be able to
   use it on the proc server).

 * Servers can label port buckets and classes, and install a function
   that creates a human-readable description of an object.  This
   information is then presented to the user doing portinfo
   --query-process, a feature that is hinted to by a #if 0ed option in
   portinfo.c.

This looks neat.

BTW, I was looking at msg.defs, which does higher-level introspection,
and noticed that, surprisingly, utils/msgport.c lacks support for
‘msg_describe_ports’, although libc does implement it.  Would be another
nice addition.

Ludo’.

Re: Release process rolling new releases

[Ludovic Courtès]

Justus Winter 4win...@informatik.uni-hamburg.de skribis:

 I understand you took care of the release process last time.  Is this
 process documented somewhere?  I think that we should make another
 round of releases.  In fact, we should make one or two releases each
 year.  At the very least it brings us quite a bit of attention.

 If it's just a matter of doing this or that, please let me know how I
 can contribute to this process.

Excellent initiative!

It would be ideal (but maybe I’m asking for too much) if this would come
with a libc tarball as well, presumably based on the sv.gnu.org repo
rebased on the latest libc released.  In the short term that would be
helpful for Guix, and in the longer term that would help the project as
a whole I think.

Thanks,
Ludo’.

nscd fails to build

[Ludovic Courtès]

Hello!

Manolis reported that nscd fails to build with Savannah’s libc, and with
libpthread used as an add-on:

--8---cut here---start-8---
i686-pc-gnu-gcc getpwnam_r.c -c -std=gnu99 -fgnu89-inline  -O2 -Wall -Winline 
-Wwrite-strings -fmerge-all-constants -frounding-math -g -Wno-parentheses 
-Wstrict-prototypes   -Wa,-mtune=i686   -fpie-I../include 
-I/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/nscd  
-I/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build  
-I../libpthread/sysdeps/mach/hurd/i386  -I../sysdeps/mach/hurd/i386  
-I../libpthread/sysdeps/mach/hurd  -I../libpthread/sysdeps/hurd  
-I../libpthread/sysdeps/pthread  -I../sysdeps/pthread  -I../sysdeps/mach/hurd  
-I../sysdeps/gnu  -I../sysdeps/unix/bsd/bsd4.4  -I../sysdeps/mach/i386  
-I../libpthread/sysdeps/mach  -I../sysdeps/mach  -I../sysdeps/i386/i686/fpu  
-I../sysdeps/i386/i686  -I../sysdeps/i386/i486  -I../sysdeps/i386/fpu  
-I../sysdeps/x86/fpu  -I../libpthread/sysdeps/i386  -I../sysdeps/i386  
-I../sysdeps/x86  -I../sysdeps/wordsize-32  -I../sysdeps/ieee754/ldbl-96  
-I../sysdeps/ieee754/dbl-64  -I../sysdeps/ieee754/flt-32  -I../sysdeps/unix/bsd 
 -I../sysdeps/unix/inet  -I../sysdeps/ieee754  -I../sysdeps/unix  
-I../sysdeps/posix  -I../libpthread/sysdeps/posix  -I../sysdeps/generic  
-I../libpthread/include -I../libpthread  -I../ports -I../hurd 
-I/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/hurd/ -I../mach 
-I/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/mach/ -I.. 
-I../libio -I. -nostdinc -isystem 
/gnu/store/r3zflcz1l7959f34i4r1hdy9x49ranjm-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/lib/gcc/i686-pc-gnu/4.8.2/include
 -isystem 
/gnu/store/r3zflcz1l7959f34i4r1hdy9x49ranjm-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/lib/gcc/i686-pc-gnu/4.8.2/include-fixed
 -isystem 
/gnu/store/0m6iv2lbkbjb3imjy29vqdrkv48wrjk0-linux-libre-headers-3.3.8/include 
-I../libpthread/include  -D_LIBC_REENTRANT -include ../include/libc-symbols.h   
-DNOT_IN_libc=1 -DIS_IN_nscd=1 -D_FORTIFY_SOURCE=2-o 
/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/nscd/getpwnam_r.o 
-MD -MP -MF 
/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/nscd/getpwnam_r.o.dt
 -MT 
/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-0/build/nscd/getpwnam_r.o
In file included from ../sysdeps/mach/hurd/tls.h:28:0,
 from ../sysdeps/mach/hurd/i386/tls.h:24,
 from ../sysdeps/mach/hurd/bits/libc-lock.h:24,
 from ../sysdeps/mach/hurd/dirstream.h:22,
 from ../include/dirent.h:3,
 from nscd.c:22:
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/mig_errors.h:71:3:
 warning: function declaration isn't a prototype [-Wstrict-prototypes]
   (*ms_routine)();
   ^
In file included from ../sysdeps/mach/hurd/tls.h:28:0,
 from ../sysdeps/mach/hurd/i386/tls.h:24,
 from ../sysdeps/i386/i486/bits/atomic.h:20,
 from ../include/atomic.h:50,
 from connections.c:21:
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/mig_errors.h:71:3:
 warning: function declaration isn't a prototype [-Wstrict-prototypes]
   (*ms_routine)();
   ^
In file included from ../sysdeps/mach/hurd/tls.h:28:0,
 from ../sysdeps/mach/hurd/i386/tls.h:24,
 from ../sysdeps/i386/i486/bits/atomic.h:20,
 from ../include/atomic.h:50,
 from ../nss/getXXbyYY_r.c:20,
 from getpwnam_r.c:31:
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/mig_errors.h:71:3:
 warning: function declaration isn't a prototype [-Wstrict-prototypes]
   (*ms_routine)();
   ^
In file included from ../sysdeps/mach/hurd/tls.h:28:0,
 from ../sysdeps/mach/hurd/i386/tls.h:24,
 from ../sysdeps/i386/i486/bits/atomic.h:20,
 from ../include/atomic.h:50,
 from nscd-client.h:28,
 from nscd.h:30,
 from pwdcache.c:36:
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/mig_errors.h:71:3:
 warning: function declaration isn't a prototype [-Wstrict-prototypes]
   (*ms_routine)();
   ^
connections.c:112:13: error: 
'PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP' undeclared here (not in a 
function)
 .lock = PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP,
 ^
connections.c: In function 'restart':
connections.c:1487:11: warning: ignoring return value of 'setuid', declared 
with attribute warn_unused_result [-Wunused-result]
setuid (server_uid);
   ^
connections.c:1502:11: warning: ignoring return value of 'setuid', declared 
with attribute warn_unused_result [-Wunused-result]
setuid (server_uid);
   ^
connections.c:1503:11: warning: ignoring return 

Re: [PATCH] nscd: Remove unused typedef and variable.

[Ludovic Courtès]

Ondřej Bílka nel...@seznam.cz skribis:

 On Sun, Jun 22, 2014 at 11:55:43AM +0200, Ludovic Courtès wrote:
 The attached patch removed the unused ‘thread_info_t’ typedef and the
 ‘thread_info’ variable from nscd.c.  The former conflicts with a GNU Mach
 typedef, and the latter conflicts with a GNU Mach function declaration:
 https://lists.gnu.org/archive/html/bug-hurd/2014-06/msg00101.html.
 
 Tested on x86_64-linux-gnu.
 
 Thanks,
 Ludo’.
 
 Looks ok so I commited it.

Thank you.

Ludo’.

Re: Conflicting ‘thread_info_t’ declaration between nscd and Mach

[Ludovic Courtès]

Pino Toscano toscano.p...@tiscali.it skribis:

 On Saturday 21 June 2014 23:29:43 Ludovic Courtès wrote:
 Roland McGrath rol...@hack.frob.com skribis:
  Send a patch upstream to libc to avoid using the *_t name space for
  non-public type names like the one in nscd.  We'll take it.
 
 Great, will do.
 
 Manolis: could you confirm that the attached patch allows you to build
 libc for the Hurd with nscd support?
 
 diff --git a/nscd/nscd.c b/nscd/nscd.c
 index 5680378..c910fd5 100644
 --- a/nscd/nscd.c
 +++ b/nscd/nscd.c
 @@ -61,14 +61,14 @@
 and under a shutdown condition to wait till all in-progress
 requests have finished before turning off the lights.  */
  
 -typedef struct
 +struct thread_info
  {
int num_active;
pthread_cond_t  thread_exit_cv;
pthread_mutex_t mutex;
 -} thread_info_t;
 +};
  
 -thread_info_t thread_info;
 +struct thread_info thread_info;

 Or what about simply getting rid of thread_info_t struct and thread_info 
 variable, since they don't seem to be used at all within nscd?

Indeed, I had overlooked that.

 (Of course, that would require testing also on Linux.)

I’ll try it.

Ludo’.

[PATCH] nscd: Remove unused typedef and variable.

[Ludovic Courtès]

The attached patch removed the unused ‘thread_info_t’ typedef and the
‘thread_info’ variable from nscd.c.  The former conflicts with a GNU Mach
typedef, and the latter conflicts with a GNU Mach function declaration:
https://lists.gnu.org/archive/html/bug-hurd/2014-06/msg00101.html.

Tested on x86_64-linux-gnu.

Thanks,
Ludo’.

2014-06-22  Ludovic Courtès  l...@gnu.org

* nscd/nscd.c (thread_info_t): Remove typedef.
(thread_info): Remove variable.

diff --git a/nscd/nscd.c b/nscd/nscd.c
index 5680378..d4faa29 100644
--- a/nscd/nscd.c
+++ b/nscd/nscd.c
@@ -56,20 +56,6 @@
 
 #define PACKAGE _libc_intl_domainname
 
-/* Structure used by main() thread to keep track of the number of
-   active threads.  Used to limit how many threads it will create
-   and under a shutdown condition to wait till all in-progress
-   requests have finished before turning off the lights.  */
-
-typedef struct
-{
-  int num_active;
-  pthread_cond_t  thread_exit_cv;
-  pthread_mutex_t mutex;
-} thread_info_t;
-
-thread_info_t thread_info;
-
 int do_shutdown;
 int disabled_passwd;
 int disabled_group;

Conflicting ‘thread_info_t’ declaration between nscd and Mach

[Ludovic Courtès]

While courageously trying to build Savannah’s libc for GNU/Hurd,
Manolis stumbled upon this issue:

--8---cut here---start-8---
nscd.c:67:3: error: conflicting types for 'thread_info_t'
 } thread_info_t;
   ^
In file included from 
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/mach_types.h:47:0,
 from ../mach/mach.h:28,
 from ../sysdeps/mach/hurd/tls.h:29,
 from ../sysdeps/mach/hurd/i386/tls.h:24,
 from ../sysdeps/mach/hurd/bits/libc-lock.h:24,
 from ../sysdeps/mach/hurd/dirstream.h:22,
 from ../include/dirent.h:3,
 from nscd.c:22:
/gnu/store/x760iihp152nxwyrsch6kmxcswbpi7zy-gnumach-headers-cross-i686-pc-gnu-1.4/include/mach/thread_info.h:47:20:
 note: previous declaration of 'thread_info_t' was here
 typedef integer_t *thread_info_t;  /* varying array of ints */
--8---cut here---end---8---

The nscd and Mach both define ‘thread_info_t’, thus preventing libc to
be built with nscd support.

Any idea how to address that?

TIA,  :-)
Ludo’.

Re: Circular dependency with glibc libpthread libihash

[Ludovic Courtès]

Samuel Thibault samuel.thiba...@gnu.org skribis:

 Ludovic Courtès, le Fri 30 May 2014 17:29:27 +0200, a écrit :
 Manolis Ragkousis manolis...@gmail.com skribis:
 
  I had to put one more line after AC_NO_EXECUTABLES, otherwise it would 
  fail with
  syntax error near unexpected token `fi'.
 
 AC_NO_EXECUTABLES expands to nothing, hence the error.  It seems that we
 don’t even have to use it in a conditional.

 ? I see in the autoconf source code that it was modified precisely so
 that it can be used in a conditional.

Yes, the comment says that, but it’s unclear to me that’s it’s still
relevant: the macro expands to nothing, so what’s gained by using it in
a conditional?

Anyway, things work as is, so maybe it’s not worth bothering.  ;-)

Ludo’.

Re: Circular dependency with glibc libpthread libihash

[Ludovic Courtès]

Manolis Ragkousis manolis...@gmail.com skribis:

 I had to put one more line after AC_NO_EXECUTABLES, otherwise it would fail 
 with
 syntax error near unexpected token `fi'.

AC_NO_EXECUTABLES expands to nothing, hence the error.  It seems that we
don’t even have to use it in a conditional.

Ludo’.

Re: [PATCH] build: Remove checks for 'getgrouplist' and 'uselocale'

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

 From a8b1c2e3c5cd89f26b6856c63fd3ba625a36ad54 Mon Sep 17 00:00:00 2001
 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= l...@gnu.org
 Date: Sun, 25 May 2014 01:06:50 +0200
 Subject: [PATCH] build: Remove checks for 'getgrouplist' and 'uselocale'.

 GNU libc has had them for a long time.

 * configure.ac: Remove checks for 'getgrouplist' and 'uselocale'.
 * libshouldbeinlibc/idvec-impgids.c (_merge_implied_gids): Remove #ifdef
   HAVE_GETGROUPLIST and remove #else arm.
 * libthreads/cthreads.c: Remove #ifdef HAVE_USELOCALE, keeping its
   bodies.

I’ve committed this one has it looked trivial.  Let me know if I should
proceed differently.

Thanks,
Ludo’.

[PATCH] build: Remove checks for 'getgrouplist' and 'uselocale'

[Ludovic Courtès]

This patch removes configure checks that have long been unneeded.

OK to apply?

Ludo’.

From a8b1c2e3c5cd89f26b6856c63fd3ba625a36ad54 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= l...@gnu.org
Date: Sun, 25 May 2014 01:06:50 +0200
Subject: [PATCH] build: Remove checks for 'getgrouplist' and 'uselocale'.

GNU libc has had them for a long time.

* configure.ac: Remove checks for 'getgrouplist' and 'uselocale'.
* libshouldbeinlibc/idvec-impgids.c (_merge_implied_gids): Remove #ifdef
  HAVE_GETGROUPLIST and remove #else arm.
* libthreads/cthreads.c: Remove #ifdef HAVE_USELOCALE, keeping its
  bodies.
---
 configure.ac  |  3 ---
 libshouldbeinlibc/idvec-impgids.c | 14 +-
 libthreads/cthreads.c |  9 +++--
 3 files changed, 4 insertions(+), 22 deletions(-)

diff --git a/configure.ac b/configure.ac
index 873ced8..e756496 100644
--- a/configure.ac
+++ b/configure.ac
@@ -156,9 +156,6 @@ else
 fi
 AC_SUBST(VERSIONING)
 
-# Check if libc contains getgrouplist and/or uselocale.
-AC_CHECK_FUNCS(getgrouplist uselocale)
-
 
 # From glibc HEAD, 2007-11-07.
 AC_CACHE_CHECK(for -fgnu89-inline, libc_cv_gnu89_inline, [dnl
diff --git a/libshouldbeinlibc/idvec-impgids.c b/libshouldbeinlibc/idvec-impgids.c
index 74d3cc1..d89f487 100644
--- a/libshouldbeinlibc/idvec-impgids.c
+++ b/libshouldbeinlibc/idvec-impgids.c
@@ -1,6 +1,6 @@
 /* Add gids implied by a user
 
-   Copyright (C) 1997, 2001 Free Software Foundation, Inc.
+   Copyright (C) 1997, 2001, 2014 Free Software Foundation, Inc.
 
Written by Miles Bader mi...@gnu.ai.mit.edu
 
@@ -56,7 +56,6 @@ _merge_implied_gids (struct idvec *implied_gids, uid_t uid)
 else
   {
 	struct idvec *cache = make_idvec ();
-#ifdef HAVE_GETGROUPLIST
 	gid_t _gids[NUM_STATIC_GIDS], *gids = _gids;
 	int maxgids = NUM_STATIC_GIDS;
 	int ngids = getgrouplist (pw-pw_name, pw-pw_gid, gids, maxgids);
@@ -79,17 +78,6 @@ _merge_implied_gids (struct idvec *implied_gids, uid_t uid)
 	if (gids != _gids)
 	  free (gids);
 	  }
-#else
-#warning getgrouplist() not available; supplementary group IDs unsupported.
-	if (! cache)
-	  err = ENOMEM;
-	else
-	  {
-	err = idvec_add_new (cache, pw-pw_gid);
-	if (err)
-	  idvec_free (cache);
-	  }
-#endif
 
 	if (! err)
 	  {
diff --git a/libthreads/cthreads.c b/libthreads/cthreads.c
index aef20be..1361b8b 100644
--- a/libthreads/cthreads.c
+++ b/libthreads/cthreads.c
@@ -150,10 +150,7 @@
 #include cthreads.h
 #include mach/mig_support.h
 #include cthread_internals.h
-
-#ifdef HAVE_USELOCALE
-# include locale.h
-#endif
+#include locale.h
 
 /*
  * Thread status bits.
@@ -292,11 +289,11 @@ cthread_body(cproc_t self)
 /*
  * Execute the fork request.
  */
-#ifdef HAVE_USELOCALE
+
 			/* A fresh thread needs to be bound to the
    global locale.  */
 			  	uselocale (LC_GLOBAL_LOCALE);
-#endif
+
 t-result = (*(t-func))(t-arg);
 			}
 			/*
-- 
1.8.4

Re: [PATCH] Remove configure check for libcrypt

[Ludovic Courtès]

Samuel Thibault samuel.thiba...@gnu.org skribis:

 Ludovic Courtès, le Sat 12 Apr 2014 18:37:12 +0200, a écrit :
 This patch removes a configure check from 1996, which has been
 unnecessary for some time.
 
 OK to apply?

 Why not :)

Done, thanks.

Ludo’.

Re: Problem with glibc and libihash

[Ludovic Courtès]

Hi Manolis,

Thanks for dealing with this, and for persevering, despite the difficult
situation.  Think of you as a pioneering explorer.  ;-)

Manolis Ragkousis manolis...@gmail.com skribis:

 First of all libihash and any of the hurd libraries cannot be built
 without a working glibc. Hurd libs depend heavily on header files
 found in glibc like lowlevellock.h, so even after making the build
 system of the hurd repository to accept not to be able to build
 programs (as Samuel suggested), we are still not able to build the
 libs.

 So our only option is to build a glibc without the libpthread addon,
 use it to build libihash, and then rebuild glibc with libpthread. But
 theres comes our second and biggest problem.

 Glibc cannot be built without libpthread.

This is terrible.  (This was actually easier when I packaged it in
Nixpkgs, before libpthread was made a glibc add-on.)

Samuel’s suggestion to install glibc headers just to be able to build
libihash  co. seems to be the easiest one.

Could you try making a ‘glibc-headers’ package whose build phase does
nothing and whose install phase does ‘make install-headers’?

Thanks,
Ludo’.

[PATCH] Remove configure check for libcrypt

[Ludovic Courtès]

This patch removes a configure check from 1996, which has been
unnecessary for some time.

OK to apply?

Thanks,
Ludo’.

From a0f93043770eb593600f1ae7ddefa9a82f2e23cb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= l...@gnu.org
Date: Sat, 12 Apr 2014 18:33:35 +0200
Subject: [PATCH] build: Remove configure check for libcrypt.

GNU libc has provided the 'crypt' function in libcrypt for a long time,
making this check unnecessary.

* configure.ac: Remove libcrypt check and 'LIBCRYPT' substitution.
* config.make.in (LIBCRYPT): Remove.
* trans/Makefile (password-LDLIBS): Change $(LIBCRYPT) to -lcrypt.
* utils/Makefile (login-LDLIBS, addauth-LDLIBS, setauth-LDLIBS):
  Likewise.
---
 config.make.in | 3 ---
 configure.ac   | 4 
 trans/Makefile | 4 ++--
 utils/Makefile | 8 
 4 files changed, 6 insertions(+), 13 deletions(-)

diff --git a/config.make.in b/config.make.in
index c5d4e68..0f1390a 100644
--- a/config.make.in
+++ b/config.make.in
@@ -65,9 +65,6 @@ gnu89-inline-CFLAGS = @libc_cv_gnu89_inline@
 # `yes' or `no' to indicate if ld --version-script is available.
 VERSIONING = @VERSIONING@
 
-# If a separate libcrypt is available, use it.
-LIBCRYPT = @LIBCRYPT@
-
 # How to link against Parted libraries, if at all.
 PARTED_LIBS = @PARTED_LIBS@
 
diff --git a/configure.ac b/configure.ac
index ecabfdf..873ced8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -106,10 +106,6 @@ fi
 dnl Let these propagate from the environment.
 AC_SUBST(CFLAGS) AC_SUBST(CPPFLAGS) AC_SUBST(LDFLAGS)
 
-# See if there's a separate libcrypt (many systems put crypt there).
-AC_CHECK_LIB(crypt, crypt, LIBCRYPT=-lcrypt)
-AC_SUBST(LIBCRYPT)
-
 hurd_MIG_RETCODE
 
 # See if --version-script is available.
diff --git a/trans/Makefile b/trans/Makefile
index a294282..71e6424 100644
--- a/trans/Makefile
+++ b/trans/Makefile
@@ -1,6 +1,6 @@
 #
 #   Copyright (C) 1994, 1995, 1996, 1997, 1999, 2000, 2001, 2002, 2006, 2007,
-#   2008, 2013 Free Software Foundation, Inc.
+#   2008, 2013, 2014 Free Software Foundation, Inc.
 #
 #   This program is free software; you can redistribute it and/or
 #   modify it under the terms of the GNU General Public License as
@@ -31,7 +31,7 @@ OBJS = $(SRCS:.c=.o) fsysServer.o ifsockServer.o passwordServer.o \
 	device_replyServer.o elfcore.o
 HURDLIBS = ports netfs trivfs iohelp fshelp pipe ihash shouldbeinlibc
 LDLIBS += -lpthread
-password-LDLIBS = $(LIBCRYPT)
+password-LDLIBS = -lcrypt
 password-MIGSFLAGS=\
 -DIO_INTRAN=trivfs_protid_t trivfs_begin_using_protid (io_t) \
 -DIO_DESTRUCTOR=trivfs_end_using_protid (trivfs_protid_t) \
diff --git a/utils/Makefile b/utils/Makefile
index 81fa293..241b060 100644
--- a/utils/Makefile
+++ b/utils/Makefile
@@ -1,4 +1,4 @@
-#   Copyright (C) 1994,95,96,97,98,99,2000,01,02,12 Free Software Foundation,
+#   Copyright (C) 1994,95,96,97,98,99,2000,01,02,12,14 Free Software Foundation,
 #   Inc.
 #
 #   This program is free software; you can redistribute it and/or
@@ -36,9 +36,9 @@ SRCS = shd.c ps.c settrans.c syncfs.c showtrans.c addauth.c rmauth.c \
 OBJS = $(filter-out %.sh,$(SRCS:.c=.o))
 HURDLIBS = ps ihash store fshelp ports ftpconn shouldbeinlibc
 LDLIBS += -lpthread
-login-LDLIBS = -lutil $(LIBCRYPT)
-addauth-LDLIBS = $(LIBCRYPT)
-setauth-LDLIBS = $(LIBCRYPT)
+login-LDLIBS = -lutil -lcrypt
+addauth-LDLIBS = -lcrypt
+setauth-LDLIBS = -lcrypt
 mount-LDLIBS = $(libblkid_LIBS)
 mount-CPPFLAGS = $(libblkid_CFLAGS)
 
-- 
1.8.4

Re: Circular dependency with glibc libpthread libihash

[Ludovic Courtès]

Manolis Ragkousis manolis...@gmail.com skribis:

 When libpthread tries to link against libihash.so I get the error:

 /lib/libihash.so: file not recognized: File format not recognized.

 This happens because libihash was built using the native compiler.
 So I changed the flag in libihash configure to --host so it will use
 the cross-compiler. But then I got the error configure:3008: error: C
 compiler cannot create executables.
 I read the config.log file and the problem arise from this
 /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
 cannot find crt1.o: No such file or directory
 /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
 cannot find crti.o: No such file or directory
 /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
 cannot find -lc
 /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
 cannot find crtn.o: No such file or directory

 It tries to compile a program, but cannot find these files, so it
 fails. But in order to have them we need glibc.

Could you show what test tries to compile a program?  (Send config.log.)

Thanks,
Ludo’.

Re: Circular dependency with glibc libpthread libihash

[Ludovic Courtès]

Manolis Ragkousis manolis...@gmail.com skribis:

 0:00 Ludovic Courtès l...@gnu.org:

 Could you show what test tries to compile a program?  (Send config.log.)


 Here it is

Thanks.

 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.

 It was created by GNU Hurd configure 0.5, which was
 generated by GNU Autoconf 2.69.  Invocation command line was

[...]

 configure:2937: checking whether the C compiler works
 configure:2959: i686-pc-gnu-gccconftest.c  5
 /gnu/store/kavylb2khfqq9rj42diyciinv3i8inj4-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
  cannot find crt1.o: No such file or directory
 /gnu/store/kavylb2khfqq9rj42diyciinv3i8inj4-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
  cannot find crti.o: No such file or directory
 /gnu/store/kavylb2khfqq9rj42diyciinv3i8inj4-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
  cannot find -lc
 /gnu/store/kavylb2khfqq9rj42diyciinv3i8inj4-gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld:
  cannot find crtn.o: No such file or directory
 collect2: error: ld returned 1 exit status

Could you try this patch for the Hurd’s configure.in?

diff --git a/configure.ac b/configure.ac
index ecabfdf..7ede6db 100644
--- a/configure.ac
+++ b/configure.ac
@@ -83,6 +83,13 @@ AC_PROG_INSTALL
 AC_PROG_AWK
 AC_PROG_SED
 
+if test x$cross_compiling = xyes; then
+   # It may be that we don't have a working libc yet, for instance
+   # because we're bootstrapping the cross-compilation tool chain.
+   # Thus, use this undocumented Autoconf macro designed for this.
+   AC_NO_EXECUTABLES
+fi
+
 AC_PROG_CC
 # Require GCC.
 if test x$GCC != xyes; then

Thanks,
Ludo’.

Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff

[Ludovic Courtès]

Indeed, few of the patches at
http://patch-tracker.debian.org/package/hurd/1:0.5.git20140326-1 look
Debian-specific.

For features that are not “fully baked” yet, like DDE, wouldn’t it make
sense to have a branch in the Hurd repo, instead of a set of patches in
Debian?

As for patches for which review “never completed”, there should really
be a timeout IMO, and the patch should be either rejected for everyone
(including Debian), or accepted, or amended.

My 2¢,
Ludo’.

Re: Glibc building procedure error report.

[Ludovic Courtès]

Samuel Thibault samuel.thiba...@gnu.org skribis:

 Manolis Ragkousis, le Wed 02 Apr 2014 00:29:13 +, a écrit :
   /../build/libc_pic.os: In function `__fork':
   /../source/posix/../sysdeps/mach/hurd/fork.c:71: undefined reference to 
  `__start__hurd_atfork_prepare_hook'
   /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld: 
  /../build/libc_pic.os: relocation R_386_GOTOFF against undefined hidden   
  symbol `__start__hurd_atfork_prepare_hook' can not be used when making a 
  shared object
   /../gcc-cross-sans-libc-i686-pc-gnu-4.8.2/libexec/gcc/i686-pc-gnu/ld: 
  final link failed: Bad value
   collect2: error: ld returned 1 exit status
   ../Makerules:614: recipe for target '/../build/libc.so' failed
   make[2]: *** 
  [/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-38/build/libc.so] 
  Error 1
   make[2]: Leaving directory 
  '/tmp/nix-build-glibc-hurd-cross-i686-pc-gnu-2.18.drv-38/source/elf'
   Makefile:233: recipe for target 'elf/subdir_lib' failed
 
 I searched the source files in posix and sysdeps/mach/hurd/ and I
 can't seem to find any reference to
 `__start__hurd_atfork_prepare_hook'.
 
 Any ideas?

 In such cases, look for subparts of the name, here atfork_prepare, and
 you'll see 

 sysdeps/mach/hurd/fork.c:DEFINE_HOOK (_hurd_atfork_prepare_hook, (void));

 The symbol is supposed to be defined automagically by the linker.  I
 don't know how exactly.

The linker magic is in the shlib.lds rules of Makerules, added in commit
56798c44 of glibc/Savannah.

However, it adds the start/stop symbols for the hurd_fork hooks, but not
for the hurd_atfork hooks.  Do we need something like this:

diff --git a/Makerules b/Makerules
index fe967ad..cbf1b39 100644
--- a/Makerules
+++ b/Makerules
@@ -506,6 +506,15 @@ $(common-objpfx)shlib.lds: $(common-objpfx)config.make $(..)Makerules
 		 PROVIDE(__start__hurd_preinit_hook = .);\
 		 _hurd_preinit_hook : { *(_hurd_preinit_hook) }\
 		 PROVIDE(__stop__hurd_preinit_hook = .);\
+		 PROVIDE(__start__hurd_atfork_prepare_hook = .);\
+		 _hurd_atfork_prepare_hook : { *(_hurd_atfork_prepare_hook) }\
+		 PROVIDE(__stop__hurd_atfork_prepare_hook = .);\
+		 PROVIDE(__start__hurd_atfork_child_hook = .);\
+		 _hurd_atfork_child_hook : { *(_hurd_atfork_child_hook) }\
+		 PROVIDE(__stop__hurd_atfork_child_hook = .);\
+		 PROVIDE(__start__hurd_atfork_parent_hook = .);\
+		 _hurd_atfork_parent_hook : { *(_hurd_atfork_parent_hook) }\
+		 PROVIDE(__stop__hurd_atfork_parent_hook = .);\
 		 PROVIDE(__start__hurd_fork_prepare_hook = .);\
 		 _hurd_fork_prepare_hook : { *(_hurd_fork_prepare_hook) }\
 		 PROVIDE(__stop__hurd_fork_prepare_hook = .);\

Thanks,
Ludo’.

Re: [PATCHv2,HURD] Add fork hooks for pthread_atfork

[Ludovic Courtès]

Thomas Schwinge tho...@codesourcery.com skribis:

 Manolis has hit this issue again,
 http://news.gmane.org/find-root.php?message_id=%3CCAFtzXzNgS8KEwRJO74SF7zZPMnwFkW_m1qvir6y7-WNwaKEsWQ%40mail.gmail.com%3E.

 On Tue, 4 Mar 2014 02:40:12 +0100, Samuel Thibault samuel.thiba...@gnu.org 
 wrote:
 Roland McGrath, le Mon 03 Mar 2014 15:36:52 -0800, a écrit :
   Roland McGrath, le Fri 20 Dec 2013 13:05:42 -0800, a ,Aicrit :
The linker is supposed to define those symbols automagically.
Did it break?

The mechanism does work on GNU/Linux, for instance with the
__libc_subfreeres hook:

--8---cut here---start-8---
$ objdump -t libc-2.19.so | grep -E '__(stop|start).*libc_subfreeres'
0039f750 l   .dtors   
__start___libc_subfreeres
0039f840 l   __libc_subfreeres    
__stop___libc_subfreeres
--8---cut here---end---8---

Manolis: could you add an #error in libc-symbols.h to make sure the
non-empty definition of ‘DEFINE_HOOK’ is getting used?

Thanks,
Ludo’.