[GSoC] Porting GuixSD to GNU Hurd draft

2016-03-21 Thread Manolis Ragkousis
Hello everyone,

Although I have uploaded and shared my draft to the GSoC website, I am
also resending it to the lists as per Ludovic's instruction.

I am sharing it as a txt + pdf.

Here is also the link that I shared to the GSoC website as per Google's
instructions.
https://docs.google.com/document/d/1NEEz_PHM2YZTm8okd2BgPRSc4c89I1fGnwbyeeEUrhw/edit?usp=sharing

This is the time to share your ideas/corrections :-)

Manolis
Porting GuixSD to GNU/Hurd

Manolis Ragkousis
manolis...@gmail.com

March 17, 2016


1. Summary

In this project, we would like to port the Guix Software Distribution to GNU 
Hurd.  By the end of the project, 
Guix will be able to handle and use the available Hurd mechanisms, have the 
required tools to produce a working, 
bootable, system vm/image with GNU Shepherd as the init daemon and offer the 
same functionality as you would 
expect from booting into a GNU/Linux GuixSD system. 


2. The Project

The project consists of four main stages

1. Modify Guix so it will be able to create and mount the file-system needed to 
boot into a system with Hurd at its core. 
2. Modify Guix so it can produce a working image, while isolating any cases of 
Linux assumptions.
3. Successfully boot into one such system using GNU Shepherd with pid 1.
4. Modify the new Guix system to take advantage of Hurd specific mechanisms.


Currently the tools Guix uses to interact with the filesystem exist inside the 
(guix build syscalls) module. 
This module provides bindings to libc's syscall wrappers, which are only 
available on a GNU/Linux system. 
In order to offer the same functionality on a GNU/Hurd system we must first 
write Guile bindings for the 
relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs for 
example. This module will be called 
(guix build hurd). This allows us to re-implement the functionality of the 
'settrans' command, as described 
here[1], in Scheme and use that in place of 'mount()', where applicable.

Additionally, we need to make sure that all modules in (guix build *) and (gnu 
build *) can offer the same 
functionalities on a GNU/Hurd system.  For example (gnu build vm) relies on 
QEMU's '-kernel' command-line 
option, which is Linux-specific.  On the other hand, in the case of modules 
like (gnu build linux-boot) or 
(gnu build linux-initrd), which by design are Linux-specific, we will need to 
provide a separate module with 
equivalent functionality. (gnu system *) modules will require changes as well, 
in order to be able to use 
modifications that will happen in the (guix build *) and (gnu build *) modules. 

At this point we will be able to generate a working vm image and boot into it.  
Guix will be able to use the 
Hurd servers (i.e. /hurd/init) to start the init manager and the system itself. 
 We will also have to account 
for the fact that servers in /hurd will be symlinks to /gnu/store/*, and modify 
Guix and/or the Hurd libraries 
accordingly to achieve functionality.  Regarding the init daemon, please note 
that a precedent already exists 
for starting GuixSD/Hurd with GNU Shepherd as pid 1 (David Michael [2015][2]).

Finally, once GuixSD is successfully ported, we can cater to the last stage of 
taking advantage of Hurd specific 
mechanisms. 
This includes but is not limited to:
1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a 
GNU/Hurd system. Subhurds can offer 
isolation similar to Linux containers as described here[3].
2)Modify the guix-daemon to run without root privileges by utilizing the Hurd 
architecture. The daemon needs root 
privileges in order to setup chrooted environments for the builds.  In the Hurd 
this can be done by setting up a 
translator as described here[4]. 
3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]. 
Stowfs creates a traditional Unix directory 
structure from all the files in the individual package directories.

 
3. Estimated Project Timeline

Before March 31
Finish merging the wip-hurd branch to upstream.
Write a (guix build hurd) module which will contain Guile bindings to the RPC 
stubs like hurd/fs.defs or hurd/exec.defs .

April 1 - April 15
Package missing dependencies (Hurd libs).
Re-implement 'settrans' in scheme.

April 16 - May 1
At this point we will have the tools needed to build a Hurd based file-system. 
(Milestone 1)
Start working on getting (guix build *) and (gnu build *) modules to work on 
Hurd.
Make sure '%base-packages' in (gnu system) module work as expected on Hurd.

May 2 - May 22
Create (gnu build hurd-boot) and (gnu build hurd-initrd).
Start working on describing the GNU/Hurd system in (gnu system).

May 23 - 12 June
Modify (gnu system *) modules as needed.
All the modules (guix build *) and (gnu build *) will be working as expected by 
now.
Try building a GuixSD image. (Milestone 2)

13 June - 23 June
Solve any problems with booting into the system and running GNU Shepherd.

24 June - 9 July
Have a fully working GNU/Hurd system. (Milestone 3)

Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-21 Thread Samuel Thibault
Hello,

Manolis Ragkousis, on Mon 21 Mar 2016 14:09:36 +0200, wrote:
> Although I have uploaded and shared my draft to the GSoC website, I am
> also resending it to the lists as per Ludovic's instruction.

Looks very promising :)

Samuel



Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-23 Thread Ludovic Courtès
Hello!

Manolis Ragkousis  skribis:

> Although I have uploaded and shared my draft to the GSoC website, I am
> also resending it to the lists as per Ludovic's instruction.

Yeah, the GSoC website makes it possible to provide a link to a text
file, so let’s do that instead of using their SaaSS.

> 2. The Project
>
> The project consists of four main stages
>
> 1. Modify Guix so it will be able to create and mount the file-system needed 
> to boot into a system with Hurd at its core. 
> 2. Modify Guix so it can produce a working image, while isolating any cases 
> of Linux assumptions.
> 3. Successfully boot into one such system using GNU Shepherd with pid 1.
> 4. Modify the new Guix system to take advantage of Hurd specific mechanisms.
>
>
> Currently the tools Guix uses to interact with the filesystem exist inside 
> the (guix build syscalls) module. 
> This module provides bindings to libc's syscall wrappers, which are only 
> available on a GNU/Linux system. 
> In order to offer the same functionality on a GNU/Hurd system we must first 
> write Guile bindings for the 
> relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs
> for example.

Note that technically the ‘file_set_translator’ function is in libc:

--8<---cut here---start->8---
scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link))
$1 = #
--8<---cut here---end--->8---

> This module will be called (guix build hurd). This allows us to
> re-implement the functionality of the 'settrans' command, as described
> here[1], in Scheme and use that in place of 'mount()', where
> applicable.

Good!  (We might as well call it (hurd …) in fact: (hurd fs) would
correspond to fs.defs, (hurd io) for io.defs, and so on.)

> Additionally, we need to make sure that all modules in (guix build *) and 
> (gnu build *) can offer the same 
> functionalities on a GNU/Hurd system.  For example (gnu build vm) relies on 
> QEMU's '-kernel' command-line 
> option, which is Linux-specific.  On the other hand, in the case of modules 
> like (gnu build linux-boot) or 
> (gnu build linux-initrd), which by design are Linux-specific, we will need to 
> provide a separate module with 
> equivalent functionality. (gnu system *) modules will require changes as 
> well, in order to be able to use 
> modifications that will happen in the (guix build *) and (gnu build *) 
> modules. 

I think it’s important to think about:

  1. How functionality equivalent to linux-{initrd,boot} will be
 implemented on the Hurd.

 In my experience the equivalent thing is simpler on the Hurd,
 esp. if relying on passive translators (see daemons/runsystem.sh in
 the Hurd), though here we’ll probably explicitly activate
 translators at boot time.

  2. How to structure GuixSD code in a way that abstracts the underlying
 OS kernel.

 For instance, we could have a (guix build os) module providing an
 API that works for both kernels, probably close to the Linux one,
 and that would dispatch to the right implementation, the Linux or
 Hurd one.

Both of these are quite a bit of design and implementation work that we
should not underestimate.

> Finally, once GuixSD is successfully ported, we can cater to the last stage 
> of taking advantage of Hurd specific 
> mechanisms. 
> This includes but is not limited to:
> 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a 
> GNU/Hurd system. Subhurds can offer 
> isolation similar to Linux containers as described here[3].

This is really super optional IMO.  This module is only used by ‘guix
environment --container’, which is an optional feature.

> 2)Modify the guix-daemon to run without root privileges by utilizing the Hurd 
> architecture. The daemon needs root 
> privileges in order to setup chrooted environments for the builds.  In the 
> Hurd this can be done by setting up a 
> translator as described here[4]. 

This is more important, and already non-trivial work.  If libc provided
‘mount’ with support for MS_BIND (which I think it should), it would
help a bit, but there’s still the problem of the other Linux name spaces
that are used (the CLONE_NEW* clone(2) flags.)

Thus it may make more sense to write this functionality in guix-daemon
using directly the Hurd interfaces.  Separate PID name spaces, UID name
spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
“just” a matter of giving the child process ports to separate proc,
auth, etc. translators.

In itself this is also a bit of work.  I wonder what the Hurd folks
think about priorities here?

> 3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]. 
> Stowfs creates a traditional Unix directory 
> structure from all the files in the individual package directories.

Fun but optional.  ;-)

> 3. Estimated Project Timeline
>
> Before March 31
> Finish merging the wip-hurd branch to upstream.

Note that this

Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-23 Thread Justus Winter
Hi,

Quoting Ludovic Courtès (2016-03-23 14:40:38)
> > 2. The Project
> >
> > The project consists of four main stages
> >
> > 1. Modify Guix so it will be able to create and mount the file-system 
> > needed to boot into a system with Hurd at its core. 
> > 2. Modify Guix so it can produce a working image, while isolating any cases 
> > of Linux assumptions.
> > 3. Successfully boot into one such system using GNU Shepherd with pid 1.
> > 4. Modify the new Guix system to take advantage of Hurd specific mechanisms.

For me, 4. is the most important bit, so we can build packages in
isolation.

> > Currently the tools Guix uses to interact with the filesystem exist inside 
> > the (guix build syscalls) module. 
> > This module provides bindings to libc's syscall wrappers, which are only 
> > available on a GNU/Linux system. 
> > In order to offer the same functionality on a GNU/Hurd system we must first 
> > write Guile bindings for the 
> > relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs
> > for example.
> 
> Note that technically the ‘file_set_translator’ function is in libc:
> 
> --8<---cut here---start->8---
> scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link))
> $1 = #
> --8<---cut here---end--->8---
> 
> > This module will be called (guix build hurd). This allows us to
> > re-implement the functionality of the 'settrans' command, as described
> > here[1], in Scheme and use that in place of 'mount()', where
> > applicable.

Right.  In fact, what settrans (or mount) does isn't that hard to
reproduce, though I wouldn't mind moving the c implementation to
libhurdutil or the like and binding to that.

> I think it’s important to think about:
> 
>   1. How functionality equivalent to linux-{initrd,boot} will be
>  implemented on the Hurd.
> 
>  In my experience the equivalent thing is simpler on the Hurd,
>  esp. if relying on passive translators (see daemons/runsystem.sh in
>  the Hurd), though here we’ll probably explicitly activate
>  translators at boot time.

Currently, there is not really a reason to have an initrd-like
solution on the Hurd.  Yes, one has to start the default pager
explicitly, but that's about it.

> > Finally, once GuixSD is successfully ported, we can cater to the last stage 
> > of taking advantage of Hurd specific 
> > mechanisms. 
> > This includes but is not limited to:
> > 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a 
> > GNU/Hurd system. Subhurds can offer 
> > isolation similar to Linux containers as described here[3].
> 
> This is really super optional IMO.  This module is only used by ‘guix
> environment --container’, which is an optional feature.

Yes, subhurds are more on the experimental side imho.

> > 2)Modify the guix-daemon to run without root privileges by utilizing the 
> > Hurd architecture. The daemon needs root 
> > privileges in order to setup chrooted environments for the builds.  In the 
> > Hurd this can be done by setting up a 
> > translator as described here[4]. 
> 
> This is more important, and already non-trivial work.  If libc provided
> ‘mount’ with support for MS_BIND (which I think it should), it would
> help a bit, but there’s still the problem of the other Linux name spaces
> that are used (the CLONE_NEW* clone(2) flags.)
> 
> Thus it may make more sense to write this functionality in guix-daemon
> using directly the Hurd interfaces.  Separate PID name spaces, UID name
> spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
> “just” a matter of giving the child process ports to separate proc,
> auth, etc. translators.
> 
> In itself this is also a bit of work.  I wonder what the Hurd folks
> think about priorities here?

I'd go for specializing guix-daemon over trying too hard to implement
Linux-compatible interfaces in the libc.  I consider the
filesystem-isolation part easy, UID isolation relatively easy, PID
isolation is a bit tricky.  Currently one cannot simply start another
proc server and expect it to work as it needs privileged kernel
interfaces.  I created an RPC to allow nested proc servers for
unprivileged subhurds.  That should do the trick, it is merely a
matter of making it actually work I guess.

> > 3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]. 
> > Stowfs creates a traditional Unix directory 
> > structure from all the files in the individual package directories.
> 
> Fun but optional.  ;-)

Agreed.

> > May 2 - May 22
> > Create (gnu build hurd-boot) and (gnu build hurd-initrd).
> > Start working on describing the GNU/Hurd system in (gnu system).
> 
> I know Debian at some point added initrd support to GNU Mach for some
> reason, but fundamentally, the whole point of multiboot is to provide a
> solution more flexible than initrds.  So, hopefully, no initrds here.
> Since there’s no initrd, there’s also no ‘switch_root’ dance (on

Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Manolis Ragkousis
Hello Justus, Hello Ludo

On 03/23/16 18:55, Justus Winter wrote:
> Quoting Ludovic Courtès (2016-03-23 14:40:38)
>>> 2. The Project
>>>
>>> The project consists of four main stages
>>>
>>> 1. Modify Guix so it will be able to create and mount the file-system 
>>> needed to boot into a system with Hurd at its core. 
>>> 2. Modify Guix so it can produce a working image, while isolating any cases 
>>> of Linux assumptions.
>>> 3. Successfully boot into one such system using GNU Shepherd with pid 1.
>>> 4. Modify the new Guix system to take advantage of Hurd specific mechanisms.
> 
> For me, 4. is the most important bit, so we can build packages in
> isolation.

I think our priority should be to first get GuixSD working and then
concentrate to achieving isolation.  From what I understand none of the
two is trivial but in the long run the ability to spawn GNU/Hurd system
vms on the fly will make it easier to work on it.

>>> Currently the tools Guix uses to interact with the filesystem exist inside 
>>> the (guix build syscalls) module. 
>>> This module provides bindings to libc's syscall wrappers, which are only 
>>> available on a GNU/Linux system. 
>>> In order to offer the same functionality on a GNU/Hurd system we must first 
>>> write Guile bindings for the 
>>> relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs
>>> for example.
>>
>> Note that technically the ‘file_set_translator’ function is in libc:
>>
>> --8<---cut here---start->8---
>> scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link))
>> $1 = #
>> --8<---cut here---end--->8---
>>
>>> This module will be called (guix build hurd). This allows us to
>>> re-implement the functionality of the 'settrans' command, as described
>>> here[1], in Scheme and use that in place of 'mount()', where
>>> applicable.
> 
> Right.  In fact, what settrans (or mount) does isn't that hard to
> reproduce, though I wouldn't mind moving the c implementation to
> libhurdutil or the like and binding to that.

Then I think it will be wiser to move the c implementation of settrans
to libhurdutil and binding to that.  Then we will only need a (guix
build hurd) module which will include that binding.  I will update the
proposal if you agree.

>> I think it’s important to think about:
>>
>>   1. How functionality equivalent to linux-{initrd,boot} will be
>>  implemented on the Hurd.
>>
>>  In my experience the equivalent thing is simpler on the Hurd,
>>  esp. if relying on passive translators (see daemons/runsystem.sh in
>>  the Hurd), though here we’ll probably explicitly activate
>>  translators at boot time.
> 
> Currently, there is not really a reason to have an initrd-like
> solution on the Hurd.  Yes, one has to start the default pager
> explicitly, but that's about it.

Then it seems we only need a hurd-boot module which will take care of
activating translators and booting the system.

>   2. How to structure GuixSD code in a way that abstracts the
> underlying OS kernel.
>
>  For instance, we could have a (guix build os) module providing an
>  API that works for both kernels, probably close to the Linux one,
>  and that would dispatch to the right implementation, the Linux or
>  Hurd one.

This module sounds like a great idea. I will add it to the proposal.


>>> Finally, once GuixSD is successfully ported, we can cater to the last stage 
>>> of taking advantage of Hurd specific 
>>> mechanisms. 
>>> This includes but is not limited to:
>>> 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a 
>>> GNU/Hurd system. Subhurds can offer 
>>> isolation similar to Linux containers as described here[3].
>>
>> This is really super optional IMO.  This module is only used by ‘guix
>> environment --container’, which is an optional feature.
> 
> Yes, subhurds are more on the experimental side imho.

So Adding subhurd support -> if there is time.
> 
>>> 2)Modify the guix-daemon to run without root privileges by utilizing the 
>>> Hurd architecture. The daemon needs root 
>>> privileges in order to setup chrooted environments for the builds.  In the 
>>> Hurd this can be done by setting up a 
>>> translator as described here[4]. 
>>
>> This is more important, and already non-trivial work.  If libc provided
>> ‘mount’ with support for MS_BIND (which I think it should), it would
>> help a bit, but there’s still the problem of the other Linux name spaces
>> that are used (the CLONE_NEW* clone(2) flags.)
>>
>> Thus it may make more sense to write this functionality in guix-daemon
>> using directly the Hurd interfaces.  Separate PID name spaces, UID name
>> spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
>> “just” a matter of giving the child process ports to separate proc,
>> auth, etc. translators.
>>
>> In itself this is also a bit of work.  I wonder what the Hurd folks
>> think about priorities here?

Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Justus Winter
Quoting Manolis Ragkousis (2016-03-24 12:18:25)
> >>> The project consists of four main stages
> >>>
> >>> 1. Modify Guix so it will be able to create and mount the file-system 
> >>> needed to boot into a system with Hurd at its core. 
> >>> 2. Modify Guix so it can produce a working image, while isolating any 
> >>> cases of Linux assumptions.
> >>> 3. Successfully boot into one such system using GNU Shepherd with pid 1.
> >>> 4. Modify the new Guix system to take advantage of Hurd specific 
> >>> mechanisms.
> > 
> > For me, 4. is the most important bit, so we can build packages in
> > isolation.
> 
> I think our priority should be to first get GuixSD working and then
> concentrate to achieving isolation.  From what I understand none of the
> two is trivial but in the long run the ability to spawn GNU/Hurd system
> vms on the fly will make it easier to work on it.

Otoh if we could properly build packages, we could provide the
substitutes, so people could try Guix on the Hurd without going
through the 12h+ bootstrap procedure.

> Currently I am working on familiarizing myself with both how Hurd and
> GuixSD booting works. I am already using GuixSD as a system and I am
> going though the source code so I can better understand what is going on.

Here is an overview of the early server bootstrap in the Hurd.  It is
slightly outdated, but still the best description that I know of:

http://teythoon.cryptobitch.de/posts/bootstrapping-the-hurd/

> >> For debugging purposes, it would be very helpful to say the least to
> >> have a working ‘guix system vm’: it would allow you to test your changes
> >> very quickly, without rebooting and so on.
> >>
> >> This in itself requires some thought: currently (guix system vm) relies
> >> on QEMU’s ‘-kernel’ option to boot the kernel Linux.  For GNU/Hurd, we
> >> instead need to boot a complete image with GRUB, like ‘guix system
> >> vm-image’ does.  You’ll have to investigate how to port this.
> > 
> > qemu can boot multiboot operating systems.
> 
> Justus is correct here. I found this
> https://www.gnu.org/software/hurd/hurd/running/qemu.html#index9h1 which
> explains how to make qemu start with gnumach.  This way guix system vm
> can work, just with the proper modifications.

Hmmm, so one still needs a filesystem, right?  That's going to be a
bit tricky too, since whatever tool you use for that purpose, it
surely does not support creating hurdish passive translator records.
Without passive translator records things get indeed more interesting.
I have a patch for some tool for creating ext2 filesystems that could
help, or we create all the passive translator records on first boot
similar to how Samuels Debian/Hurd live cds deal with that.  Or I
finish my bootshell work that can boot from filesystems without
passive translator records.

Justus



Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Manolis Ragkousis
On 03/24/16 13:38, Justus Winter wrote:
> Otoh if we could properly build packages, we could provide the
> substitutes, so people could try Guix on the Hurd without going
> through the 12h+ bootstrap procedure.

Here the problem is not isolation but the fact that we don't have a
substitutes server.  The machine I had at my house crashed and I cannot
bring it back up remotely. When I get back to my home island and revive
it, this will not be an issue.

> Here is an overview of the early server bootstrap in the Hurd.  It is
> slightly outdated, but still the best description that I know of:
> 
> http://teythoon.cryptobitch.de/posts/bootstrapping-the-hurd/
>

It definitely is!! Nice one!!

> Hmmm, so one still needs a filesystem, right?  That's going to be a
> bit tricky too, since whatever tool you use for that purpose, it
> surely does not support creating hurdish passive translator records.
> Without passive translator records things get indeed more interesting.
> I have a patch for some tool for creating ext2 filesystems that could
> help, or we create all the passive translator records on first boot
> similar to how Samuels Debian/Hurd live cds deal with that.  Or I
> finish my bootshell work that can boot from filesystems without
> passive translator records.

Where can I find more info on those?

Manolis




Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Justus Winter
Quoting Manolis Ragkousis (2016-03-24 13:36:04)
> > Hmmm, so one still needs a filesystem, right?  That's going to be a
> > bit tricky too, since whatever tool you use for that purpose, it
> > surely does not support creating hurdish passive translator records.
> > Without passive translator records things get indeed more interesting.
> > I have a patch for some tool for creating ext2 filesystems that could
> > help, or we create all the passive translator records on first boot
> > similar to how Samuels Debian/Hurd live cds deal with that.  Or I
> > finish my bootshell work that can boot from filesystems without
> > passive translator records.
> 
> Where can I find more info on those?

https://www.gnu.org/software/hurd/hurd-paper.html#translator

Passive translator records are stored in ext2 using the Hurd on-disk
format:

http://git.sceen.net/hurd/hurd.git/blob/HEAD:/ext2fs/ext2_fs.h#l252

Justus



Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Ludovic Courtès
Hi Justus,

Justus Winter <4win...@informatik.uni-hamburg.de> skribis:

> Quoting Ludovic Courtès (2016-03-23 14:40:38)
>> > 2. The Project
>> >
>> > The project consists of four main stages
>> >
>> > 1. Modify Guix so it will be able to create and mount the file-system 
>> > needed to boot into a system with Hurd at its core. 
>> > 2. Modify Guix so it can produce a working image, while isolating any 
>> > cases of Linux assumptions.
>> > 3. Successfully boot into one such system using GNU Shepherd with pid 1.
>> > 4. Modify the new Guix system to take advantage of Hurd specific 
>> > mechanisms.
>
> For me, 4. is the most important bit, so we can build packages in
> isolation.

I guess you mean isolation in guix-daemon.

>> This is more important, and already non-trivial work.  If libc provided
>> ‘mount’ with support for MS_BIND (which I think it should), it would
>> help a bit, but there’s still the problem of the other Linux name spaces
>> that are used (the CLONE_NEW* clone(2) flags.)
>> 
>> Thus it may make more sense to write this functionality in guix-daemon
>> using directly the Hurd interfaces.  Separate PID name spaces, UID name
>> spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
>> “just” a matter of giving the child process ports to separate proc,
>> auth, etc. translators.
>> 
>> In itself this is also a bit of work.  I wonder what the Hurd folks
>> think about priorities here?
>
> I'd go for specializing guix-daemon over trying too hard to implement
> Linux-compatible interfaces in the libc.

So this would become maybe half of the GSoC coding part maybe.  WDYT?

> I consider the filesystem-isolation part easy, UID isolation
> relatively easy, PID isolation is a bit tricky.  Currently one cannot
> simply start another proc server and expect it to work as it needs
> privileged kernel interfaces.  I created an RPC to allow nested proc
> servers for unprivileged subhurds.  That should do the trick, it is
> merely a matter of making it actually work I guess.

“Merely”, hmm…  :-)

So, let’s say PID isolation will be optional, and we can always adjust
later on.  Sounds good?

Manolis, make sure to read about how the various Hurd servers provides
these parts of POSIX personality: file systems, UIDs, PIDs, networking,
and so on.

As far as code integration code, I think we won’t bother syncing this
work with nix-daemon; guix-daemon has already diverged, and for instance
it does not have OS X sandbox support.

You’ll have to arrange to have the Hurd-specific bits in a separate
file, so that ‘#ifdef HURD’ are not scattered all over the place.

This is C(++) as you know.  WDYT, Manolis?

>> This in itself requires some thought: currently (guix system vm) relies
>> on QEMU’s ‘-kernel’ option to boot the kernel Linux.  For GNU/Hurd, we
>> instead need to boot a complete image with GRUB, like ‘guix system
>> vm-image’ does.  You’ll have to investigate how to port this.
>
> qemu can boot multiboot operating systems.

Great, didn’t know that.

>> > Deliver a working GuixSD system image with Hurd as the kernel.
>
> Hurd is not a kernel.

I think we use “kernel” to mean “the thing(s) that libc talks to.”

>> The main question is whether you should implement build isolation in
>> guix-daemon, in which case that would leave little time for the GuixSD
>> parts.  I think I would rather let you focus on the GuixSD stuff as you
>> wrote, but I’d like to hear what the Hurd folks think.
>
> I consider isolation more important.

OK.

So, Manolis, what about reframing the agenda such that porting
guix-daemon to GNU/Hurd comes first (I’d consider it roughly half of the
programming effort), followed by GuixSD stuff?

Thanks,
Ludo’.



Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Manolis Ragkousis
Hey

On 03/24/16 15:22, Ludovic Courtès wrote:
> So, let’s say PID isolation will be optional, and we can always adjust
> later on.  Sounds good?
> 
> Manolis, make sure to read about how the various Hurd servers provides
> these parts of POSIX personality: file systems, UIDs, PIDs, networking,
> and so on.

Already started. :-)

> As far as code integration code, I think we won’t bother syncing this
> work with nix-daemon; guix-daemon has already diverged, and for instance
> it does not have OS X sandbox support.
> 
> You’ll have to arrange to have the Hurd-specific bits in a separate
> file, so that ‘#ifdef HURD’ are not scattered all over the place.
> 
> This is C(++) as you know.  WDYT, Manolis?

I have to start studying the daemon's code more. From what I know the
part that handles builds is in libstore/build.cc. I will start there.

I think I can do it.

>>> The main question is whether you should implement build isolation in
>>> guix-daemon, in which case that would leave little time for the GuixSD
>>> parts.  I think I would rather let you focus on the GuixSD stuff as you
>>> wrote, but I’d like to hear what the Hurd folks think.
>>
>> I consider isolation more important.
> 
> OK.

Isolation first it is then.

> So, Manolis, what about reframing the agenda such that porting
> guix-daemon to GNU/Hurd comes first (I’d consider it roughly half of the
> programming effort), followed by GuixSD stuff?
> 

Current objectives then:
1) Achieve build isolation in the daemon on the Hurd.
2) Modify Guix so it can produce a working image, while isolating any
cases of Linux assumptions.
3) Boot to GuixSD

Ludo, Justus do you agree with this?

Manolis





Re: [GSoC] Porting GuixSD to GNU Hurd draft

2016-03-24 Thread Justus Winter
Hi,

Quoting Manolis Ragkousis (2016-03-24 14:55:31)
> On 03/24/16 15:22, Ludovic Courtès wrote:
> >>> The main question is whether you should implement build isolation in
> >>> guix-daemon, in which case that would leave little time for the GuixSD
> >>> parts.  I think I would rather let you focus on the GuixSD stuff as you
> >>> wrote, but I’d like to hear what the Hurd folks think.
> >>
> >> I consider isolation more important.
> > 
> > OK.
> 
> Isolation first it is then.
> 
> > So, Manolis, what about reframing the agenda such that porting
> > guix-daemon to GNU/Hurd comes first (I’d consider it roughly half of the
> > programming effort), followed by GuixSD stuff?
> > 
> 
> Current objectives then:
> 1) Achieve build isolation in the daemon on the Hurd.
> 2) Modify Guix so it can produce a working image, while isolating any
> cases of Linux assumptions.
> 3) Boot to GuixSD

Well, I didn't ment to stomp over your priorities like that, it is just
that I personally consider it more important and from my point of view
it should be easier to achieve.

Justus