Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Am Thu, Jun 06, 2024 at 11:23:20AM +0200 schrieb Ludovic Courtès:
> (It’s possible to sidestep that problem for instance by having GitLab-CI
> offload to a runner in a machine that you control, where Guix is
> installed.  But I’m looking for a solution that could work
> out-of-the-box on any GitLab-CI or similar instance.)

It turns out that our Gitlab instance enables docker containers in
privileged mode, so it just happened to work in my case...

Andreas

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Reza Housseini  skribis:

> Ludovic Courtès  writes:
>
>> That’s nice, but unless I’m mistaken, Guix is missing from the image,
>> right?
>>
>> My goal would be to be able to use Guix within the image, so I can have
>> GitLab-CI spawn ‘guix build’ commands (or similar).
>
> Oh sorry this was a misunderstanding from my side. Have you tried to use
> the guix system image command to create a docker container?

I did, but that’s not suitable:

  https://mail.gnu.org/archive/html/help-guix/2024-02/msg00066.html

Ludo’.

Re: Building a Docker image for GitLab-CI

[Reza Housseini]

Ludovic Courtès  writes:

> That’s nice, but unless I’m mistaken, Guix is missing from the image,
> right?
>
> My goal would be to be able to use Guix within the image, so I can have
> GitLab-CI spawn ‘guix build’ commands (or similar).

Oh sorry this was a misunderstanding from my side. Have you tried to use
the guix system image command to create a docker container?

Best,
Reza

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi,

Andreas Enge  skribis:

> Am Tue, Jun 04, 2024 at 01:29:22PM +0200 schrieb Ludovic Courtès:
>> My goal would be to be able to use Guix within the image, so I can have
>> GitLab-CI spawn ‘guix build’ commands (or similar).
>
> with a colleague we have set up such a system. He has started from a Debian
> image and written a docker script to install Guix "manually", which provides
> our base image. Then Gitlab CI creates a new image from a channels file in
> the git repository we want to monitor. The command we use is
> "guix system image -t docker" and not "guix pack";

To be able to do that, you need to have Guix already up and running in
the Docker image you pass to GitLab-CI: that’s the problem I’m trying to
solve.

Or did I misunderstand?

(It’s possible to sidestep that problem for instance by having GitLab-CI
offload to a runner in a machine that you control, where Guix is
installed.  But I’m looking for a solution that could work
out-of-the-box on any GitLab-CI or similar instance.)

Thanks,
Ludo’.

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Hello,

Am Tue, Jun 04, 2024 at 01:29:22PM +0200 schrieb Ludovic Courtès:
> My goal would be to be able to use Guix within the image, so I can have
> GitLab-CI spawn ‘guix build’ commands (or similar).

with a colleague we have set up such a system. He has started from a Debian
image and written a docker script to install Guix "manually", which provides
our base image. Then Gitlab CI creates a new image from a channels file in
the git repository we want to monitor. The command we use is
"guix system image -t docker" and not "guix pack"; I am not really familiar
with the second command, and as far as I understand the first one has the
difference of running the shepherd as the docker command, which in our case
starts the guix-daemon and a guix-build-coordinator-agent.
(One addition: We use the patch in
   https://issues.guix.gnu.org/70933
to enable chroot in the container and then run the container in privileged
mode.)

Maybe we should write up a little blog post once everything is settled.

Andreas

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi Reza,

Reza Housseini  skribis:

>> Has anyone succeeded in building a Docker image suitable for use in
>> GitLab-CI?
>
> I normally do the following and it seems to work fine with our gitlab
> instance:
>
> registry=registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds
> archive=$(guix time-machine -C channels.scm -- pack -f docker -S /bin=bin -S 
> /lib=lib -S /share=share -m manifest.scm)
> tag=$(docker load -i $archive)
> docker tag ${tag##*"Loaded image: "} $registry
> docker push $registry
>
> what seems to be crucial is to add the following packages to the
> manifest file:
>
> "bash" 
> "coreutils"
> "git"

That’s nice, but unless I’m mistaken, Guix is missing from the image,
right?

My goal would be to be able to use Guix within the image, so I can have
GitLab-CI spawn ‘guix build’ commands (or similar).

> PS: It would be really nice if one could provide a docker tag directly
> to `guix pack -f docker`

I think it’s already possible, see ‘guix pack --help-docker-format’.

Thanks,
Ludo’.

Re: Building a Docker image for GitLab-CI

[Reza Housseini]

Ludovic Courtès  writes:

sorry forgot to include the list...

Hi Ludo

> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?

I normally do the following and it seems to work fine with our gitlab
instance:

registry=registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds
archive=$(guix time-machine -C channels.scm -- pack -f docker -S /bin=bin -S 
/lib=lib -S /share=share -m manifest.scm)
tag=$(docker load -i $archive)
docker tag ${tag##*"Loaded image: "} $registry
docker push $registry

what seems to be crucial is to add the following packages to the
manifest file:

"bash" 
"coreutils"
"git"

than I can use the image in the gitlab-ci.yml file

generate-exercises:
  image: registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds:latest
  ...

Hope that helps!

Best,
Reza

PS: It would be really nice if one could provide a docker tag directly
to `guix pack -f docker`

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi,

Efraim Flashner  skribis:

> In the past I used a script to install guix using the shell script and
> then ran guix pull before building my package.  I suppose you could use
> a Debian image and run 'guix pull' first before building something.

I could… but that’d be cheating.  :-)

Ludo’.

Re: Building a Docker image for GitLab-CI

[Efraim Flashner]

On Tue, Feb 13, 2024 at 11:31:28AM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?  I haven’t.  Here’s what I tried.

In the past I used a script to install guix using the shell script and
then ran guix pull before building my package.  I suppose you could use
a Debian image and run 'guix pull' first before building something.

-- 
Efraim Flashner  רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Am Tue, Feb 13, 2024 at 11:31:28AM +0100 schrieb Ludovic Courtès:
> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?  I haven’t.  Here’s what I tried.

A colleague of mine just found this:
   https://gitlab.com/daym/guix-on-docker/

:-)

Andreas

Re: Building a Docker image for GitLab-CI

[Suhail]

Ludovic Courtès  writes:

> Initially, I built an image with ‘guix system image -t docker …’ but
> that doesn’t work because then the image’s “entry point” is shepherd,
> but shepherd never returns.

Did you try resetting the entrypoint in .gitlab-ci.yml using the
image:entrypoint keyword? [1]

[1]: 

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.