Re: Should commits rather be buildable or small

[Liliana Marie Prikler]

Hi,

Am Montag, dem 04.03.2024 um 21:38 + schrieb John Kehayias:
> [...]
> 1. Essentially squash to one commit where all of vulkan is updated in
> one commit. The main upside is that nothing should break (within
> vulkan, dependents to be fixed as needed) and it shows as "one"
> change; the main downside is that the proposed changes are not just
> trivial version bumps. Harder to then disentangle as needed.
> 
> 2. Make each commit updating a package, but don't use the variable
> %vulkan-sdk-version, updating each package with a version as it is
> done. Then do a commit where all the versions are replaced by the
> variable. This seems like unnecessary work to me and while it stops
> the obvious breaking (source hashes don't match once variable is
> updated but package hasn't yet) versions are still mixed which is
> likely a problem.
> 
> 3. Go with the series as proposed: this means after the first commit
> for sure all other vulkan packages and dependents don't build, as the
> source hashes won't match until the commit that updates that package.
> Along with version mixing, this perhaps doesn't give you a helpful
> git bisect either?
> 
> None are perfect. What do people think?
I think 1 would be workable if the changes to the packages are minimal.
You should also check whether you can just do the version bumps and
then the other changes – or flip the order.

I don't really see the benefit with 2.  Normally, we'd have "-next"
variants to catch nontrivial updates (among other things), but those
don't seem a good approach here.

If nothing else works, 3 is indeed an option to fall back to, albeit
begrudgingly.  As noted for 1, you could check whether bumping all the
hashes and then only fixing whatever else for the builds is an option
here.

Alternative 4 would be to build those -next variants and then replace
the base vulkan all at once.  This has the advantage of not doing any
version mixing in-between IIUC.


Cheers

Re: Should commits rather be buildable or small

[dan]

Hi John,

On 3/5/2024 5:38 AM, John Kehayias wrote:

In this case all the vulkan packages share a version through a variable name. I 
would assume packages wouldn't like mixed versions, but maybe some would work 
(I haven't tried). I'll be taking this series on mesa-updates with related 
changes, so the plan is that when it hits master there are no/few broken 
packages and full substitute coverage. So perhaps this makes this more of a 
style and convention question.

Some options:

1. Essentially squash to one commit where all of vulkan is updated in one commit. The 
main upside is that nothing should break (within vulkan, dependents to be fixed as 
needed) and it shows as "one" change; the main downside is that the proposed 
changes are not just trivial version bumps. Harder to then disentangle as needed.

2. Make each commit updating a package, but don't use the variable 
%vulkan-sdk-version, updating each package with a version as it is done. Then 
do a commit where all the versions are replaced by the variable. This seems 
like unnecessary work to me and while it stops the obvious breaking (source 
hashes don't match once variable is updated but package hasn't yet) versions 
are still mixed which is likely a problem.

3. Go with the series as proposed: this means after the first commit for sure 
all other vulkan packages and dependents don't build, as the source hashes 
won't match until the commit that updates that package. Along with version 
mixing, this perhaps doesn't give you a helpful git bisect either?

None are perfect. What do people think?

My instinct is to go with the series as proposed (after review) accepting that 
there will be for sure builds failing if time traveling to the middle of the 
series. I don't think we can really avoid that anyway, as sometimes we only see 
an issue after a commit and it is fixed some time later. We could have a note 
in the first commit that this requires the next n commits to update vulkan 
packages. That might help if someone is on an intermediate commit and can see 
quickly in git log this note.

Or perhaps we can note something is part of a dependent series when we make 
commits so this is easier for someone to tell in general?


I think to make each commit able to build, it's feasible to remove this 
%vulkan-sdk-version variable. However, this doesn't fundamentally solve 
the problem: when updating several packages in a patch series, some 
packages might be broken since their dependencies are updated.


Another question is how should we treat vulkan packages. Some distros 
package them on a per package basis (I see in Arch Linux, vulkan-headers 
and vulkan-icd-loaders have version 1.3.276 while other packages like 
spirv-headers has 1.3.275). I had to admit that I'm not that familiar 
with vulkan packages, but I feel it's safer to keep their version 
matched since each vulkan-sdk release makes sure every vulkan packages 
are compatible with others. Thus, I prefer updating them in batch.


I think maybe it's a good option that we mark these commits are a series.

--
dan

Re: cmake-build-system: build tests only if #:tests? is true?

[Hartmut Goebel]

Hi Greg,

I will submit a patchset shortly for review and if accepted we can
look to combine all the cmake patches for the large rebuild.


Yes, of course we should combine them. May I ask you to take care of it 
and include mine, which I just posted:


https://issues.guix.gnu.org/issue/69554

Many thanks in advance.

--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: GSoC 2024

[Pjotr Prins]

Hi Gábor,

https://libreplanet.org/wiki/Group:Guix/GSoC-2024

and added a project. Feel free to modify.

For the others: you can visit the older ideas pages - there is also
commented out projects in some of them. If you are looking for yours.

We still have a few days, but best to get it done this week. Sounds
like we should propose a goblins project!

PJ.

On Mon, Mar 04, 2024 at 07:10:34PM +, Gábor Boskovits wrote:
> Hello guix,
> 
> I coordinated with the GNU org admins, and we can still do this round,
> but we have to go fast to make this happen. I have already taken the
> initiative to try to get an ideas page up, now I would like to confirm
> if the mentors from last year are still available, and that the ideas
> are still valid.
> 
> Hereby I quickly collected the projects with the respective mentors,
> please pm me your availability:
> 
> Decentralized substitute distribution
> pukkamustard (pukkamustard [at] posteo [dot] net)
> attila.lendvai (ethswarm.org, scheme)
> 
> Robustify long-term support for Reproducible Research
> Simon Tournier (zimoun)
> 
> Develop a Web interface to configure Guix System
> Ludovic Courtès (civodul)
> 
> Trusted computing: Goblins for GNU Guix
> Christopher Webber, Ludovic Courtès and Pjotr Prins
> 
> Guix Data Service revision processing instrumentation and performance
> Christopher Baines
> 
> Guile based build-tool
> Pjotr Prins
> 
> GNU Guix system monitor
> Pjotr Prins
> 
> Booting via network
> Danny Milosavljevic
> 
> Syntax and semantics of systemd units in the Shepherd
> Ludovic Courtès (civodul)
> 
> GNUnet integration
> no mentor available
> 
> Adding modules in support of continuous integration to cuirass
> Ludovic Courtès (civodul)
> 
> Continue rewrite build daemon in Guile Scheme
> Ludovic Courtès (civodul)
> 
> I myself am available to co-mentor, and also to be the formal mentor
> in case someone does not feel like doing the official dance with
> Google. Currently I can commit to devoting two hours a week to this.
> 
> Regards,
> g_bor
> 

-- 

Re: Should commits rather be buildable or small

[John Kehayias]

Hi everyone,

And sorry for reviving an old thread, but I am faced with a similar issue for 
updating vulkan, with the patch series submitted by dan (cc'ed): 
. I thought I would get some opinions here, 
please see below:

On Mon, Dec 11, 2023 at 12:51 PM, Ricardo Wurmus wrote:

> Attila Lendvai  writes:
>
>> i myself also had headaches multiple times when i fixed something that
>> needed to touch several different packages, and they would only work
>> when applied in one transaction:
>>

In this case all the vulkan packages share a version through a variable name. I 
would assume packages wouldn't like mixed versions, but maybe some would work 
(I haven't tried). I'll be taking this series on mesa-updates with related 
changes, so the plan is that when it hits master there are no/few broken 
packages and full substitute coverage. So perhaps this makes this more of a 
style and convention question.

Some options:

1. Essentially squash to one commit where all of vulkan is updated in one 
commit. The main upside is that nothing should break (within vulkan, dependents 
to be fixed as needed) and it shows as "one" change; the main downside is that 
the proposed changes are not just trivial version bumps. Harder to then 
disentangle as needed.

2. Make each commit updating a package, but don't use the variable 
%vulkan-sdk-version, updating each package with a version as it is done. Then 
do a commit where all the versions are replaced by the variable. This seems 
like unnecessary work to me and while it stops the obvious breaking (source 
hashes don't match once variable is updated but package hasn't yet) versions 
are still mixed which is likely a problem.

3. Go with the series as proposed: this means after the first commit for sure 
all other vulkan packages and dependents don't build, as the source hashes 
won't match until the commit that updates that package. Along with version 
mixing, this perhaps doesn't give you a helpful git bisect either?

None are perfect. What do people think?

My instinct is to go with the series as proposed (after review) accepting that 
there will be for sure builds failing if time traveling to the middle of the 
series. I don't think we can really avoid that anyway, as sometimes we only see 
an issue after a commit and it is fixed some time later. We could have a note 
in the first commit that this requires the next n commits to update vulkan 
packages. That might help if someone is on an intermediate commit and can see 
quickly in git log this note.

Or perhaps we can note something is part of a dependent series when we make 
commits so this is easier for someone to tell in general?

Thanks!
John

GSoC 2024

[Gábor Boskovits]

Hello guix,

I coordinated with the GNU org admins, and we can still do this round,
but we have to go fast to make this happen. I have already taken the
initiative to try to get an ideas page up, now I would like to confirm
if the mentors from last year are still available, and that the ideas
are still valid.

Hereby I quickly collected the projects with the respective mentors,
please pm me your availability:

Decentralized substitute distribution
pukkamustard (pukkamustard [at] posteo [dot] net)
attila.lendvai (ethswarm.org, scheme)

Robustify long-term support for Reproducible Research
Simon Tournier (zimoun)

Develop a Web interface to configure Guix System
Ludovic Courtès (civodul)

Trusted computing: Goblins for GNU Guix
Christopher Webber, Ludovic Courtès and Pjotr Prins

Guix Data Service revision processing instrumentation and performance
Christopher Baines

Guile based build-tool
Pjotr Prins

GNU Guix system monitor
Pjotr Prins

Booting via network
Danny Milosavljevic

Syntax and semantics of systemd units in the Shepherd
Ludovic Courtès (civodul)

GNUnet integration
no mentor available

Adding modules in support of continuous integration to cuirass
Ludovic Courtès (civodul)

Continue rewrite build daemon in Guile Scheme
Ludovic Courtès (civodul)

I myself am available to co-mentor, and also to be the formal mentor
in case someone does not feel like doing the official dance with
Google. Currently I can commit to devoting two hours a week to this.

Regards,
g_bor

Re: cmake-build-system: build tests only if #:tests? is true?

[Greg Hogan]

On Sat, Mar 2, 2024 at 4:17 PM Hartmut Goebel
 wrote:
>
> Hi,
>
> I found an old and unfinished patch in my pile. It optimizes building
> with cmake by not building the test if "#:tests?" is false. (Basically
> it passes -DBUILD_TESTING=OFF/ON" depending on "#:tests?".)
>
> Is this of interest? Then I would take my time and finish the patch.

Hi Hartmut,

I have been testing a set of patches (since last summer) based on the
following mailing list discussion (from 2021) to rewrite the
cmake-build-system build, check, and install functions to use cmake
utilities rather than depend on the gnu-build-system. This allows for
parallel tests and a package-defined #:generator.
  https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00055.html

I will submit a patchset shortly for review and if accepted we can
look to combine all the cmake patches for the large rebuild. Perhaps a
version update for cmake-bootstrap (v3.24 released August 2022) at the
same time?

Greg

Re: cmake-build-system: build tests only if #:tests? is true?

[Jean-Pierre De Jesus Diaz]

On Sat, Mar 2, 2024 at 10:23 PM Hartmut Goebel
 wrote:
>
> Am 02.03.24 um 22:23 schrieb Ekaitz Zarraga:
> > core-updates?
>
> Yes, many, many packages. Thus of course, this would need to go into
> core-updates.
>
> --
> Regards
> Hartmut Goebel
>
> | Hartmut Goebel  | h.goe...@crazy-compilers.com   |
> | www.crazy-compilers.com | compilers which you thought are impossible |
>
>

I think another patch would be a good candidate for core-updates if
cmake-build-system is being updated:

https://issues.guix.gnu.org/68366

It is also needed for:

https://issues.guix.gnu.org/69476

-- 
Thanks,
Jean-Pierre De Jesus DIAZ
Foundation Devices, Inc.

Re: Building container images with nix2container

[Simon Tournier]

Hi lewo,

On lun., 26 févr. 2024 at 11:09, Antoine Eiche  wrote:

> Does your built images contains several layers?

This had recently been introduced.

0cf75c9b2f23869201144917cea7f6ad49683d3d
AuthorDate: Tue Dec 26 03:54:12 2023 +0300
CommitDate: Mon Jan 8 21:04:44 2024 +0300

> nix2container uses an heuristic to group store paths into layers. The
> goal is to share common layers between images and to avoid full image
> rebuild when only a storepath differs.

Well, I have not followed on which strategy Guix relies.  What is the
one of nix2container?  The one described here:

https://grahamc.com/blog/nix-and-layered-docker-images/

> Do you write the image tarball into your store when you build an image?
>
> nix2container is able to build layers on the fly from the Nix store. The
> goal is to reduce IOs and storage. Instead of writing an image tarball
> into the store, it generates a script which stream layers from store
> paths to the destination (a Docker registry, the Docker deamon, Podman
> or a file).

To my knowledge, this is not implemented in Guix.  And indeed, it could
improve the dance.  Currently, it reads:

docker load < $(guix pack -f docker …)


Cheers,
simon

Re: Contribute or create a channel?

[Andreas Enge]

Am Sat, Mar 02, 2024 at 11:32:37AM +0100 schrieb Hartmut Goebel:
> Maybe using one file per release (and accept duplicate code) would be a
> suitable workaround.

I think that would be okay if you think it will be easier to maintain
(not needing to "roll over" code from an old package at the inheritance
root when it is deleted), assuming that the old packages are removed
from time to time.

Andreas