Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

On 18.06.23 22:39, TOMAS FABRIZIO ORSI wrote:
Hello gentoo devs. The other day I made a feature suggestion to the 
eselect repository github page. (Here's the link: 
https://github.com/projg2/eselect-repository/issues/20 
).
Michał Górny suggested that I should contact the gentoo-dev mailing list 
with my suggestion, so here it is: My suggestion is to make it possible 
for eselect repository to manage overlay dependencies.


As it stands, and as far as I'm concerned, there is no "proper" way of 
having an ebuild dependency from another overlay. So overlay writers 
have to copy ebuilds from other overlays or rewrite them. This implies 
synchronization issues (where the "copied" ebuilds get out of sync from 
the original ebuild) and time loss (people writing the same ebuild more 
than once).


Michał Górny suggested that I should make an edit to the 
repositories.xml file in order to tackle the issue.


This is my general idea:
(I am using the github template as an example, but the idea should apply 
to all other templates. I got this file from 
https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml ).


   * GITHUB TEMPLATE
     
       X
       XX
       https://github.com// 


       
         X
         X
       
       https://github.com//.git 

       git+ssh://g...@github.com//.git 

       https://github.com///commits/master.atom 


       
           overlayName
       
     


Isn't that duplicating the information of metadata/layout.conf's 
'master' key-value pair [1]?


- Flow

1: 
https://wiki.gentoo.org/wiki/Repository_format/metadata/layout.conf#masters


OpenPGP_0x8CAC2A9678548E35.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

>
> Isn't that duplicating the information of metadata/layout.conf's
> 'master' key-value pair [1]?
>

Yes, I agree that it would be duplicating that information. As a matter of
fact, Michał Górny pointed the same thing out.
However, Michał also added, quote: "What's really lacking here is support
for specifying dependencies via repositories.xml. Once such a thing is
implemented, We can look into having eselect-repository handle that once
it's specified." (Here's the link to the conversation:
https://github.com/projg2/eselect-repository/issues/20#issuecomment-1579098528
)
That's why I added the  line in the repositories.xml template.

Having said that, maybe the information present in the masters key could be
used to add the overlay dependencies in the repositories present in the xml
file as of today.

Thank you for your reply,

--
Best regards,
- Tomas Fabrizio Orsi


El mar, 20 jun 2023 a las 10:44, Florian Schmaus ()
escribió:

> On 18.06.23 22:39, TOMAS FABRIZIO ORSI wrote:
> > Hello gentoo devs. The other day I made a feature suggestion to the
> > eselect repository github page. (Here's the link:
> > https://github.com/projg2/eselect-repository/issues/20
> > ).
> > Michał Górny suggested that I should contact the gentoo-dev mailing list
> > with my suggestion, so here it is: My suggestion is to make it possible
> > for eselect repository to manage overlay dependencies.
> >
> > As it stands, and as far as I'm concerned, there is no "proper" way of
> > having an ebuild dependency from another overlay. So overlay writers
> > have to copy ebuilds from other overlays or rewrite them. This implies
> > synchronization issues (where the "copied" ebuilds get out of sync from
> > the original ebuild) and time loss (people writing the same ebuild more
> > than once).
> >
> > Michał Górny suggested that I should make an edit to the
> > repositories.xml file in order to tackle the issue.
> >
> > This is my general idea:
> > (I am using the github template as an example, but the idea should apply
> > to all other templates. I got this file from
> >
> https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml
> <
> https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml
> >).
> >
> >* GITHUB TEMPLATE
> >  
> >X
> >XX
> >https://github.com//
> > 
> >
> >  X
> >  X
> >
> >https://github.com//.git
> > 
> >git+ssh://g...@github.com//.git
> > 
> >https://github.com///commits/master.atom
> > 
> >
> >overlayName
> >
> >  
>
> Isn't that duplicating the information of metadata/layout.conf's
> 'master' key-value pair [1]?
>
> - Flow
>
> 1:
> https://wiki.gentoo.org/wiki/Repository_format/metadata/layout.conf#masters
>

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:

Isn't that duplicating the information of metadata/layout.conf's
'master' key-value pair [1]?


Yes, I agree that it would be duplicating that information. As a matter 
of fact, Michał Górny pointed the same thing out.
However, Michał also added, quote: "What's really lacking here is 
support for specifying dependencies via |repositories.xml|


Do we need to duplicate the information in repositories.xml, with all 
the drawbacks of duplication?


Can't eselect repository add the new repository, then read the 'masters' 
value from layout.conf, and add the missing repositories recursively?


- Flow


OpenPGP_0x8CAC2A9678548E35.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus  wrote:
>
> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> > Isn't that duplicating the information of metadata/layout.conf's
> > 'master' key-value pair [1]?
> >
> >
> > Yes, I agree that it would be duplicating that information. As a matter
> > of fact, Michał Górny pointed the same thing out.
> > However, Michał also added, quote: "What's really lacking here is
> > support for specifying dependencies via |repositories.xml|
>
> Do we need to duplicate the information in repositories.xml, with all
> the drawbacks of duplication?
>
> Can't eselect repository add the new repository, then read the 'masters'
> value from layout.conf, and add the missing repositories recursively?

That would be a significant change in behavior for eselect repository.
Currently, it does not actually sync any repos; it just manages the
config in /etc/portage/repos.conf.

Re: [gentoo-dev] Eselect repository feature request

[Andrew Ammerlaan]

On 20/06/2023 19:26, Mike Gilbert wrote:

On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus  wrote:


On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:

 Isn't that duplicating the information of metadata/layout.conf's
 'master' key-value pair [1]?


Yes, I agree that it would be duplicating that information. As a matter
of fact, Michał Górny pointed the same thing out.
However, Michał also added, quote: "What's really lacking here is
support for specifying dependencies via |repositories.xml|


Do we need to duplicate the information in repositories.xml, with all
the drawbacks of duplication?

Can't eselect repository add the new repository, then read the 'masters'
value from layout.conf, and add the missing repositories recursively?


That would be a significant change in behavior for eselect repository.
Currently, it does not actually sync any repos; it just manages the
config in /etc/portage/repos.conf.

Or maybe we just output a warning after syncing if a repository listed 
in masters= is not enabled? Less automated, but simpler. I don't think 
this situation happens very often so this is probably good enough IMO.


Best regards,
Andrew

Re: [gentoo-dev] [PATCH 2/2 v4] kernel-build.eclass: add USE="modules-sign"

[Andrew Ammerlaan]

Version 4 (and that's the last one, I promise), makes this work with 
pkcs11 uri's as well. Tested with my Nitrokey, it is (unsurprisingly) 
incredibly slow but it works.




From 70415544a4aea458039f1abbbf9c7e112de846f3 Mon Sep 17 00:00:00 2001
From: Andrew Ammerlaan 
Date: Thu, 15 Jun 2023 21:10:02 +0200
Subject: [PATCH] kernel-build.eclass: add IUSE="modules-sign"

- Enable module signing configure options if requested by the user.

- Define the user variables MODULES_SIGN_HASH and MODULES_SIGN_KEY.
For controlling the used hashing algorithm and allowing the use of
external keys. These variables are the same as in linux-mod-r1.eclass

- Warn the user if we are letting the kernel build system generate the 
signing
key. This key will end up binary packages. Plus external modules will 
have to

be resigned if gentoo-kernel is re-emerged (i.e. a new key was generated).

Bug: https://bugs.gentoo.org/814344
Bug: https://bugs.gentoo.org/881651
Signed-off-by: Andrew Ammerlaan 
---
 eclass/kernel-build.eclass | 90 +-
 1 file changed, 89 insertions(+), 1 deletion(-)

diff --git a/eclass/kernel-build.eclass b/eclass/kernel-build.eclass
index abfb01720817a..7d4e2133a04d2 100644
--- a/eclass/kernel-build.eclass
+++ b/eclass/kernel-build.eclass
@@ -43,6 +43,48 @@ BDEPEND="

 IUSE="+strip"

+# @ECLASS_VARIABLE: KERNEL_IUSE_MODULES_SIGN
+# @PRE_INHERIT
+# @DEFAULT_UNSET
+# @DESCRIPTION:
+# If set to a non-null value, adds IUSE=modules-sign and required
+# logic to manipulate the kernel config while respecting the
+# MODULES_SIGN_HASH and MODULES_SIGN_KEY user variables.
+
+# @ECLASS_VARIABLE: MODULES_SIGN_HASH
+# @USER_VARIABLE
+# @DEFAULT_UNSET
+# @DESCRIPTION:
+# Used with USE=modules-sign.  Can be set to hash algorithm to use
+# during signature generation (CONFIG_MODULE_SIG_SHA256).
+#
+# Valid values: sha512,sha384,sha256,sha224,sha1
+#
+# Default if unset: sha512
+
+# @ECLASS_VARIABLE: MODULES_SIGN_KEY
+# @USER_VARIABLE
+# @DEFAULT_UNSET
+# @DESCRIPTION:
+# Used with USE=modules-sign.  Can be set to the path of the private
+# key in PEM format to use, or a PKCS#11 URI (CONFIG_MODULE_SIG_KEY).
+#
+# If path is relative (e.g. "certs/name.pem"), it is assumed to be
+# relative to the kernel build directory being used.
+#
+# If the key requires a passphrase or PIN, the used kernel sign-file
+# utility recognizes the KBUILD_SIGN_PIN environment variable.  Be
+# warned that the package manager may store this value in binary
+# packages, database files, temporary files, and possibly logs.  This
+# eclass unsets the variable after use to mitigate the issue (notably
+# for shared binary packages), but use this with care.
+#
+# Default if unset: certs/signing_key.pem
+
+if [[ ${KERNEL_IUSE_MODULES_SIGN} ]]; then
+   IUSE+=" modules-sign"
+fi
+
 # @FUNCTION: kernel-build_src_configure
 # @DESCRIPTION:
 # Prepare the toolchain for building the kernel, get the default .config
@@ -259,6 +301,9 @@ kernel-build_src_install() {
dosym "../../../${kernel_dir}" "/lib/modules/${module_ver}/build"
dosym "../../../${kernel_dir}" "/lib/modules/${module_ver}/source"

+   # unset to at least be out of the environment file in, e.g. shared 
binpkgs
+   unset KBUILD_SIGN_PIN
+
save_config build/.config
 }

@@ -268,6 +313,26 @@ kernel-build_src_install() {
 kernel-build_pkg_postinst() {
kernel-install_pkg_postinst
savedconfig_pkg_postinst
+
+   if [[ ${KERNEL_IUSE_MODULES_SIGN} ]]; then
+   if use modules-sign && [[ -z ${MODULES_SIGN_KEY} ]]; then
+   ewarn
+   ewarn "MODULES_SIGN_KEY was not set, this means the kernel 
build system"
+   ewarn "automatically generated the signing key. This key was 
installed"
+   ewarn "in 
${EROOT}/usr/src/linux-${PV}${KV_LOCALVERSION}/certs"
+   ewarn "and will also be included in any binary 
packages."
+   ewarn "Please take appropriate action to protect the 
key!"
+   ewarn
+   ewarn "Recompiling this package causes a new key to be 
generated. As"
+   ewarn "a result any external kernel modules will need to be 
resigned."
+   ewarn "Use emerge @module-rebuild, or manually sign the 
modules as"
+   ewarn "described on the wiki [1]"
+   ewarn
+			ewarn "Consider using the MODULES_SIGN_KEY variable to use an 
external key."

+   ewarn
+   ewarn "[1]: 
https://wiki.gentoo.org/wiki/Signed_kernel_module_support";
+   fi
+   fi
 }

 # @FUNCTION: kernel-build_merge_configs
@@ -290,16 +355,39 @@ kernel-build_merge_configs() {
local user_configs=( "${BROOT}"/etc/kernel/config.d/*.config )
shopt -u nullglob

+   local merge_configs=( "${@}" )
+
+   if [[ ${KERNEL_IUSE_MODULES_SIGN} ]]; then
+

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

A warning could be a great way of making the user aware of this situation.
Having said that, if eselect repository is able to check and warn the user
of a not synced overlay(ies) dependency, then the hard bit is done (Right?).
Being able to "sync it for them" should only be a tiny extra step. That
step being: "Hey, there's a dependency that you are missing. Would you like
to add it? (Yes/No)"

Best regards,
- Tomas Fabrizio Orsi


El mar, 20 jun 2023 a las 15:07, Andrew Ammerlaan (<
andrewammerl...@gentoo.org>) escribió:

> On 20/06/2023 19:26, Mike Gilbert wrote:
> > On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus  wrote:
> >>
> >> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >>>  Isn't that duplicating the information of metadata/layout.conf's
> >>>  'master' key-value pair [1]?
> >>>
> >>>
> >>> Yes, I agree that it would be duplicating that information. As a matter
> >>> of fact, Michał Górny pointed the same thing out.
> >>> However, Michał also added, quote: "What's really lacking here is
> >>> support for specifying dependencies via |repositories.xml|
> >>
> >> Do we need to duplicate the information in repositories.xml, with all
> >> the drawbacks of duplication?
> >>
> >> Can't eselect repository add the new repository, then read the 'masters'
> >> value from layout.conf, and add the missing repositories recursively?
> >
> > That would be a significant change in behavior for eselect repository.
> > Currently, it does not actually sync any repos; it just manages the
> > config in /etc/portage/repos.conf.
> >
> Or maybe we just output a warning after syncing if a repository listed
> in masters= is not enabled? Less automated, but simpler. I don't think
> this situation happens very often so this is probably good enough IMO.
>
> Best regards,
> Andrew
>
>