Re: [arch-dev-public] Consensus: DKMS modules

[Bartłomiej Piotrowski]

On 2016-04-18 02:24, Sébastien Luttringer wrote:
> On jeu., 2016-04-14 at 18:54 +0200, Andreas Radke wrote:
>> Am Thu, 14 Apr 2016 13:29:33 +0200
>> schrieb Ike Devolder :
>>  
>> I'm for  binary modules for our -ARCH main kernel pkg. I see no real
>> need for -lts modules but if there're a few people who find them
>> useful I can handle the kernel rebuilds.
>>
>> No opinion about dkms at all. DKMS could be useful if a foo-dkms pkg is
>> able to detect all local kernels and build required modules without
>> interaction. dkms packages for kernel for which we provide binary
>> modules doesn't provide any more comfort for the user to me.
>>
> 
> So far, everybody wants (or accept we need) a binary package for -arch kernel.
> So, I pushed a binary package for virtualbox -arch kernel modules.
> 
> I hope others dkms only packages (ndiswrapper, sysdig) will also provide a
> binary version for the -arch kernel. This would be a step forward in a common
> way of providing kernel modules.
> 
> Regarding binary modules for -lts,-zen,-grsec, I think we should either
> provides them or not. But not stay in each packager do is own choice.
> 

I'm happy now, thanks.



signature.asc
Description: OpenPGP digital signature

Re: [arch-dev-public] Consensus: DKMS modules

[Andrea Scarpino]

On Mon, Apr 18, 2016 at 2:24 AM, Sébastien Luttringer 
wrote:

> So, I pushed a binary package for virtualbox -arch kernel modules.
>

Thanks!

-- 
Andrea

Re: [arch-dev-public] Consensus: DKMS modules

[Sébastien Luttringer]

On jeu., 2016-04-14 at 18:54 +0200, Andreas Radke wrote:
> Am Thu, 14 Apr 2016 13:29:33 +0200
> schrieb Ike Devolder :
> 
> I'm for  binary modules for our -ARCH main kernel pkg. I see no real
> need for -lts modules but if there're a few people who find them
> useful I can handle the kernel rebuilds.
> 
> No opinion about dkms at all. DKMS could be useful if a foo-dkms pkg is
> able to detect all local kernels and build required modules without
> interaction. dkms packages for kernel for which we provide binary
> modules doesn't provide any more comfort for the user to me.
> 

So far, everybody wants (or accept we need) a binary package for -arch kernel.
So, I pushed a binary package for virtualbox -arch kernel modules.

I hope others dkms only packages (ndiswrapper, sysdig) will also provide a
binary version for the -arch kernel. This would be a step forward in a common
way of providing kernel modules.

Regarding binary modules for -lts,-zen,-grsec, I think we should either
provides them or not. But not stay in each packager do is own choice.

-- 
Sébastien "Seblu" Luttringer
https://seblu.net | Twitter: @seblu42
GPG: 0x2072D77A

signature.asc
Description: This is a digitally signed message part

Re: [arch-dev-public] Consensus: DKMS modules

[Andreas Radke]

Am Thu, 14 Apr 2016 13:29:33 +0200
schrieb Ike Devolder :

> On Thu, Apr 14, 2016 at 08:06:35PM +1000, Allan McRae wrote:
> > On 14/04/16 19:23, Ike Devolder wrote:  
> > > - use separate repo [kernel-update{-testing,-staging}]  
> > 
> > Why?   We have staging for rebuilds like these.  
> 
> So we can have easier updates when a kernel is updated in both core
> and testing.
> 
> In that case we would have to land a kernel in core and then update
> the modules as fast as possible. If this update process has its own
> repo you can make sure the updates of the kernel and its out-of-tree
> modules happen on the same time.
> 

There's no need for new repos.

I'm for  binary modules for our -ARCH main kernel pkg. I see no real
need for -lts modules but if there're a few people who find them
useful I can handle the kernel rebuilds.

No opinion about dkms at all. DKMS could be useful if a foo-dkms pkg is
able to detect all local kernels and build required modules without
interaction. dkms packages for kernel for which we provide binary
modules doesn't provide any more comfort for the user to me.

-Andy


pgpBBbxsIR2ON.pgp
Description: Digitale Signatur von OpenPGP

Re: [arch-dev-public] Consensus: DKMS modules

[Ike Devolder]

On Thu, Apr 14, 2016 at 08:06:35PM +1000, Allan McRae wrote:
> On 14/04/16 19:23, Ike Devolder wrote:
> > - use separate repo [kernel-update{-testing,-staging}]
> 
> Why?   We have staging for rebuilds like these.

So we can have easier updates when a kernel is updated in both core and
testing.

In that case we would have to land a kernel in core and then update the
modules as fast as possible. If this update process has its own repo you
can make sure the updates of the kernel and its out-of-tree modules
happen on the same time.

-- 
Ike


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Allan McRae]

On 14/04/16 19:23, Ike Devolder wrote:
> - use separate repo [kernel-update{-testing,-staging}]

Why?   We have staging for rebuilds like these.

Re: [arch-dev-public] Consensus: DKMS modules

[Ike Devolder]

On Wed, Apr 13, 2016 at 08:07:34PM +0200, Bartłomiej Piotrowski wrote:
> On 2016-04-13 15:44, Ike Devolder wrote:
> > On Wed, Apr 13, 2016 at 02:05:36PM +0200, Jan Alexander Steffens wrote:
> >> On Wed, Apr 13, 2016 at 12:14 PM, Ike Devolder  
> >> wrote:
> >>> To get this discussion back on the right track I'm going to build the
> >>> binary modules for virtualbox. Sébastien and myself already discussed
> >>> what will be done so relatively soon those binary modules will be back.
> >>>
> >>> My plan is now to provide the virtualbox modules for -arch -lts and
> >>> -zen. I think -grsec will be the exception since there are probably
> >>> protections in there that will block some modules to even build.
> >>>
> >>> And when everyone is happy again we probaly should proceed to provide
> >>> dkms for all out-of-tree modules alongside the binary modules. That
> >>> would benefit everyone and offer the greatest amount of choice. People
> >>> using custom kernels can use dkms and have everything working that way
> >>> and people using one of the kernels available in the repo's can choose
> >>> if they want dkms or binary. Everyone wins.
> >>
> >> Please don't add modules for -zen to the repos. They create a maintenance
> >> burden I don't want to support. Let -zen users use DKMS; they never had any
> >> prebuilt modules anyway.
> > 
> > That makes it easier for me. So we stick to binary modules for [core]
> > kernels and the rest does dkms as a middle way.
> > 
> 
> Let's wait for Andreas opinion on this, but I think that binary modules
> for -lts are unnecessary. I always used this kernel for servers (where I
> don't really care about Virtualbox or Nvidia…) and sometimes a fallback
> if -ARCH is broken.
> 
> Bartłomiej
> 

So after 2 or 3 mails we divert further from what I presumed was a sort
of consensus. Could we just take this to a vote or something because
this sort of impasse will only hurt the users.

1. vote for binary modules

- -ARCH
- [core]
- other?

2. vote for dkms

- all out-of-tree modules provide dkms
- dkms if the maintainer of the module is willing to do it
- no dkms (no longer an option I think)

proposal flow for kernel + binary module updates

- use separate repo [kernel-update{-testing,-staging}]
- announcement to the module maintainers there is an update for a kernel
- module maintainers build and push the packages in the respective repo
- kernel maintainer can move kernel + modules in the main repo

-- 
Ike


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Ike Devolder]

On Thu, Apr 14, 2016 at 09:08:50AM +0200, Jan Alexander Steffens wrote:
> On Wed, Apr 13, 2016 at 2:05 PM, Jan Alexander Steffens
>  wrote:
> > Please don't add modules for -zen to the repos. They create a maintenance
> > burden I don't want to support. Let -zen users use DKMS; they never had any
> > prebuilt modules anyway.
> 
> That said, if the module is interesting enough and GPL-compatible,
> I'll import it into the ZEN tree (Liquorix users would then get it,
> too). I've done this with vhba-module and tp_smapi way back when.

Aha so -zen already has some additional modules that are normally
out-of-tree.

-- 
Ike


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Jan Alexander Steffens]

On Wed, Apr 13, 2016 at 2:05 PM, Jan Alexander Steffens
 wrote:
> Please don't add modules for -zen to the repos. They create a maintenance
> burden I don't want to support. Let -zen users use DKMS; they never had any
> prebuilt modules anyway.

That said, if the module is interesting enough and GPL-compatible,
I'll import it into the ZEN tree (Liquorix users would then get it,
too). I've done this with vhba-module and tp_smapi way back when.

Re: [arch-dev-public] Consensus: DKMS modules

[Bartłomiej Piotrowski]

On 2016-04-13 15:44, Ike Devolder wrote:
> On Wed, Apr 13, 2016 at 02:05:36PM +0200, Jan Alexander Steffens wrote:
>> On Wed, Apr 13, 2016 at 12:14 PM, Ike Devolder  
>> wrote:
>>> To get this discussion back on the right track I'm going to build the
>>> binary modules for virtualbox. Sébastien and myself already discussed
>>> what will be done so relatively soon those binary modules will be back.
>>>
>>> My plan is now to provide the virtualbox modules for -arch -lts and
>>> -zen. I think -grsec will be the exception since there are probably
>>> protections in there that will block some modules to even build.
>>>
>>> And when everyone is happy again we probaly should proceed to provide
>>> dkms for all out-of-tree modules alongside the binary modules. That
>>> would benefit everyone and offer the greatest amount of choice. People
>>> using custom kernels can use dkms and have everything working that way
>>> and people using one of the kernels available in the repo's can choose
>>> if they want dkms or binary. Everyone wins.
>>
>> Please don't add modules for -zen to the repos. They create a maintenance
>> burden I don't want to support. Let -zen users use DKMS; they never had any
>> prebuilt modules anyway.
> 
> That makes it easier for me. So we stick to binary modules for [core]
> kernels and the rest does dkms as a middle way.
> 

Let's wait for Andreas opinion on this, but I think that binary modules
for -lts are unnecessary. I always used this kernel for servers (where I
don't really care about Virtualbox or Nvidia…) and sometimes a fallback
if -ARCH is broken.

Bartłomiej



signature.asc
Description: OpenPGP digital signature

Re: [arch-dev-public] Consensus: DKMS modules

[Ike Devolder]

On Wed, Apr 13, 2016 at 02:05:36PM +0200, Jan Alexander Steffens wrote:
> On Wed, Apr 13, 2016 at 12:14 PM, Ike Devolder  wrote:
> > To get this discussion back on the right track I'm going to build the
> > binary modules for virtualbox. Sébastien and myself already discussed
> > what will be done so relatively soon those binary modules will be back.
> >
> > My plan is now to provide the virtualbox modules for -arch -lts and
> > -zen. I think -grsec will be the exception since there are probably
> > protections in there that will block some modules to even build.
> >
> > And when everyone is happy again we probaly should proceed to provide
> > dkms for all out-of-tree modules alongside the binary modules. That
> > would benefit everyone and offer the greatest amount of choice. People
> > using custom kernels can use dkms and have everything working that way
> > and people using one of the kernels available in the repo's can choose
> > if they want dkms or binary. Everyone wins.
> 
> Please don't add modules for -zen to the repos. They create a maintenance
> burden I don't want to support. Let -zen users use DKMS; they never had any
> prebuilt modules anyway.

That makes it easier for me. So we stick to binary modules for [core]
kernels and the rest does dkms as a middle way.

-- 
Ike


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Jan Alexander Steffens]

On Wed, Apr 13, 2016 at 12:14 PM, Ike Devolder  wrote:
> To get this discussion back on the right track I'm going to build the
> binary modules for virtualbox. Sébastien and myself already discussed
> what will be done so relatively soon those binary modules will be back.
>
> My plan is now to provide the virtualbox modules for -arch -lts and
> -zen. I think -grsec will be the exception since there are probably
> protections in there that will block some modules to even build.
>
> And when everyone is happy again we probaly should proceed to provide
> dkms for all out-of-tree modules alongside the binary modules. That
> would benefit everyone and offer the greatest amount of choice. People
> using custom kernels can use dkms and have everything working that way
> and people using one of the kernels available in the repo's can choose
> if they want dkms or binary. Everyone wins.

Please don't add modules for -zen to the repos. They create a maintenance
burden I don't want to support. Let -zen users use DKMS; they never had any
prebuilt modules anyway.

Re: [arch-dev-public] Consensus: DKMS modules

[Ike Devolder]

On Tue, Mar 15, 2016 at 10:06:22AM +1000, Allan McRae wrote:
> On 14/03/16 09:07, Allan McRae wrote:
> > On 13/03/16 00:52, Sébastien Luttringer wrote:
> >> Please note that as an ideal target, I would have all our kernel modules
> >> available via dkms _and_ via prebuilt modules for each kernel flavor we
> >> provide. I read on the dev IRC that few modules could only be shipped from
> >> sources. Not sure of that btw.
> >>
> >> For example, we could, for simplicity says that we provide pre-built 
> >> modules
> >> only for the main kernel and dkms for all others kernels.
> >>
> >> What I would like is a team consensus/decision on how we handle kernel oot
> >> modules not complains directed on virtualbox only.
> > 
> > 
> > I vote for binary modules for all kernels in [core] and dkms versions.
> > Kernels outside of [core] can have binary modules provided at the
> > maintainer's choice.
> > 
> 
> We are going to need more opinions here to build a consensus...
> 
> A

To get this discussion back on the right track I'm going to build the
binary modules for virtualbox. Sébastien and myself already discussed
what will be done so relatively soon those binary modules will be back.

My plan is now to provide the virtualbox modules for -arch -lts and
-zen. I think -grsec will be the exception since there are probably
protections in there that will block some modules to even build.

And when everyone is happy again we probaly should proceed to provide
dkms for all out-of-tree modules alongside the binary modules. That
would benefit everyone and offer the greatest amount of choice. People
using custom kernels can use dkms and have everything working that way
and people using one of the kernels available in the repo's can choose
if they want dkms or binary. Everyone wins.

-- 
Ike


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Daniel Micay]

> So linux-grsec supports no out-of-tree module? No requirement on dkms
> for it, then. Fine by me.

NVIDIA is the one of the few that would make sense because the PaX
upstream actually maintains a patch for it:

https://www.grsecurity.net/~paxguy1/nvidia-drivers-361.28-pax.patch

If I was going to package it, I would definitely avoid DKMS. DKMS doesn't 
interact well with grsecurity. The modules need to be built with the same GCC 
used to compile the kernel. There are GCC plugins and the GCC ABI varies based 
on version and configuration (i.e. gcc-multilib won't work either).

It's not really the same thing as repackaging the nvidia driver for another 
kernel though. I'll end up having to update the patch myself when it breaks 
because I'll be one of the first people trying to use it when I rebuild 
nvidia-grsec after linux-grsec. I just haven't felt like taking on that 
responsibility.

signature.asc
Description: This is a digitally signed message part

Re: [arch-dev-public] Consensus: DKMS modules

[Gaetan Bisson]

[2016-03-15 19:49:25 -0400] Daniel Micay:
> > To me the issue is people pushing new kernels to the repos but not
> > being
> > able to provide the same level of support that we have for mainline.
> > Offloading out-of-tree module rebuilds to end users instead of doing
> > it
> > ourselves is clearly not the right solution.
> > 
> > So I say: remove each non-mainline kernel of which the maintainer is
> > unwilling to support the corresponding out-of-tree modules. After
> > all,
> > as Allan points out, rebuilding them is a simple script job...
> > 
> > Cheers.
> 
> In general, out-of-tree modules aren't compatible with linux-grsec. It
> is not enough to simply rebuild them. It would require actively keeping
> them compatible by maintaining patches for them and possibly working
> with the upstreams for the out-of-tree modules for cases where bugs are
> being uncovered rather than false positives / tweaks for compatibility.
> 
> Some out-of-tree modules aren't going to be compatible with the chosen
> configuration at all, similar to how Xen support is disabled in favour
> of having the hardening features marked as incompatible with it.
> 
> The NVIDIA driver and broadcom-wl need to be patched and and VirtualBox
> is semi-incompatible with the chosen configuration. AFAIK, users would
> need to rebuild the kernel with a couple options disabled for all the
> VirtualBox features to work.

So linux-grsec supports no out-of-tree module? No requirement on dkms
for it, then. Fine by me.

-- 
Gaetan


signature.asc
Description: PGP signature

Re: [arch-dev-public] Consensus: DKMS modules

[Daniel Micay]

> To me the issue is people pushing new kernels to the repos but not
> being
> able to provide the same level of support that we have for mainline.
> Offloading out-of-tree module rebuilds to end users instead of doing
> it
> ourselves is clearly not the right solution.
> 
> So I say: remove each non-mainline kernel of which the maintainer is
> unwilling to support the corresponding out-of-tree modules. After
> all,
> as Allan points out, rebuilding them is a simple script job...
> 
> Cheers.

In general, out-of-tree modules aren't compatible with linux-grsec. It
is not enough to simply rebuild them. It would require actively keeping
them compatible by maintaining patches for them and possibly working
with the upstreams for the out-of-tree modules for cases where bugs are
being uncovered rather than false positives / tweaks for compatibility.

Some out-of-tree modules aren't going to be compatible with the chosen
configuration at all, similar to how Xen support is disabled in favour
of having the hardening features marked as incompatible with it.

The NVIDIA driver and broadcom-wl need to be patched and and VirtualBox
is semi-incompatible with the chosen configuration. AFAIK, users would
need to rebuild the kernel with a couple options disabled for all the
VirtualBox features to work.

signature.asc
Description: This is a digitally signed message part

Re: [arch-dev-public] Consensus: DKMS modules

[Gaetan Bisson]

[2016-03-15 10:06:22 +1000] Allan McRae:
> On 14/03/16 09:07, Allan McRae wrote:
> > On 13/03/16 00:52, Sébastien Luttringer wrote:
> >> Please note that as an ideal target, I would have all our kernel modules
> >> available via dkms _and_ via prebuilt modules for each kernel flavor we
> >> provide. I read on the dev IRC that few modules could only be shipped from
> >> sources. Not sure of that btw.
> >>
> >> For example, we could, for simplicity says that we provide pre-built 
> >> modules
> >> only for the main kernel and dkms for all others kernels.
> >>
> >> What I would like is a team consensus/decision on how we handle kernel oot
> >> modules not complains directed on virtualbox only.
> > 
> > 
> > I vote for binary modules for all kernels in [core] and dkms versions.
> > Kernels outside of [core] can have binary modules provided at the
> > maintainer's choice.
> > 
> 
> We are going to need more opinions here to build a consensus...

To me the issue is people pushing new kernels to the repos but not being
able to provide the same level of support that we have for mainline.
Offloading out-of-tree module rebuilds to end users instead of doing it
ourselves is clearly not the right solution.

So I say: remove each non-mainline kernel of which the maintainer is
unwilling to support the corresponding out-of-tree modules. After all,
as Allan points out, rebuilding them is a simple script job...

Cheers.

-- 
Gaetan

Re: [arch-dev-public] Consensus: DKMS modules

[Pierre Schmitz]

On 15.03.2016 01:06, Allan McRae wrote:

On 14/03/16 09:07, Allan McRae wrote:

On 13/03/16 00:52, Sébastien Luttringer wrote:
Please note that as an ideal target, I would have all our kernel 
modules
available via dkms _and_ via prebuilt modules for each kernel flavor 
we
provide. I read on the dev IRC that few modules could only be shipped 
from

sources. Not sure of that btw.

For example, we could, for simplicity says that we provide pre-built 
modules

only for the main kernel and dkms for all others kernels.

What I would like is a team consensus/decision on how we handle 
kernel oot

modules not complains directed on virtualbox only.



I vote for binary modules for all kernels in [core] and dkms versions.
Kernels outside of [core] can have binary modules provided at the
maintainer's choice.



We are going to need more opinions here to build a consensus...

A


I agree. There is no point in having every user building the exact same 
module on every update. That's why we package precompiled packages. This 
looks more like a workaround of how kernel and module updates are 
handled atm. E.g. the kernel stays in testing for a long time and is not 
put into staging first so packagers can rebuild their modules.


Greetings,

Pierre

--
Pierre Schmitz, https://pierre-schmitz.com

Re: [arch-dev-public] Consensus: DKMS modules

[Lukas Jirkovsky]

On 15 March 2016 at 01:06, Allan McRae  wrote:
> On 14/03/16 09:07, Allan McRae wrote:
>>
>>
>> I vote for binary modules for all kernels in [core] and dkms versions.
>> Kernels outside of [core] can have binary modules provided at the
>> maintainer's choice.
>>
>
> We are going to need more opinions here to build a consensus...
>
> A

While I like the idea of having DKMS in the repo, I'm strongly in
favor of having binary modules for kernels from [core]

Lukas

Re: [arch-dev-public] Consensus: DKMS modules

[Florian Pritz]

On 15.03.2016 08:26, Maxime Gauduin wrote:
> It really doesn't take that much time to build them, even on modest
> machines (got nvidia, bbswitch, vboxhost and cdemu on my laptop).

I really feel like vbox takes way too long to build. It's probably not
longer than 15 seconds, but it produces so much output that it feels
slow (well and because it is when compared to how fast the archive would
be extracted).

> That being said, I feel that what Sebastien proposed, ie having built
> modules only for linux and linux-lts, and DKMS everywhere else would be a
> good compromise IMHO.

Please go that route. We are a binary distro and I'd prefer to have as
many packages prebuilt as possible. I'm always reminded of that when I
build perl modules from cpan. Installing 20 modules from the repos is
done in like 10 seconds while building them takes something like 1 to 10
minutes (yay tests).

Florian



signature.asc
Description: OpenPGP digital signature

Re: [arch-dev-public] Consensus: DKMS modules

[Maxime Gauduin]

On Tue, Mar 15, 2016 at 1:06 AM, Allan McRae  wrote:

> On 14/03/16 09:07, Allan McRae wrote:
> > On 13/03/16 00:52, Sébastien Luttringer wrote:
> >> Please note that as an ideal target, I would have all our kernel modules
> >> available via dkms _and_ via prebuilt modules for each kernel flavor we
> >> provide. I read on the dev IRC that few modules could only be shipped
> from
> >> sources. Not sure of that btw.
> >>
> >> For example, we could, for simplicity says that we provide pre-built
> modules
> >> only for the main kernel and dkms for all others kernels.
> >>
> >> What I would like is a team consensus/decision on how we handle kernel
> oot
> >> modules not complains directed on virtualbox only.
> >
> >
> > I vote for binary modules for all kernels in [core] and dkms versions.
> > Kernels outside of [core] can have binary modules provided at the
> > maintainer's choice.
> >
>
> We are going to need more opinions here to build a consensus...
>
> A
>

Having used the ck kernel for years, and now zen, I am somehow biased and
in favor of DKMS everywhere, that and it really puts a burden on our kernel
maintainers having to build all our OOT modules with every upgrade. It
really doesn't take that much time to build them, even on modest machines
(got nvidia, bbswitch, vboxhost and cdemu on my laptop). Also I know some
people disagree, but kernel headers don't take that much space, bandwidth
may be another story though.

That being said, I feel that what Sebastien proposed, ie having built
modules only for linux and linux-lts, and DKMS everywhere else would be a
good compromise IMHO.

Cheers,
--
Maxime

[arch-dev-public] Consensus: DKMS modules

[Allan McRae]

On 14/03/16 09:07, Allan McRae wrote:
> On 13/03/16 00:52, Sébastien Luttringer wrote:
>> Please note that as an ideal target, I would have all our kernel modules
>> available via dkms _and_ via prebuilt modules for each kernel flavor we
>> provide. I read on the dev IRC that few modules could only be shipped from
>> sources. Not sure of that btw.
>>
>> For example, we could, for simplicity says that we provide pre-built modules
>> only for the main kernel and dkms for all others kernels.
>>
>> What I would like is a team consensus/decision on how we handle kernel oot
>> modules not complains directed on virtualbox only.
> 
> 
> I vote for binary modules for all kernels in [core] and dkms versions.
> Kernels outside of [core] can have binary modules provided at the
> maintainer's choice.
> 

We are going to need more opinions here to build a consensus...

A