Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-05-21 Thread Ben Hutchings
On Sat, 2016-05-21 at 15:31 -0300, Lucas De Marchi wrote:
> On Wed, Apr 13, 2016 at 7:00 AM, Ben Hutchings  wrote:
> > 
> > On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote:
> > > 
> > > Hi,
> > > 
> > > CC'ing Rusty
> > > 
> > > On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings  wrote:
> > > > 
> > > > 
> > > > Debian will not sign modules during the kernel package build, as this
> > > > conflicts with the goal of reproducible builds.  Instead, we will
> > > > generate detached signatures offline and include them in a second
> > > > package.
> > > Is this a decision already? It doesn't look as a good reason - you
> > > would already need to provide a signing key (CONFIG_MODULE_SIG_KEY)
> > > anyway for this to work. How is leaving the module signature in
> > > another package be any better than just signing the module?  If you
> > > have the signature, the build is just as reproducible as before.
> > I think we may have different ideas about what reproducibility means.
> > When I say reproducible I mean *anyone* with the right tools installed
> > can reproduce the binary packages (.deb) from the source package (.dsc
> > and tarballs).
> > 
> > The signing key obviously isn't available to everyone, so the source
> > package has to include detached signatures prepared outside of the
> And how is this signature prepared?  Since it needs the compiled
> module it would be a matter of changing the compiler, even minor
> version, to invalidate the argument of reproducible build. It seems
> very fragile to me.

The versions of build tools have to be recorded:
https://reproducible-builds.org/docs/formal-definition/
https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification

> > package build process.  But we can't put them in the linux source
> > package, because that results in a dependency loop.
> > 
> > > 
> > > > 
> > > > 
> > > > We could attach the signatures when building this second package or at
> > > > installation time, but that leads to duplication of all modules,
> > > > either in the archive or on users' systems.
> > > > 
> > > > To avoid this, add support to libkmod for concatenating modules with
> > > > detached signatures (files with the '.sig' extension) at load time.
> > > this has the drawback that finit_module() can't be used.
> > So does module compression, but it's still a supported option.
> This is easily fixed by teaching the kernel to handle the fd as a
> compressed file.

This sounds speculative.

> The kernel already has the routines to uncompress
> them anyway. Supporting detached signatures means it can't be fixed
> anymore since we will have to use init_module() rather than
> finit_module().

Why does that matter?  init_module() isn't deprecated.

Ben.

-- 
Ben Hutchings
Experience is what causes a person to make new mistakes instead of old ones.

signature.asc
Description: This is a digitally signed message part


Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-05-21 Thread Marco d'Itri
On May 21, Lucas De Marchi  wrote:

> And how is this signature prepared?  Since it needs the compiled
> module it would be a matter of changing the compiler, even minor
> version, to invalidate the argument of reproducible build. It seems
> very fragile to me.
But this is the whole point of reproducible builds: knowing that if you 
start from the same sources and build enviroment, and distributions do, 
then you will get the same results.
Building gcc is reproducibile as well...
Reproducibility with a different compiler has never been a goal.

-- 
ciao,
Marco


signature.asc
Description: PGP signature


Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-05-21 Thread Lucas De Marchi
On Wed, Apr 13, 2016 at 7:00 AM, Ben Hutchings  wrote:
> On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote:
>> Hi,
>>
>> CC'ing Rusty
>>
>> On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings  wrote:
>> >
>> > Debian will not sign modules during the kernel package build, as this
>> > conflicts with the goal of reproducible builds.  Instead, we will
>> > generate detached signatures offline and include them in a second
>> > package.
>> Is this a decision already? It doesn't look as a good reason - you
>> would already need to provide a signing key (CONFIG_MODULE_SIG_KEY)
>> anyway for this to work. How is leaving the module signature in
>> another package be any better than just signing the module?  If you
>> have the signature, the build is just as reproducible as before.
>
> I think we may have different ideas about what reproducibility means.
> When I say reproducible I mean *anyone* with the right tools installed
> can reproduce the binary packages (.deb) from the source package (.dsc
> and tarballs).
>
> The signing key obviously isn't available to everyone, so the source
> package has to include detached signatures prepared outside of the

And how is this signature prepared?  Since it needs the compiled
module it would be a matter of changing the compiler, even minor
version, to invalidate the argument of reproducible build. It seems
very fragile to me.

> package build process.  But we can't put them in the linux source
> package, because that results in a dependency loop.
>
>> >
>> > We could attach the signatures when building this second package or at
>> > installation time, but that leads to duplication of all modules,
>> > either in the archive or on users' systems.
>> >
>> > To avoid this, add support to libkmod for concatenating modules with
>> > detached signatures (files with the '.sig' extension) at load time.
>> this has the drawback that finit_module() can't be used.
>
> So does module compression, but it's still a supported option.

This is easily fixed by teaching the kernel to handle the fd as a
compressed file. The kernel already has the routines to uncompress
them anyway. Supporting detached signatures means it can't be fixed
anymore since we will have to use init_module() rather than
finit_module().


Lucas De Marchi



Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-05-17 Thread Ben Hutchings
On Wed, 2016-04-13 at 11:00 +0100, Ben Hutchings wrote:
> On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote:
> > 
> > Hi,
> > 
> > CC'ing Rusty
> > 
> > On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings  wrote:
> > > 
> > > 
> > > Debian will not sign modules during the kernel package build, as this
> > > conflicts with the goal of reproducible builds.  Instead, we will
> > > generate detached signatures offline and include them in a second
> > > package.
> > Is this a decision already? It doesn't look as a good reason - you
> > would already need to provide a signing key (CONFIG_MODULE_SIG_KEY)
> > anyway for this to work. How is leaving the module signature in
> > another package be any better than just signing the module?  If you
> > have the signature, the build is just as reproducible as before.
> I think we may have different ideas about what reproducibility means.
> When I say reproducible I mean *anyone* with the right tools installed
> can reproduce the binary packages (.deb) from the source package (.dsc
> and tarballs).
> 
> The signing key obviously isn't available to everyone, so the source
> package has to include detached signatures prepared outside of the
> package build process.  But we can't put them in the linux source
> package, because that results in a dependency loop.

So, given these requirements, what do you think now about supporting
detached signatures?

I spoke at greater length about what I'm trying to do at Linuxwochen
Wien; see
http://meetings-archive.debian.net/pub/debian-meetings/2016/mini-debconf-vienna/webm/Secure_Boot_vs_the_Debian_linux_package.webm#t=595
from about 9'55" to 17'30".

Ben.

> > 
> > > 
> > > 
> > > We could attach the signatures when building this second package or at
> > > installation time, but that leads to duplication of all modules,
> > > either in the archive or on users' systems.
> > > 
> > > To avoid this, add support to libkmod for concatenating modules with
> > > detached signatures (files with the '.sig' extension) at load time.
> > this has the drawback that finit_module() can't be used.
> So does module compression, but it's still a supported option.
> 
> [...]
> > 
> > > 
> > > +   /* Try to open a detached signature.  If it's missing, that's OK. 
> > > */
> > > +   if (asprintf(&sig_filename, "%s.sig", filename) < 0) {
> > > +   err = -errno;
> > > +   goto error;
> > > +   }
> > > +   file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC);
> > > +   if (file->sig_fd < 0 && errno != ENOENT) {
> > > +   err = -errno;
> > > +   goto error;
> > > +   }
> > This can't really work if the module is being loaded uncompressed (I
> > think nowadays we can even add support for compressed modules...
> > Rusty, any input here?).
> > 
> > When the module is being directly loaded, the direct flag gets set so
> > kmod_module_insert_module() knows it can try to use finit_module().
> > Since you have an external signature what would happen is that we
> > would load the signature, but try to load the module in the kernel
> > without it.
> It does work.  I changed load_reg() to disable direct loading.when
> there's a detached signature.
> 
> Ben.
> 
> > 
> > I'm still not convinced the split module + signature is actually a good 
> > thing.
> > 
> > 
> > Lucas De Marchi
-- 
Ben Hutchings
Lowery's Law:
 If it jams, force it. If it breaks, it needed replacing anyway.

signature.asc
Description: This is a digitally signed message part


Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-04-13 Thread Ben Hutchings
On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote:
> Hi,
> 
> CC'ing Rusty
> 
> On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings  wrote:
> > 
> > Debian will not sign modules during the kernel package build, as this
> > conflicts with the goal of reproducible builds.  Instead, we will
> > generate detached signatures offline and include them in a second
> > package.
> Is this a decision already? It doesn't look as a good reason - you
> would already need to provide a signing key (CONFIG_MODULE_SIG_KEY)
> anyway for this to work. How is leaving the module signature in
> another package be any better than just signing the module?  If you
> have the signature, the build is just as reproducible as before.

I think we may have different ideas about what reproducibility means.
When I say reproducible I mean *anyone* with the right tools installed
can reproduce the binary packages (.deb) from the source package (.dsc
and tarballs).

The signing key obviously isn't available to everyone, so the source
package has to include detached signatures prepared outside of the
package build process.  But we can't put them in the linux source
package, because that results in a dependency loop.

> > 
> > We could attach the signatures when building this second package or at
> > installation time, but that leads to duplication of all modules,
> > either in the archive or on users' systems.
> > 
> > To avoid this, add support to libkmod for concatenating modules with
> > detached signatures (files with the '.sig' extension) at load time.
> this has the drawback that finit_module() can't be used.

So does module compression, but it's still a supported option.

[...]
> > +   /* Try to open a detached signature.  If it's missing, that's OK. */
> > +   if (asprintf(&sig_filename, "%s.sig", filename) < 0) {
> > +   err = -errno;
> > +   goto error;
> > +   }
> > +   file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC);
> > +   if (file->sig_fd < 0 && errno != ENOENT) {
> > +   err = -errno;
> > +   goto error;
> > +   }
> This can't really work if the module is being loaded uncompressed (I
> think nowadays we can even add support for compressed modules...
> Rusty, any input here?).
>
> When the module is being directly loaded, the direct flag gets set so
> kmod_module_insert_module() knows it can try to use finit_module().
> Since you have an external signature what would happen is that we
> would load the signature, but try to load the module in the kernel
> without it.

It does work.  I changed load_reg() to disable direct loading.when
there's a detached signature.

Ben.

> I'm still not convinced the split module + signature is actually a good thing.
> 
> 
> Lucas De Marchi
-- 
Ben Hutchings
It is easier to change the specification to fit the program than vice versa.

signature.asc
Description: This is a digitally signed message part


Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-04-12 Thread Lucas De Marchi
Hi,

CC'ing Rusty

On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings  wrote:
> Debian will not sign modules during the kernel package build, as this
> conflicts with the goal of reproducible builds.  Instead, we will
> generate detached signatures offline and include them in a second
> package.

Is this a decision already? It doesn't look as a good reason - you
would already need to provide a signing key (CONFIG_MODULE_SIG_KEY)
anyway for this to work. How is leaving the module signature in
another package be any better than just signing the module?  If you
have the signature, the build is just as reproducible as before.

> We could attach the signatures when building this second package or at
> installation time, but that leads to duplication of all modules,
> either in the archive or on users' systems.
>
> To avoid this, add support to libkmod for concatenating modules with
> detached signatures (files with the '.sig' extension) at load time.

this has the drawback that finit_module() can't be used.


> Signed-off-by: Ben Hutchings 
> ---
> v2: Fix syntax error in the xz case
>
> Missed this because I didn't realise the Debian package disables gzip
> and xz support.
>
> Ben.
>
>  libkmod/libkmod-file.c | 110 
> +
>  1 file changed, 103 insertions(+), 7 deletions(-)
>
> diff --git a/libkmod/libkmod-file.c b/libkmod/libkmod-file.c
> index 5eeba6a912a2..cb1da3c9e2ae 100644
> --- a/libkmod/libkmod-file.c
> +++ b/libkmod/libkmod-file.c

...

> @@ -292,12 +370,25 @@ struct kmod_file *kmod_file_open(const struct kmod_ctx 
> *ctx,
> if (file == NULL)
> return NULL;
>
> +   file->sig_fd = -1;
> +
> file->fd = open(filename, O_RDONLY|O_CLOEXEC);
> if (file->fd < 0) {
> err = -errno;
> goto error;
> }
>
> +   /* Try to open a detached signature.  If it's missing, that's OK. */
> +   if (asprintf(&sig_filename, "%s.sig", filename) < 0) {
> +   err = -errno;
> +   goto error;
> +   }
> +   file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC);
> +   if (file->sig_fd < 0 && errno != ENOENT) {
> +   err = -errno;
> +   goto error;
> +   }

This can't really work if the module is being loaded uncompressed (I
think nowadays we can even add support for compressed modules...
Rusty, any input here?).

When the module is being directly loaded, the direct flag gets set so
kmod_module_insert_module() knows it can try to use finit_module().
Since you have an external signature what would happen is that we
would load the signature, but try to load the module in the kernel
without it.

I'm still not convinced the split module + signature is actually a good thing.


Lucas De Marchi



Bug#820010: [PATCH v2] libkmod: Add support for detached module signatures

2016-04-04 Thread Ben Hutchings
Debian will not sign modules during the kernel package build, as this
conflicts with the goal of reproducible builds.  Instead, we will
generate detached signatures offline and include them in a second
package.

We could attach the signatures when building this second package or at
installation time, but that leads to duplication of all modules,
either in the archive or on users' systems.

To avoid this, add support to libkmod for concatenating modules with
detached signatures (files with the '.sig' extension) at load time.

Signed-off-by: Ben Hutchings 
---
v2: Fix syntax error in the xz case

Missed this because I didn't realise the Debian package disables gzip
and xz support.

Ben.

 libkmod/libkmod-file.c | 110 +
 1 file changed, 103 insertions(+), 7 deletions(-)

diff --git a/libkmod/libkmod-file.c b/libkmod/libkmod-file.c
index 5eeba6a912a2..cb1da3c9e2ae 100644
--- a/libkmod/libkmod-file.c
+++ b/libkmod/libkmod-file.c
@@ -52,6 +52,7 @@ struct kmod_file {
gzFile gzf;
 #endif
int fd;
+   int sig_fd;
bool direct;
off_t size;
void *memory;
@@ -60,6 +61,37 @@ struct kmod_file {
struct kmod_elf *elf;
 };
 
+static int append_detached_sig(struct kmod_file *file, size_t buf_size)
+{
+   struct stat st;
+   ssize_t read_size;
+
+   if (file->sig_fd < 0)
+   return 0;
+
+   if (fstat(file->sig_fd, &st) < 0)
+   return -errno;
+
+   /* Grow the buffer if necessary */
+   if ((size_t)st.st_size > buf_size - file->size) {
+   void *tmp = realloc(file->memory, file->size + st.st_size);
+   if (tmp == NULL)
+   return -errno;
+   file->memory = tmp;
+   }
+
+   read_size = read(file->sig_fd, (char *)file->memory + file->size,
+st.st_size);
+   if (read_size < 0)
+   return -errno;
+   if (read_size != st.st_size)
+   return -EINVAL;
+
+   file->size += read_size;
+
+   return 0;
+}
+
 #ifdef ENABLE_XZ
 static void xz_uncompress_belch(struct kmod_file *file, lzma_ret ret)
 {
@@ -144,6 +176,7 @@ static int load_xz(struct kmod_file *file)
 {
lzma_stream strm = LZMA_STREAM_INIT;
lzma_ret lzret;
+   size_t buf_size;
int ret;
 
lzret = lzma_stream_decoder(&strm, UINT64_MAX, LZMA_CONCATENATED);
@@ -155,7 +188,14 @@ static int load_xz(struct kmod_file *file)
return -EINVAL;
}
ret = xz_uncompress(&strm, file);
+   buf_size = file->size + strm.avail_out;
lzma_end(&strm);
+
+   if (!ret) {
+   ret = append_detached_sig(file, buf_size);
+   if (ret)
+   free(file->memory);
+   }
return ret;
 }
 
@@ -214,6 +254,11 @@ static int load_zlib(struct kmod_file *file)
 
file->memory = p;
file->size = did;
+
+   err = append_detached_sig(file, total);
+   if (err)
+   goto error;
+
p = NULL;
return 0;
 
@@ -254,18 +299,50 @@ static int load_reg(struct kmod_file *file)
if (fstat(file->fd, &st) < 0)
return -errno;
 
-   file->size = st.st_size;
-   file->memory = mmap(NULL, file->size, PROT_READ, MAP_PRIVATE,
-   file->fd, 0);
-   if (file->memory == MAP_FAILED)
-   return -errno;
-   file->direct = true;
+   if (file->sig_fd < 0) {
+   file->size = st.st_size;
+   file->memory = mmap(NULL, file->size, PROT_READ, MAP_PRIVATE,
+   file->fd, 0);
+   if (file->memory == MAP_FAILED)
+   return -errno;
+   file->direct = true;
+   } else {
+   size_t plain_size = st.st_size, sig_size;
+   _cleanup_free_ unsigned char *p = NULL;
+   ssize_t ret;
+
+   if (fstat(file->sig_fd, &st) < 0)
+   return -errno;
+   sig_size = st.st_size;
+
+   p = malloc(plain_size + sig_size);
+   if (!p)
+   return -errno;
+
+   ret = read(file->fd, p, plain_size);
+   if (ret < 0)
+   return -errno;
+   if ((size_t)ret != plain_size)
+   return -EINVAL;
+   file->memory = p;
+   file->size = plain_size;
+
+   ret = append_detached_sig(file, plain_size + sig_size);
+   if (ret)
+   return ret;
+
+   p = NULL;
+   }
+
return 0;
 }
 
 static void unload_reg(struct kmod_file *file)
 {
-   munmap(file->memory, file->size);
+   if (file->direct)
+   munmap(file->memory, file->size);
+   else
+   free(file->memory);
 }
 
 static const struct file_ops reg_ops = {
@@ -285,6 +362,7 @@ struct kmod_file *kmod_file_o