Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Adonay Felipe Nogueira]

Thank you very much for the replies.

If I understand it correctly, the current approach does seem to be more
guaranteed indeed.

2018-01-20T01:06:21-0200 Alexandre Oliva wrote:
> On Jan 16, 2018, Adonay Felipe Nogueira  wrote:
>
>> What if for example the kernel wouldn't reference stuff by file names,
>> but instead to bus address or something like that?
>
> It's not just about the error messages, it's also about what the kernel
> actually requests of the userland helper script or the filesystem
> serving /lib/firmware, and how distros might configure the loader or the
> filesystem to respond to such requests.
>
> When the kernel starts the helper script, it's active telling a
> third-party program "get me a copy of the program foo.bin".
>
> The idea that Jason referred to was of one-way hashing blob names, and
> having the loader script hash available firmware names until it found
> one that matched the request, or until it exhausted the list.
>
> Problem is, distros could still pretend to have plenty of files
> available (based on what's in the repos), and make the whole set visible
> to the script, even if stuff would only be installed on demand, if
> actually requested.
>
> This may have seemed far-fetched back when distros just supplied scripts
> that would search repos and offer to install firmware requested by the
> kernel but not installed, but more recently, the kernel is moving away
> from the userland hotplug script and accessing /lib/firmware directly,
> and so distros willing to retain such functionality are going to mount
> /lib/firmware as a userland filesystem serving out the complete set of
> firmware and installing, or offering to install it, when the kernel
> actually requests the files.  So our hashing would have to be done
> elsewhere, and even then it wouldn't stop the script/filesystem from
> installing stuff that was not installed.
>
>
> Even if we turned the table around and changed the kernel so as to ask
> for a list of available firmware before asking for any blobs, and only
> requested blobs that were in the list, we would be defeated by this
> filesystem: enumerating the (apparently) available files would make it
> seem like all blobs are actually installed.
>
> There doesn't seem to be a way to avoid what we're trying to avoid while
> still allowing proprietary firmware to be loaded, so...  we don't allow
> it.  It's still a bug, but apparently an unfixable one.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Alexandre Oliva]

On Jan 20, 2018, bill-auger  wrote:

> i must say though that it did not address what is the actual
> behavior preventing the debian kernel from being acceptable,

I didn't try to address that, and I'm afraid I don't know the answer
myself, not having looked at the Debian kernel in detail any further
than to get slightly acquainted with the general approach.  I'm told it
is Free Software, which is good, but I had got the idea that it did not
meet the GNU FSDG until PureOS, AFAIK carrying the (Debian main only?)
bits of the Debian kernel, made the endorsed distro list.  I'll defer
judgment and clarification to the FSF.

If I had to guess, I'd say the fact that the distro controls the hotplug
script, the repositories, the package installer, and whatnot, places
them at an advantage over what we have to do in GNU Linux-libre to
ensure it behaves within the FSDG, and maybe the requesting and logging
of firmware, even non-Free, is not deemed a problem because the other
system components take care of them, and the non-Free bits are not
available for installation anyway, unless other repos containing those
bits are configured.  But that's just a personal guess.

-- 
Alexandre Oliva, freedom fighterhttp://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Luke Shumaker]

On Mon, 22 Jan 2018 10:05:08 -0500,
Felipe Sanches wrote:
> Regarding devices that rely on non-free fw but that could also work
> "without firmware"... I think that most likely does not exist.

An example: Many Radeon graphics cards work without firmware, but have
no 2D or 3D acceleration.  The simple legacy (VGA?) ways for talking
to it are implemented in hardware and work without firmware, but the
more complex modern ways that are required for decent performance
don't.  It's fine for, say running Emacs and basic web-browsing, but
VLC won't do better than 5 FPS while fullscreen.

> There are devices that have the fw loaded into RAM and thus require
> an install at every power up otherwise they will simply not work.
>
> But there are also devices that have factory installed firmware
> stored in permanent ROM memory and then provide a firmware update
> mechanism that can be used to install newer versions of the fw (with
> bugfixes perhaps, but could also include new anti-features and
> likely new bugs as well). In this case, you can use the old in-ROM
> firmware or the new firmware provided by the kernel.
> 
> Having said that, I have the impression that some people actually
> refer to running the in-ROM factory fw whey they say the device may
> run "without a firmware", which is surely a misleading way of
> expressing what actually happens.

I believe that this is the situation with microcode updates.

-- 
Happy hacking,
~ Luke Shumaker

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Denis 'GNUtoo' Carikli]

On Sun, 21 Jan 2018 07:51:44 -0800 (PST)
"Jason Self"  wrote:

> Perhaps a more philosophical question is: *Should* a free program,
> especially one used in FSF-endorsed distros, be generating requests
> for proprietary programs in the first place? Regardless of how they
> might be handled.
I find the requests unclear. With Linux-libre 4.12.14 and Parabola I
have:
> ieee80211 phy1: rt2x00lib_request_firmware: Info - Loading firmware file 
> '/*(DEBLOBBED)*/'
> 1-5:1.0: Missing Free firmware (non-Free firmware loading is disabled)
> rt2800usb 1-5:1.0: Direct firmware load for /*(DEBLOBBED)*/ failed with error 
> -2
> ieee80211 phy1: rt2x00lib_request_firmware: Error - Failed to request Firmware

"Missing" makes me think that I should go find the firmware somewhere
and add it because for some reason it's missing.

A message that would encourage a user or developer to work toward:
- having a free firmware written
- having linux-libre working without a firmware

Sometime the device is working fine without firmware, so here it would
be nice to have an idea of what is the difference of behavior of the
hardware between no firmware and loading the non-free firmware.
This would either inform the user that everything is fine or push
towards making it work better without a firmware or with a free
firmware.

For instance the message could point to a page on the linux-libre
website that would explains all that in greater length, as it would be
complicated to explain all that in very few lines of messages (which
may even be redundant).

Denis.


pgpLvhRQhEiXJ.pgp
Description: OpenPGP digital signature

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Jason Self]

Perhaps a more philosophical question is: *Should* a free program,
especially one used in FSF-endorsed distros, be generating requests
for proprietary programs in the first place? Regardless of how they
might be handled.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Jason Self]

> i must say though that it did not address what is the actual
> behavior preventing the debian kernel from being acceptable, 

I thought it did.

> why is it not possible to simply change or suppress the error
> message printed to the log or to later or periodically scrub the 
> logs of the naughty bits

Because the kernel is not necessarily responsible for all firmware
loading.

Alexandre mentioned this in:

> It's not just about the error messages, it's also about what the
> kernel actually requests of the userland helper script or the
> filesystem serving /lib/firmware, and how distros might configure
> the loader or the filesystem to respond to such requests.

Once the kernel has identified that a firmware is needed for something
then we've got the problem he mentioned:

> When the kernel starts the helper script, it's active telling a
> third-party program "get me a copy of the program foo.bin".

Being that the userland firmware loading program is a separate program
the kernel has no control over what it might do. Absolutely none. It
might log the request, which would be separate from the kernel
logging, and so even if the kernel's logging functionality were
changed in some way it would not have any impact on what this other
program is doing.

But, of course, it could be even worse than logging. It could, for
some examples:

* Display dialog boxes "Hey, this thing is missing. Do you to install
it?" This would be along the lines of how messages are shown about
missing audio/video codecs when you try to play a video. That is
probably even worse that logging because it's right up there, in your
face.

* Automatically download and install it without even asking anything.

* Who knows what else. Being that the userland firmware loading
program is a program, it could do anything that a program could do.
Even things that we aren't currently thinking of.

One might be tempted to say "well, the solution, then, is that -- in
addition to neutering the kernel's logging functionality -- to this to
get rid of the kernel's support for loading firmware from userland,
since it can access the file system directly anyway."

But that is already reducing the kernel's support for loading
firmware, which was part of the original complaint, and doesn't really
address the potential things that could happen: Another problem is
that /lib/firmware could be treated specially by the distro where any
& all of the kernel's attempts to access files there actually succeed
even if the files are not there (Alexandre mentioned the possibility
that the distro mounts /lib/firmware as a separate file system under
FUSE but that would not be the only possible way of doing this), and
then proceed to do exactly the same things that were mentioned as
being problems with the userland firmware helper. In this case another
program is *still* responsible for handling the request and the worse
part is that the kernel would probably not even know this.

Of course, some might be tempted to say that "OK, the solution, then,
is that -- in addition to neutering the kernel's logging functionality
and getting rid of the kernel's support for loading firmware from
userland, it should only try to load firmware when it's not mounted as
a separate file system. Or something." But at point you're starting to
reduce the kernel's ability to load firmware even further, which is
one of the issues that people have raised, and I don't know that there
is a way for the kernel to be able to know the attempt to access the
file system will be handled.

The fundamental problem is with firmware loading being outsourced to
separate programs, regardless of whether the kernel can detect that
this is happening or not. It might not even be able to detect it. And
the kernel has absolutely no control over what that separate program
does after the kernel has made the request via userland or attempted
to access it directly via the file system.

Of course, FSF-endorsed distros would never do any of these things but
Linux-libre is for use by everyone, even those on what Alexandre
called "hostile" distros, and all such cases need to be considered and
handled.

Given all of these possibilities the firmware loading problem may very
well be unsolvable. It's at least very difficult, which is why it
hasn't happened already.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[bill-auger]

Alexandre -

thanks for that detailed explanation - i have been curious about this
myself - i must say though that it did not address what is the actual
behavior preventing the debian kernel from being acceptable, which as i
understand is not related to its ability to load any modules but simply
that it prints names to a log in the form of a error message - why is it
not possible to simply change or suppress the error message printed to
the log or to later or periodically scrub the logs of the naughty bits



signature.asc
Description: OpenPGP digital signature

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Alexandre Oliva]

On Jan 16, 2018, Adonay Felipe Nogueira  wrote:

> What if for example the kernel wouldn't reference stuff by file names,
> but instead to bus address or something like that?

It's not just about the error messages, it's also about what the kernel
actually requests of the userland helper script or the filesystem
serving /lib/firmware, and how distros might configure the loader or the
filesystem to respond to such requests.

When the kernel starts the helper script, it's active telling a
third-party program "get me a copy of the program foo.bin".

The idea that Jason referred to was of one-way hashing blob names, and
having the loader script hash available firmware names until it found
one that matched the request, or until it exhausted the list.

Problem is, distros could still pretend to have plenty of files
available (based on what's in the repos), and make the whole set visible
to the script, even if stuff would only be installed on demand, if
actually requested.

This may have seemed far-fetched back when distros just supplied scripts
that would search repos and offer to install firmware requested by the
kernel but not installed, but more recently, the kernel is moving away
from the userland hotplug script and accessing /lib/firmware directly,
and so distros willing to retain such functionality are going to mount
/lib/firmware as a userland filesystem serving out the complete set of
firmware and installing, or offering to install it, when the kernel
actually requests the files.  So our hashing would have to be done
elsewhere, and even then it wouldn't stop the script/filesystem from
installing stuff that was not installed.


Even if we turned the table around and changed the kernel so as to ask
for a list of available firmware before asking for any blobs, and only
requested blobs that were in the list, we would be defeated by this
filesystem: enumerating the (apparently) available files would make it
seem like all blobs are actually installed.

There doesn't seem to be a way to avoid what we're trying to avoid while
still allowing proprietary firmware to be loaded, so...  we don't allow
it.  It's still a bug, but apparently an unfixable one.

-- 
Alexandre Oliva, freedom fighterhttp://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Alexandre Oliva]

On Dec 23, 2017, "Jason Self"  wrote:

> My understanding is that there is an idea for how to
> enable the loading of the proprietary firmware without also steering
> people to it when it's not present.

That idea unfortunately proved to be insufficient for scenarios in which
e.g. a (user-hostile) distro pretends all sorts of proprietary firmware
are installed just because they're available as packages, and once the
kernel actually requests them, offers to install or goes ahead and
installs the proprietary firmware for the user.  I've heard of distros
that intended to do such stuff, so RMS and I agreed we shouldn't
implement that any more.  GNU Linux-libre is supposed to do its job even
if installed on such hostile distros.

-- 
Alexandre Oliva, freedom fighterhttp://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Jason Self]

It's probably best to run it by Alexandre Oliva. I imagine that there will 
still be challenges with this; he'd be the best person to explain them. 
It's not an easy problem to solve or it would have been by now.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Adonay Felipe Nogueira]

What if for example the kernel wouldn't reference stuff by file names,
but instead to bus address or something like that?

So instead of saying "Unable to load firmware: some-file.bin." it would
say "Unable to load firmware: 03:00.0" this would unfortunatelly have
the drawback of making the message confusing to search for the new
people, but wouldn't be a direct recommendation of non-free software.

Or simply make the message not reference anything at all ("Unable to
load firmware", simple) and leave the hard work of finding the needle to
those who dare to use some tool which is able to trace the kernel
execution (if this tool really exists, I imagine it would behave similar
to `strace').

Anyways, this email message is in no way an attempt to undermine the
movement's goals, I'm accepting any answer here as long as there is
references and friendly tone. ;)

2017-12-22T21:23:27-0800 Jason Self wrote:
> Not necessarily. My understanding is that there is an idea for how to
> enable the loading of the proprietary firmware without also steering
> people to it when it's not present. A partial patch for it already
> exists in the linux-libre mailing list archives but it's a hard
> problem and hasn't been fully solved yet. Interested people are free
> to help. It would probably be best to coordinate with Alexandre Oliva.
> My recollection is that there's a conversation about it in either the
> linux-libre list archives, or this one. Perhaps even both places at
> different times.

-- 
- https://libreplanet.org/wiki/User:Adfeno
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre. Por favor, veja formas de se comunicar
  instantaneamente comigo no endereço abaixo.
- Contato: https://libreplanet.org/wiki/User:Adfeno#vCard
- Arquivos comuns aceitos (apenas sem DRM): Corel Draw, Microsoft
  Office, MP3, MP4, WMA, WMV.
- Arquivos comuns aceitos e enviados: CSV, GNU Dia, GNU Emacs Org, GNU
  GIMP, Inkscape SVG, JPG, LibreOffice (padrão ODF), OGG, OPUS, PDF
  (apenas sem DRM), PNG, TXT, WEBM.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Jason Self]

Not necessarily. My understanding is that there is an idea for how to
enable the loading of the proprietary firmware without also steering
people to it when it's not present. A partial patch for it already
exists in the linux-libre mailing list archives but it's a hard
problem and hasn't been fully solved yet. Interested people are free
to help. It would probably be best to coordinate with Alexandre Oliva.
My recollection is that there's a conversation about it in either the
linux-libre list archives, or this one. Perhaps even both places at
different times.

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Julie Marchant]

On 2017年12月22日 17:13, Henry Jensen wrote:
> As Jason Self mentioned in his answer, the use of the Debian kernel by
> PureOS should be considered a (freedom) bug.

I personally would prefer to see the requirement for the names of
firmware files to not me mentioned anywhere lifted, and for ConnochaetOS
to be reconsidered. I've given it some thought some time ago, and I
think it's silly. Linux is going to have to have a specific name for
every firmware file, and if the only firmware file that does the job for
a particular device, of course it's going to carry that name.

Consider the case of shebangs. Back before the GNU project started,
would it have been recommending the proprietary sh program that existed
in Unix to write "!/bin/sh" at the top of a shell script even if the
intention was to not /use the script until GNU Bash was developed and
could be used instead? It just seems a bit absurd.

-- 
Julie Marchant
https://onpon4.github.io

Protect your emails with GnuPG:
https://emailselfdefense.fsf.org



signature.asc
Description: OpenPGP digital signature

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Henry Jensen]

Hi Zlatan,

Am Fri, 22 Dec 2017 18:41:01 +0100
schrieb Zlatan Todoric :

> I will not read into thread but I can also add this: Debian kernel
> itself doesn't contain non-free firmware at all (they are separate
> packages in non-free repo that we sync too). Debian does allow if
> something wants to find firmware and it has non-free repo enabled,
> that it can be installed. This gets to bottom part...

This is well-known. However, it has been suggested on this list that
the act of requesting non-free firmware is something not wanted in a
free distribution, since it can lead users to install those non-free
firmware files, even if the distro itself does not provide them.

As Jason Self mentioned in his answer, the use of the Debian kernel by
PureOS should be considered a (freedom) bug.


Regards,

Henry

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Jason Self]

Henry Jensen  asked:
> So, of course I want to know how PureOS can use this Debian based
> kernel and be endorsed while ConnochaetOS can't?

As Donald implied there is a lot of work involved in review a distros so I 
imagine that this was missed. A primary difference is a willingness to 
correct mistakes, which from Don's email it seems that they have been so 
this seems a good opportunity to raise the issue in the PureOS bug tracker 
along with a CC to report-nonf...@fsf.org. Earn a GNU Buck. :)

Re: [GNU-linux-libre] PureOS added to endorsed distro list - what about the kernel?

[Zlatan Todoric]

Hi Henry,


On 12/22/2017 05:18 PM, Henry Jensen wrote:
> Hallo,
>
> congratulations for adding a great distro to the list.
>
> However, it seems that PureOS is not using the linux-libre kernel but a
> kernel based on Debian's Linux kernel.

Yes, that is correct.

>
> When I tried to get ConnochaetOS [0] on this list the last time, people
> explained to me, that ConnochaetOS can't be endorsed because it doesn't
> use linux-libre and the Debian based kernel requests non-free firmware
> files and does write the names of those files to the system log. See the
> thread at
> https://lists.nongnu.org/archive/html/gnu-linux-libre/2017-08/msg00013.html

I will not read into thread but I can also add this: Debian kernel
itself doesn't contain non-free firmware at all (they are separate
packages in non-free repo that we sync too). Debian does allow if
something wants to find firmware and it has non-free repo enabled, that
it can be installed. This gets to bottom part...

>
> Now, I installed PureOS and it seems, PureOS does use the same Debian
> based deblob mechanism as does ConnochaetOS.
>
> To verify it, I tried the PureOS ISO from
> https://downloads.puri.sm/snapshots/2017-11-11/pureos-8.0-live-amd64.hybrid.iso
> and booted it on a notebook with a wifi chip which does need non-free
> firmware. And indeed I see in dmesg the following message (among others):
>
> iwlwifi: :03:00.0 firmware: failed to load iwlwifi-6000g2a-6.ucode (-2)

And as you see it says "failed to load" which means, yes, it wanted to
see if there is such firmware in kernel but it couldn't find it. So, in
the end, Debian kernel is Free and PureOS archive is entirely based on
Debian main archive (no contrib and non-free) and there is no code that
is non-free in (we even do Freedom bugs in tracker to make Debian main
"more" Free).

>
> So, of course I want to know how PureOS can use this Debian based
> kernel and be endorsed while ConnochaetOS can't?

Well, if the only issue was Debian's kernel, I don't see it as stop sign
and you should re-apply. Notice that Debian kernel in past didn't deblob
firmware from it but in recent times that is the case now as well, so
maybe some people remember it like that.

>
>
> Regards,
>
> Henry

Cheers,

Z

>
>
>
>
> [0] https://connochaetos.org/
>
>
> On Thu, 21 Dec 2017 10:54:29 -0500
> Donald Robertson  wrote:
>
>> Hello all,
>>
>> We just wanted to give the list a heads up on our forthcoming
>> announcement of adding PureOS to our list of endorsed distros. Quite
>> some time ago they wrote to this list and got feedback about their
>> distro. From that time, they've been working with us to address concerns
>> that were brought up in that thread, such as removing leftover
>> problematic code from Firefox, fixing up their bug tracker, and making
>> clearer the separation between Purism and PureOS. They've addressed our
>> concerns, and are actively maintaining freedom-related issues in their
>> bug tracker
>> . The
>> endorsement program depends on the community to help projects to
>> maintain their free status, and we are satisfied with PureOS's
>> commitment to maintaining that status. After this announcement, we'll be
>> moving our focus to working with other pending candidates for
>> endorsement, but maintaining a free distro is a task that needs
>> everyone's help. If there are issues that you can help PureOS with,
>> please do make use of the bug tracker, and make sure to CC
>>  so you can be rewarded with a GNU Buck
>>  if you do find a problem.
>> Thank you all for your help in bringing PureOS up to speed, and we look
>> forward to your continued help on future distros seeking endorsement.
>> -- 
>> Donald R. Robertson, III, J.D.
>> Licensing & Compliance Manager
>> Free Software Foundation
>> 51 Franklin Street, Fifth Floor
>> Boston, MA 02110
>> Phone +1-617-542-5942
>> Fax +1-617-542-2652 ex. 56
>>