Re: Logging a kernel message when blocking on entropy
On Tue, 29 Sep 2020 15:03:59 +0300, Andreas Gustafsson wrote: > Hauke Fath wrote: >> I have seen kernel subsystems spam the console to the point of making >> it unusable, which can get quite awkward on production machines. > > I have already explained why I believe it's not a relevant scenario > for this change. Okay, I guess I must have missed that. > I just don't want to end up being held responsible for problems caused by rate > limiting I never wanted in the first place. Understandable. Cheerio, Hauke -- Hauke Fath Grabengasse 57 64372 Ober-Ramstadt Germany
Re: Logging a kernel message when blocking on entropy
Hauke Fath wrote: > On Tue, 29 Sep 2020 11:09:25 +0300, Andreas Gustafsson wrote: > > I have now committed the code to log the message, without rate > > limiting. If a consensus should arise that rate limiting the message > > is a good idea, the code to do that should be committed by someone in > > favor of it. > > I have seen kernel subsystems spam the console to the point of making > it unusable, which can get quite awkward on production machines. > > While I don't know if this is a relevant scenario for your change, > declaring it "somebody else's problem" has a funny ring. I have already explained why I believe it's not a relevant scenario for this change. And I don't mean to make it someone else's problem - I take full responsibility for my commit, and if it turns out that it actually causes spammage, I will fix that one way or another. And similarly, if rate limiting is added and causes problems, I expect the developer adding it to take responsibility for that commit. I just don't want to end up being held responsible for problems caused by rate limiting I never wanted in the first place. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, 29 Sep 2020 11:09:25 +0300, Andreas Gustafsson wrote: > I have now committed the code to log the message, without rate > limiting. If a consensus should arise that rate limiting the message > is a good idea, the code to do that should be committed by someone in > favor of it. I have seen kernel subsystems spam the console to the point of making it unusable, which can get quite awkward on production machines. While I don't know if this is a relevant scenario for your change, declaring it "somebody else's problem" has a funny ring. Cheerio, Hauke -- Hauke Fath Grabengasse 57 64372 Ober-Ramstadt Germany
Re: Logging a kernel message when blocking on entropy
I have now committed the code to log the message, without rate limiting. If a consensus should arise that rate limiting the message is a good idea, the code to do that should be committed by someone in favor of it. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
nia wrote: > Unless you buy one of the snake oil devices Mr. Gustafsson sells, of > course. The devices you call "snake oil" and me pushing for NetBSD to stop generating predictable keys are both aspects of me working to address entropy related vulnerabilities ever since recognizing them as a major weak link in Internet security while working on BIND 9 some 20 years ago. But of course I have no way of proving my sincere intent to you, just like you have no way of proving to me that you are not paid by the NSA to undermine the security of NetBSD. So please, let's stop with the ad hominem attacks and focus on the technical issues. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
nia wrote: > On Tue, Sep 22, 2020 at 02:32:23PM +0200, Manuel Bouyer wrote: > > OK, so the printf should never happen when the system has been properly > > configured. In this case I have no objection. > > No, it will happen frequently in VMs and on non-recent-x86 hardware. The cases of processes blocking on randomness we tend to see reports about are pkgsrc builds and web browsers. Since entropy does not run out in -current as already discussed on this list, each hanging pkgsrc build or web browser will cause the message to be logged exactly once. How often do they hang for you? If PR 55659 is fixed, there will be more cases where the message is logged (at least ssh-keyegn), but still only one per blocking process. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
mar...@duskware.de (Martin Husemann) writes: >On all of these it *still* is a setup error that needs to be avoided. Every VM is installed manually by someone rolling dice so that such setup errors cannot occur. -- -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
Re: Logging a kernel message when blocking on entropy
On Thu, Sep 24, 2020 at 06:37:43AM +, nia wrote: > On Tue, Sep 22, 2020 at 02:32:23PM +0200, Manuel Bouyer wrote: > > OK, so the printf should never happen when the system has been properly > > configured. In this case I have no objection. > > No, it will happen frequently in VMs and on non-recent-x86 hardware. On all of these it *still* is a setup error that needs to be avoided. You are talking about a different "frequently". Many modern aarch64 SoCs will be affected, all VMs with no viornd(4) support, as well as most non x86 machines w/o special hardware (like ualea(4)). When these machines are properly setup *once* (ideally at install time), it will not happen to them ever again (that is the "frequently" Manuel was talking about). The problem is with USB images and VM clones that are not installed at all and we have no good idea (besides properly documenting and suggesting manual actions) so far. Martin
Re: Logging a kernel message when blocking on entropy
On Thu, Sep 24, 2020 at 06:37:43AM +, nia wrote: > On Tue, Sep 22, 2020 at 02:32:23PM +0200, Manuel Bouyer wrote: > > OK, so the printf should never happen when the system has been properly > > configured. In this case I have no objection. > > No, it will happen frequently in VMs and on non-recent-x86 hardware. Unless you buy one of the snake oil devices Mr. Gustafsson sells, of course.
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 02:32:23PM +0200, Manuel Bouyer wrote: > OK, so the printf should never happen when the system has been properly > configured. In this case I have no objection. No, it will happen frequently in VMs and on non-recent-x86 hardware.
Re: Logging a kernel message when blocking on entropy
Taylor R Campbell wrote:
> I think this is fine, but it should be rate-limited the same way other
> messages in kern_entropy.c are rate-limited.
>
> if (ratecheck(&lasttime, &interval))
> printf("...");
I would prefer them not to be rate limited. Repeated messages are
useful because they will typically contain different PIDs and program
names.
Consider a system logging the following message to the console at boot
time:
[ 22.8406587] entropy: pid 188 (syslogd) blocking due to lack of entropy
and then hanging. You hit control-c to recover, the boot continues
but then immediately hangs again as another process also blocks
on entropy. In this case, I would like a second message to be logged
so that it is immediately apparent that another process has blocked,
and which process that is.
Or if you start two different daemons in parallel and both block
immediately, it would be helpful to get messages identifying both of
them.
--
Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, 22 Sep 2020, Taylor R Campbell wrote:
Date: Tue, 22 Sep 2020 13:07:04 +0300
From: Andreas Gustafsson
The following patch will cause a kernel message to be logged when a
process blocks on /dev/random or some other randomness API. It may
help some users befuddled by pkgsrc builds blocking on /dev/random,
and I'm finding it useful when testing changes aimed at fixing PR
55659.
OK to commit?
I think this is fine, but it should be rate-limited the same way other
messages in kern_entropy.c are rate-limited.
if (ratecheck(&lasttime, &interval))
printf("...");
Not sure if that's really necessary. There's a maximum of one message
per thread - ever, and even if every thread on the machine blocks, the
limit on number of threads stops you at 32k messages. As soon as one
thread un-blocks, nothing will ever block again, so that feels like a
hard limit.
Of course, since the rate-limiting code is already written, it wouldn't
hurt to use it. I'm just pointing out that there's a natural limit
inherent in the code!
:)
++--+---+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses: |
| (Retired) | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org |
++--+---+
Re: Logging a kernel message when blocking on entropy
> Date: Tue, 22 Sep 2020 13:07:04 +0300
> From: Andreas Gustafsson
>
> The following patch will cause a kernel message to be logged when a
> process blocks on /dev/random or some other randomness API. It may
> help some users befuddled by pkgsrc builds blocking on /dev/random,
> and I'm finding it useful when testing changes aimed at fixing PR
> 55659.
>
> OK to commit?
I think this is fine, but it should be rate-limited the same way other
messages in kern_entropy.c are rate-limited.
if (ratecheck(&lasttime, &interval))
printf("...");
Re: Logging a kernel message when blocking on entropy
Andreas Gustafsson writes: > The following patch will cause a kernel message to be logged when a > process blocks on /dev/random or some other randomness API. It may > help some users befuddled by pkgsrc builds blocking on /dev/random, > and I'm finding it useful when testing changes aimed at fixing PR > 55659. I'm in favor. I have not dug in to the brave new entropy world. I'm sure it's better in many ways, but it also seems like people/systems that used to not end up blocked before now do, apparently because some sources that used to be considered ok (timing of events) no longer are. So I think people should be given clues - things appear a bit too difficult now. signature.asc Description: PGP signature
Re: Logging a kernel message when blocking on entropy
On Tue, 22 Sep 2020 at 12:35, Manuel Bouyer wrote: > > On Tue, Sep 22, 2020 at 02:31:50PM +0300, Andreas Gustafsson wrote: > > Manuel Bouyer wrote: > > > I'm not sure we want a user-triggerable kernel printf enabled by default. > > > This could be used to DOS the system (especially on serial consoles) > > > > You can already trigger kernel printfs as an unprivileged user. > > The first one that comes to mind is "sorry, pid %d was killed: > > orphaned traced process", but I'm sure there are many others. > > I think we should find and remove theses (or make them conditional) > instead of adding unconditional new ones Maybe a standard way of rate limiting such messages, including indicating how many were skipped due to rate limiting when the next one gets printed? David
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 03:23:59PM +0300, Andreas Gustafsson wrote: > Manuel Bouyer wrote: > > > In -current, entropy does not run out. > > > > So, how can it block ? > > When there's too little entropy to begin with. Once you have > gathered enough, it unblocks, and never blocks again. > > This is assuming default settings. If you actually want entropy > to run out, you can do "sysctl -w kern.entropy.depletion=1", but > there's no good reason to ever do that outside of testing. > -- > Andreas Gustafsson, g...@gson.org On Tue, Sep 22, 2020 at 02:25:25PM +0200, Martin Husemann wrote: > On Tue, Sep 22, 2020 at 02:12:19PM +0200, Manuel Bouyer wrote: > > So, how can it block ? > > When the system never had enough entropy. > > I would consider this a bug in the setup of the system, but as of now we do > not deal with it at all during installation, and on systems that are not > installed (bootable images) it is even harder. > > Sysinst will complain about it soon (and offer options to help fix it). OK, so the printf should never happen when the system has been properly configured. In this case I have no objection. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 02:12:19PM +0200, Manuel Bouyer wrote: > So, how can it block ? When the system never had enough entropy. I would consider this a bug in the setup of the system, but as of now we do not deal with it at all during installation, and on systems that are not installed (bootable images) it is even harder. Sysinst will complain about it soon (and offer options to help fix it). Martin
Re: Logging a kernel message when blocking on entropy
Manuel Bouyer wrote: > > In -current, entropy does not run out. > > So, how can it block ? When there's too little entropy to begin with. Once you have gathered enough, it unblocks, and never blocks again. This is assuming default settings. If you actually want entropy to run out, you can do "sysctl -w kern.entropy.depletion=1", but there's no good reason to ever do that outside of testing. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 03:05:34PM +0300, Andreas Gustafsson wrote: > Manuel Bouyer wrote: > > If you run a dd on /dev/random I guess the system will run out of > > entropy pretty fast. > > In -current, entropy does not run out. So, how can it block ? -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Logging a kernel message when blocking on entropy
Manuel Bouyer wrote: > If you run a dd on /dev/random I guess the system will run out of > entropy pretty fast. In -current, entropy does not run out. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 02:50:37PM +0300, Andreas Gustafsson wrote: > Manuel Bouyer wrote: > > I think we should find and remove theses (or make them conditional) > > instead of adding unconditional new ones > > It would already be conditional in the sense that it's only printed if > the system has no entropy. If a multi-user system is lacking entropy, > a user spamming the console about it is doing the administrator a > favor. If you run a dd on /dev/random I guess the system will run out of entropy pretty fast. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Logging a kernel message when blocking on entropy
Manuel Bouyer wrote: > I think we should find and remove theses (or make them conditional) > instead of adding unconditional new ones It would already be conditional in the sense that it's only printed if the system has no entropy. If a multi-user system is lacking entropy, a user spamming the console about it is doing the administrator a favor. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 02:31:50PM +0300, Andreas Gustafsson wrote: > Manuel Bouyer wrote: > > I'm not sure we want a user-triggerable kernel printf enabled by default. > > This could be used to DOS the system (especially on serial consoles) > > You can already trigger kernel printfs as an unprivileged user. > The first one that comes to mind is "sorry, pid %d was killed: > orphaned traced process", but I'm sure there are many others. I think we should find and remove theses (or make them conditional) instead of adding unconditional new ones -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Logging a kernel message when blocking on entropy
Manuel Bouyer wrote: > I'm not sure we want a user-triggerable kernel printf enabled by default. > This could be used to DOS the system (especially on serial consoles) You can already trigger kernel printfs as an unprivileged user. The first one that comes to mind is "sorry, pid %d was killed: orphaned traced process", but I'm sure there are many others. -- Andreas Gustafsson, g...@gson.org
Re: Logging a kernel message when blocking on entropy
On Tue, Sep 22, 2020 at 01:07:04PM +0300, Andreas Gustafsson wrote: > All, > > The following patch will cause a kernel message to be logged when a > process blocks on /dev/random or some other randomness API. It may > help some users befuddled by pkgsrc builds blocking on /dev/random, > and I'm finding it useful when testing changes aimed at fixing PR > 55659. > > OK to commit? I'm not sure we want a user-triggerable kernel printf enabled by default. This could be used to DOS the system (especially on serial consoles) -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Logging a kernel message when blocking on entropy
All,
The following patch will cause a kernel message to be logged when a
process blocks on /dev/random or some other randomness API. It may
help some users befuddled by pkgsrc builds blocking on /dev/random,
and I'm finding it useful when testing changes aimed at fixing PR
55659.
OK to commit?
Index: src/sys/kern/kern_entropy.c
===
RCS file: /bracket/repo/src/sys/kern/kern_entropy.c,v
retrieving revision 1.23
diff -u -r1.23 kern_entropy.c
--- src/sys/kern/kern_entropy.c 14 Aug 2020 00:53:16 - 1.23
+++ src/sys/kern/kern_entropy.c 20 Sep 2020 13:53:46 -
@@ -1297,6 +1297,9 @@
/* Wait for some entropy to come in and try again. */
KASSERT(E->stage >= ENTROPY_WARM);
+ printf("entropy: pid %d (%s) blocking due to lack of entropy\n",
+ curproc->p_pid, curproc->p_comm);
+
if (ISSET(flags, ENTROPY_SIG)) {
error = cv_wait_sig(&E->cv, &E->lock);
if (error)
--
Andreas Gustafsson, g...@gson.org
