Re: entropy lockup

[Ian Lepore]

On Tue, 2017-09-12 at 09:20 +1000, Morgan Reed wrote:
> In all likelihood the process wasn't "hung" per-se, more likely
> random
> hadn't been seeded yet and as such you *can't* get 4096b of entropy
> out of
> it (so the process was attempting to do its job, just nothing on the
> device).
> 
> The issue is basically that there are very few "good" entropy sources
> in a
> VM environment, and as such (particularly on a machine which is only
> running SSH) there's not enough data available to seed random.
> 
> Check 'man 4 random' for some discussion.
> 

The save-entropy files get written from a crontab entry.  That means
the system is already up and running, which means it's well past the
point of hanging because it needs to be seeded.  Once it is seeded, it
can never "run out of entropy".

So all in all, this is a genuine problem of some sort.  The fact that
the save-entropy process has accumulated 48+ minutes of cpu time is
another problem indicator.  The process state is RL, runnable and
waiting to acquire a lock, which seems contradictory (unless that means
it's waiting for a spinlock maybe).

-- Ian

> On Tue, Sep 12, 2017 at 12:02 AM, Vick Khera  wrote:
> 
> > 
> > I just had a VM running in Google's cloud become totally useless,
> > and I
> > tracked it down to the save-entropy operation.
> > 
> > Basically this process was sucking up all CPU, even when nothing
> > else
> > running other than my ssh shell:
> > 
> > % ps axuw803
> > USER PID  %CPU %MEM  VSZ  RSS TT  STAT STARTED TIME COMMAND
> > operator 803 100.0  0.1 8336 2096  -  RL   08:55   48:20.14 dd
> > if=/dev/random of=saved-entropy.1 bs=4096 count=1
> > 
> > 
> > The process is unkillable, and I cannot even get the system to shut
> > down.
> > That has been hanging for about 10 minutes so far, with the last
> > output
> > being
> > 
> > System shutdown time has arrive90 second watchdSep 11 09:50:02
> > yertle init:
> > /bin/sh on /etc/rc.shutdown terminated abnormally, going to single
> > user
> > mode
> > Sep 11 09:50:47 init: some processes would not die; ps axl advised
> > Waiting (max 60 seconds) for system process `vnlru' to stop... done
> > Waiting (max 60 seconds) for system process `bufdaemon' to stop...
> > done
> > Waiting (max 60 seconds) for system process `syncer' to stop...
> > Syncing disks, vnodes remaining... 4 4 4 4 4 4 4 timed out
> > 2 2 2 2 2 2 2
> > 
> > 
> > Running FreeBSD 11.1-p1 on a 1CPU standard machine in GCE.
> > 
> > What's the proper recovery from this kind of lockup, or how to
> > prevent it?
> > I've never encountered this on a bare metal system, or other KVM
> > based
> > machines.
> > ___
> > freebsd-stable@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebs
> > d.org"
> > 
> 
> 
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: entropy lockup

[Morgan Reed]

In all likelihood the process wasn't "hung" per-se, more likely random
hadn't been seeded yet and as such you *can't* get 4096b of entropy out of
it (so the process was attempting to do its job, just nothing on the
device).

The issue is basically that there are very few "good" entropy sources in a
VM environment, and as such (particularly on a machine which is only
running SSH) there's not enough data available to seed random.

Check 'man 4 random' for some discussion.

On Tue, Sep 12, 2017 at 12:02 AM, Vick Khera  wrote:

> I just had a VM running in Google's cloud become totally useless, and I
> tracked it down to the save-entropy operation.
>
> Basically this process was sucking up all CPU, even when nothing else
> running other than my ssh shell:
>
> % ps axuw803
> USER PID  %CPU %MEM  VSZ  RSS TT  STAT STARTED TIME COMMAND
> operator 803 100.0  0.1 8336 2096  -  RL   08:55   48:20.14 dd
> if=/dev/random of=saved-entropy.1 bs=4096 count=1
>
>
> The process is unkillable, and I cannot even get the system to shut down.
> That has been hanging for about 10 minutes so far, with the last output
> being
>
> System shutdown time has arrive90 second watchdSep 11 09:50:02 yertle init:
> /bin/sh on /etc/rc.shutdown terminated abnormally, going to single user
> mode
> Sep 11 09:50:47 init: some processes would not die; ps axl advised
> Waiting (max 60 seconds) for system process `vnlru' to stop... done
> Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
> Waiting (max 60 seconds) for system process `syncer' to stop...
> Syncing disks, vnodes remaining... 4 4 4 4 4 4 4 timed out
> 2 2 2 2 2 2 2
>
>
> Running FreeBSD 11.1-p1 on a 1CPU standard machine in GCE.
>
> What's the proper recovery from this kind of lockup, or how to prevent it?
> I've never encountered this on a bare metal system, or other KVM based
> machines.
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>



-- 
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-- Benjamin Franklin, 1759
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

entropy lockup

[Vick Khera]

I just had a VM running in Google's cloud become totally useless, and I
tracked it down to the save-entropy operation.

Basically this process was sucking up all CPU, even when nothing else
running other than my ssh shell:

% ps axuw803
USER PID  %CPU %MEM  VSZ  RSS TT  STAT STARTED TIME COMMAND
operator 803 100.0  0.1 8336 2096  -  RL   08:55   48:20.14 dd
if=/dev/random of=saved-entropy.1 bs=4096 count=1


The process is unkillable, and I cannot even get the system to shut down.
That has been hanging for about 10 minutes so far, with the last output
being

System shutdown time has arrive90 second watchdSep 11 09:50:02 yertle init:
/bin/sh on /etc/rc.shutdown terminated abnormally, going to single user mode
Sep 11 09:50:47 init: some processes would not die; ps axl advised
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 4 4 4 4 4 4 4 timed out
2 2 2 2 2 2 2


Running FreeBSD 11.1-p1 on a 1CPU standard machine in GCE.

What's the proper recovery from this kind of lockup, or how to prevent it?
I've never encountered this on a bare metal system, or other KVM based
machines.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Unusually high "Wired" memory

[Borja Marcos]

> On 11 Sep 2017, at 11:09, Borja Marcos  wrote:
> 
> 
> Hi,
> 
> Since I’ve updated a machine to 11.1-STABLE I am seeing a rather unusual 
> growth of Wired memory.
> 
> Any hints on what might have changed from 11-RELEASE to 11.1-RELEASE and 
> 11.1-STABLE?


Evil Mailman scrubbed my attachment :/

http://frobula.crabdance.com:8001/Geigerfun/memoryzones.png





Borja

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Unusually high "Wired" memory

[Borja Marcos]

Hi,

Since I’ve updated a machine to 11.1-STABLE I am seeing a rather unusual growth 
of Wired memory.

Any hints on what might have changed from 11-RELEASE to 11.1-RELEASE and 
11.1-STABLE?


Thanks!



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"