Re: hang while updating pkg_rolling-replace libvdpau
On Tue, 8 Sep 2020 at 08:07, Martin Husemann wrote: > > On Mon, Sep 07, 2020 at 11:40:58PM +0200, Riccardo Mottola wrote: > > > It mentions a workaround, but what does it mean to: > > > dd if=/dev/urandom of=/dev/random bs=32 count=1 > > > sysctl -w kern.entropy.consolidate=1 > > > > > > I tried this "quick fix" and then was able to leave the laptop several hours > > upgrading, so indeed it it solves the issue. I suppose this fix needs to > > repeated at every boot? > > Usualy not, a variation of it should happen at install time on all systems > not providing a (good enough) random number generator internally. Taylor > is working on covering more systems. Installation will (in the future) > detect this state and offer solutions. > > Once the "full entropy" state is reached, it usually persists - on proper > shutdown the seed file is written and restored on next boot. > > This is all kind of in flux currently, so there is quite a bit of confusion > and different advices floating around. To add to the confusion, perhaps related to this, I've been getting random hangs when using some of the (admittedly over the top) zsh prompts from oh_my_zsh, when they involve git invocation. Roughly 25% of all commands executed with one particular prompt setting The trace is as follows, if it is of interest: ... [Switching to LWP 27612 of process 16821] 0x74480bca88ba in ___lwp_park60 () from /usr/lib/libc.so.12 (gdb) thread apply all bt Thread 2 (LWP 16821 of process 16821): #0 0x74480bc4556a in _lwp_wait () from /usr/lib/libc.so.12 #1 0x74480c40c754 in pthread_join () from /usr/lib/libpthread.so.1 #2 0x0054ba92 in preload_index () #3 0x0055a3e8 in refresh_index () #4 0x00425510 in cmd_status () #5 0x004053fe in handle_builtin () #6 0x00406301 in cmd_main () #7 0x005ed143 in main () Thread 1 (LWP 27612 of process 16821): #0 0x74480bca88ba in ___lwp_park60 () from /usr/lib/libc.so.12 #1 0x74480c409791 in ?? () from /usr/lib/libpthread.so.1 #2 0x74480bcd6e1a in je_malloc_mutex_lock_slow () from /usr/lib/libc.so.12 #3 0x74480bd0e601 in je_arena_choose_hard () from /usr/lib/libc.so.12 #4 0x74480bcb594f in je_tsd_tcache_data_init () from /usr/lib/libc.so.12 #5 0x74480bcb5b99 in je_tsd_tcache_enabled_data_init () from /usr/lib/libc.so.12 #6 0x74480bcb1c19 in je_tsd_fetch_slow () from /usr/lib/libc.so.12 #7 0x74480bd0e93d in malloc () from /usr/lib/libc.so.12 #8 0x005ce9fb in xrealloc () #9 0x005a2e20 in strbuf_grow () #10 0x005ad07b in lstat_cache_matchlen () #11 0x005ad1b5 in threaded_has_symlink_leading_path () #12 0x0054b808 in preload_thread () #13 0x74480c40bee0 in ?? () from /usr/lib/libpthread.so.1 #14 0x74480bc924e0 in ?? () from /usr/lib/libc.so.12 #15 0x in ?? () (gdb) quit The process - again, as with cmake - resumes after quitting gdb. > > Martin Chavdar --
Re: hang while updating pkg_rolling-replace libvdpau
On Mon, Sep 07, 2020 at 11:40:58PM +0200, Riccardo Mottola wrote: > > It mentions a workaround, but what does it mean to: > > dd if=/dev/urandom of=/dev/random bs=32 count=1 > > sysctl -w kern.entropy.consolidate=1 > > > I tried this "quick fix" and then was able to leave the laptop several hours > upgrading, so indeed it it solves the issue. I suppose this fix needs to > repeated at every boot? Usualy not, a variation of it should happen at install time on all systems not providing a (good enough) random number generator internally. Taylor is working on covering more systems. Installation will (in the future) detect this state and offer solutions. Once the "full entropy" state is reached, it usually persists - on proper shutdown the seed file is written and restored on next boot. This is all kind of in flux currently, so there is quite a bit of confusion and different advices floating around. Martin
Re: hang while updating pkg_rolling-replace libvdpau
Hi! I have not seen a reply to my mail, but let me follow up. On 04/09/2020 11:59, Riccardo Mottola wrote: It mentions a workaround, but what does it mean to: dd if=/dev/urandom of=/dev/random bs=32 count=1 sysctl -w kern.entropy.consolidate=1 I tried this "quick fix" and then was able to leave the laptop several hours upgrading, so indeed it it solves the issue. I suppose this fix needs to repeated at every boot? The laptop eventually hang up, refusing login, keystrokes, ctrl-c or else (but interestingly, console swapping worked, so keyboard input was happening). Never seen something like this on this laptop, running NetBSD (current) since years. As written though, I do not have an Ivy Bridge, but an earlier system. Any hints on a fix given this information? It is evidently CPU dependant, but happens on more CPUs. Riccardo
Re: hang while updating pkg_rolling-replace libvdpau
Hi Chavdar, Chavdar Ivanov wrote: (If you do find a bug in pkg_rr, of which there are many, please do report it. But it is confusing to people to conflate what broke with the program that merely chose the sequence.) In this case I don't think it is anything to do with pkg_rolling-replace; I've reported a few of these hangs, which happen to happen during pkg_rolling-replace, but involve most often cmake, but other programs as well. Apparently there are similarities in the traces, perhaps pointing to the thread model and execution. In all these occasions the process in question continued to a successful conclusion after attaching and then detaching with gdb. I think you are right. I then tried manually updating some packages with "make replace"... and I got a hang also when building glib2. So pkg_rolling-replacing is just on top and has nothing to do with it. The place and file where the hang occours is consistent. In my case, libvdpau is not using cmake! you can see on the command line CMAKE=false It is involving make, python and meson: 9344 pts/5 T0:00.11 /usr/bin/make _MAKE OPSYS OS_VERSION LOWER_OPSYS _PKGSRCDIR PKGTOOLS_VERSION _CC _PATH_ORIG _PKGSRC_BARRIER ALLOW_VULNERABLE_PACKAGES replace 10978 pts/5 T0:00.16 make replace 18674 pts/5 O+ 0:00.01 ps -a 19416 pts/5 T0:00.52 /usr/pkg/bin/python3.7 /usr/pkg/bin/meson --prefix /usr/pkg --libdir lib --mandir man --sysconfdir /usr/pkg/etc --buildtype=plain -Degdir=/usr/pkg/share/examples 20711 pts/5 S0:00.01 -sh 20839 pts/5 T0:00.00 /bin/sh -c set -e;\t\t\t\t\t if test -n "" && /usr/sbin/pkg_info -K /var/db/pkg -qe libvdpau-1.4; then echo ===\\> "Skipping installation of already handled pa Riccardo
Re: hang while updating pkg_rolling-replace libvdpau
Hi David,
David Shao wrote:
Examine recently posted
kern/55641: Recent changes to random/entropy "pkgsrc devel brick" an
Intel Ivy Bridge system, with workaround
It mentions a workaround, but what does it mean to:
dd if=/dev/urandom of=/dev/random bs=32 count=1
sysctl -w kern.entropy.consolidate=1
copy some data from one random generator to another one? Before doing so
I want to know it is fine :)
Try Ctrl-C after the hang. Is there at the bottom of the trace a call
to a random number function?
no, see below.
Do you have something like an Intel Ivy Bridge system?
No, it is an older CPU, it should be a Penryn:
[ 1.03] cpu0 at mainbus0 apid 0
[ 1.03] cpu0: Use lfence to serialize rdtsc
[ 1.03] cpu0: Intel(R) Core(TM)2 Duo CPU T6670 @ 2.20GHz,
id 0x1067a
[ 1.03] cpu0: node 0, package 0, core 0, smt 0
[ 1.03] cpu1 at mainbus0 apid 1
[ 1.03] cpu1: Intel(R) Core(TM)2 Duo CPU T6670 @ 2.20GHz,
id 0x1067a
[ 1.03] cpu1: node 0, package 0, core 1, smt 0
Are you getting some warning message on boot about entropy?
Yes, I see two warnings:
disc$ dmesg | grep entro
[ 1.00] entropy: no seed from bootloader
[ 40596.326165] entropy: WARNING: extracting entropy too early
If I hit ctrl-c I see something inside meson and python and the last is
a random function. So there is probably some similarity.
sudo make replace
^CTraceback (most recent call last):
File "/usr/pkg/bin/meson", line 11, in
load_entry_point('meson==0.55.1', 'console_scripts', 'meson')()
File
"/usr/pkg/lib/python3.7/site-packages/pkg_resources/__init__.py", line
489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File
"/usr/pkg/lib/python3.7/site-packages/pkg_resources/__init__.py", line
2852, in load_entry_point
return ep.load()
File
"/usr/pkg/lib/python3.7/site-packages/pkg_resources/__init__.py", line
2443, in load
return self.resolve()
File
"/usr/pkg/lib/python3.7/site-packages/pkg_resources/__init__.py", line
2449, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/pkg/lib/python3.7/site-packages/mesonbuild/mesonmain.py",
line 25, in
from . import mconf, mdist, minit, minstall, mintro, msetup, mtest,
rewriter, msubprojects, munstable_coredata, mcompile
File "/usr/pkg/lib/python3.7/site-packages/mesonbuild/minstall.py",
line 22, in
from .mtest import rebuild_all
File "/usr/pkg/lib/python3.7/site-packages/mesonbuild/mtest.py", line
26, in
import multiprocessing
File "/usr/pkg/lib/python3.7/multiprocessing/__init__.py", line 16,
in
from . import context
File "/usr/pkg/lib/python3.7/multiprocessing/context.py", line 5, in
from . import process
File "/usr/pkg/lib/python3.7/multiprocessing/process.py", line 363,
in
_current_process = _MainProcess()
File "/usr/pkg/lib/python3.7/multiprocessing/process.py", line 347,
in __init__
self._config = {'authkey': AuthenticationString(os.urandom(32)),
KeyboardInterrupt
Re: hang while updating pkg_rolling-replace libvdpau
Examine recently posted kern/55641: Recent changes to random/entropy "pkgsrc devel brick" an Intel Ivy Bridge system, with workaround Try Ctrl-C after the hang. Is there at the bottom of the trace a call to a random number function? Do you have something like an Intel Ivy Bridge system? Are you getting some warning message on boot about entropy?
Re: hang while updating pkg_rolling-replace libvdpau
On Thu, 3 Sep 2020 at 12:19, Greg Troxel wrote: > > > Riccardo Mottola writes: > > > I finished updating all my core system to current on i386-64, kernel, > > userland, etc. > > > > Now I launched pkg_rolling replace, it crunches through several > > packages, but then hangs. > > > > > > I tried running it several times, rebooting in between... but > > nothing. What is "hang" ? The CPU stays idle too. Hangs exactly there. > > pkg_rr is just a shell script. As always, I ask that you see what order > it does "make replace package clean" and then run the problematic > command by itself, without pkg_rr, and then report that intead. > > (If you do find a bug in pkg_rr, of which there are many, please do > report it. But it is confusing to people to conflate what broke with > the program that merely chose the sequence.) In this case I don't think it is anything to do with pkg_rolling-replace; I've reported a few of these hangs, which happen to happen during pkg_rolling-replace, but involve most often cmake, but other programs as well. Apparently there are similarities in the traces, perhaps pointing to the thread model and execution. In all these occasions the process in question continued to a successful conclusion after attaching and then detaching with gdb. --
Re: hang while updating pkg_rolling-replace libvdpau
Riccardo Mottola writes: > I finished updating all my core system to current on i386-64, kernel, > userland, etc. > > Now I launched pkg_rolling replace, it crunches through several > packages, but then hangs. > > > I tried running it several times, rebooting in between... but > nothing. What is "hang" ? The CPU stays idle too. Hangs exactly there. pkg_rr is just a shell script. As always, I ask that you see what order it does "make replace package clean" and then run the problematic command by itself, without pkg_rr, and then report that intead. (If you do find a bug in pkg_rr, of which there are many, please do report it. But it is confusing to people to conflate what broke with the program that merely chose the sequence.) signature.asc Description: PGP signature
Re: hang while updating pkg_rolling-replace libvdpau
On Wed, Sep 02, 2020 at 11:50:52PM +0200, Riccardo Mottola wrote: > I finished updating all my core system to current on i386-64, kernel, > userland, etc. > > Now I launched pkg_rolling replace, it crunches through several packages, > but then hangs. > > > I tried running it several times, rebooting in between... but nothing. What > is "hang" ? The CPU stays idle too. Hangs exactly there. What does e.g., ps auxwwd say when it hangs? Maybe another manifestation of "cmake hanging" http://mail-index.netbsd.org/current-users/2020/05/24/msg038692.html ? Cheers, Patrick
