NetBSD on bhyve on TrueNAS Core
Hi! Has anyone had any success with running a NetBSD VM in bhyve under TrueNAS? I tried booting the install CD-ROM (both 9 and current, amd64) with UEFI and VNC. In each case, I see the bootloader, then the graphical console and the kernel starts booting. Then about 5 seconds later, VNC simply disconnects and does not let me reconnect. I assume that the hypervisor is crashing at that point. All the how-tos I found look somewhat stale and go through all sorts of gymnastics (such as using grub) to get non-UEFI boot working. Any suggestions? -- Benny
Re: NetBSD on bhyve on TrueNAS Core
On Sat, 24 Apr 2021 at 09:44, Benny Siegert wrote: > > Hi! > > Has anyone had any success with running a NetBSD VM in bhyve under > TrueNAS? I tried booting the install CD-ROM (both 9 and current, > amd64) with UEFI and VNC. My -current (still 9.99.79) runs under TrueNAS just fine, with UEFI and VNC, as you say. > > In each case, I see the bootloader, then the graphical console and the > kernel starts booting. Then about 5 seconds later, VNC simply > disconnects and does not let me reconnect. I assume that the > hypervisor is crashing at that point. > > All the how-tos I found look somewhat stale and go through all sorts > of gymnastics (such as using grub) to get non-UEFI boot working. > > Any suggestions? I can't remember, tbh; I had some stumble to begin with, but managed to sort it at the time. I use the VNC connection via the browser I am accessing the TrueNAS interface with. I'll try to install the latest 9.99.81 when the build I am doing now finishes. dmesg | grep -i bhyve [ 1.03] BHYVE (1.0) [ 1.03] ACPI: RSDP 0x1FB88014 24 (v02 BHYVE ) [ 1.03] ACPI: XSDT 0x1FB870E8 4C (v01 BHYVE BVFACP 0001 0113) [ 1.03] ACPI: FACP 0x1FB86000 F4 (v04 BHYVE BVFACP 0001 BHYV 0001) [ 1.03] ACPI: DSDT 0x1EA98000 00191A (v02 BHYVE BVDSDT 0001 INTL 20200430) [ 1.03] ACPI: HPET 0x1FB85000 38 (v01 BHYVE BVHPET 0001 BHYV 0001) [ 1.03] ACPI: APIC 0x1FB84000 4A (v01 BHYVE BVMADT 0001 BHYV 0001) [ 1.03] ACPI: MCFG 0x1FB83000 3C (v01 BHYVE BVMCFG 0001 BHYV 0001) [ 1.03] ACPI: SPCR 0x1FB82000 50 (v01 BHYVE BVSPCR 0001 BHYV 0001) [ 1.03] acpi0: X/RSDT: OemId , AslId <,0113> [ 2.989693] wd0: ... > > -- > Benny --
Re: NetBSD on bhyve on TrueNAS Core
On Sat, Apr 24, 2021 at 6:30 PM Chavdar Ivanov wrote: > My -current (still 9.99.79) runs under TrueNAS just fine, with UEFI > and VNC, as you say. That sounds good. What settings do you use? In particular, the web UI gives me Windows, Linux and FreeBSD as the choice of host OS. I chose FreeBSD but perhaps that was a mistake? > I use the VNC connection via the browser I am accessing the TrueNAS > interface with. Same. -- Benny
XDG_RUNTIME_DIR
I have built from pkgsrc 'lyx' with the qt5 libs. Typing 'lyx' at a command prompt yields: $ lyx QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-bob' I pecked through the qt5 pkgsrc dirs (for the qt5 packages that were built) hoping to find a MESSAGE containing instructions for setting XDG_RUNTIME_DIR. (Not sure why I suspected qt5 in that connection.) Any and all suggestions will be examined. From my 9.99.81 amd64 system with pkgsrc via git 4/15/21. Thank youse. -- What's going on with Bob? https://zeus.jtan.com/~wingnut
Re: NetBSD on bhyve on TrueNAS Core
On Sat, 24 Apr 2021 at 18:02, Benny Siegert wrote: > > On Sat, Apr 24, 2021 at 6:30 PM Chavdar Ivanov wrote: > > My -current (still 9.99.79) runs under TrueNAS just fine, with UEFI > > and VNC, as you say. > > That sounds good. What settings do you use? In particular, the web UI > gives me Windows, Linux and FreeBSD as the choice of host OS. I chose > FreeBSD but perhaps that was a mistake? Same... > > > I use the VNC connection via the browser I am accessing the TrueNAS > > interface with. > > Same. But - my working NetBSD system was created on my TrueNAS server ages ago - well, when 9.99.79 was -current, I mean, and it still works. I tried to install 9.99.81 from my last ISO, from 22nd of April, and it fails the same way for me as well, and I see in the host dmesg: pid 99497 (bhyve), jid 0, uid 0: exited on signal 11 The interface still shows the guest as running, and attempts to stop it lead to python3.8 error trace with an unknown reason. It is either a TrueNAS bug, or something in the latest builds is bugging bhyve. I'll try to install some Linux perhaps, t see if I can create new VMs at all. > > > > -- > Benny --
Re: NetBSD on bhyve on TrueNAS Core
On Sat, 24 Apr 2021 at 18:40, Chavdar Ivanov wrote: > > On Sat, 24 Apr 2021 at 18:02, Benny Siegert wrote: > > > > On Sat, Apr 24, 2021 at 6:30 PM Chavdar Ivanov wrote: > > > My -current (still 9.99.79) runs under TrueNAS just fine, with UEFI > > > and VNC, as you say. > > > > That sounds good. What settings do you use? In particular, the web UI > > gives me Windows, Linux and FreeBSD as the choice of host OS. I chose > > FreeBSD but perhaps that was a mistake? > > Same... > > > > > > I use the VNC connection via the browser I am accessing the TrueNAS > > > interface with. > > > > Same. > > But - my working NetBSD system was created on my TrueNAS server ages > ago - well, when 9.99.79 was -current, I mean, and it still works. > > I tried to install 9.99.81 from my last ISO, from 22nd of April, and > it fails the same way for me as well, and I see in the host dmesg: > > pid 99497 (bhyve), jid 0, uid 0: exited on signal 11 > > The interface still shows the guest as running, and attempts to stop > it lead to python3.8 error trace with an unknown reason. > > It is either a TrueNAS bug, or something in the latest builds is > bugging bhyve. I'll try to install some Linux perhaps, t see if I can > create new VMs at all. > > > > > > > > > -- > > Benny userconf disable xhci* > > > > -- > --
Re: NetBSD on bhyve on TrueNAS Core
On Sat, 24 Apr 2021 at 18:44, Chavdar Ivanov wrote: > > On Sat, 24 Apr 2021 at 18:40, Chavdar Ivanov wrote: > > > > On Sat, 24 Apr 2021 at 18:02, Benny Siegert wrote: > > > > > > On Sat, Apr 24, 2021 at 6:30 PM Chavdar Ivanov wrote: > > > > My -current (still 9.99.79) runs under TrueNAS just fine, with UEFI > > > > and VNC, as you say. > > > > > > That sounds good. What settings do you use? In particular, the web UI > > > gives me Windows, Linux and FreeBSD as the choice of host OS. I chose > > > FreeBSD but perhaps that was a mistake? > > > > Same... > > > > > > > > > I use the VNC connection via the browser I am accessing the TrueNAS > > > > interface with. > > > > > > Same. > > > > But - my working NetBSD system was created on my TrueNAS server ages > > ago - well, when 9.99.79 was -current, I mean, and it still works. > > > > I tried to install 9.99.81 from my last ISO, from 22nd of April, and > > it fails the same way for me as well, and I see in the host dmesg: > > > > pid 99497 (bhyve), jid 0, uid 0: exited on signal 11 > > > > The interface still shows the guest as running, and attempts to stop > > it lead to python3.8 error trace with an unknown reason. > > > > It is either a TrueNAS bug, or something in the latest builds is > > bugging bhyve. I'll try to install some Linux perhaps, t see if I can > > create new VMs at all. > > > > > > > > > > > > > > -- > > > Benny > > > userconf disable xhci* I mean, for the installation you will have to quickly go to the vnc window, interrupt the boot to command line and enter 'boot -c'; then disable xhci quit Then go through the installation as normal, GPT etc. After it finishes, go to the command line, mount your root dk (should be dk1) and add the above line to /boot.cfg . Chavdar > > > > > > > > > -- > > > > > > -- > --
Re: XDG_RUNTIME_DIR
On Sat 24 Apr 2021 at 13:39:03 -0400, Bob Bernstein wrote: > I have built from pkgsrc 'lyx' with the qt5 libs. Typing 'lyx' at a command > prompt yields: > > $ lyx > QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-bob' > > I pecked through the qt5 pkgsrc dirs (for the qt5 packages that were built) > hoping to find a MESSAGE containing instructions for setting > XDG_RUNTIME_DIR. (Not sure why I suspected qt5 in that connection.) > > Any and all suggestions will be examined. Did you ever have LyX installed / running before? I used it before, and checked my version, and I didn't get that. In my ~/.lyx directory I found a reference to "xdg-open" (in configure.log), which is part of xdg-utils. It is just a wild guess, but does it help if you install that? > > From my 9.99.81 amd64 system with pkgsrc via git 4/15/21. Did you install from the latest stable branch (2021Q1)? I have; if you installed from the -current version, there may be differences. And if you mixed installs from different point in time of pkgsrc, that may also confuse things. > Thank youse. -Olaf. -- ___ Q: "What's an anagram of Banach-Tarski?" -- Olaf "Rhialto" Seibert \X/ A: "Banach-Tarski Banach-Tarski." -- rhialto at falu dot nl signature.asc Description: PGP signature
Re: XDG_RUNTIME_DIR
On Sat, 24 Apr 2021, Rhialto wrote: Did you ever have LyX installed / running before? No, not on my NetBSD machine at any rate. It is just a wild guess, but does it help if you install that? xdg-utils got installed along with Lyx and the qt5 packages. Did you install from the latest stable branch (2021Q1)? I have; if you installed from the -current version, there may be differences. And if you mixed installs from different point in time of pkgsrc, that may also confuse things. I have a tendency to mix 'n match, or, rather, sometimes -- as you sagely note -- mix 'n mismatch pkgsrc, which requires me to let my NetBSD system from time to time grind through pkg_rolling-replace to straighten out the kind of problems to which you allude. That task was accomplished as recently as last week. Thank you. -- RSB
Re: XDG_RUNTIME_DIR
On Sat, 24 Apr 2021, Bob Bernstein wrote:
I have built from pkgsrc 'lyx' with the qt5 libs. Typing 'lyx' at a command
prompt yields:
$ lyx
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-bob'
That's a harmless notification. If you want to quash it add these lines:
1. /etc/rc.conf:
dbus_enable=YES
2. ~/.profile
export XDG_RUNTIME_DIR=/tmp/runtime-${LOGNAME}
test -d $XDG_RUNTIME_DIR ||
install -d -m 700 -o $(id -u) -g $(id -g) $XDG_RUNTIME_DIR
3. ~/.xinitrc
if [ -x /usr/local/bin/dbus-launch -a -z "${DBUS_SESSION_BUS_ADDRESS}" ];
theneval $(dbus-launch --sh-syntax --exit-with-session)
fi
-RVP
Re: XDG_RUNTIME_DIR
RVP wrote in
:
|On Sat, 24 Apr 2021, RVP wrote:
|
|> 3. ~/.xinitrc
|> if [ -x /usr/local/bin/dbus-launch -a -z "${DBUS_SESSION_BUS_ADDRESS}" ];
|> theneval $(dbus-launch --sh-syntax --exit-with-session)
|> fi
|>
|
|Correction: On NetBSD that would be:
If you run Linux you could also use the pam_xdg module i have
written. For example my /etc/pam.d/common-session is
session optional pam_xdg.so notroot
session requiredpam_unix.so quiet
and the stuff is as attached. It handles the other directories of
the standard as well. (Unfortunately i could not subscribe to the
RedHat hosted PAM list for inclusion upstream, i gave up after
trying several times.)
Could be ported to NetBSD with little effort (unfortunately PAM is
not really portable).
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
/*@ pam_xdg - manage XDG Base Directories (runtime dir life time, environment).
*@ Create /run/user/`id -u` when the first session is opened.
*@ It also creates according XDG_RUNTIME_DIR etc. environment variables in the
*@ user sessions, except when given the "runtime" option, in which case it
*@ only creates XDG_RUNTIME_DIR and not the others.
*@ Place for example in /etc/pam.d/common-session one of the following:
*@ session options pam_xdg.so [runtime] [notroot]
*@ Notes: - according to XDG Base Directory Specification, v0.7.
*@- Linux-only (i think).
*
* Copyright (c) 2021 Steffen Nurpmeso .
* SPDX-License-Identifier: ISC
*
* Permission to use, copy, modify, and/or distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
/* For these a leading \1 is replaced with struct passwd::pw_dir.
* Be aware we use a stack buffer for storage */
#define a_XDG_DATA_HOME_DEF "\1/.local/share"
#define a_XDG_CONFIG_HOME_DEF "\1/.config"
#define a_XDG_DATA_DIRS_DEF "/usr/local/share:/usr/share"
#define a_XDG_CONFIG_DIRS_DEF "/etc/xdg/"
#define a_XDG_CACHE_HOME_DEF "\1/.cache"
/* */
#define a_XDG "pam_xdg"
#define a_RUNTIME_DIR_OUTER "/run" /* This must exist already */
#define a_RUNTIME_DIR_BASE "user" /* We create this as necessary, thus. */
#define a_RUNTIME_DIR_BASE_MODE 0755 /* 0711? */
/* >8 -- 8< */
/*
#define _POSIX_C_SOURCE 200809L
#define _ATFILE_SOURCE
*/
#define _GNU_SOURCE /* Always the same mess */
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
/* _XOPEN_PATH_MAX POSIX 2008/Cor 1-2013 */
#ifndef PATH_MAX
# define PATH_MAX 1024
#endif
static int a_xdg(int isopen, pam_handle_t *pamh, int flags, int argc,
const char **argv);
static int
a_xdg(int isopen, pam_handle_t *pamh, int flags, int argc, const char **argv){
char uidbuf[sizeof "18446744073709551615"],
wbuf[((sizeof("XDG_RUNTIME_DIR=") + sizeof(a_RUNTIME_DIR_OUTER) +
sizeof(a_RUNTIME_DIR_BASE) + sizeof("18446744073709551615")) |
(sizeof("XDG_CONFIG_DIRS=") + PATH_MAX)
) +1];
struct stat st;
struct passwd *pwp;
char const *emsg;
int cwdfd, only_runtime, notroot, res, uidbuflen;
char const *user;
(void)flags;
user = "";
cwdfd = -1;
only_runtime = notroot = 0;
/* Command line */
if(isopen){
for(; argc > 0; ++argv, --argc){
if(!strcmp(argv[0], "runtime"))
only_runtime = 1;
else if(!strcmp(argv[0], "notroot"))
notroot = 1;
else if(!(flags & PAM_SILENT)){
emsg = "command line";
errno = EINVAL;
goto jerr;
}
}
}else
goto jok; /* No longer used, session counting does not work */
/* We need the user we go for */
if((res = pam_get_item(pamh, PAM_USER, (void const**)&user)
) != PAM_SUCCESS){
user = "";
emsg = "cannot query PAM_USER name";
goto jepam;
}
if((pwp = getpwnam(user)) == NULL){
emsg = "host machine does not know about user";
errno = EINVAL;
goto jerr;
}
if(notroot && pwp->pw_uid == 0)
goto jok;
/* I admit all this is overly complicated and expensive */
if((cwdfd = open(a_RUNTIME_DIR_OUTER, (O_PATH | O_DIRECTORY | O_NOFOLLOW))
) == -1){
emsg = "cannot obtain chdir(2) descriptor to "
Re: XDG_RUNTIME_DIR
On Sat, 24 Apr 2021, RVP wrote:
3. ~/.xinitrc
if [ -x /usr/local/bin/dbus-launch -a -z "${DBUS_SESSION_BUS_ADDRESS}" ];
theneval $(dbus-launch --sh-syntax --exit-with-session)
fi
Correction: On NetBSD that would be:
if [ -x /usr/pkg/bin/dbus-launch -a -z "${DBUS_SESSION_BUS_ADDRESS}" ];
In either ~/.xinitrc or ~/.xsession
-RVP
Re: XDG_RUNTIME_DIR
On Sat, 24 Apr 2021, RVP wrote:
Correction: On NetBSD that would be:
if [ -x /usr/pkg/bin/dbus-launch -a -z
"${DBUS_SESSION_BUS_ADDRESS}" ];
In either ~/.xinitrc or ~/.xsession
dbus is launching via standard rc.conf mechanism, with a
'starting dbus' message displayed during boot. Do you suppose I
need the 'dbus-launch' statement in addition to that?
I now have showing in 'env':
XDG_RUNTIME_DIR=/tmp/runtime-bob
and, fwiw:
QTDIR=/usr/pkg/qt5
AT THIS STAGE of the game, at least on launch from the
command-line lyx is not just crashing and dumping core. 'ps'
shows a running lyx process, but it doesn't seem able to reach
the point whereat it would display itself. I have DISPLAY set in
the env:
DISPLAY=:0
But the game's afoot!
Thank you.
--
What's going on with Bob?
https://zeus.jtan.com/~wingnut
Re: NetBSD on bhyve on TrueNAS Core
Excerpt from Chavdar Ivanov: > userconf disable xhci* > I mean, for the installation you will have to quickly go to the vnc > window, interrupt the boot to command line and enter 'boot -c'; then > disable xhci > quit > Then go through the installation as normal, GPT etc. After it > finishes, go to the command line, mount your root dk (should be dk1) > and add the above line to /boot.cfg . > Chavdar I thought USB 3 with xhci was working on NetBSD. Have some bugs recently arisen? Bad enough that I have to disable athn; athn causes the boot to hang: maybe fixed in more recent versions? Other question is whether an ext2fs would successfully copy to a partition on a USB stick or hard drive. But it was "disable xhci" in your message that raised the alarm in me. Tom
Re: XDG_RUNTIME_DIR
Qt-apps can be rather picky regarding XDG, I have the following in .xinitrc
to avoid such issues,
mkdir /tmp/${USER}-runtime && chmod -R 0700 /tmp/${USER}-runtime
export XDG_RUNTIME_DIR=/tmp/${USER}-runtime
Den sön 25 apr. 2021 06:19Bob Bernstein skrev:
> On Sat, 24 Apr 2021, RVP wrote:
>
> > Correction: On NetBSD that would be:
>
> > if [ -x /usr/pkg/bin/dbus-launch -a -z
> > "${DBUS_SESSION_BUS_ADDRESS}" ];
> >
> > In either ~/.xinitrc or ~/.xsession
>
> dbus is launching via standard rc.conf mechanism, with a
> 'starting dbus' message displayed during boot. Do you suppose I
> need the 'dbus-launch' statement in addition to that?
>
> I now have showing in 'env':
>
> XDG_RUNTIME_DIR=/tmp/runtime-bob
>
> and, fwiw:
>
> QTDIR=/usr/pkg/qt5
>
> AT THIS STAGE of the game, at least on launch from the
> command-line lyx is not just crashing and dumping core. 'ps'
> shows a running lyx process, but it doesn't seem able to reach
> the point whereat it would display itself. I have DISPLAY set in
> the env:
>
> DISPLAY=:0
>
> But the game's afoot!
>
> Thank you.
>
> --
> What's going on with Bob?
> https://zeus.jtan.com/~wingnut
>
