Re: FreeBSD 13 on bhyve freezes under load

[Michael Gmelin]

On Mon, 3 May 2021 13:14:53 +0200
Christian Stærk  wrote:

> Hi Michael,
> 
> I am not using ZFS and have plenty of free diskspace:
> Filesystem Size    Used   Avail Capacity Mounted on
> /dev/ada0p2    190G 92G 83G    52%    /
> 
> top looks like this on a not-hung system:
> 
> last pid: 38292;  load averages:  0.91, 0.87,  0.88    up 0+01:13:16  
> 13:12:12
> 32 processes:  2 running, 29 sleeping, 1 zombie
> CPU:  5.5% user,  0.0% nice,  7.8% system,  7.0% interrupt, 79.7% idle
> Mem: 59M Active, 3549M Inact, 1090M Wired, 610M Buf, 2250M Free
> Swap: 24G Total, 24G Free
> 
>    PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU
> COMMAND 38079 root  1  22    0    99M    43M RUN  0:03
> 14.02% pkg-static

Based on hanging pkg-static process I assume that top output is from
within the VM though, isn't it? Or is this on the host?

Best
Michael

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: FreeBSD 13 on bhyve freezes under load

[Michael Gmelin]

On Mon, 3 May 2021 11:14:45 +0200
Christian Stærk  wrote:

> Hello,
> 
> I recently upgraded one of my bhyve guests to FreeBSD 13.0-RELEASE.
> The host is running 12.2-RELEASE-p4.
> 
> The guest in question is used for building packages with poudriere
> and is always the first one to get upgraded to new releases.
> 
> The guest freezes consistently every time poudriere tries to build
> the thunderbird package. Although it is not a complete freeze. It
> responds to ping and programs like top and systat continue updating.
> However, all I/O operations seem to hang.
> 
> systat looks like this when it happens: 
> https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png 
> <https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png>
> 
> What is the best way to debug this?

Sharing top output might help (memory/swap and ZFS arc usage). Also,
make sure you're not low on diskspace (these are the two most common
issues I encountered with hanging bhyve VMs - out of non-virtual memory
and out of diskspace of the host machine filesystem).

Some basic info on the host and your VM configuration might also help
others to reproduce the problem.

-m

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve with wlan0

[Michael Gmelin]

> On 28. Nov 2020, at 19:30, Rodney W. Grimes  
> wrote:
> 
> 
>> 
>> As far as I remember it works just fine other than the standard caveats 
>> about bridging over wireless interfaces which involve some odd 
>> inconsistencies that could crop up from time to time especially with like 
>> DHCP, but a lot of that depends on your network config as a whole. 
>> The approach I would consider taking for a wireless host is if the guest 
>> doesn't need direct access to the network just do a NAT network or a routed 
>> network and route through the wlan interface instead of trying to do 
>> bridging with it directly.
> 
> One very problematic area for wireless and bhyve is that each guests have a 
> different MAC, wireless does NOT like that, as the AP usually associates with 
> one and only one MAC address.
> 

Like Jason wrote, it‘s best to use NAT on wireless (especially if you’re moving 
between networks frequently). Using something like vm-bhyve, it’s very easy to 
set up.

-m


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Bhyve help

[Michael Gmelin]

You might want to read https://blog.grem.de/ayvn

It also covers using vm-bhyve to manage vms and will explain some things about 
configuring the network.

> On 6. May 2020, at 01:55, Brandon helsley  wrote:
> 
> Well how do i add a tap to the bridge the VM is on as well as my physical nic
> 
> Sent from Outlook Mobile
> 
> 
> From: Aryeh Friedman 
> Sent: Tuesday, May 5, 2020 5:02:50 PM
> To: Brandon helsley 
> Cc: freebsd-virtualization@freebsd.org ; 
> Robert Crowston 
> Subject: Re: Bhyve help
> 
> When setting up a VM you also need to add the tap it is on to the bridge as 
> well your physical NIC
> 
> On Tue, May 5, 2020 at 6:28 PM Brandon helsley 
> mailto:brandon.hels...@hotmail.com>> wrote:
> Bhyve gets me to Network installation: "some installation files were not 
> found on the boot volume. The next few screens will allow you to configure 
> networking so that they can be downloaded from the internet." This happens 
> after system components.
> 
> Sent from Outlook Mobile
> 
> From: Aryeh Friedman 
> mailto:aryeh.fried...@gmail.com>>
> Sent: Tuesday, May 5, 2020 4:08:10 AM
> To: Brandon helsley 
> mailto:brandon.hels...@hotmail.com>>
> Cc: 
> freebsd-virtualization@freebsd.org 
> mailto:freebsd-virtualization@freebsd.org>>;
>  Robert Crowston mailto:crows...@protonmail.com>>
> Subject: Re: Bhyve help
> 
> 
> 
> On Tue, May 5, 2020 at 6:00 AM Brandon helsley 
> mailto:brandon.hels...@hotmail.com>> wrote:
> The handbook only says vmm doesn't it. I got bhyve to start without virtio 
> kernel module as well as the if_tap but with vmm modules.
> 
> The handbook section on virtualization (FreeBSD host) is at best awful and 
> misleading your best guide is the bhyve man page (especially the examples 
> there) here is the modules I load for the same set up I recommended you try 
> as a advanced beginners lesson in virtualization and networking.   And an 
> example script for using them:
> 
> /boot/loader.conf:
> 
> kern.geom.label.disk_ident.enable="0"
> kern.geom.label.gptid.enable="0"
> opensolaris_load="YES"
> zfs_load="YES"
> ipfw_load="YES"
> ipdivert_load="YES"
> linux_load="YES"
> 
> In the real /boot/loader.conf the following are commented out because 
> PetiteCloud takes care of them when it loads but unless you want to get into 
> more advanced stuff (like how to install non-standard ports) you should 
> uncomment them.
> 
> vmm_load="YES"
> if_tap_load="YES"
> nmdm_load="YES"
> 
> The script to load the file server VM:
> 
> #!/bin/sh
> #
> # Generated by PetiteCloud 0.2.6
> #
> 
> ifconfig tap12 destroy
> ifconfig tap12 create
> ifconfig tap12 up
> sleep 5
> ifconfig bridge0 addm tap12 up
> bhyvectl --destroy --vm=b829e56e9b4289a1
> /usr/sbin/bhyve -c cores=4 -m 4096 -AI -H -P -w -g 0 -s 0:0,hostbridge -s 
> 1,virtio-net,tap12 -s 2,ahci-hd,/vms/import/win10  -s 
> 29,fbuf,tcp=0.0.0.0:5912,w=1920,h=1080 -s 30,xhci,tablet 
>  -s 31,lpc -l com1,/dev/nmdm12B -l 
> bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd 
> b829e56e9b4289a1&>/dev/null
> echo $!>/var/run/petitecloud/b829e56e9b4289a1
> sleep 10
> 
> I'm at this screen:please choose the appropriate terminal type for your 
> system.common console types are: ansi, vt100, xterm, cons25w.
> 
> Console type [vt100]:
> 
> Sent from Outlook Mobile
> 
> From: Aryeh Friedman 
> mailto:aryeh.fried...@gmail.com>>
> Sent: Tuesday, May 5, 2020 3:52:26 AM
> To: Brandon helsley 
> mailto:brandon.hels...@hotmail.com>>
> Cc: 
> freebsd-virtualization@freebsd.org 
> mailto:freebsd-virtualization@freebsd.org>>;
>  Robert Crowston mailto:crows...@protonmail.com>>
> Subject: Re: Bhyve help
> 
> 
> 
> On Tue, May 5, 2020 at 5:21 AM Brandon helsley 
> mailto:brandon.hels...@hotmail.com>> wrote:
> I have re0 but the ip address is under wlan0 with my CenturyLink
> 
> In order to use vtnet and bridging (both required for bhyve networking) you 
> need to load the load the virtio kernel module as well as the if_tap and vmm 
> modules (the first one provides the host side of the virtual NIC and the 
> second loads the hypervisor components)
> 
> Sent from Outlook Mobile
> 
> 
> From: Robert Crowston 
> mailto:crows...@protonmail.com>>
> Sent: Tuesday, May 5, 2020 3:14:43 AM
> To: Brandon helsley 
> mailto:brandon.hels...@hotmail.com>>; 
> freebsd-virtualization@freebsd.org 
> mailto:freebsd-virtualization@freebsd.org>>
> Subject: Re: Bhyve help
> 
> What are your network interfaces called on your machine? You probably have to 
> change igb0 in the example to em0 (an Intel card) or re0 (a Realtek card). 
> Type “ifconfig” to list all the interfaces, and look for ones with an ip 
> address to figure it 

Re: syncing bhyve instances

[Michael Gmelin]

> On 22. Jan 2018, at 20:47, tech-lists  wrote:
> 
>> On 22/01/2018 16:38, Paul Vixie wrote:
>> for live sync you'll have to run software inside the guest that knows
>> how to properly freeze state. for example if there's a live database of
>> any kind you'll want it to be in its quiet state before you sync from
>> it. in those situations, i do use rsync.
> 
> Yeah, thought it might be this. Sorry I wasn't more clear initially
> about the use case.
> 
> Basically, the production server is in a datacentre and the reserve
> server is on a very fast vdsl service. The reason for the reserve server
> is, if the production server fails then I swap DNS to point at the
> reserved server and the guests on it without interruption of service.
> All guests are running databases (mysql) though they aren't especially
> busy. So I guess the best bet would be mysql replication for the
> databases and rsync for everything except mysql?
> 
> thanks everyone who took the time to answer

For the database, mysql replication is the way to go (otherwise you would need 
to stop/lock the server every time you do a snapshot and you'll get 
near-realtime replication which will always be more current than your 
snapshots). For  the system I'd suggest to create automation for the setup, so 
you can apply changes to both systems without a need to sync/clone/copy from 
one to the other. For anything else left (data files on disk, non-system, 
non-package, non-config) use regular rsync (or zfs send if your setup permits - 
I'd stick with rsync).

Yours,
Michael

p.s. Make sure to include monitoring (especially replication latency and data 
integrity)


> 
> -- 
> J.
> ___
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to 
> "freebsd-virtualization-unsubscr...@freebsd.org"

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: iocage no longer supported?

[Michael Gmelin]

On Tue, 12 Apr 2016 11:44:14 -0400
Larry Baird <l...@gta.com> wrote:

> I was beginning to look into a replacement for ezjail, mainly because
> ezjail doesn't use jail.conf. After vistiting lots of sites and doing
> some reading, I had about decided upon iocage. I am now convered
> about this decision. When I visit https://github.com/iocage/iocage is
> now says "**No longer supported. iocage is being rewritten in a
> differnt language." Does anybody have any details?
> 

https://github.com/iocage/iocage/commit/3f394561a3dde55cd3ac7911be313c5df5865183


-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: High load average inside idle FreeBSD guest running on bhyve

[Michael Gmelin]

On Thu, 02 Oct 2014 11:49:19 +0100
Karl Pielorz kpielorz_...@tdx.co.uk wrote:

 
 
 --On 02 October 2014 10:06 +0200 Christian Laursen
 x...@borderworlds.dk wrote:
 
  Hello
 
  I have a few bhyve instances running that all seem to have a higher
  load average than what would seem normal for a system that is idle.
 
  [snip]
 
  last pid:  6109;  load averages:  0.41,  0.38,  0.39up
  1+14:27:48 09:56:25
 
 Hi,
 
 I think this has been covered on other lists - and is apparently
 normal (i.e. cosmetic).
 
 I've installed a number of FBSD10 boxes recently, and they all seem
 to idle at around that (even when doing absolutely nothing) - I did
 post about it and was told it was just cosmetic... It certainly seems
 to be cosmetic.
 
 If it's the same issue - I'd guess it's not bhyve related...
 
 
 last pid:  1294;  load averages:  0.42,  0.32,  0.27 
 up 1+20:44:55  11:47:16
 18 processes:  1 running, 17 sleeping
 CPU:  0.0% user,  0.0% nice,  0.0% system,  0.1% interrupt, 99.9% idle
 
 
 That's from a bare metal amd64 10.0-RELEASE-p9 box with no services /
 users on it.
 
 Thinking about it - we had the same 'symptom' on a Xen based FBSD 10
 box as well...

For your bare metal box, could you send the output of vmstat -i ? I had
the same issue on older boxes and it went away after setting

hint.hpet.0.legacy_route=1
hint.attimer.0.clock=0
hint.atrtc.0.clock=0

in /boot/loader.conf (see hpet(4)).

 
 -Karl
 
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to
 freebsd-virtualization-unsubscr...@freebsd.org



-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Current problem reports assigned to freebsd-virtualization@FreeBSD.org

[Michael Gmelin]

On Mon, 3 Feb 2014 19:32:21 +0100
Mark Martinec mark.martinec+free...@ijs.si wrote:

 Aryeh Friedman wrote:
  By now we have made enough progress in runninng FB as a VM on FB ...
 
 Please avoid shortening FreeBSD as FB.  It is confusing for a casual
 reader as FB is commonly associated with Facebook, and it is
 unfriendly to search engines.  If FreeBSD is too much to type, use
 FBSD.
 

I restrained myself from asking the same, but I agree 100% with Mark,
FB == Facebook, FreeBSD == FreeBSD.

Also, please don't top post (I know it's hard these days, especially
with gmail, but it's FreeBSD etiquette for a reason, see
http://www.freebsd.org/doc/en/articles/mailing-list-faq/etiquette.html
).

Thank you.

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Public Service Announcement: The name is bhyve

[Michael Gmelin]

On Fri, 24 Jan 2014 02:46:06 -0800
Michael Dexter edi...@callfortesting.org wrote:

 On 1/24/14 2:41 AM, Michael Gmelin wrote:
  Haters gonna hate?
 
  Lack of attention to detail != hate.
 
 I guess you didn't see the phoronix.com article on bhyve a few months
 ago. :) They pulled the worst of it.
 
 The whole BSD is dying or is poo in their case FUD is still very
 active in various circles and you never know.
 

Oh I've seen it and I usually ignore Phoronix and its comments for
exactly this reason, but in this case serverwatch reported on the
topic in a factual/neutral way and just got the details wrong. It
doesn't seem like a high margin website to me, so they probably
didn't spend much time on it ;)

Recent bhyve progress is quite impressive by the way.

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve: reboot stops vm

[Michael Gmelin]

On Tue, 21 Jan 2014 03:31:40 +0400
Sergey Matveychuk s...@freebsd.org wrote:

 Hi.
 
 Is it expected behaviour?
 
 sem@vm0[17]% sudo reboot
 Jan 21 07:27:51 vm0 reboot: rebooted by sem
 Jan 21 07:27:51 vm0 syslogd: exiting on signal 15
 Waiting (max 60 seconds) for system process `vnlru' to stop...done
 Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
 Waiting (max 60 seconds) for system process `syncer' to stop...
 Syncing disks, vnodes remaining...2 0 done
 All buffers synced.
 Uptime: 1m17s
 Rebooting...
 cpu_reset: Stopping other CPUs
 root@bhyve[36]#

Yes

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Real Device with BHyve

[Michael Gmelin]

On Fri, 03 Jan 2014 09:43:24 -0800
Peter Grehan gre...@freebsd.org wrote:

 Hi Andrea,
  Well, the machine starts ok but when the child FreeBSD starts
  installation something strange happens. When I get to the
  partitioning screen I can see the device avaiable as /dev/vtdb0
  with the correct size and such. I choose autopartitioning, the
  installer writes the partition table but when it start to
  write /dev/vtdb0p2 a very cryptic error appears about being unable
  to write - sorry, did not write it down.
 
  The installer then stops.
 
  If I do a fdisk /dev/vtdb0 in the VM I can see the GPT partition
  being there. If I do a fdisk /dev/da2 on the host machine, I can
  see the GPT partition as well, but the VM just doesn't want to
  write on it.
 
  I even tried changing kern.geom.debugflags=16 as I thought the host
  machine could be locking somehow the drive, but that didn't seem to
  make any difference. I know it was a lame check but I was out of
  ideas.
 
  So I just wanted to understand if such a scenario is supposed to be
  supported
 
   It is. Been a while since I've done this but will try a repro.
 Other folk have supported success using zvols so I'd assumed it was
 working.

If it helps, I had the same problem using zvol as well (booting the
installer and install on the raw device) on 10-CURRENT from mid 2013.

I didn't try with a more recent release. I ended up installing FreeBSD
from the outside (create partitions and untar base.txz, kernel.txz etc.
- afaik the same thing the Dexter's vm0 script does now) and could then
boot into it just fine.

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Upcoming FreeBSD 10.x + bhyve ...

[Michael Gmelin]

On Mon, 2 Dec 2013 16:11:53 -0800
Marc Fournier scra...@hub.org wrote:

 
 On Dec 2, 2013, at 16:08 , Peter Grehan gre...@freebsd.org wrote:
 
  Hi Marc,
  
  I just read through http://bhyve.org/bhyve-manual.txt, and one
  thing that doesn’t seem to be supported (or, I’ve missed it) is
  HeadLess support … I get the impression that using this on a
  remote server isn’t currently possible, or am I missing something
  in the docs?
  
  It is. The easiest is using the recent change (r258668) which
  allows bhyveload and bhyve to direct output to a tty instead of
  just stdio.
 
 ‘k, that would work … so *if* I’m reading the docs right, the intiial
 build of the file system / install of guest OS, happens on an
 interactive remote ssh session, then I’d shutdown and restart it
 redirecting output … ?
 
 I’m plannin gon building a 10.x system over Christmas to start
 playing with this, just trying to do as much research as possible
 before … 
 
 thx
 
 

Even though it's not the official solution, my little bhyve-rc script
works reliably (requires sysutils/tmux). You can basically tmux attach
into your session anytime. It also provides bhyve_guest for a clean
shutdown of FreeBSD guests (a workaround until bhyve can inject the
necessary acpi instructions to trigger a shutdown). I've been using it
for half a year now and it gets the job done. You can find it here
http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/bhyve_rc/

Please share your experience with the changes in r258668,
quite interested if this works ok for you.

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: FreeBSD 9.1-Rel: Jail-rc startup of vnet jails with new configuration syntax

[Michael Gmelin]

On Thu, 07 Nov 2013 13:26:10 +0100
Kristen Nielsen k...@krn.dk wrote:

 Hi freebsd-virtualization list
 
 I am trying to figure out how to start configured jails within
 FreeBSD 9.1Release at boot time.
 I am using new C-style jail configuration syntax in /etc/jail.conf.
 all jail starts successfully when i do a jail -c from the commandline.
 
 I am trying to configure jails to start at boot time using the
 features in the /etc/rc.conf file.
 
 in /etc/rc.conf at base host i have
 jail_enable=YES
 jail_list= jail1 jail2 jail3 
 
 and no more jail related lines, evereything else is placed into the
 jail configuration (C-style) file.
 
 running service jail start:
 Configuring jails:.
 Starting jails:/etc/rc.d/jail: ERROR: jail: No hostname has been
 defined for jail1
 
 Looking into the /etc/rc.d/jail script it seems it expects that the 
 /etc/rc.conf contains old style jail configuration statement like
 
 jail_jname_hostname=jail1
 etc.
 
 I have looked at the patch at: 
 http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet
 but it also uses the old style (rc.config) jail configuration format, 
 but with extended number of parameters. The patch is dated before the 
 9-1 release where the C-style jail configuration syntax was taken
 into use.
 
 is there a patch for /etc/rc.d/jail script that make it handles the
 new style jail configuration file format (the C-style format). or am
 I doing this the wrong way.
 
 If you can point me into the right direction it will be very helpfull.
 
 Whishlist
 It would be great with something like a /etc/jail.d/jailname setup 
 where each jail configuration file is contained in a separate file,
 the format is the same as the /etc/jail.conf, the latter fil just
 holds configurations for all jails. Both files with the C-style
 syntax.

Have you considered using sysutils/ezjail (the de-facto standard for
managing jails under FreeBSD right now) or a similar port
(sysutils/jailrc, sysutils/qjail etc.)?

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: FreeBSD 9.1-Rel: Jail-rc startup of vnet jails with new configuration syntax

[Michael Gmelin]

On Thu, 7 Nov 2013 13:44:32 +0100
Michael Gmelin free...@grem.de wrote:

 
 
 On Thu, 07 Nov 2013 13:26:10 +0100
 Kristen Nielsen k...@krn.dk wrote:
 
  Hi freebsd-virtualization list
  
  I am trying to figure out how to start configured jails within
  FreeBSD 9.1Release at boot time.
  I am using new C-style jail configuration syntax in /etc/jail.conf.
  all jail starts successfully when i do a jail -c from the
  commandline.
  
  I am trying to configure jails to start at boot time using the
  features in the /etc/rc.conf file.
  
  in /etc/rc.conf at base host i have
  jail_enable=YES
  jail_list= jail1 jail2 jail3 
  
  and no more jail related lines, evereything else is placed into the
  jail configuration (C-style) file.
  
  running service jail start:
  Configuring jails:.
  Starting jails:/etc/rc.d/jail: ERROR: jail: No hostname has been
  defined for jail1
  
  Looking into the /etc/rc.d/jail script it seems it expects that the 
  /etc/rc.conf contains old style jail configuration statement like
  
  jail_jname_hostname=jail1
  etc.
  
  I have looked at the patch at: 
  http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet
  but it also uses the old style (rc.config) jail configuration
  format, but with extended number of parameters. The patch is dated
  before the 9-1 release where the C-style jail configuration syntax
  was taken into use.
  
  is there a patch for /etc/rc.d/jail script that make it handles the
  new style jail configuration file format (the C-style format). or am
  I doing this the wrong way.
  
  If you can point me into the right direction it will be very
  helpfull.
  
  Whishlist
  It would be great with something like a /etc/jail.d/jailname
  setup where each jail configuration file is contained in a separate
  file, the format is the same as the /etc/jail.conf, the latter fil
  just holds configurations for all jails. Both files with the C-style
  syntax.
 
 Have you considered using sysutils/ezjail (the de-facto standard for
 managing jails under FreeBSD right now) or a similar port
 (sysutils/jailrc, sysutils/qjail etc.)?
 

I missed the vnet requirement in my first reply. There has been a vnet
patch for ezjail floating around a while ago, but I think it was never
adopted to ezjail. Sorry for the noise.

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: rc-style script for bhyve VM provisioning and management

[Michael Gmelin]

On Sun, 03 Nov 2013 00:18:09 -0700
Michael Dexter edi...@callfortesting.org wrote:

 
 Hello all from the FreeBSD 20th Anniversary Party in San Francisco,
 
 I have published a bhyve virtual machine provisioning and management
 framework to simplify the building and deploying of bhyve VMs during
 the run-up to the FreeBSD 10.0 release. As you may know, bhyve is
 rapidly finalizing for release in FreeBSD 10.0 and most features are
 finally becoming stable enough to document in a canonical manner.
 These scripts demonstrate the many opportunities for customization of
 bhyve deployments and will embrace features like OpenBSD and Linux
 support as soon as key components like the userland grub* arrive
 (currently in final testing). It will also demonstrate the use of pf
 to allow bhyve networking on wireless interfaces.
 
 It can be downloaded from:
 
 http://bhyve.org/bhyve-script.tar
 
 It works like a rc script but does not comply with the FreeBSD way. I
 am working with several developers to determine to what degree it
 should be compatible with existing frameworks like ezjail.
 
 ./vm0 will give the usage:
 
 usage (start|stop|load|boot|destroy|restart|list|debug
 mount|umount|jail|format|fetch|install|provision|wipe)
 
 Some routines like provision include steps like fetch (distribution
 sets), format (disk images or volumes) and install them.
 
 The jail routine demonstrates how a virtual machine can be launched
 using the jail(8) command, because we can, given that bhyve FreeBSD
 VMs are simply instances of FreeBSD with the ABI limitations of
 different releases. bhyve will support any vision of FreeBSD with
 VirtIO, specifically FreeBSD 8.4 and 8-STABLE, 9.2 and 9-STABLE, and
 any recent version of 10 or 11.
 
 Detailed instructions are included in the vm0 script and it is
 designed to simply be copied to vm1, vm2 etc. with easily-configured
 global variables.
 
 While tmux(1) is optional and supports both captive and detached
 execution, a script is included to list and attach to running
 VMs/sessions. A script is also provided for the easy duplication of
 VMs and I have tested this with up to 15 X 16 vCPU instances though
 far more should work without issue.
 
 Please throw everything you can at bhyve to expose any remaining flaws
 prior to the FreeBSD 10.0 release.
 
 All the best,
 
 Michael Dexter

Sounds similar to how I ran my VMs for a while (well, I ln -s from the
original VM script instead of copying, so it's easier to update all at
once). I finally changed the script to use profiles a few days ago,
since it seemed much cleaner in general and better for automatic control
and also complies to the FreeBSD way. I submitted a PR for
sysutils/bhyve-rc literally hours before you released vm0
(http://www.freebsd.org/cgi/query-pr.cgi?pr=183604), great timing ;)
It's only good for one use case, but does that very well.

I like the idea of the script using the general structure of ezjail. I
would also suggest to split this it two scripts, one rc script, that
only provides non-destructive operations like
(start|stop|restart|rcvar) and one called vm-admin which provides the
provisions functions on top, e.g.

service vm start vm0 # ok
vm-admin start vm0 # ok
service vm format vm0 ... # not ok
vm-admin format vm0 ... # ok

-- 
Michael Gmelin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

rc.d script for running bhyve in tmux

[Michael Gmelin]

Hi,

I hacked together a small rc.d script for starting bhyve(4) in tmux,
maybe it's useful to others too. Place it in /usr/local/etc/rc.d and
install sysutils/tmux. Might need some love to fit your needs.

Minimal configuration example:
pkg install tmux

echo net.link.tap.up_on_open=1  /etc/sysctl.conf

cat  /boot/loader.conf  EOF
if_tap_load=YES
if_bridge_load=YES
vmm_load=YES
EOF

kldload tap; kldload bridge; kldload vmm

cat  /etc/rc.conf  EOF
cloned_interfaces=tap0 bridge0
bhyve_enable=YES
bhyve_diskdev=/dev/zvol/tank/bhyve/virt
EOF

ifconfig tap0 create
ifconfig bridge0 create

service bhyve start
tmux list-sessions
tmux attach -t bhyve
service bhyve status
service bhyve stop

Multi profile configuration example:
cat  /etc/rc.conf  EOF
cloned_interfaces=tap0 tap1 bridge0
bhyve_enable=YES
bhyve_profiles=virt1 virt2
bhyve_virt1_diskdev=/dev/zvol/tank/bhyve/virt1

bhyve_virt2_tapdev=tap1
bhyve_virt2_diskdev=/dev/zvol/tank/bhyve/virt2
bhyve_virt2_memsize=8192
bhyve_virt2_ncpu=4
EOF

ifconfig tap0 create
ifconfig tap1 create
ifconfig bridge0 create

service bhyve start # start all
service bhyve start virt2 # start individual
tmux attach -t bhyve_virt1
tmux attach -t bhyve_virt1
service bhyve stop virt2 # stop individual
service bhyve stop # stop all

(by default ctrl-b d detaches from tmux).

Cheers,
Michael

-- 
Michael Gmelin
#!/bin/sh
#
# $FreeBSD: mgmelin
#

# PROVIDE: bhyve
# REQUIRE: LOGIN
# KEYWORD: nojail
#

#
# Add the following lines to /etc/rc.conf to enable bhyve:
# bhyve_enable (bool):  Set to NO by default.
#   Set it to YES to enable bhyve
# bhyve_profiles (str): Set to  by default.
#   Define your profiles here.
# bhyve_tapdev (str):   Set to tap0 by default.
#   Set to the tap(4) device to use.
# bhyve_diskdev (str):  Must be set, no default.
#   Set to the disk device to use.
# bhyve_ncpu (int): Set to 1 by default.
#   Set to the number of CPUs for the VM.
# bhyve_memsize (int):  Set to 512 by default.
#   Set to the number of MB of memory for the VM.

. /etc/rc.subr

name=bhyve
rcvar=bhyve_enable

start_precmd=bhyve_prestart
status_cmd=bhyve_status
poll_cmd=bhyve_poll
stop_cmd=bhyve_stop
_session=$name
command=/usr/local/bin/tmux
procname=-sh

[ -z $bhyve_enable ]   bhyve_enable=NO
[ -z $bhyve_tapdev ]   bhyve_tapdev=tap0
[ -z $bhyve_diskdev ]  bhyve_diskdev=none
[ -z $bhyve_ncpu ] bhyve_ncpu=1
[ -z $bhyve_memsize ]  bhyve_memsize=512

load_rc_config $name

if [ -n $2 ]; then
profile=$2
_session=${_session}_${profile}
if [ x${bhyve_profiles} != x ]; then
eval bhyve_enable=\${${_session}_enable:-${bhyve_enable}}
eval bhyve_tapdev=\${${_session}_tapdev:-${bhyve_tapdev}}
eval bhyve_diskdev=\${${_session}_diskdev:-${bhyve_diskdev}}
eval bhyve_ncpu=\${${_session}_ncpu:-${bhyve_ncpu}}
eval bhyve_memsize=\${${_session}_memsize:-${bhyve_memsize}}
else
echo $0: extra argument ignored
fi
else
if [ x${bhyve_profiles} != x -a x$1 != x ]; then
for profile in ${bhyve_profiles}; do
eval _enable=\${bhyve_${profile}_enable}
case x${_enable:-${bhyve_enable}} in
x|x[Nn][Oo]|x[Nn][Oo][Nn][Ee])
continue
;;
x[Yy][Ee][Ss])
;;
*)
if test -z $_enable; then
_var=bhyve_enable
else
_var=bhyve_${profile}_enable
fi
echo Bad value \
'${_enable:-${bhyve_enable}}' \
for ${_var}.  \
Profile ${profile} skipped.
continue
;;
esac
echo === bhyve profile: ${profile}
/usr/local/etc/rc.d/bhyve $1 ${profile}
retcode=$?
if [ 0${retcode} -ne 0 ]; then
failed=${profile} (${retcode}) ${failed:-}
else
success=${profile} ${success:-}
fi
done
exit 0
fi
profile=$name
fi

pidfile=/var/run/${_session}.pid


bhyve_prestart()
{
case ${bhyve_diskdev} in
[Nn][Oo][Nn][Ee] | '')
echo No ${_session}_diskdev set. Quitting. 12
return 1;
;;
esac
if [ ! -c ${bhyve_diskdev} -a ! -f ${bhyve_diskdev} ]; then
echo