Re: Bhyve tests and findings
> > uefi="yes" > graphics="yes" Dear Colleagues, Can I enjoy and test all those nice "UEFI-GOP" features on a 10.3-RELEASE system, or do I have to install CURRENT for that? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
A couple of newbie questions
Colleagues, I like bhyve very much, and have sucessfully run FreeBSD and Ubuntu 16.04 server in FreeBSD 10.3 bhyve, with vm-bhyve as a shell. Now I am trying to boot Windows 7 but have not succeeded so far. However there are things I don't quite understand. A couple of questions, if you allow. 1. Why is it that for some guest systems, there are two stages: first bhyveload or grub2-bhyve and then bhyve itself. And for UEFI systems there is only one stage. Does it mean that eventually bhyveload and grub2-bhyve will become totally obsolete and the one-stage VM startup procedure will become the universal method? 2. All this fbuf/VNC stuff looks cool, but I don't quite understand. You can see the guest OS's console in VNC, like the Windows desktop, or only the UEFI shell, and then you have to access the guest OS via RDP/ssh etc ? TIA for explanations. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Lars Engels wrote: > > > > > > uefi="yes" > > > graphics="yes" > > > > Dear Colleagues, > > > > Can I enjoy and test all those nice "UEFI-GOP" features on a > > 10.3-RELEASE system, or do I have to install CURRENT for that? > > AFAIK you can checkout usr.sbin/bhyve from the 11 branch and compile it > on 10.3. I'm not sure how to compile something from a different branch without compiling the whole world. make what? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Lars Engels wrote: > On Fri, Aug 05, 2016 at 01:59:45PM +0700, Victor Sudakov wrote: > > Lars Engels wrote: > > > > > > > > > > uefi="yes" > > > > > graphics="yes" > > > > > > > > Dear Colleagues, > > > > > > > > Can I enjoy and test all those nice "UEFI-GOP" features on a > > > > 10.3-RELEASE system, or do I have to install CURRENT for that? > > > > > > AFAIK you can checkout usr.sbin/bhyve from the 11 branch and compile it > > > on 10.3. > > > > I'm not sure how to compile something from a different branch without > > compiling the whole world. make what? > > For bhyve it's easy. Check out the sources, go to usr.sbin/bhyve and run > > make depend; make obj; make; make install. At least lib/libvmmapi should also be built and installed. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Converting from VMDK to RAW
Colleagues, I would like to convert some FreeBSD VMs from VMware to bhyve. Which is the best way to convert virtual disks from the VMDK to the RAW format (which I think is the format bhyve uses)? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Converting from VMDK to RAW
Thomas Laus wrote: > > > > I would like to convert some FreeBSD VMs from VMware to bhyve. Which > > is the best way to convert virtual disks from the VMDK to the RAW > > format (which I think is the format bhyve uses)? > > > I have used /devel/vmdktool to do the job in the past. It has always > worked for me. Now it's in sysutils/vmdktool. However, it's not all that simple. I have downloaded the VM disk from VMware ESXi and the image consists of two files: myhost-flat.vmdk and myhost.vmdk. The latter is a small text file containing meta-information about the disk. The former looks like a RAW disk, at least I can mdconfig it and mount the partitions. $ file "myhost-flat.vmdk" myhost-flat.vmdk: DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 20971519 sectors the problem is that sysutils/vmdktool does not recognize either of them: $ vmdktool -i myhost.vmdk myhost.vmdk: File too small (must be at least 1024 bytes) $ vmdktool -i myhost-flat.vmdk myhost-flat.vmdk: Bad VMDK magic (got 8ec031fc, want 564d444b) $ Probably I don't need any converters at all because the *-flat.vmdk is already a RAW image. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Converting from VMDK to RAW
Victor Sudakov wrote: > > I have downloaded the VM disk from VMware ESXi and the image consists > of two files: myhost-flat.vmdk and myhost.vmdk. The latter is a small > text file containing meta-information about the disk. The former looks > like a RAW disk, at least I can mdconfig it and mount the > partitions. > > $ file "myhost-flat.vmdk" > myhost-flat.vmdk: DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS > (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 20971519 sectors > > the problem is that sysutils/vmdktool does not recognize either of > them: > > $ vmdktool -i myhost.vmdk > myhost.vmdk: File too small (must be at least 1024 bytes) > $ vmdktool -i myhost-flat.vmdk > myhost-flat.vmdk: Bad VMDK magic (got 8ec031fc, want 564d444b) > $ > > Probably I don't need any converters at all because the *-flat.vmdk is > already a RAW image. I've even been able to reduce the on-disk size of the image from 10G to 3,4G with "dd conv=sparse" without any ill effects. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Converting from VMDK to RAW
Thomas Laus wrote: [dd] > > > > > > Probably I don't need any converters at all because the *-flat.vmdk is > > > already a RAW image. > > > > I've even been able to reduce the on-disk size of the image from 10G > > to 3,4G with "dd conv=sparse" without any ill effects. > > > You have done a lot more with VMWare than I have. I just experimented with a > few tutorials and then went on to using Xen. I had known of the vmdktool > utilities and used them to move VM's between home and office, but never was a > serious VMWare customer (too expensive!). Oh, ESXi is free and very nice. My real incentive to switch from VMware to bhyve is that VMware requres a Microsoft Windows workstation for management. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Lars Engels wrote: > > > > > > > > > > uefi="yes" > > > > > graphics="yes" > > > > > > > > Dear Colleagues, > > > > > > > > Can I enjoy and test all those nice "UEFI-GOP" features on a > > > > 10.3-RELEASE system, or do I have to install CURRENT for that? > > > > > > AFAIK you can checkout usr.sbin/bhyve from the 11 branch and compile it > > > on 10.3. > > > > I'm not sure how to compile something from a different branch without > > compiling the whole world. make what? > > For bhyve it's easy. Check out the sources, go to usr.sbin/bhyve and run > > make depend; make obj; make; make install. Unfortunately, after all my efforts, with libvmmapi and bhyve compiled from CURRENT sources: Aug 09 12:46:52: warning; UEFI graphics is only available in FreeBSD 11 and newer Aug 09 12:46:52: booting Bummer! -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Matt Churchyard wrote: > > > > > > > > > > > > uefi="yes" > > > > > > graphics="yes" > > > > > > > > > > Dear Colleagues, > > > > > > > > > > Can I enjoy and test all those nice "UEFI-GOP" features on a > > > > > 10.3-RELEASE system, or do I have to install CURRENT for that? > > > > > > > > AFAIK you can checkout usr.sbin/bhyve from the 11 branch and compile it > > > > on 10.3. > > > > > > I'm not sure how to compile something from a different branch without > > > compiling the whole world. make what? > > > > For bhyve it's easy. Check out the sources, go to usr.sbin/bhyve and run > > > > make depend; make obj; make; make install. > > >Unfortunately, after all my efforts, with libvmmapi and bhyve compiled from > >CURRENT sources: > > >Aug 09 12:46:52: warning; UEFI graphics is only available in FreeBSD 11 and > >newer > >Aug 09 12:46:52: booting > > >Bummer! > > It's possible that you do have a bhyve system with graphics support. > Unfortunately vm-bhyve isn't aimed at users who have manually merged changes, > and will blindly tell anyone that isn't running 11+ that they don't have > graphics support. > > You have a couple of options - > > 1) Run bhyve manually rather than using vm-bhyve > 2) Upgrade to the 11 beta > 3) Modify vm-bhyve to not automatically skip graphics support if the version > is less than 11 Aha! I've commented out the version check from vm-run, and now when trying to install, I get: авг 09 17:01:05: initialising авг 09 17:01:05: [loader: none] авг 09 17:01:05: [uefi: yes] авг 09 17:01:05: [cpu: 1] авг 09 17:01:05: [memory: 1G] авг 09 17:01:05: [hostbridge: standard] авг 09 17:01:05: [com ports: com1] авг 09 17:01:05: [uuid: 093f9134-5e18-11e6-9502-5404a6b49a66] авг 09 17:01:05: [utctime: no] авг 09 17:01:05: [debug mode: no] авг 09 17:01:05: [primary disk: disk0.img] авг 09 17:01:05: [primary disk dev: file] авг 09 17:01:05: generated static mac 58:9c:fc:03:77:3b (based on 'win:0:1470736865:0') авг 09 17:01:05: initialising network device tap5 авг 09 17:01:05: adding tap5 -> bridge1 (isolated) авг 09 17:01:05: dynamically allocated port 5900 for vnc connections авг 09 17:01:05: booting авг 09 17:01:05: [bhyve options: -c 1 -m 1G -Hwl bootrom,/d02/vm/.config/BHYVE_UEFI.fd -U 093f9134-5e18-11e6-9502-5404a6b49a66] авг 09 17:01:05: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,ahci-hd,/d02/vm/win/disk0.img -s 5:0,virtio-net,tap5,mac=58:9c:fc:03:77:3b -s 6:0,fbuf,tcp=0.0.0.0:5900,wait] авг 09 17:01:05: [bhyve console: -l com1,/dev/nmdm0A] авг 09 17:01:05: [bhyve iso device: -s 3:0,ahci-cd,/d02/vm/.iso/bhyve_win7_x64.iso] авг 09 17:01:05: starting bhyve (run 1) авг 09 17:01:05: bhyve exited with status 1 авг 09 17:01:05: destroying network device tap5 авг 09 17:01:05: stopped -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Matt Churchyard wrote: > > > > > > > > > > > > > > uefi="yes" > > > > > > > graphics="yes" > > > > > > > > > > > > Dear Colleagues, > > > > > > > > > > > > Can I enjoy and test all those nice "UEFI-GOP" features on a > > > > > > 10.3-RELEASE system, or do I have to install CURRENT for that? > > > > > > > > > > AFAIK you can checkout usr.sbin/bhyve from the 11 branch and compile > > > > > it > > > > > on 10.3. > > > > > > > > I'm not sure how to compile something from a different branch without > > > > compiling the whole world. make what? > > > > > > For bhyve it's easy. Check out the sources, go to usr.sbin/bhyve and run > > > > > > make depend; make obj; make; make install. > > > > >Unfortunately, after all my efforts, with libvmmapi and bhyve compiled from > > >CURRENT sources: > > > > >Aug 09 12:46:52: warning; UEFI graphics is only available in FreeBSD 11 > > >and newer > > >Aug 09 12:46:52: booting > > > > >Bummer! > > > > It's possible that you do have a bhyve system with graphics support. > > Unfortunately vm-bhyve isn't aimed at users who have manually merged > > changes, and will blindly tell anyone that isn't running 11+ that they > > don't have graphics support. > > > > You have a couple of options - > > > > 1) Run bhyve manually rather than using vm-bhyve > > 2) Upgrade to the 11 beta > > 3) Modify vm-bhyve to not automatically skip graphics support if the > > version is less than 11 > > >Aha! I've commented out the version check from vm-run, and now when > >trying to install, I get: > > >авг 09 17:01:05: initialising > >авг 09 17:01:05: [loader: none] > >авг 09 17:01:05: [uefi: yes] > >авг 09 17:01:05: [cpu: 1] > >авг 09 17:01:05: [memory: 1G] > >авг 09 17:01:05: [hostbridge: standard] > >авг 09 17:01:05: [com ports: com1] > >авг 09 17:01:05: [uuid: 093f9134-5e18-11e6-9502-5404a6b49a66] > >авг 09 17:01:05: [utctime: no] > >авг 09 17:01:05: [debug mode: no] > >авг 09 17:01:05: [primary disk: disk0.img] > >авг 09 17:01:05: [primary disk dev: file] > >авг 09 17:01:05: generated static mac 58:9c:fc:03:77:3b (based on > >'win:0:1470736865:0') > >авг 09 17:01:05: initialising network device tap5 > >авг 09 17:01:05: adding tap5 -> bridge1 (isolated) > >авг 09 17:01:05: dynamically allocated port 5900 for vnc connections > >авг 09 17:01:05: booting > >авг 09 17:01:05: [bhyve options: -c 1 -m 1G -Hwl > >bootrom,/d02/vm/.config/BHYVE_UEFI.fd -U > >093f9134-5e18-11e6-9502-5404a6b49a66] > >авг 09 17:01:05: [bhyve devices: -s 0,hostbridge -s 31,lpc -s > >4:0,ahci-hd,/d02/vm/win/disk0.img -s > >5:0,virtio-net,tap5,mac=58:9c:fc:03:77:3b -s >6:0,fbuf,tcp=0.0.0.0:5900,wait] > >авг 09 17:01:05: [bhyve console: -l com1,/dev/nmdm0A] > >авг 09 17:01:05: [bhyve iso device: -s > >3:0,ahci-cd,/d02/vm/.iso/bhyve_win7_x64.iso] > >авг 09 17:01:05: starting bhyve (run 1) > >авг 09 17:01:05: bhyve exited with status 1 > >авг 09 17:01:05: destroying network device tap5 > >авг 09 17:01:05: stopped > > Looks like bhyve is exiting immediately. Easiest thing is to probably add > debug="yes" to the guest config file then try and run it again. Any output > from the bhyve command will be written to bhyve.log (note that's a different > file to vm-bhyve.log). > Tried that. root@vas:/d02/vm/win # file bhyve.log bhyve.log: empty root@vas:/d02/vm/win # more win.conf guest="windows" uefi="yes" graphics="yes" cpu=1 memory=1G network0_type="virtio-net" network0_switch="isolated" disk0_type="ahci-hd" disk0_name="disk0.img" debug="yes" uuid="093f9134-5e18-11e6-9502-5404a6b49a66" network0_mac="58:9c:fc:03:77:3b" root@vas:/d02/vm/win # I have tried replicating manually all the options vm is passing to bhyve, like bhyve -c 1 -m 1G -Hwl bootrom,/d02/vm/.config/BHYVE_UEFI.fd \ -U 093f9134-5e18-11e6-9502-5404a6b49a66 \ -s 0,hostbridge -s 31,lpc -s 4:0,ahci-hd,/d02/vm/win/disk0.img \ -s 5:0,virtio-net,tap5,mac=58:9c:fc:03:77:3b -s 6:0,fbuf,tcp=0.0.0.0:5900,wait \ -l com1,stdio \ -s 3:0,ahci-cd,/d02/vm/.iso/bhyve_win7_x64.iso But bhyve just spits out the "usage" help (that's why it exits immediately). But I cannot find out what's incorrect with the options. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Matt Churchyard wrote: > > >I have tried replicating manually all the options vm is passing to bhyve, > >like > > >bhyve -c 1 -m 1G -Hwl bootrom,/d02/vm/.config/BHYVE_UEFI.fd \ > >-U 093f9134-5e18-11e6-9502-5404a6b49a66 \ > >>-s 0,hostbridge -s 31,lpc -s 4:0,ahci-hd,/d02/vm/win/disk0.img \ > >-s 5:0,virtio-net,tap5,mac=58:9c:fc:03:77:3b -s > > 6:0,fbuf,tcp=0.0.0.0:5900,wait \ > >-l com1,stdio \ > > -s 3:0,ahci-cd,/d02/vm/.iso/bhyve_win7_x64.iso > > > But bhyve just spits out the "usage" help (that's why it exits > > immediately). But I cannot find out what's incorrect with the options. > > You just need the vm name (win) on the end: > > # bhyve [options] vm-name > Sorry, I was being stupid. With the vm name on the end, it just exits with error code 1 and without printing anytning. I have recompiled bhyve itself and the libvmmapi, maybe it's not enough? The kernel module? It still remains that from 10.3 -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve tests and findings
Yamagi Burmeister wrote: > Hi, > to be able to use the GOP instructions on FreeBSD 10.3 you'll need to > merge r296103 [0]. If it's an AMD system it's a good idea to merge > r297806 [1], too. Otherwise most Linux distros will panic at startup. I'd rather abandon the idea and get a separate box with FreeBSD 11 for UEFI-GOP testing. Matt and others, thanks for the effort. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
feature request
Dear Matt, Could you add a very useful feature? I would greatly appreciate being able to view and delete VM snapshots via the "vm" subcommand. Presently I have to use the zfs interface directly. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: feature request
Victor Sudakov wrote: > > Could you add a very useful feature? I would greatly appreciate being > able to view and delete VM snapshots via the "vm" subcommand. > Presently I have to use the zfs interface directly. Also, "vm destroy" does not destroy VMs, if they have snapshots: # vm destroy my-test1 Are you sure you want to completely remove this virtual machine (y/n)? y /usr/local/sbin/vm: ERROR: failed to destroy ZFS dataset d02/vm/my-test1 # zfs destroy d02/vm/my-test1 cannot destroy 'd02/vm/my-test1': filesystem has children use '-r' to destroy the following datasets: d02/vm/my-test1@2016-07-30-23:37:34 # I think "vm destroy" should always destroy the VM no matter the snapshots and dependent clones. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Time sync
Colleagues, Do FreeBSD guests in bhyve need time synchronization (a running ntpd or periodic ntpdate), or do they inherit accurate time from the host system? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Multiple VM hosting using bhyve
The Doctor wrote: > > All right, what bhyve sequence do I need to > distinguish the taps and the guests? I would strongly advise installing ports/sysutils/vm-bhyve, It makes VM management very easy and convenient. No need to learn the low-level bhyve options. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Time sync
Peter Grehan wrote: > > > Do FreeBSD guests in bhyve need time synchronization (a running ntpd > > or periodic ntpdate), or do they inherit accurate time from the host > > system? > > Best to run ntpd. VM guests can have much larger clock drift than > bare-metal systems. Thank you. I just thought there was a way to sync guests from host, but if there is none - let it be ntpd. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
choice of i386 vs amd64 guest
Colleagues, When I must run several FreeBSD guests in bhyve, should I prefer i386 or amd64 guests provided a guest will never need more than 2-3 GB of RAM? The bhyve host is of course amd64. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: choice of i386 vs amd64 guest
Paul Vixie wrote: > > > > > When I must run several FreeBSD guests in bhyve, should I prefer i386 > > or amd64 guests provided a guest will never need more than 2-3 GB of > > RAM? > > if you're running bind9, the sizeof(void *) will dominate your heap > size, and a 32-bit guest will be better overall, given that you don't > need the larger addressing size. Those guests will be running web servers mostly. > > for everything else, the larger register set on amd64 will matter more, > and you should probably use a 64-bit guest. There is an opinion that 64-bit systems use up more resources than 32-bit ones for the same tasks, it that not true? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tomsk.ru ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Outdated grub2-bhyve and booting Centos7
Dear Colleagues, Can I draw your attention to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 please? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
The Doctor via freebsd-virtualization wrote: > Are there issues with Current CEntos and bhyve? Sure there are, please look at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > I have tried 3 installation method. > > On 2nd phase, it looks like the xfs indoe are skewered. Duh! -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Ganbold Tsagaankhuu wrote: > On Sat, Aug 11, 2018 at 7:46 AM, The Doctor via freebsd-virtualization < > freebsd-virtualization@freebsd.org> wrote: > > > Are there issues with Current CEntos and bhyve? > > > > I have tried 3 installation method. > > > > On 2nd phase, it looks like the xfs indoe are skewered. > > > > > Maybe try to use CentOS-7-x86_64-Minimal-1511.iso. This is 7.2.1551. Later > versions default to XFS which FreeBSD grub2 doesn't seem to support. There > is no known way to switch this from anaconda so you have to use the old ISO > and then update. Does not work even if installed from CentOS-7-x86_64-Minimal-1511.iso. See what happens under FreeBSD 11.2-RELEASE, vm-bhyve-1.1.8_2: [ OK ] Reached target Basic System. [1.297080] vda: vda1 vda2 vda3 [1.508128] input: PS/2 Generic Mouse as /devices/platform/i8042/serio1/input/input2 [1.693391] tsc: Refined TSC clocksource calibration: 3308.829 MHz [1.695742] Switching to clocksource tsc seems good so far [1.693391] tsc: Refined TSC clocksource calibration: 3308.829 MHz [1.695742] Switching to clocksource tsc [ 124.909494] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 125.424725] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 125.937988] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 126.450692] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts ... [ 185.728954] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 185.729227] dracut-initqueue[219]: Warning: Could not boot. [ OK ] Started Show Plymouth Boot Screen. [ OK ] Reached target Paths. [ OK ] Reached target Basic System. [ 124.909494] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 125.424725] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 185.728954] dracut-initqueue[219]: Warning: dracut-initqueue timeout - starting timeout scripts [ 185.729227] dracut-initqueue[219]: Warning: Could not boot. [ 185.825970] dracut-initqueue[219]: Warning: /dev/mapper/centos-root does not exist Starting Dracut Emergency Shell... Warning: /dev/mapper/centos-root does not exist Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report. dracut:/# -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Shawn Webb wrote: > On Sat, Aug 11, 2018 at 12:53:05PM +0700, Victor Sudakov wrote: > > The Doctor via freebsd-virtualization wrote: > > > Are there issues with Current CEntos and bhyve? > > > > Sure there are, please look at > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > Booting in UEFI mode works. This means we need an update to /usr/local/share/examples/vm-bhyve/centos7.conf ? It says 'loader="grub"' for the present. Do you have a vm config to boot centos7 in UEFI mode you could share? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
The Doctor wrote: > > > The Doctor via freebsd-virtualization wrote: > > > > Are there issues with Current CEntos and bhyve? > > > > > > Sure there are, please look at > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > Booting in UEFI mode works. > > > > Chokes in my case. > > I have better luck with Debian. Yes, Ubuntu server works too, but do we have anything RPM-based that runs in bhyve? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Shawn Webb wrote: > > > > > Are there issues with Current CEntos and bhyve? > > > > > > > > Sure there are, please look at > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > > > Booting in UEFI mode works. > > > > This means we need an update to > > /usr/local/share/examples/vm-bhyve/centos7.conf ? > > It says 'loader="grub"' for the present. > > > > Do you have a vm config to boot centos7 in UEFI mode you could share? > > I just use /usr/share/examples/bhyve/vmrun.sh. I don't use any > third-party utility to manage bhyve VMs. vmrun.sh is pretty > straight-forward. Thanks for replying. However, I highly recommend vm-bhyve, maybe you should give it a try. You will love the ease of VM creation and provisioning, network management, ZFS integration (VM snapshots and cloning), console and datastore management etc. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Rodney W. Grimes wrote: > > > > > > > Are there issues with Current CEntos and bhyve? > > > > > > > > > > > > Sure there are, please look at > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > > > > > > > Booting in UEFI mode works. > > > > > > > > This means we need an update to > > > > /usr/local/share/examples/vm-bhyve/centos7.conf ? > > > > It says 'loader="grub"' for the present. > > > > > > > > Do you have a vm config to boot centos7 in UEFI mode you could share? > > > > > > I just use /usr/share/examples/bhyve/vmrun.sh. I don't use any > > > third-party utility to manage bhyve VMs. vmrun.sh is pretty > > > straight-forward. > > > > Thanks for replying. However, I highly recommend vm-bhyve, maybe you > > should give it a try. You will love the ease of VM creation and > > provisioning, network management, ZFS integration (VM snapshots and > > cloning), console and datastore management etc. > > Though it has a lot of features, it also has some short comings, > like you can not spec a vm to be wired in memory, which IMHO is > the only way to insure consistent VM performance. Well, we have "bhyve_options" configuration option in the vm config, why not put "-S" there, is that what you mean by wiring the vm in memory? > > Its artificial restriction of 16 character VM names is also > a fair bit annoying. Maybe. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Rodney W. Grimes wrote: > > Rodney W. Grimes wrote: > > > > > > > > > Are there issues with Current CEntos and bhyve? > > > > > > > > > > > > > > > > Sure there are, please look at > > > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > > > > > > > > > > > Booting in UEFI mode works. > > > > > > > > > > > > This means we need an update to > > > > > > /usr/local/share/examples/vm-bhyve/centos7.conf ? > > > > > > It says 'loader="grub"' for the present. > > > > > > > > > > > > Do you have a vm config to boot centos7 in UEFI mode you could > > > > > > share? > > > > > > > > > > I just use /usr/share/examples/bhyve/vmrun.sh. I don't use any > > > > > third-party utility to manage bhyve VMs. vmrun.sh is pretty > > > > > straight-forward. > > > > > > > > Thanks for replying. However, I highly recommend vm-bhyve, maybe you > > > > should give it a try. You will love the ease of VM creation and > > > > provisioning, network management, ZFS integration (VM snapshots and > > > > cloning), console and datastore management etc. > > > > > > Though it has a lot of features, it also has some short comings, > > > like you can not spec a vm to be wired in memory, which IMHO is > > > the only way to insure consistent VM performance. > > > > Well, we have "bhyve_options" configuration option in the vm config, > > why not put "-S" there, is that what you mean by wiring the vm in > > memory? > > I believe that fails as that only adds the -S to bhyve, and > you must specify it both on bhyveload and bhyve for it to > work. I think it is totally doable becase vm-bhyve is nothing but a suit of scripts. A PR with a feature request would be appropriate. What about VM that don't use bhyveload, but some other kind of loader like grub2-bhyve? > > > > > > Its artificial restriction of 16 character VM names is also > > > a fair bit annoying. > > > > Maybe. > > Maybe? No, factually. I migrated a number of ESXi VM's and > had to patch vm-bhyve to not have this restriction, so it is > annoying. Did you send your patches upstream? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Rodney W. Grimes wrote: [dd] > > > > > > > > > > Though it has a lot of features, it also has some short comings, > > > > > like you can not spec a vm to be wired in memory, which IMHO is > > > > > the only way to insure consistent VM performance. > > > > > > > > Well, we have "bhyve_options" configuration option in the vm config, > > > > why not put "-S" there, is that what you mean by wiring the vm in > > > > memory? > > > > > > I believe that fails as that only adds the -S to bhyve, and > > > you must specify it both on bhyveload and bhyve for it to > > > work. > > > > I think it is totally doable becase vm-bhyve is nothing but a suit of > > scripts. A PR with a feature request would be appropriate. > > I made several attempts to contact the author at the email address > provided at the git hub while making other bhyve changes to try > and coordinate with him. I got no response after 3 attempts, > so have stopped trying to contact them. (This was while I was > adding the -c cpu topology modifications.) You can add yourself to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230580 maybe something useful comes out of it. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Matt Churchyard wrote: > > Rodney W. Grimes wrote: > > [dd] > > > I've already commented on the bug, although I'll reply here as well. > If "-S" is found in bhyve_options it does currently affect both > commands. Sorry, this is not working for me (vm-bhyve 1.1.8_2 from official packages): Aug 13 16:43:09: bhyve exited with status 1 Aug 13 16:43:09: destroying network device tap0 Aug 13 16:43:09: stopped Aug 13 16:43:12: initialising Aug 13 16:43:12: [loader: bhyveload] Aug 13 16:43:12: [uefi: no] Aug 13 16:43:12: [cpu: 1] Aug 13 16:43:12: [memory: 1G] Aug 13 16:43:12: [hostbridge: standard] Aug 13 16:43:12: [com ports: com1] Aug 13 16:43:12: [uuid: e148afdb-d4af-11e6-9a94-5404a6b49a66] Aug 13 16:43:12: [utctime: yes] Aug 13 16:43:12: [debug mode: no] Aug 13 16:43:12: [primary disk: disk0.img] Aug 13 16:43:12: [primary disk dev: file] Aug 13 16:43:12: initialising network device tap0 Aug 13 16:43:12: adding tap0 -> bridge0 (main) Aug 13 16:43:12: booting Aug 13 16:43:12: bhyveload -c /dev/nmdm0A -m 1G -e autoboot_delay=3 -d /d02/vm/fido/disk0.img fido Aug 13 16:43:17: [bhyve options: -c 1 -m 1G -AHP -S -U e148afdb-d4af-11e6-9a94-5404a6b49a66 -u] Aug 13 16:43:17: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/d02/vm/fido/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:01:f6:ac] Aug 13 16:43:17: [bhyve console: -l com1,/dev/nmdm0A] Aug 13 16:43:17: starting bhyve (run 1) Aug 13 16:43:17: bhyve exited with status 1 Aug 13 16:43:17: destroying network device tap0 Aug 13 16:43:17: stopped Here is my complete config: guest="freebsd" loader="bhyveload" cpu=1 memory=1G network0_type="virtio-net" network0_switch="main" disk0_type="virtio-blk" disk0_name="disk0.img" utctime="yes" uuid="e148afdb-d4af-11e6-9a94-5404a6b49a66" network0_mac="58:9c:fc:01:f6:ac" bhyve_options="-S" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Matt Churchyard wrote: > wired_memory option has been added to next release. Thanks a lot, Matt! May I ask a question: how is the "-S" option supposed to work if the loader is not bhyveload, but grub2-bhyve or UEFI? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Curent Centos 7 and bhyve
Matt Churchyard wrote: > > Sorry, it looks like the change didn't make it into 1.1. You'd need > to upgrade to 1.2, which is in ports but looks like it hasn't made > it into pkg yet. portsmon.freebsd.org is still broken also so I > can't easily confirm whether it's made it into any of the > quarterly/latest pkg repos. (It was in ports over a month ago so > should at least be in latest) I see. My own poudriere has just built vm-bhyve-1.2.3, but I don't use my repo on that bhyve host, I use the official repo there. Does anyone know if I can install just one (or some) package from a different repo, the official one still remaining the default? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[vm-bhyve] Does anyone have a vm template for Linux Mint ?
Dear Colleagues, Can anyone please share a vm template for the installation of Linux Mint 19? It seems to have only a GUI installer so the loader should be set to "uefi" instead of "grub" to enable the VNC console, but how do I accomplish this? Thanks in advance for any input. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Josias L. Gonçalves wrote: > My ubuntu is something like that: > > uefi="csm" ^^^ This is supposed to be the key feature! I did not know about UEFI-CSM. However, when I start the vm: # vm install mint linuxmint-19-cinnamon-64bit-v2.iso Starting mint * found guest in /d02/vm/mint * booting... the TightVNC "console" is just a blank rectangle. Nothing ever happens. Is there anything else I'm missing? The vm is shown as running: mint defaultuefi 12G 0.0.0.0:5909 No Running (81329) My config (a copy of yours): guest="linux" uefi="csm" cpu=1 memory=2G network0_type="virtio-net" network0_switch="public" disk0_type="virtio-blk" disk0_name="disk0.img" # VNC graphics="yes" graphics_wait="no" graphics_res="1280x720" graphics_port="5909" graphics_listen="0.0.0.0" #xhci_mouse="yes" uuid="f9bbfaab-cafc-11e8-8d16-5404a6b49a66" network0_mac="58:9c:fc:08:91:3d" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Victor Sudakov wrote: > Josias L. Gonçalves wrote: > > My ubuntu is something like that: > > > > uefi="csm" > ^^^ > > This is supposed to be the key feature! I did not know about UEFI-CSM. > > However, when I start the vm: > > # vm install mint linuxmint-19-cinnamon-64bit-v2.iso > Starting mint > * found guest in /d02/vm/mint > * booting... > > > the TightVNC "console" is just a blank rectangle. Nothing ever happens. I get the same blank rectangle with Ubuntu (vm install mint ubuntu-16.04-desktop-amd64.iso) So there must be something amiss in my setup. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Victor Sudakov wrote: > Victor Sudakov wrote: > > Josias L. Gonçalves wrote: > > > My ubuntu is something like that: > > > > > > uefi="csm" > > ^^^ > > > > This is supposed to be the key feature! I did not know about UEFI-CSM. > > > > However, when I start the vm: > > > > # vm install mint linuxmint-19-cinnamon-64bit-v2.iso > > Starting mint > > * found guest in /d02/vm/mint > > * booting... > > > > > > the TightVNC "console" is just a blank rectangle. Nothing ever happens. > > I get the same blank rectangle with Ubuntu (vm install mint > ubuntu-16.04-desktop-amd64.iso) > So there must be something amiss in my setup. The log: окт. 08 21:46:29: initialising окт. 08 21:46:29: [loader: none] окт. 08 21:46:29: [uefi: csm] окт. 08 21:46:29: [cpu: 1] окт. 08 21:46:29: [memory: 2G] окт. 08 21:46:29: [hostbridge: standard] окт. 08 21:46:29: [com ports: com1] окт. 08 21:46:29: [uuid: f9bbfaab-cafc-11e8-8d16-5404a6b49a66] окт. 08 21:46:29: [utctime: yes] окт. 08 21:46:29: [debug mode: no] окт. 08 21:46:29: [primary disk: disk0.img] окт. 08 21:46:29: [primary disk dev: file] окт. 08 21:46:29: initialising network device tap1 окт. 08 21:46:29: adding tap1 -> bridge0 (main addm) окт. 08 21:46:29: bring up tap1 -> bridge0 (main addm) окт. 08 21:46:29: booting окт. 08 21:46:29: [bhyve options: -c 1 -m 2G -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd -U f9bbfaab-cafc-11e8-8d16-5404a6b49a66 -u] окт. 08 21:46:29: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/d02/vm/mint/disk0.img -s 5:0,virtio-net,tap1,mac=58:9c:fc:08:91:3d -s 6:0,fbuf,tcp=0.0.0.0:5909,wait] окт. 08 21:46:29: [bhyve console: -l com1,/dev/nmdm-mint.1A] окт. 08 21:46:29: [bhyve iso device: -s 3:0,ahci-cd,/d02/vm/.iso/ubuntu-16.04-desktop-amd64.iso] окт. 08 21:46:29: starting bhyve (run 1) окт. 08 21:50:08: bhyve exited with status 143 окт. 08 21:50:08: destroying network device tap1 окт. 08 21:50:08: stopped I waited about 4 minutes before sending "vm mint stop" looking into the blank VNC console. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Josias L. Gonçalves wrote: > > > I get the same blank rectangle with Ubuntu (vm install mint > > > ubuntu-16.04-desktop-amd64.iso) > > > So there must be something amiss in my setup. > > > Verify if everything is installed: > pkg install bhyve-firmware-1.0_1 bhyve-rc-3 grub2-bhyve-0.40_5 > libhyve-remote-0.1.4.2 uefi-edk2-bhyve-0.1,1 uefi-edk2-bhyve-csm-0.1,1 Well, not everything from your list. I have currently root@vas:~ # pkg info -a | egrep 'bhyve|uefi' grub2-bhyve-0.40_4 Grub-emu loader for bhyve uefi-edk2-bhyve-0.1,1 UEFI-EDK2 firmware for bhyve uefi-edk2-bhyve-csm-0.1,1 UEFI-EDK2 firmware for bhyve with CSM vm-bhyve-1.2.3 Management system for bhyve virtual machines root@vas:~ # bhyve-firmware: is a metaport for uefi-edk2-bhyve and uefi-edk2-bhyve-csm, I already have them both installed. But no harm installing it also. bhyve-rc-3: "FreeBSD RC script for starting bhyve guests in tmux" - I definitely don't need it because I use vm-bhyve for VM management. libhyve-remote-0.1.4.2: well, maybe this one is the culprit. We shall see tonight if its presence makes any difference. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Matt Churchyard wrote: [dd] > Personally I've never had success with the CSM firmware (other that > with SmartOS which was following instructions from the original > bhyve devs). My understanding is that CSM is supposed to emulate a > traditional BIOS, so anything that would boot on a BIOS should work > (or at least try to boot). I would LOVE to have such a thing! > However I usually just get a blank screen > and nothing else. Same happens to myself. However Josias says he has Ubuntu running with uefi="csm", let's find out his secret! :-) > > I have successfully** installed Ubuntu 16.04 (via VNC) using the following > template, which is just a basic UEFI config with virtio network/disk. > > uefi="yes" > graphics="yes" > xhci_mouse="yes" > cpu=2 > memory=2G > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0.img" > > **I did have two fairly annoying issues getting 16.04 to work - Thank you for the hints. Will try them tonight. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Matt Churchyard wrote: > > I have successfully** installed Ubuntu 16.04 (via VNC) using the following > template, which is just a basic UEFI config with virtio network/disk. > > uefi="yes" > graphics="yes" > xhci_mouse="yes" > cpu=2 > memory=2G > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0.img" This plain UEFI template worked for me too, for Linux Mint. Thank you. > > **I did have two fairly annoying issues getting 16.04 to work - > > 1) After installation it seemed to refuse to reboot without me > removing the install CD, which I obviously couldn't do. I ended up > powering off the system manually with "vm poweroff guest" Had this problem too, I sent "vm stop mint" from another xterm. > > 2) It then would not boot due to the fact that Ubuntu 16.04 puts the > bootloader in a non-default location, then uses efivars to tell UEFI > how to find it. Unfortunately the bhyve EFI firmware still does not > save efivars, so any specific boot options set by a guest are lost > on restart. Did not have this problem with Mint. > I had to escape to the EFI shell and manually choose the > EFI/ubuntu/grubx64.efi loader. See my post #11 on this forum post > for more details - > https://forums.freebsd.org/threads/how-to-install-a-ubuntu-guest-in-bhyve.66767/ Thank you again, I may need it one day. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [vm-bhyve] Does anyone have a vm template for Linux Mint ?
Josias L. Gonçalves wrote: > My ubuntu is something like that: > > guest="linux" > uefi="csm" > cpu=1 > memory=2G > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0.img" > Dear Josias, If you have a success story with uefi="csm", please share more detail with us. Does uefi="csm" fork for anyone? Can you boot MS-DOS in bhyve, for example? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve uefi="csm"
Dear Colleagues, Has anyone been able to boot *anything* with uefi="csm"? Can you please share at least one success story? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve uefi="csm"
Jason Barbier wrote: > > Has anyone been able to boot *anything* with uefi="csm"? > Can you tell us a bit about your set up, like are you using helper scripts > like libvirt? Sorry, I forgot to mention. I use sysutils/vm-bhyve with sysutils/uefi-edk2-bhyve-csm firmware (uefi-edk2-bhyve-csm-0.1,1) > To answer the question you asked when I run a vm with the CSM firmware it > works for me. Do you also use vm-bhyve and most importantly, what OS do you boot with CSM firmware? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve uefi="csm"
Jason Barbier wrote: > Nope I don't use vm-bhyve I use some scrips I home rolled. The os I was > booting with CSM was arch linux. Why would you want to boot Arch Linux with CSM? It should be possible to boot it either with grub2-bhyve or with UEFI. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve uefi="csm"
Rodney W. Grimes wrote: > > Can you tell us a bit about your set up, like are you using helper scripts > > like libvirt? > > To answer the question you asked when I run a vm with the CSM firmware it > > works for me. > > You should always be able to boot a vm that uses > and boots with the non CSM uefi firmware with the > CSM version of the firmware, failures may start > to show up if your booting something that is > either purely legacy, or efi aware but falls > back to csm because it didnt like something. If my goal were to boot FreeDOS in bhyve, would that be ever possible? > > The older ED2K uefi that we are using and > some of the hatchetry that was used to craft > its CSM are in a poor state and could use > some love. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve uefi="csm"
Rodney W. Grimes wrote: Rodney W. Grimes wrote: > > Can you tell us a bit about your set up, like are you using helper scripts like libvirt? > > To answer the question you asked when I run a vm with the CSM firmware it works for me. > > You should always be able to boot a vm that uses > and boots with the non CSM uefi firmware with the > CSM version of the firmware, failures may start > to show up if your booting something that is > either purely legacy, or efi aware but falls > back to csm because it didnt like something. If my goal were to boot FreeDOS in bhyve, would that be ever possible? Today no, due to the fact we do not have a ATA/legacy emulation, but rather only ahci disk emultaion, though I would be wrong on that point if FreeDOS has an ahci disk driver. According to http://wiki.freedos.org/wiki/index.php/Main_Page#What_makes_FreeDOS_better.3F FreeDOS has "UDMA drivers for hard disks and DVD drives", is it not the thing? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
mounting/exporting/importing a zfs volume
Dear Colleagues, vm-bhyve keeps virtual machines on zfs volumes with volmode=dev. How can I access/mount the filesystems within the volume when the virtual host is offline? If I kept virtual disks in raw files, I could access them as devices with mdconfig. But: root@newserv:~ # mdconfig -a -f /dev/zvol/zroot/vm/mail/disk0 mdconfig: /dev/zvol/zroot/vm/mail/disk0 is not a regular file root@newserv:~ # Also, how can I exchange those zfs volumes for use with other hypervisors? They are not real raw disk files so I cannot use sysutils/vmdktool etc. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: mounting/exporting/importing a zfs volume
Paul Webster wrote: > mdconfig -a -t vnode -f afaik Sorry, this does not work. root@newserv:~ # mdconfig -a -t vnode /dev/zvol/zroot/vm/mail/disk0 mdconfig: /dev/zvol/zroot/vm/mail/disk0 is not a regular file There must be some other way. > once they are mounted you could dd them Indeed, I can dd the volume to a regular file: dd if=/dev/zvol/zroot/vm/mail/disk0 of=/var/tmp/rawimage bs=1m conv=sparse and then work with the /var/tmp/rawimage (I can mdconfig it) but it's a waste of disk space. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: mounting/exporting/importing a zfs volume
Frank Leonhardt wrote: > > > > vm-bhyve keeps virtual machines on zfs volumes with volmode=dev. How can > > I access/mount the filesystems within the volume when the virtual host > > is offline? > > > > If I kept virtual disks in raw files, I could access them as devices > > with mdconfig. But: > > > > root@newserv:~ # mdconfig -a -f /dev/zvol/zroot/vm/mail/disk0 > > mdconfig: /dev/zvol/zroot/vm/mail/disk0 is not a regular file > > root@newserv:~ # > > > > Also, how can I exchange those zfs volumes for use with other > > hypervisors? They are not real raw disk files so I cannot use > > sysutils/vmdktool etc. > > > I don't know this, but I'll guess(!) > > If you've set volmode to dev then you get a cdev device in devfs, and > you'll never get it to mount. Try using geom instead (which IIRC is the > default). The default in vm-bhyve is volmode=dev, and I think this is reasonable. Do you know if I can clone an existing volmode=dev volume into a volmode=geom volume and then work with the clone? > > HOWEVER, I suspect you're doing this because you're hoping that a ZFS > volume is faster than a file. Well, not actually. > I went through this, in the hope it wouldn't do CoW and would > therefore be a lot better for databases. I was disappointed! > Bascially, it's no better than a ZFS file. If that was your plan, use > a UFS partition. A UFS partition? Where? > I don't use ZFS volumes any more; I > think they're more useful on Solaris. A md mapped on to a ZFS file seems > to be the BSD way, and for VMs just use a file in its own dataset. You > can then clone the dataset. Just what you need for nearly identical VMs. I've preferred disk0_dev="zvol" VMs for aesthetical reasons since vm-bhyve started supporting them. Those file-based VMs get in the way while backing up $vm_dir, and their disks are not visible in "zfs list -t volume" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
What's that disk0_type="nvme" thing? I'm curious. In 11.2-RELEASE there is nothing about nvme in bhyve(8). Should I be running CURRENT or what? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Jason Tubnor wrote: > > > What's that disk0_type="nvme" thing? > > > > I'm curious. In 11.2-RELEASE there is nothing about nvme in bhyve(8). > > Should I be running CURRENT or what? > > > > > > > It is valid on hosts 12.0-RELEASE and newer. Is it worth it? I mean upgrading from 11.2-RELEASE to 12.0-RELEASE for the sake of nvme, is it useful for running FreeBSD, Linux (Mint, Centos) and Windows10 guests? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Jason Tubnor wrote: > > > If you have Nvme devices or you want store disks in ram yes, otherwise no. > > > > > > > > Is it worth it? I mean upgrading from 11.2-RELEASE to 12.0-RELEASE for > > > the sake of nvme, is it useful for running FreeBSD, Linux (Mint, > > > Centos) and Windows10 guests? > > > > > > Or if you are using good SLOG and ARC for your VM zvols. > > I'm still sorting out some Windows issues with the bhyve nvme presentation > for a variety of guest (except freebsd, that works fine) at the moment. I > would avoid 12.0-RELEASE as it may introduce other issues with your I was just worried that using ahci-hd instead of paravirtualized disks causes suboptimal performance of guests. Do you think there is a hope to get paravirtualized disks support for Windows guests in bhyve? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Stefan Bethke wrote: > > > > I was just worried that using ahci-hd instead of paravirtualized disks > > causes suboptimal performance of guests. Do you think there is a hope to get > > paravirtualized disks support for Windows guests in bhyve? > > I was under the impression that the Windows VirtIO disk driver works > with the Bhyve virtio-blk device. You need to provide these drivers to > Windows somehow (for example, as a second CD image during > installation): > https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers I use NetKVM (VirtIO network) driver from there all the time, because e1000 is bad. However, I've read a number of bhyve resources including https://wiki.freebsd.org/bhyve/Windows and I've never found a success story with paravirtualized Windows disk drivers. If anyone has been successful with viostor(?), please share your experience. I would be especially grateful for instruction how to install Windows on a viostor disk instead of ahci-hd. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Victor Sudakov wrote: > > If anyone has been successful with viostor(?), please share your experience. Or at least please tell me how I can attach another iso image (with drivers) when running "vm install myhost windows.iso" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Jason Tubnor wrote: > On Mon, 18 Feb 2019 at 02:58, Victor Sudakov wrote: > > > > > Or at least please tell me how I can attach another iso image (with > > drivers) when running "vm install myhost windows.iso" > > > Technically, you should be able to define something like: > > disk1_type="ahci-cd" > disk1_name="virtio-win-0.1.164.iso" > disk1_dev="file" > > in the vm.conf file if the virtio-win-0.1.164.iso is located in > /vm/servername/ directory. Well, disk0 will be the guest HDD to install to, disk1 will be the ISO with drivers, and which disk will be the ISO attached by the "vm install" command? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Jason Tubnor wrote: > > > > > Well, disk0 will be the guest HDD to install to, disk1 will be the ISO with > > drivers, and which disk will be the ISO attached by the "vm install" > > command? > > > > > From a user perspective, this is not of any value to know. vm-bhyve > manages that in the background and usually puts the regular disk0 as -s 4:0 > regardless of normal boot or install and when the installer is invoked, > puts the installer iso in as -s 3:0 Oh, if vm-bhyve manages that the installer ISO is always attached before other ancillary ISOs, that's fine with me. > > So based on previous examples, you'd see something like this in the log: > -s 3:0,ahci-cd,/vm/.iso/myinstaliso.iso -s > 4:0,ahci-hd,/dev/zvol/mypool/vm/myserver/disk0 -s > 5:0,ahci-cd,/vm/myserver/virtio-win-0.1.164.iso I see. I have tried this and it works as you describe, which is good news. However, there is bad news too. I've tried to install Windows 10 and Windows Server 2012R2 onto the virtio-blk device, but... The guest installer says that there are no disks to install to and no drivers, and suggests a driver to be loaded. It can be told to find the appropriate driver in my virtio-win-0.1.141.iso CD: http://admin.sibptus.ru/~vas/bhyve1.png but when it begins loading the viostor driver, the VM crashes with "bhyve exited with status 134" Therefore I ask again if someone has an actual success story of running a Windows guest on a virtio-blk device. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Jason Tubnor wrote: > > > > > Well, disk0 will be the guest HDD to install to, disk1 will be the ISO with > > drivers, and which disk will be the ISO attached by the "vm install" > > command? > > > > > From a user perspective, this is not of any value to know. vm-bhyve > manages that in the background and usually puts the regular disk0 as -s 4:0 > regardless of normal boot or install and when the installer is invoked, > puts the installer iso in as -s 3:0 Oh, if vm-bhyve manages that the installer ISO is always attached before other ancillary ISOs, that's fine with me. > > So based on previous examples, you'd see something like this in the log: > -s 3:0,ahci-cd,/vm/.iso/myinstaliso.iso -s > 4:0,ahci-hd,/dev/zvol/mypool/vm/myserver/disk0 -s > 5:0,ahci-cd,/vm/myserver/virtio-win-0.1.164.iso I see. I have tried this and it works as you describe, which is good news. However, there is bad news too. I've tried to install Windows 10 and Windows Server 2012R2 onto the virtio-blk device, but... The guest installer says that there are no disks to install to and no drivers, and suggests a driver to be loaded. It can be told to find the appropriate driver in my virtio-win-0.1.141.iso CD: http://admin.sibptus.ru/~vas/bhyve1.png but when it begins loading the viostor driver, the VM crashes with "bhyve exited with status 134" Therefore I ask again if someone has an actual success story of running a Windows guest on a virtio-blk device. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: NVMe and Bhyve
Dave Cottlehuber wrote: > On Sun, 24 Feb 2019, at 21:43, Victor Sudakov wrote: > > The guest installer says that there are no disks to install to and no > > drivers, and suggests a driver to be loaded. It can be told to find the > > appropriate driver in my virtio-win-0.1.141.iso CD: > > > > http://admin.sibptus.ru/~vas/bhyve1.png > > > > but when it begins loading the viostor driver, the VM crashes with > > "bhyve exited with status 134" > > > > Therefore I ask again if someone has an actual success story of running > > a Windows guest on a virtio-blk device. > > Hi Viktor > > Looking through my notes I'm not using virtio-blk I think, but I had > no issues with the specific driver version linked to below > https://hackmd.io/s/rJvJuE-CW# albeit with win7 Hope that’s helpful. Not so much unfortunately. I've tried virtio-win-0.1.96.iso from your article. Windows 2012R2 server crashes on it, and Windows10 drivers are missing from it altogether. I've been using virtio-win-0.1.141.iso, which seems younger than yours, it contains Windows10 drivers, but it still crashes both Windows 2012R2 server and Windows10 guests. There is virtio-win-0.1.164.iso available at https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.164-1/ I'll try that and report. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: NVMe and Bhyve
Victor Sudakov wrote: > > > Therefore I ask again if someone has an actual success story of running > > > a Windows guest on a virtio-blk device. > > > > Hi Viktor > > > > Looking through my notes I'm not using virtio-blk I think, but I had > > no issues with the specific driver version linked to below > > https://hackmd.io/s/rJvJuE-CW# albeit with win7 Hope that’s helpful. > > Not so much unfortunately. I've tried virtio-win-0.1.96.iso from your > article. Windows 2012R2 server crashes on it, and Windows10 drivers are > missing from it altogether. > > I've been using virtio-win-0.1.141.iso, which seems younger than yours, > it contains Windows10 drivers, but it still crashes both Windows 2012R2 > server and Windows10 guests. > > There is virtio-win-0.1.164.iso available at > https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.164-1/ > I'll try that and report. virtio-win-0.1.164.iso still crashes the guest with Windows 2012R2 server inside. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Windows 2019 server
Florian Smeets via freebsd-virtualization wrote: > On 16.02.19 13:38, The Doctor via freebsd-virtualization wrote: > > Anyone got Windows 2019 server running in bhyve ? > > > > Yes, on stable/12 (r343339) using > https://github.com/churchers/vm-bhyve/wiki/Running-Window. Worked like a > charm. Which kind of storage did you present to the Windows 2019 guest, paravirtualized or ahcd-hd ? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Windows 2019 server
Jason Tubnor wrote: > > > Which kind of storage did you present to the Windows 2019 guest, > > paravirtualized or ahcd-hd ? > > ahci-hd is what you want with bhyve and uefi Still no paravirtualized disks support for Windows guests, even in FreeBSD 12? How sad. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Dear Colleagues, I have a FreeBSD 11.2 system with 32G RAM which I'm going to use as a bhyve host with zvols and sparse zvols, with (almost) no services/daemons of its own. Could you please clarify some points for me? 1. Does ARC actually cache zfs volumes (not files/datasets)? 2. If ARC does cache volumes, does this cache make sense on a hypervisor, because guest OSes will probably have their own disk cache anyway. 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of total RAM, so that most RAM is available to guest machines? 4. What other zfs tuning measures can you suggest for a bhyve hypervisor? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Patrick M. Hausen wrote: > > > 1. Does ARC actually cache zfs volumes (not files/datasets)? > > Yes it does. > > > 2. If ARC does cache volumes, does this cache make sense on a hypervisor, > > because guest OSes will probably have their own disk cache anyway. > > IMHO not much, because the guest OS is relying on the fact that when > it writes it’s own cached data out to „disk“, it will be committed to > stable storage. This is an important point. > > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of > > total RAM, so that most RAM is available to guest machines? > > Yes if you build your own solution on plain FreeBSD. No if you are running > FreeNAS which already tries to autotune the ARC size according to the > memory committed to VMs. > > > 4. What other zfs tuning measures can you suggest for a bhyve > > hypervisor? > > e.g. > zfs set sync=always zfs/vm > > if zfs/vm is the dataset under which you create the ZVOLs for your emulated > disks. Well, bhyve already has an option for this: The block-device-options are: nocache Open the file with O_DIRECT. directOpen the file using O_SYNC. roForce the file to be opened read-only. I think something like "-s 4:0,virtio-blk,/dev/zvol/zroot/vm/mail/disk0,direct" would do the same? > > I’m using this for all my VM „disks“ and have added a 16 GB SLOG device > to my spinning disk pool - seems to work great. This is on a home system. Is SLOG also used by zfs volumes? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Matt Churchyard wrote: > > > > > 1. Does ARC actually cache zfs volumes (not files/datasets)? > > > > Yes it does. > > I find this distinction between volumes/files/etc and what is cached > causes confusion (as well as "volumes not datasets"). > > Both ZVOLs and Z file systems are types of dataset. A dataset stores data > in records (usually up to 128kb in size). It's these records that are > cached (and that most ZFS functions such as compression/raidz/zil/etc > work with) > > As far as the ZFS lower levels are concerned, there is no difference > between a volume and a file system. Thank you Matt, this was very instructive. > > > 2. If ARC does cache volumes, does this cache make sense on a > > > hypervisor, because guest OSes will probably have their own disk cache > > > anyway. > > > > IMHO not much, because the guest OS is relying on the fact that when > > it writes it’s own cached data out to „disk“, it will be committed to > > stable storage. > > Maybe I've missed something but I don't quite get the link between > read cache (ARC) and guest writes here? Maybe there was a confusion between read and write caches, but my question still stands: Does it make sense to cache the same data (for reading too) twice: one time in host's RAM (ZFS ARC) and the other time in guest's RAM (whatever fs the guest uses, all modern OSes have disk caches)? What do VMWare or VirtualBox do for this situation? Do they ever cache their volumes in the hypervisor's RAM? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
running FreePBX SNG7 Official Distro
Dear Colleagues, Has anyone tried to run FreePBX under bhyve? That's what I get trying to start the vm after a successful automatic install from the ISO image: Boot Failed. EFI DVD/CDROM Failed to set MokListRT: Invalid Parameter Failed to open \EFI\BOOT\grubx64.efi - Not Found Failed to load image \EFI\BOOT\grubx64.efi: Not Found start_image() returned Not Found Boot Failed. EFI Misc Device . The vm config: uefi="yes" cpu=1 memory=2G network0_type="virtio-net" network0_switch="main" disk0_type="virtio-blk" disk0_name="disk0" disk0_dev="zvol" graphics="yes" graphics_wait="no" graphics_res="1280x720" graphics_port="5909" graphics_listen="192.168.4.1" xhci_mouse="yes" uuid="4c1871cb-56f1-11e9-bdbf-5404a6b49a66" network0_mac="58:9c:fc:0b:85:76" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Victor Sudakov wrote: > > Has anyone tried to run FreePBX under bhyve? That's what I get trying to > start the vm after a successful automatic install from the ISO image: > > Boot Failed. EFI DVD/CDROM > Failed to set MokListRT: Invalid Parameter > Failed to open \EFI\BOOT\grubx64.efi - Not Found > Failed to load image \EFI\BOOT\grubx64.efi: Not Found > start_image() returned Not Found > Boot Failed. EFI Misc Device Below are the partitions the automatic installer has created (looking at them from another vm): root@mfsbsd:~ # gpart show vtbd1 => 34 41942973 vtbd1 GPT (20G) 34 2014 - free - (1.0M) 2048186368 1 efi (91M) 188416 4096000 2 ms-basic-data (2.0G) 4284416 37654528 3 linux-lvm (18G) 41938944 4063 - free - (2.0M) If I "mount_msdosfs /dev/vtbd1p1 /mnt/" I see that grubx64.efi is not where bhyve expects to find it: root@mfsbsd:~ # find /mnt/ -name grubx64.efi /mnt/EFI/centos/grubx64.efi Who is to blame, bhyve or FreePBX's installer? How can I tell bhyve's UEFI loader to look for grubx64.efi in a different place? Or look for a different loader? Who says that the image to load should be "\EFI\BOOT\grubx64.efi" and not "\EFI\BOOT\BOOTX64.EFI" for example? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: [dd] > > > > root@mfsbsd:~ # find /mnt/ -name grubx64.efi > > /mnt/EFI/centos/grubx64.efi > > > > Who is to blame, bhyve or FreePBX's installer? > > > > How can I tell bhyve's UEFI loader to look for grubx64.efi in a > > different place? Or look for a different loader? > > > > Who says that the image to load should be "\EFI\BOOT\grubx64.efi" and > > not "\EFI\BOOT\BOOTX64.EFI" for example? > > I can not quickly answer that, but lets try the short quick fix > and simply copy this file to the right place and see if that > gets you up and running. Yes, copying grubx64.efi to "\EFI\BOOT\" does get the guest up and running (I used mfsbsd from a different VM to manipulate the EFI partition). Moreover, I waited (for a long time!) for the EFI interactive shell prompt and with a few commands: Shell> fs0 FS0:\> cd \EFI\centos FS0:\EFI\centos\> grubx64.efi I also managed to boot the guest OS all right. But naturally, the latter fix worked till next reboot only, I don't know how to save the new EFI setup in the guest's configuration. The hardware UFI BIOSes I've seen so far (not many, I must admit) permitted me to save which efi binary I would prefer to boot next time. > That would also tell us that we have > what is actually a common efi system failure problem in that > stuff looks in the wrong place. It seems so. > I have read many an install > instruction that just says copy this file to these too places > as some bioses look for it in one place and others look for it > someplace else. I would very much appreciate a link to some such instruction about uefi-edk2-bhyve: namely how and where it looks for what on boot, and if I can create a menu for example, or change its startup procedure. I can guess that it looks for a FAT16 partition in the GPT with the type "efi" but the rest is a mystery for me. Why is it trying to find "grubx64.efi" and not the default "boot64.efi" (which is present), for example? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: > > > > [dd] > > > > > > > > > > root@mfsbsd:~ # find /mnt/ -name grubx64.efi > > > > /mnt/EFI/centos/grubx64.efi > > > > > > > > Who is to blame, bhyve or FreePBX's installer? > > > > > > > > How can I tell bhyve's UEFI loader to look for grubx64.efi in a > > > > different place? Or look for a different loader? > > > > > > > > Who says that the image to load should be "\EFI\BOOT\grubx64.efi" and > > > > not "\EFI\BOOT\BOOTX64.EFI" for example? > > > > > > I can not quickly answer that, but lets try the short quick fix > > > and simply copy this file to the right place and see if that > > > gets you up and running. > > > > Yes, copying grubx64.efi to "\EFI\BOOT\" does get the guest up and > > running (I used mfsbsd from a different VM to manipulate the EFI > > partition). > > You can usually use the host by doing mdconfig -f Unfortunately mdconfig does not work with zvols: root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0 mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file > > Moreover, I waited (for a long time!) for the EFI interactive shell > > prompt and with a few commands: > > Yes, the timeout is very long, and I do not know that we > document anyplace that if you wait long enough at a failed > boot you do get a EFI shell prompt eventually. Can I press some key to escape to the EFI shell? > > Shell> fs0 > > FS0:\> cd \EFI\centos > > FS0:\EFI\centos\> grubx64.efi > > > > I also managed to boot the guest OS all right. > > > > But naturally, the latter fix worked till next reboot only, I don't know > > how to save the new EFI setup in the guest's configuration. > > My recommedation at this time would be to simply copy grubx64.efi > to the right place and leave it there so that it just boots without > any other change. That's what I have done for now. > > > > The hardware UFI BIOSes I've seen so far (not many, I must admit) > > permitted me to save which efi binary I would prefer to boot next time. > > That is done with an efivar, as it stands right now bhyve efi has > no persistant variable storage, a feature that needs to be implemented. I see. [dd] > > > I can guess that it looks for a FAT16 partition in the GPT with the type > > "efi" but the rest is a mystery for me. Why is it trying to find > > "grubx64.efi" and not the default "boot64.efi" (which is present), for > > example? > > I suspect that what ever guest you installed installed something > else someplace, either within the eft partition, or possibly in > the MBR? Do you mean to say, the guest installing something else someplace can influence the boot sequence of bhyve efi? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: > > > > > > You can usually use the host by doing mdconfig -f > > > > Unfortunately mdconfig does not work with zvols: > > > > root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0 > > mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file > > If its a zvol cant you just do > gpart show /dev/zvol/d02/vm/freepbx/disk0 > > and > mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 No I can't if the zvol is in the "volmode=dev" mode which is the default. This is the default for a reason: it's no good exposing scores of always coming and going guest geoms to the host system. I think you can even get a conflict of labels or something like that one day. > > > > Moreover, I waited (for a long time!) for the EFI interactive shell > > > > prompt and with a few commands: > > > > > > Yes, the timeout is very long, and I do not know that we > > > document anyplace that if you wait long enough at a failed > > > boot you do get a EFI shell prompt eventually. > > > > Can I press some key to escape to the EFI shell? > Not that I am aware of. It's a major problem! There must be a well-known way to break the boot sequence any time and enter the EFI shell. > > > > I can guess that it looks for a FAT16 partition in the GPT with the type > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > "grubx64.efi" and not the default "boot64.efi" (which is present), for > > > > example? > > > > > > I suspect that what ever guest you installed installed something > > > else someplace, either within the eft partition, or possibly in > > > the MBR? > > > > Do you mean to say, the guest installing something else someplace can > > influence the boot sequence of bhyve efi? > > The guest created all of the bits on that zvol, > it can influence many things. There is probably a tiny initial > stub that efi loads that has this bath to grubx64.efi codded in > it and that is what is causing this issue. It is very important to find and debug it because Oracle VirtualBox in UEFI mode installs and runs this guest just fine. So it must be some issue in bhyve itself. Here is the complete archive of everything the guest created in the EFI partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz can you find those confusing bits? The standard procedure should be as follows: Automated detection relies on standardized file paths to the OS loader, with the path varying depending on the computer architecture. The format of the file path is defined as /EFI/BOOT/BOOT.EFI; for example, the file path to the OS loader on an x86-64 system is /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture. Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Victor Sudakov wrote:
> > > > > I can guess that it looks for a FAT16 partition in the GPT with the
> > > > > type
> > > > > "efi" but the rest is a mystery for me. Why is it trying to find
> > > > > "grubx64.efi" and not the default "boot64.efi" (which is present), for
> > > > > example?
> > > >
> > > > I suspect that what ever guest you installed installed something
> > > > else someplace, either within the eft partition, or possibly in
> > > > the MBR?
> > >
> > > Do you mean to say, the guest installing something else someplace can
> > > influence the boot sequence of bhyve efi?
> >
> > The guest created all of the bits on that zvol,
> > it can influence many things. There is probably a tiny initial
> > stub that efi loads that has this bath to grubx64.efi codded in
> > it and that is what is causing this issue.
>
> It is very important to find and debug it because Oracle VirtualBox in
> UEFI mode installs and runs this guest just fine. So it must be some
> issue in bhyve itself.
>
> Here is the complete archive of everything the guest created in the EFI
> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
> can you find those confusing bits?
I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but
BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some
kind of chain loader.
Watch the interactive session below. It does not however mean that there is
nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and runs this
guest just fine.
FS0:\> cd EFI
FS0:\EFI\> ls
Directory of: FS0:\EFI\
04/04/2019 15:53 2,048 .
04/04/2019 15:53 0 ..
04/04/2019 16:26 2,048 centos
04/06/2019 04:19 2,048 BOOT
0 File(s) 0 bytes
4 Dir(s)
FS0:\EFI\> cd BOOT
FS0:\EFI\BOOT\> ls
Directory of: FS0:\EFI\BOOT\
04/04/2019 16:18 2,048 .
04/04/2019 16:18 2,048 ..
08/31/2017 21:30 1,296,176 BOOTX64.EFI
08/31/2017 21:30 79,048 fbx64.efi
2 File(s) 1,375,224 bytes
2 Dir(s)
FS0:\EFI\BOOT\> BOOTX64.EFI
Failed to set MokListRT: Invalid Parameter
Failed to open \EFI\BOOT\grubx64.efi - Not Found
Failed to load image \EFI\BOOT\grubx64.efi: Not Found
start_image() returned Not Found
FS0:\EFI\BOOT\>
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: > > > > > > > > > > You can usually use the host by doing mdconfig -f > > > > > > > > Unfortunately mdconfig does not work with zvols: > > > > > > > > root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0 > > > > mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file > > > > > > If its a zvol cant you just do > > > gpart show /dev/zvol/d02/vm/freepbx/disk0 > > > > > > and > > > mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 > > > > No I can't if the zvol is in the "volmode=dev" mode which is the default. > > > > This is the default for a reason: it's no good exposing scores of always > > coming and going guest geoms to the host system. I think you can even > > get a conflict of labels or something like that one day. > > So it may take a few more commands but it should be > possible to do this from the host side using host > side tools without having to boot a guest to make > these corrections. I'm not aware of such commands. If anyone knows them please share with us. Moreover, I already asked a similar question in February under the topic "mounting/exporting/importing a zfs volume" and nobody gave a recipe. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: > > > > > > > I can guess that it looks for a FAT16 partition in the GPT with > > > > > > > the type > > > > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > > > > "grubx64.efi" and not the default "boot64.efi" (which is > > > > > > > present), for > > > > > > > example? > > > > > > > > > > > > I suspect that what ever guest you installed installed something > > > > > > else someplace, either within the eft partition, or possibly in > > > > > > the MBR? > > > > > > > > > > Do you mean to say, the guest installing something else someplace can > > > > > influence the boot sequence of bhyve efi? > > > > > > > > The guest created all of the bits on that zvol, > > > > it can influence many things. There is probably a tiny initial > > > > stub that efi loads that has this bath to grubx64.efi codded in > > > > it and that is what is causing this issue. > > > > > > It is very important to find and debug it because Oracle VirtualBox in > > > UEFI mode installs and runs this guest just fine. So it must be some > > > issue in bhyve itself. > > > > > > Here is the complete archive of everything the guest created in the EFI > > > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > > > can you find those confusing bits? > > > > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but > > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some > > kind of chain loader. > > And it brobably tries to read a efivariable, and if that variable > is not set it defaults to grubx64.efi. This bootx64.efi is something > the guest installed into the EFI partition, hence my assertion that > the issue is with something the guest installed is some what valid. Do you think the guest OS installer set some efi variable during the installation process, which bhyve did not save? That would explain a lot. > > > > > > Moreover, I waited (for a long time!) for the EFI interactive shell > > > > > > prompt and with a few commands: > > > > > > > > > > Yes, the timeout is very long, and I do not know that we > > > > > document anyplace that if you wait long enough at a failed > > > > > boot you do get a EFI shell prompt eventually. > > > > > > > > Can I press some key to escape to the EFI shell? > > > Not that I am aware of. > > > > It's a major problem! There must be a well-known way to break the boot > > sequence any time and enter the EFI shell. > > Agreed, hopefully those working on edk2 take note and either > chime in with what that way is, or create a bug and track > so that someone may fix this issue. Would it be useful to create a PR in the FreeBSD bugtracker with a feature request? > > > > It is very important to find and debug it because Oracle VirtualBox in > > UEFI mode installs and runs this guest just fine. So it must be some > > issue in bhyve itself. > > As I stated earlier bhyve is missing percistant efi variables, > and that is most likely the reason that VirtualBox just works > and bhyve does not. > > Probably you well find in your VirtualBox directory a > file that is used to store efivars, that is where the I'll look into the VirtualBox directory tomorrow and report here. I was under the impression that efivars are stored in a configuration file in the EFI partition but I was probably wrong, they are kept in NVRAM somewhere, like BIOS settings, and not on a disk. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Victor Sudakov wrote: > > > > As I stated earlier bhyve is missing percistant efi variables, > > and that is most likely the reason that VirtualBox just works > > and bhyve does not. > > > > Probably you well find in your VirtualBox directory a > > file that is used to store efivars, that is where the > > I'll look into the VirtualBox directory tomorrow and report here. I searched through my disk and was unable to find a persistant efivars storage in my VirtualBox 6.0 installation. A Google search reveals some articles (rather dated I must admit) stating that VirtualBox does not support NVRAM emulation for storing efi variables: https://www.virtualbox.org/ticket/14279 https://forums.virtualbox.org/viewtopic.php?t=61970 they recommend using startup.nsh instead. I wonder if bhyve's efi implementation supports startup.nsh. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Rodney W. Grimes wrote: > > > > > > > > As I stated earlier bhyve is missing percistant efi variables, > > > > and that is most likely the reason that VirtualBox just works > > > > and bhyve does not. > > > > > > > > Probably you well find in your VirtualBox directory a > > > > file that is used to store efivars, that is where the > > > > > > I'll look into the VirtualBox directory tomorrow and report here. > > > > I searched through my disk and was unable to find a persistant efivars > > storage in my VirtualBox 6.0 installation. > > > > A Google search reveals some articles (rather dated I must admit) > > stating that VirtualBox does not support NVRAM emulation for storing efi > > variables: > > > > https://www.virtualbox.org/ticket/14279 > > https://forums.virtualbox.org/viewtopic.php?t=61970 > > My quick search turns up: > https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Devices/EFI/DevEFI.cpp My quick search turns up that it's there in the code but it is not used, or it is not enabled by default, whatever (it's written in https://forums.virtualbox.org/viewtopic.php?p=402022&sid=02333b87a8a2bba99383449fc08ca317#p402022 ) > > aka source code thet implements efivars stored in nvram. Anyway, I could not find a file in my virtualbox directory which would look like a storage with NVRAM data. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Victor Sudakov wrote: > > > > I'll look into the VirtualBox directory tomorrow and report here. > > > > > > I searched through my disk and was unable to find a persistant efivars > > > storage in my VirtualBox 6.0 installation. > > > > > > A Google search reveals some articles (rather dated I must admit) > > > stating that VirtualBox does not support NVRAM emulation for storing efi > > > variables: > > > > > > https://www.virtualbox.org/ticket/14279 > > > https://forums.virtualbox.org/viewtopic.php?t=61970 > > > > My quick search turns up: > > https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Devices/EFI/DevEFI.cpp > > My quick search turns up that it's there in the code but it is not > used, or it is not enabled by default, whatever (it's written in > https://forums.virtualbox.org/viewtopic.php?p=402022&sid=02333b87a8a2bba99383449fc08ca317#p402022 > ) > A fairly recent note: https://www.centos.org/forums/viewtopic.php?p=278745&sid=e52bd17e42d2530f496749054b0cc174#p278745 -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Windows 2019 server
The Doctor via freebsd-virtualization wrote: > Seems to be running slow on FreeBSD 12.0 p3. > > Just wondering if there are known issues. I've just installed "Windows Server 2016" on 11.2-RELEASE-p9 and it is very slow. Windows 2012 R2 ran much faster AFAIR. I don't think this is the ahci-hd issue because the Novabench Windows benchmark in the guest VM shows read and write speeds over 1 GB/s, isn't it more than enough? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Windows 2019 server
Victor Sudakov wrote: > > I've just installed "Windows Server 2016" on 11.2-RELEASE-p9 and it is very > slow. Windows 2012 R2 ran much faster AFAIR. > > I don't think this is the ahci-hd issue because the Novabench Windows > benchmark in the guest VM shows read and write speeds over 1 GB/s, isn't > it more than enough? And an attempt to install Windows Server 2016 to a virtio-blk device (guest driver taken from virtio-win-0.1.141.iso) still craches the guest as soon as the guest tries to access the disk: "bhyve exited with status 134". -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
zvol vs sparse-zvol vs file
Dear Colleagues, Is there any difference in guest performance between zvol, sparse-zvol and file backends? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
[vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Dear Colleages, I've noticed that it takes too long for Windows 2012 and 2016 servers guests to shutdown when I issue "vm stopall," several minutes even. Does vm-bhyve provide a way to power them off ungracefully if they would not stop within a predefined time? Or is there perhaps a way to tune the guest OSes to react more promptly to an ACPI shutdown? My primary concern is that my UPS and "apcupsd --kill-on-powerfail" give me about 30 seconds of grace time before the power supply is cut out. It is not sufficient for the bhyve box to shutdown shutdown properly because it waits for the guests to shutdown. If you know of a better way to configure apcupsd so that it powers down the UPS only after all the guest OSes and other daemons are safely down, I'd appreciate that too. For some reason, apctest does not allow me to change the grace period, perhaps this model does not support it. And it would mean a race condition anyway. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Jason Tubnor wrote:
> On Sun., 21 Apr. 2019, 2:51 pm Victor Sudakov, wrote:
>
> >
> >
> > Does vm-bhyve provide a way to power them off ungracefully if they would
> > not stop within a predefined time?
> >
>
> You'd have to do your own checking but to force an instant power off of a
> guest simply execute:
>
> vm poweroff {guest}
I don't need an *instant* poweroff, I need a poweroff only if a VM would
not stop gracefully within a predefined time.
Of course this should not be done manually but from the rc.d/vm script
on shutdown.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Rodney W. Grimes wrote:
> -- Start of PGP signed section.
> > Jason Tubnor wrote:
> > > On Sun., 21 Apr. 2019, 2:51 pm Victor Sudakov,
> > > wrote:
> > >
> > > >
> > > >
> > > > Does vm-bhyve provide a way to power them off ungracefully if they would
> > > > not stop within a predefined time?
> > > >
> > >
> > > You'd have to do your own checking but to force an instant power off of a
> > > guest simply execute:
> > >
> > > vm poweroff {guest}
> >
> > I don't need an *instant* poweroff, I need a poweroff only if a VM would
> > not stop gracefully within a predefined time.
> >
> > Of course this should not be done manually but from the rc.d/vm script
> > on shutdown.
>
> vm shutdown {guest} &
> (sleep N && vm poweroff {guest}) &
>
> A bit hackish, but that should work, if the guest has
> shutdown by the time you get to the poweroff an error
> might occur. This spawns a few processes, but is
> non blocking and highly parallel.
Thank you, I might give it a try though it would involve hacking the
rc.d/vm script.
>
> One thing I noted, you said rc.d/vm, so I am assuming your actually
> using the vm-bhyve port,
Yes, I thought that was pretty obvious from the Subject line.
> in which case it has a sequential shutdown
> with time delays between each guest. vm_delay specifically, and it
> shuts vm's down in reverse order of the startup.
>
> That is more likely your problem in that your sending these acpi
> shutdown requests one at a time, and they should be broadcast in
> the "power going out" case.
Whence is the idea that "vm stopall" does a sequential shutdown? What sense
would that make?
A sequential startup does make sense but a sequential shutdown? Useless
I think. The man page says that
stopall
Stop all running virtual machines. This sends a stop command to
all bhyve(8) instances, regardless of whether they were starting
using vm or not.
>
> It may be possile to adjust vm_delay to 0 and have that be better,
> though I have not locked at the code. You may also wish to discuss
> the issue with the vm-bhyve maintainer and maybe a "lights out"
> procedure needs to be added.
Even if there is a single Windows 2012 or 2016 guest running, it takes
the vm several minutes to transition to the "stopped" state.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: running FreePBX SNG7 Official Distro
Subbsd wrote: > On Sun, Apr 7, 2019 at 11:15 AM Victor Sudakov wrote: > > > I'll look into the VirtualBox directory tomorrow and report here. I was > > under the impression that efivars are stored in a configuration file in > > the EFI partition but I was probably wrong, they are kept in NVRAM > > somewhere, like BIOS settings, and not on a disk. > > I just wanted to confirm that this is indeed an EVIVARS problem, > because through CBSD (which uses Refind [1]) FreePBX 1805-2 > boots without any problems. Therefore, as Rodney said, you can try > using a third-party EFI boot manager. > And just wait until someone finds the time to add support for UEFI > VARS in bhyve. > > [1] - http://www.rodsbooks.com/refind/ Kudos to the FreeBSD team! I've just discovered that when you install FreeBSD in UEFI mode, the installer creates a /efi/boot/startup.nsh on the efi partition. So FreeBSD would boot without any problems in VirtualBox with EFI, no need for persistent efi variables. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Rodney W. Grimes wrote:
[dd]
> > > That is more likely your problem in that your sending these acpi
> > > shutdown requests one at a time, and they should be broadcast in
> > > the "power going out" case.
> >
> > Whence is the idea that "vm stopall" does a sequential shutdown? What sense
> > would that make?
>
> Well I am not sure that vm-bhyve does this, but esxi certainly does,
> and it is a royal PITA to deal with sometimes. It does make sense
> in some aspects, do not want the database server going offline before
> all the clients go down do we? Kinda makes for a bunch of nonsense
> errors logged due to missing server. I kinda like my virtual routers
> and firewalls to keep going tell the end too.
>
> This is all YMMV situations though.
OK, you have convinced me, a predictable sequential shutdown may make
sense sometimes. Anyway, it's not there im vm-bhyve so it's not the
reason for the slow shutdown.
>
> > A sequential startup does make sense but a sequential shutdown? Useless
> > I think. The man page says that
>
> For you perhaps useless, but that rarely rules out that there may be
> a totally valid and usefull case.
>
> > stopall
> > Stop all running virtual machines. This sends a stop command
> > to
> > all bhyve(8) instances, regardless of whether they were
> > starting
> > using vm or not.
>
> And the implementation is pretty brutal:
> # 'vm stopall'
> # stop all bhyve instances
> # note this will also stop instances not started by vm-bhyve
> #
> core::stopall(){
> local _pids=$(pgrep -f 'bhyve:')
>
> echo "Shutting down all bhyve virtual machines"
> killall bhyve
> sleep 1
> killall bhyve
> wait_for_pids ${_pids}
> }
>
> I wonder what the effect of the second kill is,
> that seems odd.
Indeed.
> Almost like you might cause more
> issues than you solve as now you already have a
> vm in what should be ACPI shutdown process.
>
> Also this killall operation probably puts a high stress
> on disk i/o as you just woke up and made all the vm's
> get busy all at once and your going to basically thrash
> on every single resource they are using (yet another reason
> you may actually want to serialize these shutdown operations.)
You are right.
>
> IIRC windows, especially newer ones, do a boat load of work
> on a shutdown unless they are told to shutdown quickly. Ie
> they even try to apply pending updates and such, that could
> be part of why you see windows vm's lagging around.
Do you know how to configure Windows for an unconditional ACPI shutdown?
Last time I stopped my Windows guests, they stopped pretty quickly. But
sometimes it just takes them forever to stop. Maybe it was giving a
shutdown warning to the users or something. Or maybe it was this issue:
https://serverfault.com/questions/871792/acpi-shutdown-does-not-always-work-on-a-windows-server-virtual-machine
> You may also want to: Disable Clear page file on shutdown
> that is a windows thing, if you have a huge page file that
> can do a LOT of io, if you have a few windows vm's on the
> same spindle and try to stop them all at once your going to
> trash that disk for much longer than you need.
Last time I checked on the Windows 2012 and 2016 servers, the
ClearPageFileAtShutdown setting was 0x0. I think it is the default.
> > > It may be possile to adjust vm_delay to 0 and have that be better,
> > > though I have not locked at the code. You may also wish to discuss
> > > the issue with the vm-bhyve maintainer and maybe a "lights out"
> > > procedure needs to be added.
What is needed in vm-bhyve is the feature that if ACPI does not stop the
guest for a predefined period of time, the guest is powered off.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Victor Sudakov wrote: [dd] > > > > It may be possile to adjust vm_delay to 0 and have that be better, > > > > though I have not locked at the code. You may also wish to discuss > > > > the issue with the vm-bhyve maintainer and maybe a "lights out" > > > > procedure needs to be added. > > What is needed in vm-bhyve is the feature that if ACPI does not stop the > guest for a predefined period of time, the guest is powered off. I've created a feature request https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237479 please support me for adding yourself to the CC list, or commenting. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Paul Vixie wrote:
>
> Victor Sudakov wrote on 2019-04-22 19:43:
> ...
> >> And the implementation is pretty brutal:
> >> # 'vm stopall'
> >> # stop all bhyve instances
> >> # note this will also stop instances not started by vm-bhyve
> >> #
> >> core::stopall(){
> >> local _pids=$(pgrep -f 'bhyve:')
> >>
> >> echo "Shutting down all bhyve virtual machines"
> >> killall bhyve
> >> sleep 1
> >> killall bhyve
> >> wait_for_pids ${_pids}
> >> }
>
> yow.
To be sure, I was unable to find the above code (as is) in
/usr/local/lib/vm-bhyve/vm-* (the vm-bhyve port 1.3.0). It may be that
something more intelligent is happening in a more recent version, like a
sequential shutdown. However, "kill $pid; sleep 1; kill $pid" seems to
be still present.
>
> >>
> >> I wonder what the effect of the second kill is,
> >> that seems odd.
> >
> > Indeed.
>
> the first killall will cause each client OS to see a soft shutdown
> signal. the sleep 1 gives them some time to flush their buffers. the
> second killall says, time's up, just stop.
>
> i think this is worse than brutal, it's wrong. consider freebsd's own
> work flow when trying to comply with the first soft shutdown it got:
>
> https://github.com/freebsd/freebsd/blob/master/sbin/reboot/reboot.c#L220
>
> this has bitten me more than once, because using "pageins" as a proxy
> for "my server processes are busy trying to synchronize their user mode
> state" is inaccurate. i think _any_ continuing I/O should be reason to
> wait the full 60 seconds.
Would it be beneficial to just hack /usr/local/lib/vm-bhyve/vm-* ?
>
> and so i think the "sleep 1" above should be a "sleep 65".
>
> > What is needed in vm-bhyve is the feature that if ACPI does not stop the
> > guest for a predefined period of time, the guest is powered off.
>
> i agree with this.
Will you please support the bug report:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237479
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Victor Sudakov wrote:
>
> I've noticed that it takes too long for Windows 2012 and 2016 servers
> guests to shutdown when I issue "vm stopall," several minutes even.
>
> Does vm-bhyve provide a way to power them off ungracefully if they would
> not stop within a predefined time?
>
> Or is there perhaps a way to tune the guest OSes to react more promptly
> to an ACPI shutdown?
>
> My primary concern is that my UPS and "apcupsd --kill-on-powerfail" give
> me about 30 seconds of grace time before the power supply is cut out. It
> is not sufficient for the bhyve box to shutdown shutdown properly
> because it waits for the guests to shutdown.
For the present I think I've found a workaround not directly related to
vm-bhyve. I'm going to insert "service vm stop" into the doshutdown()
procedure in /usr/local/etc/apcupsd/apccontrol before the actual
${SHUTDOWN}. It should give the VMs sufficient time to shutdown before
the actual /sbin/shutdown is executed.
>
> If you know of a better way to configure apcupsd so that it powers down
> the UPS only after all the guest OSes and other daemons are safely down,
> I'd appreciate that too.
Do you think the above hack would have any negative effects?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Rodney W. Grimes wrote:
> > >
> > > I've noticed that it takes too long for Windows 2012 and 2016 servers
> > > guests to shutdown when I issue "vm stopall," several minutes even.
> > >
> > > Does vm-bhyve provide a way to power them off ungracefully if they would
> > > not stop within a predefined time?
> > >
> > > Or is there perhaps a way to tune the guest OSes to react more promptly
> > > to an ACPI shutdown?
> > >
> > > My primary concern is that my UPS and "apcupsd --kill-on-powerfail" give
> > > me about 30 seconds of grace time before the power supply is cut out. It
> > > is not sufficient for the bhyve box to shutdown shutdown properly
> > > because it waits for the guests to shutdown.
> >
> > For the present I think I've found a workaround not directly related to
> > vm-bhyve. I'm going to insert "service vm stop" into the doshutdown()
> > procedure in /usr/local/etc/apcupsd/apccontrol before the actual
> > ${SHUTDOWN}. It should give the VMs sufficient time to shutdown before
> > the actual /sbin/shutdown is executed.
>
> Make sure it does not do the waitall for pids or your going to
> sit here waiting for that to complete for a long time, ie until
> all vm's go to power off.
In fact, this is my intention to sit here waiting for all VM's to go power off.
I can configure apcupsd to start the doshutdown() routine when there is
enough power in the UPS, like maybe for 20-30 minutes.
> > > If you know of a better way to configure apcupsd so that it powers down
> > > the UPS only after all the guest OSes and other daemons are safely down,
> > > I'd appreciate that too.
> >
> > Do you think the above hack would have any negative effects?
>
> I think this is a reasonable approach given the current situation.
>
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Victor Sudakov wrote:
>
> For the present I think I've found a workaround not directly related to
> vm-bhyve. I'm going to insert "service vm stop" into the doshutdown()
> procedure in /usr/local/etc/apcupsd/apccontrol before the actual
> ${SHUTDOWN}. It should give the VMs sufficient time to shutdown before
> the actual /sbin/shutdown is executed.
The workaround turned out useless: apcupsd sends the "kill power"
command to the UPS before calling the doshutdown() procedure.
F*ck, is there no correct, generally accepted way of powering off a busy
FreeBSD server? Why am I inventing the wheel?
> > If you know of a better way to configure apcupsd so that it powers down
> > the UPS only after all the guest OSes and other daemons are safely down,
> > I'd appreciate that too.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Stephen Stuart wrote: > I call this in /etc/rc.shutdown.local to shut all VMs down gracefully and > wait, perhaps a variant would help you. > > #!/bin/sh > > INTERVAL=5 > /usr/bin/killall -TERM bhyve > echo -n "/dev/vmm:" > while test -d /dev/vmm ; do > echo -n "" $(/bin/ls /dev/vmm | /usr/bin/wc -w) > /bin/sleep $INTERVAL > done > echo " 0" Thanks for reminding me about /etc/rc.shutdown.local. I've tested a solution today and it seems working. 1. Start apcupsd with --term-on-powerfail apcupsd_enable="YES" apcupsd_flags="--term-on-powerfail" 2. Put the following line into /etc/rc.shutdown.local test -f /var/run/powerfail && /usr/local/sbin/apcupsd --killpower Now, when the UPS power is exhausted, apcupsd starts the shutdown procedure and exits. The system shuts down without any haste, as it pleases, and at the very end "apcupsd --killpower" is called and poweroffs the UPS. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
[vm-bhyve] shutting down VMs and kern.init_shutdown_timeout
Dear Colleagues, The shutdown of vm-bhyve (and probably other bhyve-based VM managers) is commanded by "service vm shutdown." From man init(8) I've learned that "if the /etc/rc.shutdown does not terminate within 120 seconds, init will terminate it. The timeout can be configured via the sysctl variable kern.init_shutdown_timeout." Does that mean that any VM is supposed to shut down within 2 minutes in a default FreeBSD host system (in reality probably even less), or else the bhyve process will be mercilessly killed? This is completely unsuitable for Windows guests who may take a considerable longer time to shutdown. Even in my FreeBSD guests I occasionally find the message that "root was not properly dismounted." Can something be done about this? What about CBSD, FreeNAS and other more specialized FreeBSD-based systems, what do they do about VM shutdown? Or I may be assuming incorrectly about the 2 minutes timeout. Please correct me then. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout
Patrick M. Hausen wrote: > > > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > > "if the /etc/rc.shutdown does not terminate within 120 seconds, init will > > terminate it. The timeout can be configured via the sysctl variable > > kern.init_shutdown_timeout.“ > What’s keeping you from increasing that timeout for your > particular requirements? Is that really necessary? Am I correct assuming that any VM is supposed to shut down within 2 minutes or else the bhyve processes will be killed? Or is this kern.init_shutdown_timeout only about the rc.shutdown script itself and not its children? > IMHO 2 minutes are a reasonable > default for a system that does not run hypervisor VMs - > which is probably the vast majority. > > Anyone with the knowledge to implement a virtualized > environment The lack of information about this caveat in https://wiki.freebsd.org/bhyve, man vm-bhyve, https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization-host-bhyve.html and other sources suggests that this particular knowledge may be limited. Other hypervisors (non-bhyve) probably wait for all VMs to stop. > can simply set that to a more suitable value, > can’t he/she? Yes if they know about it. There should be a warning somewhere in the documentation if we want to be serious about FreeBSD as a hypervisor. Or better still, a warning that rcshutdown_timeout (if set) and kern.init_shutdown_timeout can be too low when running vm-bhyve and such is desirable. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout
Adam wrote: > > > Patrick M. Hausen wrote: > > > > > > > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > > > > "if the /etc/rc.shutdown does not terminate within 120 seconds, init > > will > > > > terminate it. The timeout can be configured via the sysctl variable > > > > kern.init_shutdown_timeout.“ > > > > > > > What’s keeping you from increasing that timeout for your > > > particular requirements? > > > > Is that really necessary? Am I correct assuming that any VM is supposed > > to shut down within 2 minutes or else the bhyve processes will be > > killed? Or is this kern.init_shutdown_timeout only about the rc.shutdown > > script itself and not its children? > > > > FreeBSD is tuned as a general purpose server OS. If you find the defaults > don't meet your needs, you can change them. That's why tunables exist. Adam, if you know the answer to my direct question above, please answer it. I also would like to know if there are any adverse effects of increasing kern.init_shutdown_timeout. > > > IMHO 2 minutes are a reasonable > > > default for a system that does not run hypervisor VMs - > > > which is probably the vast majority. > > > > > > Anyone with the knowledge to implement a virtualized > > > environment > > > > The lack of information about this caveat in > > https://wiki.freebsd.org/bhyve, man vm-bhyve, > > > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization-host-bhyve.html > > and other sources suggests that this particular knowledge may be > > limited. Other hypervisors (non-bhyve) probably wait for all VMs to stop. > > > > RH implemented this relatively recently. > > https://bugzilla.redhat.com/show_bug.cgi?id=1334982 Good for them. We need to raise awareness too, so I've created this PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237646 > > > can simply set that to a more suitable value, > > > can’t he/she? > > > > Yes if they know about it. There should be a warning somewhere in the > > documentation if we want to be serious about FreeBSD as a hypervisor. > > > > Or better still, a warning that rcshutdown_timeout (if set) and > > kern.init_shutdown_timeout can be too low when running vm-bhyve and > > such is desirable. > > > > Why don't you simply write a wrapper script which calls shutdown after > shutting down guests? A lot of things can be done to mitigate this problem, as soon as you are aware of it. > I doubt many admins on any hypervisor are > gratuitously calling shutdown on host with important guests running. Well, after I started noticing "WARNING: / was not properly dismounted" in my guest OS's logs, after the perfectly correct shutdown of the host, I stopped calling shutdown on host, and started thinking why this was happening. But not before. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
sysutils/docker-freebsd ?
Dear Colleagues, Do you know if there are plans to revive sysutils/docker-freebsd? It has been "broken" for 2 months, and the FreeBSD Wiki says that docker-freebsd is the recommended way of running docker apps. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: [vm-bhyve] adding existing vm instances to the vm-bhyve system
tech-lists wrote: > > How would one import a freebsd or linux vm currently running on a > freebsd-12 server into vm-bhyve management? I can see lots of > instructions for starting from fresh, but none for import of existing. > Is this possible? > > The guests are currently running as per > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization-host-bhyve.html > in screen. Some guests are file-backed, others zvol-backed. When I migrated a VM from one vm-bhyve installation to another (on a different host), it was sufficient to "zfs send -R" the vm dataset on the old host and "zfs receive" it on the new host under the $vm_dir. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Need X-server to run in bhyve guest
Dear Colleagues, If I were to test different desktop environments (gnome, xfce etc) in a FreeBSD guest in bhyve, what X server (probably with vnc access, or rdp access, or whatever?) would you advise? I actually want to see the *local* graphical desktop of a guest, not just X-clients forwarded from guest to host, or drawing to a remote $DISPLAY on the host. Any advice what to install on the guest and how to start it there? I would probably need to persuade the Gnome/XFCE on the guest that the guest has a monitor and video card... -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Need X-server to run in bhyve guest
Loyd Craft wrote: > I like Xephyr…. It’s a software framebuffer for X11... So it can pretend to be a local X-server (i.e. :0) in a bhyve guest? > But you need a X server and SSH X11 forwarding on your local machine. Xephyr is an X-server and X-client at the same time, correct? Then I can install it in the guest VM and make it output to my FreeBSD desktop. I don't even think SSH X11 forwarding will be required, just setting the $DISPLAY variable appropriately? But first of all, how do you make Gnome or KDE use Xephyr as a local X-server? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Need X-server to run in bhyve guest
Jason Barbier wrote: > honestly, > I just use remmina after enabling the vnc console on the guest, works like a > charm every time This may work for me too. But how do you enable the vnc console on a FreeBSD guest? I boot my FreeBSD guests with bhyveload, which does not have a vnc console. Does your FreeBSD guest see this vnc console as a video card suitable for running a local X-server? What driver? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Need X-server to run in bhyve guest
Henrik Gulbrandsen wrote: > On 2019-12-01 06:35, Victor Sudakov wrote: > > If I were to test different desktop environments (gnome, xfce etc) in a > > FreeBSD guest in bhyve, what X server (probably with vnc access, or rdp > > access, or whatever?) would you advise? > > This will work with the normal X server and bhyve's built-in VNC if > you run the guest with UEFI CSM and apply my Video BIOS patch: > > https://www.gulbra.net/freebsd-bhyve/ I'd prefer to stick to bhyveload for now. > > However, when I tested it with a GhostBSD 19.04 ISO image, the mouse > didn't work - probably because FreeBSD still didn't support the XHCI > tablet needed for absolute pointer coordinates in bhyve. With a PS/2 > mouse you will get two mouse pointers - one each for host and guest. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Need X-server to run in bhyve guest
Rodney W. Grimes wrote: > > Henrik Gulbrandsen wrote: > > > On 2019-12-01 06:35, Victor Sudakov wrote: > > > > If I were to test different desktop environments (gnome, xfce etc) in a > > > > FreeBSD guest in bhyve, what X server (probably with vnc access, or rdp > > > > access, or whatever?) would you advise? > > > > > > This will work with the normal X server and bhyve's built-in VNC if > > > you run the guest with UEFI CSM and apply my Video BIOS patch: > > > > > > https://www.gulbra.net/freebsd-bhyve/ > > > > I'd prefer to stick to bhyveload for now. > > I would strongly encourage you to use vnc and UEFI bios mode > in bhyve for what you are attempting to do. Unless your guess > is not able to run a uefi frame buffer type display, which any > recent FreeBSD should be just fine with. All right, I just tried to install FreeBSD 12.1 under vm-bhyve in UEFI mode. Well, the installation was not trivial. At first, in the VNC client, I see the loader menu, then the "Loading kernel..." message, and then a blank screen. Fine, I run "vm console" and install via the serial console, the UEFI screen still being blank. Second, unless I select ZFS, there is no option to install on GPT+UEFI. Fine, let's install on ZFS. Installation complete, reboot... The serial console is available, the UEFI VNC console is available too, though much later :-) Fine, I will experiment with that. Maybe Gnome or xfce will recognize this console as a monitor. Thank you for the hint, Rodney! -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Need X-server to run in bhyve guest
Victor Sudakov wrote: > > Fine, I will experiment with that. Maybe Gnome or xfce will recognize > this console as a monitor. > > Thank you for the hint, Rodney! Nope, would not run for me in the VNC console (see below). Did you do anything special? /usr/local/bin/startxfce4: Starting X server X.Org X Server 1.18.4 Release Date: 2016-07-19 X Protocol Version 11, Revision 0 Build Operating System: FreeBSD 12.0-RELEASE-p12 amd64 Current Operating System: FreeBSD uefi1.sibptus.ru 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC amd64 Build Date: 14 November 2019 08:46:03AM Current version of pixman: 0.38.4 Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/Xorg.0.log", Time: Mon Dec 2 23:28:28 2019 (==) Using default built-in configuration (30 lines) scfb trace: probe start scfb trace: probe done (EE) Fatal server error: (EE) Cannot run in framebuffer mode. Please specify busIDsfor all framebuffer devices (EE) (EE) Please consult the The X.Org Foundation support at http://wiki.x.org for help. (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information. (EE) (EE) Server terminated with error (1). Closing log file. xinit: giving up xinit: unable to connect to X server: Connection refused xinit: server error -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: Re: Need X-server to run in bhyve guest
Paul Webster wrote: >Use dummy video driver and tightvnc, launch the vnc server as a normal >user then connect with a VNC client from the outside, Connect to what? > or alternatively use the dummy video driver directly Is it useful for testing desktop environments (see my original message)? Does a Gnome/KDE desktop running on a dummy video driver have any value? Please elaborate. > and a remote X11 session over ssh It is not running X-clients over SSH forwarding that I'm looking for. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
