Re: OpenBSD as Xen domU
[Picking up on this old thread] Question for those of you running OpenBSD HVM DomUs, does your IO performance suck? Description: I have OpenBSD 4.3 DomU running HVM mode with 1x vcpu on top on OpenSolaris b97 xVM Dom0, which pins down 2x vcpus (box is a quad-core 1.9GHz opteron with 8GB mem). Like John (below), I also pass "model=ne2k_pci" as part of the vif. The system is reasonably responsive over console when not doing any IO, especially disk, when the performance tanks. To put this in perspective, when I was installing OpenBSD, it took 30+ mins for it to install the packages and thats not including any of the X packages! Likewise, a simple file-copy test gave me these results: Copy large file from Desktop to Dom (using `scp`) : Dom0: ~27136KB/s (opensolaris b97) DomU: ~322KB/s (openbsd 4.3, hvm) Copy large file from Dom to Desktop (using `scp`): Dom0: ~28441KB/s (opensolaris b97) DomU: ~662KB/s (openbsd 4.3, hvm) Finally, running `bonnie++` on the Dom0 in DomU gives: _*Bonnie++ on Dom0*_ Version 1.03d -- Sequential Output ----- Sequential Input -----Random- -Per Char- --Block-- -Rewrite- --Per Char-- --Block----Seeks-- Machine SizeK/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CPK/sec %CP san8G 67051 85 166574 59 126454 63 71058 96 362940 761645 11 -- Sequential Create -- Random Create -Create-- --Read--- -Delete---Create-- --Read--- -Delete-- files K/sec %CP K/sec %CP K/sec %CPK/sec %CP K/sec %CP K/sec %CP 16 15815 99 + +++ 22903 9915222 99 + +++ 22339 99 _*Bonnie++ in zvol-backed DomU (openbsd 4.3, hvm)*_ Version 1.03 -- Sequential Output ----- Sequential Input -----Random- -Per Char- --Block-- -Rewrite- --Per Char-- --Block----Seeks-- Machine SizeK/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CPK/sec %CP puffy 300M 451 0 295 0184 0 976 4 1521 2 26.8 1 -- Sequential Create -- Random Create -Create-- --Read--- -Delete---Create-- --Read--- -Delete-- files K/sec %CP K/sec %CP K/sec %CPK/sec %CP K/sec %CP K/sec %CP 16 8 1 + +++ 16 18 1 + +++ 10 0 I have already posted these results to the xVM mailing list and they said that they have never seen the HVM performance this bad before (where 30Mb/s is the norm) and also offered: If this was just disk or net then I'd suspect that the device emulation simply didn't match up well with the OpenBSD driver, but given that you're seeing both as slow it may well be a broader problem. BTW, my networking performance is also not great - its about ~7 Mb/s - over GE interfaces using Cat5e. This is also slow as running the same `iperf` test when openbsd is on the bare-metal gives ~534 Mb/s. Just the same, 7 Mb/s is way faster than my disk I/O (~0.5 Mb/s avg). So, for those of you running OpenBSD HVM DomUs, does your IO performance suck this bad too? Thanks, Kent John Jackson wrote: > OpenBSD as DomU works using hardware virtualization for me. There's > the occasional lockup that I haven't looked into too much. You can > launch vncviewer to get a console. My working config is at the bottom. > > John > > On Wed, Feb 06, 2008 at 11:55:05PM +0100, Julien Cabillot wrote: > >> It's work but I had really bad performances with the network (timeout on >> the interface re). >> Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html >> >> > > I found that setting the vif interface to 'model=ne2k_pci' helps with > the timeouts. > > >> On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: >> >>> I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. >>> Anyone any success running stable OpenBSD (FreeBSD would also suffice) >>> as domU in a Xen system? If so, willing to share config / how-to / >>> experience? >>> >>> Kind regards, >>> Doichin >>> > > Here's a working Xen config: > = > import os, re > arch = os.uname()[4] > if re.search('64', arch): > arch_libdir = 'lib64' > else: > arch_libdir = 'lib' > kernel = "/usr/lib/xen/boot/hvmloader" > builder='hvm' > memory = 256 > name = "obsd" > pae=0 > vif = [ 'type=ioemu, mac=00:16:3e:7d:be:ef, model=ne2k_pci' ] > disk = [ > 'file:/disk/homer.disk,hda,w','file:/disk/obsd42_amd64.iso,ioemu:hdc:cdrom,r' > ] > device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm' > boot='cd' > sdl=0 > vnc=1 > vncviewer=0 > nographic=0 > stdvga=0 > serial='pty' > ne2000=1 > a
Re: OpenBSD as Xen domU
On 8 Feb 2008, at 14:23, NetOne - Doichin Dokov wrote: Yup, I have successfully compiled the XENU kernel, I neither now which version it is, but it's 8 months old, I believe it was based on -current. The bad thing is that - when I try to run it with Xen, i get this: [EMAIL PROTECTED] xen]# xm create -c /etc/xen/net1-obsd.xm Using config file "/etc/xen/net1-obsd.xm". Error: (22, 'Invalid argument') As far as I went with googling info about this error, it's something PAE related - my Xen dom0 is PAE enabled I think (I'm not very much into linux, but i have HIGHMEM_64G defined, also PAE is present in / proc/cpuinfo). Seems like the OpenBSD XENU kernel is not (or the vice versa - dom0 is non-PAE, and the OpenBSD kernel is PAE-enabled). In FreeBSD, there's a PAE option in the kernel config, but I don't find such one in the OpenBSD configs (I did not have enough time to investigate this, though, so I might have overlooked it). I'm using Xen 3.0.x, and my net1-obsd.xm config is: = name = "net1-obsd" memory = "256" disk = [ 'phy:/dev/mirror/fc1,sda1,w' ] vif = [ 'mac=00:16:3e:1b:04:c9, bridge=xenbr0' ] vnc=1 vncunused=1 vcpus=2 kernel = "/boot/openbsd-xenu" extra = "boot_verbose,boot_single,vfs.root.mountfrom=ufs:/dev/ md0,kern.hz=100" on_reboot = 'restart' on_crash= 'restart' == Maybe it's best to contact Christoph directly, but I don't know if he still works on this project (the ng repository seems dead for the last 8 months), and I don't want to annoy him if not needed, so if anyone has any ideas how to proceed with this, I would be grateful to receive their opinion & knowledge. You can't intermix PAE and non-PAE enabled paravirtual guest kernels and dom0s. Since most Xen distros use PAE by default these days, it just needs support added to the OpenBSD port to let you boot it. Until that happens, you will need to use a non-PAE dom0. -anil
Re: OpenBSD as Xen domU
ropers NAPISA: On 07/02/2008, NetOne - Doichin Dokov <[EMAIL PROTECTED]> wrote: NetOne - Doichin Dokov NAPISA: I'm trying to use Christopher's work, but I get the following errors when i try to make depend a xenU kernel: (...) Anyone any hints? Is it meant to be run on -current (I see the last changes in the hg are from 8 months ago, but...)? Or do I need to compile it under an i386 kernel (I'm currently trying with an amd64 one)? # uname -a OpenBSD border2.net1.cc 4.2 GENERIC.MP#1 amd64 Any help MUCH appreciated! Kind regards, Doichin Guess that counts for cross-compilings, so the answer is yes - i need an i386 kernel to compile this. Sorry for the fuzz... Yes. The port is i386 and sadly it never got much traction within the OpenBSD project, which --this is just my hunch-- may have led Christoph to unfortunately focus on other stuff instead of keeping current a port that the project seems to be lukewarm for at best. I'm not sure what OpenBSD version the currently available revision of the port targets -- possibly 4.0 or 4.1. If you want to go anywhere with this it would probably be best to google Christoph Egger's email address (it's out there). He can give you the real deal straight from the horse's mouth. If you use Christoph's port, I would be very interested in hearing about it. -- Maybe others, too. Maybe you could post to [EMAIL PROTECTED] Thanks and regards, --ropers Yup, I have successfully compiled the XENU kernel, I neither now which version it is, but it's 8 months old, I believe it was based on -current. The bad thing is that - when I try to run it with Xen, i get this: [EMAIL PROTECTED] xen]# xm create -c /etc/xen/net1-obsd.xm Using config file "/etc/xen/net1-obsd.xm". Error: (22, 'Invalid argument') As far as I went with googling info about this error, it's something PAE related - my Xen dom0 is PAE enabled I think (I'm not very much into linux, but i have HIGHMEM_64G defined, also PAE is present in /proc/cpuinfo). Seems like the OpenBSD XENU kernel is not (or the vice versa - dom0 is non-PAE, and the OpenBSD kernel is PAE-enabled). In FreeBSD, there's a PAE option in the kernel config, but I don't find such one in the OpenBSD configs (I did not have enough time to investigate this, though, so I might have overlooked it). I'm using Xen 3.0.x, and my net1-obsd.xm config is: = name = "net1-obsd" memory = "256" disk = [ 'phy:/dev/mirror/fc1,sda1,w' ] vif = [ 'mac=00:16:3e:1b:04:c9, bridge=xenbr0' ] vnc=1 vncunused=1 vcpus=2 kernel = "/boot/openbsd-xenu" extra = "boot_verbose,boot_single,vfs.root.mountfrom=ufs:/dev/md0,kern.hz=100" on_reboot = 'restart' on_crash= 'restart' == Maybe it's best to contact Christoph directly, but I don't know if he still works on this project (the ng repository seems dead for the last 8 months), and I don't want to annoy him if not needed, so if anyone has any ideas how to proceed with this, I would be grateful to receive their opinion & knowledge. Kind regards, Doichin
Re: OpenBSD as Xen domU
NetOne - Doichin Dokov P=P0P?P8QP0: ropers P=P0P?P8QP0: You can use Christoph Egger's OpenBSD/Xen port. No need to go HVM-only. Unfortunately, my own website is down right now and I haven't gotten around to fixing that, but the Wayback Machine has the relevant page: http://web.archive.org/web/20070403174105/http://ropersonline.com/openbsd/xen/ Also, search the misc archives. This question crops up fairly regularly. and each time most people don't seem to know of Christoph Egger's port (and each time I then try to tell people about it again -- if I catch the message, but I don't always do and sometimes things fall through the cracks here). Thanks and regards, --ropers On 07/02/2008, John Jackson <[EMAIL PROTECTED]> wrote: OpenBSD as DomU works using hardware virtualization for me. There's the occasional lockup that I haven't looked into too much. You can launch vncviewer to get a console. My working config is at the bottom. John On Wed, Feb 06, 2008 at 11:55:05PM +0100, Julien Cabillot wrote: It's work but I had really bad performances with the network (timeout on the interface re). Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html I found that setting the vif interface to 'model=ne2k_pci' helps with the timeouts. On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. Anyone any success running stable OpenBSD (FreeBSD would also suffice) as domU in a Xen system? If so, willing to share config / how-to / experience? Kind regards, Doichin Here's a working Xen config: = import os, re arch = os.uname()[4] if re.search('64', arch): arch_libdir = 'lib64' else: arch_libdir = 'lib' kernel = "/usr/lib/xen/boot/hvmloader" builder='hvm' memory = 256 name = "obsd" pae=0 vif = [ 'type=ioemu, mac=00:16:3e:7d:be:ef, model=ne2k_pci' ] disk = [ 'file:/disk/homer.disk,hda,w','file:/disk/obsd42_amd64.iso,ioemu:hdc:cdrom,r' ] device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm' boot='cd' sdl=0 vnc=1 vncviewer=0 nographic=0 stdvga=0 serial='pty' ne2000=1 audio=0 localtime=1 I'm trying to use Christopher's work, but I get the following errors when i try to make depend a xenU kernel: In file included from /usr/src/openbsd-xen-sys.hg/arch/xen/i386/machdep.c:129: /usr/src/openbsd-xen-sys.hg/dev/isa/isavar.h:138:33: machine/isa_machdep.h: No such file or directory In file included from /usr/src/openbsd-xen-sys.hg/arch/xen/i386/npx.c:66: /usr/src/openbsd-xen-sys.hg/dev/isa/isavar.h:138:33: machine/isa_machdep.h: No such file or directory It does complete, though. But when i try to make the kernel, I get *lots* of errors, starting with these: /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s: Assembler messages: /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:334: Error: suffix or operands invalid for `push' /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:335: Error: suffix or operands invalid for `popf' /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:472: Error: `(((1+2)* (1<<12)))(%esi,%ecx,4)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:479: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:485: Error: `(0x0001|0x0002)(%edx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:486: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:491: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:501: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:506: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:515: Error: `(((0)* (1<<12))+832*4)(%esi)' is not a valid 64 bit base/index expression (and many more not in this file only, but also in vector.s, spl.s, mutex.S) Anyone any hints? Is it meant to be run on -current (I see the last changes in the hg are from 8 months ago, but...)? Or do I need to compile it under an i386 kernel (I'm currently trying with an amd64 one)? # uname -a OpenBSD border2.net1.cc 4.2 GENERIC.MP#1 amd64 Any help MUCH appreciated! Kind regards, Doichin Guess that counts for cross-compilings, so the answer is yes - i need an i386 kernel to compile this. Sorry for the fuzz...
Re: OpenBSD as Xen domU
ropers P=P0P?P8QP0: You can use Christoph Egger's OpenBSD/Xen port. No need to go HVM-only. Unfortunately, my own website is down right now and I haven't gotten around to fixing that, but the Wayback Machine has the relevant page: http://web.archive.org/web/20070403174105/http://ropersonline.com/openbsd/xen/ Also, search the misc archives. This question crops up fairly regularly. and each time most people don't seem to know of Christoph Egger's port (and each time I then try to tell people about it again -- if I catch the message, but I don't always do and sometimes things fall through the cracks here). Thanks and regards, --ropers On 07/02/2008, John Jackson <[EMAIL PROTECTED]> wrote: OpenBSD as DomU works using hardware virtualization for me. There's the occasional lockup that I haven't looked into too much. You can launch vncviewer to get a console. My working config is at the bottom. John On Wed, Feb 06, 2008 at 11:55:05PM +0100, Julien Cabillot wrote: It's work but I had really bad performances with the network (timeout on the interface re). Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html I found that setting the vif interface to 'model=ne2k_pci' helps with the timeouts. On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. Anyone any success running stable OpenBSD (FreeBSD would also suffice) as domU in a Xen system? If so, willing to share config / how-to / experience? Kind regards, Doichin Here's a working Xen config: = import os, re arch = os.uname()[4] if re.search('64', arch): arch_libdir = 'lib64' else: arch_libdir = 'lib' kernel = "/usr/lib/xen/boot/hvmloader" builder='hvm' memory = 256 name = "obsd" pae=0 vif = [ 'type=ioemu, mac=00:16:3e:7d:be:ef, model=ne2k_pci' ] disk = [ 'file:/disk/homer.disk,hda,w','file:/disk/obsd42_amd64.iso,ioemu:hdc:cdrom,r' ] device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm' boot='cd' sdl=0 vnc=1 vncviewer=0 nographic=0 stdvga=0 serial='pty' ne2000=1 audio=0 localtime=1 I'm trying to use Christopher's work, but I get the following errors when i try to make depend a xenU kernel: In file included from /usr/src/openbsd-xen-sys.hg/arch/xen/i386/machdep.c:129: /usr/src/openbsd-xen-sys.hg/dev/isa/isavar.h:138:33: machine/isa_machdep.h: No such file or directory In file included from /usr/src/openbsd-xen-sys.hg/arch/xen/i386/npx.c:66: /usr/src/openbsd-xen-sys.hg/dev/isa/isavar.h:138:33: machine/isa_machdep.h: No such file or directory It does complete, though. But when i try to make the kernel, I get *lots* of errors, starting with these: /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s: Assembler messages: /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:334: Error: suffix or operands invalid for `push' /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:335: Error: suffix or operands invalid for `popf' /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:472: Error: `(((1+2)* (1<<12)))(%esi,%ecx,4)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:479: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:485: Error: `(0x0001|0x0002)(%edx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:486: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:491: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:501: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:506: Error: `(%ebx)' is not a valid 64 bit base/index expression /usr/src/openbsd-xen-sys.hg/arch/xen/i386/locore.s:515: Error: `(((0)* (1<<12))+832*4)(%esi)' is not a valid 64 bit base/index expression (and many more not in this file only, but also in vector.s, spl.s, mutex.S) Anyone any hints? Is it meant to be run on -current (I see the last changes in the hg are from 8 months ago, but...)? Or do I need to compile it under an i386 kernel (I'm currently trying with an amd64 one)? # uname -a OpenBSD border2.net1.cc 4.2 GENERIC.MP#1 amd64 Any help MUCH appreciated! Kind regards, Doichin
Re: OpenBSD as Xen domU
You can use Christoph Egger's OpenBSD/Xen port. No need to go HVM-only. Unfortunately, my own website is down right now and I haven't gotten around to fixing that, but the Wayback Machine has the relevant page: http://web.archive.org/web/20070403174105/http://ropersonline.com/openbsd/xen/ Also, search the misc archives. This question crops up fairly regularly. and each time most people don't seem to know of Christoph Egger's port (and each time I then try to tell people about it again -- if I catch the message, but I don't always do and sometimes things fall through the cracks here). Thanks and regards, --ropers On 07/02/2008, John Jackson <[EMAIL PROTECTED]> wrote: > OpenBSD as DomU works using hardware virtualization for me. There's > the occasional lockup that I haven't looked into too much. You can > launch vncviewer to get a console. My working config is at the bottom. > > John > > On Wed, Feb 06, 2008 at 11:55:05PM +0100, Julien Cabillot wrote: > > It's work but I had really bad performances with the network (timeout on > > the interface re). > > Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html > > > > I found that setting the vif interface to 'model=ne2k_pci' helps with > the timeouts. > > > > > On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: > > > I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. > > > Anyone any success running stable OpenBSD (FreeBSD would also suffice) > > > as domU in a Xen system? If so, willing to share config / how-to / > > > experience? > > > > > > Kind regards, > > > Doichin > > > > Here's a working Xen config: > = > import os, re > arch = os.uname()[4] > if re.search('64', arch): > arch_libdir = 'lib64' > else: > arch_libdir = 'lib' > kernel = "/usr/lib/xen/boot/hvmloader" > builder='hvm' > memory = 256 > name = "obsd" > pae=0 > vif = [ 'type=ioemu, mac=00:16:3e:7d:be:ef, model=ne2k_pci' ] > disk = [ > 'file:/disk/homer.disk,hda,w','file:/disk/obsd42_amd64.iso,ioemu:hdc:cdrom,r' > ] > device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm' > boot='cd' > sdl=0 > vnc=1 > vncviewer=0 > nographic=0 > stdvga=0 > serial='pty' > ne2000=1 > audio=0 > localtime=1 > = > > -- www.ropersonline.com
Re: OpenBSD as Xen domU
OpenBSD as DomU works using hardware virtualization for me. There's the occasional lockup that I haven't looked into too much. You can launch vncviewer to get a console. My working config is at the bottom. John On Wed, Feb 06, 2008 at 11:55:05PM +0100, Julien Cabillot wrote: > It's work but I had really bad performances with the network (timeout on > the interface re). > Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html > I found that setting the vif interface to 'model=ne2k_pci' helps with the timeouts. > > On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: > > I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. > > Anyone any success running stable OpenBSD (FreeBSD would also suffice) > > as domU in a Xen system? If so, willing to share config / how-to / > > experience? > > > > Kind regards, > > Doichin > Here's a working Xen config: = import os, re arch = os.uname()[4] if re.search('64', arch): arch_libdir = 'lib64' else: arch_libdir = 'lib' kernel = "/usr/lib/xen/boot/hvmloader" builder='hvm' memory = 256 name = "obsd" pae=0 vif = [ 'type=ioemu, mac=00:16:3e:7d:be:ef, model=ne2k_pci' ] disk = [ 'file:/disk/homer.disk,hda,w','file:/disk/obsd42_amd64.iso,ioemu:hdc:cdrom,r' ] device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm' boot='cd' sdl=0 vnc=1 vncviewer=0 nographic=0 stdvga=0 serial='pty' ne2000=1 audio=0 localtime=1 =
Re: OpenBSD as Xen domU
It's work but I had really bad performances with the network (timeout on the interface re). Dmesg: http://www.openbsd-france.org/ml/archives/msg02494.html On jeu, 2008-02-07 at 00:29 +0200, NetOne - Doichin Dokov wrote: > I'm looking to replace a Linux domU with a BSD one, preferably OpenBSD. > Anyone any success running stable OpenBSD (FreeBSD would also suffice) > as domU in a Xen system? If so, willing to share config / how-to / > experience? > > Kind regards, > Doichin