Re: FreeBSD Xen DomU boot from drbd fails with error 19
On Mon, Oct 24, 2016 at 08:50:01PM -0400, n...@bigfoot.com wrote: > On 24-10-2016 13:02, Roger Pau Monné wrote: > > On Sat, Oct 22, 2016 at 04:14:33PM -0400, n...@bigfoot.com wrote: > >> Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian > >> jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM > >> using 'disk' /dev// in domU config. > >> > >> I have a number of Linux guests running on drbd (8.4). Booting FreeBSD > >> from the drbd device associated with drbd (with 'disk' /dev/drbdx) > >> fails. I know I can't use drbd: with HVM guest. > >> > >> Trying to mount root from ufs:/dev/xbd0p2 [rw]... > >> mountroot: waiting for device /dev/xbd0p2... > >> Mounting from ufs:/dev/xbd0p2 failed with error 19. > > > > Ops, hit send to soon. Can you also paste the output of `xenstore-ls -fp` > > executed from Dom0 when the guest is in this state? > > > > Thanks, Roger. > > > > Thanks for replying. > > Sorry for the long post. > There is one Linux PV guest on this host, not on drbd, but there are > configured drbd devices on this host that belong to linux PV guests on > another host also running debian jessie. They boot from drbd without > problems. > > To collect more data I created a fresh FreeBSD HVM guest on the same > host. Interestingly, this guest does boot from drbd. You'll find it as > domain 16 (vps13) in the attached output of xenstore-ls. > > The HVM guest that fails to mount root is domain 17 (vps10). > The error message is: > Trying to mount root from ufs:/dev/xbd0p2 [rw]... > GEOM: xbd0: corrupt or invalid GPT detected. > GEOM: xbd0: GPT rejected -- may not be recoverable. > > I should mention that there seems to be a conflict with the second GEOM > GPT table in FreeBSD and the drbd metadata both writing to the end of > the partition (we are using internal metadata). After creating metadata > (drbdadm create-md) the kernel complains about the secondary GPT table, > but that does not prevent it from mounting the fs. > > Inside the domu this can be resolved by issuing 'gpart recover', but > this in turn seems to destroy the drbd metadata, which is apparent by > the need to re-create md and re-sync at the next boot. > > Perhaps we should use external metadata in this case. I have no idea how drbd works, but it certainly shouldn't store internal metadata in the device presented to the guest, or else havoc is going to happen for sure. I would suggest that you try this "external metadata" mode. > As an aside, for your information, we tried to run the FreeBSD 11.0 > guest in PVH mode as well, having followed the instructions on > https://wiki.xen.org/wiki/FreeBSD_PVH. > > The guest failed to start with xl reporting en error: > xc: error: elf_xen_addr_calc_check: ERROR: ELF start or entries are out > of bounds.: Invalid kernel > Output of readelf -l /xenkernels/freebsd/kernel-11.0-RELEASE: > Elf file type is EXEC (Executable file) > Entry point 0x802ff000 > There are 6 program headers, starting at offset 64 That's expected, Xen 4.4 has some PVH support, but that's not enough, IIRC you should use Xen 4.6 or newer. Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen DomU boot from drbd fails with error 19
On 24-10-2016 13:02, Roger Pau Monné wrote: > On Sat, Oct 22, 2016 at 04:14:33PM -0400, n...@bigfoot.com wrote: >> Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian >> jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM >> using 'disk' /dev// in domU config. >> >> I have a number of Linux guests running on drbd (8.4). Booting FreeBSD >> from the drbd device associated with drbd (with 'disk' /dev/drbdx) >> fails. I know I can't use drbd: with HVM guest. >> >> Trying to mount root from ufs:/dev/xbd0p2 [rw]... >> mountroot: waiting for device /dev/xbd0p2... >> Mounting from ufs:/dev/xbd0p2 failed with error 19. > > Ops, hit send to soon. Can you also paste the output of `xenstore-ls -fp` > executed from Dom0 when the guest is in this state? > > Thanks, Roger. > Thanks for replying. Sorry for the long post. There is one Linux PV guest on this host, not on drbd, but there are configured drbd devices on this host that belong to linux PV guests on another host also running debian jessie. They boot from drbd without problems. To collect more data I created a fresh FreeBSD HVM guest on the same host. Interestingly, this guest does boot from drbd. You'll find it as domain 16 (vps13) in the attached output of xenstore-ls. The HVM guest that fails to mount root is domain 17 (vps10). The error message is: Trying to mount root from ufs:/dev/xbd0p2 [rw]... GEOM: xbd0: corrupt or invalid GPT detected. GEOM: xbd0: GPT rejected -- may not be recoverable. I should mention that there seems to be a conflict with the second GEOM GPT table in FreeBSD and the drbd metadata both writing to the end of the partition (we are using internal metadata). After creating metadata (drbdadm create-md) the kernel complains about the secondary GPT table, but that does not prevent it from mounting the fs. Inside the domu this can be resolved by issuing 'gpart recover', but this in turn seems to destroy the drbd metadata, which is apparent by the need to re-create md and re-sync at the next boot. Perhaps we should use external metadata in this case. As an aside, for your information, we tried to run the FreeBSD 11.0 guest in PVH mode as well, having followed the instructions on https://wiki.xen.org/wiki/FreeBSD_PVH. The guest failed to start with xl reporting en error: xc: error: elf_xen_addr_calc_check: ERROR: ELF start or entries are out of bounds.: Invalid kernel Output of readelf -l /xenkernels/freebsd/kernel-11.0-RELEASE: Elf file type is EXEC (Executable file) Entry point 0x802ff000 There are 6 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSizMemSiz Flags Align PHDR 0x0040 0x80200040 0x00200040 0x0150 0x0150 R E8 INTERP 0x0190 0x80200190 0x00200190 0x000d 0x000d R 1 [Requesting program interpreter: /red/herring] LOAD 0x 0x8020 0x0020 0x014ed860 0x014ed860 R E20 LOAD 0x014ee000 0x818ee000 0x018ee000 0x00132538 0x005ecfa0 RW 20 DYNAMIC0x014ee000 0x818ee000 0x018ee000 0x00d0 0x00d0 RW 8 GNU_STACK 0x 0x 0x 0x 0x RWE8 Section to Segment mapping: Segment Sections... 00 01 .interp 02 .interp .hash .dynsym .dynstr .text .rodata set_sysctl_set set_sysinit_set set_sysuninit_set set_modmetadata_set set_ah_chips set_ah_rfs set_kbddriver_set set_sdt_providers_set set_sdt_probes_set set_sdt_argtypes_set set_cons_set usb_host_id set_vt_drv_set set_kdb_dbbe_set set_ratectl_set set_crypto_set set_ieee80211_ioctl_getset set_ieee80211_ioctl_setset set_scanner_set set_videodriver_set set_scterm_set set_scrndr_set set_vga_set kern_conf .eh_frame 03 .dynamic .got.plt .data set_pcpu .bss 04 .dynamic 05 Finally, here is the output of xenstore-ls -fp when the FreeBSD HVM guest is hanging at mount root: /tool = "" (n0) /tool/xenstored = "" (n0) /local = "" (n0) /local/domain = "" (n0) /local/domain/0 = "" (n0) /local/domain/0/name = "Domain-0" (n0) /local/domain/0/domid = "0" (n0) /local/domain/0/memory = "" (n0) /local/domain/0/memory/target = "10730008" (n0) /local/domain/0/memory/static-max = "4294967292" (n0) /local/domain/0/memory/freemem-slack = "410368" (n0) /local/domain/0/libxl = "" (n0) /local/domain/0/libxl/disable_udev = "1" (n0) /local/domain/0/backend = "" (n0) /local/domain/0/backend/vbd = "" (n0) /local/domain/0/backend/vbd/1 = "" (n0) /local/domain/0/backend/vbd/1/51714
Re: FreeBSD Xen DomU boot from drbd fails with error 19
On Sat, Oct 22, 2016 at 04:14:33PM -0400, n...@bigfoot.com wrote: > Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian > jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM > using 'disk' /dev// in domU config. > > I have a number of Linux guests running on drbd (8.4). Booting FreeBSD > from the drbd device associated with drbd (with 'disk' /dev/drbdx) > fails. I know I can't use drbd: with HVM guest. > > Trying to mount root from ufs:/dev/xbd0p2 [rw]... > mountroot: waiting for device /dev/xbd0p2... > Mounting from ufs:/dev/xbd0p2 failed with error 19. Ops, hit send to soon. Can you also paste the output of `xenstore-ls -fp` executed from Dom0 when the guest is in this state? Thanks, Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen DomU boot from drbd fails with error 19
On Sat, Oct 22, 2016 at 04:14:33PM -0400, n...@bigfoot.com wrote: > Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian > jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM > using 'disk' /dev// in domU config. > > I have a number of Linux guests running on drbd (8.4). Booting FreeBSD > from the drbd device associated with drbd (with 'disk' /dev/drbdx) > fails. I know I can't use drbd: with HVM guest. > > Trying to mount root from ufs:/dev/xbd0p2 [rw]... > mountroot: waiting for device /dev/xbd0p2... > Mounting from ufs:/dev/xbd0p2 failed with error 19. > > Device xbd0 is present, but no GPT tables are found. Hm, this is quite weird. Are the Linux guests using drbd also HVM guests? Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
FreeBSD Xen DomU boot from drbd fails with error 19
Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM using 'disk' /dev// in domU config. I have a number of Linux guests running on drbd (8.4). Booting FreeBSD from the drbd device associated with drbd (with 'disk' /dev/drbdx) fails. I know I can't use drbd: with HVM guest. Trying to mount root from ufs:/dev/xbd0p2 [rw]... mountroot: waiting for device /dev/xbd0p2... Mounting from ufs:/dev/xbd0p2 failed with error 19. Device xbd0 is present, but no GPT tables are found. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
On Dec 1, 2009, at 8:35 AM, Josh Carter wrote: > Hi all, > > On Dec 1, 2009, at 4:37 AM, Luke Marsden wrote: > >> Thanks for your reply. I'll see what I can do about getting the precise >> version (which I believe is indeed 3.0.3) and patches for you. Amazon say on >> http://aws.amazon.com/ec2/ that "we work with our partners and community to >> provide you with the most choice possible. You are also empowered to use our >> bundling tools to upload your own operating systems"... so I'll attempt to >> hold them to that :) > > I've been in contact with Jeff Barr at Amazon over the past couple months and > he's trying to put me in contact with an "appropriate technical resource" to > support FreeBSD efforts toward a EC2-compatible system. > > Right now my company has a full-time expert working on FreeBSD on Xen, but > our focus is initially IO performance. We're running 64-bit HVM with PV > drivers. Given that Amazon runs Windows on EC2 they should have the ability > to run HVM, but the preferable solution is PV. I don't know when PV will > become a priority for somebody; for us (and some others I know) it wouldn't > matter until IO runs a lot faster. > > In any case, help from Amazon would be welcome. There are a number of > companies interested in FreeBSD on Xen in various configurations, and efforts > from any party benefit everyone else. Scott Long re-wrote blkfront, I have since updated the copyright and merged it in to HEAD. There is an extension to busdma that you may need, depending on file system consumer. It doesn't perform much better on random I/O due to design issues with blkback. We've debated re-writing linux's blkback versus simply moving all I/O to a driver domain and letting freebsd take care of it. -Kip___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
Hi all, On Dec 1, 2009, at 4:37 AM, Luke Marsden wrote: > Thanks for your reply. I'll see what I can do about getting the precise > version (which I believe is indeed 3.0.3) and patches for you. Amazon say on > http://aws.amazon.com/ec2/ that "we work with our partners and community to > provide you with the most choice possible. You are also empowered to use our > bundling tools to upload your own operating systems"... so I'll attempt to > hold them to that :) I've been in contact with Jeff Barr at Amazon over the past couple months and he's trying to put me in contact with an "appropriate technical resource" to support FreeBSD efforts toward a EC2-compatible system. Right now my company has a full-time expert working on FreeBSD on Xen, but our focus is initially IO performance. We're running 64-bit HVM with PV drivers. Given that Amazon runs Windows on EC2 they should have the ability to run HVM, but the preferable solution is PV. I don't know when PV will become a priority for somebody; for us (and some others I know) it wouldn't matter until IO runs a lot faster. In any case, help from Amazon would be welcome. There are a number of companies interested in FreeBSD on Xen in various configurations, and efforts from any party benefit everyone else. -Josh ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
Hi Kip, Thanks for your reply. I'll see what I can do about getting the precise version (which I believe is indeed 3.0.3) and patches for you. Amazon say on http://aws.amazon.com/ec2/ that "we work with our partners and community to provide you with the most choice possible. You are also empowered to use our bundling tools to upload your own operating systems"... so I'll attempt to hold them to that :) I'm also happy to contribute a paid EC2 account for development (within reason) since if we can't precisely replicate locally the environment they run then I imagine the next-best thing would be for you to have access to one of their instances which you can upload OS images to and debug FreeBSD as an EC2 DomU that way. Would you be interested in that? Let me know if there's anything else I can do to help / facilitate. I'll post again when I've got in touch with Amazon. Cheers, Luke Marsden Hybrid Logic Ltd. On Mon, Nov 30, 2009 at 5:40 AM, K. Macy wrote: > > On Nov 15, 2009, at 10:08 AM, Michael Sierchio wrote: > > > > > I second the notion - could someone please articulate what > > the problems are with running FreeBSD as a DomU? > > > > Being able to run a 8.x instance on EC2 would be very helpful. > > > > Amazon runs a very old Redhat version of Xen and they aren't forthcoming > about the specific version, presumably out of security concerns. > > 3.0.3 doesn't support linear page tables and thus requires some pmap > futzing. Apart from that I don't have enough specifics to go on. If someone > told me the precise version and gave me the corresponding RH patches I'd be > much more interested. Sorry for the inconvenience. > > > -Kip > > ___ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org" > ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
On Nov 15, 2009, at 10:08 AM, Michael Sierchio wrote: > > I second the notion - could someone please articulate what > the problems are with running FreeBSD as a DomU? > > Being able to run a 8.x instance on EC2 would be very helpful. > Amazon runs a very old Redhat version of Xen and they aren't forthcoming about the specific version, presumably out of security concerns. 3.0.3 doesn't support linear page tables and thus requires some pmap futzing. Apart from that I don't have enough specifics to go on. If someone told me the precise version and gave me the corresponding RH patches I'd be much more interested. Sorry for the inconvenience. -Kip ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
I second the notion - could someone please articulate what the problems are with running FreeBSD as a DomU? Being able to run a 8.x instance on EC2 would be very helpful. -- Michael Sierchio +1 415 378 1182 PO Box 9036 Berkeley CA 94709 US ku...@tenebras.com ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
Hi FreeBSD-Xen, I'm very interested in getting FreeBSD 8 running on either EC2 or GoGrid (or another large Xen-powered "cloud"), and I'm not afraid to get my hands dirty. I'm an experienced programmer but not a kernel programmer, but am very willing to learn. Could someone please summarise the problem(s) standing in the way of getting FreeBSD running and production-ready on these environments and give me some pointers on how I could start hacking at the code? Thanks, Luke Marsden Hybrid Logic Ltd. On Tue, 2009-11-03 at 13:07 +, Brett Sheffield wrote: > Adrian Chadd wrote: > > Take a look at my page in the wiki. wiki.freebsd.org/AdrianChadd > > > > I haven't built any Xen images lately. I may try and build an 8.0-RC1 > > or 8.0-RC2 kernel image. > > > > I think things are still a bit unstable though and I personally don't > > have the time to try and fix them at the present.. > > Thanks Adrian. I'll use that as a starting point and see how I get on. > ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
Adrian Chadd wrote: > Take a look at my page in the wiki. wiki.freebsd.org/AdrianChadd > > I haven't built any Xen images lately. I may try and build an 8.0-RC1 > or 8.0-RC2 kernel image. > > I think things are still a bit unstable though and I personally don't > have the time to try and fix them at the present.. Thanks Adrian. I'll use that as a starting point and see how I get on. -- __ Application Hosting | Virtual Servers | Monitoring | Email Tel: 0131 510 0340http://www.gladserv.com/ ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: FreeBSD Xen domU ?
Take a look at my page in the wiki. wiki.freebsd.org/AdrianChadd I haven't built any Xen images lately. I may try and build an 8.0-RC1 or 8.0-RC2 kernel image. I think things are still a bit unstable though and I personally don't have the time to try and fix them at the present.. Adrian 2009/11/3 Brett Sheffield : > Greetings all, > > What's the present status of FreeBSD running under Xen? I see some > posts in the archives of this mailing list suggesting RC2 works. Is > anyone using this in production yet? > > I was planning on offering FreeBSD virtual servers from next year, but > I've had an enquiry already so I'm looking at it sooner than I'd expected. > > Documentation at http://wiki.freebsd.org/FreeBSD/Xen is a bit thin. It > links to http://www.fsmware.com/xenofreebsd/7.0/download/ which is broken: > > $ host www.fsmware.com > Host www.fsmware.com not found: 3(NXDOMAIN) > > Is this the most up-to-date doco? Happy to contribute to improving it > (assuming I can get it working myself, of course!) > > Cheers, > > > Brett > -- > __ > Application Hosting | Virtual Servers | Monitoring | Email > Tel: 0131 510 0340 http://www.gladserv.com/ > ___ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org" > ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
FreeBSD Xen domU ?
Greetings all, What's the present status of FreeBSD running under Xen? I see some posts in the archives of this mailing list suggesting RC2 works. Is anyone using this in production yet? I was planning on offering FreeBSD virtual servers from next year, but I've had an enquiry already so I'm looking at it sooner than I'd expected. Documentation at http://wiki.freebsd.org/FreeBSD/Xen is a bit thin. It links to http://www.fsmware.com/xenofreebsd/7.0/download/ which is broken: $ host www.fsmware.com Host www.fsmware.com not found: 3(NXDOMAIN) Is this the most up-to-date doco? Happy to contribute to improving it (assuming I can get it working myself, of course!) Cheers, Brett -- __ Application Hosting | Virtual Servers | Monitoring | Email Tel: 0131 510 0340http://www.gladserv.com/ ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
