Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 04:43:32PM -0400, Chris Ross wrote: > [- tried to grub-install onto /dev/sda1 and /dev/sda2, won't boot, Illegal >Instruction. -] > So, I fear some of this is based on the fact that I'm running grub-install > into raid filesystems that are mounted as a RAID1, but I'm sticking things > into them at the same time. Or maybe it's more complicated, but the fact > that I'm writing into the raw partitions under an md RAID certainly seems > of concern. :-/ So I'm even more confused now. I was unable to boot disk0 or disk1, but also got errors booting off of disk3, which had been working. So I booted into rescue cd, re-ran the grub-install onto /dev/sdd1, then rebooted. Came up fine, as before. When it did, I did nothing but log in, then re-run "grub-install --force --skip-fs-probe" onto each of /dev/sda1 and /dev/sda2, while /dev/md0 wasn't mounted. Nothing on the first three disks had been accessed since boot. Sadly, this did not make those disks bootable either. But, surprisingly, I now can't boot off of disk3 again. Even after a "reset-all", I see: Boot device: disk3 File and args: GRUB ERROR: Last Trap: Illegal Instruction So, I've no idea how merely running grub-install onto /dev/sda1 and /dev/sdb1 could possibly have messed up the grub on /dev/sdd1? I am booting into rescue cd again now to repair that, but I'm starting to feel that grub is acting for some evil spirit and I am failing to meet whatever it's needs are - Chris ps, re-grub-install'ing to /dev/sdd1 allowed the system to boot again. I rebooted a few times, seems to have remained functional. I also after a few successful boots to sdd (aka disk3), tried booting from PROM to disk0 again, got the Illegal Instruction, but was then still able to boot disk3.
Re: Grub, sparc64, and compressed kernels
Sorry for the chatter on the list, all...
On Tue, Jul 03, 2018 at 04:14:18PM -0400, Chris Ross wrote:
> Adrian answered some of this before I asked the above questions:
>
> t5120# grub-install /dev/sda1
> Installing for sparc64-ieee1275 platform.
> grub-install: warning: File system `ext2' doesn't support embedding.
> grub-install: error: embedding is not possible, but this is required for RAID
> and LVM install.
> t5120# grub-install --force --skip-fs-probe /dev/sda1
> Installing for sparc64-ieee1275 platform.
> grub-install: warning: File system `ext2' doesn't support embedding.
> grub-install: error: embedding is not possible, but this is required for RAID
> and LVM install.
> t5120#
Okay. So, I see the above error from grub-install, when running in the
chroot, for both /dev/sda1, _and_ for /dev/sdd1 which worked fine for the
real disks I'm running on, and booted from. From the full machine, I can
also successfully grub-install onto /dev/sda1 (and /dev/sdb1). I don't
know why that is different when running in the chroot onto the target area.
Rebooting now, to see if I can load a working grub from disk0, and what
it can load. Hmm. Okay, clearly some problems there...
Boot device: disk3 File and args:
[halt sent]
{0} ok boot disk0
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0 File and args:
GRUB Loading kernel...
ERROR: /pci@0/pci@0/pci@2/scsi@0: Last Trap: Illegal Instruction
{0} ok boot disk1
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@1 File and args:
GRUB Loading kernel
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
UNKNOWN DEVICE: ieee1275//pci@0/pci@0/pci@2/scsi@0/disk@3\,0:a
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
ERROR: /pci@0: Last Trap: Fast Data Access MMU Miss
{0} ok boot disk0
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0 File and args:
GRUB Loading kernel...
ERROR: Last Trap: Illegal Instruction
{0} ok
So, I fear some of this is based on the fact that I'm running grub-install
into raid filesystems that are mounted as a RAID1, but I'm sticking things
into them at the same time. Or maybe it's more complicated, but the fact
that I'm writing into the raw partitions under an md RAID certainly seems
of concern. :-/
Again, let me know any thoughts. Thanks.
- Chris
Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 03:37:24PM -0400, Chris Ross wrote: > Sorry, stupid user trick. I had mdadm and friends in the disk I was running > on, but had _not_ installed those packages into the environment in md/ZFS > that I chrooted into. After installing mdadm (and the collection it brought), > I am able to update-grub. And grub-probe gives me the answer I'd expect: > > But, I'm seeing the following if I try to grub-install: > > (chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0 > Installing for sparc64-ieee1275 platform. > grub-install: warning: File system `ext2' doesn't support embedding. > grub-install: warning: Embedding is not possible. GRUB can only be installed > in this setup by using blocklists. However, blocklists are UNRELIABLE and > their use is discouraged.. > zsh: segmentation fault grub-install --force --skip-fs-probe /dev/md0 > > (1) Is this supposed to work? > (2) Assuming it's not actually installed correctly yet, booting will fail. > Should I grub-install onto /dev/sda1 or /dev/sda2? Whether one or both, > won't that mess up the RAID of those devices? Adrian answered some of this before I asked the above questions: On Tue, Jul 03, 2018 at 09:39:56PM +0200, John Paul Adrian Glaubitz wrote: > Well, you need to tell GRUB to install on the individual disks > and not on the RAID (md0), this won't work - with any boot loader. > [...] > Don't use /dev/md0, use /dev/sd* and install on any of the disks that > are part of the RAID. You cannot boot from /dev/md0 as the kernel needs > to be running to be able to access the software RAID device. Okay. That all makes sense, thank you. Two problems at this point (1) The segv above. I'm doing something wrong, but it shouldn't SEGV (2) Installing onto /dev/sda1 doesn't work. t5120# grub-install /dev/sda1 Installing for sparc64-ieee1275 platform. grub-install: warning: File system `ext2' doesn't support embedding. grub-install: error: embedding is not possible, but this is required for RAID and LVM install. t5120# grub-install --force --skip-fs-probe /dev/sda1 Installing for sparc64-ieee1275 platform. grub-install: warning: File system `ext2' doesn't support embedding. grub-install: error: embedding is not possible, but this is required for RAID and LVM install. t5120# Is there some other options for grub-install I'm missing? Embedding (grub-install /dev/sda) isn't possible with sun partition labels, but the above message suggests that embedding is required for RAID install. Thanks... - Chris
Re: Grub, sparc64, and compressed kernels
On 07/03/2018 09:24 PM, Chris Ross wrote: > Okay. So, in prepping a chroot'd md/zfs environemnt on this machine, while > updating the kernel packages, I see: > > /etc/kernel/postinst.d/zz-update-grub: > Generating grub configuration file ... > Found linux image: /boot/vmlinuz-4.16.0-2-sparc64-smp > Found initrd image: /boot/initrd.img-4.16.0-2-sparc64-smp > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > Found linux image: /boot/vmlinuz-4.16.0-1-sparc64-smp > Found initrd image: /boot/initrd.img-4.16.0-1-sparc64-smp > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > Found Debian GNU/Linux buster/sid on /dev/sdd2 > done > > This leads me to the same problem I had earlier, grub (grub-probe or > grub-install) saying "error: disk `md0' not found." I think this is why > I started down a path of looking for an alternative grub2. Well, you need to tell GRUB to install on the individual disks and not on the RAID (md0), this won't work - with any boot loader. > Googling for this error leads me to many people that had had this error, > sometimes an old double-free problem that isn't hitting me, > and problems with one-disk md RAID arrays, which isn't my issue. But I > don't see anyone with a solution to my problem without the others. I > fear I just don't know what I'm looking for. What double-free problem? That would indicate a software bug, but this isn't a bug. You just need to tell GRUB not to use md0 as the target block device as GRUB won't be able to map this to actual physical disks. > If anyone else is more familiar with using grub, and has any idea > how to work around this current situation, I'd appreciate a pointer. > > (chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0 > Installing for sparc64-ieee1275 platform. > grub-install: error: disk `md0' not found. > (chroot) root@t5120# ls -l /dev/md* > brw-rw 1 root disk 9, 0 Jul 3 14:26 /dev/md0 > crw--- 1 root root 10, 62 Jul 3 12:19 /dev/mdesc > (chroot) root@t5120# df /boot > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/md0 458396 63776366432 15% /boot > (chroot) root@t5120# Don't use /dev/md0, use /dev/sd* and install on any of the disks that are part of the RAID. You cannot boot from /dev/md0 as the kernel needs to be running to be able to access the software RAID device. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 03:24:12PM -0400, Chris Ross wrote: > Okay. So, in prepping a chroot'd md/zfs environemnt on this machine, while > updating the kernel packages, I see: > > /etc/kernel/postinst.d/zz-update-grub: > Generating grub configuration file ... > Found linux image: /boot/vmlinuz-4.16.0-2-sparc64-smp > Found initrd image: /boot/initrd.img-4.16.0-2-sparc64-smp > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > Found linux image: /boot/vmlinuz-4.16.0-1-sparc64-smp > Found initrd image: /boot/initrd.img-4.16.0-1-sparc64-smp > /usr/sbin/grub-probe: error: disk `md0' not found. > /usr/sbin/grub-probe: error: disk `md0' not found. > Found Debian GNU/Linux buster/sid on /dev/sdd2 > done > > This leads me to the same problem I had earlier, grub (grub-probe or > grub-install) saying "error: disk `md0' not found." I think this is why > I started down a path of looking for an alternative grub2. Sorry, stupid user trick. I had mdadm and friends in the disk I was running on, but had _not_ installed those packages into the environment in md/ZFS that I chrooted into. After installing mdadm (and the collection it brought), I am able to update-grub. And grub-probe gives me the answer I'd expect: (chroot) root@t5120# grub-probe -d /dev/md0 ext2 But, I'm seeing the following if I try to grub-install: (chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0 Installing for sparc64-ieee1275 platform. grub-install: warning: File system `ext2' doesn't support embedding. grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged.. zsh: segmentation fault grub-install --force --skip-fs-probe /dev/md0 (1) Is this supposed to work? (2) Assuming it's not actually installed correctly yet, booting will fail. Should I grub-install onto /dev/sda1 or /dev/sda2? Whether one or both, won't that mess up the RAID of those devices? - Chris
Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 12:22:34PM -0400, Chris Ross wrote: > I'll run with this for a bit, and respond back with questions about getting > it installed onto software-raid partitions later. Thanks again. Okay. So, in prepping a chroot'd md/zfs environemnt on this machine, while updating the kernel packages, I see: /etc/kernel/postinst.d/zz-update-grub: Generating grub configuration file ... Found linux image: /boot/vmlinuz-4.16.0-2-sparc64-smp Found initrd image: /boot/initrd.img-4.16.0-2-sparc64-smp /usr/sbin/grub-probe: error: disk `md0' not found. /usr/sbin/grub-probe: error: disk `md0' not found. /usr/sbin/grub-probe: error: disk `md0' not found. Found linux image: /boot/vmlinuz-4.16.0-1-sparc64-smp Found initrd image: /boot/initrd.img-4.16.0-1-sparc64-smp /usr/sbin/grub-probe: error: disk `md0' not found. /usr/sbin/grub-probe: error: disk `md0' not found. Found Debian GNU/Linux buster/sid on /dev/sdd2 done This leads me to the same problem I had earlier, grub (grub-probe or grub-install) saying "error: disk `md0' not found." I think this is why I started down a path of looking for an alternative grub2. Googling for this error leads me to many people that had had this error, sometimes an old double-free problem that isn't hitting me, and problems with one-disk md RAID arrays, which isn't my issue. But I don't see anyone with a solution to my problem without the others. I fear I just don't know what I'm looking for. If anyone else is more familiar with using grub, and has any idea how to work around this current situation, I'd appreciate a pointer. (chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0 Installing for sparc64-ieee1275 platform. grub-install: error: disk `md0' not found. (chroot) root@t5120# ls -l /dev/md* brw-rw 1 root disk 9, 0 Jul 3 14:26 /dev/md0 crw--- 1 root root 10, 62 Jul 3 12:19 /dev/mdesc (chroot) root@t5120# df /boot Filesystem 1K-blocks Used Available Use% Mounted on /dev/md0 458396 63776366432 15% /boot (chroot) root@t5120# Thanks. - Chris
Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 12:04:09PM -0400, Chris Ross wrote: > It sounds from Eric like the grub I built from him doesn't want to deal with > gzip. But the one shipping (2.02+dfsg1-4) does? Well, easy enough to try > that on the same disk to see. But, I presume I will face the same problem > with it not being willing to install onto md0. I'll face that later. > > Back shortly with data on Debian's grub 2.02+dfsg1-4 results. Okay. Took me a couple attempts, and the grub 2.02+ is a lot more alarming in what it says: % sudo grub-install --force --skip-fs-probe /dev/sdd1 Installing for sparc64-ieee1275 platform. grub-install: warning: Discarding improperly nested partition (hostdisk//dev/sdd,sun1,sun2). grub-install: warning: Discarding improperly nested partition (hostdisk//dev/sdd,sun1,sun4). grub-install: warning: Discarding improperly nested partition (hostdisk//dev/sdd,sun1,sun5). grub-install: warning: Attempting to install GRUB to a disk with multiple partition labels. This is not supported yet.. grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged.. Installation finished. No error reported. % But, after getting that installed, the machine rebooted with the default (vmlinuz, gzip'd) kernel. Thought i do notice before it puts up the initial boot menu: Boot device: disk3 File and args: GRUB Loading kernel error: out of memory. error: no suitable video mode found. ...then the screen clears and shows the boot menu, and when left alone to boot the default kernel, it succeeds. So, I'm curious to read the conversation spawned by your comment back on the bug I'd opened in grub2-sparc in github. I'll run with this for a bit, and respond back with questions about getting it installed onto software-raid partitions later. Thanks again. - Chris
Re: Grub, sparc64, and compressed kernels
On Tue, Jul 03, 2018 at 05:46:18PM +0200, John Paul Adrian Glaubitz wrote: > Hi Chris! > > Any reason why you aren't just using the normal Debian procedure to setup > and install GRUB? Well, IIRC, because it wasn't working for me. But, I must admit, I'm not 100% sure I tried that on the simple disk. I know I was having no luck getting it installed on md0 when I tried, and eventually punted back to trying on sdd just to get something working. I am no longer 100% sure I hadn't already switched to trying https://github.com/esnowberg/grub2-sparc/ before that. > Debian stores its kernel compressed by default and I don't see how this is > supposed to be a problem. All SPARCs that we have installed with Debian > for sparc64 are using the grub2 package from Debian with a compressed > kernel. It sounds from Eric like the grub I built from him doesn't want to deal with gzip. But the one shipping (2.02+dfsg1-4) does? Well, easy enough to try that on the same disk to see. But, I presume I will face the same problem with it not being willing to install onto md0. I'll face that later. Back shortly with data on Debian's grub 2.02+dfsg1-4 results. - Chris
Re: Grub, sparc64, and compressed kernels
Hi Chris! On 07/03/2018 05:38 PM, Chris Ross wrote: > As I was mentioning in other threads, I was getting grub installed onto the > "normal" sun-labeled ext2/4 partitioned disk in my T5120. Working with > Eric (https://github.com/esnowberg/grub2-sparc/issues/14) he helped me > figure out that grub wasn't working because the kernel in /boot was gzip'd. Any reason why you aren't just using the normal Debian procedure to setup and install GRUB? > Is there something that's told Debian that I want my kernels in /boot to be > gzip'd? Is there a way to make it _not_ do that, since grub apparently > needs them not to be gzip'd on this machine? Debian stores its kernel compressed by default and I don't see how this is supposed to be a problem. All SPARCs that we have installed with Debian for sparc64 are using the grub2 package from Debian with a compressed kernel. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Grub, sparc64, and compressed kernels
As I was mentioning in other threads, I was getting grub installed onto the "normal" sun-labeled ext2/4 partitioned disk in my T5120. Working with Eric (https://github.com/esnowberg/grub2-sparc/issues/14) he helped me figure out that grub wasn't working because the kernel in /boot was gzip'd. Is there something that's told Debian that I want my kernels in /boot to be gzip'd? Is there a way to make it _not_ do that, since grub apparently needs them not to be gzip'd on this machine? Thanks. - Chris
