Re: Will try debian-10.0-sparc64-grub-NETINST-1.iso

[Chris Ross]

On Sun, May 05, 2019 at 07:40:20PM -0400, Dennis Clarke wrote:
> I see https://cdimage.debian.org/cdimage/ports/grub-test/ and
> have fetched debian-10.0-sparc64-grub-NETINST-1.iso which may
> work on a very very old netra test unit. A Netra X1 in fact.
> However it has enough memory and a basic serial port for a
> console.
> 
> Can I serve up this iso image via TFTP/NFS or is it better to
> mess around with a physical CDROM ?

I don't have any answers or advice for you, but would be very 
interested to hear more about it.  I also have old Sparc's, including
(if it's still around somewhere) an X1.  It would be a lovely low-power
server if I could get it running a reasonable OS.

I've attached IDE CD-ROM drives to it in the past to load things, and
while it's possible, it's not fun.  Info on how to load from the
debian distributions over the network would be appreciated.

  - Chris

Re: ZFS root w/ debian sparc64 (was: Re: Installed kernel crash on T5120)

[Chris Ross]

On Sun, Sep 02, 2018 at 10:54:31AM +0200, Romain Dolbeau wrote:
> Any suggestion or idea welcome...
> 
> BTW, did anyone else succeeded in having ZFS root on sparc64?

:-(  I'm sorry to hear about this.  I'd been happy to think that if I backed
off of worrying about a mirrored /boot, and just used a RAID1 as you did, 
that I could follow your example and get it working.

As of now, I haven't tried that though, so I'm behind where you were and even
behind where you are.  I hadn't gotten grub to load from the ZFS root, as
I mentioned in emails a month or so ago.  I have been watching for a newer
installer ISO, and was going to reinstall from scratch the non-ZFS installation
I have on sdd, then try to replicate your success with RAID1 and a ext /boot.

Adrian, or anyone else involved, let us know how getting an updated installer
ISO is going.  And if anyone can help Romain, I'll also be listening intently.

- Chris

Re: Debian sparc64 installer ISO update

[Chris Ross]

On Mon, Aug 20, 2018 at 05:12:33PM +0200, John Paul Adrian Glaubitz wrote:
> I am currently not home but will be back on Wednesday evening (CEST). I will 
> be able to build new images then.

Excellent.  Thank you for the quick response, and all of the work you've done.

 - Chris

> > On Aug 20, 2018, at 4:59 PM, Chris Ross  wrote:
> > 
> > 
> > Adrian, we've seen your work to get things in line up stream.  Thank you for
> > keeping this list in the loop on that work.  Some of that work I know is to
> > support getting an updated installer ISO for sparc64.  I wanted to ask what
> > remains for that.  I've returned from vacation and am anxious to try to 
> > start
> > from scratch on my T5120, but if there's going to be an ISO more recent than
> > the one in mid-May, I can wait.
> > 
> > Thanks.
> > 
> > - Chris
> 

Debian sparc64 installer ISO update

[Chris Ross]

Adrian, we've seen your work to get things in line up stream.  Thank you for
keeping this list in the loop on that work.  Some of that work I know is to
support getting an updated installer ISO for sparc64.  I wanted to ask what
remains for that.  I've returned from vacation and am anxious to try to start
from scratch on my T5120, but if there's going to be an ISO more recent than
the one in mid-May, I can wait.

Thanks.

 - Chris

Re: Installed kernel crash on T5120

[Chris Ross]

On Fri, Jul 20, 2018 at 11:52:19PM +0200, Romain Dolbeau wrote:
> fdisk, force of habits...

lol.  Never would've guessed that was an option.  Well, that looks more like
yours:

Device  Start   End   Sectors   Size Id Type Flags
/dev/sda1   0979964979965 478.5M  1 Boot  
/dev/sda2  979965 286728119 285748155 136.3G 83 Linux native  
/dev/sda3   0 286728119 286728120 136.7G  5 Whole disk

I find it interesting that fdisk shows sda3, but parted does not.  Odd.  The
device does exist in /dev, though, so I guess I believe fdisk.

> I used fdisk and originally set the partition *ID* to unassigned as fdisk
> didn't suggest ZFS. ZFS doesn't seem to care.
> Parted shows a filesystem, not an id (I think).
> fdisk shows the id, not the filesystem.

Right.  As above, mine seems to be id 83.  *shrug*

> Oups sorry red herring - out of habits I always anonymize such files.
> In my installation, the UUID are real values, and I sed'ed them to all-9.

Ahh, great.  That makes sense, thanks for the explanation.

> As for grub - except for the fact my /boot is much smaller (I matched
> the size of the ext4 auto-created partition to the sector), I can't help.
> My first attempt failed - not valid. After trying again, it started working
> and I just had to fix the various issues in configuration and versions
> mentioned in a previous mail. I don't think I did anything special beyond
> slightly fixing the grub.cfg by adding the pool name...

Thanks.  So, I adjusted the UUID I had in my search lines of the menuentry
for booting with ZFS root, as it seemed to be pointing to sdb1 instead of sda1.
But, that just gives me a different problem I think I've also seen before:

error: unable to open
/pci@0/pci@0/pci@1/pci
@0/pci@1/pci@0/usb@0,2/hub@4/device@4/st.
error: unable to open
/pci@0/pci@0/pci@1/pci
@0/pci@1/pci@0/usb@0,2/hub@4/device@4/st.
ERROR: /pci@0: Last Trap: Fast Data Access MMU Miss

{0} ok 

I think I may just throw away what I have now and restart.  I can't immagine
I'll be any worse off, and it might be better starting with a newer base
image than I started with many months ago.  Unfortunately grub is much more
complicated than boot-loaders I've used in the past.  If someone builds a new
sparc64 ISO that I can be sure has the newer grub bits in the installer, I'll
just reload my "normal" image, then try to rebuild the ZFS disks again.

  - Chris

Re: Installed kernel crash on T5120

[Chris Ross]

On Fri, Jul 20, 2018 at 08:47:05AM +0200, Romain Dolbeau wrote:
> t5120:~$ dpkg -l '*grub*' | grep ^ii
> ii  grub-common   2.02+dfsg1-4 sparc64  GRand Unified
> Bootloader (common files)
> ii  grub-ieee1275 2.02+dfsg1-4 sparc64  GRand Unified
> Bootloader, version 2 (Open Firmware version)
> ii  grub-ieee1275-bin 2.02+dfsg1-4 sparc64  GRand Unified
> Bootloader, version 2 (Open Firmware binaries)
> ii  grub2-common  2.02+dfsg1-4 sparc64  GRand Unified
> Bootloader (common files for version 2)

Okay, that matches mine.  Thanks.

> > and show me how your disk is labeled/partitioned?
> 
> Device  Start   End   Sectors   Size Id Type   Flags
> /dev/sdd1   0192779192780  94.1M  1 Boot
> /dev/sdd2  192780 286707979 286515200 136.6G  0 Unassigned
> /dev/sdd3   0 286728119 286728120 136.7G  5 Whole disk
> 
> Pool on sdd2, of course.

Interesting.  What tool is showing that?  I have been using parted, which
shows (for my first ZFS disk):

Number  Start  EndSize   File system  Flags
 1  0.00B  502MB  502MB  ext2 boot
 2  502MB  147GB  146GB  zfs

So, other than the different format, I notice:
1) I show ZFS where your sdd2 shows "Unassigned"
2) You have a 'c' whole disk partition, and I don't.  I wonder if that's
   important?
3) You have an "Id" field [in your tools output] that I don't.

> Available on 

Thanks.  There are a few differences compared to mine, notably that the
"search" lines in your menuentry have a UUID of all 9's, and mine has a
UUID.  And I have hd0/ahci0 vs your hd3/ahci3, but that is expected.  I
also see that I'm due a kernel update (my grub.cfg still lists 4.16.0-2),
but since I'm not getting a kernel loaded at all, that's not my problem.

> BTW, when I installed the system, I had the sdd1 /boot mounted on
> /mnt/boot (ZFS root on /mnt).
> 
> After doing "grub-mkconfig -o /boot/grub/grub.cfg" (in chroot) the
> pool name was missing in grub.cfg, only the dataset name was there. I
> added the pool name by hand.
> 
> Then "grub-install --force  --skip-fs-probe /dev/sdd1" (also in chroot).
> 
> Don't remember anything special beyond that ... and a bit of elbow
> grease :-) (several reboots to add the network interface
> configuration, the console, right klibc, ...).

Yeah.  This is about the same as what I've tried, but I'm not even able
to have grub load from the disk in my case.  I've been thinking about 
booting back to "reinstall the whole thing from CD", since my original
install was many months ago, but haven't done that yet.

Adrian, the ISOs I see still are dated mid-May.  Let me know if you may
have time to regenerate those at some point.

Thanks Romain.

 - Chris

Re: Installed kernel crash on T5120

[Chris Ross]

On Wed, Jul 18, 2018 at 07:34:31AM +0200, Romain Dolbeau wrote:
> 2018-07-18 4:29 GMT+02:00 Chris Ross :
> > Interesting to note, thank you.
> 
> If you have a full build of zfs from source, you can try
> "zfs/cmd/raidz_test/raidz_test -B".
> 
> > I wonder why
> > my grub-install onto my sda1 isn't working, and yours onto your sdd1 is.
> 
> Barely so far. After a couple of trials (and adding the boot partition to 
> fstab,
> among other things), I got to the point where I get the grub menu & grub
> load the kernel... then it blows up because "/etc/zfs/zfs-functions" is 
> missing.
> That file is in the Debian packages (zfsutils-linux I think), but is
> not installed
> by the packages created by 'make deb' from git... so I tried rebuilding the
> Debian sources with "debuild', but that blows up with an error about missing
> "usr/lib/dracut" in "debian/tmp"...

Interesting.  I didn't go quite that far to the root, I took the suggetion
of James Clarke on this list [1], and downloaded the debian source packages
for zfs-linux and built those.  I think that may've saved me some of the
complexities that you came across.

I'm glad to see that you got it all running!  Unfortunately, I'm still stuck
with not being able to load grub from sda1 on my host.  It loads from sdd1,
and loads that disks ext4 installation.  But I'm not even getting it loaded
from sda1 where there is a grub.cfg that knows about the ZFS install.

I tried recently to add an entry to the grub.cfg on my ext layout that would
load the kernel and root from my ZFS volume.  But that only gives me:

  Booting a command list

error: unable to open
/pci@0/pci@0/pci@1/pci
@0/pci@1/pci@0/usb@0,2/hub@4/device@4/st.
error: unable to open
/pci@0/pci@0/pci@1/pci
@0/pci@1/pci@0/usb@0,2/hub@4/device@4/st.
ERROR: /pci@0: Last Trap: Fast Data Access MMU Miss

{0} ok 


so I fear I've configured it incorrectly.  Definately not well versed in
grub, and there may well be things outside of the menuentry I added that are
conflicting with what I put in borrowed from the other disks grub-mkconfig
(run when I was setting it all up in chroot)  The working entry run starts with
"Loading Linux 4.16.0-2-sparc64-smp ...", and this starts with "Booting a
command list".  So.  :-/

Can you confirm the version of grub that you've used to grub-install onto
your disk, and show me how your disk is labeled/partitioned?  Maybe mail me
your grub.cfg off-list, so I can see how that compares to the one I'm trying
to squeeze into the one on my ext disk.

(Side question, can it break things to boot grub and it's config off of one
/boot, then load root and have it configured to load a different /boot?)

- Chris





[1] https://lists.debian.org/debian-sparc/2018/05/msg5.html

Re: Installed kernel crash on T5120

[Chris Ross]

On Tue, Jul 17, 2018 at 03:04:18PM +0200, Romain Dolbeau wrote:
> Well I have ZFS up and running from Git - it has become surprisingly
> easy over the years.
> BTW, from the archive, I see you were intent on RAIDZ. I love RAIDZ,
> but after running the raidz benchmark, I strongly advise against
> anything but mirrors on a T5120 :-( The parity algorithms are really
> slow on that CPU. And looking at the VIS(1+2) instruction set, I doubt
> they can be significantly accelerated (VIS is old and lacks the
> required 8 bits operations, it doesn't even get any kind of shifts
> before VIS3). Unless we can somehow leverage the cryptographic stuff
> (the Modular Arithmetic Unit, MUA) ? But I don't see any documentation
> beyond "go through the Solaris driver" :-(

Interesting to note, thank you.  I had been expecting a 3-disk RAIDZ plus
a spare.  But, I'm sure I can readjust to a pair of mirrors instead.  Just
have to think more about which filesystems go on which volume than I had.

> Anyway for the new Debian I'm combining this page
> 
> and this ML with this:
> sudo debootstrap --no-check-gpg sid /mnt
> http://ftp.ports.debian.org/debian-ports
> /mnt is where the ZFS root is mounted (didn't do anywhere near as many
> FS as the wiki suggests).
> 
> ZFS is on /dev/sdd2, with /dev/sdd1 a "boot partition" similar to the
> one debian-installer added on /dev/sdc, as it is apparently required.

Thanks.  I tried to set up my sda that same way, with ZFS on sda2, and a boot
partition on sda1.  The one installed on my /dev/sdd by the installer looks
the same, just with two ext partitions and swap on sdd2, sdd4, and sdd5.  But,
as noted recently, that doesn't work.  Are you installing with the grub-install
that's in grub-common + grub-ieee1275 version 2.02+dfsg1-4 ?  I wonder why
my grub-install onto my sda1 isn't working, and yours onto your sdd1 is.
Can you show me the parted for your sdd?  My sda is:

Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start  EndSize   File system  Flags
 1  0.00B  502MB  502MB  ext2 boot
 2  502MB  147GB  146GB  zfs

Thanks.

- Chris

Re: Installed kernel crash on T5120

[Chris Ross]

On Mon, Jul 16, 2018 at 09:32:57PM +0200, Romain Dolbeau wrote:
> Next step - ZFS followed by trying to deboostrap a new install in a ZFS root 
> :-)

Thank you for your email, too.  I have a T5120 that I was able to get ZFS
on, with help from this list.  But I am having GRUB problems myself, as noted
in an email yesterday.  I hope that you are able to make progress and that
I can learn from it as well.  :-)

If you search the list archives for the last few months, you'll see some
of the threads where I was asking about getting ZFS downloaded, built,
installed.  I can't boot into it yet, but in theory have a ZFS system all
installed atm.

   - Chris

Grub confusion

[Chris Ross]

Putting aside briefly trying to get grub installed onto an md0 on my sparc,
I tried just getting it on a single device that would then load up the
zfs filesystems.  First, getting it running on sdd, is working.  At least,
it works until I try to install onto sda, which seems to (often, though maybe
not 100% of the time) mess up the grub on sdd as well.  :-/

I have sdd on thie T5120, which has 4 identical disks, and I got grub installed
onto it.  /boot is in sdd1, / in sdd2, and others.

Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sdd: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start   End SizeFile system Flags
 1  0.00B   312MB   312MB   ext2boot
 2  313MB   12.3GB  12.0GB  ext4
 4  30.1GB  46.5GB  16.5GB  linux-swap(v1)
 5  46.5GB  147GB   100GB   ext4

Then I broke down the md that I'd put across sda1 and sdb1, and just have
those as ext2 filesystems.  ZFS on the rest of the disk.

Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start  EndSize   File system  Flags
 1  0.00B  502MB  502MB  ext2 boot
 2  502MB  147GB  146GB  zfs

And, the grub/grub.cfg in that /boot looks right when I built it.  Booting
off of disk3, sdd, has been working for a while.  So, to try sda again, I ran
(while booted up on my installation on sdd):

grub-install --force --skip-fs-probe /dev/sda1

But, unlike the grub on sdd1, it doesn't boot.  Trying to load up from
OBP I see:

-
{0} ok boot disk0
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0  File and args: 
GRUB Loading kernel
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
UNKNOWN DEVICE: ieee1275//pci@0/pci@0/pci@2/scsi@0/disk@3\,0:a
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
ERROR: /pci@0: Last Trap: Fast Data Access MMU Miss

{0} ok 
-

Now, if I then try to boot disk3 again, which has been working just fine since
the last time I recovered it by reinstalling grub using the rescue CD, I see:

-
{0} ok boot disk3
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@3  File and args: 
GRUB ERROR: Last Trap: Illegal Instruction

{0} ok 
-

(tried consecutive "boot disk3", with same results each time.)

Rebooting from a rescue cd and executing a shell in sdd2 (with sdd1),
I can run:

root@t5120:/# grub-install --force --skip-fs-probe /dev/sdd1
Installing for sparc64-ieee1275 platform.
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun2).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun4).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun5).
grub-install: warning: Attempting to install GRUB to a disk with multiple 
partition labels.  This is not supported yet..
grub-install: warning: Embedding is not possible.  GRUB can only be installed 
in this setup by using blocklists.  However, blocklists are UNRELIABLE and 
their use is discouraged..
Installation finished. No error reported.
root@t5120:/# 

..and then reboot, and it will load up fine from sdd/disk3.

Any idea why grub-install onto sda1 isn't working for me?  Or more, how an
attempt there could be messing up the grub installation on sdd1?

Appreciate any ideas or pointers.  Thanks all.

- Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 04:43:32PM -0400, Chris Ross wrote:
> [- tried to grub-install onto /dev/sda1 and /dev/sda2, won't boot, Illegal
>Instruction. -]

> So, I fear some of this is based on the fact that I'm running grub-install
> into raid filesystems that are mounted as a RAID1, but I'm sticking things
> into them at the same time.  Or maybe it's more complicated, but the fact
> that I'm writing into the raw partitions under an md RAID certainly seems
> of concern.  :-/

So I'm even more confused now.  I was unable to boot disk0 or disk1, but
also got errors booting off of disk3, which had been working.  So I booted
into rescue cd, re-ran the grub-install onto /dev/sdd1, then rebooted.
Came up fine, as before.  When it did, I did nothing but log in, then 
re-run "grub-install --force --skip-fs-probe" onto each of /dev/sda1 and
/dev/sda2, while /dev/md0 wasn't mounted.  Nothing on the first three
disks had been accessed since boot.

Sadly, this did not make those disks bootable either.  But, surprisingly,
I now can't boot off of disk3 again.  Even after a "reset-all", I see:

Boot device: disk3  File and args: 
GRUB ERROR: Last Trap: Illegal Instruction

So, I've no idea how merely running grub-install onto /dev/sda1 and
/dev/sdb1 could possibly have messed up the grub on /dev/sdd1?  I am booting
into rescue cd again now to repair that, but I'm starting to feel that
grub is acting for some evil spirit and I am failing to meet whatever
it's needs are

- Chris

ps, re-grub-install'ing to /dev/sdd1 allowed the system to boot again.
I rebooted a few times, seems to have remained functional.  I also after
a few successful boots to sdd (aka disk3), tried booting from PROM to
disk0 again, got the Illegal Instruction, but was then still able to 
boot disk3.

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

Sorry for the chatter on the list, all...

On Tue, Jul 03, 2018 at 04:14:18PM -0400, Chris Ross wrote:
> Adrian answered some of this before I asked the above questions:
> 
> t5120# grub-install /dev/sda1
> Installing for sparc64-ieee1275 platform.
> grub-install: warning: File system `ext2' doesn't support embedding.
> grub-install: error: embedding is not possible, but this is required for RAID 
> and LVM install.
> t5120# grub-install --force --skip-fs-probe /dev/sda1
> Installing for sparc64-ieee1275 platform.
> grub-install: warning: File system `ext2' doesn't support embedding.
> grub-install: error: embedding is not possible, but this is required for RAID 
> and LVM install.
> t5120# 

Okay.  So, I see the above error from grub-install, when running in the
chroot, for both /dev/sda1, _and_ for /dev/sdd1 which worked fine for the
real disks I'm running on, and booted from.  From the full machine, I can
also successfully grub-install onto /dev/sda1 (and /dev/sdb1).  I don't
know why that is different when running in the chroot onto the target area.

Rebooting now, to see if I can load a working grub from disk0, and what
it can load.  Hmm.  Okay, clearly some problems there...

Boot device: disk3  File and args: 
[halt sent]

{0} ok boot disk0
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0  File and args: 
GRUB Loading kernel...
ERROR: /pci@0/pci@0/pci@2/scsi@0: Last Trap: Illegal Instruction

{0} ok boot disk1

Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@1  File and args: 
GRUB Loading kernel
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
UNKNOWN DEVICE: ieee1275//pci@0/pci@0/pci@2/scsi@0/disk@3\,0:a
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
error: unable to open /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@
4/st.
ERROR: /pci@0: Last Trap: Fast Data Access MMU Miss

{0} ok boot disk0

Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0  File and args: 
GRUB Loading kernel...
ERROR: Last Trap: Illegal Instruction

{0} ok 

So, I fear some of this is based on the fact that I'm running grub-install
into raid filesystems that are mounted as a RAID1, but I'm sticking things
into them at the same time.  Or maybe it's more complicated, but the fact
that I'm writing into the raw partitions under an md RAID certainly seems
of concern.  :-/

Again, let me know any thoughts.  Thanks.

- Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 03:37:24PM -0400, Chris Ross wrote:
> Sorry, stupid user trick.  I had mdadm and friends in the disk I was running
> on, but had _not_ installed those packages into the environment in md/ZFS
> that I chrooted into.  After installing mdadm (and the collection it brought),
> I am able to update-grub.  And grub-probe gives me the answer I'd expect:
> 
> But, I'm seeing the following if I try to grub-install:
> 
> (chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0
> Installing for sparc64-ieee1275 platform.
> grub-install: warning: File system `ext2' doesn't support embedding.
> grub-install: warning: Embedding is not possible.  GRUB can only be installed 
> in this setup by using blocklists.  However, blocklists are UNRELIABLE and 
> their use is discouraged..
> zsh: segmentation fault  grub-install --force --skip-fs-probe /dev/md0
> 
> (1) Is this supposed to work?
> (2) Assuming it's not actually installed correctly yet, booting will fail.
> Should I grub-install onto /dev/sda1 or /dev/sda2?  Whether one or both,
> won't that mess up the RAID of those devices?

Adrian answered some of this before I asked the above questions:

On Tue, Jul 03, 2018 at 09:39:56PM +0200, John Paul Adrian Glaubitz wrote:
> Well, you need to tell GRUB to install on the individual disks
> and not on the RAID (md0), this won't work - with any boot loader.
> [...]
> Don't use /dev/md0, use /dev/sd* and install on any of the disks that
> are part of the RAID. You cannot boot from /dev/md0 as the kernel needs
> to be running to be able to access the software RAID device.

Okay.  That all makes sense, thank you.  Two problems at this point
(1) The segv above.  I'm doing something wrong, but it shouldn't SEGV
(2) Installing onto /dev/sda1 doesn't work.

t5120# grub-install /dev/sda1
Installing for sparc64-ieee1275 platform.
grub-install: warning: File system `ext2' doesn't support embedding.
grub-install: error: embedding is not possible, but this is required for RAID 
and LVM install.
t5120# grub-install --force --skip-fs-probe /dev/sda1
Installing for sparc64-ieee1275 platform.
grub-install: warning: File system `ext2' doesn't support embedding.
grub-install: error: embedding is not possible, but this is required for RAID 
and LVM install.
t5120# 

Is there some other options for grub-install I'm missing?  Embedding
(grub-install /dev/sda) isn't possible with sun partition labels, but the
above message suggests that embedding is required for RAID install.

Thanks...

 - Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 03:24:12PM -0400, Chris Ross wrote:
> Okay.  So, in prepping a chroot'd md/zfs environemnt on this machine, while
> updating the kernel packages, I see:
> 
> /etc/kernel/postinst.d/zz-update-grub:
> Generating grub configuration file ...
> Found linux image: /boot/vmlinuz-4.16.0-2-sparc64-smp
> Found initrd image: /boot/initrd.img-4.16.0-2-sparc64-smp
> /usr/sbin/grub-probe: error: disk `md0' not found.
> /usr/sbin/grub-probe: error: disk `md0' not found.
> /usr/sbin/grub-probe: error: disk `md0' not found.
> Found linux image: /boot/vmlinuz-4.16.0-1-sparc64-smp
> Found initrd image: /boot/initrd.img-4.16.0-1-sparc64-smp
> /usr/sbin/grub-probe: error: disk `md0' not found.
> /usr/sbin/grub-probe: error: disk `md0' not found.
> Found Debian GNU/Linux buster/sid on /dev/sdd2
> done
> 
> This leads me to the same problem I had earlier, grub (grub-probe or
> grub-install) saying "error: disk `md0' not found."  I think this is why
> I started down a path of looking for an alternative grub2.

Sorry, stupid user trick.  I had mdadm and friends in the disk I was running
on, but had _not_ installed those packages into the environment in md/ZFS
that I chrooted into.  After installing mdadm (and the collection it brought),
I am able to update-grub.  And grub-probe gives me the answer I'd expect:

(chroot) root@t5120# grub-probe -d /dev/md0
ext2

But, I'm seeing the following if I try to grub-install:

(chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0
Installing for sparc64-ieee1275 platform.
grub-install: warning: File system `ext2' doesn't support embedding.
grub-install: warning: Embedding is not possible.  GRUB can only be installed 
in this setup by using blocklists.  However, blocklists are UNRELIABLE and 
their use is discouraged..
zsh: segmentation fault  grub-install --force --skip-fs-probe /dev/md0

(1) Is this supposed to work?
(2) Assuming it's not actually installed correctly yet, booting will fail.
Should I grub-install onto /dev/sda1 or /dev/sda2?  Whether one or both,
won't that mess up the RAID of those devices?

   - Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 12:22:34PM -0400, Chris Ross wrote:
> I'll run with this for a bit, and respond back with questions about getting
> it installed onto software-raid partitions later.  Thanks again.

Okay.  So, in prepping a chroot'd md/zfs environemnt on this machine, while
updating the kernel packages, I see:

/etc/kernel/postinst.d/zz-update-grub:
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.16.0-2-sparc64-smp
Found initrd image: /boot/initrd.img-4.16.0-2-sparc64-smp
/usr/sbin/grub-probe: error: disk `md0' not found.
/usr/sbin/grub-probe: error: disk `md0' not found.
/usr/sbin/grub-probe: error: disk `md0' not found.
Found linux image: /boot/vmlinuz-4.16.0-1-sparc64-smp
Found initrd image: /boot/initrd.img-4.16.0-1-sparc64-smp
/usr/sbin/grub-probe: error: disk `md0' not found.
/usr/sbin/grub-probe: error: disk `md0' not found.
Found Debian GNU/Linux buster/sid on /dev/sdd2
done

This leads me to the same problem I had earlier, grub (grub-probe or
grub-install) saying "error: disk `md0' not found."  I think this is why
I started down a path of looking for an alternative grub2.

Googling for this error leads me to many people that had had this error,
sometimes an old double-free problem that isn't hitting me,
and problems with one-disk md RAID arrays, which isn't my issue.  But I
don't see anyone with a solution to my problem without the others.  I
fear I just don't know what I'm looking for.

If anyone else is more familiar with using grub, and has any idea
how to work around this current situation, I'd appreciate a pointer.

(chroot) root@t5120# grub-install --force --skip-fs-probe /dev/md0 
Installing for sparc64-ieee1275 platform.
grub-install: error: disk `md0' not found.
(chroot) root@t5120# ls -l /dev/md*
brw-rw 1 root disk  9,  0 Jul  3 14:26 /dev/md0
crw--- 1 root root 10, 62 Jul  3 12:19 /dev/mdesc
(chroot) root@t5120# df /boot
Filesystem 1K-blocks  Used Available Use% Mounted on
/dev/md0  458396 63776366432  15% /boot
(chroot) root@t5120# 

Thanks.

- Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 12:04:09PM -0400, Chris Ross wrote:
> It sounds from Eric like the grub I built from him doesn't want to deal with
> gzip.  But the one shipping (2.02+dfsg1-4) does?  Well, easy enough to try
> that on the same disk to see.  But, I presume I will face the same problem
> with it not being willing to install onto md0.  I'll face that later.
> 
> Back shortly with data on Debian's grub 2.02+dfsg1-4 results.

Okay.  Took me a couple attempts, and the grub 2.02+ is a lot more alarming
in what it says:

% sudo grub-install --force --skip-fs-probe /dev/sdd1
Installing for sparc64-ieee1275 platform.
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun2).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun4).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun5).
grub-install: warning: Attempting to install GRUB to a disk with multiple 
partition labels.  This is not supported yet..
grub-install: warning: Embedding is not possible.  GRUB can only be installed 
in this setup by using blocklists.  However, blocklists are UNRELIABLE and 
their use is discouraged..
Installation finished. No error reported.
% 

But, after getting that installed, the machine rebooted with the default
(vmlinuz, gzip'd) kernel.  Thought i do notice before it puts up the initial
boot menu:

Boot device: disk3  File and args: 
GRUB Loading kernel
error: out of memory.
error: no suitable video mode found.

...then the screen clears and shows the boot menu, and when left alone to
boot the default kernel, it succeeds.  So, I'm curious to read the conversation
spawned by your comment back on the bug I'd opened in grub2-sparc in github.

I'll run with this for a bit, and respond back with questions about getting
it installed onto software-raid partitions later.  Thanks again.

- Chris

Re: Grub, sparc64, and compressed kernels

[Chris Ross]

On Tue, Jul 03, 2018 at 05:46:18PM +0200, John Paul Adrian Glaubitz wrote:
> Hi Chris!
> 
> Any reason why you aren't just using the normal Debian procedure to setup
> and install GRUB?

Well, IIRC, because it wasn't working for me.  But, I must admit, I'm not 100%
sure I tried that on the simple disk.  I know I was having no luck getting it
installed on md0 when I tried, and eventually punted back to trying on sdd
just to get something working.  I am no longer 100% sure I hadn't already
switched to trying https://github.com/esnowberg/grub2-sparc/ before that.

> Debian stores its kernel compressed by default and I don't see how this is
> supposed to be a problem. All SPARCs that we have installed with Debian
> for sparc64 are using the grub2 package from Debian with a compressed
> kernel.

It sounds from Eric like the grub I built from him doesn't want to deal with
gzip.  But the one shipping (2.02+dfsg1-4) does?  Well, easy enough to try
that on the same disk to see.  But, I presume I will face the same problem
with it not being willing to install onto md0.  I'll face that later.

Back shortly with data on Debian's grub 2.02+dfsg1-4 results.

- Chris

Grub, sparc64, and compressed kernels

[Chris Ross]

As I was mentioning in other threads, I was getting grub installed onto the
"normal" sun-labeled ext2/4 partitioned disk in my T5120.  Working with
Eric (https://github.com/esnowberg/grub2-sparc/issues/14) he helped me
figure out that grub wasn't working because the kernel in /boot was gzip'd.

Is there something that's told Debian that I want my kernels in /boot to be
gzip'd?  Is there a way to make it _not_ do that, since grub apparently
needs them not to be gzip'd on this machine?

Thanks.

 - Chris

Re: GRUB is now the default bootloader on sparc64

[Chris Ross]

On Fri, May 18, 2018 at 01:44:45PM +0200, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> I have now disabled SILO support in debian-installer meaning that
> it's no longer possible to install the SILO bootloader when installing
> Debian on sparc64.
> 
> [...]
> 
> GRUB is fully supported on sparc64 and, from the various feedback we got
> on the mailing list, should be compatible with even the oldest UltraSPARC
> machines. So it will work on any machine that run Debian's sparc64 port.

Hello Adrian, all.  I have been working on getting grub2 running as a step
in getting my t5120 running, as the list has been hearing from me.  I was/am
building from https://github.com/esnowberg/grub2-sparc, using the guide
https://github.com/esnowberg/grub2-sparc/wiki.  But, this makes me wonder,
what is it you've included in the installer if the core grub2 code is not
yet inclusive of these changes?  (I did try to use the grub2 2.02++ that
was on the host, but wasn't able to get it to install, and shifted to the
above.)

Thanks.  I have enough balls in the air on this machine that I'm starting to
think I might be better starting from scratch with grub2 now the default.
I don't know where to find built ISO's since approximately the date of this
email, so I want to ensure the above is effective in an ISO before trying
to reinstall from such.

- Chris

Re: updaing to grub on sparc64

[Chris Ross]

On Mon, Jun 11, 2018 at 08:00:13AM +0200, John Paul Adrian Glaubitz wrote:
> Could you please have this discussion in a separate thread and not hi-jack 
> Chris’ thread for that?

Thanks Adrian, and forgiven Phillip.  Back to my question, does noone have
grub installed on a sun labeled sparc64?  Could still use help on that issue...

- Chris

Re: updaing to grub on sparc64

[Chris Ross]

On Sat, Jun 09, 2018 at 12:06:29PM -0400, Chris Ross wrote:
> Is it any harder than just
> "apt install grub-ieee1275", then "grub-install /dev/sdd"?  And for a sun
> disk with /boot on sdd1 and / on sdd2, should I target grub-install to
> sdd1 or sdd2 (assuming that /dev/sdd, despite my mention above, is likely to
> not work on sparc)?  And which partition should I point OBP to load?

Sorry for the noise, I should've just finished the first experiements first.
installing grub-common and grub-ieee1275 allowed me to run grub-install, but
each of "grub-install /dev/sdd" and "grub-install /dev/sdd1" yield approximately
the same result:

root@t5120:/# grub-install /dev/sdd1
Installing for sparc64-ieee1275 platform.
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun2).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun4).
grub-install: warning: Discarding improperly nested partition 
(hostdisk//dev/sdd,sun1,sun5).
grub-install: error: hostdisk//dev/sdd appears to contain a sun partition map 
which isn't known to reserve space for DOS-style boot.  Installing GRUB there 
could result in FILESYSTEM DESTRUCTION if valuable data is overwritten by 
grub-setup (--skip-fs-probe disables this check, use at your own risk).
root@t5120:/# 

(the first three warnings occur when I select /dev/sdd1, and not when I select
/dev/sdd, but otherwise the same output)

This is a sun label disk, partitioned like so:

Partition Table: sun

Number  Start   End SizeFile system Flags
 1  0.00B   98.7MB  98.7MB  ext2boot
 2  98.7MB  30.1GB  30.0GB  ext4
 4  30.1GB  46.5GB  16.5GB  linux-swap(v1)
 5  46.5GB  147GB   100GB   ext4

(parted)  

sdd1 is /boot, 2 is /, 5 is /home.

>From the earlier conversation, I know it was tested using grub with sun labeled
disks.  Does there have to be special magic space at the front to make it work?

Thanks.

updaing to grub on sparc64

[Chris Ross]

This list was discussing grub on sparc64 recently, and IIRC, was suggesting that
that should now be the recommended bootloader.  I loaded silo on the disk I
installed earlier this year, but was expecting to use grub when I get it booting
off of ZFS (or mostly ZFS).  I am trying to install grub onto my one-disk
prototype system that I have silo on now.  Is it any harder than just
"apt install grub-ieee1275", then "grub-install /dev/sdd"?  And for a sun
disk with /boot on sdd1 and / on sdd2, should I target grub-install to
sdd1 or sdd2 (assuming that /dev/sdd, despite my mention above, is likely to
not work on sparc)?  And which partition should I point OBP to load?
IIRC, it's loading just sdd now with/for silo, so the root of /dev/sdd1.

I'll be experimening with this momentarily, just thought I'd ask in case there
were pointers already somewhere I hadn't found.  Google for debian sparc mostly
talks about silo, obviously, since that was the normal choice.

Thanks.

   - Chris

Re: sparc64 crc32c opcode not available

[Chris Ross]

On Wed, May 30, 2018 at 11:34:18AM +0200, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> > [27482.619730] Kernel unaligned access at TPC[120bc270] 
> > sa_build_layouts+0x5f0/0x8c0 [zfs]
> 
> This is a bug in the ZFS kernel code which should be reported upstream.

Yup.  I've done that, at least sent email to that list.

> > [18307.649038] crc32c_sparc64: sparc64 crc32c opcode not available.
> 
> This just means that your CPU doesn't support this particular instruction,
> this is merely an information and not an indicator of a problem.

Okay.  If it happens alot, the noise might be considered a problem in and
of itself, but for now I'll just keep an eye on it and know it's informational.
Thanks!

  - Chris

sparc64 crc32c opcode not available

[Chris Ross]

Hey.  I have my T5120 running, recently updated packages including kernel
(4.16.0-1-sparc64-smp).  On the system's console I'm seeing two messages
of concern, periodically:

[27482.619730] Kernel unaligned access at TPC[120bc270] 
sa_build_layouts+0x5f0/0x8c0 [zfs]

and

[18307.649038] crc32c_sparc64: sparc64 crc32c opcode not available.

The former I'll need to ask the zfs-linux folks about, but let me know if you
already know anything about it.

The second is the real question for this list.  Is this something you've seen
before?  Is it indicative of a real problem, or just noisy notification of
something that causes no real problem?

 - Chris

parted problems making partitions on sun disks

[Chris Ross]

This came up in my efforts to build a ZFS-booting sparc.  But, I want to pull
out some my question about disk partitioning into a separate discussion.  I
don't know of a parted mailing list, and I suspect this is specific to sun
disk label anyway, so debian-sparc seemed the best place to ask.

I have a T5120 with 4 140GB disks.  I want to put a md /boot on the first two,
then use the remainder of the disks for other filesystem(s).  However, I am
getting what seems to me to be strange messages from parted, and am not sure
how to get what I'm looking for.

First, make a sun disk label.  No problems.


(parted) mklabel sun
Warning: The existing disk label on /dev/sda will be destroyed and all data on
this disk will be lost. Do you want to continue?
Yes/No? y
(parted) p
Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start  End  Size  File system  Flags

(parted)


Then, make a 500MB partition.  Again, I have no troubles


(parted) mkpart ext4 0 500m
(parted) p
Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start  EndSize   File system  Flags
 1  0.00B  500MB  500MB  ext4

(parted)


The problem comes when I try to add a partition after that:


(parted) mkpart ext4 500m 100%
Warning: You requested a partition from 500MB to 147GB (sectors
976562..286739328).
The closest location we can manage is 502MB to 147GB (sectors
979965..286728119).
Is this still acceptable to you?
Yes/No? yes
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? cancel
(parted)


The thing that concerns me most, is that it suggests it should _start_ this
new parition on an odd-numbered sector.  I can't immagine how that would ever
make a properly aligned partition.  At least, if any alignment outside of 
single-sector was required, which is clearly is based on the next message

Below are a few more things I tried, including manually calculating 2048s
alignment, and trying to tell parted to do that, which only makes it tell
me it can't manage that.  Whah?

I would appreciate any pointers.

- Chris




(parted) unit s
(parted) p
Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 286739329s
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start  End  Size File system  Flags
 1  0s 976895s  976896s  ext4

(parted) mkpart ext4 976896s 25%
Warning: You requested a partition from 976896s to 71684832s (sectors
976896..71684832).
The closest location we can manage is 979965s to 71682029s (sectors
979965..71682029).
Is this still acceptable to you?
Yes/No? n
(parted) mkpart ext4 980992s 25%
Warning: You requested a partition from 980992s to 71684832s (sectors
980992..71684832).
The closest location we can manage is 979965s to 71682029s (sectors
979965..71682029).
Is this still acceptable to you?
Yes/No? n
(parted) mkpart ext4 980992s 7167s
Warning: You requested a partition from 980992s to 7167s (sectors
980992..7167).
The closest location we can manage is 979965s to 71682029s (sectors
979965..71682029).
Is this still acceptable to you?
Yes/No? n
(parted) mkpart ext4 980992s 7167s
Warning: You requested a partition from 980992s to 7167s (sectors
980992..7167).
The closest location we can manage is 979965s to 71682029s (sectors
979965..71682029).
Is this still acceptable to you?
Yes/No? y
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? i
(parted) p
Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 286739329s
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  StartEndSize   File system  Flags
 1  0s   976895s976896sext4
 2  979965s  71682029s  70702065s  ext4

(parted)

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Sat, May 26, 2018 at 09:19:20PM +0200, John Paul Adrian Glaubitz wrote:
> You can just mirror the partition to each disk, then GRUB can just boot
> from any of the disks.

Thanks.  I started down this path yesterday, using mdadm.  I assume that's
what you mean by mirror above?  If so, I'm good, I was able to figure out
how to get that done.  (sun partition question remains*)

> The Sun partition table is only needed for a given disk if you want
> to be able to boot from that disk. The kernel itself supports any
> of the partition tables that parted supports.
> [...]
> If all of the disks shall be bootable from GRUB, they have to have a Sun
> partition table.

Right.  I figured that, that it's an "OBP doesn't know GPT", which is fine.
I'm back to all sun labels now, working on laying out my disks again.

> But you should maybe post your GRUB-specific questions on the GRUB mailing
> list. I have never tried ZFS as a boot partition so I don't really know
> which configurations are supported.

Thanks.  I will do that when I get back to grub.

Another question I will ask here, because despite being mostly a parted
question, it is I suspect specific to the sun label.

I've made a 500mb software-raid (md) partition (side question: what is a good
size for /boot?  10s or 100s of mb?  500m seems plenty large.), and now am
trying to make the "rest of disk" zfs partition.  However, I can't figure out
how to make a partition that parted thinks is aligned.  I'm sure this is just
me not knowing these tools well, but I can't for the life of me find a way
to get it to actually make the partition where I want it to (without "the
closest location we can manage is..." or get it to feel it's aligned.  Web
research suggests making calulations based on
/sys/block/sdb/queue/optimal_io_size, but that's 0 for me.  I should find
a parted list to ask on, but if anyone here has advice for using parted to
make partitions parted is happy with on a sun disk. Let me know.

  - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Sat, May 26, 2018 at 06:15:30PM +0200, John Paul Adrian Glaubitz wrote:
> That depends on what someone means when they say "GRUB supports ZFS". Is
> it supported on Sun partition tables or on GPT partition tables? Do you
> still need a separate BIOS partition or not? Does it work differently
> on x86 systems and SPARC machines and so on?
> 
> There are so many possible configurations that GRUB support that you cannot
> derive the fact that your particular configuration works or not from a
> generic statement. GRUB works quite differently on different platforms.

Understood.  That's why I said I was confused in the space between multiple
statements.  I don't expect to ever understand all of the variables, but am
also hoping I don't need to.  :-)

> Just add a boot partition for GRUB. It doesn't have to be large.

Okay.  But, as I mentioned, I don't want to have anything that exists on
only one spinning disk.  Is there a way to mirror a partition onto a second
disk that can still be something GRUB can load to/from?  It doesn't have to
be ZFS, but it does have to be replicated so that if a disk fails, I can
boot from another.

> >> Btw, I'm surprised to read that the T5120 supports GPT disk labels? I
> >> thought that GPT is supported on T4 and newer only.
> > 
> > I believe the same, I don't know what that error message means.  I know that
> > I do have sun labels, not GPT labels, on my disks.
> 
> Can you print your partition layout with parted and post it here?

Attached.  Oh.  I see now that I was wrong, only the fourth disk (what I did
the initial install onto, and plan to make a ZFS spare) is sun.  The other
three are gpt.  So, may be totally unusable anyway.  But, it explains the
message grub gave me.  Clearly I've gone astray somewhere. :-)

So, if we presume that my T5120 won't handle gpt disks, then I'll have to
repartition the first three disks back to sun.  At which point, I may or
may not have the exact same problem with grub if I make them only have one
bit zfs partition.  Before I do that, I'm happy to hear suggestions of what
partitions to put on those disks when switching them to a sun partition table.

 - Chris

Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   EndSizeFile system  Name  Flags
 1  1049kB  147GB  147GB   zfs  zfs-4c7e86efe57e2048
 9  147GB   147GB  8389kB


Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sdb: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   EndSizeFile system  Name  Flags
 1  1049kB  147GB  147GB   zfs  zfs-31469670368b53e5
 9  147GB   147GB  8389kB


Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sdc: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   EndSizeFile system  Name  Flags
 1  1049kB  147GB  147GB   zfs  zfs-b9a4576a0888cf77
 9  147GB   147GB  8389kB


Model: SEAGATE ST914603SSUN146G (scsi)
Disk /dev/sdd: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: sun
Disk Flags: 

Number  Start   End SizeFile system Flags
 1  0.00B   98.7MB  98.7MB  ext2boot
 2  98.7MB  30.1GB  30.0GB  ext4
 4  30.1GB  46.5GB  16.5GB  linux-swap(v1)
 5  46.5GB  147GB   100GB   ext4

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Fri, May 25, 2018 at 11:39:49PM +0200, John Paul Adrian Glaubitz wrote:
> On 05/25/2018 09:38 PM, Chris Ross wrote:
> > root@t5120:/var/tmp# grub-install /dev/sda
> > Installing for sparc64-ieee1275 platform.
> > grub-install: warning: this GPT partition label contains no BIOS Boot 
> > Partition; embedding won't be possible.
> > grub-install: error: filesystem `zfs' doesn't support blocklists.
> > root@t5120:/var/tmp# 
> 
> That error message is quite clear: You have to set up a separate non-ZFS
> partition which serves as a boot partition. Booting from a ZFS partition
> using blocklists doesn't work.

Though, the conversation earlier covered that grub supports ZFS.  You say above
that booting from a ZFS partition using blocklists doesn't work.  There's
something I'm not understanding in the space across those two statements.

FreeBSD has a zfsbootloader, which is what I've been using.  And, whether
I can use ZFS or not, I don't think I want to start down the path of booting
from magnetic media that isn't replicated onto another spindle.  If debian
will let me boot from ZFS, how do I do it?  If it won't, are there other options
to boot from mirrored filesystems?

> Btw, I'm surprised to read that the T5120 supports GPT disk labels? I
> thought that GPT is supported on T4 and newer only.

I believe the same, I don't know what that error message means.  I know that
I do have sun labels, not GPT labels, on my disks.

  - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Fri, May 25, 2018 at 09:53:04PM +0200, Frans van Berckel wrote:
> > root@t5120:/var/tmp# grub-install /dev/sda
> > Installing for sparc64-ieee1275 platform.
> > grub-install: warning: this GPT partition label contains no BIOS Boot
> > Partition; embedding won't be possible.
> > grub-install: error: filesystem `zfs' doesn't support blocklists.
> > root@t5120:/var/tmp# 
> 
> And did you create a BIOS boot partition? ;-)
> 
> https://github.com/esnowberg/grub2-sparc/wiki

Well, no, of course I didn't.  :-/  The recent conversations on the list about
grub and such failed to convey to my brain that I needed to.  But, if grub
supports zfs (which I thought it did), can I install grub into a zfs partition?
right now, I just have one big zfs partition that's part of a zraid.  I
understood that grub supported zfs.  So, I guess i'm not clear on what I need
to do to "create a BIOS boot partition".  nb, I _don't_ want to have a single
filesystem on a single disk, I want to be able to handle disk failure, thus
much the point of root-on-ZFS.

Thanks.

 - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

Okay.  Got debootstrap to install onto my ZFS filesystems.  After chroot'ing,
and installing various packages (including klibc-utils which I had to build
from source, still don't know why) I have all of the zfs packages installed.
Installed grub2, but seem to be unable to get it to install onto the zfs
disk(s):

root@t5120:/var/tmp# grub-install /dev/sda
Installing for sparc64-ieee1275 platform.
grub-install: warning: this GPT partition label contains no BIOS Boot 
Partition; embedding won't be possible.
grub-install: error: filesystem `zfs' doesn't support blocklists.
root@t5120:/var/tmp# 

Googling'ing around finds lots of i386-pc vs x86_64-efi comments, where the
latter works and the former comtains with an error as I see above.  But,
I don't know that I should be seeing similar issues.  For that matter, I don't
have a GPT partition, I have sun.  Am I simply missing the right command-line
arguments to grub-install, or is something else wrong?

Thanks.

   - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Wed, May 23, 2018 at 02:27:36PM -0400, Chris Ross wrote:
> Oh!  Okay, if I run it with a target of "./sid-sparc64-sbuild" from my home
> directory, it works.  Or at least, retrieves and begins to validate packages.
> If I run it with a target of /z, the ZFS filesystems I want to install onto,
> it gives the "Invalid Release file" error.

It was looking at something in /z/var/lib/apt or /z/var/cache/apt.  I saw those
directories in the "sid-sparc64-sbuild", and then nuked them in the proto
filesystems.  It's moving along now.  Thanks, will report more later.

-Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Wed, May 23, 2018 at 02:21:40PM -0400, Chris Ross wrote:
> Okay.  I tried that command, with sudo and without a --variant or --arch, and
> got the same failure as earlier:
> 
> % sudo debootstrap --no-check-gpg sid /z http://ftp.de.debian.org/debian-ports
> I: Target architecture can be executed
> E: Invalid Release file, no entry for main/binary-sparc64/Packages

Oh!  Okay, if I run it with a target of "./sid-sparc64-sbuild" from my home
directory, it works.  Or at least, retrieves and begins to validate packages.
If I run it with a target of /z, the ZFS filesystems I want to install onto,
it gives the "Invalid Release file" error.

That help in figuring out what I need to do?  I presume debootstrap is able
to install into created/mounted filesystems, I suspect there's just something
"off" somewhere that's confusing it.

-Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Wed, May 23, 2018 at 07:54:15PM +0200, John Paul Adrian Glaubitz wrote:
> 
> I always use:
> 
> $ debootstrap --no-check-gpg --arch=sparc64 --variant=buildd sid 
> sid-sparc64-sbuild http://ftp.de.debian.org/debian-ports
> 
> You can use a different variant for your usecase, of course or just omit
> this parameter.

Okay.  I tried that command, with sudo and without a --variant or --arch, and
got the same failure as earlier:

% sudo debootstrap --no-check-gpg sid /z http://ftp.de.debian.org/debian-ports
I: Target architecture can be executed
E: Invalid Release file, no entry for main/binary-sparc64/Packages

And James Clarke wrote:
> The second command should work. Do you have gunzip or unxz in your PATH? You
> need one of them for debootstrap to let you use that Packages.?? file, and
> there's no uncompressed version. 

I have both gunzip and unxz in my path.

It's not running it as root with sudo that's the problem, I assume.  Other
than that, Adrian, I think I'm doing the same as you but it's not working.

   - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Tue, May 22, 2018 at 10:56:49PM -0400, Chris Ross wrote:
> Booting an installer may not work, though, since I need ZFS support in the
> system I'm running.  To get a d-i with ZFS built in, I presume I'd have to
> build my own.  Which I could, but don't know that I know how.  If I can use
> something like debian-installer, I'd like to, but need one that can mount
> up my ZFS filesystems.  Otherwise, I guess I need to find a guide of how to
> do it the lower-level way.  ([1] looks valuable)
> 
> [1] https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS

Well, I tried running debootstrap, but I don't know how to point it somewhere
for a ports installation.  I assume it should work, I just need the right
URL to give it?  I tried all of:

sudo debootstrap sid /z
sudo debootstrap sid /z http://ftp.ports.debian.org/debian-ports/
sudo debootstrap sid /z http://ftp.ports.debian.org/debian-ports/pool-sparc64
sudo debootstrap sid /z http://ftp.ports.debian.org/debian-ports/dists
sudo debootstrap sid /z http://ftp.ports.debian.org/debian-ports/dists/sid

And it always fails the same way:

I: Target architecture can be executed
I: Checking Release signature
I: Valid Release signature (key id 126C0D24BD8A2942CC7DF8AC7638D0442B90D010)
E: Invalid Release file, no entry for main/binary-sparc64/Packages

Anyone have a debootstrap recipie for sprac64?  Thanks.

  - Chris

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Tue, May 22, 2018 at 08:54:21PM +0100, James Clarke wrote:
> On 22 May 2018, at 20:40, Chris Ross  wrote:
> > 1) It it suspected that I can boot from a RAIDZ on sparc64? (Does zfs-linux
> > and grub allow for that on any arch?)
> 
> In theory; raidz1/2/3 are listed in the GRUB manual[1], and later[2] has an 
> example
> FreeBSD ZFS menuentry.

Thanks.  That makes it looks like the support for ZFS within grub can handle
raidz, so a good sign.

> The Arch Wiki also has some information on ZFS[3] that may prove useful.

Ooh.  That's a little odd, but also notes that it frequently goes out of date.
I may have to look for an alternate reference to find out what features grub
is known to be confused by, and make sure none of those are turned on.  If 
someone has a good pointer to info on these in grub, of the grub version
we were recently discussing for use on sparc64 debian, I'd appreciate it.

> You might be able to boot d-i, skip partitioning and install to the root.
> Alternatively you could install to a small ext4 partition and then copy it to 
> the
> ZFS volume(s). Otherwise you'll need to do a manual debootstrap and whatever 
> other
> required configuration steps d-i performs.

Booting an installer may not work, though, since I need ZFS support in the
system I'm running.  To get a d-i with ZFS built in, I presume I'd have to
build my own.  Which I could, but don't know that I know how.  If I can use
something like debian-installer, I'd like to, but need one that can mount
up my ZFS filesystems.  Otherwise, I guess I need to find a guide of how to
do it the lower-level way.  ([1] looks valuable)

Thanks.

 - Chris

[1] https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS

Re: ZFS'ing my T5120 with Debian

[Chris Ross]

On Mon, May 21, 2018 at 01:54:36PM -0400, Chris Ross wrote:
> I have a T5120 currently running on sdd, and want to install a fresh debian
> on ZFS datasets on sda through sdc.  I want to install a fresh OS onto those
> zvols.  How much of that process is built into just running the debian
> installer, and how much will I need to manually set up?  I assume I need to
> create the zfs volumes, which I'm comfortable doing. (First, does anyone know
> if booting from RAIDZ works, or if I need to use ZFS mirror (as I did on
> FreeBSD)?
> 
> After that is there a "install everything" process, or a set of steps I
> need to take one at a time to install pieces, set up disks for booting, and/or
> other things.

  Hello again.  Am hoping for answers to two questions:

1) It it suspected that I can boot from a RAIDZ on sparc64? (Does zfs-linux
and grub allow for that on any arch?)
2) What do I need to do after creating ZFS volumes to perform a base install
onto those filesystems.  I have them all mounted onto an alt-root, is the next
step getting/running debian-installer, or another "please install the whole OS"
process?

  Thanks.

 - Chris

ZFS'ing my T5120 with Debian

[Chris Ross]

Thanks all for your assistance in recent weeks.  After the zfs-linux update
I was able to get zfs built, and all of the work on grub and partition/install
that I've been following on the list gets me to a point to ask for my next
steps.

I have a T5120 currently running on sdd, and want to install a fresh debian
on ZFS datasets on sda through sdc.  I want to install a fresh OS onto those
zvols.  How much of that process is built into just running the debian
installer, and how much will I need to manually set up?  I assume I need to
create the zfs volumes, which I'm comfortable doing. (First, does anyone know
if booting from RAIDZ works, or if I need to use ZFS mirror (as I did on
FreeBSD)?

After that is there a "install everything" process, or a set of steps I
need to take one at a time to install pieces, set up disks for booting, and/or
other things.

Thanks!

 - Chris

Re: Starting up Debian on a T5120

[Chris Ross]

On Sun, May 06, 2018 at 10:40:07PM +0200, John Paul Adrian Glaubitz wrote:
> Hi Chris!
> 
> You've run into a know bug, see [1].
> 
> > [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=897568

Ahh, thanks.  Well, at least it's not unique to sparc64, so I assume it'll
get remedied in a small number of days. Thanks, I'll keep an eye on that
bug number...

- Chris

Re: Starting up Debian on a T5120

[Chris Ross]

Attaching log file, for reference.  

On Sun, May 06, 2018 at 04:13:39PM -0400, Chris Ross wrote:
> On Sun, May 06, 2018 at 01:41:01PM +0100, James Clarke wrote:
> > > https://github.com/zfsonlinux/zfs/wiki/Debian
> > 
> > zfs-initramfs, along with zfs-dkms and various other packages, are arch:all 
> > and
> > thus should be installable normally with apt if you added the sources 
> > entries
> > like I said.  It's only the userland tools and libraries that come as
> > arch-specific packages and therefore needed to be built by you.
> 
>   Ahh.  Thanks for pointing that out.  I presumed "zfs-dkms" meant kernel
> modules, therefore clearly not arch independant.  Though, it looks like I
> have problems along the path.  When trying to install the built modules,
> it skipped compiling kernel modules because I didn't have headers.  Installing
> linux-headers-sparc64-smp, then trying to install the locally built .debs
> again, it proceeded well, and seemed to build kernel modules for spl.  But,
> later in the same "apt install" running, failed to build zfs DKMS modules:
> 
> Setting up zfs-dkms (0.7.6-1) ...
> Loading new zfs-0.7.6 DKMS files...
> Building for 4.16.0-1-sparc64-smp
> Building initial module for 4.16.0-1-sparc64-smp
> Error! Bad return status for module build on kernel: 4.16.0-1-sparc64-smp 
> (sparc64)
> Consult /var/lib/dkms/zfs/0.7.6/build/make.log for more information.
> 
> Looking in that log file, there are a small number of incompatible pointer 
> type
> errors:
> 
> /usr/src/linux-headers-4.16.0-1-common/include/linux/posix_acl.h:45:16: 
> error: passing argument 1 of ‘refcount_inc’ from incompatible pointer type 
> [-Werror=incompatible-pointer-types]
>refcount_inc(&acl->a_refcount);
> [...]
> /var/lib/dkms/zfs/0.7.6/build/include/linux/vfs_compat.h:288:26: error: 
> passing argument 2 of ‘atomic_sub_return’ from incompatible pointer type 
> [-Werror=incompatible-pointer-types]
>   if (atomic_dec_and_test(&acl->a_refcount))
> 
> and a few others.  I don't know if I should try to rejigger it to not error on
> that condition, or if this indicates some problem.  I'd like to hear from 
> others
> that have built linux kernel modules on sparc64 if they have any idea...
> 
> > > [...]  And, I don't know much about
> > > grub vs silo vs anything else that Linux uses to boot.
> > 
> > Do you know which you're using?
> 
>   As I have silo [package] installed, and not grub, I assume I'm using silo.
> Silo won't let me boot from ZFS?
> 
> > I believe switching to grub is as simple as:
> > 1. apt install grub2
> > 2. grub-install --skip-fs-probe --force /dev/sdX
> > 
> > though it's been a while since I had to deal with that process.
> > 
> > >  Hopefully someone else has some more pointers for me at this point.  
> > > Thank
> > > you much for these!
> 
>   Looking forward to more feedback from folks, I think I'm stalled trying to
> get ZFS kernel modules at the moment.  Thanks all!
> 
>   - Chris
DKMS make.log for zfs-0.7.6 for kernel 4.16.0-1-sparc64-smp (sparc64)
Sun May  6 10:44:40 EDT 2018
make  all-recursive
make[1]: Entering directory '/var/lib/dkms/zfs/0.7.6/build'
Making all in module
make[2]: Entering directory '/var/lib/dkms/zfs/0.7.6/build/module'
list='icp'; for targetdir in $list; do \
make -C $targetdir; \
done
make[3]: Entering directory '/var/lib/dkms/zfs/0.7.6/build/module/icp'
mkdir -p api core spi io os algs algs/aes algs/edonr algs/modes algs/sha1 
algs/sha2 algs/skein asm-x86_64 asm-x86_64/aes asm-x86_64/modes asm-x86_64/sha1 
asm-x86_64/sha2 asm-i386 asm-generic
make[3]: Leaving directory '/var/lib/dkms/zfs/0.7.6/build/module/icp'
make -C /lib/modules/4.16.0-1-sparc64-smp/build SUBDIRS=`pwd`  
O=/lib/modules/4.16.0-1-sparc64-smp/build CONFIG_ZFS=m modules
make[3]: Entering directory '/usr/src/linux-headers-4.16.0-1-sparc64-smp'
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/avl/avl.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/nvpair/nvpair.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/unicode/u8_textprep.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/zpios/pios.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/zcommon/zfs_deleg.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/icp/illumos-crypto.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/nvpair/fnvpair.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/unicode/uconv.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/zcommon/zfs_prop.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/icp/api/kcf_cipher.o
  CC [M]  /var/lib/dkms/zfs/0.7.6/build/module/nvpair/nvpair_alloc_spl.o

Re: Starting up Debian on a T5120

[Chris Ross]

On Sun, May 06, 2018 at 01:41:01PM +0100, James Clarke wrote:
> > https://github.com/zfsonlinux/zfs/wiki/Debian
> 
> zfs-initramfs, along with zfs-dkms and various other packages, are arch:all 
> and
> thus should be installable normally with apt if you added the sources entries
> like I said.  It's only the userland tools and libraries that come as
> arch-specific packages and therefore needed to be built by you.

  Ahh.  Thanks for pointing that out.  I presumed "zfs-dkms" meant kernel
modules, therefore clearly not arch independant.  Though, it looks like I
have problems along the path.  When trying to install the built modules,
it skipped compiling kernel modules because I didn't have headers.  Installing
linux-headers-sparc64-smp, then trying to install the locally built .debs
again, it proceeded well, and seemed to build kernel modules for spl.  But,
later in the same "apt install" running, failed to build zfs DKMS modules:

Setting up zfs-dkms (0.7.6-1) ...
Loading new zfs-0.7.6 DKMS files...
Building for 4.16.0-1-sparc64-smp
Building initial module for 4.16.0-1-sparc64-smp
Error! Bad return status for module build on kernel: 4.16.0-1-sparc64-smp 
(sparc64)
Consult /var/lib/dkms/zfs/0.7.6/build/make.log for more information.

Looking in that log file, there are a small number of incompatible pointer type
errors:

/usr/src/linux-headers-4.16.0-1-common/include/linux/posix_acl.h:45:16: error: 
passing argument 1 of ‘refcount_inc’ from incompatible pointer type 
[-Werror=incompatible-pointer-types]
   refcount_inc(&acl->a_refcount);
[...]
/var/lib/dkms/zfs/0.7.6/build/include/linux/vfs_compat.h:288:26: error: passing 
argument 2 of ‘atomic_sub_return’ from incompatible pointer type 
[-Werror=incompatible-pointer-types]
  if (atomic_dec_and_test(&acl->a_refcount))

and a few others.  I don't know if I should try to rejigger it to not error on
that condition, or if this indicates some problem.  I'd like to hear from others
that have built linux kernel modules on sparc64 if they have any idea...

> > [...]  And, I don't know much about
> > grub vs silo vs anything else that Linux uses to boot.
> 
> Do you know which you're using?

  As I have silo [package] installed, and not grub, I assume I'm using silo.
Silo won't let me boot from ZFS?

> I believe switching to grub is as simple as:
> 1. apt install grub2
> 2. grub-install --skip-fs-probe --force /dev/sdX
> 
> though it's been a while since I had to deal with that process.
> 
> >  Hopefully someone else has some more pointers for me at this point.  Thank
> > you much for these!

  Looking forward to more feedback from folks, I think I'm stalled trying to
get ZFS kernel modules at the moment.  Thanks all!

  - Chris

Re: Starting up Debian on a T5120

[Chris Ross]

On Sat, May 05, 2018 at 01:35:06AM +0100, James Clarke wrote:
> For building, I imagine the way to do it is:
> 
> 1. Add:
>deb [arch=all] http://deb.debian.org/debian unstable contrib non-free
>deb-src http://deb.debian.org/debian unstable main contrib non-free
>to your sources.list (or another mirror of your choice), so you can get the
>arch:all .debs from the main archive (note /debian rather than 
> /debian-ports),
>as well as the sources. Also note that there's no "main" component listed 
> for
>the first entry.
> 
> 2. apt update
> 
> 3. apt source zfs-linux
> 
> 4. apt build-dep zfs-linux
> 
> 5. cd zfs-linux-0.7.6
> 
> 6. dpkg-buildpackage -us -uc -B
> 
> You should then have all the sparc64 .debs in the parent directory which you 
> can
> install with "apt install /path/to/foo.deb".

  Thanks.  This worked, as you expected.  The question I have now is what to
install.  I have 9 .deb files, but none of them is "zfs-dkms", which is one
of the packages that zfsonlinux says to install:

https://github.com/zfsonlinux/zfs/wiki/Debian

  I have:

-rw-r--r-- 1 cross   41444 May  5 22:50 libnvpair1linux_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross   50680 May  5 22:50 libuutil1linux_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross  119312 May  5 22:50 libzfs2linux_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross  966516 May  5 22:51 libzfslinux-dev_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross  460448 May  5 22:50 libzpool2linux_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross 3580280 May  5 22:50 zfs-dbg_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross 2481164 May  5 22:51 zfs-test_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross  328128 May  5 22:50 zfsutils-linux_0.7.6-1_sparc64.deb
-rw-r--r-- 1 cross   55252 May  5 22:50 zfs-zed_0.7.6-1_sparc64.deb

  Looking at the build logs I see some references to dkms, including:

checking for dkms.conf file... not found

  ...but I don't assume that indicates a problem.  I see it do:

make[1]: Entering directory '/home/cross/zfs-linux-0.7.6'
dh_dkms -V 0.7.6
make[1]: Leaving directory '/home/cross/zfs-linux-0.7.6'

  but with no other information, I don't know what happened (or didn't) there.

> As far as booting goes, I have no idea. I'm almost certain you'd need to be 
> using
> grub not silo, but beyond that your prior experience likely means you know 
> more
> than me. Generating installer images is awkward, so it may just be easiest to 
> clone
> your system to a new ZFS volume, boot to that and then reclaim the ext4 space.
> Perhaps there's a better way that someone here knows?

  It looks like I had the same thought.  I realize now I'm running on sdd, and
sda sdb and sdc are waiting for me to do something useful with them. So at least
I don't need to figure out the installer stuff.  I will still need to figure
out how to boot, which the above debian page suggests I want zfs-initramfs
package.  But I doubt that alone is enough.  And, I don't know much about
grub vs silo vs anything else that Linux uses to boot.  

  Hopefully someone else has some more pointers for me at this point.  Thank
you much for these!

- Chris

Re: Starting up Debian on a T5120

[Chris Ross]

On Fri, May 04, 2018 at 07:14:16PM +0200, John Paul Adrian Glaubitz wrote:
> The problem here, again, is that sparc64 is part of Debian Ports which
> doesn't build any packages from the "contrib" and "non-free" distributions.

  Ahh.  So the zfs-dkms is contrib or non-free, so doesn't get built?  Okay.
There will probably be a number of other things that will fall into that,
what's a good way to know if something is not available for that reason?

> I think this is an issue we can get resolved once we have replaced "mini-DAK"
> with "DAK" in Debian Ports (the software that maintains the FTP server
> archive contents). But that's a long-time project.

  Okay.  And one that I don't have the context to understand, but don't need
to for now.

> Just apt update && apt upgrade && apt dist-upgrade should be enough.
> 
> Let us know if there are any issues with the dist-upgrade.

  Doing that now.  360+ packages being upgraded...

  It failed to update initramfs-tools, because it ran out of space in my /boot.
I ran autoremove (getting rid of 4.14.0-3) and it was able to complete.

  Then, apt dist-upgrade found nothing to do.  So, I'm up to date.  Do you have
a pointer to a guide of how to build the ZFS kernel modules, and what will need
to be done to get it into the boot system so I can boot off of a ZFS vol?
(presumably a zmirror, which is how I've been doing it on other systems)
Then, do I need to generate a new install CD so that I can get that all
installed onto the disks that now have ext volumnes on them?

  Thanks!

   - Chris

Re: Starting up Debian on a T5120

[Chris Ross]

On Fri, May 04, 2018 at 06:38:32PM +0200, John Paul Adrian Glaubitz wrote:
> Yes, Debian has the packages "zfs-linux" and "spl-linux" which provide
> various binary packages you can install for getting ZFS support in your
> kernel. ZFS does support sparc64, of course. Although ZFS upstream introduced
> support for something called "channels" and I think this particular wasn't
> matured on sparc64.
> 
> Also, since zfs-linux is part of Debian's contrib distribution, you would
> have to build the package from source yourself. I can help you with that
> either way.

  Ahh, okay.  That's what I didn't know yet.  I saw that there was an available
zfs-fuse and installed that, since I didn't see a kernel ZFS.  I started
considering how I might manage if I didn't have a "everything is in ZFS"
system which is what I'm used to.  Then next step was "ask for help".

> So, first you should go ahead and install your machine with Debian's
> sparc64 port. Then I'll let you know how to proceed with getting ZFS
> set up.

  Got that.  A couple months ago, and just brought it back online.  Should
I reinstall it, upgrade it, or just start where I am?  Looks like a late-Feb
build is what got installed.  And I'm definately an old hand with building
and installing OSes.  Been doing it that way for 20+ years.  :-)

  Thanks for your assitance!

 - Chris

Starting up Debian on a T5120

[Chris Ross]

Hello all.  I have been a sparc64 user for many years, across a variety
of BSDs.  Most recently, FreeBSD, largely for their implementation of ZFS.

Since FreeBSD doesn't support sun4v, and isn't likely to, I looked at 
alternatives, and am looking to try debian sparc64.  I have experience
with Debian and Ubuntu, so the learning curve shouldn't be too bad.

I got a sparc64 build installed onto my t5120 earlier this year, then
got distracted by other things.  Getting back to it, I'm curious if there
is ZFS support for Debian sparc64 other than the FUSE module I found and
am using.  I'd like to configure the disks as ZFS and boot off of them.
I'm sure this is possible with the kernel modules, but want to know if
anyone knows how to set this up with the sparc64 support.

Thanks!  I look forward to learning more about my options here...

  - Chris