Re: dumping on RAIDframe

2023-09-25 Thread Michael van Elst
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes:

>> you dump a memory block that isn't a multiple of a disk sector 
>> (according to disklabel)
>You mean this one (from disklabel raid0):
>   bytes/sector: 512
>?

Yes. Which makes it unlikely.

amd64/machdep.c:

this dumps at least DEV_BSIZE:

to_write = roundup(dump_headerbuf_ptr - dump_headerbuf, dbtob(1));
error = bdev->d_dump(dumpdev, dump_header_blkno,
dump_headerbuf, to_write);

and this is called with bytes being a multiple of the page size:

for (i = 0; i < bytes; i += n, dump_totalbytesleft -= n) {
n = bytes - i; 
if (n > BYTES_PER_DUMP)
n = BYTES_PER_DUMP;
...
error = (*dump)(dumpdev, blkno, (void *)dumpspace, n);
...
}




Re: dumping on RAIDframe

2023-09-25 Thread Edgar Fuß
> you dump a memory block that isn't a multiple of a disk sector 
> (according to disklabel)
You mean this one (from disklabel raid0):
bytes/sector: 512
?


Re: dumping on RAIDframe

2023-09-25 Thread Michael van Elst
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes:

>GO>Dumping to a RAID 1 set is supported in -8.  But yes, none of those 
>GO>values seem to align with each other.  18,1 is 'raid0b' thouugh, so that 
>GO>part seems correct.

>MvE> offset and size relate to the dump data (dumplo and dumpsize), not
>MvE> the partition.


The "device not ready" comes from the driver dump routine
returning EFAULT. The error code is abused, it is reported
when a) you dump a memory block that isn't a multiple of
a disk sector (according to disklabel) or b) you start a dump
while a dump is already running.

Maybe dump errors shouldn't be printed with DPRINTF.



Re: dumping on RAIDframe

2023-09-25 Thread Edgar Fuß
EF> dumping to dev 18,1 (offset=1090767, size=8252262):

GO>Dumping to a RAID 1 set is supported in -8.  But yes, none of those 
GO>values seem to align with each other.  18,1 is 'raid0b' thouugh, so that 
GO>part seems correct.

MvE> offset and size relate to the dump data (dumplo and dumpsize), not
MvE> the partition.

So here are the various configs (raid0.conf from raidctl -G raid0); I omit
sd1's info (normally identical/analogous to that of sd0) because I just 
pulled the disk to simulate a RAID failure. I'm unsure about whether the dump 
attempt was with a healthy or (artificially) failed RAID, I think it was 
with a healthy one.
  start   size  index  contents
  0  1 PMBR
  1  1 Pri GPT header
  2 32 Pri GPT table
 34   2014 Unused
   2048 262144  1  GPT part - EFI System
 264192  930869248  2  GPT part - NetBSD RAIDFrame component
  931133440   2015 Unused
  931135455 32 Sec GPT table
  931135487  1 Sec GPT header
/dev/rsd0: 2 wedges:
dk0: efi0, 262144 blocks at 2048, type: msdos
dk1: raid0, 930869248 blocks at 264192, type: raidframe
# raidctl config file for /dev/rraid0

START array
# numRow numCol numSpare
1 2 0

START disks
/dev/dk1
/dev/dk3

START layout
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_1
128 1 1 1

START queue
fifo 100
# /dev/rraid0:
type: RAID
disk: raid0
label: lahn
flags:
bytes/sector: 512
sectors/track: 128
tracks/cylinder: 8
sectors/cylinder: 1024
cylinders: 909051
total sectors: 930869120
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0   # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

6 partitions:
#sizeoffset fstype [fsize bsize cpg/sgs]
 a:   2097152 0 4.2BSD  0 0 0  # (Cyl.  0 -   2047)
 b:  67108864   2097152   swap # (Cyl.   2048 -  67583)
 d: 930869120 0 unused  0 0# (Cyl.  0 - 909051*)
 e:  62914560  69206016 4.2BSD  0 0 0  # (Cyl.  67584 - 129023)
 f: 629145600 132120576 4.2BSD  0 0 0  # (Cyl. 129024 - 743423)


Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Robert Elz
Date:Mon, 25 Sep 2023 09:45:25 - (UTC)
From:mlel...@serpens.de (Michael van Elst)
Message-ID:  

  | There can be multiple EFI system partitions on a drive,

That was my understanding from reading the spec.

  | but it sometimes confuses software,

What can I say to that...

  | some boot procedures will only handle the first ESP.

See my reply to RVP's reply just below.

  | But you "should" be able to select an ESP in UEFI just like you would
  | select a boot device in BIOS.

There's no problem picking the correct ESP from which to load efibootx64.efi
(or whatever the magic name is) - that part works fine, and if I have multiple
ESPs with NetBSD's efiboot in them I can pick whichever one I want (I think,
which probably gives me another option for testing updated efiboot code).

It is just boot.cfg which is giving problems.

r...@sdf.org said:
  | The first EFI partition. Without a bootme etc., flag, the NetBSD UEFI
  | bootloader tries to read /EFI/NetBSD/boot.cfg from there, first. 

I wondered about that when I saw Edgar's first message earlier, it
suddenly occurred to me that I normally never go near that ESP (that one
was set up by the system builders to boot wintrash - a subject which has
no interest to me, but I have retained intact (just reduced the size of the
wintrash partition) so it can be used for repairs, which I suspect is what
just happened.

So, I went and checked, but it turns out that I was (apparently) very
thorough when I was scattering boot.cfg files around, I discovered that
I have:

./EFI/Boot/boot.cfg
./EFI/NetBSD/boot.cfg
./EFI/boot.cfg
./boot.cfg

(that is relative to /mnt where I briefly temporarily mounted that parition).

None of those are used...  (there's no NetBSD boot code there, EFI/Boot
is a wintrash boot directory, but the boot.cfg file doesn't seem to bother 
that).

  | I've hacked my bootloader locally to make it try a /EFI/NetBSD/boot.cfg on
  | the partition where it itself was loaded from

I'd suggest that's what it should always attempt first, absent other info,
rather than being a local hack for you.

kre



Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread RVP

On Mon, 25 Sep 2023, Robert Elz wrote:


   Date:Mon, 25 Sep 2023 05:57:49 +
   From:Emmanuel Dreyfus 
   Message-ID:  

 | bootme.cfg is searched in EFI paririon /EFI/NetBSD/boot.cfg

Which EFI partition?   I think I have about 5 or 6, sprinkled around
various bootable devices (more than one on some).   None of them (boot.cfg
files) are used.



The first EFI partition. Without a bootme etc., flag, the NetBSD
UEFI bootloader tries to read /EFI/NetBSD/boot.cfg from there,
first. Then it tries to read it from the first NetBSD partition
in the EFI table.

I've hacked my bootloader locally to make it try a /EFI/NetBSD/boot.cfg
on the partition where it itself was loaded from: this is for 2
reasons:

1. I like separate NetBSD EFI partitions for 9.x, 10.x, -HEAD etc.,
   and want each to read its own boot.cfg.

2. Some commands are subtly different between 9.x and 10.x. Eg.

```
gop=0
```

on its own line will work with a 9.x bootloader. But the 10.x
bootloader needs a `gop 0' in a `menu=' entry to have the same effect.

-RVP


Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Michael van Elst
k...@munnari.oz.au (Robert Elz) writes:

>Date:Mon, 25 Sep 2023 05:57:49 +
>From:Emmanuel Dreyfus 
>Message-ID:  

>  | bootme.cfg is searched in EFI paririon /EFI/NetBSD/boot.cfg

>Which EFI partition?   I think I have about 5 or 6, sprinkled around
>various bootable devices (more than one on some).


There can be multiple EFI system partitions on a drive, but it sometimes
confuses software, some boot procedures will only handle the first ESP.

But you "should" be able to select an ESP in UEFI just like you would
select a boot device in BIOS.



Re: dumping on RAIDframe

2023-09-25 Thread Michael van Elst
os...@netbsd.org (Greg Oster) writes:

>> dumping to dev 18,1 (offset=1090767, size=8252262):
>> 
>Dumping to a RAID 1 set is supported in -8.  But yes, none of those 
>values seem to align with each other.  18,1 is 'raid0b' thouugh, so that 
>part seems correct.

offset and size relate to the dump data (dumplo and dumpsize), not
the partition.



Re: boot.cfg location (was: GPT attributes in dkwedge [PATCH])

2023-09-25 Thread Martin Husemann
On Mon, Sep 25, 2023 at 10:43:56AM +0200, Edgar Fuß wrote:
> > boot[].cfg is > searched in EFI par[tit]ion /EFI/NetBSD/boot.cfg 
> > and root partition /boot.cfg. 
> But how can EFI locate it on the root partition if it tells where the root 
> partition lives?

Not EFI, but the bootloader. It guesses a partition where /netbsd is and
checks /boot.cfg the same way. The bootme flag is a strong hint in that 
guessing step.

This has been carried over from the BIOS boot loader, where this was the only
supported location for /boot.cfg (as it has no ESP).

Martin


boot.cfg location (was: GPT attributes in dkwedge [PATCH])

2023-09-25 Thread Edgar Fuß
> boot[].cfg is > searched in EFI par[tit]ion /EFI/NetBSD/boot.cfg 
> and root partition /boot.cfg. 
But how can EFI locate it on the root partition if it tells where the root 
partition lives?


Locating boot.cfg on ESP (was: GPT attributes in dkwedge [PATCH])

2023-09-25 Thread Edgar Fuß
>   | It's not obviously where efiboot finds boot.cfg, since that's in
>   | esp:/EFI/NetBSD/boot.cfg or,
> 
> And we correctly interpret that, always?
It works for me on four servers I recently set up if I put it into /EFI/NetBSD 
on the ESP. It also, for reasons unknown to me, works on one other identical 
server I set up earlier (the prototype for the others) if put into the root 
of the ESP, but that doesn't work on the four others. I didn't find out why.


Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Martin Husemann
On Mon, Sep 25, 2023 at 05:57:49AM +, Emmanuel Dreyfus wrote:
> Bootme tels bootstrap where to look root partition. bootme.cfg is 
> searched in EFI paririon /EFI/NetBSD/boot.cfg  and root partition /boot.cfg. 

I would describe it differently:

 - firmware finds bootloader "somewhere" on some ESP, bootloader finds
   boot.cfg in the same directory (if it exists)
 - otherwise bootloader guesses a partition from where the kernel is
   loaded and also checks for a boot.cfg in the root directory of that
   partition. The bootme flag may be involved in this step (but is not
   required).
 - kernel gets passed the location where it was loaded from and optional
   other arguments if boot.cfg was found. kernel decides what to use
   as root partition. A /boot.cfg on this partition is not needed nor
   usefull unless it happens to be the same partition as where the kernel
   is loaded from (the item before) and there was none in the ESP
   directory (the first item).

Martin


Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Robert Elz
Date:Mon, 25 Sep 2023 05:57:49 +
From:Emmanuel Dreyfus 
Message-ID:  

  | bootme.cfg is searched in EFI paririon /EFI/NetBSD/boot.cfg

Which EFI partition?   I think I have about 5 or 6, sprinkled around
various bootable devices (more than one on some).   None of them (boot.cfg
files) are used.   But in most of them I have (with a different banner)
boot.cfg files in their root directory, EFI directory, EFI/NetBSD
directory - and I think in a "boot" directory when one of those happens
to exist - maybe created by either wintrash (builders used) or linsux
(which I installed briefly in the very early days to test the system more
thoroughly - all removed now, except some vestigal boot nonsense that
sometimes gets in the way until I tell the firmware (again) to not use it,
as it has nothing to boot any more).

  | and root partition /boot.cfg. 

That's impossible - boot.cfg says where the root partition is to be
found, so boot.cfg has to be found first.   (But see just below for an
interpretation; and it is of much less consequence to me what is chosen
as root if boot.cfg doesn't say, one way or another, which filesystem that
should be, fallback defaults are of far less interest to me.)

I have several candidate roots, for NetBSD 10, current, whatever I
am testing this week, an experimental one (perhaps not working), ...
There's no sane way to load /boot.cfg from "it" (though I have one in at
least the root partition I use most of the time, which is my testing one,
which unsurprisingly, isn't used either).

However, if what you meant by:

  | Bootme tels bootstrap where to look root partition

is that it tells the bootstrap code where to locate boot.cfg,  whether
or not that also later happens to become the NetBSD root, via specification,
or as a default if there is no spec, then I think we are more or less in
agreement.

Of course I'd like it more if it worked, and I'll admit that updating
boot code is the thing I like to do least, as it is the most likely thing
I can do which will wedge the system and need real assistance (as in booting
from optical or USB, or something) to recover - so the efiboot that I am
currently using must be 16 months old now (late May 2022) - I have been
meaning to work out a safe way to update and try the current version - and
I actually have a fairly easy way now, when my system was repaired, I had a
new SSD added, a clone of the one which holds all of my potential roots,
and which has 2 EFI partitions, one of which boots NetBSD - ie: has the
NetBSD efiboot that actually gets used - but that one is connected to an
add in SATA controller, from which the firmware can't boot (it probably
could given an add on EFI driver to use, but that's above my pay grade).
All I need to do is swap the SATA cables to those two SSDs (after populating
the new one, of course) and I'll be able to boot from the new one - if that
fails, I can just swap back to get to the original one.

I know a lot of work has been done on efiboot since the one I am using, so,
before resorting to attempt to work out why it is unable to find any boot.cfg
for me (I don't really care which it finds, I can work with any of them) I
want to make sure the current version is still failing for me, and if so,
work out why (if not already fixed, it is probably something trivial).

kre



Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Emmanuel Dreyfus
On Mon, Sep 25, 2023 at 08:13:29AM +0200, Michael van Elst wrote:
> So, the bootme flag effectively specifies the root partition, but only
> by virtue of defaults being passed down the chain. The kernel should
> not outguess things and interpret the flag itself.

I am working with oster@ on that. There will be no bootme reference in
kernel code.

-- 
Emmanuel Dreyfus
m...@netbsd.org


Re: GPT attributes in dkwedge [PATCH]

2023-09-25 Thread Michael van Elst
On Mon, Sep 25, 2023 at 05:57:49AM +, Emmanuel Dreyfus wrote:

> On Mon, Sep 25, 2023 at 12:20:00PM +0700, Robert Elz wrote:
> [bootme flag]
> > I'd always assumed it to be where efiboot should locate boot.cfg.
> > Where the kernel and root filesystems are located are in boot.cfg.
> 
> Bootme tels bootstrap where to look root partition. bootme.cfg is 
> searched in EFI paririon /EFI/NetBSD/boot.cfg  and root partition /boot.cfg. 

Boot partition and Root partition are something separate, even when the
default root partition is the same as the boot partition.

For EFI boot, the boot partition is where boot.cfg is searched
and where the kernel is loaded from, unless you specify something else
to the bootloader (e.g. in boot.cg). Where the kernel is loaded
from is passed to the kernel.

The kernel will use the informatiun to find the root partition,
unless you specify something else (or magic like the raidframe
hack comes into play).

So, the bootme flag effectively specifies the root partition, but only
by virtue of defaults being passed down the chain. The kernel should
not outguess things and interpret the flag itself.


-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."