Please can we do something about programmatic interfaces to ZFS?

[David Chisnall]

Hi,

Since updating to 15-CURRENT, I have been unable to get some existing code that 
used libzfs_core to take snapshots.  There are a lot of reasons that this could 
have broken and it’s hard to track it down:

 - We ship both libnv and libnvpair.  These define the same data structure but 
with different APIs and are incompatible.  I believe libnv can create the 
serialised data structures that the ZFS ioctls expect.
 - We don’t install headers for libnvpair or libzfs_core (or libzfs) and so any 
code using these has to either depend on things in the src tree (which depend 
on OpenSolaris headers that are incompatible with FreeBSD ones, so must be in 
separate compilation units) or provide its own definitions, which may get out 
of sync with the libraries.
 - We don’t provide any documentation of the underlying ZFS ioctls (and there 
is some code that suggests that these vary between platforms), and so the 
*only* API for interacting with ZFS is libzfs_core.
 - The APIs in libzfs_core are also poorly documented.

This makes it incredibly difficult to interact with ZFS via anything other than 
the `zfs` command-line tool.  When things break, I have no idea which of these 
layers caused the breakage.

David

Re: Booting in bhyve always sets currdev to ZFS

[Rick Macklem]

On Sun, Jun 30, 2024 at 8:23 AM Rodney W. Grimes
 wrote:
>
> > On Fri, Jun 28, 2024 at 6:26?PM Rick Macklem  wrote:
> > >
> > > Hi,
> > >
> > > I've installed FreeBSD current in a bhvye instance. Everything
> > > went ok, with UFS as a root partition.
> > > Then I created a zpool in another partition...
> > > - Now, every time I boot it I have to
> > >   OK set currdev=disk0s1a:
> > > to get it to boot.
> > >
> > > What is the trick to keep ZFS from messing up the boot variables?
> > I've been poking around, but haven't learned much.
> > I think it is userboot (although there are so many boot programs in /boot,
> > I am not 100% sure?) that sets currdev=zfs:example: since it sees there is 
> > ZFS
> > in a partition on the drive. It is not the boot partition and doesn't have
> > any boot stuff in it.
> >
> > When I look at userboot, it appears that it always sets userboot_zfs_found
> > to 1 whenever userboot_zfs_probe() is called, given that there is a ZFS
> > partition with a pool on it. This makes extract_currdev get set to the
> > ZFS stuff,
> > assuming I am reading the code correctly.
> > What I do not understand is why I have not seen this before?
> > (Was the a change to building it with USERBOOT_ZFS_SUPPORT done?)
> >
> > It seems that userboot_zfs_probe() should check for boot files on the volume
> > and not just that a pool exists on the partition, maybe?
> >
> > Anyhow, manually setting currdev=disk0s1a: gets around the problem.
>
> You should be able to set that in your ufs partition /boot/loader.conf
> file and at least then your not having to drop to the OK prompt and
> manually entering this.
Doesn't help, because it is not reading files from the boot ufs file system.
(I could copy all the boot stuff over into the ZFS file system, but that kinda
defeats the purpose of having UFS as the boot fs.)

For example, it always fails with:
ERROR: cannot open /boot/lua/loader.lua: no such file or directory.

When I look in stand/efi/loader/main.c, there is a function that checks
for boot files called sanity_check_currdev(). However, userboot does not
have a similar function and just seems to assume that the ZFS partition
in the bootable one, if it finds one.
--> I am wondering if sanity_check_currdev() shoud be added to userboot
  to handle this case?

rick

>
> There is much magic in boot code that bites, and not just in FreeBSD.
> Often assumptions are made that "this" boot code, and "this" OS are
> the only things on the disk(s).
>
> The fact that Linux uses a "apple-zfs" uuid, and FreeBSD a "FreeBSD-zfs"
> drove me nuts for 2 days until I found this little tid bit.
>
> Zpool import doesnt care, but the boot code does!
>
>
> --
> Rod Grimes rgri...@freebsd.org