Thanks to everyone who participated. The video recording is now available:
https://www.youtube.com/watch?v=_097QvCwT4Y
Meeting notes (thanks Christian!):
-
ZFS Interface for Accelerators (“Z.I.A.”); Jason Lee (Los Alamos
National Laboratory)
-
Slides
<https://drive.google.com/file/d/1KTkR-ArSj7Pt2KTarNeWxqGsHNIQA4cf/view>
-
Summary: make use of hardware accelerators for compression, checksum,
raidz
-
Main idea:
-
Data is owned by either the Host CPU or the accelerator.
-
ZIO pipeline refers to data owned by the accelerator through a
“handle”.
-
If the accelerator doesn’t support one step of the pipeline, data
is moved back to the host for that step. But as long as the
next step is
supported, we can avoid copying back-and-forth.
-
Discussion about what safeguards should be put in place to keep
track of ownership / source-of-truth of the data. E.g.,
during ABD copy.
-
Software architecture provider/consumer of accelerators:
-
“Data Processing Unit Services Module” decouples consumers from
providers
-
Non-Restrictive License towards consumers; GPLv2 towards providers
-
QA / Discussion
-
Watch out for byte-for-byte compatibility, otherwise incompatible
with “uncompressed ARC + L2ARC” use case.
-
Maintainability / Testing: at least a software provider should be
part of the OpenZFS repo so that we can prevent accidental
breakage (e.g.
when changing ZIO pipeline implementation)
-
Public availability of actual hardware & corresponding providers
would be critical for upstreaming.
-
Discussion on resource contention (e.g. two zpools sharing one
accelerator)
-
Native encryption needs some work (Rich Ercolani)
-
Spreadsheet of encryption bugs
-
Biggest and scariest: incorrect dnode refcounting
-
Panic stacks look very different, trapping at different ASSERTs
-
Not reproducible on x86(_64), but on SPARC and on PowerPC
-
Reproduces within 24 hours on a PowerPC 64 KVM VM (but not
qemu-emulated)
-
Earliest occurrence dates all the way back to introduction of native
encryption
-
Other scary category: encrypted send/recv corrupts receive-side
dataset
-
One can be triggered by zfs change-key => will use wrong wrapping
key
-
Problem: the community is not aware of a company that could fund
fixing this.
-
Strategy: bring it up at future meetings to raise awareness.
-
Request for reviews:
-
Allan: calculate a more realistic allocation inflation multiplier:
https://github.com/openzfs/zfs/pull/12773
Alek: healing recv patch - https://github.com/openzfs/zfs/pull/9372
On Mon, Dec 6, 2021 at 9:11 AM Matthew Ahrens <[email protected]> wrote:
> The next OpenZFS Leadership meeting will be held tomorrow, December 7, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time. The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>
------------------------------------------
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/T24bdaa2886c6cbf5-Mc039a11c3f1507ea0664817b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
