Thanks to everyone who participated. Sorry for the delay in getting this
email out!
Meeting recording is posted: https://youtu.be/hij7PGGjevc
Notes (thanks Christian):
-
Update on BRT (Block Reference Table) (Pawel)
-
Pawel claims 98% ready.
-
Design changes since last update
-
If a block is already in dedup table, we now bump the dedup
refcount instead of starting a second reference counter in the BRT.
-
ZIL: now implemented. Log records just use block pointers. No use
of object IDs, since ZIL is per-objset, but BRT operation
logged in the ZIL
depends on state of source and dest objset.
-
Further potential design changes discussed in the meeting:
-
Have a per-VDEV BRT to avoid repeating the VDEV ID in each entry.
-
=> on-disk & memory space savings
-
Discussion about generality of the implementation
-
Current implementation defines two new FreeBSD-specific syscalls
-
Having the Linux copy-reflink IOCTL would make the mechanism “just
work” with existing Linux tools.
-
Also, there’s Linux copy_file_range, which “gives filesystems an
opportunity to implement "copy acceleration" techniques, such
as the use of
reflinks.
-
The MacOS syscall is path-based, but Lundman says in chat that the
filesystem implements VNOP_CLONEFILE, which operates on vnodes.
-
Ideas about a command line tool / daemon to trigger dedup/BRT
offline (i.e., after duplicates were written).
-
Timeline
-
Pawel first needs https://github.com/openzfs/zfs/pull/13027 to be
merged.
-
Then he can make a PR.
-
Code will need more reviewers.
-
Encryption Bugs (Rich Ercolani)
-
(Was not able to make it to the meeting, summary written ahead of
time.)
-
One set of bugs (#12981 <https://github.com/openzfs/zfs/pull/12981>
et al) got a workaround from George Amanakis - thanks George!
-
Another bug (#12720 <https://github.com/openzfs/zfs/issues/12720>)
manifested as an error during raw send/recv. The underlying
cause is faulty
on-disk dnodes with contradicting bonuslength/spill pointer flag
in < 0.6.4
versions of OpenZFS. Related PR #13014
<https://github.com/openzfs/zfs/pull/13014>.
-
WIP PR #12943 <https://github.com/openzfs/zfs/pull/12943> for issue
#11679 <https://github.com/openzfs/zfs/issues/11679> had an issue
reported, going to try and ameliorate with even more locking,
but could use
someone with familiarity with send/recv a/o the ARC code to help, as I’m
pretty sure this is just papering over something being done incorrectly.
-
Issue #11679 has drawn lots of attention. Someone familiar with
the DMU should take a look.
-
WIP I should circulate for extending FORCE_INHERIT/FORCE_NEW_KEY to
allow you to escape situations like #12614
<https://github.com/openzfs/zfs/issues/12614>, need to write more
tests, feel free to ping me if you’re in this situation and want
to try it.
(Also trying to figure out what a reasonable thing to do in most
cases when
you receive a change-key in an incremental send is - so far, all of the
options seem to violate POLP sometimes.)
-
Downside: not insignificantly sized foot-gun to allow you to
change-key -f
-
Storing last N copies of the wrapped key and trying them all would
help you avoid that, but then you have N ways to unlock the key…
-
Someone reported issues with receiving unencrypted under an encrypted
and not unlocked parent (#13033
<https://github.com/openzfs/zfs/issues/13033>; not data loss or
anything, just mentioning for completeness)
-
Add Blake3 Checksum (PR #12918
<https://github.com/openzfs/zfs/pull/12918>) (Tino)
-
Are there open issues before inclusion?
-
Benchmarks
<https://github.com/openzfs/zfs/pull/12918#issuecomment-1008202352>
(Unit is MiB/s, more is better)
-
Is Blake3 a “strong” checksum, in the ZFS sense (= is it dedup-safe)?
-
It is a cryptographic checksum
<https://en.wikipedia.org/wiki/BLAKE_(hash_function)> .
-
The patch adds blake3 to the list of supported dedup checksums
-
Looking for additional reviews:
-
https://github.com/openzfs/zfs/pull/12773 (reduced spa_inflation)
-
https://github.com/openzfs/zfs/pull/12868 (write smoothing)
-
https://github.com/openzfs/zfs/pull/12263 (linux user namespace
support)
-
https://github.com/openzfs/zfs/pull/12789 (log spacemap load time)
-
https://github.com/openzfs/zfs/pull/12767 (use-after free of znode_t)
On Mon, Jan 31, 2022 at 11:20 AM Matthew Ahrens <mahr...@delphix.com> wrote:
> The next OpenZFS Leadership meeting will be held tomorrow, February 1, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time. The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>
------------------------------------------
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-Me844c6f99850f181fe89f619
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
