Thanks to everyone who participated. Sorry for the delay in getting this email sent! Meeting recording is now posted: https://www.youtube.com/watch?v=tyCk376bC9A
Notes (thanks Christian): - Encryption Bugs (Rich Ercolani) - Rich’s spreadsheet is now publicly accessible <https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg/edit#gid=1560550070> - One new bug was reported that doesn’t involve replication: #1293 <https://github.com/openzfs/zfs/issues/12931>1 (dereferencing a wild pointer in the checksum code with encryption+dedup on under heavy load) - Compressors (Rich Ercolani) - Issue that sparked the discussion: #12840 <https://github.com/openzfs/zfs/issues/12840> - Summary: Replacing the compressor code causes problems if the output is not bit-for-bit identical. - Problem #1: Uncompressed ARC + Compressed On-Disk + L2ARC - Assume uncompressed ARC, which holds the data of a block that is compressed on disk. - The L2ARC invariant is that it stores a bit-for-bit identical copy of the on-disk data. - So, the L2ARC write must recompress the uncompressed copy. - But if the compressor changed, the recompressed copy will not be bit-for-bit identical to the on-disk data, and the L2ARC read will fail with a checksum error, resulting in an L2ARC miss. - Problem #2: NOP writes - TODO explain - Problem #3: Dedup - Hashing is against the compressed blocks. Thus, blocks compressed with the new compressor will not dedup with blocks compressed by the old compressor. - Initial consensus: all three problematic cases are very minor. We can just replace-upgrade decompressor and compressor. - We already did that inadvertently with the different gzip implementations (QAT, Linux, FreeBSD). - Uncompressed ARC in general should be considered an edge case. Users should expect that it might not interact perfectly with all other features (such as L2ARC). - For L2ARC, we can add a small check that marks recompressed buffers which are not bit-for-bit identical ineligible for L2ARC. - LZ4 (Rich) and ZSTD are two concrete upgrades that are planned. - Special treatment of the LZ4 compressor upgrade? - LZ4 is the default on many systems. - A replacement-upgrade of the LZ4 compressor would thus affect many users of dedup. - So, should we require users to opt-in for LZ4 upgrades, e.g., via a feature flag? This would require keeping multiple LZ4 compressor versions around. - Decision: Wait for 1 month to see whether someone comes up with a scheme that requires user opt-in. If there is no solution, go ahead with a replacement-upgrade. - NB: For ZSTD, which changes more frequently, we might not want to maintain multiple compressor versions. So, whatever will be decided for LZ4 does not set a precedent for ZSTD. On Mon, Jan 3, 2022 at 3:52 PM Matthew Ahrens <mahr...@delphix.com> wrote: > The next OpenZFS Leadership meeting will be held tomorrow, January 4, 1pm > Pacific time. > Everyone is welcome to attend and participate, and we will try to keep the > meeting on agenda and on time. The meetings will be held online via Zoom, > and recorded and posted to the website and YouTube after the meeting. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit > > --matt > ------------------------------------------ openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-Md73cea97245493e922c7e523 Delivery options: https://openzfs.topicbox.com/groups/developer/subscription