[developer] June OpenZFS Leadership Meeting

[Matthew Ahrens]

Apologies for the late notification, the next OpenZFS Leadership meeting
will be held tomorrow, June 18, at 1pm Pacific time.  (Note that this is
the 3rd meeting in a row at 1pm - normally we would have it at 9am, but the
calendar and doc say 1pm, and I can't make it at 9am so we'll keep it at
1pm this month).

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

The agenda for the meeting will be a discussion of the projects listed in
the agenda doc.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td25434f74c9df977-M0fa4c07fdffb3fb7b47b4068
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] May OpenZFS Leadership Meeting

[Matthew Ahrens]

Apologies for the late notification, the next OpenZFS Leadership meeting
will be held today, May 21, at 1pm Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

The agenda for the meeting will be a discussion of the projects listed in
the agenda doc.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tbc2723be3e4d22d4-Mc138ce66869b6f32e32bab33
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video recording: https://www.youtube.com/watch?v=habANXNHPWQ

agenda:

   -

   ZIO pipeline synchronicity problem (Alexander Motin)
   -

   NULL zios
   -

   OpenZFS Conference
   -

   Fast Dedup Reviews: https://github.com/openzfs/zfs/discussions/15896
   -

  ZAP shrinking has been accepted
  -

  Dedup quota needs a 2nd reviewer
  -

   Direct IO PR https://github.com/openzfs/zfs/pull/10018


On Tue, Apr 23, 2024 at 8:56 AM Matthew Ahrens  wrote:

> Apologies for the late notification, the next OpenZFS Leadership meeting
> will be held today, April 23 at 9am Pacific time.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-M03ba7d3fdfc3492aa15c8822
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Apologies for the late notification, the next OpenZFS Leadership meeting
will be held today, April 23 at 9am Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-Ma95dfef80845a5b4ca2ae8d0
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.


Video recording <https://www.youtube.com/watch?v=z7ond6CCLH0>

   -

   As the zfs encryption layer developer isn’t there anymore is zfs native
   encryption in actual state useful for production systems related to version
   2.1.15 and 2.2.3 or if not probably when?
   -

   Fast Dedup Reviews: https://github.com/openzfs/zfs/discussions/15896
   -

  ZAP shrinking has been extensively reviewed by Alexander Motin, need
  a 2nd reviewer to sign off
  -

   Improvements to zinject: https://github.com/openzfs/zfs/pull/15953
   -

   Single file scrubbing: https://github.com/openzfs/zfs/pull/16018
   -

   JSON kstat for zpool status: https://github.com/openzfs/zfs/pull/16026
   -

   Speculative prefetch for reordered requests:
   https://github.com/openzfs/zfs/pull/16022
   -

   OpenZFS DevSummit survey
   
<https://docs.google.com/forms/d/e/1FAIpQLSdANHOwTC58ykmqutduaSGYCSEk9-NvoZ76-fVCB-Y3bjcp4g/viewform>
   for planning the upcoming 2024 conference
   -

   When a program want to start and needs memory the kernel free up memory
   from used filesystem cache in linux but when you try that with zfs arc will
   not free memory and the application cannot get its requests, so you have to
   deal with the arc max kernel parameter before start the app. Will that
   change anytime in the future ?


On Mon, Mar 25, 2024 at 8:00 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, March 26 at 1pm
> Pacific time.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ta79ce30cc90661d8-Mc023aac84b05d3588152df31
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, March 26 at 1pm
Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ta79ce30cc90661d8-Ma66bb5d240f7cf550239b8be
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

>
> Thanks to everyone who participated.

Video recording <https://www.youtube.com/watch?v=GhSNdTg8RiE>
agenda:

   - Update ZFSd to match recent new features added to ZED (vdev properties
   for io/checksum/slow counts that cause device replacement):
   https://reviews.freebsd.org/D44043
   - Fast Dedup is up for review:
   https://github.com/openzfs/zfs/discussions/15896
   - Don’t stall MMP thread on slow I/O:
   https://github.com/openzfs/zfs/pull/15839
   - Support snapdir=disabled: https://github.com/openzfs/zfs/pull/15891
   - Klara has started work on improving arc_evict() for large memory
   systems
   - FYI, (0) days since last block cloning bug (
   https://github.com/openzfs/zfs/issues/15933)


On Mon, Feb 26, 2024 at 8:21 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, February 27th
> at 1pm Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-M935278348cc07044437b83c7
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, February 27th at
1pm Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-Mb6e757f713a90b13ba482461
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Second January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video recording is available:
https://www.youtube.com/watch?v=fbvkgSFy5Oc

topics discussed include:
>
>
   -

   OpenZFS 2.2.3 release: https://github.com/openzfs/zfs/pull/15836
   -

   Status of force pool export work:
   https://github.com/openzfs/zfs/pull/11082
   -

   Status of additional testings via QEMU:
   https://github.com/openzfs/zfs/pull/15838 (sample:
   https://github.com/mcmilk/zfs/actions/runs/7709789531) (Tino Reichardt)
   -

   MMP pool suspend from slow disk (Don Brady)
   -

   Do we really still want part9? (Rich Ercolani)

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T33d1b5f194b72a39-Ma2226c109fde212ba4866d42
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video recording is available:
https://www.youtube.com/watch?v=dOhRPyQzTA0

topics discussed include:

   -

   Block cloning bug recap
   -

   Ubuntu packages for releases (and main branch?)
   -

   Parallel import
   -

   Asynchronous ARC draining
   -

   abd/bio alignment and splitting


On Tue, Jan 2, 2024 at 9:56 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held today, January 2, at 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-Me926aed464edd4c20f8dff92
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

On Tue, Jan 9, 2024 at 11:04 AM Mai Ling  wrote:

> On Mon, Jan 4, 2021 at 7:28 PM Matthew Ahrens via openzfs-developer
>  wrote:
> >
> > The next OpenZFS Leadership meeting will be held tomorrow, December 8,
> 1pm-2pm Pacific time.  We don't have many topics on the agenda for
> tomorrow's meeting, so let me know if you'd like to add anything.
> [...]
> > --matt
> > openzfs / openzfs-developer / see discussions + participants + delivery
> options Permalink
>
> Hi,
>
> Sorry to bug you via this, but is there any chance that on one of
> these future meeting, the solving of issue of unable to export a
> suspended pool can be discussed? I know it doesn't really fall into
> LLNL ZFS use case, but to home users external usb storage and it
> causes much frustration.
>
> The work is stuck in https://github.com/openzfs/zfs/pull/11082


Sure, I'll put it on the agenda for the next meeting.

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-M4b9d1bf71a14448bc2a0617c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held today, January 2, at 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-Md00365695ab43fd05c9bd3b6
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] December OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video recording: https://youtu.be/OnuaWyt8QZo

agenda:

   -

   #15526 and related fallout: https://github.com/openzfs/zfs/issues/15526
   -

   Add slow disk diagnosis to ZED https://github.com/openzfs/zfs/pull/15469
   -

   BIO page splitting: https://github.com/openzfs/zfs/pull/15588
   -

   Block cloning tests: https://github.com/openzfs/zfs/pull/15631
   -

   ZIL prediction: https://github.com/openzfs/zfs/pull/15635
   -

   OverlayFS: https://github.com/openzfs/zfs/issues/15581

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T24bdaa2886c6cbf5-M490c8d8ff738fd41975c495b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: November OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated!

video recording: https://youtu.be/wy9M39Yb_7E

agenda:

   -

   Add slow disk diagnosis to ZED https://github.com/openzfs/zfs/pull/15469
   -

   Online rollback with mmap()
   -

   arc_prune()
   -

   Arc_evict/dbuf_evict CPU usage
   -

   Integrating ARC with page cache



On Mon, Nov 6, 2023 at 9:02 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, November 7, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc979815a83f938a9-M6e246e82c5909f50e320bbb3
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] RAIDZ Expansion Feature

[Matthew Ahrens via openzfs-developer]

That's right - RAIDZ (and mirroring) only use the space of the smallest
disk in the group, so they should normally be all the same size.  As with
other uses of `zpool attach` (to mirrors), the new drive needs to be no
smaller than the existing disks.

--matt

On Wed, Nov 8, 2023 at 4:09 PM Anthony D'Atri 
wrote:

> Nice.  Is it the case that added drives must be the same -- or no smaller
> than -- existing drives in the RAID-Z group?  The documentarian in me would
> call that out explicitly.
>
> On Nov 8, 2023, at 19:01, Don Brady  wrote:
>
> I'm happy to announce that the long-awaited *RAIDZ Expansion* feature has
> officially landed in the OpenZFS master branch. This feature will need some
> soak time, but will be available in the OpenZFS 2.3 release, which is
> probably about a year out.
>
> A huge thanks and shout out to Matt Ahrens for the design, implementation,
> and ongoing consultations. Also many thanks to the sponsors, The FreeBSD
> Foundation, iXsystems, Inc., and vStack.
>
> Thanks to our OpenZFS community members for code contributions, testing
> and feedback.  Including:
> Fedor Uporov, Stuart Maybee, Thorsten Behrens, Fmstrat, Mark Maybee, Brian
> Behlendorf, and others.
>
> Cheers,
>   Don Brady
>
> https://github.com/openzfs/zfs/commit/5caeef02fa531
>
>
> *openzfs * / openzfs-developer / see
> discussions  + participants
>  + delivery options
>  Permalink
> 
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T4db1687ae03a5df6-M8b714d5d80afec217489d8f7
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] November OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, November 7, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc979815a83f938a9-M486cfa1bdc3b0413a6777d5b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: October OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated in this short meeting.  FYI, we should
be posting videos from the DevSummit in the coming week.

recording: https://www.youtube.com/watch?v=7WtPHzPEcx4

topics:
* Zil performance

On Mon, Oct 9, 2023 at 8:46 AM Matthew Ahrens  wrote:

> Remember to register
> <https://www.eventbrite.com/e/openzfs-developer-summit-2023-tickets-685152027157>
> for next week's OpenZFS Developer Summit
> <https://openzfs.org/wiki/OpenZFS_Developer_Summit_2023> if you are able
> to attend!
>
> The next OpenZFS Leadership meeting will be held tomorrow, October 10, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td0c338d3ec8cf020-Ma5387b2e7ccf329d84cb240f
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] October OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Remember to register

for next week's OpenZFS Developer Summit
 if you are able to
attend!

The next OpenZFS Leadership meeting will be held tomorrow, October 10, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td0c338d3ec8cf020-M946bf8333d96554b77cebc54
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2023 speakers & talks announcement

[Matthew Ahrens via openzfs-developer]

I'm pleased to announce the talks that we'll have at the
OpenZFS Developer Summit
, which will be
held in San Francisco, October 16-17.  Please register

to
attend!  The conference will also be live-streamed and recorded, for those
who can't attend in person.

I hope to see you in person in 4 weeks!
TitleSpeakerCompany
Dedup Performance

Allan
Jude Klara Systems
OpenZFS at scale: learning, challenges, and awesome customers

Sam
Atkinson AWS
Idmapped Mount Support in ZFS and its Application

Youzhong
Yang MathWorks
Shared Log Pool

Paul
Dagnelie Delphix
RAIDZ Expansion

Matt
Ahrens; Don Brady Delphix, FreeBSD Foundation; iXsystems
Z.I.A. Accelerates ZFS Compression, Checksumming, and RAIDZ

Jason
Lee Los Alamos National Labs

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Teacceadd8e85484e-M7b3f333c85ec435c228a4062
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: September OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  For once I get the video posted
promptly!
https://youtu.be/0b6uAdtMgnc

Registration
<https://www.eventbrite.com/e/openzfs-developer-summit-2023-tickets-685152027157>
is open for the 2023 OpenZFS Developer Summit
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2023>, which will be
held October 16-17th in San Francisco.  The list of talks will be posted
next week.

Topics discussed:

   -

   2.2 release is close to final RC
   -

   Parallel dataset syncing PR
   -

   Fast Dedup design
   -

   Scrub specific txg range design
   -

   Verifying data on pool import
   -

   Flushing various logs to optimize pool import time
   -

   BRT for file concatenation
   -

   “Zfs receive” error handling



On Mon, Sep 11, 2023 at 6:56 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, September 12,
> 1pm Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T6cf0d73c6ec6e0cd-M55c871753c8561fadc8f07d4
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] September OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, September 12,
1pm Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T6cf0d73c6ec6e0cd-Mb54cc3cc60359c5eecdc3b10
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: OpenZFS Developer Summit 2023 - announcement & CFP

[Matthew Ahrens via openzfs-developer]

Reminder that there's less than a week until the deadline to submit talks.
If you've improved OpenZFS in any way, come talk about it!

We welcome proposals for any kinds of talks related to ZFS, and we are
particularly interested in fostering a diverse group of speakers. In the
past, the most popular talks have featured internal details of new or
upcoming features in OpenZFS, and explanations of how existing subsystems
work. Mentorship is available to help make your presentation the best it
can be.

Submit your proposal by September 5th by emailing m...@mahrens.org

--matt

On Tue, Aug 15, 2023 at 7:45 AM Matthew Ahrens  wrote:

> The eleventh annual OpenZFS Developer Summit will be held in person in San
> Francisco, *October 16-17, 2023*.  All OpenZFS developers are invited to
> participate, and registration
> <https://www.eventbrite.com/e/openzfs-developer-summit-2023-tickets-685152027157>
> is now open.
>
> See https://openzfs.org/wiki/OpenZFS_Developer_Summit_2023 for all of the
> details, including slides and videos from previous years' conferences.
>
> The goal of the event is to foster cross-community discussions of OpenZFS work
> and to make progress on some of the projects we have proposed.
>
> The first day of the conference will be set aside for presentations. *If
> you would like to give a talk, submit a proposal via email
> to m...@mahrens.org , including a 1-2 paragraph abstract.*
>
> We welcome proposals for any kinds of talks related to ZFS, and we are
> particularly interested in fostering a diverse group of speakers. In the
> past, the most popular talks have featured internal details of new or
> upcoming features in OpenZFS, and explanations of how existing subsystems
> work. Mentorship is available to help make your presentation the best it
> can be.
>
> The deadlines are as follows:
>
> *September 5, 2023 All abstracts/proposals submitted* to m...@mahrens.org
> September 15, 2023 Proposal submitters notified
> October 16-17, 2023 OpenZFS Developer Summit
>
> The second day of the event will be a hackathon, time set aside to work
> with OpenZFS developers, with the goal of having some progress to report
> at the end of the day.
>
> This event is only possible because of the efforts and contributions of
> the community of developers and companies that sponsor the event.  Our
> costs are higher this year due to being in-person as well as inflation; 
> sponsorship
> opportunities are available! Please see the website for details and send
> email to victo...@vgfevents.com if you have any questions.
>
> --matt
>>
>>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tbc593eaa318c743d-M3b3cebb05a31be111d80cb4b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: August OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  Meeting notes:

Video recording <https://youtu.be/sLX-bKEDpNE>

   -

   Reminder: OpenZFS Developer Summit October 16-17. CFP by September 5th
   Email: m...@mahrens.org, including a 1-2 paragraph abstract.
   -

   ZED slow I/O auto-spare [Allan]
   -

   ZED enc-path auto-replace [Allan]
   -

   RAID-Z Expansion [Don]
   -

   Fast Dedup prototype update (with performance comparison) [Allan]
   -

   Per-user reservations [Allan]
   -

   ZIL locking refactor v2 #15122
   <https://github.com/openzfs/zfs/pull/15122> [Alexander]
   - Tunable Compression Threshold #15174
   <https://github.com/openzfs/zfs/pull/15174> [Rich]


On Mon, Aug 14, 2023 at 9:21 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, August 14, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc8646411dbe3459a-M68480f423989942a9f1ffa1c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2023 - announcement & CFP

[Matthew Ahrens via openzfs-developer]

The eleventh annual OpenZFS Developer Summit will be held in person in San
Francisco, *October 16-17, 2023*.  All OpenZFS developers are invited to
participate, and registration

is now open.

See https://openzfs.org/wiki/OpenZFS_Developer_Summit_2023 for all of the
details, including slides and videos from previous years' conferences.

The goal of the event is to foster cross-community discussions of OpenZFS work
and to make progress on some of the projects we have proposed.

The first day of the conference will be set aside for presentations. *If
you would like to give a talk, submit a proposal via email
to m...@mahrens.org , including a 1-2 paragraph abstract.*

We welcome proposals for any kinds of talks related to ZFS, and we are
particularly interested in fostering a diverse group of speakers. In the
past, the most popular talks have featured internal details of new or
upcoming features in OpenZFS, and explanations of how existing subsystems
work. Mentorship is available to help make your presentation the best it
can be.

The deadlines are as follows:

*September 5, 2023 All abstracts/proposals submitted* to m...@mahrens.org
September 15, 2023 Proposal submitters notified
October 16-17, 2023 OpenZFS Developer Summit

The second day of the event will be a hackathon, time set aside to work
with OpenZFS developers, with the goal of having some progress to report at
the end of the day.

This event is only possible because of the efforts and contributions of the
community of developers and companies that sponsor the event.  Our costs
are higher this year due to being in-person as well as inflation; sponsorship
opportunities are available! Please see the website for details and send
email to victo...@vgfevents.com if you have any questions.

--matt
>
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tbc593eaa318c743d-M2daae74b3ae57f2fdb20f740
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] August OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, August 14, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc8646411dbe3459a-M8ef58e7d172b5cc73ec8b3b3
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] July OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated at last week's OpenZFS Leadership
Meeting

meeting recording: https://youtu.be/HjkV7vuKATM

agenda:

   -

   Raid-Z Expansion https://github.com/openzfs/zfs/pull/15022 (Don)
   -

   Gang ABDs causing alignment issues on 4Kn drives (Allan Jude)
   -

   Review Request: https://github.com/openzfs/zfs/pull/15050 BRT for Linux
   (Allan Jude)
   -

   ZED Enhancements re: slow I/Os
   -

   Are there any side effects to NOT unloading metaslabs
   (metaslab_debug_unload)?
   -

   Test failures

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ted9f300b994b7c2a-M98f122e3b79963db077dca72
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: June OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

meeting recording: https://youtu.be/2p32m-7FNpM

Notes:

   -

   RAIDZ Expansion (Don Brady)
   -

   Add more Github Action runners (arm64, amd64, ppc64 via openstack
   machines of the Oregon State University Open Source Lab (Tino Reichardt)
   -

   Dedup improvements (Allan Jude)


On Mon, Jun 19, 2023 at 5:09 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, June 20, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td25434f74c9df977-Ma8898052c4d121b9b59dd7d1
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] June OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, June 20, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td25434f74c9df977-M537821d80a9db6c8bf504ace
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] May OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, May 23rd, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tbc2723be3e4d22d4-M8469a60ee6bc031e104c1115
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

meeting recording: https://www.youtube.com/watch?v=sZJMFvjqXvE

notes:

   -

   Block cloning status after FreeBSD merge (Pawel Dawidek)
   -

   Hierarchical rate limiting demo (Pawel Dawidek)
   -

   I get in some days / weeks 6x POWER and 6x AARCH VM’s for Github action
   runners (4 Cores, 4GB RAM, 40GB Disk)  /Tino
   -

   Update on 2.2 timing perhaps? Lots of cool stuff queued up
   -

   RAIDZ Expansion
   -

   OSX support update
   -

   Direct IO Support


On Tue, Apr 25, 2023 at 9:37 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held today, April 25, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-M91885d114f25d76bf51178f7
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held today, April 25, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-M5ad66c0b23e5e291c3e505b7
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video: https://www.youtube.com/watch?v=vrTSukIwQzQ

Notes:

   -

   The OpenZFSonosx developers suggest it is time to decide if it is
   desired to merge the macOS port. #12110 The PR is currently up to date, and
   is the latest installable release.
   -

   IO rate limiting (Pawel Dawidek, Allan Jude)


On Tue, Mar 28, 2023 at 8:42 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held today, March 28, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ta79ce30cc90661d8-M9a5ece5e991edd85ccd2379b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held today, March 28, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ta79ce30cc90661d8-Maf89248f64309c910bef35eb
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] Introduction and request to join #openzfs slack

[Matthew Ahrens via openzfs-developer]

Welcome Benjamin, I sent you an invite to the openzfs slack.

--matt

On Mon, Mar 20, 2023 at 6:58 AM Benjamin Coddington 
wrote:

> Hello open-zfs developers,
>
> I've been hacking on openzfs to implement linux copy_file_range() and
> remap_file_range() file_operations to take advantage of the recently-merged
> BRT work.  I have a first pass at that here:
> https://github.com/bcodding/zfs/tree/linux-brt
>
> Normally, I am caring for NFS in a common linux distro, or AFK doing
> various personal projects.
>
> I'd like to potentially submit the work I have for review and/or testing,
> but since I am new here I will be making mistakes and asking for
> direction.  To that end, I'd like to be able to communicate with other
> developers on the #openzfs slack channel.  May I have an invite?
>
> Best,
> Benjamin Coddington
>
> *openzfs * / openzfs-developer / see
> discussions  + participants
>  + delivery options
>  Permalink
> 
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T4044e4a8e2e8ba6e-Mb42ba52738795b6363fbb45c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video recording: https://www.youtube.com/watch?v=LcHmXy34seA

notes:

   -

   Block Cloning status update (Pawel Dawidek)
   -

   Shared Slog PR introduction (Paul Dagnelie)
   -

  https://github.com/openzfs/zfs/pull/14520
  -

   Discussed mirrored L2ARC and what it is not allowed, and possibly
   allowing RAID-Z for special vdevs
   -

   I/O rate limiting properties issues (Pawel / Allan)
   -

   fsync to return error on failmode=continue (Allan Jude)

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7864f270c9f36e85-Me9972b91c29a659764c5c773
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: 2nd January 2023 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video recording: https://youtu.be/94HaCotHBAA

notes:
 * Will 2.2 be branched at some point soon? (asking _because overlayfs support 
is not planned to be backported to 2.1._ 
*)* 


 * Spurious EIO with zfs-2.1.8 _https://github.com/openzfs/zfs/issues/14413_


 * I recently submitted _PR#14249_ , 
adding Chacha20-Poly1305 as an encryption option. I'd like to talk to someone 
about next steps towards getting this PR reviewed and merged. (Rob N)


 * ZFS Space calculations (raidz deflation)


 * More adaptive (data/metadata) ARC eviction: _More adaptive ARC eviction. 
#14359 - openzfs/zfs · GitHub_ 




--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T43125b87531d70aa-M836961f62cffa3f18a1d61c0
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] 2nd January 2023 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, January 31, 1pm
Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T43125b87531d70aa-Md2e1826658747bb847e377bb
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: January 2023 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

Video recording <https://youtu.be/D6gOgkEG9X0>

   -

   I recently submitted PR#14249 <https://github.com/openzfs/zfs/pull/14249>,
   adding Chacha20-Poly1305 as an encryption option. I'd like to talk to
   someone about next steps towards getting this PR reviewed and merged. (Rob
   N)
   -

   Should the documentation have warnings about using native encryption,
   similar to dedup? (Rich E)
   -

  Several people who got burned by native encryption recently asked me
  why there were no warnings around it if it has known bad failure
modes, and
  I didn’t really have a good answer.
  -

  Klara investigating: https://github.com/openzfs/zfs/issues/12775
  -

   mmap issue: https://github.com/openzfs/zfs/issues/13608
   -

   Curious about the effort to update the ZFS On Disk Format document.  Has
   this been done?  If so, where is the latest version?
   -

  https://github.com/ahrens/zfsondisk



On Tue, Jan 3, 2023 at 8:08 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held today, January 3, 1pm
> Pacific time.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T4cec65b4195af58e-Mf1235b2652e4ede8cb471a90
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] January 2023 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held today, January 3, 1pm
Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T4cec65b4195af58e-Mdd1fbc551233fe9c59291e95
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: December 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.
video posted: https://youtu.be/atbW3Ecqm5s

topics

   -

   Unmapped prefetch (requesting review)
   https://github.com/openzfs/zfs/pull/14243
   -

   ZED configuration via vdev properties (requesting review)
   https://github.com/openzfs/zfs/pull/13805
   -

   Top-level vdev properties (add a ZAP for the root of the pool for
   property inheritance, for newly added disks)
   -

   Bandwidth quotas


On Mon, Dec 5, 2022 at 9:16 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held Tomorrow, December 6, 1pm
> Pacific time.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T3d211061afbde05a-M6c53c1c61d314be15527f8e2
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: November 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Video recording: https://www.youtube.com/watch?v=CDe1_i37zVs

Topics included: quota performance; ARC MRU/MFU; BRT details

On Tue, Nov 8, 2022 at 7:53 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held Today, November 8, 9am
> Pacific time.  i.e. in about an hour.  Sorry for the late notice!
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2ce0c0085295790b-M39744eec2587192bc5a8c5d9
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] December 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held Tomorrow, December 6, 1pm
Pacific time.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T3d211061afbde05a-Macd2c980a4d4eb48b42feac8
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit videos

[Matthew Ahrens via openzfs-developer]

It was great seeing many of you at the OpenZFS Developer Summit last
month.  The videos of the presentations have been posted on the website
 and YouTube:

TitleSpeakerCompanySlidesVideo
State of OpenZFS Matt Ahrens Delphix slides

Operating OpenZFS at scale

Satabdi
Das Amazon slides

video

Special Announcement Jorgen Lundman slides

video

Block Cloning for OpenZFS

Pawel
Jakub Dawidek Fudo Security slides

video

CHERI memory safety

Brooks
Davis SRI International slides

video

zvol performance

Tony
Hutter LLNL slides

video

Refining OpenZFS compression

Rich
Ercolani Independent slides

video

Run ZFS in userland

Ping
Huang IOMesh slides

video

Faster ZFS scrub and other improvements

Alexander
Motin iXsystems slides

video

Storage Multi-Tenancy For Containers

Allan
Jude Klara Systems slides

video

Shared L2ARC

Christian
Schwarz Nutanix slides

video
Hackathon
presentations video


--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te6e724dc511326ea-M7714fd3d5a7a2df8127b8d00
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] November 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held Today, November 8, 9am
Pacific time.  i.e. in about an hour.  Sorry for the late notice!

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2ce0c0085295790b-Mbd1319e959fc7219a150080d
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit live stream

[Matthew Ahrens via openzfs-developer]

You can watch the OpenZFS Developer Summit live, starting in about 10
minutes, using the following link:
https://vimeo.com/event/2485132/90e1506dfb

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T3549e40c323b2d83-Ma4ba3ded39522691c2326bea
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] GPU Accelerated Checksumming

[Matthew Ahrens via openzfs-developer]

Yes, to expand on what Rich said, there was a talk about Intel QAT offload
of gzip at the 2018 OpenZFS Developer Summit:
ZFS Hardware Acceleration with QAT
 Weigang Li
Intel slides

video
The
results presented show >2x throughput with <1/2 the CPU used, and similar
compression to gzip software (I'm guessing with the default gzip level).

QAT support has been in ZFS since 0.8.0.

--matt

On Mon, Oct 17, 2022 at 12:15 PM Rich  wrote:

> I believe the Intel QAT support we have will happily offload gzip for you,
> though I don't know if it makes any promises about what level equivalent of
> gzip it hands you back...
>
> - Rich
>
> On Mon, Oct 17, 2022 at 12:02 PM Garrett D'Amore 
> wrote:
>
>> That’s about what I would have expected.
>>
>> Having an offload for high levels of compression (e.g. GZIP 9 or
>> something) would be cool, but I don’t think it exists yet.  And it would be
>> hard to write that in a way that doesn’t punish things for the folks who
>> *don’t* have the offload hardware.
>>
>>- Garrett
>>
>> On Oct 17, 2022, 8:44 AM -0700, Sanjay G Nadkarni via openzfs-developer <
>> developer@lists.open-zfs.org>, wrote:
>>
>>
>>
>> We have been doing regular performance runs using various workloads over
>> NFS(v3,v4.1), SMB3, iSCSI and FC16 & 32 for the past few years. Compression
>> is enabled for all datasets and zvols in our runs. What we have observed
>> is, under load, compression consumes the highest CPU cycles, after that it
>> is a toss up of dnode locking (a well known issue) and other things that
>> might come into play depending on the protocol.
>>
>> At least in our use cases check summing of blocks does not appear to an
>> issue.
>>
>> -Sanjay
>>
>>
>>
>>
>> On 10/14/22 10:15 AM, Garrett D'Amore wrote:
>>
>> I can tell from past experiences that offloads like what you are
>> proposing are rarely worth it.  The set up and tear down of the mappings to
>> allow the data transport are not necessarily cheap.  You can avoid that by
>> having a preallocated region, but then you need to copy the
>> data.  Fortunately for this case you only need to copy once, since the
>> result will be very small compared to the data.
>>
>> Then there is the complexity (additional branches, edge cases, etc.) that
>> have to be coded.  These become performance sapping as well.
>>
>> Add to this the fact that CPUs are always getting faster, and
>> advancements like extensions to the SIMD instructions mean that the
>> disparity between the offload and just doing the natural thing inline gets
>> ever smaller.
>>
>> At the end of the day, it’s often the case that your “offload” is
>> actually a performance killer.
>>
>> The exceptions to this are when the work is truly expensive.  For
>> example, running (in the old days) RSA on an offload engine makes a lot of
>> sense.  (I’m not sure it does for elliptic curve crypto though.)  Running
>> 3DES (again if you wanted to do that, which you should not) used to make
>> sense.  AES used to, but with AES-NI not anymore.  I suspect that for SHA2
>> its a toss up.  Fletcher probably does not make sense.  If you want to
>> compress, LZJB does not make sense, but GZIP (especially at higher levels)
>> would, if you had such a device.
>>
>> Algorithms are always getting better (newer ones that are more optimized
>> for actual CPUs etc.) and CPUs are always improving — the GPU is probably
>> best reserved for truly expensive operations for which it was designed —
>> complex transforms for 3D rendering, expensive hashing (although I wish
>> that wasn’t a thing), long running scientific analysis, machine learning,
>> etc.
>>
>> As an I/O accelerator, not so much.
>> On Oct 14, 2022, 7:52 AM -0700, Thijs Cramer 
>> , wrote:
>>
>> I've been searching the GitHub Repository and the Mailing list, but
>> couldn't find any discussion about this.
>> I know it's probably silly, but I would like to understand the workings.
>>
>> Let's say one could offload the Checksumming process to a dedicated GPU.
>> This might  save some amount of CPU, *but* might increase latency
>> incredibly.
>>
>> To my understanding ZFS uses the Fletcher4 Checksum Algorithm by default,
>> and this requires a pass of the data in-memory as it calculates the
>> checksum. If we skip this step, and instead send the data to the GPU, that
>> would also require a pass of the data (no gains there).
>>
>> The actual calculation is not that hard for a CPU it seems, there are
>> specific SIMD instructions for calculating specific Checksums, and after a
>> quick pass over the code, it seems they are already used (if available).
>>
>> I think the only time that a GPU could calculate checksums 'faster', is
>> with a form of readahead.
>> If you would  pre-read a lot of data, and dump it to 

[developer] Re: October 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

notes:

Video recording <https://youtu.be/oNNTEIYX2zc>

   -

   Are we still planning to release 2.2 in November?
   -

  Would like to see the branch created before the end of this month
  -

   We at iX has grown a need for OverlayFS for Docker purposes, so actively
   looking on https://github.com/openzfs/zfs/pull/9600 and
   https://github.com/openzfs/zfs/pull/12209 .  The first seems to mostly
   need Linux VFS expert review.
   -

   Hackathon ideas spreadsheet
   
<https://docs.google.com/spreadsheets/d/1ilt78urd1RmxACjd5OSQU6pGlRYp2RGRh1kFuVBgGeY/edit#gid=0>


On Mon, Oct 10, 2022 at 8:00 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held Tuesday, October 11th,
> 1pm Pacific time.  This will be the last meeting before the OpenZFS
> Developer Summit <https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022>.
> Please register if you'd like to attend in person.  We'll also be
> live-streaming the presentations on October 24 and 25.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T871edf51c0937013-Maa4f48d734e971bafa117515
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] October 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held Tuesday, October 11th, 1pm
Pacific time.  This will be the last meeting before the OpenZFS Developer
Summit .  Please
register if you'd like to attend in person.  We'll also be live-streaming
the presentations on October 24 and 25.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T871edf51c0937013-M16ed06b0cbfa7b01e7fb68c8
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: September 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video
<https://youtu.be/pWSXah94Tw0> is now available.

The OpenZFS Developer Summit is in 5 weeks.  Register today
<https://www.eventbrite.com/e/openzfs-developer-summit-2022-tickets-387152452407>
to join us in person for a day of presentations and a 1-day hackathon.  We
have 9 great talks from community members:

TitleSpeakerCompany
State of OpenZFS Matt Ahrens Delphix
Operating OpenZFS at scale
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Operating_OpenZFS_at_scale_(Satabdi_Das)>
Satabdi
Das Amazon
Block Cloning for OpenZFS
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Block_Cloning_for_OpenZFS_(Pawel_Dawidek)>
Pawel
Jakub Dawidek Fudo Security
CHERI memory safety
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#CHERI_memory_safety_and_ZFS_(Brooks_Davis)>
Brooks
Davis SRI International
zvol performance
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#zvol_performance_(Tony_Hutter)>
Tony
Hutter LLNL
Run ZFS in userland
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Run_ZFS_in_userland_(Ping_Huang)>
Ping
Huang SmartX
Refining OpenZFS compression
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Refining_OpenZFS_Compression_%E2%80%93_a_couple_things_that_worked,_and_many_that_didn%E2%80%99t_(Rich_Ercolani)>
Rich
Ercolani Independent
Faster ZFS scrub and other improvements
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Faster_ZFS_scrub_and_other_improvements_(Alexander_Motin)>
Alexander
Motin iXsystems
Storage Multi-Tenancy For Containers
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Enabling_Storage_Multi-Tenancy_With_ZFS_For_Containers_(Allan_Jude)>
Allan
Jude Klara Systems
Shared L2ARC
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022_Talks#Shared_L2ARC_(Christian_Schwarz)>
Christian
Schwarz Nutani
video: https://youtu.be/pWSXah94Tw0

notes:

   -

   OpenZFS DevSummit talk announcement
   <https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022> (Matt)
   -

   How ZFS partitions disks (Michael Dexter)
   -

   Status on encryption bugs (Hendrick)
   -


  
https://github.com/openzfs/openzfs/commit/619c0123a513678d5824d6b1c4d7e8fad3c63e76
  -

  panic: zfs dbuf code destroys buffer still held by others on
  encrypted pool
  -

  https://www.illumos.org/issues/14003
  -

  https://jira.ixsystems.com/browse/NAS-113491
  -

  Updating encryption properties on a dataset makes all children
  unreadable + kernel panic
  -

  https://github.com/openzfs/zfs/issues/11679
  -

  https://github.com/openzfs/zfs/issues/12123


On Tue, Sep 13, 2022 at 7:47 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held today, September 13th,
> 1pm Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2174d9f005b3d69d-M2193e494efa1fc15d77b2bd3
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2022 speakers & talks announcement

[Matthew Ahrens via openzfs-developer]

I'm pleased to announce the nine great talks that we'll have at the OpenZFS
Developer Summit ,
which will be held in San Francisco, October 24-45.  Please register

to
attend!  The conference will also be live-streamed and recorded, for those
who can't attend in person.

I hope to see you in person in 5 weeks!

TitleSpeakerCompany
State of OpenZFS Matt Ahrens Delphix
Operating OpenZFS at scale

Satabdi
Das Amazon
Block Cloning for OpenZFS

Pawel
Jakub Dawidek Fudo Security
CHERI memory safety

Brooks
Davis SRI International
zvol performance

Tony
Hutter LLNL
Run ZFS in userland

Ping
Huang SmartX
Refining OpenZFS compression

Rich
Ercolani Independent
Faster ZFS scrub and other improvements

Alexander
Motin iXsystems
Storage Multi-Tenancy For Containers

Allan
Jude Klara Systems
Shared L2ARC

Christian
Schwarz Nutanix

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc41caf47077c9515-M55ef76a60c159e075e06b375
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] September 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held today, September 13th, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2174d9f005b3d69d-Mb08700110edadd8f76f744e8
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit talk proposals due today

[Matthew Ahrens via openzfs-developer]

Today is the deadline to submit a talk proposal for the OpenZFS Developer
Summit! The conference will be in-person in San Francisco, October 24-25th.


We welcome proposals for any kinds of talks related to ZFS, and we are
particularly interested in fostering a diverse group of speakers. In the
past, the most popular talks have featured internal details of new or
upcoming features in OpenZFS, and explanations of how existing subsystems
work. Mentorship is available to help make your presentation the best it
can be.

*If you would like to give a talk, submit a proposal today via email
to m...@mahrens.org , including a 1-2 paragraph abstract.*

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T0aaf6eae21fc37ea-M9b5cb74f2a953cf8a55e279c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: August 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  Reminder: talk submissions for the
OpenZFS Developer Summit are due in less than a week - August 29th!

video recording: https://youtu.be/fi_ZKc_od54

notes:

   -

   ZED configurability: does a vdev property make the most sense? Need 4
   new properties
   -

  Possibly a root-vdev property, or recursive set
  -

   Do we need a failmode=error (or failmode=fail). Where fsync() etc do not
   hang forever, but actually return an error?
   -

   Discussed investigating changing the default maximum size of a micro-zap
   -

  Also discussed if the default size of indirect blocks should be
  lowered (already a tunable)


On Mon, Aug 15, 2022 at 9:27 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, August 16, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T0832205c7b63bfd5-M6538dd8323c561b81011cb23
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] August 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, August 16, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T0832205c7b63bfd5-M99c1f9bd700ab4c73f155d65
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: July 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Reminder that the deadline for submitting talks for this year's OpenZFS
Developer Summit <https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022>
is in 3 weeks!

Here is the video of July's meeting: https://youtu.be/GPgy9ATYeYY

notes:

   -

   OpenZFS conference announcement (Matt)
   -

  Matt will send out announcements later today
  -

   Relaxed quota enforcement for performance (Allan)
   -

  When you’re close to quota, things get slow.
  -

  Plan:
  -

 Change to allow filesystem to go over quota by one txg worth of
 dirty data or percent of quota, whatever less.
 -

 For zvols, don’t enforce quota at all, or at least allow
 substantially higher.  Quota makes sense for ZVOLs though, at
least in some
 rare scenarios, e.g, thin provisioned ZVOLs.
 -

  Discussion on whether the behavior should be opt-in via a property or
  whether we can be a little more relaxed about quotas. Sentiment was that
  avoiding a property is preferable.
  -

  Expect a PR soon.
  -

   How best to store configuration for ZED (Allan)
   -

  Currently, some variables are hard-coded in the ZED C code.
  -

  It would make sense to make some of these configurable.
  -

  Question: config file in /etc, or will some command line arguments to
  ZED suffice?
  -

  Decision: make it command line options to ZED, then users can use
  init system features to override arguments.
  -

  Still, there are use cases for making it a zpool property.
  -

 Lengthy digression on a weird phantom error that bubbles up to ZED
 on Linux with certain Broadcom chipsets. For such cases, a
zpool property
 would be appropriate.
 -

   What are the plans for 2.2 release?  Or shall I merge scrub performance
   optimizations into 2.1 branch? (Mav)
   -

  If we stick to the previous cadence, then the due date for 2.2 is
  November.
  -

  Features that are close to completion that we should include if
  possible
  -

 Direct IO
 -

 BRT
 -

   Review requests
   -

  https://github.com/openzfs/zfs/pull/12773 (spa_asize_inflation)
  (Allan)
  -

  https://github.com/openzfs/zfs/pull/9372 (corruption healing recv)
  (Alek)
  -

   Has anyone experimented with directory scaling (how performance drops
   off with a large number of files in a directory) (Allan)
   -

  Allan will provide some data next time.


On Mon, Jul 18, 2022 at 8:00 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, July 17, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td1de445c83003513-Mf69b27e605f68e3c3a022a05
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2022 - announcement & CFP

[Matthew Ahrens via openzfs-developer]

The tenth annual OpenZFS Developer Summit will be held in person in San
Francisco, *October 24-25, 2022*.  All OpenZFS developers are invited to
participate, and registration

is now open.

See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2022 for all of
the details, including slides and videos from previous years' conferences.

The goal of the event is to foster cross-community discussions of OpenZFS work
and to make progress on some of the projects we have proposed.

The first day of the conference will be set aside for presentations. *If
you would like to give a talk, submit a proposal via email
to m...@mahrens.org , including a 1-2 paragraph
abstract.* Talks
may be recorded prior to the event, and speakers' (virtual) attendance the
day of the event will allow for everyone to experience the talks together,
and for Q with attendees.

We welcome proposals for any kinds of talks related to ZFS, and we are
particularly interested in fostering a diverse group of speakers. In the
past, the most popular talks have featured internal details of new or
upcoming features in OpenZFS, and explanations of how existing subsystems
work. Mentorship is available to help make your presentation the best it
can be.

The deadlines are as follows:

*August 29, 2022 All abstracts/proposals submitted* to m...@mahrens.org
September 9, 2022 Proposal submitters notified
October 24-25, 2022 OpenZFS Developer Summit

The second day of the event will be a hackathon, time set aside to work
with OpenZFS developers, with the goal of having some progress to report at
the end of the day.

This event is only possible because of the efforts and contributions of the
community of developers and companies that sponsor the event.  Our costs
are higher this year due to being in-person as well as inflation; sponsorship
opportunities are available! Please see the website for details and send
email to victo...@vgfevents.com if you have any questions.

--matt
>
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T0aba7b42eca96262-M1c19c2e456ef9c97e0cd5c43
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] July 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, July 17, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td1de445c83003513-M402d9dbd88693e01b8e32a98
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: June 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

meeting recording: https://youtu.be/WiyrHu6R8ow

Thanks Christian for these detailed notes:

   -

   Give an update on current status of the Direct IO PR (Brian Atkinson)
   -

  Will require the app to not modify memory while write is in progress,
  but have an opt-in safety check in the ZIO pipeline to catch it. If it is
  caught, then all of the following happen:
  -

 ZED event
 -

 zpool status message
 -

 write syscall fails with EINVAL
 -

  Check is off by default because of bcopy+checksum overhead (bcopy is
  dominant)
  -

  Marking the page read-only to catch in-flight modification is not an
  option
  -

  Brainstorming to avoid overhead:
  -

 checksum without bcopy (susceptible to transient modification)
 -

 opportunistic (only check in X% of cases, where X is low enough
 that performance impact is low)
 -

Seems like this option has consensus.
-

  Concerns regarding in-flight modification & compression. Checksum
  might match, but then decompression would fail. So, we need to make sure
  that compression always copies the buffer, iff it’s a userspace buffer
  (abd_borrow_buf_copy doesn’t guarantee copy right now).
  -

  Also: Brian has one weird failure on FreeBSD. Has a branch with more
  debug statements & reproducer. Needs help from FreeBSD devs!
  -

   Zio taskq scaling for multiple pools (Allan)
   -

  zio_taskq_batch_pct defaults to 75% of CPU cores
  -

  But with N pools, we create N times those threads, which is much more
  than actually available CPU cores.
  -

  Brainstorming session:
  -

 Make such taskqs global
 -

Fairness & starvation shouldn’t be a problem because the taskqs
are FIFO
-

But, fear of deadlocks if taskq operations are interdependent.
Are there any?
-

Also: lock contention on taskq locks is amplified by making
them global.
-

  Conclusion: not an obvious solution, system-wide taskq is worth
  experimenting. (=> good hackathon project)
  -

   Multiple user-keys for encryption (Jonathan Waldrep)
   -

  UX proposal: use @ notation on the properties to distinguish
  different keys (keyformat@key1, keyformat@key2, …)
  -

   Soliciting preferences on #11679
   <https://github.com/openzfs/zfs/issues/11679> solutions (Rich)
   -

  NULL pointer deref on encrypted recv
  -

  It’s a race in which you end up taking dbuf_write’s arc_write
  codepath, which temporarily NULLs the buffers of what it’s writing, while
  something else ends up doing a dbuf_read -> … ->
  arc_buf_untransform_in_place, which unconditionally hands the
b_pabd in to
  decrypt from, and is sometimes NULL unexpectedly.
  -

  George: if the buffer is being written, it shouldn’t be discoverable
  in the first place.
  -

  Rich’s Summary
  <https://github.com/openzfs/zfs/issues/11679#issuecomment-1138941249>
  provides details of how the condition happens.
  -

  George will take a look to understand why the buffer is discoverable.
  -

   OpenZFS Conference
   -

  Dates: likely early November.
  -

  Opportunities to help out are still available (see April 26 meeting)


On Mon, Jun 20, 2022 at 9:45 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, June 21, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T9f9fe51bf1909400-Mbbc16f0e98128fda03445c07
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: May 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video is posted here: https://youtu.be/DLUIAn2jL9g

I don't have detailed notes from this meeting but the topics discussed
include:

   -

   Fiemap by rohan-puri · Pull Request #9554
   <https://github.com/openzfs/zfs/pull/9554>
   -

   Blake 3
   -

   Block Reference Table
   -

   Write throttle smoothing PR


On Mon, May 23, 2022 at 9:42 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, May 24, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2fb2d18fc7c5cd6b-M872eb8a230f4c7f0299a8161
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] June 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, June 21, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T9f9fe51bf1909400-M0e6c3cfafaf99f121879fd80
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] May 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, May 24, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T2fb2d18fc7c5cd6b-Mbe41086eccacc679aea50a78
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: April 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated, and to Christian for taking notes

meeting recording: https://youtu.be/fTIJgGpV3HE


   -

   What about PR #12918 <https://github.com/openzfs/zfs/pull/12918> ? Will
   it be integrated ? (BLAKE3)
   -

  Looks like it’s moving forward.
  -

  Needs reviews from requested reviewers.
  -

   Matt: Thoughts on the next OpenZFS Conference.
   -

  High interest in an in-person event among the meeting attendees.
  -

  Safety measures: decide closer to the conference. But don’t relax the
  guidelines after announcing them.
  -

  Matt asks for a community member to help with organizing the
  conference:
  -

 Work with sponsoring companies
 -

 Catering
 -

 AV
 -

 Venue
 -

 Date
 -

  Perk: helping organize means some control. e.g., venue location. The
  Bay Area would still be a requirement, though.
  -

   Allan: Linux Namespace PR update
   -

  Will hold fd of the namespace (procfs path) to keep it alive while
  dataset is jailed. Results in slight UI change: refer to namespace by
  procfs path.
  -

   Allan: Write Inflation PR
   -

  Needs help with a failing negative test.
  -

   Michael Dexter: Can OpenZFS native encryption be recommended with
   confidence or specific caveats, not unlike deduplication?
   -

  Send/recv has several known bugs.
  -

 See Rich’s spreadsheet
 
<https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg/edit>
 (lots of these are likely duplicates of each other)
 -

  Standalone (= without send/recv) has few bugs.
  -

 Illumos has an open bug <https://www.illumos.org/issues/14003>
 related to projectquota upgrades.
 -

 Alex Motin: some TrueNAS / FreeNAS users are using native
 encryption already, plan is to only support it, instead of GELI,
 in future releases.
 -

 zfs change-key has an open issue
 <https://github.com/openzfs/zfs/issues/12123>
 -

  No crisp answer to the question.
  -

   Rich: update on compression early-abort, using lz4 as heuristic
   -

  He pushed a bunch of more data.
  -

   BRT: what’s the status of the work?
   -

  GitHub branch was last updates in February
  -

  Pawel just didn’t get around creating the upstream PR. Will try to do
  that this week.
  -

  Limitation: with encryption, currently only copies within the same
  dataset are covered by BRT. We can lift that limitation in a
later PR, for
  datasets in the same clone family.
  -

  Linux copy_file_range has been hooked up.
  -

  FreeBSD doesn’t have an equivalent yet, so the PR adds a syscall.
  Long-term, Pawel hopes FreeBSD will grow a generic VOP interface.
  -

   Request for reviews:
   -

  #13130 <https://github.com/openzfs/zfs/pull/13130> support
  incremental receive of clone streams (DRR_FLAG_CLONE)
  -

  #12767 <https://github.com/openzfs/zfs/pull/12767> fix use-after-free
  of znode_t in race between zfs_zget and zfs_rmnode/zfs_znode_dmu_fini
  -

  #9372 <https://github.com/openzfs/zfs/pull/9372> corruption
  correcting recv


On Mon, Apr 25, 2022 at 9:20 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, April 26th, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc43536da9250bfdc-M61d4c3037c621d4c7d1911e0
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] April 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, April 26th, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc43536da9250bfdc-M55c6ef3b9939b4cede7c6433
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: 2nd March 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

ON)/1.5.2+(SSE|NEON)
 continues
 -

 Preliminary results not very compelling:
 -

(“s” versions built with SSE2 optimizations and kernel_fpu
dance)
-

5900X incompressible chart

<https://www.dropbox.com/s/f4ft7qjlpnmlfoc/zstd%205900X%20incompressible%201M.png?dl=0>
-

5900X compressible chart

<https://www.dropbox.com/s/b2o1f2a2xj7th6k/zstd%205900X%20compressible%201M.png?dl=0>
-

 Same thing about not needing a feature flag as LZ4 - though zstd
 already had a version in its header field, so no fun games needed
 -

 PR: same as LZ4, if I find a use case it’s an improvement for or a
 flaw in my approach that results in better performance
 -

  gzip update
  -

 Kidding…
 -

 …but I really did experiment with it.
 -

 Tried benchmarking a few different zlib forks instead of Linux
 builtin, none were compellingly different so far
 -

 Quick and dirty graphs of zlib-ng, which was the most different,
 here <https://github.com/openzfs/zfs/issues/13245>
 -

 Seems like at best maybe the decompressor might be worth
 examining, and the SIMD improvements don’t seem to make much
difference for
 our use cases.
 -

   Questions about tests like refquota_008_neg that are giving false fails
   after the write inflation change PR #12773
   <https://github.com/openzfs/zfs/pull/12773>
   -

  Solution: make the test lower the dirty data max tunable.
  -

   Christian: questions on TBD PR that makes ZTS detect if a test does not
   properly restore tunable values.
   -

  test-runner.py is an internal interface, zfs-tests.sh is the external
  interface of the test suite. Ok to change / cut down test-runner.py
  functionality that is no longer in use.
  -

  If we can detect the tunable change, can’t we just reset them and
  continue running tests? Maybe, if we assume any tunable can be
written can
  be written in any order. But a test that doesn’t restore tunable values
  properly is a bug. So, just fix any broken tests once in this
PR, then this
  problem should be solved going forward.


On Mon, Mar 28, 2022 at 3:38 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, March 29th, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tde9acb4c64be171e-Mf5669337fa4729706acd4beb
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] 2nd March 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, March 29th, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tde9acb4c64be171e-M4f6912ad6fd71c0ca07a2f58
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] March 2022 OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, March 1st, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc2ee7254a86e787f-M531dabb5a307736d5185529a
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

ort)
  -

  https://github.com/openzfs/zfs/pull/12789 (log spacemap load time)
  -

  https://github.com/openzfs/zfs/pull/12767 (use-after free of znode_t)


On Mon, Jan 31, 2022 at 11:20 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, February 1, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-Me844c6f99850f181fe89f619
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  Sorry for the delay in getting this
email sent!
Meeting recording is now posted: https://www.youtube.com/watch?v=tyCk376bC9A

Notes (thanks Christian):

   -

   Encryption Bugs (Rich Ercolani)
   -

  Rich’s spreadsheet is now publicly accessible
  
<https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg/edit#gid=1560550070>

  -

  One new bug was reported that doesn’t involve replication: #1293
  <https://github.com/openzfs/zfs/issues/12931>1 (dereferencing a wild
  pointer in the checksum code with encryption+dedup on under heavy load)
  -

  Compressors (Rich Ercolani)
  -

 Issue that sparked the discussion: #12840
 <https://github.com/openzfs/zfs/issues/12840>
 -

 Summary: Replacing the compressor code causes problems if the
 output is not bit-for-bit identical.
 -

 Problem #1: Uncompressed ARC + Compressed On-Disk +  L2ARC
 -

Assume uncompressed ARC, which holds the data of a block that
is compressed on disk.
-

The L2ARC invariant is that it stores a bit-for-bit identical
copy of the on-disk data.
-

So, the L2ARC write must recompress the uncompressed copy.
-

But if the compressor changed, the recompressed copy will not
be bit-for-bit identical to the on-disk data, and the
L2ARC read will fail
with a checksum error, resulting in an L2ARC miss.
-

 Problem #2: NOP writes
 -

TODO explain
-

 Problem #3: Dedup
 -

Hashing is against the compressed blocks. Thus, blocks
compressed with the new compressor will not dedup with
blocks compressed by
the old compressor.
-

 Initial consensus: all three problematic cases are very minor. We
 can just replace-upgrade decompressor and compressor.
 -

We already did that inadvertently with the different gzip
implementations (QAT, Linux, FreeBSD).
-

Uncompressed ARC in general should be considered an edge case.
Users should expect that it might not interact perfectly
with all other
features (such as L2ARC).
-

For L2ARC, we can add a small check that marks recompressed
buffers which are not bit-for-bit identical ineligible for L2ARC.
-

 LZ4 (Rich) and ZSTD are two concrete upgrades that are planned.
 -

 Special treatment of the LZ4 compressor upgrade?
 -

LZ4 is the default on many systems.
-

A replacement-upgrade of the LZ4 compressor would thus affect
many users of dedup.
-

So, should we require users to opt-in for LZ4 upgrades, e.g.,
via a feature flag? This would require keeping multiple
LZ4 compressor
versions around.
-

Decision: Wait for 1 month to see whether someone comes up with
a scheme that requires user opt-in. If there is no
solution, go ahead with
a replacement-upgrade.
-

 NB: For ZSTD, which changes more frequently, we might not want to
 maintain multiple compressor versions. So, whatever will be
decided for LZ4
 does not set a precedent for ZSTD.


On Mon, Jan 3, 2022 at 3:52 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, January 4, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-Md73cea97245493e922c7e523
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, February 1, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-M94db268a0ab16a99e5f0fa56
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: December OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video recording is now available:
https://www.youtube.com/watch?v=_097QvCwT4Y

Meeting notes (thanks Christian!):


   -

   ZFS Interface for Accelerators (“Z.I.A.”); Jason Lee (Los Alamos
   National Laboratory)
   -

  Slides
  <https://drive.google.com/file/d/1KTkR-ArSj7Pt2KTarNeWxqGsHNIQA4cf/view>
  -

  Summary: make use of hardware accelerators for compression, checksum,
  raidz
  -

  Main idea:
  -

 Data is owned by either the Host CPU or the accelerator.
 -

 ZIO pipeline refers to data owned by the accelerator through a
 “handle”.
 -

 If the accelerator doesn’t support one step of the pipeline, data
 is moved back to the host for that step. But as long as the
next step is
 supported, we can avoid copying back-and-forth.
 -

 Discussion about what safeguards should be put in place to keep
 track of ownership / source-of-truth of the data. E.g.,
during ABD copy.
 -

  Software architecture provider/consumer of accelerators:
  -

 “Data Processing Unit Services Module” decouples consumers from
 providers
 -

 Non-Restrictive License towards consumers; GPLv2 towards providers
 -

  QA / Discussion
  -

 Watch out for byte-for-byte compatibility, otherwise incompatible
 with “uncompressed ARC + L2ARC” use case.
 -

 Maintainability / Testing: at least a software provider should be
 part of the OpenZFS repo so that we can prevent accidental
breakage (e.g.
 when changing ZIO pipeline implementation)
 -

 Public availability of actual hardware & corresponding providers
 would be critical for upstreaming.
 -

 Discussion on resource contention (e.g. two zpools sharing one
 accelerator)
 -

   Native encryption needs some work (Rich Ercolani)
   -

  Spreadsheet of encryption bugs
  -

  Biggest and scariest: incorrect dnode refcounting
  -

 Panic stacks look very different, trapping at different ASSERTs
 -

 Not reproducible on x86(_64), but on SPARC and on PowerPC
 -

 Reproduces within 24 hours on a PowerPC 64 KVM VM (but not
 qemu-emulated)
 -

  Earliest occurrence dates all the way back to introduction of native
  encryption
  -

  Other scary category: encrypted send/recv corrupts receive-side
  dataset
  -

 One can be triggered by zfs change-key => will use wrong wrapping
 key
 -

  Problem: the community is not aware of a company that could fund
  fixing this.
  -

  Strategy: bring it up at future meetings to raise awareness.
  -

   Request for reviews:
   -

  Allan: calculate a more realistic allocation inflation multiplier:
  https://github.com/openzfs/zfs/pull/12773

Alek: healing recv patch - https://github.com/openzfs/zfs/pull/9372

On Mon, Dec 6, 2021 at 9:11 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, December 7, 9am
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T24bdaa2886c6cbf5-Mc039a11c3f1507ea0664817b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] December OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, December 7, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T24bdaa2886c6cbf5-M2048dff54b50591dbfaafa24
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit videos

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who presented at this year's OpenZFS Developer Summit!
The videos are now online, linked on the website
 and below:

TitleSpeakerCompanySlides/video
State of OpenZFS Matt Ahrens slides

video 
DirectIO for ZFS

Brian
Atkinson Los Alamos National Lab slides

video 
ZFS on Object Storage

George
Wilson, Matt Ahrens, Paul Dagnelie, Manoj Joseph Delphix slides

video 
ZFS performance on Windows

Imtiaz
Mohammad DataCore slides

video 
A New ZIL That Keeps Up With Persistent Memory Latency

Christian
Schwarz Independent slides

video 
ZettaCache: fast access to slow storage

Mark
Maybee, Serapheim Dimitropoulos Delphix slides

video 
Improving ZFS send/recv

Jitendra
Patidar Nutanix slides

video 
Adding Logical Quotas to ZFS

Sanjeev
Bagewadi Nutanix slides

video 
VDEV Properties

Allan
Jude, Mark Maybee Klara Systems, Delphix slides

video 

Hackathon presentations video 

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T53c709bae8967905-Mc234d05be3ff721713de30bc
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] 2 weeks to OpenZFS Developer Summit!

[Matthew Ahrens via openzfs-developer]

The ninth annual OpenZFS Developer Summit
 starts in just 2
weeks! The first day will feature talks about the future of OpenZFS, and on
the 2nd day we'll have a hackathon with a focus on quality. Register today

and join us online November 8th and 9th.  Registration is free!  The
conference will run 9am-5pm Pacific Time

.

Registered participants will be able to join the zoom call, participate in
Q, and join break rooms hosted by the speakers and other leaders of the
OpenZFS community.  The conference will also be streamed live on YouTube
 and recorded.

TitleSpeakerCompany
State of OpenZFS Matt Ahrens
DirectIO for ZFS

Brian
Atkinson Los Alamos National Lab
ZFS on Object Storage

George
Wilson, Matt Ahrens, Paul Dagnelie, Manoj Joseph Delphix
ZFS performance on Windows

Imtiaz
Mohammad DataCore
A New ZIL That Keeps Up With Persistent Memory Latency

Christian
Schwarz Independent
ZettaCache: fast access to slow storage

Mark
Maybee, Serapheim Dimitropoulos Delphix
Improving ZFS send/recv

Jitendra
Patidar Nutanix
Adding Logical Quotas to ZFS

Sanjeev
Bagewadi Nutanix
VDEV Properties

Allan
Jude, Mark Maybee Klara Systems, Delphix
--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T6569b9048c3119a4-Mf49a60e5875812b628c638c1
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: October OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  Don't forget to register
<https://www.eventbrite.com/e/openzfs-developer-summit-2021-tickets-166105683571>
for the OpenZFS Developer Summit
<https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021>, which will take
place November 8-9.  The November Leadership Meeting will be canceled, and
we'll meet again December 7 at 9am.

video recording: https://www.youtube.com/watch?v=VsBNknvrAas

Meeting notes (thanks Serapheim):

   -

   Block Reference Table update and a demo (Paweł Dawidek)
   -

  A lot of progress has been made and was demonstrated through the demo
  -

  There are still a few interesting problems to be solved specifically:
  -

 Dedup interaction - Freeing a block that is part of both the BRT
 and DDT
 -

 Making the BRT “sendable”
 -

 Overall memory consumption of feature under certain conditions
 -

   Linux User Namespace Support https://github.com/openzfs/zfs/pull/12263
   (Allan Jude)
   -

  All of the issues in the PR have been addressed
  -

  The feature should be landing soon
  -

   VDEV Properties https://github.com/openzfs/zfs/pull/11711 (Allan Jude,
   Mark Maybee)
   -

  Some of the feedback has been addressed but there is still some
  progress to be made
  -

   Encryption Incompatible On-Disk Format Change for illumos and OSX -
   (Jason King)
   -

  Jason came up with a solution that entails using one of the reserved
  bits on the object set flag field. The idea is to use that in order to
  migrate pools over time past this format change.
  -

   OpenZFS Hackathon Ideas
   -

  Allan brought up the idea of going through our bug backlog and
  picking bugs that would be nice for people to work on during the hackathon
  -

  List:
  
https://docs.google.com/spreadsheets/d/1byObqK0clYBpgJfzPUNWKMeHkiUAuFkRbTK71K8eW9g/edit?usp=sharing


On Mon, Oct 11, 2021 at 8:59 AM Matthew Ahrens  wrote:

> Reminder: Register today for this year's OpenZFS Developer Summit, which
> will be held online, November 8-9:
> https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021
>
> The next OpenZFS Leadership meeting will be held tomorrow, October 12, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td0c338d3ec8cf020-Md032def31f4e9fe621f013b7
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] October OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Reminder: Register today for this year's OpenZFS Developer Summit, which
will be held online, November 8-9:
https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021

The next OpenZFS Leadership meeting will be held tomorrow, October 12, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td0c338d3ec8cf020-Mea66d529453f6b5ce8304122
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2021 speakers & talks announcement

[Matthew Ahrens via openzfs-developer]

I'm pleased to announce the nine great talks that we'll have at the OpenZFS
Developer Summit ,
which will be held November 8-9 this year.

The conference will be live-streamed, and you can register

(for
free!) to participate in Q with the speakers, breakout discussions with
other attendees, and the hackathon!
TitleSpeakerCompany
State of OpenZFS Matt Ahrens
DirectIO for ZFS

Brian
Atkinson Los Alamos National Lab
ZFS on Object Storage

George
Wilson, Matt Ahrens, Paul Dagnelie, Manoj Joseph Delphix
ZFS performance on Windows

Imtiaz
Mohammad DataCore
A New ZIL That Keeps Up With Persistent Memory Latency

Christian
Schwarz Independent
ZettaCache: fast access to slow storage

Mark
Maybee, Serapheim Dimitropoulos Delphix
Improving ZFS send/recv

Jitendra
Patidar Nutanix
Adding Logical Quotas to ZFS

Sanjeev
Bagewadi Nutanix
VDEV Properties

Allan
Jude, Mark Maybee Klara Systems, Delphix

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T31316642a8c4b1cc-Ma0c2bfa3a3adbade5ec992c4
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: September OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

Video recording: https://www.youtube.com/watch?v=-WwwomHkAZg

meeting notes (thanks Serapheim):

   -

   OpenZFS Dev Summit: talks & schedule (Matt)
   -

  The talks and other Info can be found in the summit’s website:
  https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021
  -

  Like previous years we will be streaming the event live on Youtube
  and provide the recordings there for future reference.
  -

  There will be a hackathon on the second day as usual. This time it
  would be great if people from this meeting and the general
leadership could
  lead some of the hackathon efforts. One piece of feedback that
we got from
  previous years was that it would be nice to have people available to help
  newcomers get started.
  -

   Speed up zfs list (ready for review)
   -

  PR: https://github.com/openzfs/zfs/pull/11080
  -

  Status: Looking for reviewers
  -

  This work adds more fields to an existing optimization where we fetch
  specific fields from datasets without actually opening their objsets
  (object sets), which leads to a 300%+ performance improvement for those
  scenarios.
  -

  Future ideas/work: Would be for zfs destroy to take advantage of the
  same optimization and just print the datasets to be destroyed in no-op
  scenarios without actually calculating the space freed, which is
expensive.
  -

   ZFS allow for ‘encrypted’ send only, to only allow encrypted sends
   -

  Question for this feature was brought up in FreeBSD podcast last week
  -

  The idea is that the receiving system of such a send-receive stream
  won’t be able to access/read the plain text data of the encrypted dataset.
  -

  This level of security is not currently supported and probably
  involves some changes to be designed outside of the ZFS kernel module.
  -

  Could be a good hackathon topic for the summit!
  -

   ZTS may fail in weird ways when re-ran on the same disks
   -

  Igor has been experiencing ZTS issues in DilOS where the first time
  the ZTS runs successfully but subsequent runs experience
sporadic failures.
  His hypothesis so far is that the labels are not cleared for the
tests that
  run on disk-based pools (as opposed to file-based ones).
  -

  Historically, we’ve seen that all test cleanups work when tests pass,
  but sometimes they do experience issues when tests fail. Proper
  error-handling and cleanup for all possible scenarios during a
test failure
  can get tricky.
  -

  Igor will be filling an issue after he comes up with a minimal
  reproducer and/or a list of the troublesome tests so we can test whether
  the same issues come up in the OS’s officially supported by the OpenZFS
  repo.
  -

   Issue Triaging & Stalebot
   -

  There exist some concerns within the community about Issue Triaging
  and the use of the Stalebot to mark and close them.
  -

  Illumos has been occasionally hitting a panic described in
  https://www.illumos.org/issues/14003
  <https://www.illumos.org/issues/14003#note-1> , which apparently was
  also reported in OpenZFS under
  https://github.com/openzfs/zfs/issues/6881 once but was subsequently
  closed by Stalebot.
  -

  The underlying problem seems to be issue involvement and triaging
  from the community. There’s recently been efforts from the leadership to
  assign specific folks to help with PRs (ensuring that they get the
  attention of the right reviews and people to test them) but unfortunately
  issue triaging is something that people do on a volunteer-basis. In the
  meantime the stalebot mark old issues as Stale and if there is
no activity
  on them for 90 days, only then closes them.
  -

  The next steps on this would be to start a discussion on the issue
  and encourage hackathon ideas around it during the summit.


On Mon, Sep 13, 2021 at 6:21 PM Matthew Ahrens  wrote:

> FYI: The schedule of presentations for this year's OpenZFS Developer
> Summit have been posted to the webpage:
> https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021
>
> The next OpenZFS Leadership meeting will be held tomorrow, September 14,
> 9am Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs

[developer] September OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

FYI: The schedule of presentations for this year's OpenZFS Developer Summit
have been posted to the webpage:
https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021

The next OpenZFS Leadership meeting will be held tomorrow, September 14,
9am Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T6cf0d73c6ec6e0cd-Medca973e8cd4e284df3d470d
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: OpenZFS Developer Summit 2021 - announcement & CFP

[Matthew Ahrens via openzfs-developer]

Please share your experience developing or using ZFS with the community at
the OpenZFS Developer Summit.  Reminder that the deadline for talk
proposals is in just a few days!

--matt

On Mon, Aug 9, 2021 at 12:25 PM Matthew Ahrens  wrote:

> The ninth annual OpenZFS Developer Summit will be held online, *November
> 8-9, 2021*.  All OpenZFS developers are invited to participate, and
> (free!) registration is now open.
>
> See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2021 for all of
> the details, including slides and videos from previous years' conferences.
>
> The goal of the event is to foster cross-community discussions of OpenZFS work
> and to make progress on some of the projects we have proposed.
>
> The first day of the conference will be set aside for presentations. *If
> you would like to give a talk, submit a proposal via email
> to m...@mahrens.org , including a 1-2 paragraph abstract.* 
> Talks
> may be recorded prior to the event, and speakers' (virtual) attendance the
> day of the event will allow for everyone to experience the talks together,
> and for Q with attendees.
>
> We welcome proposals for any kinds of talks related to ZFS, and we are
> particularly interested in fostering a diverse group of speakers. In the
> past, the most popular talks have featured internal details of new or
> upcoming features in OpenZFS, and explanations of how existing subsystems
> work. Mentorship is available to help make your presentation the best it
> can be.
>
> The deadlines are as follows:
>
> *September 3, 2021 All abstracts/proposals submitted* to m...@mahrens.org
> September 10, 2021 Proposal submitters notified
> November 8-9, 2021 OpenZFS Developer Summit
>
> The second day of the event will be a hackathon, time set aside to work
> with OpenZFS developers, with the goal of having some progress to report
> at the end of the day.
>
> This event is only possible because of the efforts and contributions of
> the community of developers and companies that sponsor the event.  Although
> our costs are lower this year with an online conference, sponsorship
> opportunities are available. Please see the website for details and send
> email to victo...@vgfevents.com if you have any questions.
>
> --matt
>>
>>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T3edb89a14e0cd8a3-M9e216323e142f97ad13e4bd9
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: August OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated at this month's meeting.

The video recording is now available: https://youtu.be/D56PfP7KSCs

Reminder that the deadline for submitting a talk for this year's OpenZFS
Developer Summit <https://openzfs.org/wiki/OpenZFS_Developer_Summit_2021>
is September 3, which is just about a week away!

--matt

On Mon, Aug 16, 2021 at 4:11 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, August 17, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc8646411dbe3459a-M5656ea37358484398a18f9fa
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] August OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, August 17, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc8646411dbe3459a-M0a69275feff631a3e2ccf494
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] OpenZFS Developer Summit 2021 - announcement & CFP

[Matthew Ahrens via openzfs-developer]

The ninth annual OpenZFS Developer Summit will be held online, *November
8-9, 2021*.  All OpenZFS developers are invited to participate, and (free!)
registration is now open.

See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2021 for all of
the details, including slides and videos from previous years' conferences.

The goal of the event is to foster cross-community discussions of OpenZFS work
and to make progress on some of the projects we have proposed.

The first day of the conference will be set aside for presentations. *If
you would like to give a talk, submit a proposal via email
to m...@mahrens.org , including a 1-2 paragraph
abstract.* Talks
may be recorded prior to the event, and speakers' (virtual) attendance the
day of the event will allow for everyone to experience the talks together,
and for Q with attendees.

We welcome proposals for any kinds of talks related to ZFS, and we are
particularly interested in fostering a diverse group of speakers. In the
past, the most popular talks have featured internal details of new or
upcoming features in OpenZFS, and explanations of how existing subsystems
work. Mentorship is available to help make your presentation the best it
can be.

The deadlines are as follows:

*September 3, 2021 All abstracts/proposals submitted* to m...@mahrens.org
September 10, 2021 Proposal submitters notified
November 8-9, 2021 OpenZFS Developer Summit

The second day of the event will be a hackathon, time set aside to work
with OpenZFS developers, with the goal of having some progress to report at
the end of the day.

This event is only possible because of the efforts and contributions of the
community of developers and companies that sponsor the event.  Although our
costs are lower this year with an online conference, sponsorship
opportunities are available. Please see the website for details and send
email to victo...@vgfevents.com if you have any questions.

--matt
>
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T3edb89a14e0cd8a3-Maf2850404565d3b1c059bbb2
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: July OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated at this month's meeting, and especially
to Mark Maybee for running the meeting and to Serapheim for taking notes.

video recording: https://youtu.be/5Lby_Vl6H88

notes:


   -

   OpenZFS Dev Summit: Dates, planning, etc.
   - Registration for talk proposals opens on July 23rd and deadline is on
   August 30th
   - Conference Nov 8-9th
   - Remote or hybrid (in-person and remote) conference is being discussed.
   Would folks be interested in the hybrid option? Only Allan expressed
   interest. Most interest is in remote-only.
   -

   Run ztest with asan (Address Sanitizer malloc debugger) (Matt/Mark)
   There was some initial issue with ztest, issue 12215 that needs to be
   tracked down before we can enable ASan
   Related: MarkJ@FreeBSD also using KMSAN:
   https://github.com/openzfs/zfs/pull/12383
   -

   Status update on Panzura’s temporal dedup?

(matt:) looks like the last update was 4/28/2020, they have
“de-prioritized” this upstreaming work

No one on the call has update.

   -

   Allan Jude:
   - Linux Namespace support PR12263 is ready for review
   - About time to remove WIP status for vdev properties PR11711, new tests
   are being written.
   - Klara is working on zvol performance, if you have known pathologies,
   please share
   -

   Allan Jude brought up enabling LZ4 compression by default.


   -

   Ryan Moller suggests changing default for xattr to SA
   -

   Alexander Motin suggests changing default zvol block size from 8kb to
   16kb


   -

   Discussed possibly forcing allocations to be far apart on the disk (ala
   ditto blocks) to improve flash performance


   -

   Xattr compat (Ryan): https://github.com/openzfs/zfs/pull/11919
   Requesting design/code review. Tony to reach out to Brian.
   -

   Formalize a procedure for ‘erratas’, like the encryption+ project dnode
   issue, or even the zstd big-endian issue (like a bug@ not a feature@
   flag)

Due to zfs projects being ported after encryption, #7177 (include the
project dnode as part of the objset hash on encrypted datasets) was missed
(this appears to have happened with the macOS port as well). As a result,
there are now encrypted zfs datasets on illumos that are incompatible with
OpenZFS and vice versa.  While it is possible to re-try any objset hash
failures for compatibility between the two,  ‘fixing’ such a dataset
presents some thorny issues. One approach that was floated in the illumos
community (see
https://illumos.topicbox.com/groups/developer/T1981ce734a4a8c50-M4a3b3c0aff56de6ac0ceddfc)
was to propose an SPA feature that’s largely a dummy feature on non-illumos
platforms (i.e. is ignored), which would allow for explicit identification
of the two pools.


On Mon, Jul 19, 2021 at 1:41 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, July 20, 1pm
> Pacific time.
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ted9f300b994b7c2a-Ma3007de6669d1c1748c937bd
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] July OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, July 20, 1pm
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Ted9f300b994b7c2a-M5a4a3ab910c31bb8d444995f
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] Input on #12255? (Dedup apparently occupying nontrivial space for 100% deduplicated data)

[Matthew Ahrens via openzfs-developer]

The reason you can't find the "missing" blocks in zdb is that there aren't
any.  This is consistent with your "zpool list" output which shows that no
more space is allocated as you create more deduped-away "copies" (ALLOC is
constant).

The only thing that changes is the DMU's view of how much free space there
is.  What's happening is that the SPA tells the DMU that the pool is bigger
than it really is due to dedup (USED + AVAIL > SIZE).  The SPA also tells
the DMU how much "slop space" is unusable.  The strange thing is that the
"slop space" is based on the dedup-inflated fictional size of the pool.  So
as you have more dedup copies, the slop space increases.

This doesn't seem necessary to me.  The fix would be for
spa_get_slop_space() to not include ddt_get_dedup_dspace() in its overall
size (via spa_get_dspace()).  Instead it should just take into account
metaslab_class_get_dspace(spa_normal_class(spa)) (and probably the vdev
removal stuff from spa_get_dspace()).

--matt

On Tue, Jun 22, 2021 at 4:46 AM Rich  wrote:

> Hi everyone,
> i've spent a bit of time poking around in #12255 - the short version
> is "user reporting that, on a 120G pool with dedup on, writing
> additional copies of a 100G file where part of the first 128k is
> different occupies an additional ~3.2G each, until ENOSPC".
>
> Curiously, on my reproducing setup (git master, 120G file on an SSD
> with single file special and dedup vdevs to try and identify where the
> space goes, ashift 9, compression=lz4 even though the 100G is from
> /dev/urandom, recordsize=1M though that didn't noticably change
> matters), zfs list -o space at only one copy reports:
> NAME   AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
> dedup2 15.3G   100G0B 24K 0B   100G
> dedup2/copyme  15.3G   100G0B100G 0B 0B
> and zpool list -v reports:
> NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAGCAP
>  DEDUPHEALTH  ALTROOT
> dedup2 158G  99.9G  58.1G- - 0%63%
>  1.00xONLINE  -
>   /deduppool2  119G  99.9G  19.1G- - 0%  83.9%
>  -ONLINE
> dedup -  -  -- -  -  -
>   -  -
>   /dedup2dedup19.5G  31.3M  19.5G- - 0%  0.15%
>  -ONLINE
> special   -  -  -- -  -  -
>   -  -
>   /dedup2special  19.5G  11.9M  19.5G- - 0%  0.05%
>  -ONLINE
>
> while at n=2 (I used entirely identical files)...
> NAME   AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
> dedup2 12.1G   200G0B 24K 0B   200G
> dedup2/copyme  12.1G   200G0B200G 0B 0B
> NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAGCAP
>  DEDUPHEALTH  ALTROOT
> dedup2 158G   100G  57.9G- - 0%63%
>  2.00xONLINE  -
>   /deduppool2  119G   100G19G- - 0%  84.0%
>  -ONLINE
> dedup -  -  -- -  -  -
>   -  -
>   /dedup2dedup19.5G  43.4M  19.5G- - 0%  0.21%
>  -ONLINE
> special   -  -  -- -  -  -
>   -  -
>   /dedup2special  19.5G  22.8M  19.5G- - 0%  0.11%
>  -ONLINE
>
> ...and so on until returns ENOSPC at n=6 while zpool list still
> reports 19G free:
> NAME   AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
> dedup20B   587G0B 24K 0B   587G
> dedup2/copyme 0B   587G0B587G 0B 0B
> NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAGCAP
>  DEDUPHEALTH  ALTROOT
> dedup2 158G   100G  57.9G- - 0%63%
>  5.87xONLINE  -
>   /deduppool2  119G   100G19G- - 0%  84.0%
>  -ONLINE
> dedup -  -  -- -  -  -
>   -  -
>   /dedup2dedup19.5G  40.9M  19.5G- - 0%  0.20%
>  -ONLINE
> special   -  -  -- -  -  -
>   -  -
>   /dedup2special  19.5G  52.0M  19.4G- - 0%  0.26%
>  -ONLINE
>
> My hypothesis was metadata, even though 3% overhead seemed like a lot,
> but examining zdb - diff between the n=1 and n=2 cases closely, I
> couldn't see any obvious culprit. (and 5 or 6 ds proved too much for
> any non-diff utility to handle, while hundreds of MB of diff output
> was not particularly readable even colorized).
>
> zdb -bb also was not informative (I have copies, if anyone wants, but
> suffice it to say they just report 99.9G/200G/.../587G in total, with
> nothing besides "ZFS plain file" reporting over 100M usage).
>
> Deleting the files frees the extra space, so it's 

[developer] June OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, June 22, 9am
Pacific time.
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td25434f74c9df977-M860b64107155fecaac8237fc
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] talk tomorrow on RAIDZ Expansion

[Matthew Ahrens via openzfs-developer]

FYI, I’ll be giving a talk about RAIDZ Expansion at the FreeBSD Developer
Summit tomorrow (9am pacific).  Interested folks can watch on youtube live:
https://wiki.freebsd.org/DevSummit/202106

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T837e0a1c9d7e4b23-M128dd3fd9617403151fc760c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: May OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video recording is now available:
https://youtu.be/GKusXS8T_KY

meeting notes:

   -

   New maintainers (Matt)
   -

  Brian has been the sole maintainer for ZoL -> OpenZFS for almost a
  decade now.
  -

  The volume of contribution and users has been increasing and we need
  more people
  -

  Matt and Tony Hutter will be joining him as maintainers
  -

  Tony Nguyen, John Kennedy, and Mark Maybee will also be helping out
  by ushering incoming PRs through the review process (e.g. finding
  reviewers, applying the appropriate tags, and merging them).
  -

   Settings vdev properties at pool create time (Allan)
   -

  This is not a pressing matter for getting the vdev properties PR
  merged, but if anyone comes up with a good idea on how the
argument parsing
  should be done for this, they should definitely send an email to Allan or
  open a discussion in the mailing list.
  -

   Getting zpool status details via pool properties (scrub progress,
etc) (related
   issue <https://github.com/openzfs/zfs/issues/2626>) (Allan)
   -

  Some of these properties you can get with the vdev properties PR as
  it is right now but certain pool properties like a scan estimate or other
  things that admins see from commands like zpool-status are harder to get.
  -

  It would be useful to expose some of these properties through
  zpool-get as-is or at least parts of them.
  -

   If we add a bunch more pool and dataset properties, do we need a new
   ‘verbose’ type or something, where a lot of these new (not for human
   consumption) properties are not displayed by default?
   -

  zpool/zfs get all has been pretty verbose in general as more
  properties are piling up on it and its output is not very
friendly in terms
  of parsing it programmatically
  -

  Allan and Matt are considering a potential alternative command option
  that’s more human readable and outputs some important properties (not all
  of them).
  -

   Userland zfs (PR <https://github.com/openzfs/zfs/pull/11766>) (Mayank)
   -

  The above PR needs reviewers
  -

  There is a lot more work to be scoped and done. Ideally the
  upstreaming work from the cStor folks would involve incremental
PRs adding
  more IOCTLs or other minimal functionality that can be tested
independently
  in the userland and slowly become part of the existing testing
  -

 Allan brought up an old idea of the userland send/receive tests
 -

 Other options include but are not limited to Matt’s old
 socket-based userland ZFS testing.
 -

  Keeping in mind FUSE and laying the groundwork for it during this
  effort should also provide some long-term benefits and flexibility as we
  discover more ways on how to bring the kernel code to the userland.
  -

  There is a lot of ground to be covered but given the excitement there
  should be volunteers to help. If you are interested please reach out to
  Mayank, Matt, or Serapheim.
  -

   Corrective Recv PR <https://github.com/openzfs/zfs/pull/9372> (Alek)
   -

  The above PR seems to be passing our tests and needs
  reviewers/reviews.
  -

   FreeBSD Summit
   -

  Matt will be giving a talk about RAIDZ expansion
  -

  There will be other storage topics covered outside of ZFS for folks
  that are interested
  -

   ZFS support for LXD and similar container technologies (Allan)
   -

  A PR should be out soon (probably next week)
  -

  There will be a need for both reviewers and testers
  -

   ZFS Forced Export (Allan)
   -

  Tests are passing and a PR should be open soon
  -

   OSX port (Jorgen)
   -

  Around 71% of the tests currently pass and we’d like to start opening
  PRs soon
  -

  Besides the above, one thing that is pending in terms of
  infrastructure is figuring out which solution to use for incorporating
  testing for the new OS in our CI/CD. Github Action seems to provide some
  MacOS instances but there may be a timeout that’s not enough for
us to run
  ZTS. AWS is another option.
  -

   Add Blake3 Checksum (PR <https://github.com/openzfs/zfs/pull/11897>)
   (Tino)
   -

  Sorry, did not have time for this currently (will do it in the next 4
  weeks)
  -

  I got an account on the gcc farm project, so assembler routines for
  sha2 will also come from me (aarch64, ppc, x64) … together with
blake3 and
  benchmarks
  -

  (Moving as an agenda item for next time)


On Mon, May 24, 2021 at 10:22 PM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, May 25, 1pm
> Pacific time.  Topics on the agenda include:
>
>- Settings vdev properties at pool create time

[developer] May OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, May 25, 1pm
Pacific time.  Topics on the agenda include:

   - Settings vdev properties at pool create time
   - Getting zpool status details via pool properties (scrub progress, etc)
   (related issue )
   - If we add a bunch more pool and dataset properties, do we need a new
   ‘verbose’ type or something, where a lot of these new (not for human
   consumption) properties are not displayed by default?
   - Userland zfs (PR ) (Mayank)

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tbc2723be3e4d22d4-Mcce4897fe77bcb20bc4baebc
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] New OpenZFS maintainers/committers

[Matthew Ahrens via openzfs-developer]

Throughout OpenZFS/zfsonlinux's life, Brian Behlendorf has been nearly the
sole manager of the day-to-day operations of the project.  He's triaged
bugs, reviewed PR's, found reviewers for PR's, given feedback to PR
submitters, and decided when PR's are ready to be merged.  And that's in
addition to his work developing new ZFS features and fixing bugs!

As the project grows and the volume of PR's has increased, we need to
spread the workload among more volunteers.  To that end, I'm happy to
announce three new OpenZFS maintainers: Mark Maybee, Tony Nguyen, and John
Kennedy.  They join existing maintainers Brian Behlendorf, Tony Hutter, and
myself.  As maintainers, we are responsible for giving feedback to PR
submitters, deciding when a PR is ready to be merged, and merging it.

Note that maintainers' work does not necessarily include reviewing the code
of every PR in detail.  We continue to rely on broad community
participation by many experts, to thoroughly review the code of each PR.
The maintainers' role is to help PR submitters find appropriate code
reviewers, so you may find yourself being reminded about code review
requests by these new maintainers :-)

We have divided the codebase into several broad areas, with each maintainer
assigned some areas of responsibility.  Each PR will have a designated
maintainer (using the "Assignees" feature of GitHub).  This is the PR
submitter's point of contact with the project for getting their PR merged.

Brian Behlendorf

   -

   Other & catch-all

Matt Ahrens

   -

   Advisory
   -

   Assigning PR’s to maintainers

Tony Hutter

   -

   Backports
   -

   ZED

Tony Nguyen

   -

   Platform-specific, shared, and imported kernel code  (module/os/*,
   zcommon, ICP, LUA, zstd, etc)
   -

   Manpages & other documentation
   -

   Other userland scripts/commands (zed, arcstat, contrib/*, etc)

Mark Maybee

   -

   Core platform-agnostic kernel code (module/zfs/*)

John Kennedy

   -

   ZTS
   -

   Libzfs & consumers (zpool/zfs commands, etc)
   -

   Libzpool & consumers (ztest, zdb, libspl)
   - Developer tools (makefiles, packaging, github workflows/templates)


About the new maintainers, who coincidentally all live in Colorado:

Mark Maybee has been a developer on ZFS/OpenZFS since its infancy 20 years
ago, working on ZFS at Sun, Oracle, Cray, HPE, and Delphix.  In the early
days he implemented the ZPL.  More recently he has worked on DRAID.

Tony Nguyen has worked on ZFS primarily as a performance engineer, at Sun,
Nexenta, and Delphix.  He has worked on the ZFS Performance Tests.

John Kennedy has worked on ZFS primarily as a test/QA engineer, at Sun and
Delphix.  He implemented the ZFS Test Suite.

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T5d274eb056cc6f4d-Mccd497515da5645f8159593d
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated, and apologies for the delay in getting
this posted.

video recording: https://youtu.be/6zb6AxL26d8

notes (thanks Serapheim):

   -

   Corrective Recv PR <https://github.com/openzfs/zfs/pull/9372> (Alek)
   -

  The functionality introduced here is for being able to “heal” a
  corrupted dataset from a send file (instead of deleting and
recreating it).
  An interesting details is that this feature works regardless of the
  differences in the configuration of the target machine and the
machine that
  generated the test file (e.g. compression or other properties).
  -

  Current status is that most of the tests are passing except a couple
  from the redacted send-receive tests in FreeBSD which cause a panic.
  -

 John Kennedy and Paul Dagnelie can help with this
 -

  Besides fixing the panic - the PR needs reviews.
  -

   Add Blake3 Checksum (PR <https://github.com/openzfs/zfs/pull/11897>)
   (Tino)
   -

  Currently experiencing problems with supporting this on FreeBSD. The
  complications seem specific to the code organization and getting
the right
  directives in autoconf/automake and make to work.
  -

  Besides just adding this new checksump algorithm, the community would
  like to see how this algorithm compares to existing ones, running on a
  sufficiently modern CPU under different blocksizes.
  -

   Should I add sha_ni (x86) and maybe also aarch64 sha2 hardware support?
   -

  There is probably some low-hanging fruit for performance
  improvements, especially for our choice of CPU instructions.
  -

  Some support for this in FreeBSD has started being worked on:
  https://github.com/freebsd/freebsd-src/compare/main...allanjude:zfssha
  -

   Interface for /proc/spl/kstat/zfs/hashing_bench ? echo blake3-sse2 >
   hashing ..
   -

   ZFS and Linux Containers (LXD) (Allan Jude)
   -

  The idea here is jailing a dataset (group of datasets) to a user
  namespace in LXD.
  -

 In terms of code, this would mean at least implementing the
 IS_GLOBAL_ZONE() macro that currently in Linux always returns true.
 -

  Do people have interesting use/test cases?
  -

 If you do, please do reach out to Allan.
 -

  What makes sense for extending the ZTS to cover containers?
  -

 Besides the actual ZTS-specific additions, we’ll need to look at
 what we can reasonably add to our automated testing bots in
Github to run
 those tests.
 -

   Cross-platform compatible xattr support (Alexander Motin - amotin)
   -

  Current status: There is progress but we’d love it if more people are
  able to look at the code and/or bring alternative solutions/ideas to the
  discussion.
  -

   ZFS on Object Storage
   -

  Currently an experiment within Delphix
  -

  The idea is to run ZFS on top of an object store like S3 or MinIO.
  -

  The focus is on performance for small record sizes.
  -

  There are currently two brand-new components being introduced:
  -

 A userland agent that facilitates the interaction between ZFS and
 the object store
 -

 A new block-based local cache introduced as a vdev type to improve
 the performance of the overall design of this project.
 -

The L2ARC was briefly considered but it doesn’t satisfy the
requirements for the current use-case as its memory
requirements are high.
-

  More technical details on the upcoming OpenZFS Summit.
  -

  Until then, while the technical requirements are still being designed
  we’d love any feedback on the administrative/user interface of
this project.
  -

 Currently we just provide a couple of user properties just to test
 the design (e.g. a property for the s3 URL, credentials file, etc..).
 Eventually if we are to start supporting other protocols
we’ll need to have
 a user interface and workflow that works for all of them.


On Mon, Apr 26, 2021 at 11:49 AM Matthew Ahrens  wrote:

>
> The next OpenZFS Leadership meeting will be held tomorrow, April 27, 1pm
> Pacific time.  Topics on the agenda include:
>
>> Userland zfs (PR <https://github.com/openzfs/zfs/pull/11766>)
>>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--

[developer] April OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, April 27, 1pm
Pacific time.  Topics on the agenda include:

> Userland zfs (PR )
>
Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-Mdbf648771178c7353820251b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: (second) March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.  The video is now posted:
https://www.youtube.com/watch?v=ZXkSVkq0bgM
I used a different format for the video, "gallery view" which shows
everyone's video.  This seems a little less jarring than having the video
cut back and forth between speakers, and it's a little more interesting
than a blank screen when someone who isn't sharing video is speaking.  Let
me know if you have any feedback.

meeting notes (thanks Serapheim!):

   -

   Version numbering scheme (Bryan)
   -

  Major.Minor.Patch
  -

 Major - support for new platforms and new feature that target very
 common use-cases
 -

 Minor - new features, properties, feature flags (e.g draid),
 refactors
 -

 Patch - support for new kernels, bug fixes
 -

  Periodical Releases and LTS
  -

 We’ll be creating a new release branch about every 6 months,
 marking the occasional LTS. The LTS releases will contain only very
 critical bug fixes (e.g. security or data corruptions -
crashes for rarely
 used codepaths are not part of this).
 -

   OpenZFS Developer Summit (Matt)
   -

  Will be held online November 3rd-4th Wed-Thu
  -

  Submit Proposals - Deadline: End of August
  -

 What are you working on now or the next few months?
 -

 Ideas: New features, performance investigations/analyses,
 interesting use-cases/applications/workloads.
 -

  Will be sending announcements to the mailing list and other channels
  soon
  -

  COVID & Travelling: Depending on how things are by that time we may
  reconsider allowing folks to gather in person somewhere. That
said we plan
  to remain “remote-first” regardless of the situation (e.g. if you want to
  participate/present but don’t feel safe travelling you can still
  participate/present remotely)
  -

   Patches to fix bugs and upstreaming from DilOS (Igor K)
   -

  Igor found some bugs in the dRAID and L2ARC features, for some of
  which he has fixes for. He’ll be working with the upstream community to
  create issues for them.
  -

  He also wants to make integration with DilOS easier. The proper plan
  for this would be like adding support for any other platform (similar to
  the MacOS effort taking place currently).
  -

 This involves things like refactoring code and re-organizing files
 and folders so we end up with the least amount of ifdefs,
more common code
 among platforms, and any platform-specific code that we
cannot avoid to be
 refactored into its own files.
 -

   RAIDz Expansion (Matt)
   -

  Currently cleaning up the code and polishing the design doc
  -

  A PR should be out soon
  -

   Proposal: Optionally calculate the space deleted in zfs destroy -v
   (Allan)
   -

  Motivation: For pools with a lot of snapshots the overhead of the
  space calculation that happens by default incurs a lot of overhead.
  -

  The idea is not to change the default behavior but at least provide
  an option to get the list of the snapshots to be deleted without
the space
  calculation
  -

   Performance Improvements for zfs list (Allan)
   -

  PR: https://github.com/openzfs/zfs/pull/11080
  -

  The idea here is to improve the performance of zfs list by performing
  the least computation that is required to gather the specific
  fields/properties specified by the -o flag.

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tb368e00631ce09fa-M263730dc8568937496577f22
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] (second) March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, March 30,
9am-10am Pacific time.  Topics on the agenda include:

   -

   Version numbering scheme
   -

   OpenZFS Developer Summit

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tb368e00631ce09fa-M229b39e73aa702e877556b8f
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: (first) March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated.

video: https://www.youtube.com/watch?v=z5JoRoBbUmA

Notes:

   -

   Vdev properties
   
<https://openzfs.topicbox.com/groups/developer/Tb11994f96a39413b/vdev-properties>
   (Allan)
   -

  Allan will be posting a PR about this soon to demonstrate the state
  that the code is currently in. In terms of functionality that PR is
  expected to work only with read-only properties for the time-being.
  -

  The UI/Syntax component of this change is currently modelled after
  other existing zpool commands (e.g. zpool list, zpool iostat, etc..).
  -

  Mark Maybee will be working on the vdev_noalloc feature and could
  potentially leverage the vdev properties infrastructure for his change.
  Depending on the state of the PR he may help Allan push it through the
  finish line.
  -

   More meaningful error messages for admins using zpool import
   -

  The idea here was to propagate the debug statements produced during
  pool import to the userland in order to replace generic import
errors that
  are not particularly helpful. Could be a good hackathon project.
  -

   Cleanup and refactoring of platform independent ZVOL code (Christian)
   -

  Relevant PR: FreeBSD struct bio support for zfs_uio_t (needed by
  https://github.com/openzfs/zfs/pull/11657#issuecomment-788860549 )
  -

  Currently having problems dealing with GEOM-related code - Sean Fagan
  volunteered to help over Slack.
  -

   Pool user properties: https://github.com/openzfs/zfs/pull/11680
   -

   Next Meeting Preview:
   -

  WIll take place earlier (9am Pacific in 4 weeks)
  -

  Brian may have some information/updates in the version numbering
  scheme for OpenZFS
  -

  Matt may have information/updates for this year’s OpenZFS summit.


On Mon, Mar 1, 2021 at 8:59 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, March 2,
> 1pm-2pm Pacific time.  We don't have many topics on the agenda for
> tomorrow's meeting, so let me know if you'd like to add anything.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te98adfb02c2a480d-Mb4f058ef4e4114fa55161ae9
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] (first) March OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, March 2, 1pm-2pm
Pacific time.  We don't have many topics on the agenda for tomorrow's
meeting, so let me know if you'd like to add anything.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te98adfb02c2a480d-M787906512743431f7e027b30
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated last week.

The video is posted: https://youtu.be/VmXoHO2PDP4

meeting notes:

   -

   Seagate command priority call Feb 3
   -

  Mentioned in the last OpenZFS meeting
  -

  Get in contact with Muhammad if interested
  -

  WIll take place 2/3/2021 11:30 PST
  -

   OpenZFS Versioning Scheme and what it means
   -

  Motivation: Folks were interested in learning when would dRAID be
  included in an official release
  -

  Brian will put something together for the next meeting to be discussed
  -

   Status of specific PRs:
   -

  DirectIO
  -

 Brian Atkinson is still working on it. We expect it to be updated
 soon, at which point we’ll need reviewers
 -

  Pool Feature Compatibility Options during pool creation
  -

 Still ongoing, needs reviewers
 -

  Force Pool Export
  -

 Should be good to go soon. We’d like to get more reviewers as the
 PR touches many different parts of the codebase.


On Mon, Feb 1, 2021 at 9:48 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, February 2,
> 1pm-2pm Pacific time.  We don't have many topics on the agenda for
> tomorrow's meeting, so let me know if you'd like to add anything.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-Mbb1cecd282a40cb19b2dd843
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] February OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

The next OpenZFS Leadership meeting will be held tomorrow, February 2,
1pm-2pm Pacific time.  We don't have many topics on the agenda for
tomorrow's meeting, so let me know if you'd like to add anything.

Everyone is welcome to attend and participate, and we will try to keep the
meeting on agenda and on time.  The meetings will be held online via Zoom,
and recorded and posted to the website and YouTube after the meeting.

For more information and details on how to attend, as well as notes and
video from the previous meeting, please see the agenda document:

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit

--matt

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T7ea98f99753036e8-M2179abc5b638e4843bbcf228
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

Thanks to everyone who participated in the discussion this week.

video recording: https://www.youtube.com/watch?v=Zhx17DyCWFo

meeting notes (thanks Serapheim):

   -

   What is on the current roadmap in terms of OpenZFS releases?
   -

  2.0.1 and subsequent 2.0.X releases are coming out soon that better
  support the latest Linux kernel versions.
  -

  2.1 is expected to be released in this quarter
  -

  Main projects that are inflight that we hope to release in 2.1:
  -

 DRAID, Forced Pool Export, DirectIO
 -

  Further quarterly releases are expected to mostly have bug fixes
  -

  We expect OpenZFS 3.0 within the year~18 months (first half of 2022)


   -

   RAIDZ Expansion
   -

  Work is still underway, expect to see progress on that soon
  -

  Currently Matt is breaking out some ABD related changes from his
  branch and a PR with them is expected to be out later this week. These
  changes mostly make it possible for the abd_t to be allocated by the
  caller. The BSD-specific parts of this work are a bit more
tricky but still
  doable.


   -

   Marking vdevs as non-allocatable
   -

  Mark Maybee is expected be working on this feature
  -

  Goal: Making the removal of multiple devices easier
  -

 Ensure that all devices can be removed (e.g. there is enough space
 in the pool for the data after their removal)
 -

 Don’t spread data of device being removed to devices that will be
 removed next, mark them as non-allocatable first as you queue up their
 removal.
 -

  Current design just passivates the vdev’s metaslab group
  -

  Christian also threw the idea of removing the vdev from the metaslab
  class altogether, instead of just passivating the group
  -

  Allan Jude has some code for the vdev-properties he could try and
  upstream soon
  -

 No-alloc could be a part of this, and save Mark from having to
 write an interface to the userland for it


   -

   Potential Talks with Seagate SCSI/Firmware Folks about priority policies
   -

  Seagate Firmware Folks want to get a better understanding of how ZFS
  prioritizes operations, and coordinate with the community on
their designs
  of how operations can be prioritized best.
  -

  From our community’s perspective, getting a high-level description of
  what the driver can do, how we can take advantage of that, and if the
  benefit is enough for us to implement any logic, are the main questions.
  -

  Alexander Motin’s work/discussion of separating priorities for
  different requests (ZFS background work, user-triggered requests,
  synchronous vs asynchronous, etc..) is also relevant on the matter.
  -

  For next steps on this:
  -

 It would be nice to have some information from the firmware folks
 before setting a meeting, so we can have a better
understanding of what
 this entails and gather up the right folks that can discuss
the topic in
 more detail.
 -

 For setting up the actual meeting, we can coordinate on Slack,
 have an initial discussion there, and then send out an email
when everyone
 is prepared to talk about this.


On Mon, Jan 4, 2021 at 9:10 AM Matthew Ahrens  wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, December 8,
> 1pm-2pm Pacific time.  We don't have many topics on the agenda for
> tomorrow's meeting, so let me know if you'd like to add anything.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-M837088846c28dd7ed6702ba0
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Re: [developer] January OpenZFS Leadership Meeting

[Matthew Ahrens via openzfs-developer]

I'll try to remember to do that!

--matt

On Mon, Jan 4, 2021 at 12:40 PM Igor Kozhukhov  wrote:

> Hi Matt,
>
> thanks for update.
> is it possible paste to slack info when meeting will be started with zoom
> URL ?
> i'd like to join tomorrow.
>
> best regards,
> -Igor
>
> On 4 Jan 2021, at 23:33, Matthew Ahrens via openzfs-developer <
> developer@lists.open-zfs.org> wrote:
>
> Correction: the meeting will be tomorrow, January 5th, 9am-10am Pacific
> time.
>
> --matt
>
> On Mon, Jan 4, 2021 at 9:10 AM Matthew Ahrens  wrote:
>
>> The next OpenZFS Leadership meeting will be held tomorrow, December 8,
>> 1pm-2pm Pacific time.  We don't have many topics on the agenda for
>> tomorrow's meeting, so let me know if you'd like to add anything.
>>
>> Everyone is welcome to attend and participate, and we will try to keep
>> the meeting on agenda and on time.  The meetings will be held online via
>> Zoom, and recorded and posted to the website and YouTube after the meeting.
>>
>> For more information and details on how to attend, as well as notes and
>> video from the previous meeting, please see the agenda document:
>>
>>
>> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>>
>> --matt
>>
>
> *openzfs <https://openzfs.topicbox.com/latest>* / openzfs-developer / see
> discussions <https://openzfs.topicbox.com/groups/developer> + participants
> <https://openzfs.topicbox.com/groups/developer/members> + delivery options
> <https://openzfs.topicbox.com/groups/developer/subscription> Permalink
> <https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-Mc3c58919c4555d366f259f3d>
>

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Td6ebc6df2e2a14bd-M1a7434212a38d07509306b04
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription