I'd loveeee to have a more technical explanation in the form of a project-level blog :) Inspired by Apache Datafusion [1] Amogh did a great presentation on the current state of "V4 Adaptive Metadata Tree" [2]. It's a good resource for some of the current challenges.
Best, Kevin Liu [1] https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes/ [2] https://www.youtube.com/watch?v=X3SQHWwE1fA On Mon, Dec 29, 2025 at 11:38 AM Russell Spitzer <[email protected]> wrote: > I do want to make a higher level video about scan planning some day ... > although I may wait till we finish V4 so I can do it in that context > > On Mon, Dec 29, 2025 at 1:22 PM Kevin Liu <[email protected]> wrote: > >> The video Russell posted is a great resource. >> I don't know of any docs or threads specifically explaining the internals >> behind scan planning. There might be some in the wild from community >> contribution. >> >> Most of the design is in the code itself. For example, this is the >> pyiceberg code for scan planning [1] >> >> Best, >> Kevin Liu >> >> >> [1] >> https://github.com/apache/iceberg-python/blob/59cdf33d75cc726c92ca4fa0c7b0ad1e16f8f30a/pyiceberg/table/__init__.py#L1973 >> >> >> On Sat, Dec 27, 2025 at 10:12 AM Vaibhav Kumar <[email protected]> >> wrote: >> >>> This was a great video. I have been through this already last month. I >>> would really appreciate if you have some docs/thread handy for the same. >>> >>> On Sat, 27 Dec 2025 at 7:19 PM, Russell Spitzer < >>> [email protected]> wrote: >>> >>>> https://youtu.be/kJaD0WuQ1Bg?si=woefIj2FJlFVMtmV >>>> >>>> I did a pretty long overview of it earlier this year , this may be more >>>> in depth than you are looking for >>>> >>>> On Sat, Dec 27, 2025 at 6:35 AM Vaibhav Kumar <[email protected]> >>>> wrote: >>>> >>>>> Hi Team >>>>> >>>>> I was trying to understand the internals behind scan planning. Can >>>>> someone please help me with relevant design doc/thread or any >>>>> community sync video where we would have it in detail? It would help >>>>> me to better understand iceberg as a project. >>>>> >>>>> Regards, >>>>> Vaibhav >>>>> >>>>
