Thanks for starting the thread! I've added a comment on the Github issue. Super exciting to see the iceberg-rust integration with Comet. As mentioned, I'll help take a look at https://github.com/apache/iceberg/pull/13786 and see if we can bring it to the finish line.
Best, Kevin Liu On Thu, Dec 18, 2025 at 6:30 PM Renjie Liu <[email protected]> wrote: > Thanks Andy for raising this. > > +1 for the iceberg-rust solution. This could benefit the whole oss > community in the long term and avoid duplicated efforts. As with the > technique challenges you mentioned, this seems more like a missing feature > rather than limitation. > > On Thu, Dec 18, 2025 at 5:58 AM huaxin gao <[email protected]> wrote: > >> Thanks Andy for starting this discussion! >> >> I agree we should prioritize the iceberg-rust path as the long term >> default for Comet/Iceberg scans, while it's still valuable to keep the >> Iceberg Java integration as an option, especially for users who need >> JVM-specific features. I'm hoping we can get consensus on a path to resolve >> the shading issues (e.g. the approach proposed in >> github.com/apache/iceberg/pull/13786) so the java option can move >> forward. >> >> Thanks, >> Huaxin >> >> On Wed, Dec 17, 2025 at 12:23 PM Shawn Chang <[email protected]> >> wrote: >> >>> Thanks Andy for starting this thread! >>> >>> I've replied to the issue above but am posting here as well for better >>> visibility: >>> >>> I’m leaning toward prioritizing the iceberg-rust path to avoid the >>> shading and circular dependency headaches of the Java approach. Easier >>> maintenance typically lowers the entry barrier and encourages more >>> community involvement, which is important for the project’s long-term >>> health. >>> >>> It also makes sense to keep the Iceberg Java integration available as an >>> experimental option for users who need its richer JVM-specific features >>> while Rust becomes the long-term default. We don’t want to block >>> contributions from anyone who truly needs the Java path. >>> >>> Best, >>> Shawn >>> >>> On Wed, Dec 17, 2025 at 11:00 AM Andy Grove <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> There is a discussion in the DataFusion Comet community about the >>>> future direction of our Iceberg support [1]. For example, should we >>>> continue the efforts to integrate with the Iceberg Java project or focus >>>> more on the iceberg-rust project. >>>> >>>> It would be great to also get input from the Iceberg community. >>>> >>>> Thanks, >>>> >>>> Andy. >>>> >>>> [1] https://github.com/apache/datafusion-comet/issues/2921 >>>> >>>
