Hi Anton, Comet support for Spark 4.1 has just started[1]. I'm not sure about the timeline but I'm devoted to adding 4.1 support in Comet as early as possible.
For the current, I'm using `org.apache.datafusion:comet-spark-spark4.0_2.13` as we used `org.apache.datafusion:comet-spark-spark3.5_2.13` for 4.0 before it's supported in Comet. 1.https://github.com/apache/datafusion-comet/pull/2829 Happy Thanksgiving! Manu On Thu, Nov 27, 2025 at 2:27 PM Anton Okolnychyi <[email protected]> wrote: > Thanks for driving this, Manu! > > What shall we do with Comet support? Is it broken for 4.1? If so, what's > the timeline on the Comet side to add support for 4.1? > > - Anton > > вт, 25 лист. 2025 р. о 18:17 Manu Zhang <[email protected]> пише: > >> Hi team, >> >> We've been preparing for initial Spark 4.1 support[1] for a while, >> testing, fixing bugs and compatibility issues. Meanwhile, the Spark >> community is working towards 4.1.0 (RC1) release[2]. While there's a few >> remaining issues to be resolved, Anton([email protected]) suggested >> getting it into the main branch first such that it will be easier for the >> community to collaborate on Spark 4.1 support. >> >> If there's no concerns or objections, we are going to merge it. >> >> 1. https://github.com/apache/iceberg/pull/14155 >> <https://github.com/apache/iceberg/pull/14155> >> 2. https://lists.apache.org/thread/vcdz4fpmojbclogw1zntkotcnntjtjrt >> >> Happy Thanksgiving! >> Manu >> >
