Hi I did a new pass, thanks !
Regards JB On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <[email protected]> wrote: > Here's the latest version of the Flink Quickstart PR, > https://github.com/apache/iceberg/pull/15062 > Please take a look! > > Let's start a new thread to follow up on publishing the Docker image (for > Flink and Spark) > > > > On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <[email protected]> wrote: > >> Thanks Robin! Really great work. >> >> Cheers, >> Max >> >> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <[email protected]> wrote: >> > >> > Thanks again for working on this. I took another look at #15124. Let's >> try to get this merged, it's very self contained. >> > >> > Best, >> > Kevin Liu >> > >> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev < >> [email protected]> wrote: >> >> >> >> (I realised I mistakenly sent this only to Péter instead of the whole >> list) >> >> >> >> Could I get a review on https://github.com/apache/iceberg/pull/15124 >> please. >> >> Once that's sorted, I'll update the quickstart docs PR (15062) for the >> necessary changes. >> >> >> >> thanks, Robin. >> >> >> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <[email protected]> wrote: >> >>> >> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for >> review please. Once that's sorted, I'll update the quickstart docs PR >> (15062) for the necessary changes. >> >>> >> >>> thanks, Robin. >> >>> >> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <[email protected]> >> wrote: >> >>>> >> >>>> In there are no objections then I would prefer it in the >> docker/iceberg-flink-quickstart >> >>>> >> >>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: >> 2026. febr. 6., P, 11:13): >> >>>>> >> >>>>> Hi Peter, >> >>>>> >> >>>>> Thanks for the direction. I'll remove the publish step so that we >> can get the quickstart published, and then work on the publishing >> subsequently. >> >>>>> >> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or >> docker/iceberg-flink-quickstart ? >> >>>>> >> >>>>> thanks, Robin >> >>>>> >> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry < >> [email protected]> wrote: >> >>>>>> >> >>>>>> I think we have two options: >> >>>>>> >> >>>>>> Remove the image publication from this PR ( >> https://github.com/apache/iceberg/pull/15124) for now, and proceed with >> adding the Docker image and updating the documentation. >> >>>>>> Alternatively, we could discuss publishing the Flink quickstart >> image at the next Iceberg Community Sync and use that as an opportunity to >> simplify both the documentation and the overall user experience. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: >> 2026. febr. 4., Sze, 18:52): >> >>>>>>> >> >>>>>>> Hi, >> >>>>>>> >> >>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate >> some help untangling it. The recap is in my previous email (below). >> >>>>>>> >> >>>>>>> thanks, Robin. >> >>>>>>> >> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <[email protected]> >> wrote: >> >>>>>>>> >> >>>>>>>> Hi Kevin, >> >>>>>>>> >> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :) >> >>>>>>>> I have two related PRs: >> >>>>>>>> >> >>>>>>>> #15124: Add Flink Quickstart docker image >> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too) >> >>>>>>>> >> >>>>>>>> I can see a few routes forward: >> >>>>>>>> >> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with >> the publish script (I've not seen anything raised about it yet tho?) >> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still >> relying on local image build (not sure what this would achieve vs the >> option above tho?) >> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the >> published image not the local build >> >>>>>>>> >> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile >> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart >> (#15124). >> >>>>>>>> >> >>>>>>>> LMK if I've missed an angle here. >> >>>>>>>> >> >>>>>>>> thanks, Robin >> >>>>>>>> >> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <[email protected]> >> wrote: >> >>>>>>>>> >> >>>>>>>>> Thanks for working on this, Robin! It looks like the complexity >> here is publishing the docker image. What do you think about isolating that >> part? (Just move the publish script out of #15124) We can start with the >> Dockerfile definition, which allows us to build locally. This should >> unblock us from merging the getting started docs in #15062 >> >>>>>>>>> Thoughts? >> >>>>>>>>> >> >>>>>>>>> Best, >> >>>>>>>>> Kevin Liu >> >>>>>>>>> >> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev < >> [email protected]> wrote: >> >>>>>>>>>> >> >>>>>>>>>> Hi, >> >>>>>>>>>> >> >>>>>>>>>> Thanks for the discussion and input. >> >>>>>>>>>> It sounds like there are no major blockers. Could someone >> please review https://github.com/apache/iceberg/pull/15124 ? >> >>>>>>>>>> >> >>>>>>>>>> thanks, >> >>>>>>>>>> >> >>>>>>>>>> Robin. >> >>>>>>>>>> >> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <[email protected]> >> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>> Hey folks, >> >>>>>>>>>>> >> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the >> Spark base image and installs all the necessary jars. This is used for our >> integration test setup [2] and is inspired by >> databricks/docker-spark-iceberg [3]. We've made many improvements such as >> upgrading to Spark 4, supporting Spark Connect, and better image build >> caching. >> >>>>>>>>>>> >> >>>>>>>>>>> This is already self-contained and can be reused by other >> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep >> them in sync. >> >>>>>>>>>>> I think it would be beneficial for the project to publish >> this image and something similar for Flink. >> >>>>>>>>>>> >> >>>>>>>>>>> Let me know what you think. >> >>>>>>>>>>> >> >>>>>>>>>>> Best, >> >>>>>>>>>>> Kevin Liu >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> [1] >> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile >> >>>>>>>>>>> [2] >> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21 >> >>>>>>>>>>> [3] >> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile >> >>>>>>>>>>> [4] >> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile >> >>>>>>>>>>> >> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu < >> [email protected]> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives >> in our repository, it might make sense to also store the Docker images and >> the corresponding scripts there. >> >>>>>>>>>>>> >> >>>>>>>>>>>> I agree with Peter here. >> >>>>>>>>>>>> >> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg project >> should host those docker images. Not sure if the opinions have changed. >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev < >> [email protected]> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread. >> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be that >> ideally Flink would publish a Docker image that included the Iceberg >> dependencies? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> However we do this, I feel that the user coming to run the >> Flink quickstart should not have to build their own Docker image; this adds >> unnecessary friction that is easily alleviated. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy >> to discuss this idea with the Flink community; please let me know before I >> do so. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> thanks, Robin. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat < >> [email protected]> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Hi Robin and Peter, >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> I discussed community-maintained Docker images previously: >> >>>>>>>>>>>>>> >> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker >> image (now at 100K+ total downloads) and use Docker images published by the >> main engines in the quickstart, instead of maintaining these images >> ourselves. >> >>>>>>>>>>>>>> See the thread above for more details. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in >> favor of adding it and relying on the Docker images provided by Flink >> rather than maintaining our own images. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> - Ajantha >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry < >> [email protected]> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Hi Robin, >> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will >> have some extra stuff to do with the docker image. For example make sure >> that we have ci in place to build it. >> >>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>> Peter >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev < >> [email protected]> wrote: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea. >> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :) >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as >> separate PRs, or combine them? >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> thanks, Robin. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124 >> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062 >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré < >> [email protected]> wrote: >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Hi, >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> This is a great idea. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker >> image published by the project, we must ensure it is fully compliant with >> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem >> straightforward, it is a detail that is often overlooked. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> I would be happy to help with this process. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Regards, >> >>>>>>>>>>>>>>>>> JB >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels < >> [email protected]> wrote: >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Hey Robin, >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for >> new users to get >> >>>>>>>>>>>>>>>>>> all the dependencies in the right place. >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images: >> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> -Max >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry < >> [email protected]> wrote: >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > Hi Robin, >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in >> the Iceberg repo. >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: >> https://github.com/apache/iceberg/issues/13519 >> >>>>>>>>>>>>>>>>>> > The conclusion was: >> >>>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and >> bring this back up to date. My preference would be to rely on the Spark >> image <https://hub.docker.com/r/apache/spark> provided by the Apache >> Spark project, similar to what we do for the Hive < >> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be >> able to load all the Iceberg-specific JARs through the spark.jars.packages >> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>. >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list >> thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > Thanks for working on this, >> >>>>>>>>>>>>>>>>>> > Peter >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <[email protected]> ezt >> írta (időpont: 2026. jan. 22., Cs, 19:23): >> >>>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>>> >> Hi, >> >>>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR >> [1], what do people think about adding an official quickstart Docker image >> for Flink to the project? >> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses >> tabulario/spark-iceberg so perhaps that could be brought into the project >> too. >> >>>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>>> >> thanks, Robin. >> >>>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062 >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Robin Moffatt >> >> Sr. Principal Advisor, Streaming Data Technologies >> >
