Thanks again for working on this. I took another look at #15124. Let's try to get this merged, it's very self contained.
Best, Kevin Liu On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev < [email protected]> wrote: > (I realised I mistakenly sent this only to Péter instead of the whole list) > > Could I get a review on https://github.com/apache/iceberg/pull/15124 > please. > Once that's sorted, I'll update the quickstart docs PR (15062) for the > necessary changes. > > thanks, Robin. > > On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <[email protected]> wrote: > >> OK, https://github.com/apache/iceberg/pull/15124 is now ready for review >> please. Once that's sorted, I'll update the quickstart docs PR (15062) for >> the necessary changes. >> >> thanks, Robin. >> >> On Fri, 6 Feb 2026 at 11:20, Péter Váry <[email protected]> >> wrote: >> >>> In there are no objections then I would prefer it in the >>> docker/iceberg-flink-quickstart >>> >>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: 2026. >>> febr. 6., P, 11:13): >>> >>>> Hi Peter, >>>> >>>> Thanks for the direction. I'll remove the publish step so that we can >>>> get the quickstart published, and then work on the publishing subsequently. >>>> >>>> Do you think the Dockerfile is best kept in flink/quickstart, or >>>> docker/iceberg-flink-quickstart ? >>>> >>>> thanks, Robin >>>> >>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <[email protected]> >>>> wrote: >>>> >>>>> I think we have two options: >>>>> >>>>> 1. Remove the image publication from this PR ( >>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed >>>>> with adding the Docker image and updating the documentation. >>>>> 2. Alternatively, we could discuss publishing the Flink quickstart >>>>> image at the next Iceberg Community Sync and use that as an >>>>> opportunity to >>>>> simplify both the documentation and the overall user experience. >>>>> >>>>> >>>>> >>>>> >>>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: >>>>> 2026. febr. 4., Sze, 18:52): >>>>> >>>>>> Hi, >>>>>> >>>>>> I have perhaps managed to deadlock this process :) I'd appreciate >>>>>> some help untangling it. The recap is in my previous email (below). >>>>>> >>>>>> thanks, Robin. >>>>>> >>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Kevin, >>>>>>> >>>>>>> Just recapping so that I'm clear, cos I'm getting confused :) >>>>>>> I have two related PRs: >>>>>>> >>>>>>> #15124: Add Flink Quickstart docker image >>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too) >>>>>>> >>>>>>> I can see a few routes forward: >>>>>>> >>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the >>>>>>> publish script (I've not seen anything raised about it yet tho?) >>>>>>> 2. Merge #15124 minus publish script, and then #15062 still relying >>>>>>> on local image build (not sure what this would achieve vs the option >>>>>>> above >>>>>>> tho?) >>>>>>> 3. Merge #15124 including publish script, then #15062 using the >>>>>>> published image not the local build >>>>>>> >>>>>>> Either way, one thing that needs resolving is the Dockerfile >>>>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart >>>>>>> (#15124). >>>>>>> >>>>>>> LMK if I've missed an angle here. >>>>>>> >>>>>>> thanks, Robin >>>>>>> >>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks for working on this, Robin! It looks like the complexity >>>>>>>> here is publishing the docker image. What do you think about isolating >>>>>>>> that >>>>>>>> part? (Just move the publish script out of #15124) We can start >>>>>>>> with the Dockerfile definition, which allows us to build locally. This >>>>>>>> should unblock us from merging the getting started docs in #15062 >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> Best, >>>>>>>> Kevin Liu >>>>>>>> >>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Thanks for the discussion and input. >>>>>>>>> It sounds like there are no major blockers. Could someone please >>>>>>>>> review https://github.com/apache/iceberg/pull/15124 ? >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Robin. >>>>>>>>> >>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hey folks, >>>>>>>>>> >>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark >>>>>>>>>> base image and installs all the necessary jars. This is used for our >>>>>>>>>> integration test setup [2] and is inspired by >>>>>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements >>>>>>>>>> such as >>>>>>>>>> upgrading to Spark 4, supporting Spark Connect, and better image >>>>>>>>>> build >>>>>>>>>> caching. >>>>>>>>>> >>>>>>>>>> This is already self-contained and can be reused by other >>>>>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to >>>>>>>>>> keep >>>>>>>>>> them in sync. >>>>>>>>>> I think it would be beneficial for the project to publish this >>>>>>>>>> image and something similar for Flink. >>>>>>>>>> >>>>>>>>>> Let me know what you think. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Kevin Liu >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile >>>>>>>>>> [2] >>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21 >>>>>>>>>> [3] >>>>>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile >>>>>>>>>> [4] >>>>>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile >>>>>>>>>> >>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> > Since the integration code for both Spark and Flink lives in >>>>>>>>>>> our repository, it might make sense to also store the Docker images >>>>>>>>>>> and the >>>>>>>>>>> corresponding scripts there. >>>>>>>>>>> >>>>>>>>>>> I agree with Peter here. >>>>>>>>>>> >>>>>>>>>>> The previous thread has some concerns if the Iceberg project >>>>>>>>>>> should host those docker images. Not sure if the opinions have >>>>>>>>>>> changed. >>>>>>>>>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread. >>>>>>>>>>>> Having looked at it, am I understanding the view to be that >>>>>>>>>>>> ideally Flink would publish a Docker image that included the >>>>>>>>>>>> Iceberg >>>>>>>>>>>> dependencies? >>>>>>>>>>>> >>>>>>>>>>>> However we do this, I feel that the user coming to run the >>>>>>>>>>>> Flink quickstart should not have to build their own Docker image; >>>>>>>>>>>> this adds >>>>>>>>>>>> unnecessary friction that is easily alleviated. >>>>>>>>>>>> >>>>>>>>>>>> If I've understood the situation correctly, then I'm happy to >>>>>>>>>>>> discuss this idea with the Flink community; please let me know >>>>>>>>>>>> before I do >>>>>>>>>>>> so. >>>>>>>>>>>> >>>>>>>>>>>> thanks, Robin. >>>>>>>>>>>> >>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Robin and Peter, >>>>>>>>>>>>> >>>>>>>>>>>>> I discussed community-maintained Docker images previously: >>>>>>>>>>>>> >>>>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq >>>>>>>>>>>>> >>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker >>>>>>>>>>>>> image <https://hub.docker.com/r/apache/iceberg-rest-fixture> >>>>>>>>>>>>> (now at 100K+ total downloads) and use Docker images published by >>>>>>>>>>>>> the main >>>>>>>>>>>>> engines in the quickstart, instead of maintaining these images >>>>>>>>>>>>> ourselves. >>>>>>>>>>>>> See the thread above for more details. >>>>>>>>>>>>> >>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor >>>>>>>>>>>>> of adding it and relying on the Docker images provided by Flink >>>>>>>>>>>>> rather than >>>>>>>>>>>>> maintaining our own images. >>>>>>>>>>>>> - Ajantha >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Robin, >>>>>>>>>>>>>> It would be nice to separate them. I expect that we will have >>>>>>>>>>>>>> some extra stuff to do with the docker image. For example make >>>>>>>>>>>>>> sure that we >>>>>>>>>>>>>> have ci in place to build it. >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Peter >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the positive reception of this idea. >>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as >>>>>>>>>>>>>>> separate PRs, or combine them? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> thanks, Robin. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124 >>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This is a great idea. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image >>>>>>>>>>>>>>>> published by the project, we must ensure it is fully compliant >>>>>>>>>>>>>>>> with ASF >>>>>>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this >>>>>>>>>>>>>>>> may seem >>>>>>>>>>>>>>>> straightforward, it is a detail that is often overlooked. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I would be happy to help with this process. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> JB >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hey Robin, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new >>>>>>>>>>>>>>>>> users to get >>>>>>>>>>>>>>>>> all the dependencies in the right place. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images: >>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -Max >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Hi Robin, >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the >>>>>>>>>>>>>>>>> Iceberg repo. >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: >>>>>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519 >>>>>>>>>>>>>>>>> > The conclusion was: >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring >>>>>>>>>>>>>>>>> this back up to date. My preference would be to rely on the >>>>>>>>>>>>>>>>> Spark image < >>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the >>>>>>>>>>>>>>>>> Apache Spark project, similar to what we do for the Hive < >>>>>>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. >>>>>>>>>>>>>>>>> We should be able to load all the Iceberg-specific JARs >>>>>>>>>>>>>>>>> through the >>>>>>>>>>>>>>>>> spark.jars.packages configuration < >>>>>>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>. >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread: >>>>>>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Thanks for working on this, >>>>>>>>>>>>>>>>> > Peter >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Robin Moffatt via dev <[email protected]> ezt írta >>>>>>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23): >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> Hi, >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], >>>>>>>>>>>>>>>>> what do people think about adding an official quickstart >>>>>>>>>>>>>>>>> Docker image for >>>>>>>>>>>>>>>>> Flink to the project? >>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses >>>>>>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into >>>>>>>>>>>>>>>>> the project >>>>>>>>>>>>>>>>> too. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> thanks, Robin. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >> >> > > -- > *Robin Moffatt* > *Sr. Principal Advisor, Streaming Data Technologies* >
