I'm working on a demo cross-language pipeline on a local flink cluster that
relies on my python row coder PR [1]. The PR includes some changes to the
Java worker code, so I need to build a Java SDK container locally and use
that in the pipeline.

Unfortunately, whenever I run the pipeline, the apachebeam/java_sdk:latest
tag is moved off of my locally built image to a newly downloaded image with
a creation date 2 weeks ago, and that image is used instead. It looks like
the reason is we run `docker pull` before running the container [2]. As the
comment says this should be a no-op if the image already exists, but that
doesn't seem to be the case. If I just run `docker pull
apachebeam/java_sdk:latest` on my local machine it downloads the 2 week old
image and happily informs me:

Status: Downloaded newer image for apachebeam/java_sdk:latest

Does anyone know how I can prevent `docker pull` from doing this? I can
unblock myself for now just by commenting out the docker pull command, but
I'd like to understand what is going on here.

Thanks,
Brian

[1] https://github.com/apache/beam/pull/9188
[2]
https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerCommand.java#L80

Reply via email to