Hi Brian If we pull docker images, it always downloads from remote repository, which is expected behavior. In case we want to run a local image and pull it only when the image is not available at local, we can use `docker run` command directly, without pulling it in advance. [1] In case we want to pull images only when they are not available at local, we can use `docker images -q` to check if images are existing at local before pulling it. Another option is re-tag your local image, pass your image to pipeline and overwrite default one, but the code is still trying to pull, so if your image is not pushed to the remote repository, it would fail.
1. https://github.com/docker/cli/pull/1498 Hannah On Tue, Sep 24, 2019 at 11:56 AM Brian Hulette <[email protected]> wrote: > I'm working on a demo cross-language pipeline on a local flink cluster > that relies on my python row coder PR [1]. The PR includes some changes to > the Java worker code, so I need to build a Java SDK container locally and > use that in the pipeline. > > Unfortunately, whenever I run the pipeline, the apachebeam/java_sdk:latest > tag is moved off of my locally built image to a newly downloaded image with > a creation date 2 weeks ago, and that image is used instead. It looks like > the reason is we run `docker pull` before running the container [2]. As the > comment says this should be a no-op if the image already exists, but that > doesn't seem to be the case. If I just run `docker pull > apachebeam/java_sdk:latest` on my local machine it downloads the 2 week old > image and happily informs me: > > Status: Downloaded newer image for apachebeam/java_sdk:latest > > Does anyone know how I can prevent `docker pull` from doing this? I can > unblock myself for now just by commenting out the docker pull command, but > I'd like to understand what is going on here. > > Thanks, > Brian > > [1] https://github.com/apache/beam/pull/9188 > [2] > https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerCommand.java#L80 >
