Something like this: options = PipelineOptions(["--runner=PortableRunner", "--environment_config=apachebeam/python3.7_sdk ", # <--- "--job_endpoint=dnnserver2:8099"])
On 2019/09/17 02:14:00, Kyle Weaver <kcwea...@google.com> wrote: > I think environment_config is the option you are looking for. > > On Mon, Sep 16, 2019 at 7:06 PM Yu Watanabe <yu.w.ten...@gmail.com> wrote: > > > Hello. > > > > For testing I would like to use image uploaded to google container > > registry. > > How can I use images pulled from other than bintray.io ? > > > > > > ============================================================================ > > admin@ip-172-31-9-89:~$ gcloud container images list --repository > > asia.gcr.io/[PROJECTNAME] <http://asia.gcr.io/%5BPROJECTNAME%5D> > > NAME > > asia.gcr.io/[PROJECTNAME]/beam <http://asia.gcr.io/%5BPROJECTNAME%5D/beam> > > > > ============================================================================ > > > > Looks like FlinkRunner (2.15.0) uses bintray repository as a default > > behavior. As a result I am not able to use > > > > Caused by: java.lang.Exception: The user defined 'open()' method caused an > > exception: java.io.IOException: Received exit code 125 for command 'docker > > run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm > > ywatanabe-docker-apache.bintray.io/beam/python3:latest --id=1 > > --logging_endpoint=localhost:33787 --artifact_endpoint=localhost:41681 > > --provision_endpoint=localhost:45089 --control_endpoint=localhost:37045'. > > stderr: Unable to find image ' > > ywatanabe-docker-apache.bintray.io/beam/python3:latest' locallydocker: > > Error response from daemon: manifest for > > ywatanabe-docker-apache.bintray.io/beam/python3:latest not found: > > manifest unknown: The named manifest is not known to the registry.See > > 'docker run -- > > > > According to online doc, there is not parameter which controls which image > > to use . > > > > https://beam.apache.org/documentation/runners/flink/ > > > > Pipeline options I am using is below. > > > > > > ============================================================================ > > options = PipelineOptions([ > > "--runner=FlinkRunner", > > "--flink_version=1.8", > > "--flink_master_url=localhost:8081", > > "--worker_harness_container_image= > > asia.gcr.io/PROJECTNAME/beam/python3", > > "--experiments=beam_fn_api" > > ]) > > > > ============================================================================ > > > > Perhaps is there any environment variable to specify which image to use ? > > > > Best Regards, > > Yu Watanabe > > > > -- > > Yu Watanabe > > Weekend Freelancer who loves to challenge building data platform > > yu.w.ten...@gmail.com > > [image: LinkedIn icon] <https://www.linkedin.com/in/yuwatanabe1> [image: > > Twitter icon] <https://twitter.com/yuwtennis> > > > -- > Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com >