Just a followup since no one replied it. My understanding is for any expanded transforms beam wants the environment self-described. So I updated boot and dockerfile for the java harness environment and use --sdk_harness_container_image_overrides in portable runner but fail to see the updated image loaded (default still), I guess only dataflow runner support it by glancing the code, but I believe it's the correct way and just need to deep dive the codes here when I turn back, then I will update this thread too.
Kobe On Wed, Sep 30, 2020 at 1:26 PM Kobe Feng <flllbls...@gmail.com> wrote: > Hi everyone, > Is there any recommended way to upload a third party jar (runtime scope) > for expanding transform like KafkaIO.Read when using the python portable > runner? Thank you! > > I tried --experiments=jar_packages=abc.jar,d.jar but just found those > artifacts in python harness with provision info, and the java harness just > uses the default environment for dependencies after expanding > transformation from the grpc server upon expansion jar for reading Kafka > messages. > > Also noticed above option will be removed in the future then tried > --files_to_stage but this option only exists in Java SDK pipeline options. > > -- > Yours Sincerely > Kobe Feng > -- Yours Sincerely Kobe Feng
