AnandInguva commented on issue #28084: URL: https://github.com/apache/beam/issues/28084#issuecomment-1687397824
> > The rootcause is: starting from 2.50.0, we no longer stage Beam SDK > > To clarify, what do you mean by this ? > > I could run a Python x-lang pipeline on the RC by installing the SDK via the zip file and providing the same file to the "--sdk_location" flag. > > https://dist.apache.org/repos/dist/dev/beam/2.50.0/python/apache-beam-2.50.0.zip Beam python downloads the sdk sdist or wheel from pypi and stages it to the staging environment. Then the boot.go looks for this staged file and installs it on Dataflow. We do this for Runner v1 since the default container for runner v1 doesn't contain Beam but from 2.50.0, runner v1 is deprecated, so we stopped staging SDK. If a tarball is passed to the `--sdk_location`, we stage it and install it on the Dataflow but by default we don't stage anything from 2.50.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
