You are passing a python 2.7 wheel to a job that was launched on python 3.6.
You need to select a correct wheel for the platform or pass source distribution (zip/tag.gz). On Thu, Aug 13, 2020, 15:20 Alex Amato <[email protected]> wrote: > I was trying to use the --sdk_location parameter in a python pipeline, to > allow users to run a snapshot SDK. Though it looks like it hit a type error > after downloading the .wdl file. > > Perhaps this code is assuming that remote files downloaded are text type, > not bytes type? Have I done something wrong? Or is this a bug? Any ideas? > > Thanks for taking a look, > Alex > > Using the --sdk_location parameter (Full command line > <https://paste.googleplex.com/5792777008840704>) > --sdk_location= > https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl > > INFO:apache_beam.runners.portability.stager:Failed to download Artifact > from > https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl > Traceback (most recent call last): > File > "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", > line 193, in _run_module_as_main > "__main__", mod_spec) > File > "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", > line 85, in _run_code > exec(code, run_globals) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", > line 142, in <module> > run() > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", > line 121, in run > result = p.run() > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", > line 521, in run > allow_proto_holders=True).run(False) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", > line 534, in run > return self.runner.run_pipeline(self, self._options) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", > line 479, in run_pipeline > artifacts=environments.python_sdk_dependencies(options))) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/transforms/environments.py", > line 611, in python_sdk_dependencies > staged_name in stager.Stager.create_job_resources(options, tmp_dir)) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", > line 235, in create_job_resources > resources.extend(Stager._create_beam_sdk(sdk_remote_location, > temp_dir)) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", > line 657, in _create_beam_sdk > Stager._download_file(sdk_remote_location, local_download_file) > File > "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", > line 375, in _download_file > f.write(content) > TypeError: write() argument must be str, not bytes > > >
