I changed the .wdl I was passing in to: --sdk_location= https://storage.googleapis.com/beam-wheels-staging/master/699f872ea1ef3bdb1588a029fc6b1e3185e986a6-207696119/apache_beam-2.25.0.dev0-cp36-cp36m-macosx_10_9_x86_64.whl
and also tried --sdk_location= https://storage.googleapis.com/beam-wheels-staging/master/699f872ea1ef3bdb1588a029fc6b1e3185e986a6-207696119/apache-beam-2.25.0.dev0.zip python --version Python 3.6.8 In both cases the same TypeError occurs. https://paste.googleplex.com/6275630654029824 On Thu, Aug 13, 2020 at 3:52 PM Valentyn Tymofieiev <[email protected]> wrote: > You are passing a python 2.7 wheel to a job that was launched on python > 3.6. > > You need to select a correct wheel for the platform or pass source > distribution (zip/tag.gz). > > On Thu, Aug 13, 2020, 15:20 Alex Amato <[email protected]> wrote: > >> I was trying to use the --sdk_location parameter in a python pipeline, to >> allow users to run a snapshot SDK. Though it looks like it hit a type error >> after downloading the .wdl file. >> >> Perhaps this code is assuming that remote files downloaded are text type, >> not bytes type? Have I done something wrong? Or is this a bug? Any ideas? >> >> Thanks for taking a look, >> Alex >> >> Using the --sdk_location parameter (Full command line >> <https://paste.googleplex.com/5792777008840704>) >> --sdk_location= >> https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl >> >> INFO:apache_beam.runners.portability.stager:Failed to download Artifact >> from >> https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl >> Traceback (most recent call last): >> File >> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", >> line 193, in _run_module_as_main >> "__main__", mod_spec) >> File >> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", >> line 85, in _run_code >> exec(code, run_globals) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", >> line 142, in <module> >> run() >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", >> line 121, in run >> result = p.run() >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", >> line 521, in run >> allow_proto_holders=True).run(False) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", >> line 534, in run >> return self.runner.run_pipeline(self, self._options) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", >> line 479, in run_pipeline >> artifacts=environments.python_sdk_dependencies(options))) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/transforms/environments.py", >> line 611, in python_sdk_dependencies >> staged_name in stager.Stager.create_job_resources(options, tmp_dir)) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", >> line 235, in create_job_resources >> resources.extend(Stager._create_beam_sdk(sdk_remote_location, >> temp_dir)) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", >> line 657, in _create_beam_sdk >> Stager._download_file(sdk_remote_location, local_download_file) >> File >> "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", >> line 375, in _download_file >> f.write(content) >> TypeError: write() argument must be str, not bytes >> >> >>
