[ https://issues.apache.org/jira/browse/BEAM-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Valentyn Tymofieiev updated BEAM-6158: -------------------------------------- Priority: Major (was: Critical) > Enable support for save_main_session in Python 3 > ------------------------------------------------ > > Key: BEAM-6158 > URL: https://issues.apache.org/jira/browse/BEAM-6158 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness > Reporter: Mark Liu > Assignee: Valentyn Tymofieiev > Priority: Major > > This happened when I run wordcount example with portable Dataflow runner in > Python 3.5. The failure shows in worker log (unfortunately unformatted) of > [this > job|https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-29_11_47_38-6731484595556255542?project=google.com:clouddfe]: > {code:java} > Could not load main session: Traceback (most recent call last): File > "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 125, in main _load_main_session(semi_persistent_directory) File > "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 201, in _load_main_session pickler.load_session(session_file) File > "/usr/local/lib/python3.5/site-packages/apache_beam/internal/pickler.py", > line 269, in load_session return dill.load_session(file_path) File > "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 402, in > load_session module = unpickler.load() File > "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 465, in > find_class return StockUnpickler.find_class(self, module, name) > AttributeError: Can't get attribute 'WordExtractingDoFn' on <module > 'apache_beam.runners.worker.sdk_worker_main' from > '/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py'> > Traceback (most recent call last): File > "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 125, in main _load_main_session(semi_persistent_directory) File > "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 201, in _load_main_session pickler.load_session(session_file) File > "/usr/local/lib/python3.5/site-packages/apache_beam/internal/pickler.py", > line 269, in load_session return dill.load_session(file_path) File > "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 402, in > load_session module = unpickler.load() File > "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 465, in > find_class return StockUnpickler.find_class(self, module, name) > AttributeError: Can't get attribute 'WordExtractingDoFn' on <module > 'apache_beam.runners.worker.sdk_worker_main' from > '/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker_main.py'> > {code} > Looks like saved main session didn't work properly in Python 3. > +cc: [~tvalentyn] [~robertwb] [~altay] -- This message was sent by Atlassian JIRA (v7.6.3#76005)