[ https://issues.apache.org/jira/browse/BEAM-8402?focusedWorklogId=376714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376714 ]
ASF GitHub Bot logged work on BEAM-8402: ---------------------------------------- Author: ASF GitHub Bot Created on: 24/Jan/20 05:50 Start Date: 24/Jan/20 05:50 Worklog Time Spent: 10m Work Description: aaltay commented on issue #9811: [BEAM-8402] Create a class hierarchy to represent environments URL: https://github.com/apache/beam/pull/9811#issuecomment-578000835 Beam 2.18 broke some users. Reverting back to 2.17 resolves the issues. I believe this PR is the root cause. I am trying to get more information to file a JIRA. A stack trace I got looks like this: ``` File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 339, in <module> main() File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 332, in main execution_info = launcher.launch() File "/tfx-src/tfx/orchestration/launcher/base_component_launcher.py", line 205, in launch execution_decision.exec_properties) File "/tfx-src/tfx/orchestration/launcher/in_process_component_launcher.py", line 67, in _run_executor executor.Do(input_dict, output_dict, exec_properties) File "/tfx-src/tfx/components/example_gen/base_example_gen_executor.py", line 230, in Do artifact_utils.get_split_uri(output_dict['examples'], split_name))) File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 436, in __exit__ self.run().wait_until_finish() File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 416, in run self._options).run(False) File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 429, in run return self.runner.run_pipeline(self, self._options) File "/opt/venv/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner.py", line 389, in run_pipeline default_environment=self._default_environment)) File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 637, in to_runner_api default_environment=default_environment) File "/opt/venv/lib/python3.6/site-packages/apache_beam/runners/pipeline_context.py", line 134, in __init__ default_environment, label='default_environment') File "/opt/venv/lib/python3.6/site-packages/apache_beam/runners/pipeline_context.py", line 66, in get_id if obj not in self._obj_to_id: TypeError: unhashable type: 'Environment' ``` I am suspecting this is related to the changes in the Environment class. @violalyu @chadrik or other reviewers, do you know what this might be about? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 376714) Time Spent: 5h 10m (was: 5h) > Create a class hierarchy to represent environments > -------------------------------------------------- > > Key: BEAM-8402 > URL: https://issues.apache.org/jira/browse/BEAM-8402 > Project: Beam > Issue Type: New Feature > Components: sdk-py-core > Reporter: Chad Dombrova > Assignee: Chad Dombrova > Priority: Major > Time Spent: 5h 10m > Remaining Estimate: 0h > > As a first step towards making it possible to assign different environments > to sections of a pipeline, we first need to expose environment classes to the > pipeline API. Unlike PTransforms, PCollections, Coders, and Windowings, > environments exists solely in the portability framework as protobuf objects. > By creating a hierarchy of "native" classes that represent the various > environment types -- external, docker, process, etc -- users will be able to > instantiate these and assign them to parts of the pipeline. The assignment > portion will be covered in a follow-up issue/PR. -- This message was sent by Atlassian Jira (v8.3.4#803005)