[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588549#comment-16588549 ]
Ankur Goenka commented on BEAM-5190: ------------------------------------ Options are rewritten to URN causing the issue. https://github.com/apache/beam/blob/7c41e0a915083bd3b1fe52c2a417fa38a00e6463/sdks/python/apache_beam/runners/portability/portable_runner.py#L107 > Python pipeline options are not picked correctly by PortableRunner > ------------------------------------------------------------------ > > Key: BEAM-5190 > URL: https://issues.apache.org/jira/browse/BEAM-5190 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness > Reporter: Ankur Goenka > Assignee: Ankur Goenka > Priority: Major > > Python SDK worker is deserializing the pipeline options to dictionary instead > of PipelineOptions > Sample log > [grpc-default-executor-2] INFO sdk_worker_main.main - Python sdk harness > started with pipeline_options: \{u'beam:option:flink_master:v1': u'[auto]', > u'beam:option:streaming:v1': False, u'beam:option:experiments:v1': > [u'beam_fn_api', u'worker_threads=50'], u'beam:option:dry_run:v1': False, > u'beam:option:runner:v1': None, u'beam:option:profile_memory:v1': False, > u'beam:option:runtime_type_check:v1': False, u'beam:option:region:v1': > u'us-central1', u'beam:option:options_id:v1': 1, u'beam:option:no_auth:v1': > False, u'beam:option:dataflow_endpoint:v1': > u'https://dataflow.googleapis.com', u'beam:option:sdk_location:v1': > u'/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/dist/apache-beam-2.7.0.dev0.tar.gz', > u'beam:option:direct_runner_use_stacked_bundle:v1': True, > u'beam:option:save_main_session:v1': True, > u'beam:option:type_check_strictness:v1': u'DEFAULT_TO_ANY', > u'beam:option:profile_cpu:v1': False, u'beam:option:job_endpoint:v1': > u'localhost:8099', u'beam:option:job_name:v1': > u'BeamApp-goenka-0822071645-48ae1008', u'beam:option:temp_location:v1': > u'gs://clouddfe-goenka/tmp/', u'beam:option:app_name:v1': None, > u'beam:option:project:v1': u'google.com:clouddfe', > u'beam:option:pipeline_type_check:v1': True, > u'beam:option:staging_location:v1': u'gs://clouddfe-goenka/tmp/staging'} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)