[jira] [Commented] (BEAM-9754) Dataflow runner currently does not support Python 3.8
[
https://issues.apache.org/jira/browse/BEAM-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17107439#comment-17107439
]
yoshiki obata commented on BEAM-9754:
-
[~kamilwu] made dataflow tests skipped at python 3.8 in [PR
11707|https://github.com/apache/beam/pull/11707]
> Dataflow runner currently does not support Python 3.8
> -
>
> Key: BEAM-9754
> URL: https://issues.apache.org/jira/browse/BEAM-9754
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: yoshiki obata
>Assignee: yoshiki obata
>Priority: Major
>
> Following tests fail due to the dataflow runner not supporting Python 3.8:
> - test_default_ip_configuration
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_experiment_use_multiple_sdk_containers
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_flexrs_blank
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_flexrs_cost
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_flexrs_speed
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_absent_in_released_sdks_with_runner_v2
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_absent_in_unreleased_sdk
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_custom_in_released_sdks
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_custom_in_released_sdks_with_runner_v2
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_default_in_released_sdks
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_harness_override_uses_base_version_in_rc_releases
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_number_of_worker_harness_threads
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_pinned_worker_harness_image_tag_used_in_dev_sdk
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_pipeline_url
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_private_ip_configuration
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_public_ip_configuration
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_sdk_harness_container_images_get_set
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_set_network
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_set_subnetwork
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_worker_harness_image_tag_matches_base_sdk_version_of_an_rc
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_worker_harness_image_tag_matches_released_sdk_version
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_worker_harness_override_takes_precedence_over_sdk_defaults
> (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
> - test_biqquery_read_streaming_fail
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_dataflow_worker_jar_flag_adds_use_staged_worker_jar_experiment
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_dataflow_worker_jar_flag_non_fnapi_noop
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_min_cpu_platform_flag_is_propagated_to_experiments
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_remote_runner_display_data
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_remote_runner_translation
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_streaming_create_translation
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_streaming_engine_flag_adds_windmill_experiments
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_unsupported_fnapi_features
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_upload_graph_experiment
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_use_fastavro_experiment_is_added_on_py3_and_onwards
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_use_fastavro_experiment_is_not_added_when_use_avro_is_present
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
> - test_bad_path
> (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest)
> - test_full_completion
> (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest)
> Sample error message:
> {code}
[jira] [Commented] (BEAM-9754) Dataflow runner currently does not support Python 3.8
[ https://issues.apache.org/jira/browse/BEAM-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088856#comment-17088856 ] Valentyn Tymofieiev commented on BEAM-9754: --- On Dataflow side, at a minimum we need to release a worker container with Python 3.8 to unblock this support. > Dataflow runner currently does not support Python 3.8 > - > > Key: BEAM-9754 > URL: https://issues.apache.org/jira/browse/BEAM-9754 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: yoshiki obata >Assignee: yoshiki obata >Priority: Major > > Following tests fail due to the dataflow runner not supporting Python 3.8: > - test_default_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_experiment_use_multiple_sdk_containers > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_blank > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_cost > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_speed > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_absent_in_released_sdks_with_runner_v2 > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_absent_in_unreleased_sdk > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_custom_in_released_sdks > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_custom_in_released_sdks_with_runner_v2 > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_default_in_released_sdks > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_uses_base_version_in_rc_releases > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_number_of_worker_harness_threads > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_pinned_worker_harness_image_tag_used_in_dev_sdk > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_pipeline_url > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_private_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_public_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_sdk_harness_container_images_get_set > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_set_network > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_set_subnetwork > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_image_tag_matches_base_sdk_version_of_an_rc > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_image_tag_matches_released_sdk_version > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_override_takes_precedence_over_sdk_defaults > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_biqquery_read_streaming_fail > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_dataflow_worker_jar_flag_adds_use_staged_worker_jar_experiment > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_dataflow_worker_jar_flag_non_fnapi_noop > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_min_cpu_platform_flag_is_propagated_to_experiments > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_remote_runner_display_data > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_remote_runner_translation > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_streaming_create_translation > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_streaming_engine_flag_adds_windmill_experiments > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_unsupported_fnapi_features > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_upload_graph_experiment > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_use_fastavro_experiment_is_added_on_py3_and_onwards > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_use_fastavro_experiment_is_not_added_when_use_avro_is_present > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_bad_path > (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest) > - test_full_completion > (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest) > Sample error
[jira] [Commented] (BEAM-9754) Dataflow runner currently does not support Python 3.8
[ https://issues.apache.org/jira/browse/BEAM-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088854#comment-17088854 ] Valentyn Tymofieiev commented on BEAM-9754: --- We can temporarily disable these tests on Python 3.8 until Dataflow runner supports 3.8. I can help add this support to Dataflow. > Dataflow runner currently does not support Python 3.8 > - > > Key: BEAM-9754 > URL: https://issues.apache.org/jira/browse/BEAM-9754 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: yoshiki obata >Assignee: yoshiki obata >Priority: Major > > Following tests fail due to the dataflow runner not supporting Python 3.8: > - test_default_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_experiment_use_multiple_sdk_containers > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_blank > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_cost > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_flexrs_speed > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_absent_in_released_sdks_with_runner_v2 > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_absent_in_unreleased_sdk > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_custom_in_released_sdks > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_custom_in_released_sdks_with_runner_v2 > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_default_in_released_sdks > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_harness_override_uses_base_version_in_rc_releases > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_number_of_worker_harness_threads > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_pinned_worker_harness_image_tag_used_in_dev_sdk > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_pipeline_url > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_private_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_public_ip_configuration > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_sdk_harness_container_images_get_set > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_set_network > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_set_subnetwork > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_image_tag_matches_base_sdk_version_of_an_rc > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_image_tag_matches_released_sdk_version > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_worker_harness_override_takes_precedence_over_sdk_defaults > (apache_beam.runners.dataflow.internal.apiclient_test.UtilTest) > - test_biqquery_read_streaming_fail > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_dataflow_worker_jar_flag_adds_use_staged_worker_jar_experiment > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_dataflow_worker_jar_flag_non_fnapi_noop > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_min_cpu_platform_flag_is_propagated_to_experiments > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_remote_runner_display_data > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_remote_runner_translation > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_streaming_create_translation > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_streaming_engine_flag_adds_windmill_experiments > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_unsupported_fnapi_features > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_upload_graph_experiment > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_use_fastavro_experiment_is_added_on_py3_and_onwards > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_use_fastavro_experiment_is_not_added_when_use_avro_is_present > (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) > - test_bad_path > (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest) > - test_full_completion > (apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest)
