Beam High Priority Issue Report (65)
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/30593 The PostCommit XVR PythonUsingJavaSQL Dataflow job is flaky https://github.com/apache/beam/issues/30530 The LoadTests Java GBK Smoke job is flaky https://github.com/apache/beam/issues/30529 The PostCommit Java Sickbay job is flaky https://github.com/apache/beam/issues/30528 The Dask Runner Tests job is flaky https://github.com/apache/beam/issues/30527 The PostCommit Java IO Performance Tests job is flaky https://github.com/apache/beam/issues/30526 The PerformanceTests xlang KafkaIO Python job is flaky https://github.com/apache/beam/issues/30525 The PostCommit Python ValidatesContainer Dataflow With RC job is flaky https://github.com/apache/beam/issues/30524 The PerformanceTests SingleStoreIO job is flaky https://github.com/apache/beam/issues/30523 The PostCommit Java SingleStoreIO IT job is flaky https://github.com/apache/beam/issues/30522 The Build and Version Runner Docker Image job is flaky https://github.com/apache/beam/issues/30521 The LoadTests Go Combine Flink Batch job is flaky https://github.com/apache/beam/issues/30520 The LoadTests Python Combine Flink Streaming job is flaky https://github.com/apache/beam/issues/30519 The PostCommit XVR GoUsingJava Dataflow job is flaky https://github.com/apache/beam/issues/30518 The Update Python Depedencies job is flaky https://github.com/apache/beam/issues/30517 The PostCommit XVR Direct job is flaky https://github.com/apache/beam/issues/30516 The PostCommit Java ValidatesRunner SparkStructuredStreaming job is flaky https://github.com/apache/beam/issues/30515 The LoadTests Go SideInput Flink Batch job is flaky https://github.com/apache/beam/issues/30514 The Clean Up Prebuilt SDK Images job is flaky https://github.com/apache/beam/issues/30513 The PostCommit Python job is flaky https://github.com/apache/beam/issues/30512 The PostCommit Java PVR Spark Batch job is flaky https://github.com/apache/beam/issues/30511 The LoadTests Python Smoke job is flaky https://github.com/apache/beam/issues/30510 The PostCommit Java InfluxDbIO Integration Test job is flaky https://github.com/apache/beam/issues/30508 The PostCommit Java ValidatesRunner Flink job is flaky https://github.com/apache/beam/issues/30507 The LoadTests Go GBK Flink Batch job is flaky https://github.com/apache/beam/issues/30506 The TypeScript Tests job is flaky https://github.com/apache/beam/issues/30505 The PostRelease Nightly Snapshot job is flaky https://github.com/apache/beam/issues/30504 The LoadTests Python Combine Dataflow Streaming job is flaky https://github.com/apache/beam/issues/30503 The PostCommit Java ValidatesRunner Flink Java11 job is flaky https://github.com/apache/beam/issues/30502 The LoadTests Go CoGBK Flink Batch job is flaky https://github.com/apache/beam/issues/30498 [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss https://github.com/apache/beam/issues/30447 [Failing Test]: beam_PostRelease_NightlySnapshot red due to various flakes in Dataflow example tests https://github.com/apache/beam/issues/29971 [Bug]: FixedWindows not working for large Kafka topic https://github.com/apache/beam/issues/29926 [Bug]: FileIO: lack of timeouts may cause the pipeline to get stuck indefinitely https://github.com/apache/beam/issues/29902 [Bug]: Messages are not ACK on Pubsub starting Beam 2.52.0 on Flink Runner in detached mode https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java SDK Harness doesn't update user counters in OnTimer callback functions https://github.com/apache/beam/issues/28760 [Bug]: EFO Kinesis IO reader provided by apache beam does not pick the event time for watermarking https://github.com/apache/beam/issues/28383 [Failing Test]: org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testMaxThreadMetric https://github.com/apache/beam/issues/28326 Bug: apache_beam.io.gcp.pubsublite.ReadFromPubSubLite not working https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not working when using CreateDisposition.CREATE_IF_NEEDED https://github.com/apache/beam/issues/27616 [Bug]: Unable to use applyRowMutations() in bigquery IO apache beam java https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with inequality filters https://github.com/apache/beam/issues/27314 [Failing Test]: bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1] https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when using Kafka and GroupByKey on Dataflow Runner https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested ROW (described below) https://github.com/apache/beam/issues/26343 [Bug]:
Beam High Priority Issue Report (65)
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/30530 The LoadTests Java GBK Smoke job is flaky https://github.com/apache/beam/issues/30529 The PostCommit Java Sickbay job is flaky https://github.com/apache/beam/issues/30528 The Dask Runner Tests job is flaky https://github.com/apache/beam/issues/30527 The PostCommit Java IO Performance Tests job is flaky https://github.com/apache/beam/issues/30526 The PerformanceTests xlang KafkaIO Python job is flaky https://github.com/apache/beam/issues/30525 The PostCommit Python ValidatesContainer Dataflow With RC job is flaky https://github.com/apache/beam/issues/30524 The PerformanceTests SingleStoreIO job is flaky https://github.com/apache/beam/issues/30523 The PostCommit Java SingleStoreIO IT job is flaky https://github.com/apache/beam/issues/30522 The Build and Version Runner Docker Image job is flaky https://github.com/apache/beam/issues/30521 The LoadTests Go Combine Flink Batch job is flaky https://github.com/apache/beam/issues/30520 The LoadTests Python Combine Flink Streaming job is flaky https://github.com/apache/beam/issues/30519 The PostCommit XVR GoUsingJava Dataflow job is flaky https://github.com/apache/beam/issues/30518 The Update Python Depedencies job is flaky https://github.com/apache/beam/issues/30517 The PostCommit XVR Direct job is flaky https://github.com/apache/beam/issues/30516 The PostCommit Java ValidatesRunner SparkStructuredStreaming job is flaky https://github.com/apache/beam/issues/30515 The LoadTests Go SideInput Flink Batch job is flaky https://github.com/apache/beam/issues/30514 The Clean Up Prebuilt SDK Images job is flaky https://github.com/apache/beam/issues/30513 The PostCommit Python job is flaky https://github.com/apache/beam/issues/30512 The PostCommit Java PVR Spark Batch job is flaky https://github.com/apache/beam/issues/30511 The LoadTests Python Smoke job is flaky https://github.com/apache/beam/issues/30510 The PostCommit Java InfluxDbIO Integration Test job is flaky https://github.com/apache/beam/issues/30508 The PostCommit Java ValidatesRunner Flink job is flaky https://github.com/apache/beam/issues/30507 The LoadTests Go GBK Flink Batch job is flaky https://github.com/apache/beam/issues/30506 The TypeScript Tests job is flaky https://github.com/apache/beam/issues/30505 The PostRelease Nightly Snapshot job is flaky https://github.com/apache/beam/issues/30504 The LoadTests Python Combine Dataflow Streaming job is flaky https://github.com/apache/beam/issues/30503 The PostCommit Java ValidatesRunner Flink Java11 job is flaky https://github.com/apache/beam/issues/30502 The LoadTests Go CoGBK Flink Batch job is flaky https://github.com/apache/beam/issues/30498 [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss https://github.com/apache/beam/issues/30459 [Failing Test]: Python PostCommit failing hdfsIntegrationTest in generate_external_transform_wrappers https://github.com/apache/beam/issues/30447 [Failing Test]: beam_PostRelease_NightlySnapshot red due to various flakes in Dataflow example tests https://github.com/apache/beam/issues/29971 [Bug]: FixedWindows not working for large Kafka topic https://github.com/apache/beam/issues/29926 [Bug]: FileIO: lack of timeouts may cause the pipeline to get stuck indefinitely https://github.com/apache/beam/issues/29902 [Bug]: Messages are not ACK on Pubsub starting Beam 2.52.0 on Flink Runner in detached mode https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java SDK Harness doesn't update user counters in OnTimer callback functions https://github.com/apache/beam/issues/28760 [Bug]: EFO Kinesis IO reader provided by apache beam does not pick the event time for watermarking https://github.com/apache/beam/issues/28383 [Failing Test]: org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testMaxThreadMetric https://github.com/apache/beam/issues/28326 Bug: apache_beam.io.gcp.pubsublite.ReadFromPubSubLite not working https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not working when using CreateDisposition.CREATE_IF_NEEDED https://github.com/apache/beam/issues/27616 [Bug]: Unable to use applyRowMutations() in bigquery IO apache beam java https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with inequality filters https://github.com/apache/beam/issues/27314 [Failing Test]: bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1] https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when using Kafka and GroupByKey on Dataflow Runner https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested ROW (described below) https://github.com/apache/beam/issues/26343 [Bug]:
Beam High Priority Issue Report (65)
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/24021 [Bug]: SparkReceriverIO and KafkaIO performance tests failing due to unschedulable pods on k8s https://github.com/apache/beam/issues/24016 Failure in test: "beam_PerformanceTests_Kafka_IO" - timed out waiting for the condition on jobs/kafka-config https://github.com/apache/beam/issues/23982 [Bug]: Could not find a version that matches protobuf https://github.com/apache/beam/issues/23974 [Bug]: The top tesult for "Beam godocs" in Google points to an old Godocs page of Beam https://github.com/apache/beam/issues/23944 beam_PreCommit_Python_Cron regularily failing - test_pardo_large_input flaky https://github.com/apache/beam/issues/23940 beam_PostCommit_Py_ValCont failing https://github.com/apache/beam/issues/23815 [Bug]: Neo4j tests failing https://github.com/apache/beam/issues/23745 [Bug]: Samza AsyncDoFnRunnerTest.testSimplePipeline is flaky https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle https://github.com/apache/beam/issues/22969 Discrepancy in behavior of `DoFn.process()` when `yield` is combined with `return` statement, or vice versa https://github.com/apache/beam/issues/22913 [Bug]: beam_PostCommit_Java_ValidatesRunner_Flink is flakes in org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState https://github.com/apache/beam/issues/22321 PortableRunnerTestWithExternalEnv.test_pardo_large_input is regularly failing on jenkins https://github.com/apache/beam/issues/21713 404s in BigQueryIO don't get output to Failed Inserts PCollection https://github.com/apache/beam/issues/21561 ExternalPythonTransformTest.trivialPythonTransform flaky https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: Connection refused https://github.com/apache/beam/issues/21462 Flake in org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use https://github.com/apache/beam/issues/21261 org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer is flaky https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit data at GC time https://github.com/apache/beam/issues/21123 Multiple jobs running on Flink session cluster reuse the persistent Python environment. https://github.com/apache/beam/issues/21113 testTwoTimersSettingEachOtherWithCreateAsInputBounded flaky https://github.com/apache/beam/issues/20976 apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics is flaky https://github.com/apache/beam/issues/20975 org.apache.beam.runners.flink.ReadSourcePortableTest.testExecution[streaming: false] is flaky https://github.com/apache/beam/issues/20974 Python GHA PreCommits flake with grpc.FutureTimeoutError on SDK harness startup https://github.com/apache/beam/issues/20689 Kafka commitOffsetsInFinalize OOM on Flink https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit empty pane when it should https://github.com/apache/beam/issues/19814 Flink streaming flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful https://github.com/apache/beam/issues/19734 WatchTest.testMultiplePollsWithManyResults flake: Outputs must be in timestamp order (sickbayed) https://github.com/apache/beam/issues/19241 Python Dataflow integration tests should export the pipeline Job ID and console output to Jenkins Test Result section P1 Issues with no update in the last week: https://github.com/apache/beam/issues/23906 [Bug]: Dataflow jpms tests fail on the 2.43.0 release branch https://github.com/apache/beam/issues/23875 [Bug]: beam.Row.__eq__ returns true for unequal rows https://github.com/apache/beam/issues/23855 [FLAKY-WORKFLOW] [10535797]: THIS IS A TEST, PLEASE IGNORE #1 https://github.com/apache/beam/issues/23627 [Bug]: Website precommit flaky https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder will drop message id and orderingKey https://github.com/apache/beam/issues/23489 [Bug]: add DebeziumIO to the connectors page https://github.com/apache/beam/issues/23306 [Bug]: BigQueryBatchFileLoads in python loses data when using WRITE_TRUNCATE https://github.com/apache/beam/issues/23286 [Bug]: beam_PerformanceTests_InfluxDbIO_IT Flaky > 50 % Fail https://github.com/apache/beam/issues/22891 [Bug]: beam_PostCommit_XVR_PythonUsingJavaDataflow is flaky https://github.com/apache/beam/issues/22605 [Bug]: Beam Python
Beam High Priority Issue Report (65)
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/24021 [Bug]: SparkReceriverIO and KafkaIO performance tests failing due to unschedulable pods on k8s https://github.com/apache/beam/issues/24016 Failure in test: "beam_PerformanceTests_Kafka_IO" - timed out waiting for the condition on jobs/kafka-config https://github.com/apache/beam/issues/23982 [Bug]: Could not find a version that matches protobuf https://github.com/apache/beam/issues/23974 [Bug]: The top tesult for "Beam godocs" in Google points to an old Godocs page of Beam https://github.com/apache/beam/issues/23944 beam_PreCommit_Python_Cron regularily failing - test_pardo_large_input flaky https://github.com/apache/beam/issues/23940 beam_PostCommit_Py_ValCont failing https://github.com/apache/beam/issues/23815 [Bug]: Neo4j tests failing https://github.com/apache/beam/issues/23745 [Bug]: Samza AsyncDoFnRunnerTest.testSimplePipeline is flaky https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle https://github.com/apache/beam/issues/22969 Discrepancy in behavior of `DoFn.process()` when `yield` is combined with `return` statement, or vice versa https://github.com/apache/beam/issues/22913 [Bug]: beam_PostCommit_Java_ValidatesRunner_Flink is flakes in org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState https://github.com/apache/beam/issues/22321 PortableRunnerTestWithExternalEnv.test_pardo_large_input is regularly failing on jenkins https://github.com/apache/beam/issues/21713 404s in BigQueryIO don't get output to Failed Inserts PCollection https://github.com/apache/beam/issues/21561 ExternalPythonTransformTest.trivialPythonTransform flaky https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: Connection refused https://github.com/apache/beam/issues/21462 Flake in org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use https://github.com/apache/beam/issues/21261 org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer is flaky https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit data at GC time https://github.com/apache/beam/issues/21123 Multiple jobs running on Flink session cluster reuse the persistent Python environment. https://github.com/apache/beam/issues/21113 testTwoTimersSettingEachOtherWithCreateAsInputBounded flaky https://github.com/apache/beam/issues/20976 apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics is flaky https://github.com/apache/beam/issues/20975 org.apache.beam.runners.flink.ReadSourcePortableTest.testExecution[streaming: false] is flaky https://github.com/apache/beam/issues/20974 Python GHA PreCommits flake with grpc.FutureTimeoutError on SDK harness startup https://github.com/apache/beam/issues/20689 Kafka commitOffsetsInFinalize OOM on Flink https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit empty pane when it should https://github.com/apache/beam/issues/19814 Flink streaming flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful https://github.com/apache/beam/issues/19734 WatchTest.testMultiplePollsWithManyResults flake: Outputs must be in timestamp order (sickbayed) https://github.com/apache/beam/issues/19241 Python Dataflow integration tests should export the pipeline Job ID and console output to Jenkins Test Result section P1 Issues with no update in the last week: https://github.com/apache/beam/issues/23906 [Bug]: Dataflow jpms tests fail on the 2.43.0 release branch https://github.com/apache/beam/issues/23875 [Bug]: beam.Row.__eq__ returns true for unequal rows https://github.com/apache/beam/issues/23855 [FLAKY-WORKFLOW] [10535797]: THIS IS A TEST, PLEASE IGNORE #1 https://github.com/apache/beam/issues/23627 [Bug]: Website precommit flaky https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder will drop message id and orderingKey https://github.com/apache/beam/issues/23489 [Bug]: add DebeziumIO to the connectors page https://github.com/apache/beam/issues/23306 [Bug]: BigQueryBatchFileLoads in python loses data when using WRITE_TRUNCATE https://github.com/apache/beam/issues/23286 [Bug]: beam_PerformanceTests_InfluxDbIO_IT Flaky > 50 % Fail https://github.com/apache/beam/issues/22891 [Bug]: beam_PostCommit_XVR_PythonUsingJavaDataflow is flaky https://github.com/apache/beam/issues/22605 [Bug]: Beam Python
