[jira] [Created] (FLINK-34984) Disaggregated State Storage and Management (Umbrella FLIP)
Yuan Mei created FLINK-34984: Summary: Disaggregated State Storage and Management (Umbrella FLIP) Key: FLINK-34984 URL: https://issues.apache.org/jira/browse/FLINK-34984 Project: Flink Issue Type: New Feature Components: API / Core, API / DataStream, Runtime / Checkpointing, Runtime / State Backends Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32651) Benchmark Support for Changelog Statebackend
Yuan Mei created FLINK-32651: Summary: Benchmark Support for Changelog Statebackend Key: FLINK-32651 URL: https://issues.apache.org/jira/browse/FLINK-32651 Project: Flink Issue Type: Improvement Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29082) Clean-up Leftovers for changelog pre-uploading files after failover
Yuan Mei created FLINK-29082: Summary: Clean-up Leftovers for changelog pre-uploading files after failover Key: FLINK-29082 URL: https://issues.apache.org/jira/browse/FLINK-29082 Project: Flink Issue Type: Improvement Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-27214) Build Failed on state backend benchmark
Yuan Mei created FLINK-27214: Summary: Build Failed on state backend benchmark Key: FLINK-27214 URL: https://issues.apache.org/jira/browse/FLINK-27214 Project: Flink Issue Type: Bug Reporter: Yuan Mei Failed build 72 of flink-statebackend-benchmarks-java11 ([Open|http://codespeed.dak8s.net:8080/job/flink-statebackend-benchmarks-java11/72/]): hudson.AbortException: script returned exit code 1 Failed build 214 of flink-statebackend-benchmarks-java8 ([Open|http://codespeed.dak8s.net:8080/job/flink-statebackend-benchmarks-java8/214/]): hudson.AbortException: script returned exit code 1 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26992) PojoSerializer may cause concurrent exception passing directly between threads
Yuan Mei created FLINK-26992: Summary: PojoSerializer may cause concurrent exception passing directly between threads Key: FLINK-26992 URL: https://issues.apache.org/jira/browse/FLINK-26992 Project: Flink Issue Type: Bug Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25512) Materialization Files are not cleaned up if no checkpoint is using it
Yuan Mei created FLINK-25512: Summary: Materialization Files are not cleaned up if no checkpoint is using it Key: FLINK-25512 URL: https://issues.apache.org/jira/browse/FLINK-25512 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25511) Leftovers after truncation are not be cleaned up if pre-uploading is enabled
Yuan Mei created FLINK-25511: Summary: Leftovers after truncation are not be cleaned up if pre-uploading is enabled Key: FLINK-25511 URL: https://issues.apache.org/jira/browse/FLINK-25511 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25470) Add/Expose/differentiate metrics of checkpoint size between changelog size vs materialization size
Yuan Mei created FLINK-25470: Summary: Add/Expose/differentiate metrics of checkpoint size between changelog size vs materialization size Key: FLINK-25470 URL: https://issues.apache.org/jira/browse/FLINK-25470 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-24436) FsStateChangelogWriter#lastAppendedSequenceNumber return different seq number with no writes
Yuan Mei created FLINK-24436: Summary: FsStateChangelogWriter#lastAppendedSequenceNumber return different seq number with no writes Key: FLINK-24436 URL: https://issues.apache.org/jira/browse/FLINK-24436 Project: Flink Issue Type: Bug Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-24435) FsStateChangelogWriter#lastAppendedSequenceNumber return different seq number with no writes
Yuan Mei created FLINK-24435: Summary: FsStateChangelogWriter#lastAppendedSequenceNumber return different seq number with no writes Key: FLINK-24435 URL: https://issues.apache.org/jira/browse/FLINK-24435 Project: Flink Issue Type: Bug Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-23765) PythonTableFunctionOperatorTestBase
Yuan Mei created FLINK-23765: Summary: PythonTableFunctionOperatorTestBase Key: FLINK-23765 URL: https://issues.apache.org/jira/browse/FLINK-23765 Project: Flink Issue Type: Bug Reporter: Yuan Mei PythonTableFunctionOperatorTest>PythonTableFunctionOperatorTestBase.testFinishBundleTriggeredByTime:147 » NullPointer PythonTableFunctionOperatorTest>PythonTableFunctionOperatorTestBase.testFinishBundleTriggeredOnCheckpoint:93 » NullPointer PythonTableFunctionOperatorTest>PythonTableFunctionOperatorTestBase.testLeftJoin:179 » NullPointer PythonTableFunctionOperatorTest>PythonTableFunctionOperatorTestBase.testRetractionFieldKept:68 » NullPointer -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-23441) Remove CheckpointOptions Argument away from Snapshotable#snapshot
Yuan Mei created FLINK-23441: Summary: Remove CheckpointOptions Argument away from Snapshotable#snapshot Key: FLINK-23441 URL: https://issues.apache.org/jira/browse/FLINK-23441 Project: Flink Issue Type: Bug Components: Runtime / State Backends Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-23367) testKeyGroupedInternalPriorityQueue does not dispose rocksdb properly, and fails the test
Yuan Mei created FLINK-23367: Summary: testKeyGroupedInternalPriorityQueue does not dispose rocksdb properly, and fails the test Key: FLINK-23367 URL: https://issues.apache.org/jira/browse/FLINK-23367 Project: Flink Issue Type: Bug Components: Runtime / State Backends Reporter: Yuan Mei The set of `testKeyGroupedInternalPriorityQueue` for `ChangelogDelegateEmbeddedRocksDBStateBackendTest` does not dispose rocksdb properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22678) Fix Loading Changelog Statebackend with configs set in job-level and cluster-level separately
Yuan Mei created FLINK-22678: Summary: Fix Loading Changelog Statebackend with configs set in job-level and cluster-level separately Key: FLINK-22678 URL: https://issues.apache.org/jira/browse/FLINK-22678 Project: Flink Issue Type: Bug Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22403) Update Chinese version of JDBC sink docs
Yuan Mei created FLINK-22403: Summary: Update Chinese version of JDBC sink docs Key: FLINK-22403 URL: https://issues.apache.org/jira/browse/FLINK-22403 Project: Flink Issue Type: Task Components: Connectors / JDBC, Documentation Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22239) Improve support for JdbcXaSinkFunction
Yuan Mei created FLINK-22239: Summary: Improve support for JdbcXaSinkFunction Key: FLINK-22239 URL: https://issues.apache.org/jira/browse/FLINK-22239 Project: Flink Issue Type: Task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22004) Translate Flink Roadmap to Chinese.
Yuan Mei created FLINK-22004: Summary: Translate Flink Roadmap to Chinese. Key: FLINK-22004 URL: https://issues.apache.org/jira/browse/FLINK-22004 Project: Flink Issue Type: Task Components: Documentation Reporter: Yuan Mei https://flink.apache.org/roadmap.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21448) Test Changelog State backend
Yuan Mei created FLINK-21448: Summary: Test Changelog State backend Key: FLINK-21448 URL: https://issues.apache.org/jira/browse/FLINK-21448 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21018) Follow up of FLINK-20488 to update checkpoint related documentation
Yuan Mei created FLINK-21018: Summary: Follow up of FLINK-20488 to update checkpoint related documentation Key: FLINK-21018 URL: https://issues.apache.org/jira/browse/FLINK-21018 Project: Flink Issue Type: Task Reporter: Yuan Mei Follow up of FLINK-20488 to update checkpoint-related documentation as well. https://ci.apache.org/projects/flink/flink-docs-stable/ops/monitoring/checkpoint_monitoring.html#checkpoint-details -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20252) Manual Test for Approximate Local Recovery
Yuan Mei created FLINK-20252: Summary: Manual Test for Approximate Local Recovery Key: FLINK-20252 URL: https://issues.apache.org/jira/browse/FLINK-20252 Project: Flink Issue Type: Bug Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20048) Make Approximate Local Recovery Compatible With PipelinedRegionSchedulingStrategy
Yuan Mei created FLINK-20048: Summary: Make Approximate Local Recovery Compatible With PipelinedRegionSchedulingStrategy Key: FLINK-20048 URL: https://issues.apache.org/jira/browse/FLINK-20048 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19988) Make Approximate Local Recovery Compatible With Unaligned Checkpoint
Yuan Mei created FLINK-19988: Summary: Make Approximate Local Recovery Compatible With Unaligned Checkpoint Key: FLINK-19988 URL: https://issues.apache.org/jira/browse/FLINK-19988 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19895) Unify life cycle management of ResultPartitionType Pipelined Family
Yuan Mei created FLINK-19895: Summary: Unify life cycle management of ResultPartitionType Pipelined Family Key: FLINK-19895 URL: https://issues.apache.org/jira/browse/FLINK-19895 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19774) Introduce sub partition view version for approximate Failover
Yuan Mei created FLINK-19774: Summary: Introduce sub partition view version for approximate Failover Key: FLINK-19774 URL: https://issues.apache.org/jira/browse/FLINK-19774 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19693) Scheduler Change for Approximate Local Recovery
Yuan Mei created FLINK-19693: Summary: Scheduler Change for Approximate Local Recovery Key: FLINK-19693 URL: https://issues.apache.org/jira/browse/FLINK-19693 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19632) Introduce a new ResultPartitionType for Approximate Local Recovery
Yuan Mei created FLINK-19632: Summary: Introduce a new ResultPartitionType for Approximate Local Recovery Key: FLINK-19632 URL: https://issues.apache.org/jira/browse/FLINK-19632 Project: Flink Issue Type: Sub-task Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-19547) Partial record clean Up after receiver task fails and restart
Yuan Mei created FLINK-19547: Summary: Partial record clean Up after receiver task fails and restart Key: FLINK-19547 URL: https://issues.apache.org/jira/browse/FLINK-19547 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.12.0 Reporter: Yuan Mei In the mode of approximate local recovery, the problem of partial record occurs when a record is spanning over multiple buffers. If the first (several) buffers have already been sent over the network and then get lost due to the failure of the receiver task, the remaining data belonging to the same record in transition should be cleaned up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-18113) Single Task Failure Recovery API Abstraction
Yuan Mei created FLINK-18113: Summary: Single Task Failure Recovery API Abstraction Key: FLINK-18113 URL: https://issues.apache.org/jira/browse/FLINK-18113 Project: Flink Issue Type: New Feature Components: API / Core, Runtime / Checkpointing, Runtime / Coordination, Runtime / Network Reporter: Yuan Mei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-18112) Single Task Failure Recovery Prototype
Yuan Mei created FLINK-18112: Summary: Single Task Failure Recovery Prototype Key: FLINK-18112 URL: https://issues.apache.org/jira/browse/FLINK-18112 Project: Flink Issue Type: New Feature Components: Runtime / Checkpointing, Runtime / Coordination, Runtime / Network Affects Versions: 1.12.0 Environment: Build a prototype of single task failure recovery to address and answer the following questions: Step 1: Scheduling part, restart a single node without restarting the upstream or downstream nodes. Step 2: Checkpointing part, as my understanding of how regional failover works, this part might not need modification. Step 3: Network part - how the recovered node able to link to the upstream ResultPartitions, and continue getting data - how the downstream node able to link to the recovered node, and continue getting node - how different netty transit mode affects the results - what if the failed node buffered data pool is full Step 4: Failover process verification Reporter: Yuan Mei Fix For: 1.12.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-18079) KafkaShuffle Manual Tests
Yuan Mei created FLINK-18079: Summary: KafkaShuffle Manual Tests Key: FLINK-18079 URL: https://issues.apache.org/jira/browse/FLINK-18079 Project: Flink Issue Type: Improvement Components: API / DataStream, Runtime / Checkpointing Affects Versions: 1.11.0 Reporter: Yuan Mei Fix For: 1.11.0 Manual Tests and Results to demonstrate KafkaShuffle is working as expected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17916) Separate KafkaShuffle read/write to different environments
Yuan Mei created FLINK-17916: Summary: Separate KafkaShuffle read/write to different environments Key: FLINK-17916 URL: https://issues.apache.org/jira/browse/FLINK-17916 Project: Flink Issue Type: Improvement Components: API / DataStream, Connectors / Kafka Affects Versions: 1.11.0 Reporter: Yuan Mei Fix For: 1.12.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17873) Add check for max concurrent checkpoints under UC mode
Yuan Mei created FLINK-17873: Summary: Add check for max concurrent checkpoints under UC mode Key: FLINK-17873 URL: https://issues.apache.org/jira/browse/FLINK-17873 Project: Flink Issue Type: Sub-task Components: Runtime / Checkpointing Reporter: Yuan Mei Fix For: 1.11.0 Currently, the UC mode only supports max concurrent checkpoint number = 1. So we need to check whether the configured max allowed checkpoints are more than 1 under the UC mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
