Re: [PR] [FLINK-34996][Connectors/Kafka] Use UserCodeCL to instantiate Deserializer [flink-connector-kafka]

2024-04-07 Thread via GitHub
morazow commented on PR #89: URL: https://github.com/apache/flink-connector-kafka/pull/89#issuecomment-2041992850 Thanks @hugogu for the PR! I have added some comments, please have a look. Do you know why the CI is failing? -- This is an automated message from the Apache Git Servic

Re: [PR] [FLINK-34996][Connectors/Kafka] Use UserCodeCL to instantiate Deserializer [flink-connector-kafka]

2024-04-07 Thread via GitHub
morazow commented on code in PR #89: URL: https://github.com/apache/flink-connector-kafka/pull/89#discussion_r1555308164 ## flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/sink/KafkaSerializerWrapperTest.java: ## @@ -0,0 +1,63 @@ +package org.apache.flink.c

[jira] [Created] (FLINK-35049) Implement Async State API for ForStStateBackend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35049: Summary: Implement Async State API for ForStStateBackend Key: FLINK-35049 URL: https://issues.apache.org/jira/browse/FLINK-35049 Project: Flink Issue Type: S

[jira] [Created] (FLINK-35048) Implement all methods of AsyncKeyedStateBakend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35048: Summary: Implement all methods of AsyncKeyedStateBakend Key: FLINK-35048 URL: https://issues.apache.org/jira/browse/FLINK-35048 Project: Flink Issue Type: S

[jira] [Updated] (FLINK-35047) Introduce ForStStateBackend to manage ForSt

2024-04-07 Thread Hangxiang Yu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hangxiang Yu updated FLINK-35047: - Summary: Introduce ForStStateBackend to manage ForSt (was: Introduce ForStStateBackend) > Intr

[jira] [Created] (FLINK-35047) Introduce ForStStateBackend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35047: Summary: Introduce ForStStateBackend Key: FLINK-35047 URL: https://issues.apache.org/jira/browse/FLINK-35047 Project: Flink Issue Type: Sub-task Co

[jira] [Created] (FLINK-35046) Introduce New KeyedStateBackend related Async interfaces

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35046: Summary: Introduce New KeyedStateBackend related Async interfaces Key: FLINK-35046 URL: https://issues.apache.org/jira/browse/FLINK-35046 Project: Flink Issu

Re: [PR] [cdc-cli] When using miniCluster, don't check 'FLINK_HOME'. [flink-cdc]

2024-04-07 Thread via GitHub
joyCurry30 closed pull request #2813: [cdc-cli] When using miniCluster, don't check 'FLINK_HOME'. URL: https://github.com/apache/flink-cdc/pull/2813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] [cdc-cli] When using miniCluster, don't check 'FLINK_HOME'. [flink-cdc]

2024-04-07 Thread via GitHub
joyCurry30 commented on code in PR #2813: URL: https://github.com/apache/flink-cdc/pull/2813#discussion_r1555290167 ## flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/CliFrontend.java: ## @@ -86,6 +87,18 @@ static CliExecutor createExecutor(CommandLine commandLine) throws

[jira] [Created] (FLINK-35045) Introduce ForStFileSystem to support reading and writing with ByteBuffer

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35045: Summary: Introduce ForStFileSystem to support reading and writing with ByteBuffer Key: FLINK-35045 URL: https://issues.apache.org/jira/browse/FLINK-35045 Project: Fli

[jira] [Commented] (FLINK-35009) Change on getTransitivePredecessors breaks connectors

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834799#comment-17834799 ] Zhu Zhu commented on FLINK-35009: - Thanks for looking into the issue. [~Weijie Guo] So i

[jira] [Updated] (FLINK-35044) Introduce statebackend-forst module

2024-04-07 Thread Hangxiang Yu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hangxiang Yu updated FLINK-35044: - Component/s: Runtime / State Backends > Introduce statebackend-forst module > --

[jira] [Created] (FLINK-35044) Introduce statebackend-forst module

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35044: Summary: Introduce statebackend-forst module Key: FLINK-35044 URL: https://issues.apache.org/jira/browse/FLINK-35044 Project: Flink Issue Type: Sub-task

Re: [PR] [cdc-cli][cdc-composer] Applying 'flink-config' for pipeline yaml [flink-cdc]

2024-04-07 Thread via GitHub
joyCurry30 commented on code in PR #3187: URL: https://github.com/apache/flink-cdc/pull/3187#discussion_r1555283731 ## flink-cdc-cli/src/test/resources/definitions/pipeline-definition-full.yaml: ## @@ -53,3 +53,8 @@ pipeline: name: source-database-sync-pipe parallelism: 4

[jira] [Comment Edited] (FLINK-32513) Job in BATCH mode with a significant number of transformations freezes on method StreamGraphGenerator.existsUnboundedSource()

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834798#comment-17834798 ] Zhu Zhu edited comment on FLINK-32513 at 4/8/24 6:27 AM: - {{Tran

[jira] [Created] (FLINK-35043) Release beta version of ForSt

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35043: Summary: Release beta version of ForSt Key: FLINK-35043 URL: https://issues.apache.org/jira/browse/FLINK-35043 Project: Flink Issue Type: Sub-task

Re: [PR] [cdc-cli][cdc-composer] Applying 'flink-config' for pipeline yaml [flink-cdc]

2024-04-07 Thread via GitHub
joyCurry30 commented on code in PR #3187: URL: https://github.com/apache/flink-cdc/pull/3187#discussion_r1555283731 ## flink-cdc-cli/src/test/resources/definitions/pipeline-definition-full.yaml: ## @@ -53,3 +53,8 @@ pipeline: name: source-database-sync-pipe parallelism: 4

[jira] [Commented] (FLINK-32513) Job in BATCH mode with a significant number of transformations freezes on method StreamGraphGenerator.existsUnboundedSource()

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834798#comment-17834798 ] Zhu Zhu commented on FLINK-32513: - {{Transformation}} is not a public interface, it is a

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on PR #24614: URL: https://github.com/apache/flink/pull/24614#issuecomment-2041936516 @masteryhx Thanks for your comments! Applied some suggestions, PTAL thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Assigned] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2024-04-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-33934: Assignee: Yuan Kui > Flink SQL Source use raw format maybe lead to data lost > --

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555253454 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Updated] (FLINK-35042) Streaming File Sink s3 end-to-end test failed as TM lost

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-35042: --- Description: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58782&view=logs&

[jira] [Created] (FLINK-35042) Streaming File Sink s3 end-to-end test failed as TM lost

2024-04-07 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-35042: -- Summary: Streaming File Sink s3 end-to-end test failed as TM lost Key: FLINK-35042 URL: https://issues.apache.org/jira/browse/FLINK-35042 Project: Flink Issue Ty

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555247481 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555245552 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555244839 ## flink-runtime/src/test/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionControllerTest.java: ## @@ -0,0 +1,305 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (FLINK-34273) git fetch fails

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34273: --- Affects Version/s: 1.20.0 > git fetch fails > --- > > Key: FLINK-34273 >

[jira] [Commented] (FLINK-35023) YARNApplicationITCase failed on Azure

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834783#comment-17834783 ] Weijie Guo commented on FLINK-35023: jdk17 https://dev.azure.com/apache-flink/apach

[jira] [Commented] (FLINK-34273) git fetch fails

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834784#comment-17834784 ] Weijie Guo commented on FLINK-34273: https://dev.azure.com/apache-flink/apache-flink

[jira] [Updated] (FLINK-35041) IncrementalRemoteKeyedStateHandleTest.testSharedStateReRegistration failed

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-35041: --- Description: {code:java} Apr 08 03:22:45 03:22:45.450 [ERROR] org.apache.flink.runtime.state.Increm

[jira] [Created] (FLINK-35041) IncrementalRemoteKeyedStateHandleTest.testSharedStateReRegistration failed

2024-04-07 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-35041: -- Summary: IncrementalRemoteKeyedStateHandleTest.testSharedStateReRegistration failed Key: FLINK-35041 URL: https://issues.apache.org/jira/browse/FLINK-35041 Project: Flink

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555241719 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/KeyAccountingUnit.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555239594 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateExecutor.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555237399 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Jiabao Sun (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834779#comment-17834779 ] Jiabao Sun commented on FLINK-34955: I have rechecked the dependency of `commons-cod

[jira] [Commented] (FLINK-35010) Bump org.apache.commons:commons-compress from 1.24.0 to 1.26.0 for Flink Mongodb connector

2024-04-07 Thread Jiabao Sun (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834778#comment-17834778 ] Jiabao Sun commented on FLINK-35010: I have rechecked the dependency of `commons-cod

[jira] [Commented] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.0 for Flink Kafka connector

2024-04-07 Thread Jiabao Sun (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834777#comment-17834777 ] Jiabao Sun commented on FLINK-35008: I have rechecked the dependency of `commons-cod

Re: [PR] [FLINK-34955] Upgrade commons-compress to 1.26.0. [flink]

2024-04-07 Thread via GitHub
Jiabao-Sun commented on code in PR #24580: URL: https://github.com/apache/flink/pull/24580#discussion_r1555174874 ## flink-end-to-end-tests/flink-sql-client-test/pom.xml: ## @@ -69,6 +69,13 @@ under the License. kafka test

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
fredia commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555208732 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
masteryhx commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1555175989 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2024-04-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-33934: - Affects Version/s: 1.19.0 1.18.0 1.17.0

[jira] [Commented] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2024-04-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834765#comment-17834765 ] Yun Tang commented on FLINK-33934: -- I think it's easy to let data lost when using {{raw

[jira] [Updated] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-07 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan updated FLINK-35040: Description: The performance of serializerHeavyString regresses since April 3, and had not yet recovered

[jira] [Updated] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-07 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan updated FLINK-35040: Attachment: screenshot-1.png > The performance of serializerHeavyString regresses since April 3 >

Re: [PR] [FLINK-34487][ci] Adds Python Wheels nightly GHA workflow [flink]

2024-04-07 Thread via GitHub
morazow commented on PR #24426: URL: https://github.com/apache/flink/pull/24426#issuecomment-2041795538 Thanks @XComp, I am going to address your suggestions. Please have a look once you are back -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] [FLINK-34487][ci] Adds Python Wheels nightly GHA workflow [flink]

2024-04-07 Thread via GitHub
morazow commented on code in PR #24426: URL: https://github.com/apache/flink/pull/24426#discussion_r1555171619 ## .github/workflows/template.python-wheels-ci.yml: ## @@ -0,0 +1,67 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agr

Re: [PR] [FLINK-34955] Upgrade commons-compress to 1.26.0. [flink]

2024-04-07 Thread via GitHub
Jiabao-Sun commented on code in PR #24580: URL: https://github.com/apache/flink/pull/24580#discussion_r1555174874 ## flink-end-to-end-tests/flink-sql-client-test/pom.xml: ## @@ -69,6 +69,13 @@ under the License. kafka test

[jira] [Created] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-07 Thread Rui Fan (Jira)
Rui Fan created FLINK-35040: --- Summary: The performance of serializerHeavyString regresses since April 3 Key: FLINK-35040 URL: https://issues.apache.org/jira/browse/FLINK-35040 Project: Flink Issue

Re: [PR] [FLINK-34487][ci] Adds Python Wheels nightly GHA workflow [flink]

2024-04-07 Thread via GitHub
morazow commented on code in PR #24426: URL: https://github.com/apache/flink/pull/24426#discussion_r1555166750 ## .github/workflows/template.python-wheels-ci.yml: ## @@ -0,0 +1,67 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agr

Re: [PR] [FLINK-34634]Fix that Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed [flink-cdc]

2024-04-07 Thread via GitHub
loserwang1024 commented on code in PR #3134: URL: https://github.com/apache/flink-cdc/pull/3134#discussion_r1555165225 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/enumerator/MySqlSourceEnumerator.j

Re: [PR] [FLINK-34634]Fix that Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed [flink-cdc]

2024-04-07 Thread via GitHub
loserwang1024 commented on code in PR #3134: URL: https://github.com/apache/flink-cdc/pull/3134#discussion_r1555164983 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/events/BinlogSplitMetaEvent.java:

[jira] [Commented] (FLINK-35039) Create Profiling JobManager/TaskManager Instance failed

2024-04-07 Thread JJJJude (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834758#comment-17834758 ] ude commented on FLINK-35039: - [~Yu Chen] Please have a look, If my modification pla

[jira] [Created] (FLINK-35039) Create Profiling JobManager/TaskManager Instance failed

2024-04-07 Thread JJJJude (Jira)
ude created FLINK-35039: --- Summary: Create Profiling JobManager/TaskManager Instance failed Key: FLINK-35039 URL: https://issues.apache.org/jira/browse/FLINK-35039 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-34966) Support to read snapshot by table partitions in MySQL CDC Source

2024-04-07 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834756#comment-17834756 ] Qingsheng Ren commented on FLINK-34966: --- [~wanghe] Could you provide some context

[jira] [Assigned] (FLINK-34661) TaskExecutor supports retain partitions after JM crashed.

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-34661: --- Assignee: Junrui Li > TaskExecutor supports retain partitions after JM crashed. > -

[jira] [Assigned] (FLINK-33983) Introduce JobEvent and JobEventStore for Batch Job Recovery

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-33983: --- Assignee: Junrui Li > Introduce JobEvent and JobEventStore for Batch Job Recovery > ---

[jira] [Assigned] (FLINK-33986) Extend shuffleMaster to support batch snapshot.

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-33986: --- Assignee: Junrui Li > Extend shuffleMaster to support batch snapshot. > ---

[jira] [Closed] (FLINK-33984) Introduce SupportsBatchSnapshot for operator coordinator

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-33984. --- Fix Version/s: 1.20.0 Resolution: Done master: 38255652406becbfbcb7cbec557aa5ba9a1ebbb3 558ca75da2fce

[jira] [Commented] (FLINK-35036) Flink CDC Job cancel with savepoint failed

2024-04-07 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834755#comment-17834755 ] Qingsheng Ren commented on FLINK-35036: --- [~fly365] Thanks for reporting the issue!

[jira] [Assigned] (FLINK-34945) Support recover shuffle descriptor and partition metrics from tiered storage

2024-04-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-34945: --- Assignee: Junrui Li > Support recover shuffle descriptor and partition metrics from tiered storage

[jira] [Commented] (FLINK-35035) Reduce job pause time when cluster resources are expanded in adaptive mode

2024-04-07 Thread Biao Geng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834752#comment-17834752 ] Biao Geng commented on FLINK-35035: --- I am not very familiar with adaptive scheduler, m

Re: [PR] [FLINK-33984][runtime] Support batch snapshot for OperatorCoordinator. [flink]

2024-04-07 Thread via GitHub
zhuzhurk closed pull request #24415: [FLINK-33984][runtime] Support batch snapshot for OperatorCoordinator. URL: https://github.com/apache/flink/pull/24415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (FLINK-35009) Change on getTransitivePredecessors breaks connectors

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834751#comment-17834751 ] Weijie Guo commented on FLINK-35009: This change can be done in a compatible way. Bu

[jira] [Comment Edited] (FLINK-35009) Change on getTransitivePredecessors breaks connectors

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834751#comment-17834751 ] Weijie Guo edited comment on FLINK-35009 at 4/8/24 2:13 AM:

[jira] [Commented] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Zhongqiang Gong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834750#comment-17834750 ] Zhongqiang Gong commented on FLINK-34955: - Hi [~slfan1989]  ,I apologize for the

[jira] [Commented] (FLINK-35036) Flink CDC Job cancel with savepoint failed

2024-04-07 Thread Biao Geng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834749#comment-17834749 ] Biao Geng commented on FLINK-35036: --- Hi [~fly365], according to the attached screensho

[jira] [Closed] (FLINK-34573) the task is stuck on the high presure

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-34573. -- Resolution: Won't Fix > the task is stuck on the high presure > -

Re: [PR] [hotfix] Correct the option key for sortPartition's java doc [flink]

2024-04-07 Thread via GitHub
reswqa merged PR #24627: URL: https://github.com/apache/flink/pull/24627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

[jira] [Closed] (FLINK-35006) Use try with-resource for StandaloneAutoscalerExecutor

2024-04-07 Thread Kirill Plugatarev (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Plugatarev closed FLINK-35006. - Resolution: Not A Problem > Use try with-resource for StandaloneAutoscalerExecutor > ---

Re: [PR] [FLINK-35006] Use try with-resource for StandaloneAutoscalerExecutor [flink-kubernetes-operator]

2024-04-07 Thread via GitHub
plugatarev closed pull request #811: [FLINK-35006] Use try with-resource for StandaloneAutoscalerExecutor URL: https://github.com/apache/flink-kubernetes-operator/pull/811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] [FLINK-35006] Use try with-resource for StandaloneAutoscalerExecutor [flink-kubernetes-operator]

2024-04-07 Thread via GitHub
plugatarev commented on code in PR #811: URL: https://github.com/apache/flink-kubernetes-operator/pull/811#discussion_r1555135563 ## flink-autoscaler-standalone/src/main/java/org/apache/flink/autoscaler/standalone/StandaloneAutoscalerEntrypoint.java: ## @@ -60,9 +60,11 @@ publi

[jira] [Created] (FLINK-35038) Bump test dependency org.yaml:snakeyaml to 2.2

2024-04-07 Thread Ufuk Celebi (Jira)
Ufuk Celebi created FLINK-35038: --- Summary: Bump test dependency org.yaml:snakeyaml to 2.2 Key: FLINK-35038 URL: https://issues.apache.org/jira/browse/FLINK-35038 Project: Flink Issue Type: Tec

Re: [PR] [FLINK-34915][table] Complete `DESCRIBE CATALOG` syntax [flink]

2024-04-07 Thread via GitHub
liyubin117 commented on PR #24630: URL: https://github.com/apache/flink/pull/24630#issuecomment-2041527525 @LadyForest Hi, CI passed now, Looking forward your review, Thanks very much :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (FLINK-34898) Cannot create ARRAY of named STRUCTs

2024-04-07 Thread Feng Jin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834685#comment-17834685 ] Feng Jin commented on FLINK-34898: -- [~chloehe]   Thank you for the update, sorry for th

[jira] [Comment Edited] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834682#comment-17834682 ] Shilun Fan edited comment on FLINK-34955 at 4/7/24 2:17 PM:

[jira] [Updated] (FLINK-34898) Cannot create ARRAY of named STRUCTs

2024-04-07 Thread Feng Jin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Jin updated FLINK-34898: - Attachment: 截屏2024-04-07 22.05.40.png > Cannot create ARRAY of named STRUCTs > -

[jira] [Commented] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834682#comment-17834682 ] Shilun Fan commented on FLINK-34955: [~gongzhongqiang] Of course, if upgrading is po

[jira] [Commented] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834681#comment-17834681 ] Shilun Fan commented on FLINK-34955: [~gongzhongqiang] From my personal perspective,

Re: [PR] [FLINK-35010][connectors/mongodb] Bump org.apache.commons:commons-compress from 1.24.0 to 1.26.0 for Flink Mongodb connector [flink-connector-mongodb]

2024-04-07 Thread via GitHub
GOODBOY008 commented on PR #32: URL: https://github.com/apache/flink-connector-mongodb/pull/32#issuecomment-2041478551 @Jiabao-Sun PR updated and version to `1.26.1`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[jira] [Commented] (FLINK-34955) Upgrade commons-compress to 1.26.0

2024-04-07 Thread Zhongqiang Gong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834678#comment-17834678 ] Zhongqiang Gong commented on FLINK-34955: - [~slfan1989] [~mbalassi] According to

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
Zakelly commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1554965456 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-34986][Runtime/State] Basic framework of async execution for state [flink]

2024-04-07 Thread via GitHub
ljz2051 commented on code in PR #24614: URL: https://github.com/apache/flink/pull/24614#discussion_r1554960979 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequest.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Updated] (FLINK-35037) Optimize uniqueKeys and upsertKeys inference of windows with ROW_NUMBER

2024-04-07 Thread yisha zhou (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yisha zhou updated FLINK-35037: --- Description: In current Implementation, relNodes with Window type will only deliver upsert/unique k

[jira] [Updated] (FLINK-35037) Optimize uniqueKeys and upsertKeys inference of windows with ROW_NUMBER

2024-04-07 Thread yisha zhou (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yisha zhou updated FLINK-35037: --- Description: In current Implementation, relNodes with Window type will only deliver upsert/unique k

[jira] [Created] (FLINK-35037) Optimize uniqueKeys and upsertKeys inference of windows with ROW_NUMBER

2024-04-07 Thread yisha zhou (Jira)
yisha zhou created FLINK-35037: -- Summary: Optimize uniqueKeys and upsertKeys inference of windows with ROW_NUMBER Key: FLINK-35037 URL: https://issues.apache.org/jira/browse/FLINK-35037 Project: Flink

Re: [PR] [FLINK-32440][checkpoint] Introduce file merging configurations [flink]

2024-04-07 Thread via GitHub
fredia commented on PR #22973: URL: https://github.com/apache/flink/pull/22973#issuecomment-2041450107 @Zakelly I have rebased this PR, could you please take a look if you're free? thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [FLINK-34969][cdc-cli]Add support for both new and old Flink config files in Flink… [flink-cdc]

2024-04-07 Thread via GitHub
PatrickRen commented on code in PR #3194: URL: https://github.com/apache/flink-cdc/pull/3194#discussion_r1554892220 ## flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/utils/FlinkEnvironmentUtils.java: ## @@ -20,18 +20,33 @@ import org.apache.flink.cdc.common.configuration.

Re: [PR] [FLINK-34915][table] Complete `DESCRIBE CATALOG` syntax [flink]

2024-04-07 Thread via GitHub
liyubin117 commented on PR #24630: URL: https://github.com/apache/flink/pull/24630#issuecomment-2041411609 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [hotfix][doc] Generate and use docs of RpcOptions instead of AkkaOptions [flink]

2024-04-07 Thread via GitHub
Zakelly commented on PR #24629: URL: https://github.com/apache/flink/pull/24629#issuecomment-2041409660 @XComp Would you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [FLINK-34908][pipeline-connector][doris] Fix Mysql pipeline to doris will lost precision for timestamp [flink-cdc]

2024-04-07 Thread via GitHub
gong commented on PR #3207: URL: https://github.com/apache/flink-cdc/pull/3207#issuecomment-2041406982 > I wonder if changing `DorisEventSerializer.DATE_TIME_FORMATTER` to `-MM-dd HH:mm:ss.SS` fixes this problem, too? @yuxiqian I think that it can fix this problem, too. --

[jira] [Updated] (FLINK-34915) Complete `DESCRIBE CATALOG` syntax

2024-04-07 Thread Yubin Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yubin Li updated FLINK-34915: - Attachment: image-2024-04-07-17-54-51-203.png Description: Describe the metadata of an existing cat

Re: [PR] [FLINK-34915][table] Complete `DESCRIBE CATALOG` syntax [flink]

2024-04-07 Thread via GitHub
flinkbot commented on PR #24630: URL: https://github.com/apache/flink/pull/24630#issuecomment-2041394595 ## CI report: * a1146244ce8a490c2ca1557f4c2410b59effb333 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-34915) Complete `DESCRIBE CATALOG` syntax

2024-04-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34915: --- Labels: pull-request-available (was: ) > Complete `DESCRIBE CATALOG` syntax > -

[PR] [FLINK-34915][table] Complete `DESCRIBE CATALOG` syntax [flink]

2024-04-07 Thread via GitHub
liyubin117 opened a new pull request, #24630: URL: https://github.com/apache/flink/pull/24630 ## What is the purpose of the change Describe the metadata of an existing catalog. The metadata information includes the catalog’s name, type, and comment. If the optional EXTENDED option is

Re: [PR] [FLINK-34952][cdc-composer][sink] Flink CDC pipeline supports SinkFunction [flink-cdc]

2024-04-07 Thread via GitHub
loserwang1024 commented on code in PR #3204: URL: https://github.com/apache/flink-cdc/pull/3204#discussion_r1554863395 ## flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-values/src/main/java/org/apache/flink/cdc/connectors/values/sink/ValuesDataSinkOptions.j

Re: [PR] [hotfix] Fix YARN ContainerId.getId Deprecated Used. [flink]

2024-04-07 Thread via GitHub
slfan1989 commented on PR #24601: URL: https://github.com/apache/flink/pull/24601#issuecomment-2041389025 @1996fanrui Thank you very much for reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Created] (FLINK-35036) Flink CDC Job cancel with savepoint failed

2024-04-07 Thread Fly365 (Jira)
Fly365 created FLINK-35036: -- Summary: Flink CDC Job cancel with savepoint failed Key: FLINK-35036 URL: https://issues.apache.org/jira/browse/FLINK-35036 Project: Flink Issue Type: Bug Comp

Re: [PR] [cdc-connector][cdc-base] Shade guava31 to avoid dependency conflict with flink below 1.18 [flink-cdc]

2024-04-07 Thread via GitHub
PatrickRen commented on code in PR #3083: URL: https://github.com/apache/flink-cdc/pull/3083#discussion_r1554857536 ## pom.xml: ## @@ -462,8 +462,15 @@ under the License. submodules, ${flink.version} will be resolved as the actual Flink

[jira] [Comment Edited] (FLINK-35023) YARNApplicationITCase failed on Azure

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834639#comment-17834639 ] Weijie Guo edited comment on FLINK-35023 at 4/7/24 9:24 AM:

[jira] [Commented] (FLINK-35023) YARNApplicationITCase failed on Azure

2024-04-07 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834639#comment-17834639 ] Weijie Guo commented on FLINK-35023: https://dev.azure.com/apache-flink/apache-flink

  1   2   >