Re: [PR] [FLINK-27355][runtime] Unregister JobManagerRunner after it's closed [flink]

2024-09-23 Thread via GitHub
kumar-mallikarjuna commented on code in PR #25027: URL: https://github.com/apache/flink/pull/25027#discussion_r1771279162 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/OnMainThreadJobManagerRunnerRegistry.java: ## @@ -84,9 +84,10 @@ public Collection getJobM

Re: [PR] [FLINK-27355][runtime] Unregister JobManagerRunner after it's closed [flink]

2024-09-23 Thread via GitHub
kumar-mallikarjuna commented on code in PR #25027: URL: https://github.com/apache/flink/pull/25027#discussion_r1771276366 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/cleanup/LocallyCleanableInMainThreadResource.java: ## @@ -0,0 +1,52 @@ +/* + * Licensed to

Re: [PR] [FLINK-27355][runtime] Unregister JobManagerRunner after it's closed [flink]

2024-09-23 Thread via GitHub
kumar-mallikarjuna commented on code in PR #25027: URL: https://github.com/apache/flink/pull/25027#discussion_r1771262937 ## flink-runtime/src/test/java/org/apache/flink/runtime/dispatcher/cleanup/TestingResourceCleanerFactory.java: ## @@ -26,27 +26,42 @@ import java.util.Array

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-23 Thread via GitHub
XComp commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1771262343 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/StateLocalitySlotAssigner.java: ## @@ -139,6 +154,36 @@ public Collection assignSlots

Re: [PR] [FLINK-36351][pipeline-connector/doris] support the conversion of Flink TIME type to Doris String type [flink-cdc]

2024-09-23 Thread via GitHub
vinlee19 closed pull request #3618: [FLINK-36351][pipeline-connector/doris] support the conversion of Flink TIME type to Doris String type URL: https://github.com/apache/flink-cdc/pull/3618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[PR] [FLINK-36315][cdc-base]The flink-cdc-base module supports source metric statistics [flink-cdc]

2024-09-23 Thread via GitHub
liuxiao2shf opened a new pull request, #3619: URL: https://github.com/apache/flink-cdc/pull/3619 The MySQL source already supports embedding observability metrics, but this feature cannot be reused by other source types. Therefore, we hope to port this feature to the base for easy reuse by

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-23 Thread via GitHub
1996fanrui commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1771234937 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/StateLocalitySlotAssigner.java: ## @@ -139,6 +154,36 @@ public Collection assign

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-23 Thread via GitHub
Zakelly merged PR #25367: URL: https://github.com/apache/flink/pull/25367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-23 Thread via GitHub
1996fanrui commented on PR #25218: URL: https://github.com/apache/flink/pull/25218#issuecomment-2367911956 Hi @XComp , thank you for the quick review and feedback. > I'm not sure about adding this feature to the StateLocalitySlotAssigner. Shouldn't state locality have priority over t

Re: [PR] [FLINK-36295] [runtime] Test `testCheckpointStatsPersistedAcrossRescale` waits until the desired parallelism is reached [flink]

2024-09-23 Thread via GitHub
XComp merged PR #25375: URL: https://github.com/apache/flink/pull/25375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-23 Thread via GitHub
XComp commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770926488 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssigner.java: ## @@ -32,4 +46,77 @@ Collection assignSlots( Collect

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-23 Thread via GitHub
XComp commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770953081 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/DefaultSlotAssigner.java: ## @@ -43,35 +47,43 @@ public Collection assignSlots(

Re: [PR] [FLINK-36287] Disallow UC for inner sink channels [flink]

2024-09-23 Thread via GitHub
fapaul commented on code in PR #25353: URL: https://github.com/apache/flink/pull/25353#discussion_r1771171220 ## flink-runtime/src/main/java/org/apache/flink/streaming/runtime/operators/sink/committables/CommittableCollector.java: ## @@ -254,4 +249,9 @@ private CheckpointCommitt

Re: [PR] [FLINK-36351][pipeline-connector/doris] Add the conversion from Flink TIME type to Doris [flink-cdc]

2024-09-23 Thread via GitHub
vinlee19 commented on PR #3618: URL: https://github.com/apache/flink-cdc/pull/3618#issuecomment-2367783473 @JNSimba @yuxiqian PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [hotfix] Fix UnifiedSinkMigrationITCase [flink]

2024-09-23 Thread via GitHub
AHeise merged PR #25368: URL: https://github.com/apache/flink/pull/25368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-3992][core] Remove deprecated Key interface [flink]

2024-09-23 Thread via GitHub
flinkbot commented on PR #25376: URL: https://github.com/apache/flink/pull/25376#issuecomment-236730 ## CI report: * 05260cc235106b120ea18caab4993a19f03d68e8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-28770][table-planner] CREATE TABLE AS SELECT supports explain [flink]

2024-09-23 Thread via GitHub
twalthr merged PR #25359: URL: https://github.com/apache/flink/pull/25359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

[PR] [FLINK-3992][core] Remove deprecated Key interface [flink]

2024-09-23 Thread via GitHub
reswqa opened a new pull request, #25376: URL: https://github.com/apache/flink/pull/25376 ## What is the purpose of the change *Remove deprecated Key interface* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Do

Re: [PR] [FLINK-36335][runtime] Improving Method Reusability in StreamGraphGenerator with JobVertexBuildContext [flink]

2024-09-23 Thread via GitHub
JunRuiLee commented on code in PR #25366: URL: https://github.com/apache/flink/pull/25366#discussion_r1770890500 ## flink-runtime/src/main/java/org/apache/flink/streaming/api/graph/StreamingJobGraphGenerator.java: ## @@ -517,7 +537,8 @@ private void preValidate() {

Re: [PR] [hotfix] Fix UnifiedSinkMigrationITCase [flink]

2024-09-23 Thread via GitHub
fapaul commented on code in PR #25368: URL: https://github.com/apache/flink/pull/25368#discussion_r1770987987 ## flink-tests/src/test/java/org/apache/flink/test/streaming/runtime/UnifiedSinkMigrationITCase.java: ## @@ -265,16 +266,24 @@ private static class TestCommitter impleme

Re: [PR] [FLINK-35282][FLINK-35520] PyFlink Support for Apache Beam > 2.49 [flink]

2024-09-23 Thread via GitHub
snuyanzin commented on PR #24908: URL: https://github.com/apache/flink/pull/24908#issuecomment-2367556535 I was here since I thought the only thing which should be fixed here is license related staff However it seems some more things should be done currently I don't have time for that

Re: [PR] [FLINK-36273][table] Remove deprecated Table/SQL configuration in 2.0 [flink]

2024-09-23 Thread via GitHub
xuyangzhong commented on PR #25334: URL: https://github.com/apache/flink/pull/25334#issuecomment-2367514032 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [FLINK-36295] [runtime] Test `testCheckpointStatsPersistedAcrossRescale` waits until the desired parallelism is reached [flink]

2024-09-23 Thread via GitHub
flinkbot commented on PR #25375: URL: https://github.com/apache/flink/pull/25375#issuecomment-2367419327 ## CI report: * 7c27a13fed82288fe511af00badeb9bb1e4ffc96 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] Flink 36295 [flink]

2024-09-23 Thread via GitHub
ztison opened a new pull request, #25375: URL: https://github.com/apache/flink/pull/25375 ## What is the purpose of the change Flaky test fix and additional logging in Adaptive Scheduler. ## Verifying this change This change is a trivial rework / code cleanup without any

Re: [PR] [FLINK-36175][python] Remove `register_table_source` and `register_table_sink` in python module [flink]

2024-09-23 Thread via GitHub
HuangXingBo merged PR #25266: URL: https://github.com/apache/flink/pull/25266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

Re: [PR] [FLINK-36175][python] Remove `register_table_source` and `register_table_sink` in python module [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on PR #25266: URL: https://github.com/apache/flink/pull/25266#issuecomment-2367368865 The failed test is traced by https://issues.apache.org/jira/browse/FLINK-36293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [FLINK-36188] Fix disable buffer flush lose efficacy [flink-connector-hbase]

2024-09-22 Thread via GitHub
MOBIN-F commented on PR #49: URL: https://github.com/apache/flink-connector-hbase/pull/49#issuecomment-2367334732 @MartijnVisser can you help to trigger a CI run, thanks~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
XComp commented on PR #25218: URL: https://github.com/apache/flink/pull/25218#issuecomment-2367315487 Sure, I can take a pass over it today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [FLINK-36251][streaming-java] Remove StateBackend-related configuration getters/setters that return/set complex Java objects. [flink]

2024-09-22 Thread via GitHub
JunRuiLee commented on PR #25352: URL: https://github.com/apache/flink/pull/25352#issuecomment-2367290298 Thanks @zhuzhurk , I've updated this pr. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [FLINK-36257][API] Remove easy-to-drop deprecated API [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25347: URL: https://github.com/apache/flink/pull/25347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-36257][API] Remove easy-to-drop deprecated API [flink]

2024-09-22 Thread via GitHub
reswqa commented on PR #25347: URL: https://github.com/apache/flink/pull/25347#issuecomment-2367285342 Thanks for the review, merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] [FLINK-36251][streaming-java] Remove StateBackend-related configuration getters/setters that return/set complex Java objects. [flink]

2024-09-22 Thread via GitHub
zhuzhurk commented on code in PR #25352: URL: https://github.com/apache/flink/pull/25352#discussion_r1768319833 ## flink-state-backends/flink-statebackend-changelog/src/test/java/org/apache/flink/state/changelog/ChangelogStateBackendLoadingTest.java: ## @@ -219,29 +218,14 @@ pub

Re: [PR] [FLINK-33678][streaming-java] Remove CheckpointStorage related public API. [flink]

2024-09-22 Thread via GitHub
JunRuiLee closed pull request #25264: [FLINK-33678][streaming-java] Remove CheckpointStorage related public API. URL: https://github.com/apache/flink/pull/25264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [FLINK-36273][table] Remove deprecated Table/SQL configuration in 2.0 [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on PR #25334: URL: https://github.com/apache/flink/pull/25334#issuecomment-2367274893 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
RocMarshal commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770790841 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770772924 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractKeyedStateBackend.java: ## @@ -250,6 +250,13 @@ public void setCurrentKey(K newKey) {

Re: [PR] [FLINK-35165][runtime/coordination] AdaptiveBatch Scheduler should not restrict the default source parall… [flink]

2024-09-22 Thread via GitHub
venkata91 commented on PR #24736: URL: https://github.com/apache/flink/pull/24736#issuecomment-2367214026 To illustrate with an example test case, try changing the `executeJob` and `createConfiguration` with below code and run `testScheduling`: ``` private void executeJob(Boolean u

Re: [PR] [Flink 36245] Remove legacy SourceFunction / SinkFunction / Sink V1 API in 2.0 [flink]

2024-09-22 Thread via GitHub
lvyanquan commented on PR #25331: URL: https://github.com/apache/flink/pull/25331#issuecomment-2367194793 Hi @PatrickRen, could you please help to review this as I'm not really familiar with the API norms. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] [FLINK-35165][runtime/coordination] AdaptiveBatch Scheduler should not restrict the default source parall… [flink]

2024-09-22 Thread via GitHub
venkata91 commented on PR #24736: URL: https://github.com/apache/flink/pull/24736#issuecomment-2367194478 @JunRuiLee First of all, apologies for very late reply! I think the issue is, if source parallelism is not explicitly set and if `execution.batch.adaptive.auto-parallelism.max-par

Re: [PR] [FLINK-36321][state/forst] Execute read/write state request in different executors [flink]

2024-09-22 Thread via GitHub
fredia merged PR #25360: URL: https://github.com/apache/flink/pull/25360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770746753 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object r

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770742198 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object r

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
fredia commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770726474 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object re

Re: [PR] [cdc-connector][cdc-base] Add SNAPSHOT mode [flink-cdc]

2024-09-22 Thread via GitHub
loserwang1024 commented on PR #2901: URL: https://github.com/apache/flink-cdc/pull/2901#issuecomment-2367145529 > Not supporting BATCH execution mode hi, @LinMingQiang, still not support execution.runtime-mode=BATCH now, though snapshot mode will read in batch. Because we still don't su

Re: [PR] [FLINK-36335][runtime] Improving Method Reusability in StreamGraphGenerator with JobVertexBuildContext [flink]

2024-09-22 Thread via GitHub
JunRuiLee commented on code in PR #25366: URL: https://github.com/apache/flink/pull/25366#discussion_r1768349683 ## flink-runtime/src/main/java/org/apache/flink/streaming/api/graph/util/ChainedOperatorHashInfo.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
1996fanrui commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770708552 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
1996fanrui commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770708552 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-36248][table] Introduce new Join Operator with Async State API [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on code in PR #25320: URL: https://github.com/apache/flink/pull/25320#discussion_r1770706342 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/JoinITCase.scala: ## @@ -99,6 +106,33 @@ class JoinITCase(miniBa

Re: [PR] [FLINK-36248][table] Introduce new Join Operator with Async State API [flink]

2024-09-22 Thread via GitHub
xishuaidelin commented on code in PR #25320: URL: https://github.com/apache/flink/pull/25320#discussion_r1770576947 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/JoinITCase.scala: ## @@ -99,6 +106,33 @@ class JoinITCase(miniB

Re: [PR] [FLINK-35780] Support state migration between disabling and enabling State TTL for RocksDBState [flink]

2024-09-22 Thread via GitHub
xiangyuf commented on PR #25035: URL: https://github.com/apache/flink/pull/25035#issuecomment-2366757734 @Zakelly Currently, the serializer's compatibility check assumes the new serializer and previous serializer should have the same datatype. This does not apply to compatibility check betw

[PR] [FLINK-35874][cdc-connector][mysql] Obtain the high watermark offset and table data in the same transaction during the Snapshot phase [flink-cdc]

2024-09-22 Thread via GitHub
qiaozongmi opened a new pull request, #3616: URL: https://github.com/apache/flink-cdc/pull/3616 Obtain the high watermark offset and table data in the same transaction during the Snapshot phase,This ensures that the data retrieved is consistent and avoids any potential data loss. -- This

Re: [PR] [FLINK-36346][API] Remove deprecated API in flink-streaming-java module [flink]

2024-09-22 Thread via GitHub
flinkbot commented on PR #25374: URL: https://github.com/apache/flink/pull/25374#issuecomment-2366135001 ## CI report: * 530bf52976a08ad3b847e6c1edd2b0304e329090 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-36346][API] Remove deprecated API in flink-streaming-java module [flink]

2024-09-22 Thread via GitHub
reswqa opened a new pull request, #25374: URL: https://github.com/apache/flink/pull/25374 ## What is the purpose of the change *Remove deprecated API in flink-streaming-java module* ## Brief change log - *Remove deprecated API in flink-streaming-java module*

Re: [PR] [FLINK-36310] Remove per-job and run-application [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25348: URL: https://github.com/apache/flink/pull/25348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-29741] Remove flink-scala and flink-streaming-scala dependency exclude table module [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25324: URL: https://github.com/apache/flink/pull/25324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-36292][Connectors/Common]: remove timeout to avoid timeout exception [flink]

2024-09-22 Thread via GitHub
showuon commented on PR #25371: URL: https://github.com/apache/flink/pull/25371#issuecomment-2365680553 @becketqin , please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [FLINK-36293][state] make sure register is closed before verification in testAsyncCancellation [flink]

2024-09-21 Thread via GitHub
flinkbot commented on PR #25373: URL: https://github.com/apache/flink/pull/25373#issuecomment-2365568081 ## CI report: * 9dd04b94f241c1f04b9737424cad93aea81ce0c8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36293][state] make sure register is closed before verification in testAsyncCancellation [flink]

2024-09-21 Thread via GitHub
showuon commented on PR #25373: URL: https://github.com/apache/flink/pull/25373#issuecomment-2365559457 @StefanRRichter @rkhachatryan , please take a look when available. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] [FLINK-36293][state] make sure register is closed before verification in testAsyncCancellation [flink]

2024-09-21 Thread via GitHub
showuon opened a new pull request, #25373: URL: https://github.com/apache/flink/pull/25373 ## What is the purpose of the change Saw this flaky `testAsyncCancellation` test failed with ``` org.apache.flink.contrib.streaming.state.RocksDBWriteBatchWrapperTest.testAsyncCanc

Re: [PR] [FLINK-34702][table] Do not deduplicate GroupAggregate and some joins for rank [flink]

2024-09-21 Thread via GitHub
snuyanzin commented on PR #25372: URL: https://github.com/apache/flink/pull/25372#issuecomment-2365480047 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [cdc-runtime]SchemaRegistry Log output is repeated [flink-cdc]

2024-09-21 Thread via GitHub
github-actions[bot] commented on PR #3487: URL: https://github.com/apache/flink-cdc/pull/3487#issuecomment-2365369297 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [FLINK-34702][table] Do not deduplicate GroupAggregate and some joins for rank [flink]

2024-09-21 Thread via GitHub
flinkbot commented on PR #25372: URL: https://github.com/apache/flink/pull/25372#issuecomment-2365367123 ## CI report: * 0bcb9216feab685ea4b7087c82f4c40f2466c86d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-34702][table] Do not deduplicate GroupAggregate and some joins for rank [flink]

2024-09-21 Thread via GitHub
snuyanzin opened a new pull request, #25372: URL: https://github.com/apache/flink/pull/25372 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them

Re: [PR] [FLINK-36319][Connector/Prometheus] Fixed error handling behavior on non-retriable errors [flink-connector-prometheus]

2024-09-21 Thread via GitHub
nicusX commented on PR #7: URL: https://github.com/apache/flink-connector-prometheus/pull/7#issuecomment-2365215355 Fixed two additional bugs detected while testing: 1. Sink builder throwing an exception if maxBatchSizeInSamples or retryConfiguration were not explicitly configured 2.

Re: [PR] [FLINK-36345][feature][cdc-connector][oracle] Oracle cdc support partition table [flink-cdc]

2024-09-21 Thread via GitHub
zhuxt2015 commented on PR #3593: URL: https://github.com/apache/flink-cdc/pull/3593#issuecomment-2365169738 > Thanks for @zhuxt2015's contribution! Would you mind adding a test case to verify this change? 我参考其他测试用例写的oracle分区表的测试用例,但是运行会报错,一直没找到解决办法,您能帮忙看一下吗? I refer to the test cas

Re: [PR] [FLINK-36292][Connectors/Common]: remote timeout to avoid timeout exception [flink]

2024-09-21 Thread via GitHub
flinkbot commented on PR #25371: URL: https://github.com/apache/flink/pull/25371#issuecomment-2365160222 ## CI report: * 99828512ccc35f7c04dc75ad93a1092f38286bfc UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-36292][Connectors/Common]: remote timeout to avoid timeout exception [flink]

2024-09-21 Thread via GitHub
showuon opened a new pull request, #25371: URL: https://github.com/apache/flink/pull/25371 ## What is the purpose of the change Removed timeouts setting in `testCloseCleansUpPreviouslyClosedFetcher` tests. If a test runs into a deadlock, Flink's CI would produce a thread d

Re: [PR] [minor][cdc-runtime]The processedTableIds add is executed twice [flink-cdc]

2024-09-20 Thread via GitHub
github-actions[bot] commented on PR #3483: URL: https://github.com/apache/flink-cdc/pull/3483#issuecomment-2364767272 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [minor][cdc-connector][mysql] catch all exceptions to display the table info [flink-cdc]

2024-09-20 Thread via GitHub
github-actions[bot] commented on PR #3485: URL: https://github.com/apache/flink-cdc/pull/3485#issuecomment-2364767260 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [master][FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
snuyanzin merged PR #25358: URL: https://github.com/apache/flink/pull/25358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [master][FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
snuyanzin commented on PR #25358: URL: https://github.com/apache/flink/pull/25358#issuecomment-2364397774 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-36330] Session Window TVFs with named parameters don't support column expansion [flink]

2024-09-20 Thread via GitHub
snuyanzin merged PR #25362: URL: https://github.com/apache/flink/pull/25362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [1.20][FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
snuyanzin merged PR #25369: URL: https://github.com/apache/flink/pull/25369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [FLINK-36342][runtime] Rename misleading variable name 'targetTotalBu… [flink]

2024-09-20 Thread via GitHub
flinkbot commented on PR #25370: URL: https://github.com/apache/flink/pull/25370#issuecomment-2364000848 ## CI report: * da167a5cddf4469e18a2eff2100b2ed73180176b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-36342][runtime] Rename misleading variable name 'targetTotalBu… [flink]

2024-09-20 Thread via GitHub
XuHao41 opened a new pull request, #25370: URL: https://github.com/apache/flink/pull/25370 ## What is the purpose of the change Incorrect variable naming can cause confusion for readers of flink source code. Given that this variable is involved in the computation within the Bu

Re: [PR] [FLINK-28770][table-planner] CREATE TABLE AS SELECT supports explain [flink]

2024-09-20 Thread via GitHub
spena commented on code in PR #25359: URL: https://github.com/apache/flink/pull/25359#discussion_r1768799456 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/connectors/DynamicSinkUtils.java: ## @@ -209,6 +213,78 @@ public static RelNode convertSin

Re: [PR] [FLINK-28770][table-planner] CREATE TABLE AS SELECT supports explain [flink]

2024-09-20 Thread via GitHub
spena commented on code in PR #25359: URL: https://github.com/apache/flink/pull/25359#discussion_r1768799089 ## flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/utils/ParserResource.java: ## @@ -41,6 +41,10 @@ public interface ParserResource { @Resourc

Re: [PR] [1.20][FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
flinkbot commented on PR #25369: URL: https://github.com/apache/flink/pull/25369#issuecomment-2363956380 ## CI report: * 93b2e0c0fe437940921a972056625e709633107a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
dawidwys closed pull request #25357: [FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types URL: https://github.com/apache/flink/pull/25357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [FLINK-36276] Fix the description of MySqlDataSourceOptions.`scan.sta… [flink-cdc]

2024-09-20 Thread via GitHub
yuxiqian commented on PR #3606: URL: https://github.com/apache/flink-cdc/pull/3606#issuecomment-2363847956 Could @ruanhang1993 please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [hotfix] Remove UnifiedSinkMigrationITCase [flink]

2024-09-20 Thread via GitHub
flinkbot commented on PR #25368: URL: https://github.com/apache/flink/pull/25368#issuecomment-2363846967 ## CI report: * c73bf9a1ac308e6c93dccf3f8ce59df4ee11985c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [hotfix] Remove UnifiedSinkMigrationITCase [flink]

2024-09-20 Thread via GitHub
AHeise opened a new pull request, #25368: URL: https://github.com/apache/flink/pull/25368 ## What is the purpose of the change UnifiedSinkMigrationITCase tests that SinkV1 can be read with Flink 1.14 SinkV2. Since we remove SinkV1 entirely from Flink 2.0, we can go ahe

Re: [PR] [FLINK-36276] Fix the description of MySqlDataSourceOptions.`scan.sta… [flink-cdc]

2024-09-20 Thread via GitHub
LinMingQiang commented on PR #3606: URL: https://github.com/apache/flink-cdc/pull/3606#issuecomment-2363810910 @yuxiqian Can you help me merge this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [FLINK-33750][config] Remove deprecated runtime config option [flink]

2024-09-20 Thread via GitHub
Sxnan merged PR #25355: URL: https://github.com/apache/flink/pull/25355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-20 Thread via GitHub
flinkbot commented on PR #25367: URL: https://github.com/apache/flink/pull/25367#issuecomment-2363459765 ## CI report: * 90aa9496b9954906ec1cce3c37d67a1ed4f682ea UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-20 Thread via GitHub
Zakelly opened a new pull request, #25367: URL: https://github.com/apache/flink/pull/25367 ## What is the purpose of the change After [FLINK-36117](https://issues.apache.org/jira/browse/FLINK-36117), we port old state backends implementation to new api using AsyncKeyedStateBackendAda

Re: [PR] [FLINK-28770][table-planner] CREATE TABLE AS SELECT supports explain [flink]

2024-09-20 Thread via GitHub
twalthr commented on code in PR #25359: URL: https://github.com/apache/flink/pull/25359#discussion_r1768388152 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/connectors/DynamicSinkUtils.java: ## @@ -209,6 +213,78 @@ public static RelNode convertS

Re: [PR] [FLINK-36330] Session Window TVFs with named parameters don't support column expansion [flink]

2024-09-20 Thread via GitHub
twalthr commented on code in PR #25362: URL: https://github.com/apache/flink/pull/25362#discussion_r1768379745 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/calcite/FlinkCalciteSqlValidator.java: ## @@ -373,20 +375,30 @@ protected void addToSele

Re: [PR] [FLINK-36330] Session Window TVFs with named parameters don't support column expansion [flink]

2024-09-20 Thread via GitHub
snuyanzin commented on code in PR #25362: URL: https://github.com/apache/flink/pull/25362#discussion_r1768373194 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/calcite/FlinkCalciteSqlValidator.java: ## @@ -373,20 +375,30 @@ protected void addToSe

Re: [PR] [FLINK-36284][format-avro] Support extracting Avro type with new DataType stack [flink]

2024-09-20 Thread via GitHub
lincoln-lil merged PR #25336: URL: https://github.com/apache/flink/pull/25336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

Re: [PR] [FLINK-36335][runtime] Improving Method Reusability in StreamGraphGenerator with JobVertexBuildContext [flink]

2024-09-20 Thread via GitHub
flinkbot commented on PR #25366: URL: https://github.com/apache/flink/pull/25366#issuecomment-2363372428 ## CI report: * 4d46ec3534f888d7ff54c9519acfba024de6f2ed UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36330] Session Window TVFs with named parameters don't support column expansion [flink]

2024-09-20 Thread via GitHub
snuyanzin commented on code in PR #25362: URL: https://github.com/apache/flink/pull/25362#discussion_r1768355815 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/calcite/FlinkCalciteSqlValidator.java: ## @@ -373,20 +375,30 @@ protected void addToSe

Re: [PR] [FLINK-36330] Session Window TVFs with named parameters don't support column expansion [flink]

2024-09-20 Thread via GitHub
snuyanzin commented on code in PR #25362: URL: https://github.com/apache/flink/pull/25362#discussion_r1768348432 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/calcite/FlinkCalciteSqlValidator.java: ## @@ -373,20 +375,30 @@ protected void addToSe

Re: [PR] [FLINK-36318] Fix deserializing from 1.18 savepoint with RAW types [flink]

2024-09-20 Thread via GitHub
dawidwys commented on code in PR #25357: URL: https://github.com/apache/flink/pull/25357#discussion_r1768343224 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/typeutils/RowDataSerializer.java: ## @@ -314,7 +314,8 @@ public void readSnapshot(int r

Re: [PR] Bump com.google.protobuf:protobuf-java from 3.21.7 to 3.25.5 [flink-connector-aws]

2024-09-20 Thread via GitHub
hlteoh37 commented on PR #169: URL: https://github.com/apache/flink-connector-aws/pull/169#issuecomment-2363341100 Merging as tests pass (title validator is not valid test for dependabot) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[PR] [FLINK-36335][runtime] Improving Method Reusability in StreamGraphGenerator with JobVertexBuildContext [flink]

2024-09-20 Thread via GitHub
noorall opened a new pull request, #25366: URL: https://github.com/apache/flink/pull/25366 ## What is the purpose of the change Refactor the StreamGraphGenerator by introducing the JobVertexBuildContext to make more methods reusable. ## Brief change log - In

Re: [PR] [FLINK-33750][config] Remove deprecated runtime config option [flink]

2024-09-20 Thread via GitHub
Sxnan commented on PR #25355: URL: https://github.com/apache/flink/pull/25355#issuecomment-2363353508 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [minor][cdc-connector][mongodb] Add `op_type` metadata column [flink-cdc]

2024-09-20 Thread via GitHub
Jiabao-Sun commented on code in PR #3615: URL: https://github.com/apache/flink-cdc/pull/3615#discussion_r1768333821 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mongodb-cdc/src/test/java/org/apache/flink/cdc/connectors/mongodb/source/MongoDBFullChangelogITCase.java:

Re: [PR] Bump com.google.protobuf:protobuf-java from 3.21.7 to 3.25.5 [flink-connector-aws]

2024-09-20 Thread via GitHub
hlteoh37 merged PR #169: URL: https://github.com/apache/flink-connector-aws/pull/169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] [minor][cdc-connector][mongodb] Add `op_type` metadata column [flink-cdc]

2024-09-20 Thread via GitHub
qg-lin commented on PR #3615: URL: https://github.com/apache/flink-cdc/pull/3615#issuecomment-2363291818 @Jiabao-Sun @yuxiqian PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

<    3   4   5   6   7   8   9   10   11   12   >