Re: [PR] [FLINK-32440][checkpoint] Introduce file merging configurations [flink]

2024-04-11 Thread via GitHub
fredia commented on code in PR #22973: URL: https://github.com/apache/flink/pull/22973#discussion_r1562003338 ## docs/layouts/shortcodes/generated/checkpointing_configuration.html: ## @@ -44,6 +44,42 @@ String The default directory used for storing the

Re: [PR] [FLINK-32440][checkpoint] Introduce file merging configurations [flink]

2024-04-11 Thread via GitHub
Zakelly commented on code in PR #22973: URL: https://github.com/apache/flink/pull/22973#discussion_r1561983143 ## docs/layouts/shortcodes/generated/checkpointing_configuration.html: ## @@ -44,6 +44,42 @@ String The default directory used for storing

Re: [PR] [FLINK-35089][runtime] Serialize the lastRecordAttributes in AbstractStreamOperator [flink]

2024-04-11 Thread via GitHub
Sxnan commented on PR #24655: URL: https://github.com/apache/flink/pull/24655#issuecomment-2050918953 @xintongsong Can you help review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [FLINK-35089][runtime] Serialize the lastRecordAttributes in AbstractStreamOperator [flink]

2024-04-11 Thread via GitHub
flinkbot commented on PR #24655: URL: https://github.com/apache/flink/pull/24655#issuecomment-205094 ## CI report: * b8758babe99b2fc7f080026b3615e2b5ff7da164 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-35089) Two input AbstractStreamOperator may throw NPE when receiving RecordAttributes

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35089: --- Labels: pull-request-available (was: ) > Two input AbstractStreamOperator may throw NPE

[PR] [FLINK-35089][runtime] Serialize the lastRecordAttributes in AbstractStreamOperator [flink]

2024-04-11 Thread via GitHub
Sxnan opened a new pull request, #24655: URL: https://github.com/apache/flink/pull/24655 ## What is the purpose of the change Currently the `lastRecordAttributes1` and `lastRecordAttributes2` in the `AbstractStreamOperator` are transient. The two fields will be null when it is

[jira] [Updated] (FLINK-35089) Two input AbstractStreamOperator may throw NPE when receiving RecordAttributes

2024-04-11 Thread Xuannan Su (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuannan Su updated FLINK-35089: --- Description: Currently the `lastRecordAttributes1` and `lastRecordAttributes2` in the

[jira] [Commented] (FLINK-35093) Postgres source connector support SPECIFIC_OFFSETS start up mode from an existed replication slot.

2024-04-11 Thread Hongshun Wang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836403#comment-17836403 ] Hongshun Wang commented on FLINK-35093: --- I'd like to do it > Postgres source connector support

[jira] [Created] (FLINK-35093) Postgres source connector support SPECIFIC_OFFSETS start up mode from an existed replication slot.

2024-04-11 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35093: - Summary: Postgres source connector support SPECIFIC_OFFSETS start up mode from an existed replication slot. Key: FLINK-35093 URL:

[jira] [Updated] (FLINK-35090) Doris sink fails to create table when database does not exist

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35090: --- Labels: pull-request-available (was: ) > Doris sink fails to create table when database

Re: [PR] [FLINK-35090][cdc][doris] Add database auto-creating support for Doris sink pipeline connector [flink-cdc]

2024-04-11 Thread via GitHub
yuxiqian commented on PR #3222: URL: https://github.com/apache/flink-cdc/pull/3222#issuecomment-2050900067 @lvyanquan PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-34903][MySQL][Feature] Add mysql-pipeline-connector with table.exclude.list option to exclude unnecessary tables [flink-cdc]

2024-04-11 Thread via GitHub
shiyiky commented on code in PR #3186: URL: https://github.com/apache/flink-cdc/pull/3186#discussion_r1561958443 ##

Re: [PR] [FLINK-32440][checkpoint] Introduce file merging configurations [flink]

2024-04-11 Thread via GitHub
fredia commented on code in PR #22973: URL: https://github.com/apache/flink/pull/22973#discussion_r1561957606 ## docs/layouts/shortcodes/generated/checkpointing_configuration.html: ## @@ -44,6 +44,42 @@ String The default directory used for storing the

Re: [PR] [FLINK-34969][cdc-cli]Add support for both new and old Flink config files in Flink… [flink-cdc]

2024-04-11 Thread via GitHub
PatrickRen commented on code in PR #3194: URL: https://github.com/apache/flink-cdc/pull/3194#discussion_r1561954297 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/utils/MemorySize.java: ## @@ -0,0 +1,421 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-32440][checkpoint] Introduce file merging configurations [flink]

2024-04-11 Thread via GitHub
fredia commented on code in PR #22973: URL: https://github.com/apache/flink/pull/22973#discussion_r1561952908 ## docs/layouts/shortcodes/generated/checkpointing_configuration.html: ## @@ -44,6 +44,42 @@ String The default directory used for storing the

[jira] [Commented] (FLINK-35035) Reduce job pause time when cluster resources are expanded in adaptive mode

2024-04-11 Thread yuanfenghu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836395#comment-17836395 ] yuanfenghu commented on FLINK-35035: [~echauchot]  Thank you for your reply. You should look at

Re: [PR] [FLINK-20398][e2e] Migrate test_batch_sql.sh to Java e2e tests framework [flink]

2024-04-11 Thread via GitHub
morazow commented on code in PR #24471: URL: https://github.com/apache/flink/pull/24471#discussion_r1561946899 ## flink-end-to-end-tests/flink-batch-sql-test/src/test/java/org/apache/flink/sql/tests/Generator.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

Re: [PR] [FLINK-35091][Metrics][Minor] Fix incorrect warning msg in JM log when use metric reporter [flink]

2024-04-11 Thread via GitHub
flinkbot commented on PR #24654: URL: https://github.com/apache/flink/pull/24654#issuecomment-2050853597 ## CI report: * 682a59db0c1587c46c9fe72fd0dd8e1283e51e8a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Created] (FLINK-35092) Add integrated test for Doris sink pipeline connector

2024-04-11 Thread Xiqian YU (Jira)
Xiqian YU created FLINK-35092: - Summary: Add integrated test for Doris sink pipeline connector Key: FLINK-35092 URL: https://issues.apache.org/jira/browse/FLINK-35092 Project: Flink Issue Type:

Re: [PR] [FLINK-34853] Draft: Submit CDC Job To Flink K8S Native Application Mode [flink-cdc]

2024-04-11 Thread via GitHub
PatrickRen commented on PR #3093: URL: https://github.com/apache/flink-cdc/pull/3093#issuecomment-2050850214 @czy006 Could you rebase the PR to the latest master? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] [FLINK-35091][Metrics][Minor] Fix incorrect warning msg in JM log when use metric reporter [flink]

2024-04-11 Thread via GitHub
leosanqing opened a new pull request, #24654: URL: https://github.com/apache/flink/pull/24654 ## Desc I encountered an issue while upgrading Flink from version 1.14 to 1.18. After the upgrade, I noticed that some monitoring metrics were not being reported to InfluxDB. Upon checking

[jira] [Updated] (FLINK-35091) Incorrect warning msg in JM when use metric reporter

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35091: --- Labels: pull-request-available (was: ) > Incorrect warning msg in JM when use metric

Re: [PR] [FLINK-35024][Runtime/State] Implement the record buffer of AsyncExecutionController [flink]

2024-04-11 Thread via GitHub
fredia commented on PR #24633: URL: https://github.com/apache/flink/pull/24633#issuecomment-2050847516 @Zakelly Thanks for the review, updated and squashed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (FLINK-34634) Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed

2024-04-11 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836393#comment-17836393 ] Qingsheng Ren commented on FLINK-34634: --- flink-cdc master:

[jira] [Resolved] (FLINK-34634) Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed

2024-04-11 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingsheng Ren resolved FLINK-34634. --- Assignee: Hongshun Wang Resolution: Fixed > Restarting the job will not read the

Re: [PR] [FLINK-34634]Fix that Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed [flink-cdc]

2024-04-11 Thread via GitHub
PatrickRen merged PR #3134: URL: https://github.com/apache/flink-cdc/pull/3134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Resolved] (FLINK-34689) check binlog_row_value_optoins

2024-04-11 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingsheng Ren resolved FLINK-34689. --- Fix Version/s: cdc-3.1.0 Resolution: Fixed > check binlog_row_value_optoins >

Re: [PR] [FLINK-35072][cdc][doris] Support applying compatible `AlterColumnTypeEvent` to Doris sink [flink-cdc]

2024-04-11 Thread via GitHub
yuxiqian commented on PR #3215: URL: https://github.com/apache/flink-cdc/pull/3215#issuecomment-2050843987 I'm not familiar with Doris, could @lvyanquan @JNSimba take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (FLINK-34689) check binlog_row_value_optoins

2024-04-11 Thread Qingsheng Ren (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836391#comment-17836391 ] Qingsheng Ren commented on FLINK-34689: --- flink-cdc master:

Re: [PR] [FLINK-34689][MySQL][Feature] check binlog_row_value_options [flink-cdc]

2024-04-11 Thread via GitHub
PatrickRen merged PR #3148: URL: https://github.com/apache/flink-cdc/pull/3148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (FLINK-35091) Incorrect warning msg in JM when use metric reporter

2024-04-11 Thread sanqingleo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sanqingleo updated FLINK-35091: --- Summary: Incorrect warning msg in JM when use metric reporter (was: Incorrect Warning msg in JM

[jira] [Created] (FLINK-35091) Incorrect Warning msg in JM when use influxdb

2024-04-11 Thread sanqingleo (Jira)
sanqingleo created FLINK-35091: -- Summary: Incorrect Warning msg in JM when use influxdb Key: FLINK-35091 URL: https://issues.apache.org/jira/browse/FLINK-35091 Project: Flink Issue Type:

[jira] [Commented] (FLINK-35090) Doris sink fails to create table when database does not exist

2024-04-11 Thread Xiqian YU (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836386#comment-17836386 ] Xiqian YU commented on FLINK-35090: --- [~renqs] I'm willing to take this ticket. > Doris sink fails to

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on PR #20990: URL: https://github.com/apache/flink/pull/20990#issuecomment-2050824391 Thank you @XComp @1996fanrui @Samrat002 very much for the review and sorry for the late response. I make some change based on your comments. Would you mind helping to have a

[jira] [Created] (FLINK-35090) Doris sink fails to create table when database does not exist

2024-04-11 Thread Xiqian YU (Jira)
Xiqian YU created FLINK-35090: - Summary: Doris sink fails to create table when database does not exist Key: FLINK-35090 URL: https://issues.apache.org/jira/browse/FLINK-35090 Project: Flink

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on code in PR #20990: URL: https://github.com/apache/flink/pull/20990#discussion_r1561915638 ## flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/JavaProgramTestBaseJUnit5.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the

[jira] [Created] (FLINK-35089) Two input AbstractStreamOperator may throw NPE when receiving RecordAttributes

2024-04-11 Thread Xuannan Su (Jira)
Xuannan Su created FLINK-35089: -- Summary: Two input AbstractStreamOperator may throw NPE when receiving RecordAttributes Key: FLINK-35089 URL: https://issues.apache.org/jira/browse/FLINK-35089 Project:

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on code in PR #20990: URL: https://github.com/apache/flink/pull/20990#discussion_r1561911292 ## flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/JavaProgramTestBaseJUnit5.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on code in PR #20990: URL: https://github.com/apache/flink/pull/20990#discussion_r1561910994 ## flink-connectors/flink-hadoop-compatibility/src/test/java/org/apache/flink/test/hadoopcompatibility/mapred/HadoopIOFormatsITCase.java: ## @@ -35,40 +38,54 @@

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on code in PR #20990: URL: https://github.com/apache/flink/pull/20990#discussion_r1561910064 ## flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/JavaProgramTestBaseJUnit5.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the

[jira] [Commented] (FLINK-34127) Kafka connector repo runs a duplicate of `IntegrationTests` framework tests

2024-04-11 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836382#comment-17836382 ] Mason Chen commented on FLINK-34127: Sure. Can I assign myself now with committer permissions? :D >

Re: [PR] [FLINK-29050][JUnit5 Migration] Module: flink-hadoop-compatibility [flink]

2024-04-11 Thread via GitHub
RocMarshal commented on code in PR #20990: URL: https://github.com/apache/flink/pull/20990#discussion_r1296008112 ## flink-connectors/flink-hadoop-compatibility/src/test/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopInputFormatTest.java: ## @@ -137,27 +132,26 @@ public

[jira] [Closed] (FLINK-30238) Unified Sink committer does not clean up state on final savepoint

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-30238. -- Resolution: Invalid > Unified Sink committer does not clean up state on final savepoint >

[jira] [Commented] (FLINK-35088) watermark alignment maxAllowedWatermarkDrift and updateInterval param need check

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836364#comment-17836364 ] Martijn Visser commented on FLINK-35088: [~elon] Please verify this with later version of Flink,

[jira] [Commented] (FLINK-35076) Watermark alignment will cause data flow to experience serious shake

2024-04-11 Thread Kenneth William Krugler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836325#comment-17836325 ] Kenneth William Krugler commented on FLINK-35076: - Hi [~elon] - please post these

[jira] [Comment Edited] (FLINK-35076) Watermark alignment will cause data flow to experience serious shake

2024-04-11 Thread elon_X (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836238#comment-17836238 ] elon_X edited comment on FLINK-35076 at 4/11/24 3:23 PM: - [~kkrugler]  Thank

[jira] [Commented] (FLINK-35076) Watermark alignment will cause data flow to experience serious shake

2024-04-11 Thread elon_X (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836238#comment-17836238 ] elon_X commented on FLINK-35076: [~kkrugler]  Thank you for your reply. Setting the idle time is not

Re: [PR] [FLINK-20398][e2e] Migrate test_batch_sql.sh to Java e2e tests framework [flink]

2024-04-11 Thread via GitHub
affo commented on PR #24471: URL: https://github.com/apache/flink/pull/24471#issuecomment-2049852124 @XComp Glad for your vacation! Finally I also addressed the deprecation warnings and went through the implementation of a custom connector through `DynamicTableSource`. It turned

[jira] [Commented] (FLINK-34224) ChangelogStorageMetricsTest.testAttemptsPerUpload(ChangelogStorageMetricsTest timed out

2024-04-11 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836218#comment-17836218 ] Ryan Skraba commented on FLINK-34224: - 1.20 Java 8: Test (module: core)

Re: [PR] [FLINK-34996][Connectors/Kafka] Use UserCodeCL to instantiate Deserializer [flink-connector-kafka]

2024-04-11 Thread via GitHub
hugogu commented on PR #89: URL: https://github.com/apache/flink-connector-kafka/pull/89#issuecomment-2049754379 @morazow Thanks for your approval. I have just rebased this PR to include the build fix made in main branch. Hopefully the build would success this time. I also noticed

Re: [PR] [FLINK-34738][cdc][docs-zh] "Deployment - YARN" Page for Flink CDC Chinese Documentation [flink-cdc]

2024-04-11 Thread via GitHub
Vincent-Woo commented on PR #3205: URL: https://github.com/apache/flink-cdc/pull/3205#issuecomment-2049708639 @leonardBang @PatrickRen @lvyanquan @loserwang1024 Excuse me, do you have time to take a look at this? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [FLINK-32084][checkpoint] Migrate current file merging of channel state snapshot into the unify file merging framework [flink]

2024-04-11 Thread via GitHub
flinkbot commented on PR #24653: URL: https://github.com/apache/flink/pull/24653#issuecomment-2049623349 ## CI report: * ecce541d1b05cd350ca635fc8e0f6611743d583d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-32084) Migrate current file merging of channel state into the file merging framework

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32084: --- Labels: pull-request-available (was: ) > Migrate current file merging of channel state

[PR] [FLINK-32084][checkpoint] Migrate current file merging of channel state snapshot into the unify file merging framework [flink]

2024-04-11 Thread via GitHub
fredia opened a new pull request, #24653: URL: https://github.com/apache/flink/pull/24653 ## What is the purpose of the change Migrate current file merging of channel state snapshot into the unify file merging framework. This PR only focuses on the snapshot/writing part.

[jira] [Commented] (FLINK-34127) Kafka connector repo runs a duplicate of `IntegrationTests` framework tests

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836149#comment-17836149 ] Martijn Visser commented on FLINK-34127: [~mason6345] Is this something that you can check? I

[jira] [Created] (FLINK-35088) watermark alignment maxAllowedWatermarkDrift and updateInterval param need check

2024-04-11 Thread elon_X (Jira)
elon_X created FLINK-35088: -- Summary: watermark alignment maxAllowedWatermarkDrift and updateInterval param need check Key: FLINK-35088 URL: https://issues.apache.org/jira/browse/FLINK-35088 Project: Flink

Re: [PR] [FLINK-35024][Runtime/State] Implement the record buffer of AsyncExecutionController [flink]

2024-04-11 Thread via GitHub
Zakelly commented on code in PR #24633: URL: https://github.com/apache/flink/pull/24633#discussion_r1560916376 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/StateRequestBuffer.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software

Re: [PR] [FLINK-34661][runtime] TaskExecutor supports retain partitions after JM crashed. [flink]

2024-04-11 Thread via GitHub
zhuzhurk commented on code in PR #24582: URL: https://github.com/apache/flink/pull/24582#discussion_r1560848973 ## flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskExecutor.java: ## @@ -1886,6 +1911,33 @@ private void disconnectJobManagerConnection(

[jira] [Commented] (FLINK-34704) Process checkpoint barrier in AsyncWaitOperator when the element queue is full

2024-04-11 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836142#comment-17836142 ] Zakelly Lan commented on FLINK-34704: - Thanks for your detailed explanation! Now I see your point.

[jira] [Closed] (FLINK-35038) Bump test dependency org.yaml:snakeyaml to 2.2

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-35038. -- Fix Version/s: kafka-4.0.0 kafka-3.1.1 (was: 3.1.0)

[jira] [Updated] (FLINK-35038) Bump test dependency org.yaml:snakeyaml to 2.2 for Flink Kafka connector

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-35038: --- Summary: Bump test dependency org.yaml:snakeyaml to 2.2 for Flink Kafka connector (was:

[jira] [Closed] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink Kafka connector

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-35008. -- Fix Version/s: kafka-4.0.0 kafka-3.1.1 Resolution: Fixed Fixed in

Re: [PR] [FLINK-35038] Bump `org.yaml:snakeyaml` to `2.2` [flink-connector-kafka]

2024-04-11 Thread via GitHub
boring-cyborg[bot] commented on PR #93: URL: https://github.com/apache/flink-connector-kafka/pull/93#issuecomment-2049520227 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [FLINK-35038] Bump `org.yaml:snakeyaml` to `2.2` [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser merged PR #93: URL: https://github.com/apache/flink-connector-kafka/pull/93 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink Kafka connector

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35008: --- Labels: pull-request-available (was: ) > Bump org.apache.commons:commons-compress from

Re: [PR] [FLINK-35008] Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser merged PR #87: URL: https://github.com/apache/flink-connector-kafka/pull/87 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Closed] (FLINK-35007) Update Flink Kafka connector to support 1.19

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-35007. -- Fix Version/s: kafka-4.0.0 kafka-3.1.1 Resolution: Fixed Fixed in

Re: [PR] [FLINK-35007][BP v3.1] Add support for Flink 1.19 (#90) [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser merged PR #94: URL: https://github.com/apache/flink-connector-kafka/pull/94 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (FLINK-20217) More fine-grained timer processing

2024-04-11 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski reassigned FLINK-20217: -- Assignee: Piotr Nowojski > More fine-grained timer processing >

[jira] [Updated] (FLINK-34961) GitHub Actions runner statistcs can be monitored per workflow name

2024-04-11 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin updated FLINK-34961: Fix Version/s: kafka-4.0.0 > GitHub Actions runner statistcs can be monitored per

[jira] [Commented] (FLINK-34961) GitHub Actions runner statistcs can be monitored per workflow name

2024-04-11 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836128#comment-17836128 ] Sergey Nuyanzin commented on FLINK-34961: - Merged to flink-connector-kafka main as

Re: [PR] [FLINK-34961] Use dedicated CI name for Kafka connector to differentiate it in infra-reports [flink-connector-kafka]

2024-04-11 Thread via GitHub
snuyanzin merged PR #92: URL: https://github.com/apache/flink-connector-kafka/pull/92 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink Kafka connector

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836125#comment-17836125 ] Martijn Visser commented on FLINK-35008: I've updated to PR to use 1.26.1 > Bump

[jira] [Updated] (FLINK-35022) Add TypeInformed Element Converter for DynamoDbSink

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35022: --- Labels: pull-request-available (was: ) > Add TypeInformed Element Converter for

[PR] [FLINK-35022] Add TypeInformed DDB Element Converter [flink-connector-aws]

2024-04-11 Thread via GitHub
vahmed-hamdy opened a new pull request, #136: URL: https://github.com/apache/flink-connector-aws/pull/136 ## Purpose of the change Add `DynamoDbTypeInformedElementConverter` to convert Elements to dynamoDb Sink using its provided type Info. ## Verifying this change

Re: [PR] [FLINK-XXXXX] Bump org.yaml:snakeyaml from 1.31 to 2.0 in /flink-connector-kafka [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser commented on PR #85: URL: https://github.com/apache/flink-connector-kafka/pull/85#issuecomment-2049475263 Superseded by https://github.com/apache/flink-connector-kafka/pull/93 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [FLINK-XXXXX] Bump org.yaml:snakeyaml from 1.31 to 2.0 in /flink-connector-kafka [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser closed pull request #85: [FLINK-X] Bump org.yaml:snakeyaml from 1.31 to 2.0 in /flink-connector-kafka URL: https://github.com/apache/flink-connector-kafka/pull/85 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Comment Edited] (FLINK-29050) [JUnit5 Migration] Module: flink-hadoop-compatibility

2024-04-11 Thread RocMarshal (Jira)
[ https://issues.apache.org/jira/browse/FLINK-29050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798856#comment-17798856 ] RocMarshal edited comment on FLINK-29050 at 4/11/24 10:56 AM: -- Based on

[jira] [Commented] (FLINK-34704) Process checkpoint barrier in AsyncWaitOperator when the element queue is full

2024-04-11 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836113#comment-17836113 ] Piotr Nowojski commented on FLINK-34704: Sounds good to me (y) Indeed we could later provide

Re: [PR] [FLINK-35079] Fallback to timestamp startup mode when resume token has expired [flink-cdc]

2024-04-11 Thread via GitHub
yuxiqian commented on code in PR #3221: URL: https://github.com/apache/flink-cdc/pull/3221#discussion_r1560819276 ##

Re: [PR] [FLINK-35045][state] Introduce Internal State for Async State API [flink]

2024-04-11 Thread via GitHub
Zakelly commented on code in PR #24651: URL: https://github.com/apache/flink/pull/24651#discussion_r1560803119 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/internal/InternalKvState.java: ## @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Comment Edited] (FLINK-34704) Process checkpoint barrier in AsyncWaitOperator when the element queue is full

2024-04-11 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836106#comment-17836106 ] Gyula Fora edited comment on FLINK-34704 at 4/11/24 10:41 AM: -- I agree with

[jira] [Commented] (FLINK-34704) Process checkpoint barrier in AsyncWaitOperator when the element queue is full

2024-04-11 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836107#comment-17836107 ] Gyula Fora commented on FLINK-34704: So restricting the optimisation to the head of the operator

[jira] [Commented] (FLINK-34704) Process checkpoint barrier in AsyncWaitOperator when the element queue is full

2024-04-11 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836106#comment-17836106 ] Gyula Fora commented on FLINK-34704: I agree with [~pnowojski] here, the currently blocked element

Re: [PR] [FLINK-35079] Fallback to timestamp startup mode when resume token has expired [flink-cdc]

2024-04-11 Thread via GitHub
Jiabao-Sun commented on code in PR #3221: URL: https://github.com/apache/flink-cdc/pull/3221#discussion_r1560798123 ##

Re: [PR] [FLINK-35040] Revert `commons-io` to 2.11.0 [flink]

2024-04-11 Thread via GitHub
slfan1989 commented on PR #24652: URL: https://github.com/apache/flink/pull/24652#issuecomment-2049386534 > Thanks @slfan1989 for the quick review! > > > @1996fanrui If we confirm that it's an issue with commons-io, couldn't we resolve it by upgrading commons-io instead? > >

[jira] [Commented] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-11 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836099#comment-17836099 ] Shilun Fan commented on FLINK-35040: [~fanrui] During the compilation process, we found that

Re: [PR] [FLINK-35040] Revert `commons-io` to 2.11.0 [flink]

2024-04-11 Thread via GitHub
1996fanrui commented on PR #24652: URL: https://github.com/apache/flink/pull/24652#issuecomment-2049377985 Thanks @slfan1989 for the quick review! > @1996fanrui If we confirm that it's an issue with commons-io, couldn't we resolve it by upgrading commons-io instead? Do you

Re: [PR] [FLINK-35079] Fallback to timestamp startup mode when resume token has expired [flink-cdc]

2024-04-11 Thread via GitHub
yuxiqian commented on code in PR #3221: URL: https://github.com/apache/flink-cdc/pull/3221#discussion_r1560784586 ##

Re: [PR] [FLINK-35079] Fallback to timestamp startup mode when resume token has expired [flink-cdc]

2024-04-11 Thread via GitHub
yuxiqian commented on code in PR #3221: URL: https://github.com/apache/flink-cdc/pull/3221#discussion_r1560784586 ##

[jira] [Comment Edited] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-11 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836091#comment-17836091 ] Rui Fan edited comment on FLINK-35040 at 4/11/24 10:16 AM: --- Hi [~slfan1989] ,

Re: [PR] [FLINK-35040] Revert `commons-io` to 2.11.0 [flink]

2024-04-11 Thread via GitHub
slfan1989 commented on PR #24652: URL: https://github.com/apache/flink/pull/24652#issuecomment-2049370922 @1996fanrui If we confirm that it's an issue with commons-io, couldn't we resolve it by upgrading commons-io instead? -- This is an automated message from the Apache Git Service. To

[jira] [Comment Edited] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-11 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836091#comment-17836091 ] Rui Fan edited comment on FLINK-35040 at 4/11/24 10:15 AM: --- Hi [~slfan1989] ,

Re: [PR] [FLINK-35040] Revert `commons-io` to 2.11.0 [flink]

2024-04-11 Thread via GitHub
flinkbot commented on PR #24652: URL: https://github.com/apache/flink/pull/24652#issuecomment-2049368111 ## CI report: * fbdcbb96a3d6cb0a766aef51eb332b8eb98f6759 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink Kafka connector

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-35008: --- Summary: Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink Kafka

[jira] [Commented] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-11 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836091#comment-17836091 ] Rui Fan commented on FLINK-35040: - Hi [~slfan1989] , thanks for your quick feedback! FLINK-34955 wants

[jira] [Updated] (FLINK-35040) The performance of serializerHeavyString regresses since April 3

2024-04-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35040: --- Labels: pull-request-available (was: ) > The performance of serializerHeavyString

[PR] [FLINK-35040] Revert `commons-io` to 2.11.0 [flink]

2024-04-11 Thread via GitHub
1996fanrui opened a new pull request, #24652: URL: https://github.com/apache/flink/pull/24652 ## What is the purpose of the change The performance of serializerHeavyString regresses since April 3, and had not yet recovered on April 8th. After running a series of benchmarks, I

[jira] [Commented] (FLINK-32582) Move TypeSerializerUpgradeTestBase from Kafka connector into flink-connector-common

2024-04-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836087#comment-17836087 ] Martijn Visser commented on FLINK-32582: [~mapohl] What was the plan on this ticket? We're now

Re: [PR] [FLINK-35007][BP v3.1] Add support for Flink 1.19 (#90) [flink-connector-kafka]

2024-04-11 Thread via GitHub
MartijnVisser commented on code in PR #94: URL: https://github.com/apache/flink-connector-kafka/pull/94#discussion_r1560765488 ## .github/workflows/weekly.yml: ## @@ -38,11 +38,20 @@ jobs: jdk: '8, 11, 17, 21', branch: main }, { - flink:

  1   2   >