Re: [PR] [FLINK-36151] Add schema evolution related docs [flink-cdc]

2024-08-27 Thread via GitHub
monologuist commented on PR #3575: URL: https://github.com/apache/flink-cdc/pull/3575#issuecomment-2311755677 @yuxiqian Hello, I would like to know which Flink CDC version your document corresponds to. I only seem to see three schema change Behaviors in Flink CDC 3.0.1. -- This is an aut

Re: [PR] [hotfix] Run schema coordinator logic asynchronously to avoid blocking the main thread [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang commented on PR #3577: URL: https://github.com/apache/flink-cdc/pull/3577#issuecomment-2311762537 @loserwang1024 Would you like to review this PR when you have time ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [FLINK-36151] Add schema evolution related docs [flink-cdc]

2024-08-27 Thread via GitHub
yuxiqian commented on PR #3575: URL: https://github.com/apache/flink-cdc/pull/3575#issuecomment-2311766702 Hi @monologuist, this PR reflects current `master` situation (`release-3.3-SNAPSHOT`). Will backport it to previous release branches later. -- This is an automated message from the A

Re: [PR] [FLINK-36148][pipeline-connector][mysql] Add custom parser for CreateTableEvent [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang merged PR #3570: URL: https://github.com/apache/flink-cdc/pull/3570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Resolved] (FLINK-36148) Support parsing DDL to new CreateTableEvent in binlog reading phase.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36148. Resolution: Fixed fixed in master: 3c1517f70f728e1ef8a4b6bed942d87d66d874df > Support parsing DDL

Re: [PR] [FLINK-36150][pipeline-connector/mysql] tables.exclude is still valid if scan.binlog.newly-added-table.enabled is true. [flink-cdc]

2024-08-27 Thread via GitHub
ruanhang1993 commented on code in PR #3573: URL: https://github.com/apache/flink-cdc/pull/3573#discussion_r1732312221 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/schema/Selectors.java: ## @@ -0,0

[jira] [Updated] (FLINK-36133) Remove subclass implementations of TableSource and TableSink, along with their corresponding Factory classes.

2024-08-27 Thread dalongliu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dalongliu updated FLINK-36133: -- Parent: FLINK-33748 Issue Type: Sub-task (was: Technical Debt) > Remove subclass implementati

Re: [PR] [hotfix] Run schema coordinator logic asynchronously to avoid blocking the main thread [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang commented on code in PR #3577: URL: https://github.com/apache/flink-cdc/pull/3577#discussion_r1732395696 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/coordinator/SchemaRegistry.java: ## @@ -113,18 +119,27 @@ public class SchemaRegi

[jira] [Assigned] (FLINK-36066) Introducing the AdaptiveGraphGenerator component

2024-08-27 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36066: --- Assignee: Lei Yang > Introducing the AdaptiveGraphGenerator component > ---

[jira] [Assigned] (FLINK-36033) FLIP-469: Supports Adaptive Optimization of StreamGraph

2024-08-27 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36033: --- Assignee: Junrui Li > FLIP-469: Supports Adaptive Optimization of StreamGraph > ---

[jira] [Assigned] (FLINK-36032) FLIP-468: Introducing StreamGraph-Based Job Submission

2024-08-27 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36032: --- Assignee: Junrui Li > FLIP-468: Introducing StreamGraph-Based Job Submission >

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-27 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2311939034 > Caused by: org.rocksdb.RocksDBException: `/dev/null' exists but is not a directory This is because when run e2e test, the directory of rocksdb log will be specified as` /d

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-27 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2311939990 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Created] (FLINK-36159) FLIP-470: Support Adaptive Broadcast Join

2024-08-27 Thread xingbe (Jira)
xingbe created FLINK-36159: -- Summary: FLIP-470: Support Adaptive Broadcast Join Key: FLINK-36159 URL: https://issues.apache.org/jira/browse/FLINK-36159 Project: Flink Issue Type: New Feature

[jira] [Updated] (FLINK-36159) FLIP-470: Support Adaptive Broadcast Join

2024-08-27 Thread xingbe (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xingbe updated FLINK-36159: --- Description: This is the umbrella ticket for FLIP-470 https://cwiki.apache.org/confluence/display/FLINK/FLIP

Re: [PR] [FLINK-35773][docs] Document s5cmd [flink]

2024-08-27 Thread via GitHub
pnowojski commented on code in PR #25235: URL: https://github.com/apache/flink/pull/25235#discussion_r1732443462 ## docs/content/docs/deployment/filesystems/s3.md: ## @@ -164,4 +164,38 @@ The `s3.entropy.key` defines the string in paths that is replaced by the random If a file

Re: [PR] [FLINK-35773][docs] Document s5cmd [flink]

2024-08-27 Thread via GitHub
pnowojski commented on PR #25235: URL: https://github.com/apache/flink/pull/25235#issuecomment-2311981366 > And I guess we also need to update zh version? Yep, I wanted to wait until the english version is mostly done/approved, in order to avoid copy pasting fixups. -- This is an a

Re: [PR] [FLINK-36124][filesystem][s3]Make S3RecoverableFsDataOutputStream.sync not to close the stream [flink]

2024-08-27 Thread via GitHub
ferenc-csaky commented on PR #25231: URL: https://github.com/apache/flink/pull/25231#issuecomment-2312059662 Thanks for the fix @gaborgsomogyi! As I tried to understand the problem we have here, I checked the history of what happened here before and why. The previous change that was m

Re: [PR] [FLINK-36143][ES6][ES7] Intro retry-on-conflict param to resolve Sink ES occurred "version conflict" [flink-connector-elasticsearch]

2024-08-27 Thread via GitHub
leosanqing commented on PR #109: URL: https://github.com/apache/flink-connector-elasticsearch/pull/109#issuecomment-2312079949 @schulzp @alpinegizmo @aljoscha @rmetzger Has any body to review my code? thk ❤️❤️ -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] [FLINK-36117] Implement AsyncKeyedStateBackend for RocksDBKeyedStateBackend and HeapKeyedStateBackend [flink]

2024-08-27 Thread via GitHub
Zakelly commented on code in PR #25233: URL: https://github.com/apache/flink/pull/25233#discussion_r1732498188 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/StateDescriptorUtils.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] fixup! [FLINK-35886][task] Fix watermark idleness timeout accounting when subtask is backpressured/blocked [flink]

2024-08-27 Thread via GitHub
pnowojski commented on code in PR #25245: URL: https://github.com/apache/flink/pull/25245#discussion_r1732604961 ## pom.xml: ## @@ -2355,8 +2355,6 @@ under the License. @org.apache.flink.annotation.Experimental

Re: [PR] fixup! [FLINK-35886][task] Fix watermark idleness timeout accounting when subtask is backpressured/blocked [flink]

2024-08-27 Thread via GitHub
pnowojski commented on PR #25245: URL: https://github.com/apache/flink/pull/25245#issuecomment-2312186890 Thanks @rkhachatryan for looking into this. Let's try without reverting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] fixup! (1.20 backport) [FLINK-35886][task] Fix watermark idleness timeout accounting when subtask is backpressured/blocked [flink]

2024-08-27 Thread via GitHub
pnowojski commented on PR #25246: URL: https://github.com/apache/flink/pull/25246#issuecomment-2312192390 Thanks @rkhachatryan for looking into this. Let's try without reverting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Closed] (FLINK-36063) Migrate StreamGraph and its related classes to flink-runtime

2024-08-27 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-36063. --- Fix Version/s: 2.0.0 Resolution: Fixed master: 51f1684be0214a1a8c47ffb4ae20b6ebf6f0fc0c > Migrate St

Re: [PR] [FLINK-36063][runtime] Migrate StreamGraph and its related classes from flink-streaming-java to flink-runtime. [flink]

2024-08-27 Thread via GitHub
zhuzhurk closed pull request #25219: [FLINK-36063][runtime] Migrate StreamGraph and its related classes from flink-streaming-java to flink-runtime. URL: https://github.com/apache/flink/pull/25219 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] [FLINK-34510][Runtime/State]Rename RestoreMode to RecoveryClaimMode [flink]

2024-08-27 Thread via GitHub
Zakelly commented on code in PR #25192: URL: https://github.com/apache/flink/pull/25192#discussion_r1732616249 ## flink-runtime-web/src/test/resources/rest_api_v1.snapshot: ## @@ -4190,4 +4190,4 @@ } } } ] -} Review Comment: Ah, it's better to revert this chan

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-27 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2312392245 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [FLINK-35709][Table SQL / API] Allow defining a schema in REPLACE TABLE AS (RTAS) [flink]

2024-08-27 Thread via GitHub
spena commented on code in PR #25247: URL: https://github.com/apache/flink/pull/25247#discussion_r1732752338 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/MergeTableAsUtil.java: ## @@ -63,9 +79,90 @@ public class MergeTableAsUtil {

Re: [PR] [FLINK-36124][filesystem][s3]Make S3RecoverableFsDataOutputStream.sync not to close the stream [flink]

2024-08-27 Thread via GitHub
gaborgsomogyi commented on code in PR #25231: URL: https://github.com/apache/flink/pull/25231#discussion_r1732792672 ## flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/writer/S3RecoverableFsDataOutputStream.java: ## @@ -129,10 +129,9 @@ public void

Re: [PR] [FLINK-36124][filesystem][s3]Make S3RecoverableFsDataOutputStream.sync not to close the stream [flink]

2024-08-27 Thread via GitHub
gaborgsomogyi commented on code in PR #25231: URL: https://github.com/apache/flink/pull/25231#discussion_r1732814002 ## flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/writer/S3RecoverableFsDataOutputStream.java: ## @@ -129,10 +129,9 @@ public void

Re: [PR] Backport bug fixes from `master` to `release 3.2` branch [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang commented on PR #3566: URL: https://github.com/apache/flink-cdc/pull/3566#issuecomment-2312518458 Thanks @yuxiqian for the backport, all CI passed, I'll organize the commits and merge it soon. -- This is an automated message from the Apache Git Service. To respond to the messa

[jira] [Created] (FLINK-36160) Support hive advanced configuration

2024-08-27 Thread zhuanshenbsj1 (Jira)
zhuanshenbsj1 created FLINK-36160: - Summary: Support hive advanced configuration Key: FLINK-36160 URL: https://issues.apache.org/jira/browse/FLINK-36160 Project: Flink Issue Type: Improvement

[PR] [FLINK-36160] Support hive advanced configuration [flink]

2024-08-27 Thread via GitHub
zhuanshenbsj1 opened a new pull request, #25258: URL: https://github.com/apache/flink/pull/25258 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring t

[jira] [Created] (FLINK-36161) Integration Test Example uses deprecated SinkFunction

2024-08-27 Thread Wolff Bock von Wuelfingen (Jira)
Wolff Bock von Wuelfingen created FLINK-36161: - Summary: Integration Test Example uses deprecated SinkFunction Key: FLINK-36161 URL: https://issues.apache.org/jira/browse/FLINK-36161 Projec

Re: [PR] [FLINK-36160] Support hive advanced configuration [flink]

2024-08-27 Thread via GitHub
flinkbot commented on PR #25258: URL: https://github.com/apache/flink/pull/25258#issuecomment-2312648935 ## CI report: * 65a84f3a32e91294c6227f460c6ce97bc0d73868 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-35709][Table SQL / API] Allow defining a schema in REPLACE TABLE AS (RTAS) [flink]

2024-08-27 Thread via GitHub
twalthr merged PR #25247: URL: https://github.com/apache/flink/pull/25247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

[jira] [Closed] (FLINK-35708) Allow defining a schema in REPLACE TABLE AS (RTAS)

2024-08-27 Thread Timo Walther (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Walther closed FLINK-35708. Fix Version/s: 2.0.0 Resolution: Fixed Fixed in master: c1fa29ac91aeef64701d8d97b13644a4267

Re: [PR] [FLINK-35790] Update docs for new schema definition in CTAS and RTAS [flink]

2024-08-27 Thread via GitHub
twalthr merged PR #25238: URL: https://github.com/apache/flink/pull/25238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

[jira] [Closed] (FLINK-35790) Update docs for new schema definition in CTAS and RTAS

2024-08-27 Thread Timo Walther (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Walther closed FLINK-35790. Fix Version/s: 2.0.0 Assignee: Sergio Peña Resolution: Fixed Fixed in master: 8c11

[PR] [FLINK-36145] Rename flinkStateSnapshotReference to initialState [flink-kubernetes-operator]

2024-08-27 Thread via GitHub
gyfora opened a new pull request, #873: URL: https://github.com/apache/flink-kubernetes-operator/pull/873 ## What is the purpose of the change Rename JobSpec.flinkStateSnapshotReference to JobSpec.initialState. This should make less verbose and easier to memorize for users. fl

Re: [PR] [FLINK-36145][snapshot] Rename flinkStateSnapshotReference to snapshotReference [flink-kubernetes-operator]

2024-08-27 Thread via GitHub
gyfora commented on PR #870: URL: https://github.com/apache/flink-kubernetes-operator/pull/870#issuecomment-2312765821 I have opened a new PR with the further shortened `initialState` version -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
GOODBOY008 commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1733042779 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DataGeneratorSource.java: ## @@ -76,14 +76,14 @@ * Long values

Re: [PR] [FLINK-36145] Rename flinkStateSnapshotReference to initialState [flink-kubernetes-operator]

2024-08-27 Thread via GitHub
gyfora commented on PR #873: URL: https://github.com/apache/flink-kubernetes-operator/pull/873#issuecomment-2312807325 Actually @tweise @mateczagany , I have a slight concern about this initialState reference field overall. It may make sense to simply keep the current `initialSavepointPath

[jira] [Created] (FLINK-36162) Remove flinkStateSnapshotReference and namespace from FlinkStateSnapshot jobReference

2024-08-27 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-36162: -- Summary: Remove flinkStateSnapshotReference and namespace from FlinkStateSnapshot jobReference Key: FLINK-36162 URL: https://issues.apache.org/jira/browse/FLINK-36162 Pro

Re: [PR] [FLINK-36145] Rename flinkStateSnapshotReference to initialState [flink-kubernetes-operator]

2024-08-27 Thread via GitHub
gyfora closed pull request #873: [FLINK-36145] Rename flinkStateSnapshotReference to initialState URL: https://github.com/apache/flink-kubernetes-operator/pull/873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[jira] [Commented] (FLINK-36145) Change JobSpec.flinkStateSnapshotReference to snapshotReference

2024-08-27 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877085#comment-17877085 ] Gyula Fora commented on FLINK-36145: I will probably close this in favour of https:

[jira] [Commented] (FLINK-36162) Remove flinkStateSnapshotReference and namespace from FlinkStateSnapshot jobReference

2024-08-27 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877086#comment-17877086 ] Gyula Fora commented on FLINK-36162: cc [~thw]  > Remove flinkStateSnapshotReferenc

[jira] [Updated] (FLINK-36150) tables.exclude is not valid if scan.binlog.newly-added-table.enabled is true.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36150: --- Issue Type: Bug (was: Improvement) > tables.exclude is not valid if scan.binlog.newly-added-table.e

[jira] [Assigned] (FLINK-36150) tables.exclude is not valid if scan.binlog.newly-added-table.enabled is true.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-36150: -- Assignee: Hongshun Wang > tables.exclude is not valid if scan.binlog.newly-added-table.enable

Re: [PR] [FLINK-36150][pipeline-connector/mysql] tables.exclude is still valid if scan.binlog.newly-added-table.enabled is true. [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang merged PR #3573: URL: https://github.com/apache/flink-cdc/pull/3573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [minor][cdc-runtime] Run schema coordinator logic asynchronously to avoid blocking the main thread [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang merged PR #3577: URL: https://github.com/apache/flink-cdc/pull/3577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Resolved] (FLINK-36150) tables.exclude is not valid if scan.binlog.newly-added-table.enabled is true.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36150. Resolution: Fixed master: afe9c3c0adb77594565b24c004daa6385d50d5fc 3.2: todo > tables.exclude is

Re: [PR] Backport bug fixes from `master` to `release 3.2` branch [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang merged PR #3566: URL: https://github.com/apache/flink-cdc/pull/3566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-35890][cdc] Select only the increment but still enter the SNAPSHOT lead to ckp failed [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang commented on PR #3568: URL: https://github.com/apache/flink-cdc/pull/3568#issuecomment-2313070670 Thanks @leosanqing for the contribution, could you also check other connectors? I think some start up mode of debezium should not pass to underlying debeziunm engine once the Flink

[jira] [Updated] (FLINK-36128) The in-flight data will meet unrecoverable issue when restore from checkpoint once old schema has been flushed to downstream systems

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Fix Version/s: cdc-3.3.0 > The in-flight data will meet unrecoverable issue when restore from checkp

[jira] [Resolved] (FLINK-36128) The in-flight data will meet unrecoverable issue when restore from checkpoint once old schema has been flushed to downstream systems

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36128. Resolution: Fixed master: 2e938a92f5335515f7eb46077254aee25acd9107 release-3.2: debd43cdd75eb53ce

[jira] [Resolved] (FLINK-35056) when initial sqlserver table that's primary key is datetime type, it org.apache.flink.table.api.ValidationException: Timestamp precision must be between 0 and 9 (both

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-35056. Resolution: Fixed > when initial sqlserver table that's primary key is datetime type, it > org.a

[jira] [Comment Edited] (FLINK-35056) when initial sqlserver table that's primary key is datetime type, it org.apache.flink.table.api.ValidationException: Timestamp precision must be between 0 and 9

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876041#comment-17876041 ] Leonard Xu edited comment on FLINK-35056 at 8/27/24 4:56 PM: -

[jira] [Comment Edited] (FLINK-36114) Schema registry should block when handling existing requests

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875816#comment-17875816 ] Leonard Xu edited comment on FLINK-36114 at 8/27/24 4:58 PM: -

[jira] [Comment Edited] (FLINK-36092) Transform doesn't fully support with schema evolution

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875826#comment-17875826 ] Leonard Xu edited comment on FLINK-36092 at 8/27/24 4:57 PM: -

[jira] [Updated] (FLINK-36081) Flink CDC MySQL source connector missing some columns data of newly added tables

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36081: --- Fix Version/s: cdc-3.2.0 > Flink CDC MySQL source connector missing some columns data of newly added

[jira] [Resolved] (FLINK-36094) CDC SchemaRegistryRequestHandler should throw exception which is not SchemaEvolveException

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36094. Resolution: Fixed fixed master: 6205a5a0f16d2cf72ff751573351c4e15ea59efb release-3.2: 5a80757443e

[jira] [Comment Edited] (FLINK-36081) Flink CDC MySQL source connector missing some columns data of newly added tables

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875326#comment-17875326 ] Leonard Xu edited comment on FLINK-36081 at 8/27/24 5:00 PM: -

[jira] [Comment Edited] (FLINK-36088) Fix NPE in PaimonDataSink.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874938#comment-17874938 ] Leonard Xu edited comment on FLINK-36088 at 8/27/24 5:01 PM: -

[jira] [Comment Edited] (FLINK-36076) Hotfix: Set isSchemaChangeApplying as volatile for thread safe.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874940#comment-17874940 ] Leonard Xu edited comment on FLINK-36076 at 8/27/24 5:01 PM: -

[jira] [Resolved] (FLINK-36088) Fix NPE in PaimonDataSink.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36088. Resolution: Fixed > Fix NPE in PaimonDataSink. > -- > > Ke

[jira] [Resolved] (FLINK-36076) Hotfix: Set isSchemaChangeApplying as volatile for thread safe.

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36076. Resolution: Fixed > Hotfix: Set isSchemaChangeApplying as volatile for thread safe. >

[jira] [Resolved] (FLINK-36082) Fix NotSerializableException for KafkaDataSink

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36082. Resolution: Fixed > Fix NotSerializableException for KafkaDataSink > -

[jira] [Comment Edited] (FLINK-36082) Fix NotSerializableException for KafkaDataSink

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874876#comment-17874876 ] Leonard Xu edited comment on FLINK-36082 at 8/27/24 5:02 PM: -

[jira] [Resolved] (FLINK-36056) change flink.connector.elasticsearch.version to a release version in branch release-3.2

2024-08-27 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36056. Resolution: Fixed fixed in master(3.2-SNAPSHOT): ab64eb3b574ebbeb81cf6b6500acbd211eea39ab > chang

Re: [PR] [cdc-connector][sqlserver]Flink cdc pipeline support sqlserver source [flink-cdc]

2024-08-27 Thread via GitHub
zakhalex commented on PR #3445: URL: https://github.com/apache/flink-cdc/pull/3445#issuecomment-2313494810 @ChengJie1053 - do you have the privilege to approve the workflows? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (FLINK-36162) Remove flinkStateSnapshotReference and namespace from FlinkStateSnapshot jobReference

2024-08-27 Thread Thomas Weise (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877194#comment-17877194 ] Thomas Weise commented on FLINK-36162: -- [~gyfora] I was surprised when you closed y

Re: [PR] [FLINK-22748] Allow dynamic target topic selection in SQL Kafka sinks [flink-connector-kafka]

2024-08-27 Thread via GitHub
klam-shop commented on code in PR #109: URL: https://github.com/apache/flink-connector-kafka/pull/109#discussion_r1733569499 ## flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/KafkaTableITCase.java: ## @@ -188,6 +188,120 @@ public void test

Re: [PR] [FLINK-22748] Allow dynamic target topic selection in SQL Kafka sinks [flink-connector-kafka]

2024-08-27 Thread via GitHub
klam-shop commented on code in PR #109: URL: https://github.com/apache/flink-connector-kafka/pull/109#discussion_r1733569499 ## flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/KafkaTableITCase.java: ## @@ -188,6 +188,120 @@ public void test

Re: [PR] [WIP] [FLINK-XXXX] Generalized watermark wip [flink]

2024-08-27 Thread via GitHub
jeyhunkarimov commented on PR #24997: URL: https://github.com/apache/flink/pull/24997#issuecomment-2313665000 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [FLINK-35718][mysql] cherry pick DBZ-5333 from debezium MySqlErrorHandler and add logs. [flink-cdc]

2024-08-27 Thread via GitHub
github-actions[bot] commented on PR #3440: URL: https://github.com/apache/flink-cdc/pull/3440#issuecomment-2313776551 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

[jira] [Updated] (FLINK-35718) Cherrypick DBZ-5333 to fix frequently failover by EOFException.

2024-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35718: --- Labels: pull-request-available (was: ) > Cherrypick DBZ-5333 to fix frequently failover by

Re: [PR] [FLINK-35277][cdc-connector][db2] Fix the error in the `asncdcaddremove.sql` script for the DB2 test container. [flink-cdc]

2024-08-27 Thread via GitHub
github-actions[bot] commented on PR #3286: URL: https://github.com/apache/flink-cdc/pull/3286#issuecomment-2313776592 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [e2e][tests] Improve the stability of pipeline e2e tests [flink-cdc]

2024-08-27 Thread via GitHub
yuxiqian commented on code in PR #3578: URL: https://github.com/apache/flink-cdc/pull/3578#discussion_r1733730877 ## flink-cdc-e2e-tests/flink-cdc-pipeline-e2e-tests/src/test/java/org/apache/flink/cdc/pipeline/tests/RouteE2eITCase.java: ## @@ -719,6 +718,11 @@ public void testMe

Re: [PR] [cdc-connector][sqlserver]Flink cdc pipeline support sqlserver source [flink-cdc]

2024-08-27 Thread via GitHub
ChengJie1053 commented on PR #3445: URL: https://github.com/apache/flink-cdc/pull/3445#issuecomment-2313885858 > @ChengJie1053 - do you have the privilege to approve the workflows? I'm sorry. I don't have that clearance -- This is an automated message from the Apache Git Service

[jira] [Updated] (FLINK-35917) Failed to deserialize data of EventHeaderV4

2024-08-27 Thread wangkang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangkang updated FLINK-35917: - Description: Caused by: java.lang.RuntimeException: One or more fetchers have encountered exception at

Re: [PR] [e2e][tests] Improve the stability of pipeline e2e tests [flink-cdc]

2024-08-27 Thread via GitHub
leonardBang commented on code in PR #3578: URL: https://github.com/apache/flink-cdc/pull/3578#discussion_r1733762640 ## flink-cdc-e2e-tests/flink-cdc-pipeline-e2e-tests/src/test/java/org/apache/flink/cdc/pipeline/tests/RouteE2eITCase.java: ## @@ -719,6 +718,11 @@ public void tes

[jira] [Updated] (FLINK-35917) Failed to deserialize data of EventHeaderV4

2024-08-27 Thread wangkang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangkang updated FLINK-35917: - Description: it occurred during the full to incremental phase: Caused by: java.lang.RuntimeException: O

Re: [PR] [FLINK-35599][cdc-connector][jdbc-mysql]Flink cdc pipeline sink jdbc mysql [flink-cdc]

2024-08-27 Thread via GitHub
cobolbaby commented on PR #3433: URL: https://github.com/apache/flink-cdc/pull/3433#issuecomment-2313956284 Does it only support jdbc-mysql? Can the jdbc be extended to a general sink connector? -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (FLINK-35599) Add JDBC Sink Plugin to Flink-CDC-Pipeline

2024-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35599: --- Labels: pull-request-available (was: ) > Add JDBC Sink Plugin to Flink-CDC-Pipeline > -

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
GOODBOY008 merged PR #24692: URL: https://github.com/apache/flink/pull/24692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.ap

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
GOODBOY008 commented on PR #24692: URL: https://github.com/apache/flink/pull/24692#issuecomment-2314045037 @morozov LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Resolved] (FLINK-35177) Datagen examples in documentation do not compile

2024-08-27 Thread Zhongqiang Gong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongqiang Gong resolved FLINK-35177. - Fix Version/s: 2.0.0 Resolution: Fixed Merged to branch: Master , Via : bc3afba5c

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
GOODBOY008 commented on PR #24692: URL: https://github.com/apache/flink/pull/24692#issuecomment-2314102983 @morozov Can you open another pr to branch `release-1.20` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Created] (FLINK-36163) Kafka cdc pipeline source supports protobuf format.

2024-08-27 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-36163: - Summary: Kafka cdc pipeline source supports protobuf format. Key: FLINK-36163 URL: https://issues.apache.org/jira/browse/FLINK-36163 Project: Flink Issue T

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-27 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2314121275 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Created] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
tchivs created FLINK-36164: -- Summary: JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table Key: FLINK-36164 URL: https://issues.apach

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Description: When synchronizing a PostgreSQL table using a connector, if the table contains a large number

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Description: What's Wrong? When synchronizing a PostgreSQL table using a connector, if the table contains a

[PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
morozov opened a new pull request, #25259: URL: https://github.com/apache/flink/pull/25259 This is a backport of https://github.com/apache/flink/pull/24692. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-27 Thread via GitHub
morozov commented on PR #24692: URL: https://github.com/apache/flink/pull/24692#issuecomment-2314127171 @GOODBOY008, please see https://github.com/apache/flink/pull/25259. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Description: What's Wrong? When synchronizing a PostgreSQL table using a connector, if the table contains a

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Attachment: image-2024-08-28-11-28-33-016.png > JdbcIncrementalSource CheckPoint Timeout Due to Retrieving S

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Attachment: image-2024-08-28-11-28-21-496.png > JdbcIncrementalSource CheckPoint Timeout Due to Retrieving S

[jira] [Updated] (FLINK-36164) JdbcIncrementalSource CheckPoint Timeout Due to Retrieving Schemas for All Subpartitions When Synchronizing a Partitioned PostgreSQL Table

2024-08-27 Thread tchivs (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tchivs updated FLINK-36164: --- Description: What's Wrong? When synchronizing a PostgreSQL table using a connector, if the table contains a

  1   2   >