Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-05-22 Thread via GitHub
hk-lrzy commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1609878354 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -127,11 +142,23 @@ public Schema load(TableId tableId) {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-05-22 Thread via GitHub
hk-lrzy commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1609878354 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -127,11 +142,23 @@ public Schema load(TableId tableId) {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-05-22 Thread via GitHub
hk-lrzy commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1609878354 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -127,11 +142,23 @@ public Schema load(TableId tableId) {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-24 Thread via GitHub
leonardBang merged PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-23 Thread via GitHub
lvyanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2072115980 > There is an error in the scenario of using route from CI, and I am trying to fix it. Done fix. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-23 Thread via GitHub
lvyanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2072051290 There is an error in the scenario of using route from CI, and I am trying to fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-22 Thread via GitHub
lvyanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2071458830 @leonardBang Can you help to start a CI workflow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-17 Thread via GitHub
lvyanquan commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1568374817 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/pipeline/PipelineOptions.java: ## @@ -85,5 +89,12 @@ public class PipelineOptions {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-16 Thread via GitHub
leonardBang commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1567483688 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/pipeline/PipelineOptions.java: ## @@ -85,5 +89,12 @@ public class PipelineOptions {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
lvyanquan commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1554820774 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/pipeline/PipelineOptions.java: ## @@ -85,5 +89,12 @@ public class PipelineOptions {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
leonardBang commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1554795627 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/pipeline/PipelineOptions.java: ## @@ -85,5 +89,12 @@ public class PipelineOptions {

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
leonardBang commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1554795438 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -127,4 +136,13 @@ RESPONSE

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
lvyanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2041034979 > Thanks @lvyanquan for the fix, I left some comments. And I think tests is necessary, e.g. a test mocked RPC timeout when processing schema change event is welcome. Add new

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
lvyanquan commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1554559194 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -127,4 +136,13 @@ RESPONSE

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-06 Thread via GitHub
lvyanquan commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1554559016 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -111,8 +115,13 @@ private SchemaChangeResponse

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-04-03 Thread via GitHub
leonardBang commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1549367497 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/SchemaOperator.java: ## @@ -111,8 +115,13 @@ private SchemaChangeResponse

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-18 Thread via GitHub
LYanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2003238562 > Should we clear the flushedSinkWriters as well? Thanks for this suggestion, address it. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-18 Thread via GitHub
LYanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2003214710 @fanqiejiang8 Hi, I have added `getRuntimeContext().getIndexOfThisSubtask() == 0` check to make sure that it won't send many times. -- This is an automated message from the Apache

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-18 Thread via GitHub
BaoPiao commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2003029727 > @BaoPiao Yeah, this is because SchemaRegistryRequestHandler doesn't clear pendingSchemaChanges when SchemaOperator restarted, I've added one commit

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-18 Thread via GitHub
yanghuaiGit commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2003023705 如果schemaRegister的pending里有多个待执行schema事件,isSchemaChangeApplying代表的是正在执行schema变更事件,但是无法得知是哪个schema事件执行,isSchemaChangeApplying无法和客户端的事件绑定,导致schemaoperator可能需要等待所有的schema事件执行完才会转为非阻塞

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-17 Thread via GitHub
fanqiejiang8 commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2002855730 @lvyanquan @BaoPiao When task parallelism is multiple, task retry all sent SchemaRegistry task more the RefreshPendingListsRequest request, that each request will not cause data

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-17 Thread via GitHub
LYanquan commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-2002529227 @BaoPiao Yeah, this is because SchemaRegistryRequestHandler doesn't clear pendingSchemaChanges when SchemaOperator restarted, I've added one commit

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-14 Thread via GitHub
BaoPiao commented on PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#issuecomment-1996731322 If the SchemaRegistryRequestHandler is in the midst of executing a scheme change, and the task manager where the SchemaOperator resides undergoes a restart, could this potentially

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-14 Thread via GitHub
lvyanquan commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1524312973 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/coordinator/SchemaRegistryRequestHandler.java: ## @@ -70,6 +81,8 @@ public

Re: [PR] [FLINK-34648] add waitChangeResultRequest and WaitChangeResultResponse to avoid RPC timeout. [flink-cdc]

2024-03-13 Thread via GitHub
ruanhang1993 commented on code in PR #3128: URL: https://github.com/apache/flink-cdc/pull/3128#discussion_r1524276055 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/coordinator/SchemaRegistryRequestHandler.java: ## @@ -70,6 +81,8 @@ public