Re: [PR] [FLINK-34180][docs] Migrate doc website from ververica to flink [flink-web]

2024-03-08 Thread via GitHub
PatrickRen commented on code in PR #722: URL: https://github.com/apache/flink-web/pull/722#discussion_r1517379722 ## docs/content.zh/getting-started/with-flink-cdc.md: ## @@ -0,0 +1,27 @@ +--- +weight: 6 +title: With Flink CDC +bookHref: "https://nightlies.apache.org/flink/flin

[jira] [Created] (FLINK-34623) Flink creating main.jar files in jobmanager jars upload folder

2024-03-08 Thread Nikhil_D (Jira)
Nikhil_D created FLINK-34623: Summary: Flink creating main.jar files in jobmanager jars upload folder Key: FLINK-34623 URL: https://issues.apache.org/jira/browse/FLINK-34623 Project: Flink Issue

[jira] [Updated] (FLINK-34623) Flink creating main.jar files in jobmanager jars upload folder

2024-03-08 Thread Nikhil_D (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil_D updated FLINK-34623: - Description: Flink is creating a main.jar and uploading it to jars upload folder. This would cause lot

[jira] [Commented] (FLINK-31663) Add ARRAY_EXCEPT supported in SQL & Table API

2024-03-08 Thread Dawid Wysakowicz (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824656#comment-17824656 ] Dawid Wysakowicz commented on FLINK-31663: -- [~jackylau] Please see the discussi

Re: [PR] [FLINK-34180][docs] Migrate doc website from ververica to flink [flink-web]

2024-03-08 Thread via GitHub
GOODBOY008 commented on code in PR #722: URL: https://github.com/apache/flink-web/pull/722#discussion_r1517395740 ## docs/content.zh/getting-started/training-course.md: ## @@ -1,5 +1,5 @@ --- -weight: 6 Review Comment: @PatrickRen **Weight** will effect sort. I insert befo

Re: [PR] [FLINK-34566] Pass a FixedThreadPool to set reconciliation parallelism correctly [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
ysymi commented on code in PR #790: URL: https://github.com/apache/flink-kubernetes-operator/pull/790#discussion_r1517409537 ## flink-kubernetes-operator/src/test/java/org/apache/flink/kubernetes/operator/FlinkOperatorTest.java: ## @@ -70,10 +70,21 @@ public void testConfigurat

Re: [PR] [FLINK-34622][docs] fix typo in execution_mode.md [flink]

2024-03-08 Thread via GitHub
Myasuka merged PR #24467: URL: https://github.com/apache/flink/pull/24467 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [FLINK-34617][docs] Correct the Javadoc of org.apache.flink.api.common.time.Time [flink]

2024-03-08 Thread via GitHub
Myasuka merged PR #24464: URL: https://github.com/apache/flink/pull/24464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [FLINK-34484][state] Split 'state.backend.local-recovery' into two options for checkpointing and recovery [flink]

2024-03-08 Thread via GitHub
Zakelly commented on code in PR #24402: URL: https://github.com/apache/flink/pull/24402#discussion_r1517422921 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java: ## @@ -219,6 +219,18 @@ public TaskStateSnapshot retrieveLocalState(long ch

[jira] [Resolved] (FLINK-34622) Typo of execution_mode configuration name in Chinese document

2024-03-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-34622. -- Fix Version/s: 1.18.2 1.20.0 1.19.1 Assignee: Yu Chen

[jira] [Resolved] (FLINK-34617) Correct the Javadoc of org.apache.flink.api.common.time.Time

2024-03-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-34617. -- Fix Version/s: 1.20.0 1.19.1 Resolution: Fixed merged master: 9617598de33b2b

Re: [PR] [FLINK-34566] Pass a FixedThreadPool to set reconciliation parallelism correctly [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
gyfora commented on code in PR #790: URL: https://github.com/apache/flink-kubernetes-operator/pull/790#discussion_r1517465337 ## flink-kubernetes-operator/src/test/java/org/apache/flink/kubernetes/operator/FlinkOperatorTest.java: ## @@ -70,10 +70,21 @@ public void testConfigura

Re: [PR] [FLINK-34483][docs] Improve the documentation of 'state.checkpoints.dir' and 'state.checkpoint-storage' [flink]

2024-03-08 Thread via GitHub
fredia commented on PR #24401: URL: https://github.com/apache/flink/pull/24401#issuecomment-1985348174 Thanks all for the review ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [FLINK-34483][docs] Improve the documentation of 'state.checkpoints.dir' and 'state.checkpoint-storage' [flink]

2024-03-08 Thread via GitHub
fredia merged PR #24401: URL: https://github.com/apache/flink/pull/24401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

[jira] [Commented] (FLINK-34483) Improve the documentation of 'state.checkpoints.dir' and 'state.checkpoint-storage'

2024-03-08 Thread Yanfei Lei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824683#comment-17824683 ] Yanfei Lei commented on FLINK-34483: Merged 1d8cbe9 into master. > Improve the docu

[jira] [Closed] (FLINK-34483) Improve the documentation of 'state.checkpoints.dir' and 'state.checkpoint-storage'

2024-03-08 Thread Yanfei Lei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanfei Lei closed FLINK-34483. -- > Improve the documentation of 'state.checkpoints.dir' and > 'state.checkpoint-storage' > ---

[jira] [Resolved] (FLINK-34483) Improve the documentation of 'state.checkpoints.dir' and 'state.checkpoint-storage'

2024-03-08 Thread Yanfei Lei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanfei Lei resolved FLINK-34483. Resolution: Resolved > Improve the documentation of 'state.checkpoints.dir' and > 'state.checkpoi

[jira] [Commented] (FLINK-31663) Add ARRAY_EXCEPT supported in SQL & Table API

2024-03-08 Thread Jacky Lau (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824686#comment-17824686 ] Jacky Lau commented on FLINK-31663: --- [~dwysakowicz] i see, thanks for your response.

[jira] [Commented] (FLINK-34494) Migrate ReplaceIntersectWithSemiJoinRule

2024-03-08 Thread Jacky Lau (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824687#comment-17824687 ] Jacky Lau commented on FLINK-34494: --- hi [~dwysakowicz] will you help review this? > M

[jira] [Commented] (FLINK-31663) Add ARRAY_EXCEPT supported in SQL & Table API

2024-03-08 Thread Dawid Wysakowicz (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824688#comment-17824688 ] Dawid Wysakowicz commented on FLINK-31663: -- I don't see how `array_union` is re

Re: [PR] [FLINK-33668] Decoupling Shuffle network memory and job topology [flink]

2024-03-08 Thread via GitHub
reswqa merged PR #23851: URL: https://github.com/apache/flink/pull/23851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

[jira] [Resolved] (FLINK-33668) Decoupling Shuffle network memory and job topology

2024-03-08 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo resolved FLINK-33668. Fix Version/s: 1.20.0 (was: 1.19.0) Resolution: Done master(1.20) vi

[jira] [Assigned] (FLINK-33668) Decoupling Shuffle network memory and job topology

2024-03-08 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo reassigned FLINK-33668: -- Assignee: Jiang Xin > Decoupling Shuffle network memory and job topology > --

[jira] [Closed] (FLINK-33668) Decoupling Shuffle network memory and job topology

2024-03-08 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-33668. -- > Decoupling Shuffle network memory and job topology > --

[jira] [Closed] (FLINK-33004) Decoupling topology and network memory to support complex job topologies

2024-03-08 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-33004. -- Resolution: Duplicate > Decoupling topology and network memory to support complex job topologies > ---

[jira] [Commented] (FLINK-31663) Add ARRAY_EXCEPT supported in SQL & Table API

2024-03-08 Thread Jacky Lau (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824699#comment-17824699 ] Jacky Lau commented on FLINK-31663: --- [~dwysakowicz] Even though they are different fun

[PR] Dev 0306 2025 [flink]

2024-03-08 Thread via GitHub
WencongLiu opened a new pull request, #24468: URL: https://github.com/apache/flink/pull/24468 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them

Re: [PR] Dev 0306 2025 [flink]

2024-03-08 Thread via GitHub
flinkbot commented on PR #24468: URL: https://github.com/apache/flink/pull/24468#issuecomment-1985399113 ## CI report: * 2db314a32a656a2bf2a768428770799a545422bb UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-34571][test] Fix flaky test `SortMergeResultPartitionReadSchedulerTest.testOnReadBufferRequestError` [flink]

2024-03-08 Thread via GitHub
reswqa opened a new pull request, #24469: URL: https://github.com/apache/flink/pull/24469 ## What is the purpose of the change *We have a race condition in `SortMergeResultPartitionReadSchedulerTest.testOnReadBufferRequestError`, sometimes the reader finish reading before buffer pool

[jira] [Updated] (FLINK-34571) SortMergeResultPartitionReadSchedulerTest.testOnReadBufferRequestError failed due an assertion

2024-03-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34571: --- Labels: github-actions pull-request-available test-stability (was: github-actions test-stab

Re: [PR] [FLINK-34571][test] Fix flaky test `SortMergeResultPartitionReadSchedulerTest.testOnReadBufferRequestError` [flink]

2024-03-08 Thread via GitHub
flinkbot commented on PR #24469: URL: https://github.com/apache/flink/pull/24469#issuecomment-1985416994 ## CI report: * f73314da56535ff2ed9e148ab7266040287fb5da UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-34566] Pass a FixedThreadPool to set reconciliation parallelism correctly [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
gyfora merged PR #790: URL: https://github.com/apache/flink-kubernetes-operator/pull/790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr

[jira] [Updated] (FLINK-34566) Flink Kubernetes Operator reconciliation parallelism setting not work

2024-03-08 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora updated FLINK-34566: --- Fix Version/s: kubernetes-operator-1.8.0 > Flink Kubernetes Operator reconciliation parallelism sett

[jira] [Closed] (FLINK-34566) Flink Kubernetes Operator reconciliation parallelism setting not work

2024-03-08 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-34566. -- Resolution: Fixed merged to main 726e484c6a9b4121563829bc094b3eebeb8ddcf3 > Flink Kubernetes Operator

[jira] [Created] (FLINK-34624) Enable local recovery in ChangelogRescalingITCase

2024-03-08 Thread Yanfei Lei (Jira)
Yanfei Lei created FLINK-34624: -- Summary: Enable local recovery in ChangelogRescalingITCase Key: FLINK-34624 URL: https://issues.apache.org/jira/browse/FLINK-34624 Project: Flink Issue Type: Tec

Re: [PR] [FLINK-34524] Scale down JM deployment to 0 before deletion [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
mateczagany commented on code in PR #791: URL: https://github.com/apache/flink-kubernetes-operator/pull/791#discussion_r1517507683 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/service/AbstractFlinkService.java: ## @@ -1134,4 +1141,28 @@ privat

[jira] [Commented] (FLINK-29114) TableSourceITCase#testTableHintWithLogicalTableScanReuse sometimes fails with result mismatch

2024-03-08 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/FLINK-29114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824724#comment-17824724 ] Ryan Skraba commented on FLINK-29114: - * [https://dev.azure.com/apache-flink/apache

[jira] [Commented] (FLINK-31472) AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread

2024-03-08 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824726#comment-17824726 ] Ryan Skraba commented on FLINK-31472: - * [https://github.com/apache/flink/actions/r

Re: [PR] [FLINK-34537] Autoscaler JDBC Support HikariPool [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
czy006 commented on PR #785: URL: https://github.com/apache/flink-kubernetes-operator/pull/785#issuecomment-1985477162 > Thanks @czy006 for the improvement! > > I'm on vacation recently, I will review it after it. Sorry for the delay. > > Btw, `1.8.0` will be released soon, wou

[jira] [Updated] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-34581: --- Priority: Critical (was: Blocker) > streaming code throws java.lang.reflect.InaccessibleObj

[jira] [Commented] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824731#comment-17824731 ] Martijn Visser commented on FLINK-34581: I've downgraded this from Blocker to Ma

[jira] [Updated] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-34581: --- Component/s: Runtime / Checkpointing (was: API / DataStream) > streamin

[jira] [Updated] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-34581: --- Priority: Major (was: Critical) > streaming code throws java.lang.reflect.InaccessibleObjec

[jira] [Commented] (FLINK-18476) PythonEnvUtilsTest#testStartPythonProcess fails

2024-03-08 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824732#comment-17824732 ] Ryan Skraba commented on FLINK-18476: - https://github.com/apache/flink/actions/runs/

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824733#comment-17824733 ] Martijn Visser commented on FLINK-34554: How did you migrate from the FlinkKafka

[jira] [Updated] (FLINK-34624) Enable local recovery in ChangelogRescalingITCase

2024-03-08 Thread Yanfei Lei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanfei Lei updated FLINK-34624: --- Description: Enable local recovery in ChangelogRescalingITCase, since the local recovery of changelo

[PR] [FLINK-34624][state/changelog] Enable local recovery in ChangelogRescalingITCase [flink]

2024-03-08 Thread via GitHub
fredia opened a new pull request, #24470: URL: https://github.com/apache/flink/pull/24470 ## What is the purpose of the change Enable local recovery in ChangelogRescalingITCase, since the local recovery of changelog state backend was supported by [FLINK-27693](https://issues.

[jira] [Updated] (FLINK-34624) Enable local recovery in ChangelogRescalingITCase

2024-03-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34624: --- Labels: pull-request-available (was: ) > Enable local recovery in ChangelogRescalingITCase

Re: [PR] [FLINK-34624][state/changelog] Enable local recovery in ChangelogRescalingITCase [flink]

2024-03-08 Thread via GitHub
fredia commented on PR #24470: URL: https://github.com/apache/flink/pull/24470#issuecomment-1985501915 @Zakelly @masteryhx Would you help to take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824737#comment-17824737 ] Hilmi Al Fatih commented on FLINK-34554: Hi [~martijnvisser] , yes. We stop with

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-34554: --- Priority: Major (was: Blocker) > Using EXACTLY_ONCE with KafkaSink cause broker's OOM due t

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-34554: --- Fix Version/s: (was: 1.16.3) (was: 1.17.2)

Re: [PR] [FLINK-34624][state/changelog] Enable local recovery in ChangelogRescalingITCase [flink]

2024-03-08 Thread via GitHub
flinkbot commented on PR #24470: URL: https://github.com/apache/flink/pull/24470#issuecomment-1985509794 ## CI report: * 801073e943446b953a394bd27527f5246bf97ef9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824742#comment-17824742 ] Martijn Visser commented on FLINK-34554: I think it will be hard to find somethi

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824748#comment-17824748 ] Hilmi Al Fatih commented on FLINK-34554: [~martijnvisser] IIUC my situation corr

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824749#comment-17824749 ] Hilmi Al Fatih commented on FLINK-34554: > The only thing that you could try is

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824750#comment-17824750 ] Martijn Visser commented on FLINK-34554: IIRC it was a deliberate choice to not

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824752#comment-17824752 ] Martijn Visser commented on FLINK-34554: {quote}> The only thing that you could

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824757#comment-17824757 ] Hilmi Al Fatih commented on FLINK-34554: {quote}You're not using the latest vers

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824757#comment-17824757 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:49 AM: -

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824758#comment-17824758 ] Martijn Visser commented on FLINK-34554: You can definitely stick with FlinkKafk

[jira] [Created] (FLINK-34625) TTL doesn't seem to work in pyflink

2024-03-08 Thread Mark Lidenberg (Jira)
Mark Lidenberg created FLINK-34625: -- Summary: TTL doesn't seem to work in pyflink Key: FLINK-34625 URL: https://issues.apache.org/jira/browse/FLINK-34625 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih commented on FLINK-34554: Honestly I am afraid if you will remove

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:58 AM: -

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:58 AM: -

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824766#comment-17824766 ] Martijn Visser commented on FLINK-34554: The problem is that I can't be sure tha

[jira] [Commented] (FLINK-34625) TTL doesn't seem to work in pyflink

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824767#comment-17824767 ] Martijn Visser commented on FLINK-34625: [~dianfu] [~hxbks2ks] Any thoughts? >

Re: [PR] [FLINK-34417] Log Job ID via MDC [flink]

2024-03-08 Thread via GitHub
rkhachatryan commented on PR #24292: URL: https://github.com/apache/flink/pull/24292#issuecomment-1985585478 Re-ran benchmarks - no regressions found: [jmh-result-runtime.csv](https://github.com/apache/flink/files/14537980/jmh-result-runtime.csv) [jmh-result-scheduler.csv](https://git

[jira] [Created] (FLINK-34626) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
WangMinChao created FLINK-34626: --- Summary: Pulsar source lost topic subscribe Key: FLINK-34626 URL: https://issues.apache.org/jira/browse/FLINK-34626 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-34628) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
WangMinChao created FLINK-34628: --- Summary: Pulsar source lost topic subscribe Key: FLINK-34628 URL: https://issues.apache.org/jira/browse/FLINK-34628 Project: Flink Issue Type: Bug Co

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih commented on FLINK-34554: I see. Let me get back to you by reprod

[jira] [Created] (FLINK-34627) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
WangMinChao created FLINK-34627: --- Summary: Pulsar source lost topic subscribe Key: FLINK-34627 URL: https://issues.apache.org/jira/browse/FLINK-34627 Project: Flink Issue Type: Bug Co

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 12:19 PM: -

[jira] [Created] (FLINK-34629) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
WangMinChao created FLINK-34629: --- Summary: Pulsar source lost topic subscribe Key: FLINK-34629 URL: https://issues.apache.org/jira/browse/FLINK-34629 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-34630) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
WangMinChao created FLINK-34630: --- Summary: Pulsar source lost topic subscribe Key: FLINK-34630 URL: https://issues.apache.org/jira/browse/FLINK-34630 Project: Flink Issue Type: Bug Co

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

2024-03-08 Thread Hilmi Al Fatih (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 12:21 PM: -

[jira] [Updated] (FLINK-34623) Flink creating main.jar files in jobmanager jars upload folder

2024-03-08 Thread Nikhil_D (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil_D updated FLINK-34623: - Attachment: image-2024-03-08-17-59-25-982.png > Flink creating main.jar files in jobmanager jars upload

[jira] [Updated] (FLINK-34623) Flink creating main.jar files in jobmanager jars upload folder

2024-03-08 Thread Nikhil_D (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil_D updated FLINK-34623: - Description: Flink is creating a main.jar and uploading it to jars upload folder. This would cause lot

Re: [PR] [FLINK-34524] Scale down JM deployment to 0 before deletion [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
gyfora commented on PR #791: URL: https://github.com/apache/flink-kubernetes-operator/pull/791#issuecomment-1985612180 > Added some comments, but I really like this change, I think it makes a lot of sense to let the JMs shut down the TMs themselves in native mode. > > But I wonder, h

[jira] [Updated] (FLINK-34623) Flink creating main.jar files in jobmanager jars upload folder

2024-03-08 Thread Nikhil_D (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil_D updated FLINK-34623: - Description: Flink is creating a main.jar and uploading it to jars upload folder. This would cause lot

[PR] Fix non-partition topic subscribe lost. [flink-connector-pulsar]

2024-03-08 Thread via GitHub
minchowang opened a new pull request, #84: URL: https://github.com/apache/flink-connector-pulsar/pull/84 ## Purpose of the change *We subscribe to multiple non-partition topics the part of topics may subscribe lost when `startIndex` equals 0* ## Brief change log

Re: [PR] Fix non-partition topic subscribe lost. [flink-connector-pulsar]

2024-03-08 Thread via GitHub
boring-cyborg[bot] commented on PR #84: URL: https://github.com/apache/flink-connector-pulsar/pull/84#issuecomment-1985615031 Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) -- This is

[jira] [Commented] (FLINK-34629) Pulsar source lost topic subscribe

2024-03-08 Thread WangMinChao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824771#comment-17824771 ] WangMinChao commented on FLINK-34629: - Fix https://github.com/apache/flink-connector

Re: [PR] Fix non-partition topic subscribe lost. [flink-connector-pulsar]

2024-03-08 Thread via GitHub
minchowang commented on PR #84: URL: https://github.com/apache/flink-connector-pulsar/pull/84#issuecomment-1985637571 If `startIndex` = 0, `partitionId` = -1, and results of plus equals `-1`, this value as `readerId` cannot match `pendingReaders`, leading to cannot be assigned for one `p

[jira] [Closed] (FLINK-34629) Pulsar source lost topic subscribe

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-34629. -- Resolution: Duplicate > Pulsar source lost topic subscribe > -

[jira] [Closed] (FLINK-34628) Pulsar source lost topic subscribe

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-34628. -- Resolution: Duplicate > Pulsar source lost topic subscribe > -

[jira] [Closed] (FLINK-34630) Pulsar source lost topic subscribe

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-34630. -- Resolution: Duplicate > Pulsar source lost topic subscribe > -

[jira] [Closed] (FLINK-34626) Pulsar source lost topic subscribe

2024-03-08 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser closed FLINK-34626. -- Resolution: Duplicate > Pulsar source lost topic subscribe > -

Re: [PR] [FLINK-34417] Log Job ID via MDC [flink]

2024-03-08 Thread via GitHub
rkhachatryan merged PR #24292: URL: https://github.com/apache/flink/pull/24292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

[jira] [Commented] (FLINK-34417) Add JobID to logging MDC

2024-03-08 Thread Roman Khachatryan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824806#comment-17824806 ] Roman Khachatryan commented on FLINK-34417: --- Merged into master as d6a4eb966fb

[jira] [Closed] (FLINK-34417) Add JobID to logging MDC

2024-03-08 Thread Roman Khachatryan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Khachatryan closed FLINK-34417. - Resolution: Fixed > Add JobID to logging MDC > > >

Re: [PR] feat: autoscaling decision parallelism improvement [FLINK-34563] [flink-kubernetes-operator]

2024-03-08 Thread via GitHub
gyfora commented on PR #787: URL: https://github.com/apache/flink-kubernetes-operator/pull/787#issuecomment-1985808993 I have some concerns about this change: 1. It doesn't work with custom slot sharing configuration which is very common 2. It provides almost no benefit with large t

Re: [PR] Add announcement blog post for Flink 1.19 [flink-web]

2024-03-08 Thread via GitHub
lincoln-lil commented on code in PR #721: URL: https://github.com/apache/flink-web/pull/721#discussion_r1517781917 ## docs/content/posts/2024-03-xx-release-1.19.0.md: ## @@ -0,0 +1,466 @@ +--- +authors: +- LincolnLee: + name: "Lincoln Lee" + twitter: lincoln_86xy + +date: "202

[PR] [FLINK-20398][e2e] Migrate test_batch_sql.sh to Java e2e tests framework [flink]

2024-03-08 Thread via GitHub
affo opened a new pull request, #24471: URL: https://github.com/apache/flink/pull/24471 ## What is the purpose of the change Migrate `test_batch_sql.sh` to end-to-end test frameworks. ## Brief change log - implement `BatchSQLTest` porting `test_batch_sql.sh` - fi

[jira] [Updated] (FLINK-20398) Migrate test_batch_sql.sh

2024-03-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20398: --- Labels: pull-request-available (was: ) > Migrate test_batch_sql.sh > --

[jira] [Created] (FLINK-34631) Memory leak in pyflink when using state with RocksDB

2024-03-08 Thread Mark Lidenberg (Jira)
Mark Lidenberg created FLINK-34631: -- Summary: Memory leak in pyflink when using state with RocksDB Key: FLINK-34631 URL: https://issues.apache.org/jira/browse/FLINK-34631 Project: Flink Iss

Re: [PR] [FLINK-20398][e2e] Migrate test_batch_sql.sh to Java e2e tests framework [flink]

2024-03-08 Thread via GitHub
flinkbot commented on PR #24471: URL: https://github.com/apache/flink/pull/24471#issuecomment-1985868527 ## CI report: * 35acd12f6acdd8c132f3dbc365c5c24549b7bc18 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [hotfix] A couple of hotfixes related to e2e tests [flink]

2024-03-08 Thread via GitHub
pnowojski opened a new pull request, #24472: URL: https://github.com/apache/flink/pull/24472 ## What is the purpose of the change Fixing a couple of smaller issues. ## Brief change log Please check individual commit messages. ## Verifying this change This ch

Re: [PR] [hotfix] A couple of hotfixes related to e2e tests [flink]

2024-03-08 Thread via GitHub
flinkbot commented on PR #24472: URL: https://github.com/apache/flink/pull/24472#issuecomment-1985886384 ## CI report: * 1715f47deaa42dd31cb35298348c69bee811d09e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-34631) Memory leak in pyflink when using state with RocksDB

2024-03-08 Thread Mark Lidenberg (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Lidenberg updated FLINK-34631: --- Description: I have had issues with memory constantly growing on pyflink task managers, whi

  1   2   >