[jira] [Created] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to RPC issue

2024-01-24 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34225: - Summary: HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to RPC issue Key: FLINK-34225 URL: https://issues.apache.org/jira/browse/FLINK-34225 Pro

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to RPC issue

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Attachment: FLINK-34225.log > HybridShuffleITCase.testHybridSelectiveExchangesRestart failed d

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Summary: HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerExce

[jira] [Commented] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810252#comment-17810252 ] Matthias Pohl commented on FLINK-34225: --- I attached the logs of the test failure t

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Component/s: Runtime / Network (was: Runtime / Coordination) > HybridShuf

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Priority: Blocker (was: Critical) > HybridShuffleITCase.testHybridSelectiveExchangesRestart f

[jira] [Commented] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810254#comment-17810254 ] Matthias Pohl commented on FLINK-34225: --- I'm raising this as a blocker because it'

[jira] [Comment Edited] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810254#comment-17810254 ] Matthias Pohl edited comment on FLINK-34225 at 1/24/24 8:15 AM: --

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Description: This test failed in a master nightly workflow run in GitHub Actions ([FLIP-396|h

[jira] [Updated] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34225: -- Description: This test failed in a master nightly workflow run in GitHub Actions ([FLIP-396|h

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
JunRuiLee commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1907622959 @mayuehappy I noticed that the creation of the CheckpointCoordinator currently depends on the isCheckpointingEnabled condition. This implies that if we disable checkpointing, the entire

[jira] [Assigned] (FLINK-34007) Flink Job stuck in suspend state after losing leadership in HA Mode

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl reassigned FLINK-34007: - Assignee: Matthias Pohl > Flink Job stuck in suspend state after losing leadership in H

Re: [PR] [FLINK-34015][checkpoint] fix that passing ignore-unclaimed-state through dynamic props does not take effect [flink]

2024-01-24 Thread via GitHub
xiangforever2014 commented on code in PR #24058: URL: https://github.com/apache/flink/pull/24058#discussion_r1464515517 ## flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettings.java: ## @@ -163,14 +163,16 @@ public static SavepointRestoreSettings

Re: [PR] [FLINK-34015][checkpoint] fix that passing ignore-unclaimed-state through dynamic props does not take effect [flink]

2024-01-24 Thread via GitHub
xiangforever2014 commented on PR #24058: URL: https://github.com/apache/flink/pull/24058#issuecomment-1907643393 > Thanks for the pr. PTAL my comments. BTW, Could we also provide a UT to verify this ? Sorry to reply so late, I have refactor the code according to your comments, and ad

[jira] [Comment Edited] (FLINK-34007) Flink Job stuck in suspend state after losing leadership in HA Mode

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810038#comment-17810038 ] Matthias Pohl edited comment on FLINK-34007 at 1/24/24 8:37 AM: --

Re: [PR] [FLINK-34104][autoscaler] Improve the ScalingReport format of autoscaling [flink-kubernetes-operator]

2024-01-24 Thread via GitHub
1996fanrui commented on code in PR #757: URL: https://github.com/apache/flink-kubernetes-operator/pull/757#discussion_r1464522145 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoscalerEventUtils.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache So

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
mayuehappy commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1907651213 @JunRuiLee Thanks for joining the disscussion . > the PR skips the creation of the checkpoint coordinator when disabling the checkpoint scheduling Yes, actually I think that's

Re: [PR] [FLINK-24024][table-planner] support session window tvf in plan [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on PR #23505: URL: https://github.com/apache/flink/pull/23505#issuecomment-1907653922 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[jira] [Commented] (FLINK-34225) HybridShuffleITCase.testHybridSelectiveExchangesRestart failed due to NullPointerException

2024-01-24 Thread Yunfeng Zhou (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810280#comment-17810280 ] Yunfeng Zhou commented on FLINK-34225: -- Hi [~mapohl] thanks for raising this bug. I

[jira] [Commented] (FLINK-33683) FLIP-407 Improve Flink Client performance in interactive scenarios

2024-01-24 Thread xiangyu feng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810282#comment-17810282 ] xiangyu feng commented on FLINK-33683: -- [~guoyangze]  Hi, this Flip(https://cwiki.

[jira] [Assigned] (FLINK-33683) FLIP-407 Improve Flink Client performance in interactive scenarios

2024-01-24 Thread Yangze Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yangze Guo reassigned FLINK-33683: -- Assignee: xiangyu feng > FLIP-407 Improve Flink Client performance in interactive scenarios >

Re: [PR] [FLINK-34117][bugfix][table] Fix CompactCoordinator data loss upon termination [flink]

2024-01-24 Thread via GitHub
leonardBang merged PR #24120: URL: https://github.com/apache/flink/pull/24120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

[jira] [Updated] (FLINK-34117) CompactCoordinator for table file sink loses data upon job termination

2024-01-24 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-34117: --- Affects Version/s: 1.18.1 > CompactCoordinator for table file sink loses data upon job termination >

[jira] [Updated] (FLINK-34117) CompactCoordinator for table file sink loses data upon job termination

2024-01-24 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-34117: --- Component/s: Connectors / FileSystem > CompactCoordinator for table file sink loses data upon job te

[jira] [Updated] (FLINK-34117) CompactCoordinator for table file sink loses data upon job termination

2024-01-24 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-34117: --- Fix Version/s: 1.19.0 > CompactCoordinator for table file sink loses data upon job termination > ---

[jira] [Created] (FLINK-34226) Add the backoff-multiplier configuration in ExponentialBackoffRetryStrategy

2024-01-24 Thread xiangyu feng (Jira)
xiangyu feng created FLINK-34226: Summary: Add the backoff-multiplier configuration in ExponentialBackoffRetryStrategy Key: FLINK-34226 URL: https://issues.apache.org/jira/browse/FLINK-34226 Project:

[jira] [Commented] (FLINK-34117) CompactCoordinator for table file sink loses data upon job termination

2024-01-24 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810283#comment-17810283 ] Leonard Xu commented on FLINK-34117: master: 1d0f34e5a4a9127724bd09ecd0d9c0c0dba431f

Re: [PR] [FLINK-34120] Introduce unified serialization config option for all Kryo, POJO and customized serializers [flink]

2024-01-24 Thread via GitHub
JunRuiLee commented on code in PR #24182: URL: https://github.com/apache/flink/pull/24182#discussion_r1464545618 ## flink-core/src/main/java/org/apache/flink/api/common/serialization/SerializerConfig.java: ## @@ -408,4 +431,104 @@ private T loadClass( throw new Ill

Re: [PR] [FLINK-34111][table] Add JSON_QUOTE and JSON_UNQUOTE function [flink]

2024-01-24 Thread via GitHub
xjmdoo commented on code in PR #24156: URL: https://github.com/apache/flink/pull/24156#discussion_r1464565498 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/SqlFunctionUtils.java: ## @@ -1174,6 +1174,107 @@ public static String toBase64

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
WencongLiu commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1907709816 @mayuehappy The key point is that if the users set the job with streaming execution mode and set the checkpoint-interval with Long.MAX_VALUE, the savepoint will be not supported at all

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
JunRuiLee commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1907724157 > @JunRuiLee Thanks for joining the disscussion . > > > the PR skips the creation of the checkpoint coordinator when disabling the checkpoint scheduling > > Yes, actually I

Re: [PR] [FLINK-34120] Introduce unified serialization config option for all Kryo, POJO and customized serializers [flink]

2024-01-24 Thread via GitHub
reswqa commented on code in PR #24182: URL: https://github.com/apache/flink/pull/24182#discussion_r1464593996 ## flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java: ## @@ -255,6 +256,51 @@ public class PipelineOptions {

Re: [PR] [FLINK-34209] Migrate FileSink to the new SinkV2 API [flink]

2024-01-24 Thread via GitHub
pvary commented on code in PR #24180: URL: https://github.com/apache/flink/pull/24180#discussion_r1464617858 ## flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java: ## @@ -307,6 +313,14 @@ public AsyncSinkWriter(

Re: [PR] [FLINK-34209] Migrate FileSink to the new SinkV2 API [flink]

2024-01-24 Thread via GitHub
pvary commented on PR #24180: URL: https://github.com/apache/flink/pull/24180#issuecomment-1907753610 The `AsyncFlinkWriter` changes are not covered by the FLIP-372, but they are need to follow the same deprecation patterns for the writer interface so the API is consistent. -- This i

Re: [PR] [FLINK-34208] Migrate SinkV1Adapter to the new SinkV2 API [flink]

2024-01-24 Thread via GitHub
gyfora merged PR #24174: URL: https://github.com/apache/flink/pull/24174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

[jira] [Closed] (FLINK-34208) Migrate SinkV1Adapter to the new SinkV2 API

2024-01-24 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-34208. -- Fix Version/s: 1.19.0 Assignee: Peter Vary Resolution: Fixed merged to master 2cf213ec

Re: [PR] [FLINK-23687][table-planner] Introduce partitioned lookup join to enforce input of LookupJoin to hash shuffle by lookup keys [flink]

2024-01-24 Thread via GitHub
LB-Yu commented on code in PR #24104: URL: https://github.com/apache/flink/pull/24104#discussion_r1464624954 ## flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/physical/batch/BatchPhysicalLookupJoinRule.scala: ## @@ -73,7 +75,19 @@ object

Re: [PR] [FLINK-23687][table-planner] Introduce partitioned lookup join to enforce input of LookupJoin to hash shuffle by lookup keys [flink]

2024-01-24 Thread via GitHub
LB-Yu commented on code in PR #24104: URL: https://github.com/apache/flink/pull/24104#discussion_r1464627984 ## docs/content/docs/dev/table/sql/queries/joins.md: ## @@ -326,6 +326,17 @@ FROM Orders AS o In the example above, the Orders table is enriched with data from the Cu

Re: [PR] [FLINK-23687][table-planner] Introduce partitioned lookup join to enforce input of LookupJoin to hash shuffle by lookup keys [flink]

2024-01-24 Thread via GitHub
LB-Yu commented on code in PR #24104: URL: https://github.com/apache/flink/pull/24104#discussion_r1464628624 ## docs/content/docs/dev/table/sql/queries/joins.md: ## @@ -326,6 +326,17 @@ FROM Orders AS o In the example above, the Orders table is enriched with data from the Cu

Re: [PR] [FLINK-24024][table-planner] support session window tvf in plan [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on PR #23505: URL: https://github.com/apache/flink/pull/23505#issuecomment-1907771837 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [FLINK-33263][table-planner] Implement ParallelismProvider for sources in the table planner [flink]

2024-01-24 Thread via GitHub
BIOINSu commented on code in PR #24128: URL: https://github.com/apache/flink/pull/24128#discussion_r1464639516 ## flink-streaming-java/src/main/java/org/apache/flink/streaming/api/transformations/SourceTransformationWrapper.java: ## @@ -0,0 +1,72 @@ +/* +Licensed to the Apache S

[jira] [Commented] (FLINK-34147) TimestampData to/from LocalDateTime is ambiguous

2024-01-24 Thread Yunhong Zheng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810300#comment-17810300 ] Yunhong Zheng commented on FLINK-34147: --- Hi, [~lirui] . I also think it ambiguous.

Re: [PR] [FLINK-33263][table-planner] Implement ParallelismProvider for sources in the table planner [flink]

2024-01-24 Thread via GitHub
BIOINSu commented on code in PR #24128: URL: https://github.com/apache/flink/pull/24128#discussion_r1464649772 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/TableScanTest.xml: ## @@ -727,4 +727,60 @@ Calc(select=[ts, a, b],

Re: [PR] [FLINK-28915] Support fetching artifacts in native K8s and standalone application mode [flink]

2024-01-24 Thread via GitHub
ferenc-csaky commented on code in PR #24065: URL: https://github.com/apache/flink/pull/24065#discussion_r1464653979 ## docs/layouts/shortcodes/generated/artifact_fetch_configuration.html: ## @@ -0,0 +1,36 @@ + + + +Key +Default +Ty

Re: [PR] [FLINK-34120] Introduce unified serialization config option for all Kryo, POJO and customized serializers [flink]

2024-01-24 Thread via GitHub
X-czh commented on code in PR #24182: URL: https://github.com/apache/flink/pull/24182#discussion_r1464658457 ## flink-core/src/main/java/org/apache/flink/api/common/serialization/SerializerConfig.java: ## @@ -352,6 +361,20 @@ public void configure(ReadableConfig configuration,

[jira] [Commented] (FLINK-34218) AutoRescalingITCase#testCheckpointRescalingInKeyedState fails

2024-01-24 Thread Stefan Richter (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810313#comment-17810313 ] Stefan Richter commented on FLINK-34218: Hi, I also cannot reproduce the problem

[PR] [WIP][FLINK-34204][core] Dispatch legacy serializer related methods in ExecutionConfig to SerializerConfig [flink]

2024-01-24 Thread via GitHub
reswqa opened a new pull request, #24187: URL: https://github.com/apache/flink/pull/24187 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on

[jira] [Updated] (FLINK-34204) Dispatch legacy serializer related methods in ExecutionConfig to SerializerConfig

2024-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34204: --- Labels: pull-request-available (was: ) > Dispatch legacy serializer related methods in Exec

[PR] [FLINK-32684][rpc] Introduces RpcOptions and deprecates AkkaOptions [flink]

2024-01-24 Thread via GitHub
XComp opened a new pull request, #24188: URL: https://github.com/apache/flink/pull/24188 ## What is the purpose of the change The name {{AkkaOptions}} is wrong after replacing Akka with Apache Pekko in FLINK-32468. The new options should be named more generically to avoid running int

[jira] [Updated] (FLINK-32684) Renaming AkkaOptions into PekkoOptions

2024-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32684: --- Labels: 2.0-related pull-request-available (was: 2.0-related) > Renaming AkkaOptions into P

Re: [PR] [WIP][FLINK-34204][core] Dispatch legacy serializer related methods in ExecutionConfig to SerializerConfig [flink]

2024-01-24 Thread via GitHub
flinkbot commented on PR #24187: URL: https://github.com/apache/flink/pull/24187#issuecomment-1907857611 ## CI report: * f2e9c8652fa9b9f71867d913ebc61843b1de03a7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-32684) Renaming AkkaOptions into RpcOptions

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-32684: -- Summary: Renaming AkkaOptions into RpcOptions (was: Renaming AkkaOptions into PekkoOptions)

[jira] [Assigned] (FLINK-32684) Renaming AkkaOptions into PekkoOptions

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl reassigned FLINK-32684: - Assignee: Matthias Pohl > Renaming AkkaOptions into PekkoOptions >

[jira] [Commented] (FLINK-32684) Renaming AkkaOptions into RpcOptions

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810319#comment-17810319 ] Matthias Pohl commented on FLINK-32684: --- I'm working on that one now. I decided ag

Re: [PR] [FLINK-34120] Introduce unified serialization config option for all Kryo, POJO and customized serializers [flink]

2024-01-24 Thread via GitHub
JunRuiLee commented on code in PR #24182: URL: https://github.com/apache/flink/pull/24182#discussion_r1464718228 ## flink-core/src/main/java/org/apache/flink/api/common/serialization/SerializerConfig.java: ## @@ -352,6 +361,20 @@ public void configure(ReadableConfig configuratio

Re: [PR] [FLINK-32684][rpc] Introduces RpcOptions and deprecates AkkaOptions [flink]

2024-01-24 Thread via GitHub
flinkbot commented on PR #24188: URL: https://github.com/apache/flink/pull/24188#issuecomment-1907874885 ## CI report: * 7e95ca97cb6a0b2d8f04c08dee403927efe77c8e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
mayuehappy commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1907878203 @JunRuiLee @WencongLiu Thanks for replying > In my opinion, disabling checkpoints does not necessarily equate to disabling savepoints or other functionalities of the Checkpoint

Re: [PR] [FLINK-34152] Tune heap memory of autoscaled jobs [flink-kubernetes-operator]

2024-01-24 Thread via GitHub
1996fanrui commented on code in PR #762: URL: https://github.com/apache/flink-kubernetes-operator/pull/762#discussion_r1464567468 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/utils/MemoryTuningUtils.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-33287] Using the flink shaded jackson for flink-autoscaler [flink-kubernetes-operator]

2024-01-24 Thread via GitHub
1996fanrui merged PR #761: URL: https://github.com/apache/flink-kubernetes-operator/pull/761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsu

[PR] [DRAFT] new implementation of FLIP-403 [flink]

2024-01-24 Thread via GitHub
KarmaGYZ opened a new pull request, #24189: URL: https://github.com/apache/flink/pull/24189 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them o

[jira] [Commented] (FLINK-33287) Using the flink shaded jackson for flink-autoscaler

2024-01-24 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810330#comment-17810330 ] Rui Fan commented on FLINK-33287: - Merged to main(1.9.0) via : ca5587445e1cde945e26bd6aa

[jira] [Resolved] (FLINK-33287) Using the flink shaded jackson for flink-autoscaler

2024-01-24 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan resolved FLINK-33287. - Fix Version/s: kubernetes-operator-1.8.0 Resolution: Fixed > Using the flink shaded jackson for f

Re: [PR] [FLINK-33668] Decoupling Shuffle network memory and job topology [flink]

2024-01-24 Thread via GitHub
jiangxin369 commented on code in PR #23851: URL: https://github.com/apache/flink/pull/23851#discussion_r1464734165 ## flink-runtime/src/main/java/org/apache/flink/runtime/io/network/metrics/CreditBasedInputBuffersUsageGauge.java: ## @@ -28,27 +31,35 @@ */ public class CreditB

[jira] [Comment Edited] (FLINK-33287) Using the flink shaded jackson for flink-autoscaler

2024-01-24 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810330#comment-17810330 ] Rui Fan edited comment on FLINK-33287 at 1/24/24 10:55 AM: --- Me

Re: [PR] [DRAFT] new implementation of FLIP-403 [flink]

2024-01-24 Thread via GitHub
flinkbot commented on PR #24189: URL: https://github.com/apache/flink/pull/24189#issuecomment-1907898418 ## CI report: * 0faeb94bea8f83cf2aa8c533ccf936edd66a6249 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464755151 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/window/tvf/operator/UnalignedWindowTableFunctionOperator.java: ## @@ -0,0

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464763918 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/window/tvf/operator/UnalignedWindowTableFunctionOperator.java: ## @@ -0,0

Re: [PR] [FLINK-34120] Introduce unified serialization config option for all Kryo, POJO and customized serializers [flink]

2024-01-24 Thread via GitHub
X-czh commented on code in PR #24182: URL: https://github.com/apache/flink/pull/24182#discussion_r1464765997 ## flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java: ## @@ -255,6 +256,51 @@ public class PipelineOptions {

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464766029 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/batch/sql/WindowTableFunctionITCase.scala: ## @@ -1327,4 +1328,146 @@ class

Re: [PR] [FLINK-34220][table-runtime] Introduce BufferBundle for minibatch join [flink]

2024-01-24 Thread via GitHub
xishuaidelin commented on code in PR #24159: URL: https://github.com/apache/flink/pull/24159#discussion_r1464767224 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/join/stream/bundle/InputSideHasNoUniqueKeyBundle.java: ## @@ -0,0 +1,145

Re: [PR] [FLINK-34220][table-runtime] Introduce BufferBundle for minibatch join [flink]

2024-01-24 Thread via GitHub
xishuaidelin commented on code in PR #24159: URL: https://github.com/apache/flink/pull/24159#discussion_r1464767765 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/join/stream/bundle/BufferBundle.java: ## @@ -0,0 +1,86 @@ +/* + * License

[jira] [Updated] (FLINK-34220) introduce buffer bundle for minibatch join

2024-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34220: --- Labels: pull-request-available (was: ) > introduce buffer bundle for minibatch join > -

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464769669 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/WindowAggregateTest.xml: ## @@ -2170,6 +2360,386 @@ Sink(ta

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464774215 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/WindowAggregateTest.xml: ## @@ -1186,6 +1186,196 @@ Calc(se

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
xuyangzhong commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464774651 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/WindowAggregateTest.xml: ## @@ -1186,6 +1186,196 @@ Calc(se

Re: [PR] [FLINK-33976] AdaptiveScheduler cooldown period should be taken from JobManager's configuration [flink]

2024-01-24 Thread via GitHub
echauchot commented on code in PR #24021: URL: https://github.com/apache/flink/pull/24021#discussion_r1464780022 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/Executing.java: ## @@ -310,22 +312,25 @@ static class Factory implements StateFactory {

Re: [PR] [FLINK-34121][core] Introduce pipeline.force-kryo-avro to control whe… [flink]

2024-01-24 Thread via GitHub
X-czh commented on code in PR #24129: URL: https://github.com/apache/flink/pull/24129#discussion_r1464784119 ## docs/layouts/shortcodes/generated/pipeline_configuration.html: ## @@ -56,6 +56,12 @@ Boolean If enabled, forces TypeExtractor to use Kryo ser

[jira] [Commented] (FLINK-32315) Support local file upload in K8s mode

2024-01-24 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810338#comment-17810338 ] Ferenc Csaky commented on FLINK-32315: -- Hi! I wrapped up the work on FLINK-28915 an

Re: [PR] [FLINK-34121][core] Introduce pipeline.force-kryo-avro to control whe… [flink]

2024-01-24 Thread via GitHub
X-czh commented on code in PR #24129: URL: https://github.com/apache/flink/pull/24129#discussion_r1464796642 ## flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/DataTypeFactoryImpl.java: ## @@ -141,6 +141,16 @@ private static Supplier createSerializ

Re: [PR] [FLINK-33976] AdaptiveScheduler cooldown period should be taken from JobManager's configuration [flink]

2024-01-24 Thread via GitHub
echauchot commented on code in PR #24021: URL: https://github.com/apache/flink/pull/24021#discussion_r1464780022 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/Executing.java: ## @@ -310,22 +312,25 @@ static class Factory implements StateFactory {

[jira] [Commented] (FLINK-34016) Janino compile failed when watermark with column by udf

2024-01-24 Thread Yunhong Zheng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810339#comment-17810339 ] Yunhong Zheng commented on FLINK-34016: --- Hi, [~wczhu] , I cann't reproduce this er

[jira] [Commented] (FLINK-34147) TimestampData to/from LocalDateTime is ambiguous

2024-01-24 Thread Rui Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810347#comment-17810347 ] Rui Li commented on FLINK-34147: I think having two separate classes is fine, but it mig

Re: [PR] Introduce operator for minibatch join [flink]

2024-01-24 Thread via GitHub
xishuaidelin commented on code in PR #24160: URL: https://github.com/apache/flink/pull/24160#discussion_r1464818953 ## flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/runtime/operators/join/stream/StreamingMiniBatchJoinOperatorTest.java: ## @@ -0,0 +1,1667 @

[jira] [Created] (FLINK-34227) ResourceRequirements couldn't be matched in WindowDistinctAggregateITCase with AdaptiveScheduler causing a timeout

2024-01-24 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34227: - Summary: ResourceRequirements couldn't be matched in WindowDistinctAggregateITCase with AdaptiveScheduler causing a timeout Key: FLINK-34227 URL: https://issues.apache.org/jira/

Re: [PR] [FLINK-33263][table-planner] Implement ParallelismProvider for sources in the table planner [flink]

2024-01-24 Thread via GitHub
BIOINSu commented on code in PR #24128: URL: https://github.com/apache/flink/pull/24128#discussion_r1464827047 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/common/CommonExecTableSourceScan.java: ## @@ -105,54 +117,131 @@ protect

[jira] [Updated] (FLINK-34227) WindowDistinctAggregateITCase with AdaptiveScheduler times out

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34227: -- Summary: WindowDistinctAggregateITCase with AdaptiveScheduler times out (was: ResourceRequire

Re: [PR] [FLINK-34145][connector/filesystem] support dynamic source parallelism inference in batch jobs [flink]

2024-01-24 Thread via GitHub
zhuzhurk commented on code in PR #24186: URL: https://github.com/apache/flink/pull/24186#discussion_r1464840583 ## flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/src/FileSourceTextLinesITCase.java: ## @@ -126,8 +127,9 @@ private void testBou

Re: [PR] [FLINK-34210][Runtime/Checkpointing] Fix DefaultExecutionGraphBuilder#isCheckpointingEnabled return wrong value when checkpoint disabled [flink]

2024-01-24 Thread via GitHub
WencongLiu commented on PR #24173: URL: https://github.com/apache/flink/pull/24173#issuecomment-1908029486 Thanks @mayuehappy . In adaptive batch scenarios, the creation of a CheckpointCoordinator is definitely not necessary. In other scenarios, we may need to further sort out whether we ne

[jira] [Commented] (FLINK-34227) WindowDistinctAggregateITCase with AdaptiveScheduler times out

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810357#comment-17810357 ] Matthias Pohl commented on FLINK-34227: --- There are several test failures in {{Wind

Re: [PR] [FLINK-34015][checkpoint] fix that passing ignore-unclaimed-state through dynamic props does not take effect [flink]

2024-01-24 Thread via GitHub
xiangforever2014 commented on PR #24058: URL: https://github.com/apache/flink/pull/24058#issuecomment-1908037053 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] [FLINK-34144][docs] Update the documentation about dynamic source parallelism inference [flink]

2024-01-24 Thread via GitHub
zhuzhurk commented on code in PR #24184: URL: https://github.com/apache/flink/pull/24184#discussion_r1464845836 ## docs/content/docs/deployment/elastic_scaling.md: ## @@ -230,6 +230,22 @@ In addition, there are several related configuration options that may need adjus - Set

[jira] [Created] (FLINK-34228) Add long UTF serializer/deserializer

2024-01-24 Thread Peter Vary (Jira)
Peter Vary created FLINK-34228: -- Summary: Add long UTF serializer/deserializer Key: FLINK-34228 URL: https://issues.apache.org/jira/browse/FLINK-34228 Project: Flink Issue Type: Improvement

Re: [PR] [FLINK-33914][ci] Adds basic Flink CI workflow [flink]

2024-01-24 Thread via GitHub
XComp commented on PR #23970: URL: https://github.com/apache/flink/pull/23970#issuecomment-1908054230 * [This force push](https://github.com/apache/flink/compare/2d8452c5abe034a48df6b08eabd1e7088f54c1d4..1557192d0ed323a1265d11512384f720ae3bc268) updates the workflow name to include `(beta)`

[jira] [Commented] (FLINK-34227) WindowDistinctAggregateITCase with AdaptiveScheduler times out

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810366#comment-17810366 ] Matthias Pohl commented on FLINK-34227: --- :facepalm: The test run was cancelled bec

[jira] [Updated] (FLINK-34227) table stage timed out

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34227: -- Summary: table stage timed out (was: WindowDistinctAggregateITCase with AdaptiveScheduler tim

[jira] [Updated] (FLINK-34227) Github Actions: table stage timed out in AdaptiveScheduler profile

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34227: -- Summary: Github Actions: table stage timed out in AdaptiveScheduler profile (was: table stage

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
lsyldliu commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464871853 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/batch/sql/WindowTableFunctionITCase.scala: ## @@ -1327,4 +1328,146 @@ class Wi

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
lsyldliu commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464873767 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/window/tvf/operator/UnalignedWindowTableFunctionOperator.java: ## @@ -0,0 +1,

Re: [PR] [FLINK-34100][table] Support session window table function without pulling up with window agg [flink]

2024-01-24 Thread via GitHub
lsyldliu commented on code in PR #24162: URL: https://github.com/apache/flink/pull/24162#discussion_r1464873767 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/window/tvf/operator/UnalignedWindowTableFunctionOperator.java: ## @@ -0,0 +1,

[jira] [Commented] (FLINK-34227) Github Actions: table stage timed out in AdaptiveScheduler profile

2024-01-24 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810373#comment-17810373 ] Matthias Pohl commented on FLINK-34227: --- Ok, {{WindowDistinctAggregateITCase}} see

  1   2   3   >