Re: [Discuss] Planning Flink 1.14

2021-06-06 Thread Benchao Li
Hi all, Thanks Xintong for bringing this up. I would like to share some experience of the usage of Flink in our company (ByteDance). 1. We started building our SQL platform in mid 2019, using v1.9 blink planner, and it's amazing. Also we added many internal features which is still missing in

[jira] [Created] (FLINK-22895) Table common document misspelled

2021-06-06 Thread sujun (Jira)
sujun created FLINK-22895: - Summary: Table common document misspelled Key: FLINK-22895 URL: https://issues.apache.org/jira/browse/FLINK-22895 Project: Flink Issue Type: Bug Components:

Re: Add control mode for flink

2021-06-06 Thread 刘建刚
Thank you for the reply. I have checked the post you mentioned. The dynamic config may be useful sometimes. But it is hard to keep data consistent in flink, for example, what if the dynamic config will take effect when failover. Since dynamic config is a desire for users, maybe flink can support

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2021-06-06 Thread Jingsong Li
Thanks Yingjie for the great effort! This is really helpful to Flink Batch users! Best, Jingsong On Mon, Jun 7, 2021 at 10:11 AM Yingjie Cao wrote: > Hi devs & users, > > The FLIP-148[1] has been released with Flink 1.13 and the final > implementation has some differences compared with the

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2021-06-06 Thread Yingjie Cao
Hi devs & users, The FLIP-148[1] has been released with Flink 1.13 and the final implementation has some differences compared with the initial proposal in the FLIP document. To avoid potential misunderstandings, I have updated the FLIP document[1] accordingly and I also drafted another

[jira] [Created] (FLINK-22894) Window Top-N should allow n=1

2021-06-06 Thread David Anderson (Jira)
David Anderson created FLINK-22894: -- Summary: Window Top-N should allow n=1 Key: FLINK-22894 URL: https://issues.apache.org/jira/browse/FLINK-22894 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-22893) ResumeCheckpointManuallyITCase hangs on azure

2021-06-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22893: Summary: ResumeCheckpointManuallyITCase hangs on azure Key: FLINK-22893 URL: https://issues.apache.org/jira/browse/FLINK-22893 Project: Flink Issue

[jira] [Created] (FLINK-22892) Resuming Externalized Checkpoint (rocks, incremental, scale down) end-to-end test fails

2021-06-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22892: Summary: Resuming Externalized Checkpoint (rocks, incremental, scale down) end-to-end test fails Key: FLINK-22892 URL: https://issues.apache.org/jira/browse/FLINK-22892

[jira] [Created] (FLINK-22891) FineGrainedSlotManagerDefaultResourceAllocationStrategyITCase fails on azure

2021-06-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22891: Summary: FineGrainedSlotManagerDefaultResourceAllocationStrategyITCase fails on azure Key: FLINK-22891 URL: https://issues.apache.org/jira/browse/FLINK-22891

[jira] [Created] (FLINK-22890) Few tests fail in HiveTableSinkITCase

2021-06-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22890: Summary: Few tests fail in HiveTableSinkITCase Key: FLINK-22890 URL: https://issues.apache.org/jira/browse/FLINK-22890 Project: Flink Issue Type:

[jira] [Created] (FLINK-22889) JdbcExactlyOnceSinkE2eTest hangs on azure

2021-06-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22889: Summary: JdbcExactlyOnceSinkE2eTest hangs on azure Key: FLINK-22889 URL: https://issues.apache.org/jira/browse/FLINK-22889 Project: Flink Issue

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

2021-06-06 Thread Steven Wu
> Converter function relies on the specific enumerator capabilities to set the new start position (e.g. fileSourceEnumerator.getEndTimestamp() and kafkaSourceEnumerator.setTimestampOffsetsInitializer(..) I guess the premise is that a converter is for a specific tuple of (upstream source,

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

2021-06-06 Thread Thomas Weise
Hi Steven, Thank you for the thorough review of the PR and for bringing this back to the mailing list. All, I updated the FLIP-150 page to highlight aspects in which the PR deviates from the original proposal [1]. The goal would be to update the FLIP soon and bring it to a vote, as previously

Job gets stuck with exactly once checkpointing and Kafka when initializing.

2021-06-06 Thread Anil Singh
Dear Flink Community, When using exactly once checkpointing along kafka and connecting Kafka using SASL_SSL the job gets stuck with status of Kafka producer in Initialising state for around 10 mins and post that it functions normally across multiple

[jira] [Created] (FLINK-22888) Matches results may be wrong when using notNext as the last part of the pattern with Window

2021-06-06 Thread Yue Ma (Jira)
Yue Ma created FLINK-22888: -- Summary: Matches results may be wrong when using notNext as the last part of the pattern with Window Key: FLINK-22888 URL: https://issues.apache.org/jira/browse/FLINK-22888