[RESULT][VOTE] FLIP-211: Kerberos delegation token framework

2022-02-07 Thread Gabor Somogyi
Hi devs, FLIP-211 [1] Has been accepted. There were 3 binding votes and 2 non-binding in favor. None against. Votes are in the order of arrival: Binding: Gyula Fora Marton Balassi Chesnay Schepler Non-binding: Junfan Zhang David Moravek [1]

[jira] [Created] (FLINK-25998) Flink akka runs into NoClassDefFoundError on shutdown

2022-02-07 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-25998: -- Summary: Flink akka runs into NoClassDefFoundError on shutdown Key: FLINK-25998 URL: https://issues.apache.org/jira/browse/FLINK-25998 Project: Flink

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Yangze Guo
Thanks @Thomas and @Gyula. +1 to only introduce necessary and reasonable shorthand proxy parameters. Best, Yangze Guo On Tue, Feb 8, 2022 at 12:47 PM Thomas Weise wrote: > > @Yangze thanks for bringing up the configuration priority. This is > quite important indeed and should be mentioned in

[jira] [Created] (FLINK-25997) FsStateChangelogWriter cannot work well when there is no change appended

2022-02-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-25997: Summary: FsStateChangelogWriter cannot work well when there is no change appended Key: FLINK-25997 URL: https://issues.apache.org/jira/browse/FLINK-25997 Project:

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Thomas Weise
@Yangze thanks for bringing up the configuration priority. This is quite important indeed and should be mentioned in the FLIP. I agree with the sentiment that whenever possible we should use the native configuration directly (either Flink native settings or k8s pod template), rather than

[jira] [Created] (FLINK-25996) Introduce job property isDynamicGraph to ExecutionConfig

2022-02-07 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-25996: --- Summary: Introduce job property isDynamicGraph to ExecutionConfig Key: FLINK-25996 URL: https://issues.apache.org/jira/browse/FLINK-25996 Project: Flink Issue Type:

[jira] [Created] (FLINK-25995) Make implicit assumption of SQL local keyBy/groupBy explicit

2022-02-07 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-25995: --- Summary: Make implicit assumption of SQL local keyBy/groupBy explicit Key: FLINK-25995 URL: https://issues.apache.org/jira/browse/FLINK-25995 Project: Flink Issue

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Gen Luo
Hi, I'm thinking about Yuan's case. Let's assume that the case is running in current Flink: 1. CP8 finishes 2. For some reason, PR2 stops consuming records from the source (but is not stuck), and PR1 continues consuming new records. 3. CP9 and CP10 finish 4. PR2 starts to consume quickly to catch

[jira] [Created] (FLINK-25994) Implement FileStoreExpire

2022-02-07 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-25994: --- Summary: Implement FileStoreExpire Key: FLINK-25994 URL: https://issues.apache.org/jira/browse/FLINK-25994 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-25993) Option to disable Kryo.setRegistrationRequired(false)

2022-02-07 Thread Shane Bishop (Jira)
Shane Bishop created FLINK-25993: Summary: Option to disable Kryo.setRegistrationRequired(false) Key: FLINK-25993 URL: https://issues.apache.org/jira/browse/FLINK-25993 Project: Flink Issue

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Thomas Weise
Hi, Thanks for opening this discussion! The proposed enhancement would be interesting for use cases in our infrastructure as well. There are scenarios where it makes sense to have multiple disconnected subgraphs in a single job because it can significantly reduce the operational burden as well

[jira] [Created] (FLINK-25992) JobDispatcherITCase..testRecoverFromCheckpointAfterLosingAndRegainingLeadership fails on azure

2022-02-07 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-25992: - Summary: JobDispatcherITCase..testRecoverFromCheckpointAfterLosingAndRegainingLeadership fails on azure Key: FLINK-25992 URL:

[jira] [Created] (FLINK-25991) Allow custom metadata to be committed alongside the offsets

2022-02-07 Thread Frederic Hemery (Jira)
Frederic Hemery created FLINK-25991: --- Summary: Allow custom metadata to be committed alongside the offsets Key: FLINK-25991 URL: https://issues.apache.org/jira/browse/FLINK-25991 Project: Flink

[jira] [Created] (FLINK-25990) Expose uid generator for DataStream/Transformation providers

2022-02-07 Thread Timo Walther (Jira)
Timo Walther created FLINK-25990: Summary: Expose uid generator for DataStream/Transformation providers Key: FLINK-25990 URL: https://issues.apache.org/jira/browse/FLINK-25990 Project: Flink

[jira] [Created] (FLINK-25989) EventTimeWindowCheckpointingITCase failed with exit code 137

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25989: --- Summary: EventTimeWindowCheckpointingITCase failed with exit code 137 Key: FLINK-25989 URL: https://issues.apache.org/jira/browse/FLINK-25989 Project: Flink Issue

[jira] [Created] (FLINK-25988) UnalignedCheckpointRescaleITCase failed with exit code 137

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25988: --- Summary: UnalignedCheckpointRescaleITCase failed with exit code 137 Key: FLINK-25988 URL: https://issues.apache.org/jira/browse/FLINK-25988 Project: Flink Issue Type:

[jira] [Created] (FLINK-25987) IllegalArgumentException thrown from FsStateChangelogWriter.truncate

2022-02-07 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-25987: - Summary: IllegalArgumentException thrown from FsStateChangelogWriter.truncate Key: FLINK-25987 URL: https://issues.apache.org/jira/browse/FLINK-25987

[jira] [Created] (FLINK-25986) Add FLIP-190 new API methods to python

2022-02-07 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-25986: --- Summary: Add FLIP-190 new API methods to python Key: FLINK-25986 URL: https://issues.apache.org/jira/browse/FLINK-25986 Project: Flink Issue

[jira] [Created] (FLINK-25985) e2e test covering the main functionality of the JobResultStore

2022-02-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25985: - Summary: e2e test covering the main functionality of the JobResultStore Key: FLINK-25985 URL: https://issues.apache.org/jira/browse/FLINK-25985 Project: Flink

Re: [VOTE] Release 1.13.6, release candidate #1

2022-02-07 Thread Chesnay Schepler
+1 (binding) - signatures OK - checksums OK - tag OK - all artifacts accounted for - PR looks good On 05/02/2022 21:06, Konstantin Knauf wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 1.13.6, as follows: [ ] +1, Approve the release [ ] -1, Do not

[jira] [Created] (FLINK-25984) Add types to ConfigOptions and replace usages of the untyped methods defaultValue and noDefaultValue with the corresponding typed ones.

2022-02-07 Thread Marios Trivyzas (Jira)
Marios Trivyzas created FLINK-25984: --- Summary: Add types to ConfigOptions and replace usages of the untyped methods defaultValue and noDefaultValue with the corresponding typed ones. Key: FLINK-25984 URL:

[jira] [Created] (FLINK-25983) Add WatermarkStrategy#withWatermarkAlignment

2022-02-07 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-25983: Summary: Add WatermarkStrategy#withWatermarkAlignment Key: FLINK-25983 URL: https://issues.apache.org/jira/browse/FLINK-25983 Project: Flink Issue

[jira] [Created] (FLINK-25982) Support idleness with watermark alignment

2022-02-07 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-25982: -- Summary: Support idleness with watermark alignment Key: FLINK-25982 URL: https://issues.apache.org/jira/browse/FLINK-25982 Project: Flink Issue Type:

[jira] [Created] (FLINK-25981) ZooKeeperMultipleComponentLeaderElectionDriverTest failed

2022-02-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25981: - Summary: ZooKeeperMultipleComponentLeaderElectionDriverTest failed Key: FLINK-25981 URL: https://issues.apache.org/jira/browse/FLINK-25981 Project: Flink

[jira] [Created] (FLINK-25980) remove unnecessary condition in IntervalJoinOperator

2022-02-07 Thread hongshu (Jira)
hongshu created FLINK-25980: --- Summary: remove unnecessary condition in IntervalJoinOperator Key: FLINK-25980 URL: https://issues.apache.org/jira/browse/FLINK-25980 Project: Flink Issue Type:

RE: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Alexis Sarda-Espinosa
Danny Cranmer mentioned they are interested in standalone mode, and I am too, so I just wanted to say that if that development starts in parallel, I might be able to contribute a little. Regarding the CRD, I agree it would be nice to avoid as many "duplications" as possible if pod templates

[jira] [Created] (FLINK-25979) Suspicious Classloading error during close of KafkaEnumerator

2022-02-07 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25979: Summary: Suspicious Classloading error during close of KafkaEnumerator Key: FLINK-25979 URL: https://issues.apache.org/jira/browse/FLINK-25979 Project: Flink

[jira] [Created] (FLINK-25978) DispatcherTest.testJobDataAreCleanedUpInCorrectOrderOn*Job can be removed

2022-02-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25978: - Summary: DispatcherTest.testJobDataAreCleanedUpInCorrectOrderOn*Job can be removed Key: FLINK-25978 URL: https://issues.apache.org/jira/browse/FLINK-25978 Project:

[jira] [Created] (FLINK-25977) Close sink client and sink http client for KDS/KDF Sinks

2022-02-07 Thread Zichen Liu (Jira)
Zichen Liu created FLINK-25977: -- Summary: Close sink client and sink http client for KDS/KDF Sinks Key: FLINK-25977 URL: https://issues.apache.org/jira/browse/FLINK-25977 Project: Flink Issue

[jira] [Created] (FLINK-25976) Update the KDS and KDF Sink's defaults & update the docs

2022-02-07 Thread Zichen Liu (Jira)
Zichen Liu created FLINK-25976: -- Summary: Update the KDS and KDF Sink's defaults & update the docs Key: FLINK-25976 URL: https://issues.apache.org/jira/browse/FLINK-25976 Project: Flink Issue

[jira] [Created] (FLINK-25975) add doc for how to use AvroParquetRecordFormat

2022-02-07 Thread Jing Ge (Jira)
Jing Ge created FLINK-25975: --- Summary: add doc for how to use AvroParquetRecordFormat Key: FLINK-25975 URL: https://issues.apache.org/jira/browse/FLINK-25975 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-25974) Make cancellation of jobs depend on the JobResultStore

2022-02-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25974: - Summary: Make cancellation of jobs depend on the JobResultStore Key: FLINK-25974 URL: https://issues.apache.org/jira/browse/FLINK-25974 Project: Flink

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Till Rohrmann
Thanks for the clarification Yuan and Gen, I agree that the checkpointing of the sources needs to support the rescaling case, otherwise it does not work. Is there currently a source implementation where this wouldn't work? For Kafka it should work because we store the offset per assigned

[jira] [Created] (FLINK-25973) Rename ArchivedExecutionGraph.createFromInitializingJob into more generic createSparseArchivedExecutionGraph

2022-02-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25973: - Summary: Rename ArchivedExecutionGraph.createFromInitializingJob into more generic createSparseArchivedExecutionGraph Key: FLINK-25973 URL:

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Yuan Mei
Hey Till, > Why rescaling is a problem for pipelined regions/independent execution subgraphs: Take a simplified example : job graph : source (2 instances) -> sink (2 instances) execution graph: source (1/2) -> sink (1/2) [pieplined region 1] source (2/2) -> sink (2/2) [pieplined region 2]

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Gen Luo
Hi Till, I agree that a failing task is much like a very slow or deadlock task to the checkpointing. The main difference is whether a checkpoint of the region the task in can be triggered. Triggering a checkpoint on a failing region makes no sense since the checkpoint should be discarded right

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Yangze Guo
+1 (binding) Best, Yangze Guo On Mon, Feb 7, 2022 at 5:04 PM K Fred wrote: > > +1 (non-binding) > > Best Regards > Peng Yuan > > On Mon, Feb 7, 2022 at 4:49 PM Danny Cranmer > wrote: > > > +1 (binding) > > > > Thanks for this effort! > > Danny Cranmer > > > > On Mon, Feb 7, 2022 at 7:59 AM

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread K Fred
+1 (non-binding) Best Regards Peng Yuan On Mon, Feb 7, 2022 at 4:49 PM Danny Cranmer wrote: > +1 (binding) > > Thanks for this effort! > Danny Cranmer > > On Mon, Feb 7, 2022 at 7:59 AM Biao Geng wrote: > > > +1 (non-binding) > > > > Best, > > Biao Geng > > > > Peter Huang 于2022年2月7日周一

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Danny Cranmer
+1 (binding) Thanks for this effort! Danny Cranmer On Mon, Feb 7, 2022 at 7:59 AM Biao Geng wrote: > +1 (non-binding) > > Best, > Biao Geng > > Peter Huang 于2022年2月7日周一 14:31写道: > > > +1 (non-binding) > > > > > > Best Regards > > Peter Huang > > > > On Sun, Feb 6, 2022 at 7:35 PM Yang Wang

[jira] [Created] (FLINK-25972) org.apache.calcite.rel.metadata.JaninoRelMetadataProvider#HANDLERS might cause class leaks

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25972: --- Summary: org.apache.calcite.rel.metadata.JaninoRelMetadataProvider#HANDLERS might cause class leaks Key: FLINK-25972 URL: https://issues.apache.org/jira/browse/FLINK-25972

[jira] [Created] (FLINK-25971) org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeUtil#OBJECT_MAPPER_INSTANCE might cause class leaks

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25971: --- Summary: org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeUtil#OBJECT_MAPPER_INSTANCE might cause class leaks Key: FLINK-25971 URL:

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread K Fred
Hi Gyula! You are right. I think some common flink config options can be put in the CR, other expert settings continue to be overwritten by flink, and then the user can choose to customize the configuration. Best Wishes, Peng Yuan On Mon, Feb 7, 2022 at 4:16 PM Gyula Fóra wrote: > Hi Yangze!

[jira] [Created] (FLINK-25970) SerializedThrowable should record type of the original throwable.

2022-02-07 Thread Xintong Song (Jira)
Xintong Song created FLINK-25970: Summary: SerializedThrowable should record type of the original throwable. Key: FLINK-25970 URL: https://issues.apache.org/jira/browse/FLINK-25970 Project: Flink

[jira] [Created] (FLINK-25969) org.apache.flink.table.runtime.generated.CompileUtils might cause class leaks

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25969: --- Summary: org.apache.flink.table.runtime.generated.CompileUtils might cause class leaks Key: FLINK-25969 URL: https://issues.apache.org/jira/browse/FLINK-25969 Project: Flink

[jira] [Created] (FLINK-25968) Possible class leaks in flink-table / sql modules

2022-02-07 Thread Yun Gao (Jira)
Yun Gao created FLINK-25968: --- Summary: Possible class leaks in flink-table / sql modules Key: FLINK-25968 URL: https://issues.apache.org/jira/browse/FLINK-25968 Project: Flink Issue Type: Bug

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-07 Thread Till Rohrmann
Hi everyone, Yuan and Gen could you elaborate why rescaling is a problem if we say that separate pipelined regions can take checkpoints independently? Conceptually, I somehow think that a pipelined region that is failed and cannot create a new checkpoint is more or less the same as a pipelined

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-07 Thread Gyula Fóra
Hi Yangze! This is not set in stone at the moment but the way I think it should work is that first class config options in the CR should always take precedence over the Flink config. In general we should not introduce too many arbitrary config options that duplicate the flink configs without

Re: [VOTE] FLIP-211: Kerberos delegation token framework

2022-02-07 Thread Chesnay Schepler
+1 (binding) On 21/01/2022 15:57, Gabor Somogyi wrote: Hi devs, I would like to start the vote for FLIP-211 [1], which was discussed and reached a consensus in the discussion thread [2]. The vote will be open for at least 72h, unless there is an objection or not enough votes. BR, G [1]