[jira] [Created] (FLINK-21093) Support StreamExecTableSource json serialization/deserialization

godfrey he created FLINK-21093: -- Summary: Support StreamExecTableSource json serialization/deserialization Key: FLINK-21093 URL: https://issues.apache.org/jira/browse/FLINK-21093 Project: Flink

[jira] [Created] (FLINK-21092) Introduce getJsonPlan, explainJsonPlan and executeJsonPlan in TableEnvironmentInternal

godfrey he created FLINK-21092: -- Summary: Introduce getJsonPlan, explainJsonPlan and executeJsonPlan in TableEnvironmentInternal Key: FLINK-21092 URL: https://issues.apache.org/jira/browse/FLINK-21092

[jira] [Created] (FLINK-21091) Support ExecNode json serialization/deserialization

godfrey he created FLINK-21091: -- Summary: Support ExecNode json serialization/deserialization Key: FLINK-21091 URL: https://issues.apache.org/jira/browse/FLINK-21091 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

Thanks @Timo for the detailed reply, let's go on this topic on this discussion, I've merged all mails to this discussion. > LOCALDATE / LOCALTIME / LOCALTIMESTAMP > > --> uses session time zone, returns DATE/TIME/TIMESTAMP > > CURRENT_DATE/CURRENT_TIME/CURRENT_TIMESTAMP > > --> uses session

Re: [DISCUSS] Correct time-related function behavior in Flink SQL

Thanks @Timo for the detailed reply. Let’s continue the technical discussion on the former FLIP discussion thread[1]: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior. We can share our decision to user and user-zh on this thread once we have a final proposal. Best, Leonard

[jira] [Created] (FLINK-21088) CheckpointCoordinator pass the flag about whether a operator is fully finished on recovery

Yun Gao created FLINK-21088: --- Summary: CheckpointCoordinator pass the flag about whether a operator is fully finished on recovery Key: FLINK-21088 URL: https://issues.apache.org/jira/browse/FLINK-21088

[jira] [Created] (FLINK-21087) StreamTask waits for all the asynchronous step of pending checkpoint to finish

Yun Gao created FLINK-21087: --- Summary: StreamTask waits for all the asynchronous step of pending checkpoint to finish Key: FLINK-21087 URL: https://issues.apache.org/jira/browse/FLINK-21087 Project: Flink

[jira] [Created] (FLINK-21086) CheckpointBarrierHandler Insert barriers into channels received EndOfPartition

Yun Gao created FLINK-21086: --- Summary: CheckpointBarrierHandler Insert barriers into channels received EndOfPartition Key: FLINK-21086 URL: https://issues.apache.org/jira/browse/FLINK-21086 Project: Flink

[jira] [Created] (FLINK-21085) Refactor StreamTask hierarchy to support triggering checkpoint via RPC for non-source tasks

Yun Gao created FLINK-21085: --- Summary: Refactor StreamTask hierarchy to support triggering checkpoint via RPC for non-source tasks Key: FLINK-21085 URL: https://issues.apache.org/jira/browse/FLINK-21085

[jira] [Created] (FLINK-21084) Allows tasks to report operator subtask finish state

Yun Gao created FLINK-21084: --- Summary: Allows tasks to report operator subtask finish state Key: FLINK-21084 URL: https://issues.apache.org/jira/browse/FLINK-21084 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

FGRuntimeInterface.png Thank you~ Xintong Song On Fri, Jan 22, 2021 at 11:11 AM Xintong Song wrote: > I think Chesnay's proposal could actually work. IIUC, the keypoint is to > derive operator

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

I think Chesnay's proposal could actually work. IIUC, the keypoint is to derive operator requirements from SSG requirements on the API side, so that the runtime only deals with operator requirements. It's debatable how the deriving should be done though. E.g., an alternative could be to evenly

[jira] [Created] (FLINK-21083) Streaming write partitioned table needs to handle null partition value

Rui Li created FLINK-21083: -- Summary: Streaming write partitioned table needs to handle null partition value Key: FLINK-21083 URL: https://issues.apache.org/jira/browse/FLINK-21083 Project: Flink

[jira] [Created] (FLINK-21082) Document how to start multiple JobManagers in HA mode for Kubernetes deployment

Yang Wang created FLINK-21082: - Summary: Document how to start multiple JobManagers in HA mode for Kubernetes deployment Key: FLINK-21082 URL: https://issues.apache.org/jira/browse/FLINK-21082 Project:

Re: [ANNOUNCE] Welcome Guowei Ma as a new Apache Flink Committer

Congrats, Guowei! Best, Zhijiang -- From:Biao Liu Send Time:2021年1月21日(星期四) 14:45 To:dev Subject:Re: [ANNOUNCE] Welcome Guowei Ma as a new Apache Flink Committer Congrats, Guowei! Thanks, Biao /'bɪ.aʊ/ On Thu, 21 Jan 2021

[jira] [Created] (FLINK-21081) Re-compute tasks to trigger when tasks get triggered before finished

Yun Gao created FLINK-21081: --- Summary: Re-compute tasks to trigger when tasks get triggered before finished Key: FLINK-21081 URL: https://issues.apache.org/jira/browse/FLINK-21081 Project: Flink

[jira] [Created] (FLINK-21080) Identify JobVertex containing legacy source operators and abort checkpoint with legacy source operators partially finished

Yun Gao created FLINK-21080: --- Summary: Identify JobVertex containing legacy source operators and abort checkpoint with legacy source operators partially finished Key: FLINK-21080 URL:

[jira] [Created] (FLINK-21079) Fink startup failed when $JAVA_HOME have space

Raghav Kumar Gautam created FLINK-21079: --- Summary: Fink startup failed when $JAVA_HOME have space Key: FLINK-21079 URL: https://issues.apache.org/jira/browse/FLINK-21079 Project: Flink

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

You're raising a good point, but I think I can rectify that with a minor adjustment. Default requirements are whatever the default requirements are, setting the requirements for one operator has no effect on other operators. With these rules, and some API enhancements, the following mockup

[jira] [Created] (FLINK-21078) Add DeclarativeSlotPoolService

Till Rohrmann created FLINK-21078: - Summary: Add DeclarativeSlotPoolService Key: FLINK-21078 URL: https://issues.apache.org/jira/browse/FLINK-21078 Project: Flink Issue Type: Sub-task

Re: [Vote] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

Hi all, Looks like 157 has passed :) I will open the tickets and move forward with the migration. Look out for a follow-up email soon about the transition processes and short doc freeze. Seth On Wed, Jan 20, 2021 at 5:19 AM Arvid Heise wrote: > +1 > > On Wed, Jan 20, 2021 at 12:07 PM

[jira] [Created] (FLINK-21077) Abstract SlotPool from JobManager by introducing SlotPoolService

Till Rohrmann created FLINK-21077: - Summary: Abstract SlotPool from JobManager by introducing SlotPoolService Key: FLINK-21077 URL: https://issues.apache.org/jira/browse/FLINK-21077 Project: Flink

[jira] [Created] (FLINK-21076) Document how to use the declarative scheduler and its limitations

Till Rohrmann created FLINK-21076: - Summary: Document how to use the declarative scheduler and its limitations Key: FLINK-21076 URL: https://issues.apache.org/jira/browse/FLINK-21076 Project: Flink

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

Now we have 2 discussion threads on 3 mailing lists. Which one should have prioity? Should I repost my large email here again? I think it is good to inform and invite in the user mailing lists but let's keep the FLIP discussion on the dev@ ML only. Regards, Timo On 21.01.21 16:50, Leonard

[jira] [Created] (FLINK-21075) FLIP-160: Declarative scheduler

Till Rohrmann created FLINK-21075: - Summary: FLIP-160: Declarative scheduler Key: FLINK-21075 URL: https://issues.apache.org/jira/browse/FLINK-21075 Project: Flink Issue Type: New Feature

Re: Flink Table from KeyedStream

Hey, Thanks for the answer. That's what I've been observing but wanted to know for sure. Best Regards, Dom.

Re: [DISCUSS] Correct time-related function behavior in Flink SQL

Hi Leonard, thanks for working on this topic. I agree that time handling is not easy in Flink at the moment. We added new time data types (and some are still not supported which even further complicates things like TIME(9)). We should definitely improve this situation for users. This is a

[jira] [Created] (FLINK-21074) Document how to use the reactive mode

Till Rohrmann created FLINK-21074: - Summary: Document how to use the reactive mode Key: FLINK-21074 URL: https://issues.apache.org/jira/browse/FLINK-21074 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-161: Configuration through environment variables

Hi everyone, I've updated the FLIP (https://cwiki.apache.org/confluence/x/ngtRCg) according to these discussions. Regards Ingo On Thu, Jan 21, 2021 at 11:37 AM Ingo Bürk wrote: > Hi Ufuk, Till, > > I definitely agree that having the Configuration be (or at least feel) > immutable and

[DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

Hello, everyone I want to start the discussion of FLIP-162: Consistent Flink SQL time function behavior[1]. We’ve some initial discussion of several problematic functions in dev mail list[2], and I think it's the right time to resolve them by a FLIP. Currently some time function behaviors

Re: Flink Table from KeyedStream

Hi Dom, AFAIK, Table API will apply a key partitioner based on the join key for the join operator, [id, data] and [numbeer, metadata] in your case. So the partitioner in the KeyedStreaem is not respected. Best, Jark On Thu, 21 Jan 2021 at 21:39, Dominik Wosiński wrote: > Hey, > I was

Re: [VOTE] Release 1.10.3, release candidate #1

+1 (non-binding) - verified checksums - build Flink from the provided source code - double-checked that the security fix is included in the source code - started the a local Flink cluster and ran a job checking the logs for exceptions - verified that CVE-2020-17518 cannot be reproduced using the

[jira] [Created] (FLINK-21073) Mention that RocksDB ignores equals/hashCode because it works on binary data

Robert Metzger created FLINK-21073: -- Summary: Mention that RocksDB ignores equals/hashCode because it works on binary data Key: FLINK-21073 URL: https://issues.apache.org/jira/browse/FLINK-21073

Re: Re: [DISCUSS] Dealing with deprecated and legacy code in Flink

> > I would prefer not to rely on the Jira for marking when something is > supposed to be deleted. If `@Deprecated(since, planned_to_remove_on)` would > have two obligatory parameters, there would be no way to "forget" about > marking it and it would be also self documenting (I don't imagine users

Re: [DISCUSS] Correct time-related function behavior in Flink SQL

> Before the changes, as I am writing this reply, the local time here is > 2021-01-21 12:03:35 (Beijing time, UTC+8). > And I tried these 5 functions in sql client, and got: > > Flink SQL> select now(), PROCTIME(), CURRENT_TIMESTAMP, CURRENT_DATE, > CURRENT_TIME; >

Flink Table from KeyedStream

Hey, I was wondering if that's currently possible to use KeyedStream to create a properly partitioned Table in Flink 1.11 ? I have a use case where I wanted to first join two streams using Flink SQL and then process them via *KeyedProcessFunction.* So I do something like: implicit val env =

[jira] [Created] (FLINK-21072) Refactor the SnapshotStrategy hierarchy

Dawid Wysakowicz created FLINK-21072: Summary: Refactor the SnapshotStrategy hierarchy Key: FLINK-21072 URL: https://issues.apache.org/jira/browse/FLINK-21072 Project: Flink Issue Type:

[jira] [Created] (FLINK-21071) Kubernetes e2e test failed

Dawid Wysakowicz created FLINK-21071: Summary: Kubernetes e2e test failed Key: FLINK-21071 URL: https://issues.apache.org/jira/browse/FLINK-21071 Project: Flink Issue Type: Bug

Re: [DISCUSS] Flink configuration from environment variables

Hi, sorry, I almost forgot, so just to update this thread: I now started a new thread for the actual FLIP: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-161-Configuration-through-environment-variables-td48094.html Ingo On Wed, Jan 20, 2021 at 11:10 AM Ingo Bürk

Re: [DISCUSS] FLIP-161: Configuration through environment variables

Hi Ufuk, Till, I definitely agree that having the Configuration be (or at least feel) immutable and complete seems like a better choice, and it is probably worth the trade-off in EV naming flexibility. Let me reshape the FLIP to propose something along the lines of solution (3). Regarding

[jira] [Created] (FLINK-21070) Overloaded aggregate functions cause converter errors

Timo Walther created FLINK-21070: Summary: Overloaded aggregate functions cause converter errors Key: FLINK-21070 URL: https://issues.apache.org/jira/browse/FLINK-21070 Project: Flink Issue

[jira] [Created] (FLINK-21069) Configuration "parallelism.default" doesn't take effect for TableEnvironment#explainSql

Jark Wu created FLINK-21069: --- Summary: Configuration "parallelism.default" doesn't take effect for TableEnvironment#explainSql Key: FLINK-21069 URL: https://issues.apache.org/jira/browse/FLINK-21069

[jira] [Created] (FLINK-21068) Add new timeout options for Elasticsearch connector

jinfeng created FLINK-21068: --- Summary: Add new timeout options for Elasticsearch connector Key: FLINK-21068 URL: https://issues.apache.org/jira/browse/FLINK-21068 Project: Flink Issue Type:

[jira] [Created] (FLINK-21067) Modify the logic of computing tasks to trigger/wait/commit to consider finished tasks

Yun Gao created FLINK-21067: --- Summary: Modify the logic of computing tasks to trigger/wait/commit to consider finished tasks Key: FLINK-21067 URL: https://issues.apache.org/jira/browse/FLINK-21067 Project:

[jira] [Created] (FLINK-21066) Refactor the CheckpointCoordinator to compute the tasks to trigger/wait/commit dynamically

Yun Gao created FLINK-21066: --- Summary: Refactor the CheckpointCoordinator to compute the tasks to trigger/wait/commit dynamically Key: FLINK-21066 URL: https://issues.apache.org/jira/browse/FLINK-21066

Re: [DISCUSS] FLIP-161: Configuration through environment variables

Thanks for starting the discussion, Ingo! Regarding approach 1: I like the idea of having a mapping scheme from ConfigOption to env var(s), but I'm concerned about the implications of lazy eval. I think it would be preferable to keep the Configuration object as the source of truth, requiring

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

I second Till's concern about implicitly interpreting zero resource requirements for unspecified operators. I'm not against allowing both specifying SSG requirements as shortcuts and further refining operator requirements as needed. Combining Till's idea, we can do the following. - Prefer using

[jira] [Created] (FLINK-21065) Passing configuration from TableEnvironmentImpl to MiniCluster is not supported

Robert Metzger created FLINK-21065: -- Summary: Passing configuration from TableEnvironmentImpl to MiniCluster is not supported Key: FLINK-21065 URL: https://issues.apache.org/jira/browse/FLINK-21065

[jira] [Created] (FLINK-21064) Format docs incorrectly contain "format" property

Ingo Bürk created FLINK-21064: - Summary: Format docs incorrectly contain "format" property Key: FLINK-21064 URL: https://issues.apache.org/jira/browse/FLINK-21064 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-161: Configuration through environment variables

Thanks for starting this discussion Ingo! I like the flexibility in terms of supported EV formats the lazy evaluation gives us. However, I am a bit concerned by the fact that it is quite hard to compute the configurational ground truth w/o knowing all requested config keys. Moreover, it makes the

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

If I understand you correctly Chesnay, then you want to decouple the resource requirement specification from the slot sharing group assignment. Hence, per default all operators would be in the same slot sharing group. If there is no operator with a resource specification, then the system would