[DISCUSS] Creating a new repo to host Flink benchmarks

Hi Flink devs, As Flink develops rapidly with more and more features added, how to ensure no performance regression existed has become more and more important. And we would like to create a new repo under apache project to host previous flink-benchmarks [1] repo, which is inspired when we disc

Re: [VOTE] FLIP-108: Add GPU support in Flink

+1 Thanks for driving this effort, Ynagze. The latest FLIP wiki looks good to me. Cheers, Jiangjie (Becket) Qin On Wed, Apr 8, 2020 at 4:10 PM Yangze Guo wrote: > Edit: RuntimeContext interface > From: Map> > getExternalResourceInfo(ResourceSpec resourceSpec); > To: Map> getExternalResourceIn

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

Thanks for the inputs, Andrey & Zhijiang. # Mapped I did some experiments myself. It seems I was wrong about mapped buffer being accounted into max direct limit. Therefore, I agree with Zhijiang's suggestion that these metrics might be less valuable for users. # Overlapped Metrics It seems that

[jira] [Created] (FLINK-17061) Unset process/flink memory size from configuration once dynamic worker resource is activated.

Xintong Song created FLINK-17061: Summary: Unset process/flink memory size from configuration once dynamic worker resource is activated. Key: FLINK-17061 URL: https://issues.apache.org/jira/browse/FLINK-17061

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

Hi everyone ~ Thanks for the feedback, i would start a new vote again if there are no new input objections after 24 hours ~ Best, Danny Jark Wu 于2020年4月8日周三 下午7:19写道： > `table.dynamic-table-options.enabled` and `TableConfigOptions` sounds good > to me. > > Best, > Jark > > On Wed, 8 Apr 2020 a

[jira] [Created] (FLINK-17060) "{flink}" isn't being replaced by the Apache Flink trademark in Stateful Functions docs

Tzu-Li (Gordon) Tai created FLINK-17060: --- Summary: "{flink}" isn't being replaced by the Apache Flink trademark in Stateful Functions docs Key: FLINK-17060 URL: https://issues.apache.org/jira/browse/FLINK-17

[jira] [Created] (FLINK-17059) flink-shaded-hadoop-2-uber:2.8.3-10.0 has invalid pom in Gradle

John Lonergan created FLINK-17059: - Summary: flink-shaded-hadoop-2-uber:2.8.3-10.0 has invalid pom in Gradle Key: FLINK-17059 URL: https://issues.apache.org/jira/browse/FLINK-17059 Project: Flink

[jira] [Created] (FLINK-17058) Adding TimeoutTrigger support nested triggers

Roey Shem Tov created FLINK-17058: - Summary: Adding TimeoutTrigger support nested triggers Key: FLINK-17058 URL: https://issues.apache.org/jira/browse/FLINK-17058 Project: Flink Issue Type: I

[jira] [Created] (FLINK-17057) Add OpenSSL micro-benchmarks

Nico Kruber created FLINK-17057: --- Summary: Add OpenSSL micro-benchmarks Key: FLINK-17057 URL: https://issues.apache.org/jira/browse/FLINK-17057 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-17056) JMH main() methods call unrelated benchmarks

Nico Kruber created FLINK-17056: --- Summary: JMH main() methods call unrelated benchmarks Key: FLINK-17056 URL: https://issues.apache.org/jira/browse/FLINK-17056 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] Apache Flink Stateful Functions 2.0.0 released

Excited to see the stateful functions release! Thanks for the great work of manager Gordon and everyone who ever contributed to this. Best Yun Tang From: Till Rohrmann Sent: Wednesday, April 8, 2020 14:30 To: dev Cc: Oytun Tez ; user Subject: Re: [ANNOUNCE] Apa

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

I didn't know that. We should definitely implement this asap. Please open a JIRA issue. Thanks, Timo On 08.04.20 14:29, Zhenghua Gao wrote: Hi Timo, Actually "TEMPORARY" is not supported in table DDL now. But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP. And may be we s

[jira] [Created] (FLINK-17055) List jobs via bin/flink throws FlinkException indicating no cluster id was specified

Canbin Zheng created FLINK-17055: Summary: List jobs via bin/flink throws FlinkException indicating no cluster id was specified Key: FLINK-17055 URL: https://issues.apache.org/jira/browse/FLINK-17055

[jira] [Created] (FLINK-17054) TaskMailboxProcessorTest.testIdleTime() unstable

Robert Metzger created FLINK-17054: -- Summary: TaskMailboxProcessorTest.testIdleTime() unstable Key: FLINK-17054 URL: https://issues.apache.org/jira/browse/FLINK-17054 Project: Flink Issue Ty

[jira] [Created] (FLINK-17053) RecordWriterTest.testClearBuffersAfterInterruptDuringBlockingBufferRequest fails

Robert Metzger created FLINK-17053: -- Summary: RecordWriterTest.testClearBuffersAfterInterruptDuringBlockingBufferRequest fails Key: FLINK-17053 URL: https://issues.apache.org/jira/browse/FLINK-17053

[jira] [Created] (FLINK-17052) Introduce PlanGenerator

godfrey he created FLINK-17052: -- Summary: Introduce PlanGenerator Key: FLINK-17052 URL: https://issues.apache.org/jira/browse/FLINK-17052 Project: Flink Issue Type: Sub-task Components

[jira] [Created] (FLINK-17051) Persist spilled records of SpillingAdaptiveSpanningRecordDeserializer

Piotr Nowojski created FLINK-17051: -- Summary: Persist spilled records of SpillingAdaptiveSpanningRecordDeserializer Key: FLINK-17051 URL: https://issues.apache.org/jira/browse/FLINK-17051 Project: F

[jira] [Created] (FLINK-17050) Remove methods getVertex() and getResultPartition() from SchedulingTopology

Gary Yao created FLINK-17050: Summary: Remove methods getVertex() and getResultPartition() from SchedulingTopology Key: FLINK-17050 URL: https://issues.apache.org/jira/browse/FLINK-17050 Project: Flink

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

Hi Timo, Actually "TEMPORARY" is not supported in table DDL now. But you are right I could support "CREATE TEMPORARY VIEW" in this FLIP. And may be we should open a separate JIRA ticket to track supporting it in table DDL？ *Best Regards,* *Zhenghua Gao* On Wed, Apr 8, 2020 at 7:48 PM Timo Walth

[jira] [Created] (FLINK-17049) Support implicit conversions in nested rows

Dawid Wysakowicz created FLINK-17049: Summary: Support implicit conversions in nested rows Key: FLINK-17049 URL: https://issues.apache.org/jira/browse/FLINK-17049 Project: Flink Issue Typ

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

Hi Zhenghua, FLINK-10232 is quite old and a lot of stuff was discussed and agreed on since then. I don't like to postpone the 'TEMPORARY' keyword because it is a important concept that is already part of the Table API (see TableEnvironment.createTemporaryView) and in function DDL and table DDL

[PSA] Please check your github email configuration when merging on Github

Hi Everyone, we have a lot of commits recently that were committed by "GitHub ". This happens when your GitHub account is not configured correctly with respect to your email address. Please make sure that your commits somehow show who is the committer. For reference, check out https://help.

Re: [DISCUSS] Consolidated log4j2-properties file

INFO output usually not enough for debugging. In some cases it may be sufficient, but if you go beyond that then the dev still has to know to look into one specific module, bringing us back to square one. I do think though that a dedicated module for sharing these resources would actually be

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

`table.dynamic-table-options.enabled` and `TableConfigOptions` sounds good to me. Best, Jark On Wed, 8 Apr 2020 at 18:59, Danny Chan wrote: > `table.dynamic-table-options.enabled` seems fine to me, I would make a new > `TableConfigOptions` class and put the config option there ~ > > What do you

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

`table.dynamic-table-options.enabled` seems fine to me, I would make a new `TableConfigOptions` class and put the config option there ~ What do you think about the new class to put ? Best, Danny Chan 在 2020年4月8日 +0800 PM5:33，dev@flink.apache.org，写道： > > `table.dynamic-table-options.enabled`

[jira] [Created] (FLINK-17048) Add memory related JVM args to Mesos JM startup scripts

Andrey Zagrebin created FLINK-17048: --- Summary: Add memory related JVM args to Mesos JM startup scripts Key: FLINK-17048 URL: https://issues.apache.org/jira/browse/FLINK-17048 Project: Flink

[jira] [Created] (FLINK-17047) Simplify SchedulingStrategy#onPartitionConsumable(...) parameters

Zhu Zhu created FLINK-17047: --- Summary: Simplify SchedulingStrategy#onPartitionConsumable(...) parameters Key: FLINK-17047 URL: https://issues.apache.org/jira/browse/FLINK-17047 Project: Flink Issu

[jira] [Created] (FLINK-17046) SavepointWriterITCase failed on travis

Zhu Zhu created FLINK-17046: --- Summary: SavepointWriterITCase failed on travis Key: FLINK-17046 URL: https://issues.apache.org/jira/browse/FLINK-17046 Project: Flink Issue Type: Bug Compon

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

On 08.04.20 04:27, Jark Wu wrote: I have a minor concern about the global configuration `table.optimizer.dynamic-table-options.enabled`, does it belong to optimizer? From my point of view, it is just an API to set table options and uses Calcite in the implementation. I'm also thinking about wha

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

Another concern about "CREATE DDL" is: FLINK-10232 proposes using "IF NOT EXISTS" to control the behavior when a view or table with the same name already exists. And "OR REPLACE" for type/library/function DDL. @godfrey he I will keep the "IF NOT EXISTS" syntax and postpone the "OR REPLACE" synta

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

Hi Timo, Shall we postpone the support of 'TEMPORARY' keyword since it's not mentioned in FLINK-10232? *Best Regards,* *Zhenghua Gao* On Wed, Apr 8, 2020 at 3:30 PM Timo Walther wrote: > Hi Zhenghua, > > VIEWS should also support the TEMPORA

Re: [VOTE] FLIP-108: Add GPU support in Flink

Edit: RuntimeContext interface From: Map> getExternalResourceInfo(ResourceSpec resourceSpec); To: Map> getExternalResourceInfo(); Best, Yangze Guo On Wed, Apr 8, 2020 at 11:36 AM Yangze Guo wrote: > > Hi, there > > I have updated the FLIP, mainly target to make it more detailed and > clear. The

Re: [DISCUSS] FLIP-105: Support to Interpret and Emit Changelog in Flink SQL

Hi Kurt, The JSON encoding of Debezium can be configured to include or exclude the message schema using the `value.converter.schemas.enable` properties. That's why we propose to have a `format.schema-include` property key to config how to parse the json. Besides, the encoding format of debezium i

Re: [DISCUSS] FLIP-71 - E2E View support in Flink SQL

Hi Zhenghua, VIEWS should also support the TEMPORARY keyword according to FLIP-64. Otherwise the FLIP looks good to me. Regards, Timo On 08.04.20 07:31, Zhenghua Gao wrote: @Danny Chan you‘re right. I have updated the doc. *Best Regards,* *Zhenghua Gao* On Wed, Apr 8, 2020 at 1:20 PM Da