[jira] [Created] (FLINK-18760) Redundant task managers should be released when there's no job running in session cluster

Xintong Song created FLINK-18760: Summary: Redundant task managers should be released when there's no job running in session cluster Key: FLINK-18760 URL: https://issues.apache.org/jira/browse/FLINK-18760

Re: [DISCUSS] Introduce SupportsParallelismReport and SupportsStatisticsReport for Hive and Filesystem

Hi Jingsong, Regarding SupportsParallelismReport, I think the streaming connectors can also benefit from it. I see some requirements from user ML that they want to control source/sink's parallelism instead to set them to global parallelism. Also, in our compony, we did this too. Jingsong Li

Re: [DISCUSS] Align the semantic of returning null from InputFormat#nextRecord

Hi Jingsong, Thanks for the input. Our current implementation is very like the way you posted. Btw, we are doing the unification of streaming/batch work using our Flink SQL engine, hope to contribute more in this realm. Jingsong Li 于2020年7月30日周四 上午11:49写道： > Hi Benchao, > > I'm very glad that

[jira] [Created] (FLINK-18759) Add readme.md for TPC-DS tools

Leonard Xu created FLINK-18759: -- Summary: Add readme.md for TPC-DS tools Key: FLINK-18759 URL: https://issues.apache.org/jira/browse/FLINK-18759 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-18758) Support debezium data type in debezium format

Leonard Xu created FLINK-18758: -- Summary: Support debezium data type in debezium format Key: FLINK-18758 URL: https://issues.apache.org/jira/browse/FLINK-18758 Project: Flink Issue Type:

Re: [DISCUSS] Align the semantic of returning null from InputFormat#nextRecord

Hi Benchao, I'm very glad that you are developing the ecology of batch~ More detail for #1 is in `CsvInputFormat`: - There are dirty records (Comments) in `CsvInputFormat.readRecord`, so unfortunately, it must return null. - So go on like this, will return null in `nextRecord` like #2, so

Re: [DISCUSS] Align the semantic of returning null from InputFormat#nextRecord

Thanks Jingsong for the response. If we plan to deprecate InputFormat in the near future, I think it's ok to keep current behavior as it is for now. I raised this discussion because I found that InputFormatSourceFunction is not working as expected while I'm developing a batch table source

[DISCUSS] Introduce SupportsParallelismReport and SupportsStatisticsReport for Hive and Filesystem

Hi all, ## SupportsParallelismReport Now that FLIP-95 [1] is ready, only Hive and Filesystem are still using the old interfaces. We are considering migrating to the new interface. However, one problem is that in the old interface implementation, connectors infer parallelism by itself instead

[jira] [Created] (FLINK-18757) FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce failed with "Topic XX not present in metadata after 60000 ms"

Dian Fu created FLINK-18757: --- Summary: FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce failed with "Topic XX not present in metadata after 6 ms" Key: FLINK-18757 URL:

[jira] [Created] (FLINK-18756) Support IF NOT EXISTS for CREATE TABLE statement

Jark Wu created FLINK-18756: --- Summary: Support IF NOT EXISTS for CREATE TABLE statement Key: FLINK-18756 URL: https://issues.apache.org/jira/browse/FLINK-18756 Project: Flink Issue Type: New

Re: [DISCUSS] Align the semantic of returning null from InputFormat#nextRecord

Hi Benchao, My understanding is that #1 treats null as the end of input. This means we should try our best to avoid returning null before the end of input in the implementation of `InputFormat`. Even if we have to return null, we have to return it at the end of input. I think this is the most

[jira] [Created] (FLINK-18755) RabbitMQ QoS Chinese Documentation

Austin Cawley-Edwards created FLINK-18755: - Summary: RabbitMQ QoS Chinese Documentation Key: FLINK-18755 URL: https://issues.apache.org/jira/browse/FLINK-18755 Project: Flink Issue

Re: [Discuss] Questions about SortedMapState in Stream API

Thanks for the reply. Yes, it's true that RocksDB only supports bytes-comparator which makes it challenging. I checked FLINK-6219 has already been closed with reason "The thing is this issue will be addressed after blink code gets merged, and we will open another Jira to track the requirements. "

Re: [DISCUSS] FLIP-132: Temporal Table DDL

Hi, Konstantin > > 1) A "Versioned Temporal Table DDL on source" can only be joined on the > PRIMARY KEY attribute, correct? Yes, the PRIMARY KEY would be join key. > > 2) Isn't it the time attribute in the ORDER BY clause of the VIEW definition > that defines > whether a event-time or

[jira] [Created] (FLINK-18754) Display Python streaming metrics in Grafana dashboard

Maximilian Michels created FLINK-18754: -- Summary: Display Python streaming metrics in Grafana dashboard Key: FLINK-18754 URL: https://issues.apache.org/jira/browse/FLINK-18754 Project: Flink

Re: [VOTE] FLIP-130: Support for Python DataStream API (Stateless Part)

Hi everyone, The voting time for FLIP-130 is up. I'm going to finalize the vote. Up till now, there are 4 +1 votes, 3 of which are binding: - Jincheng Sun (binding) - Dian Fu - Aljoscha (binding) - Hequn (binding) And there are no disapproving votes. Hereby, the FLIP-130 has been accepted.

[jira] [Created] (FLINK-18753) Support local recovery for Unaligned checkpoints

Roman Khachatryan created FLINK-18753: - Summary: Support local recovery for Unaligned checkpoints Key: FLINK-18753 URL: https://issues.apache.org/jira/browse/FLINK-18753 Project: Flink

[DISCUSS] FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)

Hi Everyone, my colleagues (in cc) and I would like to propose this FLIP for discussion. In short, we want to reduce the number of APIs that we have by deprecating the DataSet API. This is a big step for Flink, that's why I'm also cross-posting this to the User Mailing List. FLIP-131:

[jira] [Created] (FLINK-18752) Yarn ship logic should support files

Gyula Fora created FLINK-18752: -- Summary: Yarn ship logic should support files Key: FLINK-18752 URL: https://issues.apache.org/jira/browse/FLINK-18752 Project: Flink Issue Type: Improvement

Re: flink 1.11 CREATE VIEW + LATERAL TABLE 语法校验问题

Hi XiaChang, Dev mailing list is used for discussing technical designs and proposals. Please ask user questions in u...@flink.apache.org or user...@flink.apache.org mailing list. Thanks, Jark On Wed, 29 Jul 2020 at 15:30, Wei Zhong wrote: > Hi XiaChang, > > I think this is a bug. Others have

Re: [VOTE] FLIP-129: Refactor Descriptor API to register connector in Table API

Hi all, The voting time for FLIP-129 has passed. I'm closing the vote now. There were 5 +1 votes, 4 of which are binding: - Timo (binding) - Jark (binding) - jincheng sun (binding) - Leonard Xu - Jingsong (binding) There were no disapproving votes. Thus, FLIP-129 has been accepted. Thanks

[jira] [Created] (FLINK-18751) Implement SlotSharingExecutionSlotAllocator

Andrey Zagrebin created FLINK-18751: --- Summary: Implement SlotSharingExecutionSlotAllocator Key: FLINK-18751 URL: https://issues.apache.org/jira/browse/FLINK-18751 Project: Flink Issue

Re: [DISCUSS] FLIP-132: Temporal Table DDL

Hi Leonard, Thank you for preparing this FLIP. I have a few questions. 1) A "Versioned Temporal Table DDL on source" can only be joined on the PRIMARY KEY attribute, correct? 2) The "processing time temporal table join" example under "how to correlate a temporal table" does not make sense to

Re: [VOTE] FLIP-129: Refactor Descriptor API to register connector in Table API

+1 On 29/07/2020 07:11, Jingsong Li wrote: > +1 > > Thanks Jark for driving this. > > On Mon, Jul 27, 2020 at 4:14 PM Leonard Xu wrote: > >> Thanks Jark >> >> +1(non-binding) >> >> Best >> Leonard >>> 在 2020年7月27日，15:32，jincheng sun 写道： >>> >>> +1（binding) >>> >>> Best, >>> Jincheng >>> >>> >>>

[jira] [Created] (FLINK-18750) SqlValidatorException thrown when select from a view which contains a UDTF call

Wei Zhong created FLINK-18750: - Summary: SqlValidatorException thrown when select from a view which contains a UDTF call Key: FLINK-18750 URL: https://issues.apache.org/jira/browse/FLINK-18750 Project:

Re: flink 1.11 CREATE VIEW + LATERAL TABLE 语法校验问题

Hi XiaChang, I think this is a bug. Others have encountered similar problems before. I have create a JIRA for it: https://issues.apache.org/jira/browse/FLINK-18750 Best, Wei > 在 2020年7月29日，15:00，XiaChang <13628620...@163.com> 写道： > >

flink 1.11 CREATE VIEW + LATERAL TABLE 语法校验问题

您好，请教一个问题： 背景： 运行环境-->flink1.11.1 source_table-->kafka输入表，kafka中每一个行为一个json数组。（[{},{}]） -- source CREATE TABLE source_table( __message STRING ) WITH( 'connector.type' = 'kafka', 'connector.version' = 'universal-xx', 'connector.properties.bootstrap.servers' = 'xxx', 'connector.topic' = 'xxx',

Re: flink1.11 可以使用processtime开窗，但是无法使用eventtime开窗

Sorry for the typo. The dev@flink.apache.org is used to discuss Flink Development like new features, vote, and releases, it’s improper to discuss user question here. Please discuss user question in u...@flink.apache.org , if you

Re: flink1.11 可以使用processtime开窗，但是无法使用eventtime开窗

Hi, taochabglian The u...@flink.apache.org is used to discuss Flink Development like new features, vote, and releases, it’s improper to discuss user question here. Please discuss user question in u...@flink.apache.org , if you prefer

[jira] [Created] (FLINK-18749) Correct dependencies in Kubernetes pom and notice file

Yang Wang created FLINK-18749: - Summary: Correct dependencies in Kubernetes pom and notice file Key: FLINK-18749 URL: https://issues.apache.org/jira/browse/FLINK-18749 Project: Flink Issue Type:

回复：flink1.11 可以使用processtime开窗，但是无法使用eventtime开窗

我们这都是大部分都是以使用eventtime进行处理居多，需要使用eventtime，则要在3个地方进行设置 第一： environment.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); 第二： SingleOutputStreamOperator add_event_time = hitchSPVLoggerSingleOutputStreamOperator.uid("add event time").assignTimestampsAndWatermarks(new