[jira] [Created] (FLINK-16643) flink-connector-hive_2.11 build fails on retrieving javax.jms:jms:jar:1.1

2020-03-18 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-16643: -- Summary: flink-connector-hive_2.11 build fails on retrieving javax.jms:jms:jar:1.1 Key: FLINK-16643 URL: https://issues.apache.org/jira/browse/FLINK-16643 Project

Re: Watermark generation in Temporal Table Join

2020-03-18 Thread Kurt Young
Hi, AFAIK there is no special watermark generation logic for temporal table join operator. Could you share your example's codes then I can help to analyze and debug? Best, Kurt On Tue, Mar 17, 2020 at 9:53 PM Dominik Wosiński wrote: > Hey Guys, > I have observed a weird behavior on using the

[jira] [Created] (FLINK-16644) HadoopS3FileSystemBehaviorITCase failed with null uri host

2020-03-18 Thread Zhijiang (Jira)
Zhijiang created FLINK-16644: Summary: HadoopS3FileSystemBehaviorITCase failed with null uri host Key: FLINK-16644 URL: https://issues.apache.org/jira/browse/FLINK-16644 Project: Flink Issue Type

[jira] [Created] (FLINK-16645) Limit the maximum backlogs in subpartitions for data skew case

2020-03-18 Thread Zhijiang (Jira)
Zhijiang created FLINK-16645: Summary: Limit the maximum backlogs in subpartitions for data skew case Key: FLINK-16645 URL: https://issues.apache.org/jira/browse/FLINK-16645 Project: Flink Issue

Re: [VOTE] FLIP-106: Support Python UDF in SQL Function DDL

2020-03-18 Thread zoudan
+1 (non-binding) Best, Dan Zou

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-18 Thread Benchao Li
Hi Timo, Thank you and others for the efforts to prepare this FLIP. The FLIP LGTM generally. +1 for moving blink data structures to table-common, it's useful to udf too in the future. A little question is, do we plan to support the new interfaces and data types in legacy planner? Or we only plan

[jira] [Created] (FLINK-16646) flink read orc file throw a NullPointerException

2020-03-18 Thread Jun Zhang (Jira)
Jun Zhang created FLINK-16646: - Summary: flink read orc file throw a NullPointerException Key: FLINK-16646 URL: https://issues.apache.org/jira/browse/FLINK-16646 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-16647) Miss file extension when inserting to hive table with compression

2020-03-18 Thread Rui Li (Jira)
Rui Li created FLINK-16647: -- Summary: Miss file extension when inserting to hive table with compression Key: FLINK-16647 URL: https://issues.apache.org/jira/browse/FLINK-16647 Project: Flink Issue

[jira] [Created] (FLINK-16648) Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment

2020-03-18 Thread jinhai (Jira)
jinhai created FLINK-16648: -- Summary: Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment Key: FLINK-16648 URL: https://issues.apache.org/jira/browse/FLINK-16648 Project: Flin

[jira] [Created] (FLINK-16649) Support Java 14

2020-03-18 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-16649: Summary: Support Java 14 Key: FLINK-16649 URL: https://issues.apache.org/jira/browse/FLINK-16649 Project: Flink Issue Type: Sub-task Compon

[jira] [Created] (FLINK-16650) Support LocalZonedTimestampType for Python UDF of blink planner

2020-03-18 Thread Dian Fu (Jira)
Dian Fu created FLINK-16650: --- Summary: Support LocalZonedTimestampType for Python UDF of blink planner Key: FLINK-16650 URL: https://issues.apache.org/jira/browse/FLINK-16650 Project: Flink Issue

[jira] [Created] (FLINK-16651) flink stream SQL INNER JOIN乱序

2020-03-18 Thread Jira
王太阳 created FLINK-16651: --- Summary: flink stream SQL INNER JOIN乱序 Key: FLINK-16651 URL: https://issues.apache.org/jira/browse/FLINK-16651 Project: Flink Issue Type: Bug Reporter: 王太阳 flink

[DISCUSS] (FLINK-16648) Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment

2020-03-18 Thread jinhai wang
Hi Devs I would like to start the formal discussion about FLINK-16648. StreamQueryConfig is deprecated and replaced by TableConfig in the future, and TableEnvironment class has config object. Do we need to delete the interfaces in StreamTableEnvironment and StreamTableEnvironmentImpl: toAppe

Re: [DISCUSS] (FLINK-16648) Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment

2020-03-18 Thread Jingsong Li
Hi Jinhai, thanks for driving. +1 to remove, I think we can remove StreamQueryConfig too. since we have deprecated StreamQueryConfig two versions. Remember to record it in release notes of issue. Best, Jingsong Lee On Wed, Mar 18, 2020 at 5:58 PM jinhai wang wrote: > Hi Devs > > I would like

Re: [DISCUSS] (FLINK-16648) Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment

2020-03-18 Thread jinhai wang
Also, QueryConfig is deprecated,we can remove BatchTableEnvironment and BatchTableEnvironmentImpl api about BatchQueryConfig? And remove QueryConfig interface finally Best Regards jinhai...@gmail.com > 2020年3月18日 下午6:05,Jingsong Li 写道: > > Hi Jinhai, thanks for driving. > > +1 to remove, I

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-18 Thread Stephan Ewen
>> The FLIP is "Filesystem connector in Table", it's about building up Flink Table's capabilities. That is exactly what worries me. The whole effort is not thinking about Flink as a whole any more. This proposal is not trying to build a consistent user experience across batch and streaming, across

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-18 Thread Till Rohrmann
Thanks for creating this FLIP Andrey. I like the general idea pretty much. I tried to group some of the above-mentioned points in order to give my 50 cent. # Configuration How to configure the Flink process seems to be the biggest question. Due to historical reasons we have a plethora of differe

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-18 Thread Kostas Kloudas
Hi all, I also agree with Stephan on this! It has been more than a year now that most of our efforts have had the "unify" / "unification"/ etc either on their title or in their core and this has been the focus of all our resources. By deviating from this now, we only put more stress on other team

[jira] [Created] (FLINK-16652) Hive 3.1.1 test is broken

2020-03-18 Thread Rui Li (Jira)
Rui Li created FLINK-16652: -- Summary: Hive 3.1.1 test is broken Key: FLINK-16652 URL: https://issues.apache.org/jira/browse/FLINK-16652 Project: Flink Issue Type: Test Components: Connecto

[jira] [Created] (FLINK-16653) Introduce ResultPartitionWriterTestBase for simplifying tests

2020-03-18 Thread Zhijiang (Jira)
Zhijiang created FLINK-16653: Summary: Introduce ResultPartitionWriterTestBase for simplifying tests Key: FLINK-16653 URL: https://issues.apache.org/jira/browse/FLINK-16653 Project: Flink Issue

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-18 Thread Timo Walther
Hi Benchao, this is a very good question. I will update the FLIP about this. The legacy planner will not support the new interfaces. It will only support the old interfaces. With the next release, I think the Blink planner is stable enough to be the default one as well. Regards, Timo On 18.

Re: [DISCUSS] (FLINK-16648) Clean up the input parameter StreamQueryConfig of the API in StreamTableEnvironment

2020-03-18 Thread Timo Walther
Hi Jinhai, yes, we can remove this deprecated method. Actually, I started with it but I stopped because we were missing a validation feature in ConfigOptions that ensures that min and max retention time satisfies some conditions (e.g. min is always smaller than max and some minimum difference

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-18 Thread Kurt Young
Hi all, Thanks for the discuss and feedbacks. I think this FLIP doesn't imply the implementation of such connector yet, it only describes the functionality and expected behaviors from user's perspective. Reusing current StreamingFileSink is definitely one of the possible ways to implement it. Sinc

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-18 Thread Timo Walther
Hi everyone, +1 to Kurt's suggestion. Let's just have it in source and sink factories for now. We can still move this method up in the future. Currently, I don't see a need for catalogs or formats. Because how would you target a format in the query? @Danny: Can you send a link to your PoC? I

[jira] [Created] (FLINK-16654) Implement Application Mode according to FLIP-85

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16654: -- Summary: Implement Application Mode according to FLIP-85 Key: FLINK-16654 URL: https://issues.apache.org/jira/browse/FLINK-16654 Project: Flink Issue Typ

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-18 Thread Kurt Young
Sorry maybe I didn't make myself clear. I think some format property is very suitable to be hinted, like "ignore errors during parsing". Maybe we should have a dedicated Hintable interface, and have `supportedHintOptions` method inside. All factories supports hint could implement from it. Best, K

[jira] [Created] (FLINK-16655) Introduce the EmbeddedExecutor

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16655: -- Summary: Introduce the EmbeddedExecutor Key: FLINK-16655 URL: https://issues.apache.org/jira/browse/FLINK-16655 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-16656) Introduce the DispatcherBootstrap

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16656: -- Summary: Introduce the DispatcherBootstrap Key: FLINK-16656 URL: https://issues.apache.org/jira/browse/FLINK-16656 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-16657) Wire the EmbeddedExecutor to the Web Submission logic.

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16657: -- Summary: Wire the EmbeddedExecutor to the Web Submission logic. Key: FLINK-16657 URL: https://issues.apache.org/jira/browse/FLINK-16657 Project: Flink Is

[jira] [Created] (FLINK-16658) Wire the EmbeddedExecutor to the StandaloneClusterEntrypoint.

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16658: -- Summary: Wire the EmbeddedExecutor to the StandaloneClusterEntrypoint. Key: FLINK-16658 URL: https://issues.apache.org/jira/browse/FLINK-16658 Project: Flink

[jira] [Created] (FLINK-16659) flink 1.9.1 流排序(普通模式、CEP模式)多并发情况下分组异常

2020-03-18 Thread wuchangjun (Jira)
wuchangjun created FLINK-16659: -- Summary: flink 1.9.1 流排序(普通模式、CEP模式)多并发情况下分组异常 Key: FLINK-16659 URL: https://issues.apache.org/jira/browse/FLINK-16659 Project: Flink Issue Type: Bug C

[jira] [Created] (FLINK-16660) Introduce the ApplicationDispatcherBootstrap

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16660: -- Summary: Introduce the ApplicationDispatcherBootstrap Key: FLINK-16660 URL: https://issues.apache.org/jira/browse/FLINK-16660 Project: Flink Issue Type:

[jira] [Created] (FLINK-16661) Introduce the YarnApplicationClusterEntrypoint

2020-03-18 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-16661: -- Summary: Introduce the YarnApplicationClusterEntrypoint Key: FLINK-16661 URL: https://issues.apache.org/jira/browse/FLINK-16661 Project: Flink Issue Type

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-18 Thread Danny Chan
My POC is here for the hints options merge [1]. Personally, I have no strong objections for splitting hints with the CatalogTable, the only cons is a more complex implementation but the concept is more clear, and I have updated the WIKI. I think it would be nice if we can support the format “ig

PackagedProgram and ProgramDescription

2020-03-18 Thread Flavio Pompermaier
Hi all, what do you think if we exploit this job-submission sprint to address also the problem discussed in https://issues.apache.org/jira/browse/FLINK-10862? Best, Flavio

Re: [DISCUSS] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Andrey Zagrebin
Hi all, Thanks for the feedback, Xintong and Till. > rename jobmanager.memory.direct.size into jobmanager.memory.off-heap.size I am ok with that to align it with TM and avoid further complications for users. I will adjust the FLIP. > change the default value of JM Metaspace size to 256 MB Inde

[VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Andrey Zagrebin
Hi All, The discussion for FLIP-116 looks to be resolved [1]. Therefore, I start the vote for it. The vote will end at 6pm CET on Monday, 23 March. Best, Andrey [1] http://mail-archives.apache.org/mod_mbox/flink-dev/202003.mbox/%3CCAJNyZN7AJAU_RUVhnWa7r%2B%2BtXpmUqWFH%2BG0hfoLVBzgRMmAO2w%40mail.

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-18 Thread Piotr Nowojski
Hi Kurt, +1 for having some offline discussion on this topic. But I think the question about using StreamingFileSink or implementing subset of it’s feature from scratch is quite fundamental design decision, with impact on the behaviour of Public API, so I wouldn’t discard it as technical detail

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-18 Thread Jark Wu
Hi everyone, Sorry, but I'm not sure about the `supportedHintOptions`. I'm afraid it doesn't solve the problems but increases some development and learning burdens. # increase development and learning burden According to the discussion so far, we want to support overriding a subset of options in

[jira] [Created] (FLINK-16662) Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)

2020-03-18 Thread chenxyz (Jira)
chenxyz created FLINK-16662: --- Summary: Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name) Key: FLINK-16662 URL: https://issues.apache.org/jira/browse/FL

Re: [DISCUSS] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Andrey Zagrebin
Hi all, One thing more thing to mention, the current calculations can lead to arbitrary small JVM Heap, maybe even zero. I suggest to introduce a check where we at least recommend to set the JVM heap to e.g. 128Mb. Additionally, we can demand some minimum value to function and fail if it is not f

[jira] [Created] (FLINK-16663) Docs version 1.10 missing from version picker dropdown

2020-03-18 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-16663: Summary: Docs version 1.10 missing from version picker dropdown Key: FLINK-16663 URL: https://issues.apache.org/jira/browse/FLINK-16663 Project: Flink Issue

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-18 Thread Jingsong Li
Hi Stephan & Kostas & Piotrek, thanks for these inputs, Maybe what I expressed is not clear. For the implementation, I want to know what you think, rather than must making another set from scratch. Piotrek you are right, implementation is the part of this FLIP too, because we can not list all deta

[jira] [Created] (FLINK-16664) Unable to set DataStreamSource parallelism to default (-1)

2020-03-18 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-16664: --- Summary: Unable to set DataStreamSource parallelism to default (-1) Key: FLINK-16664 URL: https://issues.apache.org/jira/browse/FLINK-16664 Project: Flink Issu

Re: [VOTE] FLIP-106: Support Python UDF in SQL Function DDL

2020-03-18 Thread Wei Zhong
Thanks everyone for the votes! I’ll summarize the voting result in a separate email. Best, Wei > 在 2020年3月18日,15:37,zoudan 写道: > > +1 (non-binding) > > Best, > Dan Zou >

[RESULT][VOTE] FLIP-106: Support Python UDF in SQL Function DDL

2020-03-18 Thread Wei Zhong
Hi all, The voting time for FLIP-106 has passed. I'm closing the vote now. There were 5 +1 votes, 3 of which are binding: - Jincheng (binding) - Hequn (binding) - Dian (binding) - Benchao (non-binding) - Dan (non-binding) There were no disapproving votes. Thus, FLIP-106 has been accepted. Than

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-18 Thread Wei Zhong
+1 (non-binding) Best, Wei > 在 2020年3月17日,19:15,Dian Fu 写道: > > +1 (binding) > > On Tue, Mar 17, 2020 at 10:35 AM jincheng sun > wrote: > >> +1 >> >> Best, >> Jincheng >> >> >> >> Hequn Cheng 于2020年3月16日周一 上午10:01写道: >> >>> Hi everyone, >>> >>> I'd like to start the vote of FLIP-112[1

[jira] [Created] (FLINK-16665) Support Python UDF in SQL Function DDL (FLIP-106)

2020-03-18 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-16665: - Summary: Support Python UDF in SQL Function DDL (FLIP-106) Key: FLINK-16665 URL: https://issues.apache.org/jira/browse/FLINK-16665 Project: Flink Issue Type: New F

[jira] [Created] (FLINK-16666) Support new Python dependency configuration options in flink-java, flink-streaming-java and flink-table

2020-03-18 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-1: - Summary: Support new Python dependency configuration options in flink-java, flink-streaming-java and flink-table Key: FLINK-1 URL: https://issues.apache.org/jira/browse/FLINK-1

[jira] [Created] (FLINK-16667) Support new Python dependency configuration options in flink-client

2020-03-18 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-16667: - Summary: Support new Python dependency configuration options in flink-client Key: FLINK-16667 URL: https://issues.apache.org/jira/browse/FLINK-16667 Project: Flink

[jira] [Created] (FLINK-16668) Support new Python dependency configuration options in flink-python

2020-03-18 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-16668: - Summary: Support new Python dependency configuration options in flink-python Key: FLINK-16668 URL: https://issues.apache.org/jira/browse/FLINK-16668 Project: Flink

[jira] [Created] (FLINK-16669) Support Python UDF in SQL function DDL

2020-03-18 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-16669: - Summary: Support Python UDF in SQL function DDL Key: FLINK-16669 URL: https://issues.apache.org/jira/browse/FLINK-16669 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Xintong Song
I think recommend a minimum value in docs and throw a warning if the heap size is too small should be good enough. Not sure about failing job if the min heap is not fulfilled. As already mentioned, it would be hard to determine the min heap size. And if we make the min heap configurable, then in an

Re: [VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Xintong Song
+1 (non-binding) from my side. Thank you~ Xintong Song On Wed, Mar 18, 2020 at 10:16 PM Andrey Zagrebin wrote: > Hi All, > > The discussion for FLIP-116 looks to be resolved [1]. > Therefore, I start the vote for it. > The vote will end at 6pm CET on Monday, 23 March. > > Best, > Andrey > >

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-18 Thread Hequn Cheng
+1 (binding) Best, Hequn On Thu, Mar 19, 2020 at 10:39 AM Wei Zhong wrote: > +1 (non-binding) > > Best, > Wei > > > 在 2020年3月17日,19:15,Dian Fu 写道: > > > > +1 (binding) > > > > On Tue, Mar 17, 2020 at 10:35 AM jincheng sun > > wrote: > > > >> +1 > >> > >> Best, > >> Jincheng > >> > >> > >> > >

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-18 Thread Hequn Cheng
Thanks all for the votes. So far, we have - 3 binding +1 votes (Jincheng, Dian, and Hequn) - 1 non-binding +1 votes (Wei) - 0 -1 votes I will move the FLIP to accepted! Best, Hequn On Thu, Mar 19, 2020 at 2:07 PM Hequn Cheng wrote: > +1 (binding) > > Best, Hequn > > On Thu, Mar 19, 2

[jira] [Created] (FLINK-16670) Support User-Defined Metrics in Python UDF

2020-03-18 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16670: --- Summary: Support User-Defined Metrics in Python UDF Key: FLINK-16670 URL: https://issues.apache.org/jira/browse/FLINK-16670 Project: Flink Issue Type: New Fea

[jira] [Created] (FLINK-16671) Support define scopes and variables on Python metric group

2020-03-18 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16671: --- Summary: Support define scopes and variables on Python metric group Key: FLINK-16671 URL: https://issues.apache.org/jira/browse/FLINK-16671 Project: Flink Issu

[jira] [Created] (FLINK-16672) Support Counter, Gauge, Meter, Distribution metric type for Python UDF

2020-03-18 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16672: --- Summary: Support Counter, Gauge, Meter, Distribution metric type for Python UDF Key: FLINK-16672 URL: https://issues.apache.org/jira/browse/FLINK-16672 Project: Flink

[jira] [Created] (FLINK-16673) Support Counter, Gauge, Meter, Distribution metric type for Python UDTF

2020-03-18 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16673: --- Summary: Support Counter, Gauge, Meter, Distribution metric type for Python UDTF Key: FLINK-16673 URL: https://issues.apache.org/jira/browse/FLINK-16673 Project: Flink

[jira] [Created] (FLINK-16674) Add documentation about how to use user-defined metrics for Python UDF

2020-03-18 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16674: --- Summary: Add documentation about how to use user-defined metrics for Python UDF Key: FLINK-16674 URL: https://issues.apache.org/jira/browse/FLINK-16674 Project: Flink

Re: [VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-18 Thread Yang Wang
+1 (non-binding). This FLIP will make jobmanager memory configuration keep up with taskmanager. Best, Yang Xintong Song 于2020年3月19日周四 上午11:23写道: > +1 (non-binding) from my side. > > Thank you~ > > Xintong Song > > > > On Wed, Mar 18, 2020 at 10:16 PM Andrey Zagrebin > wrote: > > > Hi All, >

[jira] [Created] (FLINK-16675) TableEnvironmentITCase. testClearOperation fails on travis nightly build

2020-03-18 Thread Yu Li (Jira)
Yu Li created FLINK-16675: - Summary: TableEnvironmentITCase. testClearOperation fails on travis nightly build Key: FLINK-16675 URL: https://issues.apache.org/jira/browse/FLINK-16675 Project: Flink I