[jira] [Created] (FLINK-28089) Hive dialect support "tablesample (xx rows)"

2022-06-16 Thread luoyuxia (Jira)
luoyuxia created FLINK-28089: Summary: Hive dialect support "tablesample (xx rows)" Key: FLINK-28089 URL: https://issues.apache.org/jira/browse/FLINK-28089 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-28090) Support attachAsDatastream

2022-06-16 Thread Zixuan Rao (Jira)
Zixuan Rao created FLINK-28090: -- Summary: Support attachAsDatastream Key: FLINK-28090 URL: https://issues.apache.org/jira/browse/FLINK-28090 Project: Flink Issue Type: Improvement Comp

[RESULT][VOTE] FLIP-228: Support Within between events in CEP Pattern

2022-06-16 Thread Nicholas
Hi dev, FLIP-228: Support Within between events in CEP Pattern[1] has been accepted. There are 3 binding votes, 5 non-binding votes[2]. - Martijn Visser(binding) - Dian Fu(binding) - Xingbo Huang(binding) - md peng(non-binding) - Rui Fan(non-binding) - Utopia(non-binding) - yue ma(no

[jira] [Created] (FLINK-28091) Thread leak in CheckpointResourcesCleanupRunnerTest#testCancellationAfterStart

2022-06-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-28091: Summary: Thread leak in CheckpointResourcesCleanupRunnerTest#testCancellationAfterStart Key: FLINK-28091 URL: https://issues.apache.org/jira/browse/FLINK-28091

[jira] [Created] (FLINK-28092) Support ASCII and CHR built-in function in the Table API

2022-06-16 Thread LuNing Wang (Jira)
LuNing Wang created FLINK-28092: --- Summary: Support ASCII and CHR built-in function in the Table API Key: FLINK-28092 URL: https://issues.apache.org/jira/browse/FLINK-28092 Project: Flink Issue

[jira] [Created] (FLINK-28093) Flink SQL Kafka Source Can Not Support Recovery On Checkpoint

2022-06-16 Thread Jiangfei Liu (Jira)
Jiangfei Liu created FLINK-28093: Summary: Flink SQL Kafka Source Can Not Support Recovery On Checkpoint Key: FLINK-28093 URL: https://issues.apache.org/jira/browse/FLINK-28093 Project: Flink

Re: [VOTE] Deprecate SourceFunction API

2022-06-16 Thread Jing Ge
+1 Thanks for driving this! Best regards, Jing On Wed, Jun 15, 2022 at 8:03 PM Alexander Fedulov wrote: > Hi everyone, > > following the discussion in [1], I would like to open up a vote for > deprecating the SourceFunction API. > > An overview of the steps required for being able to drop this

Re: [VOTE] FLIP-222: Support full job lifecycle statements in SQL client

2022-06-16 Thread godfrey he
+1 Best, Godfrey Martijn Visser 于2022年6月13日周一 20:25写道: > > +1 (binding) > > Op ma 13 jun. 2022 om 13:52 schreef Jark Wu : > > > +1 (binding) > > > > nit: the stop job syntax should be > > STOP JOB '' [WITH SAVEPOINT] [WITH DRAIN] > > which means savepoint and drain can be enabled at the same tim

Re: Re: [DISCUSS] FLIP-240: Introduce "ANALYZE TABLE" Syntax

2022-06-16 Thread Jark Wu
Hi Godfrey, > we just need a JIRA to support it. Could you create the JIRA issue? I think it would be better if we can support `DESC EXTENDED` and `ANALYZE TABLE` together in the 1.16 release. Otherwise, it's hard for users to determine when to call ANALYZE TABLE. Best, Jark On Thu, 16 Jun 2022

[jira] [Created] (FLINK-28094) [aws][Kinesis][Firehose] Upgrade AWS SDK to support ap-southeast-3

2022-06-16 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-28094: - Summary: [aws][Kinesis][Firehose] Upgrade AWS SDK to support ap-southeast-3 Key: FLINK-28094 URL: https://issues.apache.org/jira/browse/FLINK-28094 Project: Flink

[jira] [Created] (FLINK-28095) Replace IOUtils dependency on oss filesystem

2022-06-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-28095: Summary: Replace IOUtils dependency on oss filesystem Key: FLINK-28095 URL: https://issues.apache.org/jira/browse/FLINK-28095 Project: Flink Issue Ty

[jira] [Created] (FLINK-28096) Hive dialect support set variable

2022-06-16 Thread luoyuxia (Jira)
luoyuxia created FLINK-28096: Summary: Hive dialect support set variable Key: FLINK-28096 URL: https://issues.apache.org/jira/browse/FLINK-28096 Project: Flink Issue Type: Sub-task Re

[DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Gabor Somogyi
Hi All, I've just tried to delete long time deprecated configs and faced some inconveniences. My main intention is to understand what was the idea when japicmp-maven-plugin introduced related configs/functions marked with @Deprecated annotation. When I've removed the mentioned configs from source

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Chesnay Schepler
The japicm-plugin did exactly what it was supposed to be doing. We can't remove these fields because they are part of a @Public class and hence have to remain until Flink 2.0. Adding an exclusion for @Deprecated is a bad idea because it makes it way to easy to break the API. On 16/06/2022 1

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Gabor Somogyi
OK, I see your point. Just a question here. Mesos is deprecated in 1.13 here: https://issues.apache.org/jira/browse/FLINK-22352 and now I don't see any Mesos related classes in 1.16-SNAPSHOT. No matter from which angle I take a look at it this is breaking change. What makes the 2 cases different?

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Chesnay Schepler
a) Nothing in there was @Public afaict b) even if a) weren't the case, we voted on it c) there was an appreciable benefit in dropping Mesos, since it paved the way for Scala-free Flink On 16/06/2022 12:59, Gabor Somogyi wrote: OK, I see your point. Just a question here. Mesos is deprecated in

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Chesnay Schepler
d) The japicmp plugin is about programmatic API compatibility, which doesn't cover Mesos support. On 16/06/2022 12:59, Gabor Somogyi wrote: OK, I see your point. Just a question here. Mesos is deprecated in 1.13 here: https://issues.apache.org/jira/browse/FLINK-22352 and now I don't see any Me

Re: [VOTE] Deprecate SourceFunction API

2022-06-16 Thread Martijn Visser
+1 (binding) Thanks again for opening this discussion Alex. Cheers, Martijn Op do 16 jun. 2022 om 11:36 schreef Jing Ge : > +1 > Thanks for driving this! > > Best regards, > Jing > > On Wed, Jun 15, 2022 at 8:03 PM Alexander Fedulov > > wrote: > > > Hi everyone, > > > > following the discussio

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Gyula Fóra
Chesnay, I think as Gabor said, we have removed modules and functionality in the past which basically made some old configuration keys useless (effectively removed). In this particular case we don't need to remove the static constants that store this config key, we can simply ignore them. This wou

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Yang Wang
Thanks Xintong for the explanation. It makes sense to leave the discussion about job result store in a dedicated thread. Best, Yang Xintong Song 于2022年6月16日周四 13:40写道: > My impression of JobResultStore is more about fault tolerance and high > availability. Using it for providing information t

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Gyula Fóra
In any case these configs are related to supporting some very very old Hadoop versions. Which are likely broken anyways. I think we can also vote on removing support for some of these keys together with discussing (and voting on) the minimum supported hadoop version. In other words I think we shou

Re: [VOTE] Deprecate SourceFunction API

2022-06-16 Thread Lijie Wang
+1 (non-binding) Thanks for driving this. Best, Lijie Martijn Visser 于2022年6月16日周四 19:07写道: > +1 (binding) > > Thanks again for opening this discussion Alex. > > Cheers, Martijn > > Op do 16 jun. 2022 om 11:36 schreef Jing Ge : > > > +1 > > Thanks for driving this! > > > > Best regards, > > Ji

[jira] [Created] (FLINK-28097) Remove bundling of formats-common in json/csv format

2022-06-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-28097: Summary: Remove bundling of formats-common in json/csv format Key: FLINK-28097 URL: https://issues.apache.org/jira/browse/FLINK-28097 Project: Flink

Re: [DISCUSS] How to delete deprecated configs/functions

2022-06-16 Thread Gabor Somogyi
Now I see the difference. All in all the main reason of this discussion is that I've found many super old Hadoop workarounds/unused configs/not working solutions. I think it would be beneficial for the community to remove/fix them somehow. There are 4-7 years old issues inside... This is a good ex

Re: [VOTE] Deprecate SourceFunction API

2022-06-16 Thread Becket Qin
+1 (binding) Thanks Alex. Jiangjie (Becket) Qin On Thu, Jun 16, 2022 at 7:16 PM Lijie Wang wrote: > +1 (non-binding) > > Thanks for driving this. > > Best, > Lijie > > Martijn Visser 于2022年6月16日周四 19:07写道: > > > +1 (binding) > > > > Thanks again for opening this discussion Alex. > > > > Cheer

[jira] [Created] (FLINK-28098) Refactor table store compactor

2022-06-16 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28098: Summary: Refactor table store compactor Key: FLINK-28098 URL: https://issues.apache.org/jira/browse/FLINK-28098 Project: Flink Issue Type: Improvement

Re: [VOTE] Release 1.14.5, release candidate #1

2022-06-16 Thread Dian Fu
+1 (binding) - Verified the checksum and signature - Reviewed the website PR - Go through the artifacts to be deployed to maven central repository, there should be no missing artifacts Regards, Dian On Tue, Jun 14, 2022 at 11:27 AM Xingbo Huang wrote: > +1 (non-binding) > > - verify signatures

Re: Contributing to Apache Flink

2022-06-16 Thread Martijn Visser
Hi Shubham, Apologies for the late reply. It's great to hear that you're so willing and enthusiastic to contribute towards Flink. As a next step after your starter tasks, one of the best ways to get involved is to have a look at some tickets that are test stabilities. For example, you could have

Re:Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-16 Thread Lihe Ma
Thanks for the suggestion, Yuan. How about naming the newly in-memory state-backend, which supports incremental checkpoint, as HeapStateBackend . And let the default state-backend still stay as HashMapStateBackend. By doing so, we can: 1) the default value of parameter state.backend.incremental

[jira] [Created] (FLINK-28099) KafkaSink timeout exception

2022-06-16 Thread Jira
Крючкова марина Сергеевна created FLINK-28099: - Summary: KafkaSink timeout exception Key: FLINK-28099 URL: https://issues.apache.org/jira/browse/FLINK-28099 Project: Flink Issue T

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Aitozi
Thanks Junhan for driving this. It a great improvement for the batch jobs. I'm looking forward to this feature in our internal use case. +1 for it. One more question: Are each attempts of the TaskManager or JobManager pods (if failure occurs) all be shown in the ui ? Best, Aitozi. Yang Wang 于2

Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-16 Thread Zakelly Lan
Thanks for bringing this up. I'm +1 on enabling the incremental checkpoint by default on RocksDB. But I also agree with Yuan about not enabling this on newly implemented incremental checkpoint for hashmap statebackend. I am wondering can we make it behave differently for different state backends w

Re: [DISCUSS] FLIP-238: Introduce FLIP-27-based Data Generator Source

2022-06-16 Thread Alexander Fedulov
Hi Jing, thanks for your thorough analysis. I agree with the points you make and also with the idea to approach the larger task of providing a universal (DataStream + SQL) data generator base iteratively. Regarding the name, the SourceFunction-based *DataGeneratorSource* resides in the *org.apache

Re: Access to Apache Flink wiki (FLIP submission)

2022-06-16 Thread Andriy Redko
Hi Folks, Gentle reminder please, :-) I would like to ask for write permissions to Apache Flink wiki space in order to create a FLIP proposal (username is reta). Thank you in advance. Best Regards, Andriy Redko rao> Hi Folks, rao> Following the FLIP submission process, I would like to ask

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Jing Ge
Hi Junhan These are must-to-have information for batch processing. Thanks for bringing it up. I have some comments: 1. It seems that most of these metrics are more interesting to batch jobs. Does it make sense to calculate them for pure streaming jobs too? 2. Why "duration is less interesting" w

[jira] [Created] (FLINK-28100) rocketmq-flink checkpoint

2022-06-16 Thread yantao (Jira)
yantao created FLINK-28100: -- Summary: rocketmq-flink checkpoint Key: FLINK-28100 URL: https://issues.apache.org/jira/browse/FLINK-28100 Project: Flink Issue Type: Technical Debt Components

[jira] [Created] (FLINK-28101) YARNHighAvailabilityITCase failed on azure

2022-06-16 Thread dalongliu (Jira)
dalongliu created FLINK-28101: - Summary: YARNHighAvailabilityITCase failed on azure Key: FLINK-28101 URL: https://issues.apache.org/jira/browse/FLINK-28101 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.14.5, release candidate #1

2022-06-16 Thread Shuiqiang Chen
+1(non-binding) - Verified the source code tag with release-1.14.5-rc1 - Verified the checksum and signature - Go through the artifacts to be deployed to maven central repository - Reviewed the website PR Thanks for driving this. Best, Shuiqiang Dian Fu 于2022年6月16日周四 20:41写道: > +1 (binding)

Re: Re: [DISCUSS] FLIP-240: Introduce "ANALYZE TABLE" Syntax

2022-06-16 Thread godfrey he
Hi, Jing. Thanks for the feedback. >When will the converted SELECT statement of the ANALYZE TABLE be > submitted? right after the CREATE TABLE? The SELECT job will be submitted only when `ANALYZE TABLE` is executed, and there is nothing to do with CREATE TABLE. Because the `ANALYZE TABLE` is tri

Re: Re: [DISCUSS] FLIP-240: Introduce "ANALYZE TABLE" Syntax

2022-06-16 Thread godfrey he
Hi Jark, I have created the issue and will be done in release 1.16, see https://issues.apache.org/jira/browse/FLINK-28074 Best, Godfrey Jark Wu 于2022年6月16日周四 18:03写道: > > Hi Godfrey, > > > we just need a JIRA to support it. > Could you create the JIRA issue? I think it would be better if we can

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Yangze Guo
Thanks for the feedback, Aitozi and Jing. > Are each attempts of the TaskManager or JobManager pods (if failure occurs) all be shown in the ui? The info of the prior execution attempts will be archived, you could refer to `ArchivedExecutionVertex$priorExecutions`. > It seems that most of these m

Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-16 Thread LuNing Wang
Strongly +1 Best, LuNing Wang Zakelly Lan 于2022年6月17日周五 00:15写道: > Thanks for bringing this up. > I'm +1 on enabling the incremental checkpoint by default on RocksDB. But I > also agree with Yuan about not enabling this on newly implemented > incremental checkpoint for hashmap statebackend. >

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Jiangang Liu
Thanks for the FLIP. It is helpful to track detail infos for completed jobs. I want to ask another question. In our environment, sometimes it is hard to distinguish jobs since the same job names may appear multi times in the completed jobs. Because a job may run multi times or different jobs have

[jira] [Created] (FLINK-28102) Flink AkkaRpcSystemLoader fails when temporary directory is a symlink

2022-06-16 Thread Prabhu Joseph (Jira)
Prabhu Joseph created FLINK-28102: - Summary: Flink AkkaRpcSystemLoader fails when temporary directory is a symlink Key: FLINK-28102 URL: https://issues.apache.org/jira/browse/FLINK-28102 Project: Flin

Re: [DISCUSS] FLIP-241: Completed Jobs Information Enhancement

2022-06-16 Thread Yangze Guo
Thanks for the input, Jiangang. I think it's a valid demand to distinguish completed jobs with the same name. - If they are different jobs, I think users need to give them different meaningful names respectively. - If they are exactly the same job, IIUC, what you need is to figure out the order. A

[jira] [Created] (FLINK-28103) Job cancelling api returns 404 when job is actually running

2022-06-16 Thread David (Jira)
David created FLINK-28103: - Summary: Job cancelling api returns 404 when job is actually running Key: FLINK-28103 URL: https://issues.apache.org/jira/browse/FLINK-28103 Project: Flink Issue Type: Bu