Re: [VOTE] Release 1.12.1, release candidate #1

Hi Matthias, I think Becket's opinion is not to block this release on FLINK-20781, thus not to cancel the vote for this rc1. Since you have already voted -1, I'd like to understand your opinion on this release candidate now. Do you still believe we should not approve rc1 because of FLINK-20781, or

[jira] [Created] (FLINK-20798) Service temporarily unavailable due to an ongoing leader election. Please refresh.

hayden zhou created FLINK-20798: --- Summary: Service temporarily unavailable due to an ongoing leader election. Please refresh. Key: FLINK-20798 URL: https://issues.apache.org/jira/browse/FLINK-20798 Proj

Re: Support local aggregate push down for Blink batch planner

Hi Sebastian, Thanks for the proposal. I think this is a great improvement for Flink SQL. I went through the design doc and have following thoughts: 1) Flink has deprecated the legacy TableSource in 1.11 and proposed a new set of DynamicTableSource interfaces. Could you update your proposal to u

Re: [VOTE] Release 1.12.1, release candidate #1

I think Becket's proposal in [1] sounds reasonable having the quickfix in place for 1.12.1. Thanks, Matthias [1] https://issues.apache.org/jira/browse/FLINK-20781 On Mon, Dec 28, 2020 at 1:51 PM Xintong Song wrote: > @Becket, > > Thanks for the clarification. Sorry, I misunderstood what you me

Re: [ANNOUNCE] New formatting rules are now in effect

Yes, thanks for driving this, Aljoscha. ...and Chesnay as well for helping to finalize it. Good job. Matthias On Tue, Dec 29, 2020 at 5:23 AM Jark Wu wrote: > Thanks Aljoscha and Chesnay for the great work! > > Best, > Jark > > On Tue, 29 Dec 2020 at 11:11, Xintong Song wrote: > > > Great! Tha

[jira] [Created] (FLINK-20797) can flink on k8s use pv using NFS and pvc as the hight avalibility storagedir

hayden zhou created FLINK-20797: --- Summary: can flink on k8s use pv using NFS and pvc as the hight avalibility storagedir Key: FLINK-20797 URL: https://issues.apache.org/jira/browse/FLINK-20797 Project:

[jira] [Created] (FLINK-20796) Support unnest in the Table API

Dian Fu created FLINK-20796: --- Summary: Support unnest in the Table API Key: FLINK-20796 URL: https://issues.apache.org/jira/browse/FLINK-20796 Project: Flink Issue Type: Sub-task Componen

Re: [ANNOUNCE] New formatting rules are now in effect

Thanks Aljoscha and Chesnay for the great work! Best, Jark On Tue, 29 Dec 2020 at 11:11, Xintong Song wrote: > Great! Thanks Aljoscha and Chesnay for driving this. > > Thank you~ > > Xintong Song > > > > On Tue, Dec 29, 2020 at 1:28 AM Chesnay Schepler > wrote: > > > Hello everyone, > > > > I

[jira] [Created] (FLINK-20795) add a parameter to decide whether or not print dirty record when `ignore-parse-errors` is true

zoucao created FLINK-20795: -- Summary: add a parameter to decide whether or not print dirty record when `ignore-parse-errors` is true Key: FLINK-20795 URL: https://issues.apache.org/jira/browse/FLINK-20795 Pr

Re: [ANNOUNCE] New formatting rules are now in effect

Great! Thanks Aljoscha and Chesnay for driving this. Thank you~ Xintong Song On Tue, Dec 29, 2020 at 1:28 AM Chesnay Schepler wrote: > Hello everyone, > > I have just merged the commits for FLINK-20651 > to master, > release-1.12 and releas

[jira] [Created] (FLINK-20794) Support to select distinct columns in the Table API

Dian Fu created FLINK-20794: --- Summary: Support to select distinct columns in the Table API Key: FLINK-20794 URL: https://issues.apache.org/jira/browse/FLINK-20794 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-20793) Fix NamesTest due to code style refactor

Huang Xingbo created FLINK-20793: Summary: Fix NamesTest due to code style refactor Key: FLINK-20793 URL: https://issues.apache.org/jira/browse/FLINK-20793 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.12.1, release candidate #1

I think we may hold announcing this release until the 1.12.1 official image is available. For the other approaches discussed in FLINK-20632 (hosting Docker images ourselves, moving most of entrypoint scripts into the Flink repository, etc.), maybe we can address them in future releases. Thank you

Re: [VOTE] Release 1.12.1, release candidate #1

Should we maybe wait with this release until we have the docker issue sorted out? On 12/28/2020 1:51 PM, Xintong Song wrote: @Becket, Thanks for the clarification. Sorry, I misunderstood what you meant on the jira. Seems it's not necessary to cancel this vote. WDYT, @Matthias? Thank you~ Xi

[jira] [Created] (FLINK-20792) Allow shorthand invocation of spotless

Chesnay Schepler created FLINK-20792: Summary: Allow shorthand invocation of spotless Key: FLINK-20792 URL: https://issues.apache.org/jira/browse/FLINK-20792 Project: Flink Issue Type: Im

[ANNOUNCE] New formatting rules are now in effect

Hello everyone, I have just merged the commits for FLINK-20651 to master, release-1.12 and release-11, which enforces new formatting rules using the spotless plugin, following the google-java-format. This change touched every single java fi

[Discussion][FLINK-20416] Need a cached catalog for batch SQL job

Hi all, I‘d like to discuss a feature which supports the Flink OLAP scenario. For OLAP scenarios, There are usually some analytical queries which running time is relatively short. These queries are also sensitive to latency. In the current Blink sql processing, parse/validate/optimize stages are

Support local aggregate push down for Blink batch planner

Hi all, I'd like to discuss a new feature for the Blink Planner. Aggregate operator of Flink SQL is currently fully done at Flink layer. With the developing of storage, many downstream storage of Flink SQL has the ability to deal with Aggregation operator. Pushing down Aggregate to data source lay

Re: Insufficient number of network buffers after restarting

Hi Yufei, My prime suspect would be changes to the memory configuration introduced in 1.11 [1] Piotrek [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/release-notes/flink-1.11.html#memory-management pon., 28 gru 2020 o 09:52 Till Rohrmann napisał(a): > Hi Yufei, > > I cannot

[jira] [Created] (FLINK-20791) Support local aggregate push down for Blink batch planner

Sebastian Liu created FLINK-20791: - Summary: Support local aggregate push down for Blink batch planner Key: FLINK-20791 URL: https://issues.apache.org/jira/browse/FLINK-20791 Project: Flink I

[jira] [Created] (FLINK-20790) Generated classes should not be put under src/ directory

Chesnay Schepler created FLINK-20790: Summary: Generated classes should not be put under src/ directory Key: FLINK-20790 URL: https://issues.apache.org/jira/browse/FLINK-20790 Project: Flink

[jira] [Created] (FLINK-20789) Add a metric named `deserializeFaildCount` for kafka connectors

xiaozilong created FLINK-20789: -- Summary: Add a metric named `deserializeFaildCount` for kafka connectors Key: FLINK-20789 URL: https://issues.apache.org/jira/browse/FLINK-20789 Project: Flink

Re: [DISCUSS][FLINK-20726] Introduce Pulsar connector

Hi Jianyun, Thanks a lot for reviving this discussion. I think it would be great to have a well working Pulsar connector for Flink. Before diving into the detailed plan on how to do it technically, I think we should discuss where exactly the connector should live. From the Flink community's perspe

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Given that there might still be users who want to upgrade and, thus, are reading the release notes for the 1.12.0 release, I would also update the 1.12.0 release notes. Moreover, I would consider the risk of losing state because unaligned checkpoints might break recovery as quite a serious problem.

Re: [VOTE] Release 1.12.1, release candidate #1

@Becket, Thanks for the clarification. Sorry, I misunderstood what you meant on the jira. Seems it's not necessary to cancel this vote. WDYT, @Matthias? Thank you~ Xintong Song On Mon, Dec 28, 2020 at 8:42 PM Becket Qin wrote: > Hi Xintong, > > Thanks for driving the release. I just replie

Re: [VOTE] Release 1.12.1, release candidate #1

Hi Xintong, Thanks for driving the release. I just replied to FLINK-20781. Personally I don't think that is a blocker for release-1.12.1. The reasons are: 1. The root cause of the issue is thread unsafe shutdown sequence of the mailbox task. It is an issue for a while, SourceOperator just exposes

[jira] [Created] (FLINK-20788) It doesn't support to use cube/rollup/grouping sets in the Table API

Dian Fu created FLINK-20788: --- Summary: It doesn't support to use cube/rollup/grouping sets in the Table API Key: FLINK-20788 URL: https://issues.apache.org/jira/browse/FLINK-20788 Project: Flink I

Re: [VOTE] Release 1.12.1, release candidate #1

@Becket has confirmed on the jira ticket that FLINK-20781 is indeed a bug. I‘m hereby officially canceling this vote. At the same time, it is appreciated to continue verifying the release candidate and reporting issues, so they can be fixed by the next release candidate. The RC#2 will be created

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Adding it as a warning of known issues to 1.12.1 release notes makes sense to me. (If it doesn't get fixed in this release. I'm canceling 1.12.1-rc1 for another blocker.) I'm not entirely sure about adding a warning to 1.12.0 release notes. Is it how we usually do, adding warnings to release notes

[jira] [Created] (FLINK-20787) Improve the Table API to make it usable

Dian Fu created FLINK-20787: --- Summary: Improve the Table API to make it usable Key: FLINK-20787 URL: https://issues.apache.org/jira/browse/FLINK-20787 Project: Flink Issue Type: Improvement

Re: Changing application configuration when restoring from checkpoint/savepoint

On Wed Dec 16, 2020 at 6:41 PM CET, vishalovercome wrote: > 1. Is there any way to restore from a checkpoint as well as recreate > client > using newer configuration? I think that would only work if you somehow read the configuration from an external system > 2. If we take a savepoint (drain and

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Alright, thanks for the clarification. Should we issue a warning to not use unaligned checkpoints for the time being because it can lead to corrupted data streams on recovery? I can envision that some of our users might be surprised about it. Maybe adding it to the 1.12.0 and 1.12.1 release notes?

[jira] [Created] (FLINK-20786) flink-playgrounds Git branch(release-1.12) does not exist

billyxie created FLINK-20786: Summary: flink-playgrounds Git branch(release-1.12) does not exist Key: FLINK-20786 URL: https://issues.apache.org/jira/browse/FLINK-20786 Project: Flink Issue Type

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Hi, Yes, as Xintong wrote above, I've wrote offline to him: > I’m going to remove release blocker status from FLINK-20654. After all we already have released it, at least in 1.12.0, and maybe even sooner. There is no point from blocking a release (which has probably some important bug fixes) in t

[jira] [Created] (FLINK-20785) MapRelDataType for legacy planner has wrong digest

Danny Chen created FLINK-20785: -- Summary: MapRelDataType for legacy planner has wrong digest Key: FLINK-20785 URL: https://issues.apache.org/jira/browse/FLINK-20785 Project: Flink Issue Type: Ta

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Hi Till, @Piotr and @Roman mentioned offline that FLINK-20648 is not a new bug and they don't think we should block a release on it. I guess we should have made the conversation public visible. Sorry for the confusion. Thank you~ Xintong Song On Mon, Dec 28, 2020 at 4:59 PM Till Rohrmann

Re: [DISCUSS] Releasing Apache Flink 1.12.1

Hi Xintong, quick question, what about FLINK-20654? Previously it was listed as a release blocker but has not been fixed yet. Cheers, Till On Fri, Dec 25, 2020 at 10:52 AM Xintong Song wrote: > Hi devs, > > I'm very glad to announce that all known blocker issues for release-1.12.1 > have been

Re: Insufficient number of network buffers after restarting

Hi Yufei, I cannot remember exactly the changes in this area between Flink 1.10.0 and Flink 1.12.0. It sounds a bit as if we were not releasing memory segments fast enough or had a memory leak. One thing to try out is to increase the restart delay to see whether it is the first problem. Alternativ

[jira] [Created] (FLINK-20784) .staging_xxx does not exist, when insert into hive

macdoor615 created FLINK-20784: -- Summary: .staging_xxx does not exist, when insert into hive Key: FLINK-20784 URL: https://issues.apache.org/jira/browse/FLINK-20784 Project: Flink Issue Type: Bu