[jira] [Created] (FLINK-21553) WindowDistinctAggregateITCase#testHopWindow_Cube is unstable

2021-03-01 Thread Jark Wu (Jira)
Jark Wu created FLINK-21553: --- Summary: WindowDistinctAggregateITCase#testHopWindow_Cube is unstable Key: FLINK-21553 URL: https://issues.apache.org/jira/browse/FLINK-21553 Project: Flink Issue

[jira] [Created] (FLINK-21552) The managed memory was not released if exception was thrown in createPythonExecutionEnvironment

2021-03-01 Thread Dian Fu (Jira)
Dian Fu created FLINK-21552: --- Summary: The managed memory was not released if exception was thrown in createPythonExecutionEnvironment Key: FLINK-21552 URL: https://issues.apache.org/jira/browse/FLINK-21552

Re: [VOTE] Release 1.12.2, release candidate #2

2021-03-01 Thread Kurt Young
+1 (binding) - We mainly checked the patch of FLINK-20663 [1] and confirmed there is no OutOfManagedMemory error anymore. [1] https://issues.apache.org/jira/browse/FLINK-20663 Best, Kurt On Tue, Mar 2, 2021 at 12:41 PM Yu Li wrote: > +1 (binding) > > - Checked the diff between 1.12.1 and

[jira] [Created] (FLINK-21551) FlinkKafkaProducerITCase.testHappyPath fail

2021-03-01 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-21551: - Summary: FlinkKafkaProducerITCase.testHappyPath fail Key: FLINK-21551 URL: https://issues.apache.org/jira/browse/FLINK-21551 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-21550) ZooKeeperHaServicesTest.testSimpleClose fail

2021-03-01 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-21550: - Summary: ZooKeeperHaServicesTest.testSimpleClose fail Key: FLINK-21550 URL: https://issues.apache.org/jira/browse/FLINK-21550 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.12.2, release candidate #2

2021-03-01 Thread Yu Li
+1 (binding) - Checked the diff between 1.12.1 and 1.12.2-rc2: OK ( https://github.com/apache/flink/compare/release-1.12.1...release-1.12.2-rc2) - jackson version has been bumped to 2.10.5.1 through FLINK-21020 and all NOTICE files updated correctly - beanutils version has been bumped to

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-03-01 Thread Jark Wu
I prefer option#2 and I think this can make everyone happy. Best, Jark On Mon, 1 Mar 2021 at 18:22, Shengkai Fang wrote: > Hi, everyone. > > After the long discussion, I am fine with both choices. But I prefer the > second option that applies to both table modules and sql client. Just as >

[VOTE] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Leonard Xu
Hi all, I would like to start the vote for FLIP-162 [1], which has been discussed and reached a consensus in the discussion thread [2]. Please vote +1 to approve the FLIP, or -1 with a comment. The vote will be open until March 5th (72h), unless there is an objection or not enough votes.

[jira] [Created] (FLINK-21549) Support json serialization/deserialization for the push-down results

2021-03-01 Thread godfrey he (Jira)
godfrey he created FLINK-21549: -- Summary: Support json serialization/deserialization for the push-down results Key: FLINK-21549 URL: https://issues.apache.org/jira/browse/FLINK-21549 Project: Flink

Re: [DISCUSS] FLIP-165: Operator's Flame Graphs

2021-03-01 Thread Alexander Fedulov
Thanks Henry, I have some issues with subscribing with our domain (it is an alias). @All, this thread is a duplicate caused by some technical issues, sorry for that. Please ignore it and use the previous one with the same title instead for the discussion:

Re: [DISCUSS] FLIP-165: Operator's Flame Graphs

2021-03-01 Thread Henry Saputra
Hi Alexander, I had to moderate and accept your email to dev@ list. Could you subscribe to dev@ list for Apache Flink [1] to continue getting updates from your discussion thread? Thanks, Henry [1] https://flink.apache.org/community.html#mailing-lists On Mon, Mar 1, 2021 at 3:42 PM Alexander

[DISCUSS] FLIP-165: Operator's Flame Graphs

2021-03-01 Thread Alexander Fedulov
Hi All, I would like to start a discussion for FLIP-165: Operator's Flame Graphs [1] A Flame Graph [2] is a visualization that is very effective for providing answers to the questions like: - Which methods are currently consuming CPU resources? - How CPU utilization by one method compares to the

[DISCUSS] FLIP-165: Operator's Flame Graphs

2021-03-01 Thread Alexander Fedulov
Hi All, I would like to start a discussion for FLIP-165: Operator's Flame Graphs [1] A Flame Graph [2] is a visualization that is very effective for providing answers to the questions like: - Which methods are currently consuming CPU resources? - How CPU utilization by one method compares to the

Re: Watermarks when reading from file

2021-03-01 Thread Dominik Wosiński
Hey Till, You were obviously right, my bad here. My math was incorrect. The correct reasoning is that indeed first 5 days of october will be added to the window number 1 and the rest of days will end up in the second window. Solved! Thanks a lotte, Best Regards, Dom.

Re: Watermarks when reading from file

2021-03-01 Thread Dominik Wosiński
Hey, Thanks for the answer, as I've mentioned in the email the data range is only 30 days, for the tests I've used the data from october so I basically have timestamps starting at midningt of 1st october 2020 and finishing at 23:59 30 october 2020, so if I understand correctly this shouldn't

Re: [DISCUSS] Apache Flink Jira Process

2021-03-01 Thread Roman Khachatryan
Hi, Thanks for the proposal Konstantin, I like the ideas expressed there. I am a bit concerned about the new issue type "Technical Debt". In contrast to other issue types, it doesn't imply that someone will likely work on that. So it can linger until the bot closes it. Probably we need some

[jira] [Created] (FLINK-21548) keyBy operation produces skewed event distribution with low-cardinality keys

2021-03-01 Thread Iaroslav Zeigerman (Jira)
Iaroslav Zeigerman created FLINK-21548: -- Summary: keyBy operation produces skewed event distribution with low-cardinality keys Key: FLINK-21548 URL: https://issues.apache.org/jira/browse/FLINK-21548

Re: Watermarks when reading from file

2021-03-01 Thread Till Rohrmann
Hi Dominik, I think the problem could be that TumblingTimeWindows don't start with the timestamp of the first arriving event but start at a multiple of the window length. So when defining a 90 day tumbling window you define a window from 0 - 89, 90 - 179, If your data ranges from day 79 -

Re: [DISCUSS] Enabling more dynamic, or metadata-driven behaviors

2021-03-01 Thread Till Rohrmann
Hi Maciej, The Flink community highly appreciates any kind of feedback and improvement suggestions. W/o going into more detail for the problems you've encountered, it is very likely that other people have run into something similar as well. Hence, it is a good idea to share these limitations and

Re: [VOTE] Release 1.12.2, release candidate #2

2021-03-01 Thread Piotr Nowojski
+1 (binding) For the RC2 I have additionally confirmed that "stop-with-savepoint", and "stop-with-savepoint --drain" seems to be working. Piotrek pon., 1 mar 2021 o 11:18 Matthias Pohl napisał(a): > Thanks for managing release 1.12.2, Yuan & Roman. > > +1 (non-binding) > > - Verified

[jira] [Created] (FLINK-21547) Fix improper log level in TwoPhaseCommitSinkFunction

2021-03-01 Thread tim yu (Jira)
tim yu created FLINK-21547: -- Summary: Fix improper log level in TwoPhaseCommitSinkFunction Key: FLINK-21547 URL: https://issues.apache.org/jira/browse/FLINK-21547 Project: Flink Issue Type: Bug

Watermarks when reading from file

2021-03-01 Thread Dominik Wosiński
Hey, I have a question regarding DataStream created from multiple files in s3. I have several files in AWS s3, say the path is s3://files/, and then there are several folders for different days, so in the end the full paths look like : s3://files/day=1/file.parquet, s3://files/day=2/file.parquet.

Re: [VOTE] FLIP-151: Incremental snapshots for heap-based state backend

2021-03-01 Thread Piotr Nowojski
Thanks Roman for coming up with this proposal and driving this topic: +1 (binding) from my side Piotrek pon., 1 mar 2021 o 10:12 Roman Khachatryan napisał(a): > Hi everyone, > > since the discussion [1] about FLIP-151 [2] seems to have reached a > consensus, I'd like to start a formal vote

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Roman Khachatryan
@tzuli...@apache.org > instead of splitting into “sub-lists”, we should simply have dedicated “sub-topic maintainers” assigned. I think this could also work, but some mails may fall between the filters. @i...@ververica.com I guess the previous decision about StateFun ML was made in a bit

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Leonard Xu
> I'm fine with your proposal. But once we see users asking for better unified > semantics, we should not hesitate to introduce an option to give them more > flexibility. Yes, I agree that we should introduce the option once we received feedback requirement from user input. I will update this

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Igal Shilman
Hi Roman, Regarding StateFun having a separate mailing list, I'm ok with it going either-way, however when we first contributed the project there was already a discussion about having a separate mailing list for StateFun [1] and the feedback was having StateFun be part of the regular mailing

[jira] [Created] (FLINK-21546) Upgrade io.netty netty-codec in Flink (four findings)

2021-03-01 Thread Adam Roberts (Jira)
Adam Roberts created FLINK-21546: Summary: Upgrade io.netty netty-codec in Flink (four findings) Key: FLINK-21546 URL: https://issues.apache.org/jira/browse/FLINK-21546 Project: Flink Issue

[jira] [Created] (FLINK-21545) Running Kerberized YARN per-job on Docker test stalls on azure

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21545: Summary: Running Kerberized YARN per-job on Docker test stalls on azure Key: FLINK-21545 URL: https://issues.apache.org/jira/browse/FLINK-21545 Project:

[jira] [Created] (FLINK-21544) Upgrade Jackson databind version from 2.10.1 used in, at least, Flink Python jar

2021-03-01 Thread Adam Roberts (Jira)
Adam Roberts created FLINK-21544: Summary: Upgrade Jackson databind version from 2.10.1 used in, at least, Flink Python jar Key: FLINK-21544 URL: https://issues.apache.org/jira/browse/FLINK-21544

[jira] [Created] (FLINK-21543) when using FIFO compaction, I found sst being deleted on the first checkpoint

2021-03-01 Thread xiaogang zhou (Jira)
xiaogang zhou created FLINK-21543: - Summary: when using FIFO compaction, I found sst being deleted on the first checkpoint Key: FLINK-21543 URL: https://issues.apache.org/jira/browse/FLINK-21543

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Timo Walther
It is true that your proposal is kind of a conservative plan. I'm fine with your proposal. But once we see users asking for better unified semantics, we should not hesitate to introduce an option to give them more flexibility. Regards, Timo On 01.03.21 12:59, Leonard Xu wrote: Thanks Kurt

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Leonard Xu
Thanks Kurt and Timo for the feedbacks. >> I prefer to not introduce such config until we have to. Leonard's proposal >> already makes almost all users happy thus I think we can still wait. I could understand Kurt’s concern that we don't need rush to introduce this option util we have to,

[DISCUSS] Enabling more dynamic, or metadata-driven behaviors

2021-03-01 Thread Maciej Obuchowski
While working on project that's strongly metadata-driven I've had to overcome several deficiencies, or assumptions in Flink code. Project involves reading from hundreds (possibly thousands) kafka topics, each containing avro messages with different schemas. Then, various transformations and

[jira] [Created] (FLINK-21542) Add documentation for supporting INSERT INTO specific columns

2021-03-01 Thread Jark Wu (Jira)
Jark Wu created FLINK-21542: --- Summary: Add documentation for supporting INSERT INTO specific columns Key: FLINK-21542 URL: https://issues.apache.org/jira/browse/FLINK-21542 Project: Flink Issue

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Tzu-Li (Gordon) Tai
Hi, I feel that the issues Roman has pointed out so far, is less a problem of all topics (SQL / PyFlink / StateFun) being on the same list, and more a problem that we are missing dedicated groups of “user support shepherds” who are specifically responsible for individual topics on a day-to-day

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Roman Khachatryan
Thanks for your replies! @Konstantin Knauf > Why do you think the quality and speed of answers would improve with dedicated lists? If there is a question on something that you are not an expert in; then you either have to - pull in someone who is more experienced in it (more time on hops, esp.

Re: [DISCUSS] Apache Flink Jira Process

2021-03-01 Thread Konstantin Knauf
Hi Dawid, Thanks for the feedback. Do you think we should simply get rid of the "Trivial" priority then and use the "starter" label more aggressively? Best, Konstantin On Mon, Mar 1, 2021 at 11:44 AM Dawid Wysakowicz wrote: > Hi Konstantin, > > I also like the idea. > > Two comments: > > *

Re: [DISCUSS] Apache Flink Jira Process

2021-03-01 Thread Dawid Wysakowicz
Hi Konstantin, I also like the idea. Two comments: * you describe the "Trivial" priority as one that needs to be implemented immediately. First of all it is not used to often, but I think the way it works now is similar with a "starter" label. Tasks that are not bugs, are easy to implement and

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Dawid Wysakowicz
As others I'd also rather be -1 on splitting (even splitting out the statefun). Personally I don't find it problematic. I often find the subjects quite descriptive, they often include tags or mention which API they refer to. If they don't I am quite sure having separate sub-lists would not help

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Timo Walther
I would vote -0 here. I fear that we are creating potential silos where a team doesn't know what is going on in the other teams. Regards, Timo On 01.03.21 10:47, Jark Wu wrote: I also have some concerns about splitting python and sql. Because I have seen some SQL questions users reported

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-03-01 Thread Shengkai Fang
Hi, everyone. After the long discussion, I am fine with both choices. But I prefer the second option that applies to both table modules and sql client. Just as Timo said, the option `table.dml-sync` can improve the SQL script portability. Users don't need to modify the script and execute the

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Timo Walther
I agree that Leonard's last proposal makes "almost all" users happy. However, a config option (as Joe said) would make "all" user happy because they have the power to choose. I don't have a strong opinion on this proposal as it is bascially a mixture of both approaches: 1) "some magic using

Re: [VOTE] Release 1.12.2, release candidate #2

2021-03-01 Thread Matthias Pohl
Thanks for managing release 1.12.2, Yuan & Roman. +1 (non-binding) - Verified checksums and GPG of artifacts in [1] - Build the sources locally without errors - Started a local standalone cluster and deployed WordCount without problems (no suspicious logs identified) - Verified FLINK-21030 [2]

[jira] [Created] (FLINK-21541) Update the existing K8s application e2e test to also verify pod template

2021-03-01 Thread Yang Wang (Jira)
Yang Wang created FLINK-21541: - Summary: Update the existing K8s application e2e test to also verify pod template Key: FLINK-21541 URL: https://issues.apache.org/jira/browse/FLINK-21541 Project: Flink

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Jark Wu
I also have some concerns about splitting python and sql. Because I have seen some SQL questions users reported but is related to deployment or state backend. Best, Jark On Mon, 1 Mar 2021 at 17:15, Konstantin Knauf wrote: > Hi Roman, > > I slightly +1 for a list dedicated to Statefun users,

[jira] [Created] (FLINK-21540) finegrained_resource_management tests hang on azure

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21540: Summary: finegrained_resource_management tests hang on azure Key: FLINK-21540 URL: https://issues.apache.org/jira/browse/FLINK-21540 Project: Flink

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Konstantin Knauf
Hi Roman, I slightly +1 for a list dedicated to Statefun users, but -1 for splitting up the rest. I think there are still a lot of crosscutting concerns between Python, DataStream, Table API and SQL where users of another API can also help out, too. It also requires users to think about which

[jira] [Created] (FLINK-21539) 'SQL Client end-to-end test (Blink planner) Elasticsearch (v6.3.1)' fails during download

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21539: Summary: 'SQL Client end-to-end test (Blink planner) Elasticsearch (v6.3.1)' fails during download Key: FLINK-21539 URL: https://issues.apache.org/jira/browse/FLINK-21539

[VOTE] FLIP-151: Incremental snapshots for heap-based state backend

2021-03-01 Thread Roman Khachatryan
Hi everyone, since the discussion [1] about FLIP-151 [2] seems to have reached a consensus, I'd like to start a formal vote for the FLIP. Please vote +1 to approve the FLIP, or -1 with a comment. The vote will be open at least until Wednesday, Mar 3rd. [1]

[jira] [Created] (FLINK-21538) Elasticsearch6DynamicSinkITCase.testWritingDocuments fails when submitting job

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21538: Summary: Elasticsearch6DynamicSinkITCase.testWritingDocuments fails when submitting job Key: FLINK-21538 URL: https://issues.apache.org/jira/browse/FLINK-21538

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-03-01 Thread Kurt Young
I'm +1 for either: 1. introduce a sql client specific option, or 2. Introduce a table config option and make it apply to both table module & sql client. It would be the FLIP owner's call to decide. Best, Kurt On Mon, Mar 1, 2021 at 3:25 PM Timo Walther wrote: > We could also think about

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread xiao...@ysstech.com
Hi Roman, This is a very good idea. I will look forward to the official setting up "sub-lists" as soon as possible and sharing development experience and problems with friends in a certain field. Regards, yue xiao...@ysstech.com From: Roman Khachatryan Date: 2021-03-01 16:48 To: dev

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Kurt Young
I'm +1 to Leonard's last proposal, which: 1. Keep CURRENT_TIMESTAMP row level behavior in streaming mode, and make it evaluated at query start in batch mode. 2. Introduce CURRENT_ROW_TIMESTAMP for batch users who want such semantic. I'm slightly -1 for introducing an option because we are

[DISCUSS] Splitting User support mailing list

2021-03-01 Thread Roman Khachatryan
Hi everyone, I'd like to propose to extract several "sub-lists" from our user mailing list (u...@flink.apache.org). For example, - user-sql@flink.a.o (Python) - user-statefun@f.a.o (StateFun) - user-py@f.a.o. (SQL/TableAPI) And u...@flink.apache.org will remain the main or "default" list. That

Re: [DISCUSS] Apache Flink Jira Process

2021-03-01 Thread Konstantin Knauf
Hi Xintong, yes, such labels would make a lot of sense. I added a sentence to the document. Thanks, Konstantin On Mon, Mar 1, 2021 at 8:51 AM Xintong Song wrote: > Thanks for driving this discussion, Konstantin. > > I like the idea of having a bot reminding reporter/assignee/watchers about >

[jira] [Created] (FLINK-21537) SavepointITCase fails on azure

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21537: Summary: SavepointITCase fails on azure Key: FLINK-21537 URL: https://issues.apache.org/jira/browse/FLINK-21537 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-21536) AdaptiveSchedulerSlotSharingITCase.testSchedulingOfJobRequiringSlotSharing fails on azure

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21536: Summary: AdaptiveSchedulerSlotSharingITCase.testSchedulingOfJobRequiringSlotSharing fails on azure Key: FLINK-21536 URL:

[jira] [Created] (FLINK-21535) UnalignedCheckpointITCase.execute failed with "OutOfMemoryError: Java heap space"

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21535: Summary: UnalignedCheckpointITCase.execute failed with "OutOfMemoryError: Java heap space" Key: FLINK-21535 URL: https://issues.apache.org/jira/browse/FLINK-21535

[jira] [Created] (FLINK-21534) Test jars are uploaded twice during Flink maven deployment

2021-03-01 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-21534: -- Summary: Test jars are uploaded twice during Flink maven deployment Key: FLINK-21534 URL: https://issues.apache.org/jira/browse/FLINK-21534 Project: Flink

[jira] [Created] (FLINK-21533) Kafka011ITCase#testAllDeletes fails on azure

2021-03-01 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21533: Summary: Kafka011ITCase#testAllDeletes fails on azure Key: FLINK-21533 URL: https://issues.apache.org/jira/browse/FLINK-21533 Project: Flink Issue