[jira] [Created] (FLINK-25214) FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on AZP with incorrect result

2021-12-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25214: - Summary: FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on AZP with incorrect result Key: FLINK-25214 URL: https://issues.apache.org/jira/browse/FLINK

Re: [DISCUSS] FLIP-197: API stability graduation process

2021-12-06 Thread Till Rohrmann
release cycle, a duration of one release would mean you fully mature the > > API in the same release in which things are still being fixed; > intuitively, > > it makes sense to me that the step to Public would come after a period of > > no changes needed, however. > >

Re: [VOTE] FLIP-194: Introduce the JobResultStore

2021-12-06 Thread Till Rohrmann
+1 (binding) Cheers, Till On Mon, Dec 6, 2021 at 10:00 AM David Morávek wrote: > Hi everyone, > > I'd like to open a vote on FLIP-194: Introduce the JobResultStore [1] which > has been > discussed in this thread [2]. > The vote will be open for at least 72 hours unless there is an objection or

Re: [DISCUSS] FLIP-196: Source API stability guarantees

2021-12-06 Thread Till Rohrmann
7;t > for example add an experimental method to an interface. > > This would also include all classes used as argument and return type of > such methods too, which seems quite restrictive. > > > Best > Ingo > > On Fri, Dec 3, 2021, 17:51 Till Rohrmann wrote: > > > >

[jira] [Created] (FLINK-25186) KafkaDynamicTableFactoryTest and UpsertKafkaDynamicTableFactoryTest fails on AZP

2021-12-06 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25186: - Summary: KafkaDynamicTableFactoryTest and UpsertKafkaDynamicTableFactoryTest fails on AZP Key: FLINK-25186 URL: https://issues.apache.org/jira/browse/FLINK-25186

[jira] [Created] (FLINK-25185) StreamFaultToleranceTestBase hangs on AZP

2021-12-06 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25185: - Summary: StreamFaultToleranceTestBase hangs on AZP Key: FLINK-25185 URL: https://issues.apache.org/jira/browse/FLINK-25185 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-25181) KafkaSourceITCase.testValueOnlyDeserializer fails on AZP

2021-12-06 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25181: - Summary: KafkaSourceITCase.testValueOnlyDeserializer fails on AZP Key: FLINK-25181 URL: https://issues.apache.org/jira/browse/FLINK-25181 Project: Flink

[jira] [Created] (FLINK-25180) Jepsen test fails while setting up libzip4

2021-12-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25180: - Summary: Jepsen test fails while setting up libzip4 Key: FLINK-25180 URL: https://issues.apache.org/jira/browse/FLINK-25180 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-196: Source API stability guarantees

2021-12-03 Thread Till Rohrmann
else entirely, which leads to > unfavorable design. I don't think we should disallow extending classes with > methods of a weaker stability. > > > Best > Ingo > > On Fri, Dec 3, 2021 at 4:53 PM Till Rohrmann wrote: > > > Hi Ingo, thanks a lot for your thought

Re: [DISCUSS] FLIP-197: API stability graduation process

2021-12-03 Thread Till Rohrmann
hat; if we mature APIs too quickly, we risk losing out > on important feedback. Therefore, I would propose starting slower here, and > rather think about shortening that cycle in the future. > > > Best > Ingo > > On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann wrote: > > &

Re: [DISCUSS] FLIP-196: Source API stability guarantees

2021-12-03 Thread Till Rohrmann
seems to be quite incompatible with the current interpretation in the > code base, and it would prevent valid (and in-use) use cases like marking > e.g. a single method experimental (or even internal) in an otherwise public > (evolving) API. > > [1] https://github.com/apache/fli

Re: [DISCUSS] FLIP-196: Source API stability guarantees

2021-12-03 Thread Till Rohrmann
/test/java/org/apache/flink/architecture/rules/ApiAnnotationRules.java Cheers, Till On Fri, Dec 3, 2021 at 11:59 AM Till Rohrmann wrote: > Hi Konstantin and Francesco, > > > * Are you planning to implement the required tests via ArchUnit? > > If ArchUnit is the best tool for th

Re: [DISCUSS] FLIP-196: Source API stability guarantees

2021-12-03 Thread Till Rohrmann
bility > > guarantees are important for our users as well as any downstream project > of > > Apache Flink. Documenting the de-facto status quo and tackling existing > > inconsistencies sounds like a good first step. So, +1 from my side. > > > > Two questions for clar

Re: [DISCUSS] Conventions on assertions to use in tests

2021-12-03 Thread Till Rohrmann
missed it? > > > > > > > > > > Matthias > > > > > > > > > > [1] > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Best+Practices+and+Lesso > > > > > ns+Learned> > > > &g

Re: [DISCUSS] Change some default config values of blocking shuffle

2021-12-03 Thread Till Rohrmann
Thanks for starting this discussion Yingjie, How will our tests be affected by these changes? Will Flink require more resources and, thus, will it risk destabilizing our testing infrastructure? I would propose to create a FLIP for these changes since you propose to change the default behaviour. I

Re: [DISCUSS] FLIP-197: API stability graduation process

2021-12-03 Thread Till Rohrmann
s, then a strict rule that is > broken from the start. > * If I could choose whether we focus on this FLIP or another follow up of > FLIP-196 like establishing stability guarantees for the REST API, I would > choose the latter. > > Cheers, > > Konstantin > > > > On Th

[ANNOUNCE] New Apache Flink Committer - Matthias Pohl

2021-12-02 Thread Till Rohrmann
Hi everyone, On behalf of the PMC, I'm very happy to announce Matthias Pohl as a new Flink committer. Matthias has worked on Flink since August last year. He helped review a ton of PRs. He worked on a variety of things but most notably the tracking and reporting of concurrent exceptions, fixing H

[ANNOUNCE] New Apache Flink Committer - Ingo Bürk

2021-12-02 Thread Till Rohrmann
Hi everyone, On behalf of the PMC, I'm very happy to announce Ingo Bürk as a new Flink committer. Ingo has started contributing to Flink since the beginning of this year. He worked mostly on SQL components. He has authored many PRs and helped review a lot of other PRs in this area. He actively re

Re: [DISCUSS] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-02 Thread Till Rohrmann
and effort to put together > this > >>>>>> proposal! > >>>>>> > >>>>>> Regarding: > >>>>>>> For simplification of the design, we assume that upgrades use a > >>>>>>> step size > >>>&

[DISCUSS] FLIP-197: API stability graduation process

2021-12-02 Thread Till Rohrmann
Hi everyone, As promised, here is the follow-up FLIP [1] for discussing how we can ensure that newly introduced APIs are being stabilized over time. This FLIP is related to FLIP-196 [2]. The idea of FLIP-197 is to introduce an API graduation process that forces us to increase the API stability gu

[DISCUSS] FLIP-196: Source API stability guarantees

2021-12-02 Thread Till Rohrmann
Hi everyone, I would like to start a discussion about what kind of API stability guarantees we want to provide to our users. The Flink community already agreed on some stability guarantees, but these guarantees were only communicated via the ML and not properly documented [2]. Moreover, I've seen

Re: Be able to use a custom KafkaSubscriber to have dynamic custom topic selection

2021-12-01 Thread Till Rohrmann
Hi Etienne, I think this is a reasonable request. The question we need to answer is what kind of stability guarantees we want to provide for the KafkaSubscriber interface. Once this is done, I am sure that we can also expose this part for advanced use cases. I am pulling in Arvid and Fabian who ar

Re: [VOTE] FLIP-193: Snapshots ownership

2021-12-01 Thread Till Rohrmann
Thanks for creating this FLIP Dawid. +1 Cheers, Till On Wed, Dec 1, 2021 at 2:03 PM Konstantin Knauf wrote: > Thanks, Dawid. > > +1 > > On Wed, Dec 1, 2021 at 1:23 PM Dawid Wysakowicz > wrote: > > > Dear devs, > > > > I'd like to open a vote on FLIP-193: Snapshots ownership [1] which was > >

Re: [ANNOUNCE] Open source of remote shuffle project for Flink batch data processing

2021-11-30 Thread Till Rohrmann
Great news, Yingjie. Thanks a lot for sharing this information with the community and kudos to all the contributors of the external shuffle service :-) Cheers, Till On Tue, Nov 30, 2021 at 2:32 PM Yingjie Cao wrote: > Hi dev & users, > > We are happy to announce the open source of remote shuffl

Re: [DISCUSS] FLIP-194: Introduce the JobResultStore

2021-11-30 Thread Till Rohrmann
ighAvailabilityServices interface. > > Kind regards, > Mika > > On 26.11.2021 18:28, Till Rohrmann wrote: > >Thanks for creating this FLIP Matthias, Mika and David. > > > >I think the JobResultStore is an important piece for fixing Flink's last > >high-

Re: [DISCUSS] FLIP-194: Introduce the JobResultStore

2021-11-26 Thread Till Rohrmann
Thanks for creating this FLIP Matthias, Mika and David. I think the JobResultStore is an important piece for fixing Flink's last high-availability problem (afaik). Once we have this piece in place, users no longer risk to re-execute a successfully completed job. I have one comment concerning brea

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Till Rohrmann
Thanks for writing this FLIP Dawid. Just to clarify one thing for the support of forced full snapshots. If a state backend does not support this feature, then the user either has to copy the snapshot manually or resume using --claim mode, create a savepoint in canonical format and then change the s

Re: [DISCUSS] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-11-26 Thread Till Rohrmann
Thanks for writing this FLIP Timo. I think this will be a very important improvement for Flink and our SQL user :-) Similar to Francesco I would like to understand the statement > For simplification of the design, we assume that upgrades use a step size of a single minor version. We don't guarant

Re: [DISCUSS] Creating an external connector repository

2021-11-26 Thread Till Rohrmann
> > > > Hi everyone, > > > > > > I created the flink-connectors repo [1] to advance the topic. We would > > > create a proof-of-concept in the next few weeks as a special branch > > > that I'd then use for discussions. If the community agrees with the >

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-26 Thread Till Rohrmann
x27;m not sure about ZK compatibility, but we'd also upgrade Curator to > 5.x, which doesn't support ookeeperK 3.4 anymore. > > On 25/11/2021 21:56, Till Rohrmann wrote: > > Should we ask on the user mailing list whether anybody is still using > > ZooKeeper 3.4 and

[jira] [Created] (FLINK-25069) YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP

2021-11-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25069: - Summary: YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP Key: FLINK-25069 URL: https://issues.apache.org/jira/browse/FLINK-25069

Re: [DISCUSS] Deprecate Java 8 support

2021-11-25 Thread Till Rohrmann
+1 for the deprecation and reaching out to the user ML to ask for feedback from our users. Thanks for driving this Chesnay! Cheers, Till On Thu, Nov 25, 2021 at 10:15 AM Roman Khachatryan wrote: > The situation is probably a bit different now compared to the previous > upgrade: some users might

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-25 Thread Till Rohrmann
Should we ask on the user mailing list whether anybody is still using ZooKeeper 3.4 and thus needs support for this version or can a ZooKeeper 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that not a lot of users depend on it but just to make sure that we aren't annoying a lot of o

Re: [DISCUSS] Definition of Done for Apache Flink

2021-11-25 Thread Till Rohrmann
When I started writing this response I thought that the current PR template is mostly ignored by the community. In order to verify this I went through the first two pages of open PRs and I was very positively surprised that almost all PRs filled out the current template. Hence, I had to redact my o

[jira] [Created] (FLINK-25053) Document how to use the usrlib to load code in the user code class loader

2021-11-25 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25053: - Summary: Document how to use the usrlib to load code in the user code class loader Key: FLINK-25053 URL: https://issues.apache.org/jira/browse/FLINK-25053 Project

[jira] [Created] (FLINK-25040) FlinkKafkaInternalProducerITCase.testInitTransactionId failed on AZP

2021-11-24 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25040: - Summary: FlinkKafkaInternalProducerITCase.testInitTransactionId failed on AZP Key: FLINK-25040 URL: https://issues.apache.org/jira/browse/FLINK-25040 Project

[jira] [Created] (FLINK-25039) AZP fails with license check

2021-11-24 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25039: - Summary: AZP fails with license check Key: FLINK-25039 URL: https://issues.apache.org/jira/browse/FLINK-25039 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-25037) Compilation of compile_cron_python_wheels failed on AZP

2021-11-24 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25037: - Summary: Compilation of compile_cron_python_wheels failed on AZP Key: FLINK-25037 URL: https://issues.apache.org/jira/browse/FLINK-25037 Project: Flink

[jira] [Created] (FLINK-25026) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP

2021-11-23 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25026: - Summary: UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP Key: FLINK-25026 URL: https://issues.apache.org/jira/browse/FLINK-25026

[jira] [Created] (FLINK-25025) BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount failed on AZP

2021-11-23 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25025: - Summary: BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount failed on AZP Key: FLINK-25025 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] Shall casting functions return null or throw exceptions for invalid input

2021-11-18 Thread Till Rohrmann
Hi everyone, personally I would also prefer the system telling me that something is wrong instead of silently ignoring records. If there is a TRY_CAST function that has the old behaviour, people can still get the old behaviour. For backwards compatibility reasons it is a good idea to introduce a s

Re: [ANNOUNCE] New Apache Flink Committer - Yingjie Cao

2021-11-17 Thread Till Rohrmann
Congratulations Yingjie! Cheers, Till On Wed, Nov 17, 2021 at 2:07 PM Jing Zhang wrote: > Congratulations! > > Best, > Jing Zhang > > Dian Fu 于2021年11月17日周三 下午8:07写道: > > > Congratulations! > > > > Regards, > > Dian > > > > On Wed, Nov 17, 2021 at 5:14 PM Yang Wang wrote: > > > > > Congratula

Re: [DISCUSS] Conventions on assertions to use in tests

2021-11-16 Thread Till Rohrmann
; I'm personally in favor of assertj (disclaimer I contributed to the >> project >> a bit). But I'm not sure if it's wise to change everything at once also >> from the perspective of less active contributors. We may alleviate that >> pain by providing good gui

Re: Re: [ANNOUNCE] New Apache Flink Committer - Fabian Paul

2021-11-15 Thread Till Rohrmann
Congratulations Fabian :-) Cheers, Till On Mon, Nov 15, 2021 at 3:03 PM Francesco Guardiani wrote: > Congratulations Fabian! > > On Mon, Nov 15, 2021 at 2:29 PM Yun Gao > wrote: > > > Congratulations Fabian! > > > > Best, > > Yun > > --Original Mail -- > > Send

Re: [ANNOUNCE] New Apache Flink Committer - Jing Zhang

2021-11-15 Thread Till Rohrmann
Congratulations Jing :-) Cheers, Till On Mon, Nov 15, 2021 at 2:48 PM Martijn Visser wrote: > Congratulations Jing! > > On Mon, 15 Nov 2021 at 14:39, Timo Walther wrote: > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce Jing Zhang as a new > > Flink committer. > > > >

Re: [DISCUSS] Update Policy for old releases

2021-11-15 Thread Till Rohrmann
Hi everyone, I want to second Stephan's points. I think that supporting LTS versions rather fights symptoms than addressing the underlying problem that are incompatible/behaviour changing changes between versions. I believe if we can address this problem, then our Flink users will probably upgrade

Re: [DISCUSS] Conventions on assertions to use in tests

2021-11-12 Thread Till Rohrmann
Thanks for starting this discussion Francesco. I think there is a lot of value in consistency because it makes it a lot easier to navigate and contribute to the code base. The testing tools are definitely one important aspect of consistency. It is a bit unfortunate that we have tests that follow d

Re: [ANNOUNCE] New Apache Flink Committer - Leonard Xu

2021-11-12 Thread Till Rohrmann
Congratulations! Cheers, Till On Fri, Nov 12, 2021 at 9:19 AM Martijn Visser wrote: > Well deserved, congratulations Leonard! > > Best regards, > > Martijn > > On Fri, 12 Nov 2021 at 08:44, Caizhi Weng wrote: > > > Congrats, Leonard! > > > > Shuo Cheng 于2021年11月12日周五 下午3:12写道: > > > > > Congr

Re: [ANNOUNCE] New Apache Flink Committer - Yangze Guo

2021-11-12 Thread Till Rohrmann
Congratulations! Cheers, Till On Fri, Nov 12, 2021 at 9:19 AM Martijn Visser wrote: > Congratulations! > > Best regards, > > Martijn > > On Fri, 12 Nov 2021 at 07:49, Jark Wu wrote: > > > Congratulations Yangze! > > > > Best, > > Jark > > > > On Fri, 12 Nov 2021 at 14:06, JING ZHANG wrote: >

Re: Reserve FLIP numbers

2021-11-11 Thread Till Rohrmann
Good reminder Dawid! Cheers, Till On Thu, Nov 11, 2021 at 4:02 PM Dawid Wysakowicz wrote: > Hey devs, > > Just a quick reminder to increase the counter[1] when picking up a > number for your FLIPs. > > Thanks ;) > > Best, > > Dawid > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/F

Re: [DISCUSS] Improve the name and structure of job vertex and operator name for job

2021-11-11 Thread Till Rohrmann
This improvement looks like it makes the life of our users a lot easier when it comes to understanding logs and reading the UI. Hence +1. Cheers, Till On Thu, Nov 11, 2021 at 11:59 AM JING ZHANG wrote: > Big +1. > > This is a problem frequently encountered in our production platform, look > for

[jira] [Created] (FLINK-24872) PulsarSourceITCase.testMultipleSplits failed on AZP

2021-11-11 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24872: - Summary: PulsarSourceITCase.testMultipleSplits failed on AZP Key: FLINK-24872 URL: https://issues.apache.org/jira/browse/FLINK-24872 Project: Flink Issue

[jira] [Created] (FLINK-24867) E2e tests take longer than the maximum 310 minutes on AZP

2021-11-10 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24867: - Summary: E2e tests take longer than the maximum 310 minutes on AZP Key: FLINK-24867 URL: https://issues.apache.org/jira/browse/FLINK-24867 Project: Flink

[jira] [Created] (FLINK-24866) AZP crashed in Post-job: Cache Maven local repo

2021-11-10 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24866: - Summary: AZP crashed in Post-job: Cache Maven local repo Key: FLINK-24866 URL: https://issues.apache.org/jira/browse/FLINK-24866 Project: Flink Issue Type

Re: [VOTE] FLIP-187: Adaptive Batch Job Scheduler

2021-11-10 Thread Till Rohrmann
+1 (binding) Cheers, Till On Wed, Nov 10, 2021 at 6:37 AM Zhu Zhu wrote: > +1 (binding) > > Thanks, > Zhu > > Jingsong Li 于2021年11月9日周二 下午4:01写道: > > > +1 (non-binding) > > > > This greatly enhances the ease of use of batch jobs. (Parallelism > > setting is really a challenge) > > > > (non-bin

[jira] [Created] (FLINK-24844) CassandraConnectorITCase.testCassandraBatchPojoFormat fails on AZP

2021-11-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24844: - Summary: CassandraConnectorITCase.testCassandraBatchPojoFormat fails on AZP Key: FLINK-24844 URL: https://issues.apache.org/jira/browse/FLINK-24844 Project: Flink

[jira] [Created] (FLINK-24841) ChainOrderTest.testMigrationAndRestore fails on AZP

2021-11-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24841: - Summary: ChainOrderTest.testMigrationAndRestore fails on AZP Key: FLINK-24841 URL: https://issues.apache.org/jira/browse/FLINK-24841 Project: Flink Issue

[jira] [Created] (FLINK-24840) JobManagerHAProcessFailureRecoveryITCase crashed on AZP

2021-11-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24840: - Summary: JobManagerHAProcessFailureRecoveryITCase crashed on AZP Key: FLINK-24840 URL: https://issues.apache.org/jira/browse/FLINK-24840 Project: Flink

[jira] [Created] (FLINK-24839) CsvFilesystemStreamSinkITCase.testPart times out on AZP

2021-11-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24839: - Summary: CsvFilesystemStreamSinkITCase.testPart times out on AZP Key: FLINK-24839 URL: https://issues.apache.org/jira/browse/FLINK-24839 Project: Flink

Re: [NOTICE] Please keep flink-examples up to date

2021-11-08 Thread Till Rohrmann
Thanks for raising awareness for keeping our examples up to date. I think this is a very good practice and it also helps our users better understand how to use Flink. Cheers, Till On Mon, Nov 8, 2021 at 4:57 PM Fabian Paul wrote: > Hi Seth, > > Thanks for brining up this topic. > > Huge appreci

Re: [VOTE] FLIP-189: SQL Client Usability Improvements

2021-11-08 Thread Till Rohrmann
+1 (binding) Cheers, Till On Fri, Nov 5, 2021 at 9:24 PM Ingo Bürk wrote: > +1 (non-binding) > > I think it has all been said before. :-) > > On Fri, Nov 5, 2021, 17:45 Konstantin Knauf wrote: > > > +1 (binding) > > > > On Fri, Nov 5, 2021 at 4:03 PM Martijn Visser > > wrote: > > > > > +1 (no

[jira] [Created] (FLINK-24821) FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails on AZP

2021-11-08 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24821: - Summary: FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails on AZP Key: FLINK-24821 URL: https://issues.apache.org/jira/browse/FLINK-24821 Project

[jira] [Created] (FLINK-24795) FlinkKafkaProducerITCase.testRunOutOfProducersInThePool times out on AZP

2021-11-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24795: - Summary: FlinkKafkaProducerITCase.testRunOutOfProducersInThePool times out on AZP Key: FLINK-24795 URL: https://issues.apache.org/jira/browse/FLINK-24795 Project

[jira] [Created] (FLINK-24794) pyflink/datastream/tests/test_data_stream.py fails on AZP

2021-11-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24794: - Summary: pyflink/datastream/tests/test_data_stream.py fails on AZP Key: FLINK-24794 URL: https://issues.apache.org/jira/browse/FLINK-24794 Project: Flink

[jira] [Created] (FLINK-24793) DefaultSchedulerLocalRecoveryITCase fails on AZP

2021-11-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24793: - Summary: DefaultSchedulerLocalRecoveryITCase fails on AZP Key: FLINK-24793 URL: https://issues.apache.org/jira/browse/FLINK-24793 Project: Flink Issue

[jira] [Created] (FLINK-24792) OperatorCoordinatorSchedulerTest crashed JVM on AZP

2021-11-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24792: - Summary: OperatorCoordinatorSchedulerTest crashed JVM on AZP Key: FLINK-24792 URL: https://issues.apache.org/jira/browse/FLINK-24792 Project: Flink Issue

[jira] [Created] (FLINK-24791) py37-cython failed on AZP

2021-11-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24791: - Summary: py37-cython failed on AZP Key: FLINK-24791 URL: https://issues.apache.org/jira/browse/FLINK-24791 Project: Flink Issue Type: Bug

Re: Contributor

2021-11-04 Thread Till Rohrmann
Hi Huwei, Welcome to the Flink community :-) There is no need for any JIRA permissions. If you want to get started, then I'd recommend reading the contribution guide [1] and then look for a ticket you like to work on. Once you've found one, then write on it so that a committer can assign you to th

[jira] [Created] (FLINK-24764) pyflink/table/tests/test_udf.py hang on Azure

2021-11-04 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24764: - Summary: pyflink/table/tests/test_udf.py hang on Azure Key: FLINK-24764 URL: https://issues.apache.org/jira/browse/FLINK-24764 Project: Flink Issue Type

[jira] [Created] (FLINK-24763) ParquetFileSystemITCase.testLimitableBulkFormat failed on Azure

2021-11-04 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24763: - Summary: ParquetFileSystemITCase.testLimitableBulkFormat failed on Azure Key: FLINK-24763 URL: https://issues.apache.org/jira/browse/FLINK-24763 Project: Flink

Re: [DISCUSS] FLIP-191: Extend unified Sink interface to support small file compaction

2021-11-04 Thread Till Rohrmann
Thanks for the detailed description Arvid. I might misunderstand things but one comment concerning: | We could even optimize the writer to only emit the committables after notifyCheckpointComplete as long as we retain them in the state of the writer. Emitting records for downstream operators in

Re: [DISCUSS] FLIP-191: Extend unified Sink interface to support small file compaction

2021-11-03 Thread Till Rohrmann
) to parallelism of the aggregator. > > To be really effective here, compaction would ideally be able to compact > files from multiple checkpoints. However there is a huge tradeoff between > latency a efficiency (especially with exactly once). Is this something > worth exploring? > &

Re: [DISCUSS] FLIP-191: Extend unified Sink interface to support small file compaction

2021-11-03 Thread Till Rohrmann
Thanks for creating this FLIP Fabian. >From your description I would be in favour of option 2 for the following reasons: Assuming that option 2 solves all our current problems, it seems like the least invasive change and smallest in scope. Your main concern is that it might not cover future use ca

Re: [DISCUSS] FLIP-187: Adaptive Batch Job Scheduler

2021-11-03 Thread Till Rohrmann
> > bounded and > > > > unbounded workloads can be supported. Given that there can be > multiple > > > > stages scheduled at the same time, the design of the scheduling may > not > > > be > > > > very straightforward and need

[jira] [Created] (FLINK-24744) FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on Azure because org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does

2021-11-03 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24744: - Summary: FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on Azure because org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition

[jira] [Created] (FLINK-24743) New File Sink end-to-end test fails on Azure

2021-11-03 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24743: - Summary: New File Sink end-to-end test fails on Azure Key: FLINK-24743 URL: https://issues.apache.org/jira/browse/FLINK-24743 Project: Flink Issue Type

[jira] [Created] (FLINK-24739) State requirements for Flink's application mode in the documentation

2021-11-02 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24739: - Summary: State requirements for Flink's application mode in the documentation Key: FLINK-24739 URL: https://issues.apache.org/jira/browse/FLINK-24739 Pr

[jira] [Created] (FLINK-24727) Running Kerberized YARN per-job on Docker test (no fat-jar) fails on Azure

2021-11-02 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24727: - Summary: Running Kerberized YARN per-job on Docker test (no fat-jar) fails on Azure Key: FLINK-24727 URL: https://issues.apache.org/jira/browse/FLINK-24727 Project

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Till Rohrmann
t; video! > > > > > > > > > > > > > > > Best > > > > > Ingo > > > > > > > > > > On Mon, Nov 1, 2021, 17:15 Sergey Nuyanzin > > > wrote: > > > > > > > > > > > Thanks for t

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-11-01 Thread Till Rohrmann
ign in mind. At some point we > re-evaluate the decision whether > to put it inside or outside the Flink core. What do you think? > > Best, > Kurt > > > On Fri, Oct 29, 2021 at 11:53 PM Till Rohrmann > wrote: > > > Hi Jingsong, > > > > Thanks for c

Re: [DISCUSS] Creating an external connector repository

2021-10-29 Thread Till Rohrmann
Hi everyone, >From the discussion, it seems to me that we have different opinions whether to have an ASF umbrella repository or to host them outside of the ASF. It also seems that this is not really the problem to solve. Since there are many good arguments for either approach, we could simply star

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-10-29 Thread Till Rohrmann
Hi Jingsong, Thanks for creating this FLIP. I don't have a lot to add because I am not very familiar with the SQL components. While reading the FLIP I was wondering what would we need in Flink to build something like the BDT feature outside of Flink as a kind of extension? Would something like thi

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-10-29 Thread Till Rohrmann
Thanks for creating this FLIP Sergey. I think what you propose sounds like very good improvements for the SQL client. This should make the client a lot more ergonomic :-) Cheers, Till On Fri, Oct 29, 2021 at 11:26 AM Sergey Nuyanzin wrote: > Hi Martijn, > > Thank you for your suggestion with PO

Re: [DISCUSS] FLIP-187: Adaptive Batch Job Scheduler

2021-10-29 Thread Till Rohrmann
Hi Lijie, Thanks for drafting this FLIP together with Zhu Zhu :-) I like the idea of making the parallelism of operators of a bounded job dependent on the data size. This makes the job adjust automatically when the data sources/sizes change. I can see this work well in combination with the activ

Re: [Discuss] Planning Flink 1.15

2021-10-28 Thread Till Rohrmann
I think it is important that most of the people who contribute new features are available during the testing/stabilization period because otherwise there is a risk that the release gets delayed. Hence +1 for moving the feature freeze to the 6th of February. Cheers, Till On Thu, Oct 28, 2021 at 7:

Re: [NOTICE] flink-streaming-java no longer depends on Scala and lost it's suffix

2021-10-27 Thread Till Rohrmann
Thanks a lot for your efforts Chesnay. The Scala beast will soon be tamed :-) On Tue, Oct 26, 2021 at 2:29 PM Martijn Visser wrote: > This is an awesome accomplishment, many thanks! > Ideally we also manage to complete > https://issues.apache.org/jira/browse/FLINK-24427 for 1.15 to get Flink in

Re: [VOTE] FLIP-176: Unified Iteration to Support Algorithms (Flink ML)

2021-10-22 Thread Till Rohrmann
Thanks for creating this FLIP. +1 (binding) Cheers, Till On Fri, Oct 22, 2021 at 6:29 AM Guowei Ma wrote: > +1 (binding) > > Best, > Guowei > > > On Thu, Oct 21, 2021 at 3:58 PM Yun Gao > wrote: > > > > > Hi all, > > > > We would like to start the vote for FLIP-176: Unified Iteration to > Su

Re: [ANNOUNCE] Apache Flink 1.13.3 released

2021-10-22 Thread Till Rohrmann
Thanks Chesnay and Martijn for managing this release and to everyone who contributed to it. Cheers, Till On Fri, Oct 22, 2021 at 11:04 AM Yangze Guo wrote: > Thank Chesnay, Martijn, and everyone involved! > > Best, > Yangze Guo > > On Fri, Oct 22, 2021 at 4:25 PM Yun Tang wrote: > > > > Thanks

Re: Disable usage of TumblingEventTimeWindows when EventTime is disabled?

2021-10-22 Thread Till Rohrmann
Hi Dario, I think adding a sanity check that checks whether the user uses some event time constructs and prints a warning if WatermarkStrategy.noWatermarks() is set, could be a good improvement. It might even be ok to fail to make this misconfiguration explicit. Hence, I think it is a good idea to

Re: Custom Sink Object attribute issue

2021-10-11 Thread Till Rohrmann
Hi Jigar, in order to run the Sink function on the Flink cluster, it will be serialized. Since you marked the repository as transient, it won't be shipped to the cluster. So if Repository is Serializable, you can ship it to the cluster. If not, then you need to reconstruct the Repository on the cl

Re: [NOTICE] CiBot improvements

2021-10-11 Thread Till Rohrmann
Thanks a lot for this effort Chesnay! The improvements sound really good. Cheers, Till On Mon, Oct 11, 2021 at 8:46 AM David Morávek wrote: > Nice! Thanks for the effort Chesnay, this is really a huge step forward! > > Best, > D. > > On Mon, Oct 11, 2021 at 6:02 AM Xintong Song > wrote: > > >

[jira] [Created] (FLINK-24482) KafkaShuffleTestBase.testMetrics fails on Azure

2021-10-08 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24482: - Summary: KafkaShuffleTestBase.testMetrics fails on Azure Key: FLINK-24482 URL: https://issues.apache.org/jira/browse/FLINK-24482 Project: Flink Issue Type

[jira] [Created] (FLINK-24437) Remove unhandled exception handler from CuratorFramework before closing it

2021-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24437: - Summary: Remove unhandled exception handler from CuratorFramework before closing it Key: FLINK-24437 URL: https://issues.apache.org/jira/browse/FLINK-24437 Project

[jira] [Created] (FLINK-24430) Clean up log pollution

2021-09-30 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24430: - Summary: Clean up log pollution Key: FLINK-24430 URL: https://issues.apache.org/jira/browse/FLINK-24430 Project: Flink Issue Type: Technical Debt

Re: [Flink blogs]

2021-09-30 Thread Till Rohrmann
Hi Etienne, Great to see that you want to write about one of the topics you have worked on! Spreading the word about changes/improvements/new features is always super important. As a general recommendation I think you should write about the topic you are most excited about. This usually results i

Re: [ANNOUNCE] Apache Flink 1.14.0 released

2021-09-30 Thread Till Rohrmann
Thanks a lot for being our release managers Dawid, Xintong and Joe. Also a big thanks to everyone who contributed to this release and helped make it happen! Cheers, Till On Thu, Sep 30, 2021 at 10:12 AM Caizhi Weng wrote: > Congratulations! > > As a side note, I notice that the MigrationVersion

Re: Beam with Flink runner - Issues when writing to S3 in Parquet Format

2021-09-30 Thread Till Rohrmann
Hi Sandeep, I am not a Beam expert. The problem might be caused by the used S3 filesystem implementation. Have you tried whether the same problem occurs when using vanilla Flink's latest version? Alternatively, you could also reach out to the Beam community or ask on Flink's user ML whether people

Re: Is anyone still using the 1.11-1.13 Burndown/Bugs/Sprint Kanban boards?

2021-09-30 Thread Till Rohrmann
I think it should be fine to remove these boards. So +1 for removing them from my side. Cheers, Till On Mon, Sep 6, 2021 at 1:13 PM Chesnay Schepler wrote: > Is anyone still using one of these boards? I would like to remove them > unless they are still of value to someone. > > > https://issues.

Re: [ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

2021-09-30 Thread Till Rohrmann
Thanks for the hint with the managed search engines Matthias. I think this is quite helpful. Cheers, Till On Wed, Sep 15, 2021 at 4:27 PM Matthias Pohl wrote: > Thanks Leonard for the announcement. I guess that is helpful. > > @Robert is there any way we can change the default setting to someth

Re: [DISCUSS] About migrating to new mail list archive service

2021-09-03 Thread Till Rohrmann
+1 for announcing it. Making our users aware of this change is definitely helpful. Do you wanna write the announcement Leonard? Cheers, Till On Fri, Sep 3, 2021 at 10:29 AM Chesnay Schepler wrote: > Announcing it cannot hurt I suppose. > > On 03/09/2021 10:21, Leonard Xu wrote: > >> When search

<    1   2   3   4   5   6   7   8   9   10   >