Re: [ DISCUSS ] FLIP-XXX : [Plugin] Enhancing Flink Failure Management in Kubernetes with Dynamic Termination Log Integration

Hi Gyula and Ahmed, I totally agree that there is an interlap in the final goal that both the FLIPs are achieving here and infact FLIP-304 is more comprehensive for job failures. But as a proposal to move forward can we make Swathi's FLIP/JIRA as a sub task for FLIP-304 and continue with the PR

Re: Issue with flink 1.16 and hive dialect

cy.kind'='metastore,success-file'); > > > [1] https://issues.apache.org/jira/browse/FLINK-27450 > [2] https://issues.apache.org/jira/browse/HIVE-21584 > [3] https://issues.apache.org/jira/browse/FLINK-32596 > > > Best regards, > Yuxia > > --

Issue with flink 1.16 and hive dialect

Hi All, I am not sure if this was already discussed in this forum. In our set up with 1.16.0 flink we have ensured that the setup has all the necessary things for Hive catalog to work. The flink dialect works fine functionally (with some issues will come to that later). But when i follow the

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

Congratulations !!! On Mon, Mar 27, 2023 at 2:54 PM Yu Li wrote: > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new project still

Re: StreamingFileSink's DefaultRollingPolicy

Hi Keith, With default rolling policy, when you try to restore from the older save points, there is a high chance that the inProgressPart file would have been renamed to a newer file. So when we try to restore from older save points this might not work. The onCheckpointRolling policy might be

Re: Flink k8s native support - pod deployments and upgrades

t; On Sat, 4 Mar 2023 at 06:51, ramkrishna vasudevan > > wrote: > > > Thanks for the reply. Yes I get that the operator helps out in upgrades > > also . But when the operator works with flink k8s native clusters, it can > > help with savepoints and helps in creatin

Re: Flink k8s native support - pod deployments and upgrades

he.org/flink/flink-kubernetes-operator-docs-main/ > > Thanks, > Thomas > > > On Fri, Mar 3, 2023 at 2:02 AM ramkrishna vasudevan < > ramvasu.fl...@gmail.com> > wrote: > > > Hi All, > > > > The native implementation of the App mode and session mode - do

Flink k8s native support - pod deployments and upgrades

Hi All, The native implementation of the App mode and session mode - does not have any replica set . Instead it just allows the JM to create TM pods on demand. This is simple and easy in terms of creation of resources, but for an upgrade story, how is this managed? Leaving K8s to manage a

Re: Re: [DISCUSS] Extract core autoscaling algorithm as new SubModule in flink-kubernetes-operator

Hi Samrat/Prabhu My preliminary question here would be that though YARN is the platform for Flink, but the yarn also runs over K8s ? Is that the reason why you wanted the autoscale logic to be generic but inside the operator itself? So if the case is that Yarn is the resource management, then

Re: Re: [ANNOUNCE] New Apache Flink Committer - Weijie Guo

Congratulations!!! Weijie Regards Ram On Mon, Feb 13, 2023 at 2:17 PM Yuepeng Pan wrote: > Congratulations, Weijie! > > Best, > > Best, Yuepeng Pan. > > 在 2023-02-13 16:26:32，"Lincoln Lee" 写道： > >Congratulations, Weijie! > > > >Best, > >Lincoln Lee > > > > > >Martijn Visser 于2023年2月13日周一

Re: ABFS/WASB IT test cases not running

> On Wed, Dec 7, 2022 at 5:54 AM ramkrishna vasudevan < > ramvasu.fl...@gmail.com> > wrote: > > > Thank you for your email. So with this PR I should be adding ABFS related > > config and secrets and also ensure that the existing WASB secrets are > also > > up

Re: ABFS/WASB IT test cases not running

ory section of the CI docs [3]. > > I hope that helps, > Matthias > > [1] https://issues.apache.org/jira/browse/FLINK-12654 > [2] https://issues.apache.org/jira/browse/FLINK-29810 > [3] > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > >

ABFS/WASB IT test cases not running

Hi devs, I was trying to write test cases for the Streaming sink support for ABFS as part of https://issues.apache.org/jira/browse/FLINK-30128. I found that the AzureFileSystemBehaviorITCase was not running in the CI build too. So if I want to add the Recoverable support for ABFS , is there a

Fwd: Physical memory and Auto Memory settings in TM

Hi All, (Sending to dev@ ) Installation of flink clusters where we govern the memory configs by specifying 'taskmanager.memory.process.size' TM memory is auto calculated. But in this calculation we generally don't check the actual physical memory available. So in case of a wrong configuration

Re: [ANNOUNCE] New Apache Flink Committer - Matyas Orhidi

Congrats Matayas On Mon, Nov 21, 2022 at 10:06 PM Jim Busche wrote: > Congratulations Matyas! > > Jim > -- > James Busche | Sr. Software Engineer, Watson AI and Data Open Technology | > 408-460-0737 | jbus...@us.ibm.com > > > > > From: Márton Balassi > Date: Monday,