Re: Discussion: Condition field in the CR status

Hi Lajith! Can you please include some examples in the document to help reviewers? Just some examples with the status and the proposed conditions. Cheers, Gyula On Wed, May 1, 2024 at 9:06 AM Lajith Koova wrote: > Hello, > > > Starting discussion thread here to discuss a proposal to add

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

That's my fault @Robert Metzger , since the new FLIP process and a lack of confluent access for non-committers this is a bit tricky to get it sync :) Gyula On Thu, Apr 25, 2024 at 4:17 PM Robert Metzger wrote: > In principle I'm +1 on the proposal, but I think the FLIP in the wiki is > not in

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

+1 (binding) Gyula On Wed, Apr 24, 2024 at 10:07 AM Ferenc Csaky wrote: > +1 (non-binding), looking forward to this! > > Best, > Ferenc > > > > > On Wednesday, April 24th, 2024 at 10:03, Mate Czagany > wrote: > > > > > > > Hi everyone, > > > > I'd like to start a vote on the FLIP-446:

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

Thank you for driving this effort +1 Cheers Gyula On Wed, 24 Apr 2024 at 06:25, Yuan Mei wrote: > Hey Yue, > > Thanks for all the great efforts significantly improving rescaling and > upgrading rocksdb. > > +1 for this. > > Best > Yuan > > On Wed, Apr 24, 2024 at 10:46 AM Zakelly Lan >

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

he following conditions: >> > - Triggered >> > - Completed >> > - Failed >> > The only benefit of this proposal that I see is that it would tell the >> > user how long it took to create the savepoint. >> > >> > 5. You mention that "JobSpec.i

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

description, the Operator will > fallback > > to the old mode if the FlinkStateSnapshot CRD cannot be found on the > > Kubernetes cluster. > > > > Regards, > > Mate > > > > Gyula Fóra ezt írta (időpont: 2024. ápr. 16., K, > > 17:01): > > &g

Re: [ DISCUSS ] FLIP-XXX : [Plugin] Enhancing Flink Failure Management in Kubernetes with Dynamic Termination Log Integration

Hi Swathi! Thank you for creating this proposal. I really like the general idea of increasing the K8s native observability of Flink job errors. I took a quick look at your reference PR, the termination log related logic is contained completely in the ClusterEntrypoint. What type of errors will

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

Thanks Mate, this is great stuff. Mate, I think the new configs should probably default to the new mode and they should only be useful for users to fall back to the old behaviour. We could by default use the new Snapshot CRD if the CRD is installed, otherwise use the old mode by default and log a

Re: [DISCUSS] Flink Website Menu Adjustment

+1 for the proposal Gyula On Mon, Mar 25, 2024 at 12:49 PM Leonard Xu wrote: > Thanks Zhongqiang for starting this discussion, updating documentation > menus according to sub-projects' activities makes sense to me. > > +1 for the proposed menus: > > > After: > > > > With Flink > > With Flink

Re: Flink Kubernetes Operator Failing Over FlinkDeployments to a New Cluster

I agree, we would need some FLIPs to cover this. Actually there is already some work on this topic initiated by Matthias Pohl (ccd). Please see this:

Re: [VOTE] FLIP-433: State Access on DataStream API V2

+1 (binding) Gyula On Thu, Mar 21, 2024 at 3:33 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Thanks to Weijie for driving this proposal, which solves the problem that I > raised in FLIP-359. > > Best, > Rui > > On Thu, Mar 21, 2024 at 10:10 AM Hangxiang Yu wrote: > > > +1

Re: Flink Kubernetes Operator Failing Over FlinkDeployments to a New Cluster

Sorry for the late reply Kevin. I think what you are suggesting makes sense, it would be basically a `last-state` startup mode. This would also help in cases where the current last-state mechanism fails to locate HA metadata (and the state). This is somewhat of a tricky feature to implement: 1.

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

+1 (binding) Thanks! Gyula On Wed, Mar 20, 2024 at 3:36 PM Mate Czagany wrote: > +1 (non-binding) > > Thank you, > Mate > > Ferenc Csaky ezt írta (időpont: 2024. márc. > 20., Sze, 15:11): > > > Hello devs, > > > > I would like to start a vote about FLIP-439 [1]. The FLIP is about to > >

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.8.0, release candidate #1

Hi Max! +1 (binding) - Verified source release, helm chart + checkpoints / signatures - Helm points to correct image - Deployed operator, stateful example and executed upgrade + savepoint redeploy - Verified logs - Flink web PR looks good +1 A minor correction is that [3] in the email

Re: Unaligned checkpoint blocked by long Async operation

e code modifications on > the mailbox executor. > > > Best, > Zakelly > > On Thu, Mar 14, 2024 at 9:15 PM Gyula Fóra wrote: > >> Thank you for the detailed analysis Zakelly. >> >> I think we should consider whether yield should process checkpoint

Re: Flink Kubernetes Operator Failing Over FlinkDeployments to a New Cluster

Hey Kevin! The general mismatch I see here is that operators and resources are pretty cluster dependent. The operator itself is running in the same cluster so it feels out of scope to submit resources to different clusters, this doesn't really sound like what any Kubernetes Operator should do in

Re: [DISCUSS] Manual savepoint triggering in flink-kubernetes-operator

s Kubernetes > workflows a lot better. > > If there are no objections, I can investigate it during the next few weeks > and see how this could be implemented in the current code. > > Cheers, > Mate > > Gyula Fóra ezt írta (időpont: 2024. márc. 12., K, > 16:01): > > >

Re: [DISCUSS] Manual savepoint triggering in flink-kubernetes-operator

gt; > On Wed, Mar 2, 2022 at 12:47 PM Yang Wang > wrote: > > > > > > > I also like the annotation approach since it is more natural. > > > > But I am not sure about whether the meta data change will trigger an > > > event > > > > in ja

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

Hey! I may have missed this in the FLIP but did not find it spelled out explicitly. Can the user use the entire record processing context in their async execution? In other words can they access all functionality from the callback? For example can they simply reference the keyed context and

Re: [DISCUSS] FLIP-423 ～FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0

Hey all! This is a massive improvement / work. I just started going through the Flips and have a more or less meta comment. While it's good to keep the overall architecture discussion here, I think we should still have separate discussions for each FLIP where we can discuss interface details

Re: [DISCUSS] FLIP-433: State Access on DataStream API V2

Hi Weijie! Thank you for the proposal. I have some initial questions to start the discussion: 1. What is the semantics of the usesStates method? When is it called? Can the used state change dynamically at runtime? Can the logic depend on something computed in open(..) for example? Currently

Re: Temporal join on rolling aggregate

Hi Everyone! I have discussed this with Sébastien Chevalley, he is going to prepare and drive the FLIP while I will assist him along the way. Thanks Gyula On Tue, Mar 5, 2024 at 9:57 AM wrote: > I do agree with Ron Liu. > This would definitely need a FLIP as it would impact SQL and extend it

Re: [VOTE] FLIP-314: Support Customized Job Lineage Listener

+1 (binding) Gyula On Wed, Feb 28, 2024 at 11:10 AM Maciej Obuchowski wrote: > +1 (non-binding) > > Best, > Maciej Obuchowski > > śr., 28 lut 2024 o 10:29 Zhanghao Chen > napisał(a): > > > +1 (non-binding) > > > > Best, > > Zhanghao Chen > > > > From: Yong

Re: Temporal join on rolling aggregate

Posting this to dev as well as it potentially has some implications on development effort. What seems to be the problem here is that we cannot control/override Timestamps/Watermarks/Primary key on VIEWs. It's understandable that you cannot create a PRIMARY KEY on the view but I think the temporal

Re: FLINK-21672

his point. > > On Fri, Feb 16, 2024 at 5:00 PM Gyula Fóra wrote: > > > > Depending on the scope of this change, it may not be considered a feature > > right Martijn? > > If it's a test improvement, can it still be part of the release? > > > > Gyula >

Re: FLINK-21672

Depending on the scope of this change, it may not be considered a feature right Martijn? If it's a test improvement, can it still be part of the release? Gyula On Fri, Feb 16, 2024 at 4:45 PM Martijn Visser wrote: > Hi David, > > Happy to assign it to you. It can't be merged for Flink 1.19

Re: [DISCUSS] Kubernetes Operator 1.8.0 release planning

on vacation starting this Friday, and I have some > > other work before I go on vacation. After evaluating my time today, > > I found that I cannot complete the development, testing, and merging > > of the JDBC event handler this week. So I tend to put the JDBC > > event handle

[DISCUSS] Kubernetes Operator 1.8.0 release planning

Hi all! I would like to kick off the release planning for the operator 1.8.0 release. The last operator release was November 22 last year. Since then we have added a number of fixes and improvements to both the operator and the autoscaler logic. There are a few outstanding PRs currently,

Re: DataOutputSerializer serializing long UTF Strings

Hi Peter! I think this is a good additional serialization utility to Flink that may benefit different data formats / connectors in the future. +1 Cheers, Gyula On Mon, Jan 22, 2024 at 8:04 PM Steven Wu wrote: > I think this is a reasonable extension to `DataOutputSerializer`. Although > 64

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

+1 (binding) On Sun, Jan 14, 2024 at 9:01 AM Yun Tang wrote: > +1 (non-binding) > > On 2024/01/14 02:26:13 Yun Gao wrote: > > +1 (binding) > > > > On Sat, Jan 13, 2024 at 10:08 AM Rodrigo Meneses > wrote: > > > > > > +1 (non binding) > > > > > > On Fri, Jan 12, 2024 at 5:45 PM Dong Lin wrote:

Re: Flink pending record metric weired after autoscaler rescaling

Could this be related to the issue reported here? https://issues.apache.org/jira/browse/FLINK-34063 Gyula On Wed, Jan 10, 2024 at 4:04 PM Yang LI wrote: > Just to give more context, my setup uses Apache Flink 1.18 with the > adaptive scheduler enabled, issues happen randomly particularly >

Re: [VOTE] FLIP-372: Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

+1 (binding) Gyula On Mon, 18 Dec 2023 at 13:04, Márton Balassi wrote: > +1 (binding) > > On Mon 18. Dec 2023 at 09:34, Péter Váry > wrote: > > > Hi everyone, > > > > Since there were no further comments on the discussion thread [1], I > would > > like to start the vote for FLIP-372 [2]. > >

Re: [DISCUSS] Should Configuration support getting value based on String key?

encouraging developers to use ConfigOption as much as > possible" vs. "a bit less convenient in 5% of the cases". I guess there's > no right and wrong, just different engineer opinions. While I'm personally > stand with removing the string-key access methods, I'd also be f

Re: [DISCUSS] Should Configuration support getting value based on String key?

Hi Xintong, I don’t really see the actual practical benefit from removing the getstring and setstring low level methods. I understand that ConfigOptions are nicer for 95% of the cases but from a technical point of view there will always be cases where a more flexible dynamic handling is

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

in approach for easier evolvability and better > > consistency, but I would like to hear your thoughts, so I can flash out > the > > chosen approach in FLIP-372 > > > > Thanks, > > Peter > > > > [1] - https://lists.apache.org/thread/h6nkgth838dlh5s90sd95zd6h

Re: Flink Kubernetes Operator Observed Generation

Hi Ryan! If you think this is useful/required to integrate with some tooling, I don't really see a problem. We already have a concept of reconciled generation as part of the lastReconciledSpec status field but that is semantically a bit different. Cheers, Gyula

Re: [VOTE] FLIP-401: REST API JSON response deserialization unknown field tolerance

+1 Gyula On Mon, Dec 11, 2023 at 1:26 PM Gabor Somogyi wrote: > Hi All, > > I'd like to start a vote on FLIP-401: REST API JSON response > deserialization unknown field tolerance [1] which has been discussed in > this thread [2]. > > The vote will be open for at least 72 hours unless there is

Re: [DISCUSS] FLIP-401: REST API JSON response deserialization unknown field tolerance

Thanks Gabor! +1 from my side, this sounds like a reasonable change that will improve integration and backward compatibility. Let's keep this open for at least another day before starting the vote. Cheers, Gyula On Thu, Dec 7, 2023 at 10:48 AM Gabor Somogyi wrote: > Hi All, > > I'd like to

Re: Kubernetes Operator Managed Ingress with TLS

Hi Ryan! There is not really a Flink Kubernetes Operator team :) Everyone is welcome to contribute and bring new improvement and feature ideas! Sounds like a valuable addition, I will assign the ticket to you (please comment on it so I can see your jira handle) so you can follow up with your PR.

Re: [DISCUSS] REST API response parsing throws exception on new fields

; > https://cwiki.apache.org/confluence/display/FLINK/FLIP-401%3A+REST+API+JSON+response+deserialization+unknown+field+tolerance > > Let's wait on voices until beginning of next week. > > G > > > On Wed, Dec 6, 2023 at 11:05 AM Gyula Fóra wrote: > > > Thanks G, I think this is

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

ecause flink > operator announced support from 1.13 to 1.17/1.18 > > Has other solution will work in our situation? > > Thanks > Richard Su > > > 2023年12月7日 16:22，Gyula Fóra 写道： > > > > Hi! > > > > What Flink version are you using? > &g

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

Hi! What Flink version are you using? The operator always sets: execution.shutdown-on-application-finish to false so that finished / failed application clusters should not exit immediately and we can observe them. This is however only available in Flink 1.15 and above. Cheers, Gyula On Thu,

Re: Flink Operator - Supporting Recovery from Snapshot

> wrote: > > > > > Thanks for the response Gyula! Those caveats make sense, and I see, > > there's > > > a bit of a complexity to consider if the feature is implemented. I do > > think > > > it would be useful, so would also love to hear what others

Re: [DISCUSS] REST API response parsing throws exception on new fields

Thanks G, I think this is a reasonable proposal which will increase compatibility between different Flink clients and versions such as SQL Gateway, CLI, Flink operator etc. I don't really see any harm in ignoring unknown json fields globally, but this probably warrants a FLIP and a proper vote.

Re: Discussion: [FLINK-24150] Support to configure cpu resource request and limit in pod template

I hope the container resources's request can be decoupling from flink > configuration. > > > > From flink's current implementation, this could not be done. > > > > Thanks. > > > > Richard Su > > > >> 2023年12月5日 20:28，Gyula Fóra 写道： > >&

Re: Discussion: [FLINK-24150] Support to configure cpu resource request and limit in pod template

es > user config and actually make sconfig more intuitive and easier but not > make the container resource decoupling flink configuration. > > > > So from my side, I think it need to add new configuration to support > this proposal, and it need more discussion. > >

Re: Discussion: [FLINK-24150] Support to configure cpu resource request and limit in pod template

code of flink operator, > > which should import something new configuration to support the native > mode. > > > > > > I think it will be better to import some configuration, which > > FlinkConfigManager.class can override it using the resource of > > JobmanagerSp

Re: Discussion: [FLINK-24150] Support to configure cpu resource request and limit in pod template

which indeed will override > the process size by TaskmanagerSpec.resources or JobmanagerSpec.resources, > which really confused, I had modified the code of flink operator to avoid > the override. > > Looking for your response. > > Thank you. > Richard Su > > > > 2023年12月5日

Re: Discussion: [FLINK-24150] Support to configure cpu resource request and limit in pod template

Hi! Please see the discussion in https://lists.apache.org/thread/6p5tk6obmk1qxf169so498z4vk8cg969 and the ticket: https://issues.apache.org/jira/browse/FLINK-33548 We should follow the approach outlined there. If you are interested you are welcome to pick up the operator ticket. Unfortunately

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

Hi All! Based on the discussion above, I feel that the most reasonable approach from both developers and users perspective at this point is what Becket lists as Option 1: Revert the naming change to the backward compatible version and accept that the names are not perfect (treat it as legacy).

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

> > Thanks > Surendra > > > On Sat, Nov 25, 2023, 12:28 AM Gyula Fóra wrote: > > > I think a better and more Kubernetes canonical approach would be what is > > outlined in this ticket: > > > > https://issues.apache.org/jira/browse/FLINK-33548 > > >

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

LINK-24150 > <https://issues.apache.org/jira/browse/FLINK-24150> > > Thanks > Surendra > > On Fri, Nov 24, 2023 at 4:16 PM Gyula Fóra wrote: > > > Hi Surendra! > > > > The resource configuration in Flink is pretty well established and > > covers sett

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

Hi Surendra! The resource configuration in Flink is pretty well established and covers setting both memory requests and limits (through the limit factor.) Could you please elaborate why you think this change is a good addition? I see a few downsides: - It complicates memory configuration by

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

Hi Peter! Thank you for the analysis of the options. I don't really have a strong opinion, but in general I am in favor of the mix in style interfaces. We follow the same approach for table sources / sinks as well. Some other benefits I see in addition to what you mentioned: - Easier to

[ANNOUNCE] Apache Flink Kubernetes Operator 1.7.0 released

The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.7.0. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Release highlights: - Standalone

[RESULT] [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Marton Balassi (binding) * Gyula Fora (binding) * Maximilian Mixhels (binding) * Rui Fan (non-binding) * Mate Czagany (non-binding) There are no disapproving votes.

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

s > > - Ran built-in tests, built jars + docker image from source successfully > > - Upgraded the operator and the CRD from 1.6.1 to 1.7.0 > > > > Best, > > Marton > > > > On Mon, Nov 20, 2023 at 2:03 PM Gyula Fóra wrote: > > > > > +1

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

+1 (binding) Gyula On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: > +1 (non-binding) > > Thanks for driving this. > > Best, > Xiangyu Feng > > > Ferenc Csaky 于2023年11月21日周二 20:07写道： > > > +1 (non-binding) > > > > Lookgin forward to this! > > > > Best, > > Ferenc > > > > > > > > > > On

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

l source files contain the license header > > - All pom files point to the correct version > > - Verified Helm chart version and appVersion > > - Verified Docker image tag > > - Ran flink-autoscaler-standalone JAR downloaded from the maven > repository > > - Tested autoscale

Re: [VOTE] FLIP-378: Support Avro timestamp with local timezone

+1 Also, while technically enough , i prefer to exclude weekend from the 72 hours. So this vote should have run for a little longer ideally But no harm done :) Gyula On Mon, 20 Nov 2023 at 03:46, Peter Huang wrote: > Thanks for pointing this out. I will follow the process in the future. > >

SQL / Table input format for Kafka that can access full record

Hey All! I have been browsing through the codebase but don't really have a good answer. Is there any way to access the entire Kafka consumer record (key, value, headers) when deserializing into a dynamic table? I see that we can specify, key format , value format and metadata columns

[VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

Hi Everyone, Please review and vote on the release candidate #1 for the version 1.7.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of

Re: [DISCUSS] FLIP-378: Support Avro timestamp with local timezone

+1 Gyula On Thu, Nov 2, 2023 at 6:18 AM Martijn Visser wrote: > +1 > > On Thu, Nov 2, 2023 at 12:44 PM Leonard Xu wrote: > > > > > > > Thanks @Leonard Xu . Two minor versions are > definitely needed for flip the configs. > > > > Sorry, Peter. I thought the next minor

[DISCUSS] Kubernetes Operator 1.7.0 release planning

Hi all! I would like to kick off the release planning for the operator 1.7.0 release. We have added quite a lot of new functionality over the last few weeks and I think the operator is in a good state to kick this off. Based on the original release schedule we had Nov 1 as the proposed feature

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 1.6.1 released

Thank you Rui for taking care of this! On Mon, Oct 30, 2023 at 11:55 AM Rui Fan <1996fan...@gmail.com> wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink Kubernetes Operator 1.6.1. > > Please check out the release blog post for an overview of the

Re: Operator 1.6 to Olm

versions of the Flink operator. When 1.6.1 is out > I will do the same again, > Kind regards, David. > > > > From: Gyula Fóra > Date: Tuesday, 10 October 2023 at 13:27 > To: dev@flink.apache.org > Subject: [EXTERNAL] Re: Operator 1.6 to Olm > That wou

Re: [DISCUSS] FLIP-360: Merging ExecutionGraphInfoStore and JobResultStore into a single component

y used for information display, and the other is > for > > failover. So after unifying storage, can we use different interfaces to > > meet the different requirements for jobs? Adding all these methods for > > different components into one interface such as `CompletedJobStore` ma

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

+1 (binding) - Verified checksums, signatures, source release content - Helm repo works correctly and points to the correct image / version - Installed operator, ran stateful example Gyula On Sat, Oct 21, 2023 at 1:43 PM Rui Fan <1996fan...@gmail.com> wrote: > +1(non-binding) > > - Downloaded

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

the > 1.6.1-rc1 based on the latest release-1.6 branch. > > Please let me know if any critical commit is missed at release-1.6 branch, > thanks~ > > And thanks Gyula's help! > > Best, > Rui > > On Thu, Oct 19, 2023 at 2:25 PM Gyula Fóra wrote: > >> Thank

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

inor version. For those who have never released before, > a minor version may be a good entry point. Would you mind > if I volunteer as the release manager for 1.6.1? > > [1] https://lists.apache.org/thread/5ynjv18nfoj6rvyhlz1g5y8qtxx6v1gd > > Best, > Rui > > On Thu, Oct 19,

[DISCUSS] Creating Kubernetes Operator release 1.6.1

Hi All! I would like to propose to release the 1.6.1 patch version for the Kubernetes operator. The release branch currently contains 2-3 critical fixes for issues that many users have hit over time. Making this release now would allow us more time to wrap up and finalize the 1.7.0 release

Re: [VOTE] FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink

Thanks , Peter. +1 Gyula On Wed, 11 Oct 2023 at 14:47, Péter Váry wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-371[1]. > Based on the discussion thread [2], I think we are ready to take a vote to > contribute this to Flink. > I'd like to start a vote for it. > The vote

Re: Operator 1.6 to Olm

That would be great David, thank you! Gyula On Tue, 10 Oct 2023 at 14:13, David Radley wrote: > Hi, > I notice that the latest version in olm of the operator is 1.5. I plan to > run the scripts to publish the 1.6 Flink operator to olm, > Kind regards, David. > > Unless otherwise stated

Re: [DISCUSS] Java Record support

; > > > On Wed, Oct 4, 2023 at 5:04 PM Gyula Fóra wrote: > > > > > I will share my initial implementation soon, it seems to be pretty > > > straightforward. > > > > > > Biggest challenge so far is setting tests so we can still compile > agai

Re: [DISCUSS] Java Record support

uctor in the right order. > > From what I remember from my own experiments this shouldn't exactly > /difficult/, but just a bit tedious to integrate into the Type > extraction stack. > > On 04/10/2023 16:14, Őrhidi Mátyás wrote: > > +1 This would be great > > >

[DISCUSS] Java Record support

Hi All! Flink 1.18 contains experimental Java 17 support but it misses out on Java records which can be one of the nice benefits of actually using newer java versions. There is already a Jira to track this feature [1] but I am not aware of any previous efforts so far. Since records have pretty

[RESULT][VOTE] FLIP-361: Improve GC Metrics

Hi Everyone, The proposal, FLIP-361: Improve GC Metrics, has been unanimously accepted with 15 votes (8 binding) . +1 votes: - Gyula Fora (binding) - Rui Fan (binding) - Ahmed Hamdy - Chen Zhanghao - Conrad Jam - Samrat Deb - Xintong Song (binding) - Matt Wang - Dong Lin (binding) -

Re: [VOTE] FLIP-361: Improve GC Metrics

FLIP-361: Improve GC Metrics | > > >> > > > +1 (binding) > > >> > > > > > >> > > > Best, > > >> > > > > > >> > > > Xintong > > >> > > > > > >> > > > > > >>

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

anagement > requirements, so having an operator manage them would be a bit wasted. > Therefore, atm, we can allow users to directly submit jobs using JDBC > or REST API. WDYT? > > Best, > Yangze Guo > > On Mon, Sep 18, 2023 at 4:08 PM Gyula Fóra wrote: > > > > As I wr

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

ghtlies.apache.org/flink/flink-docs-master/docs/dev/table/olap_quickstart/ > > Best, > Shammon FY > > > > On Sun, Sep 17, 2023 at 2:02 PM Gyula Fóra wrote: > > > Hi! > > It sounds pretty easy to deploy the gateway automatically with session > > cluster deployments from

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

n operator can provide additional benefits and be > more user-friendly, especially for users who are less familiar with > Kubernetes. By using an operator, users can benefit from consistent > version management with the session cluster and upgrade capabilities. > > > Best, > Y

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

gt; > > > > valuable if we can do more work to start it in k8s. > > > > > > > > > > > > For xiaolong: > > > > > > Do you want to start a sql-gateway instance in the jobmanager > pod? I > > > > > think > > > >

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

Regards, > Jing > > On Thu, Sep 14, 2023 at 3:13 PM Ahmed Hamdy wrote: > > > Makes sense, > > Thanks for the clarification. > > Best Regards > > Ahmed Hamdy > > > > > > On Thu, 14 Sept 2023 at 14:07, Gyula Fóra wrote: > > > > > Hi Ahme

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

Hi! I don't completely understand what would be a content of such CRD, could you give a minimal example how the Flink SQL Gateway CR yaml would look like? Adding a CRD would mean you need to add some operator/controller logic as well. Why not simply use a Deployment / StatefulSet in Kubernetes?

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

ang > > > > > > Biao Geng 于2023年9月6日周三 18:15写道： > > > > > > > +1 for the proposal. > > > > > > > > Best, > > > > Biao Geng > > > > > > > > Gyula Fóra 于2023年9月6日周三 16:10写道： > > > > > > > > >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

n > > > On Wed, Sep 13, 2023 at 6:13 AM Gyula Fóra wrote: > > > Thanks for all the feedback, I will start the vote on this. > > > > Gyula > > > > On Wed, Sep 6, 2023 at 11:11 AM Xintong Song > > wrote: > > > > > > > > > >

[VOTE] FLIP-361: Improve GC Metrics

Hi All! Thanks for all the feedback on FLIP-361: Improve GC Metrics [1][2] I'd like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. Cheers, Gyula [1]

Re: [DISCUSS] FLIP-361: Improve GC Metrics

gt; make > > too much sense as collection times usually differ greatly depending on > the > > collector. > > > > LGTM > > > Best, > > Xintong > > > > On Wed, Sep 6, 2023 at 4:31 PM Gyula Fóra wrote: > > > I added the average time metr

Re: [VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

+1 (binding) Gyula On Wed, 13 Sep 2023 at 09:33, Matt Wang wrote: > Thank you for driving this FLIP, > > +1 (non-binding) > > > -- > > Best, > Matt Wang > > > Replied Message > | From | ConradJam | > | Date | 09/13/2023 15:28 | > | To | | > | Subject | Re: [VOTE] FLIP-334:

Re: [DISCUSS] FLIP-360: Merging ExecutionGraphInfoStore and JobResultStore into a single component

Hi Matthias! Thank you for the detailed proposal, overall I am in favor of making this unification to simplify the logic and make the integration for external components more straightforward. I will try to read through the proposal more carefully next week and provide some detailed feedback. +1

Re: [DISCUSS] FLIP-361: Improve GC Metrics

: > Thank you :) > > Best, > > Xintong > > > > On Wed, Sep 6, 2023 at 4:17 PM Gyula Fóra wrote: > > > Makes sense Xintong, I am happy to extend the proposal with the average > gc > > time metric +1 > > > > Gyula > > > > On Wed, Sep

Re: [DISCUSS] FLIP-361: Improve GC Metrics

happen at a > similar time usually take a similar length of time. Admittedly, this is not > a hard guarantee. > > > Best, > > Xintong > > > > On Wed, Sep 6, 2023 at 3:59 PM Gyula Fóra wrote: > > > Matt Wang, > > > > I think the currently expose

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

ote: > > > > +1 > > > > On Tue, Sep 5, 2023 at 8:03 AM Thomas Weise wrote: > > > > > +1, thanks for the proposal > > > > > > On Tue, Sep 5, 2023 at 8:13 AM Gyula Fóra > wrote: > > > > > > > Hi All! > > > > > > &

Re: [DISCUSS] FLIP-361: Improve GC Metrics

count. For example, > for G1, there is no way to distinguish whether it is the > young generation or the old generation. > > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Gyula Fóra | > | Date | 09/6/2023 15:03 | > | To | | >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

ood alternative. > WDYT? > > Best, > > Xintong > > > > On Wed, Sep 6, 2023 at 2:16 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Thanks for the clarification! > > > > By default the meterview measures for 1 minute sounds good to me! > > >

Re: [DISCUSS] Drop python 3.7 support in 1.19

split them is to separate the concerns and reduce > the > > > amount of code in a PR to help reviewers. > > > > > > BR, > > > G > > > > > > > > > On Mon, Sep 4, 2023 at 12:57 PM Sergey Nuyanzin > > > wrote: > > > > &g

Re: [DISCUSS] FLIP-361: Improve GC Metrics

: > GC Total.Time of current time - GC total time of one miunte ago. > > Best, > Rui > > On Tue, Sep 5, 2023 at 11:05 PM Maximilian Michels wrote: > > > Hi Gyula, > > > > +1 The proposed changes make sense and are in line with what is > > available fo

[DISCUSS] FLIP-361: Improve GC Metrics

Hi Devs, I would like to start a discussion on FLIP-361: Improve GC Metrics [1]. The current Flink GC metrics [2] are not very useful for monitoring purposes as they require post processing logic that is also dependent on the current runtime environment. Problems: - Total time is not very

[DISSCUSS] Kubernetes Operator Flink Version Support Policy

Hi All! @Maximilian Michels has raised the question of Flink version support in the operator before the last release. I would like to open this discussion publicly so we can finalize this before the next release. Background: Currently the Flink Operator supports all Flink versions since Flink

Re: [DISCUSS] Drop python 3.7 support in 1.19

+1 Thanks for looking into this. Gyula On Mon, Sep 4, 2023 at 8:26 AM Matthias Pohl wrote: > Thanks Gabor for looking into it. It sounds reasonable to me as well. > > +1 > > On Sun, Sep 3, 2023 at 5:44 PM Márton Balassi > wrote: > > > Hi Gabor, > > > > Thanks for bringing this up. Similarly

  1   2   3   4   5   6   7   >