Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Biao Geng
+1 (non-binding) Best, Biao Geng Peter Huang 于2022年2月7日周一 14:31写道: > +1 (non-binding) > > > Best Regards > Peter Huang > > On Sun, Feb 6, 2022 at 7:35 PM Yang Wang wrote: > > > +1 (binding) > > > > Best, > > Yang > > > > Xintong Song 于2022年2月7日周一 10:25写道: > > > > > +1 (binding) > > > > > > Th

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-06 Thread Gen Luo
Hi Gyula, Thanks for sharing the idea. As Yuan mentioned, I think we can discuss this within two scopes. One is the job subgraph, the other is the execution subgraph, which I suppose is the same as PipelineRegion. An idea is to individually checkpoint the PipelineRegions, for the recovering in a

[jira] [Created] (FLINK-25967) StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on azure

2022-02-06 Thread Yun Gao (Jira)
Yun Gao created FLINK-25967: --- Summary: StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on azure Key: FLINK-25967 URL: https://issues.apache.org/jira/browse/FLINK-25967 Project: F

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Yangze Guo
Thanks everyone for the great effort. The FLIP looks really good. I just want to make sure the configuration priority in the CR example. It seems the requests resources or "taskManager. taskSlots" will be transferred to Flink internal config, e.g. "taskmanager.memory.process.size" and "taskmanager

[jira] [Created] (FLINK-25966) KafkaSourceITCase failed on azure due to Create test topic : topic1 failed

2022-02-06 Thread Yun Gao (Jira)
Yun Gao created FLINK-25966: --- Summary: KafkaSourceITCase failed on azure due to Create test topic : topic1 failed Key: FLINK-25966 URL: https://issues.apache.org/jira/browse/FLINK-25966 Project: Flink

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Peter Huang
+1 (non-binding) Best Regards Peter Huang On Sun, Feb 6, 2022 at 7:35 PM Yang Wang wrote: > +1 (binding) > > Best, > Yang > > Xintong Song 于2022年2月7日周一 10:25写道: > > > +1 (binding) > > > > Thank you~ > > > > Xintong Song > > > > > > > > On Mon, Feb 7, 2022 at 12:52 AM Márton Balassi > > > wro

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-06 Thread Yuan Mei
Hey Gyula, That's a very interesting idea. The discussion about the `Individual` vs `Global` checkpoint was raised before, but the main concern was from two aspects: - Non-deterministic replaying may lead to an inconsistent view of checkpoint - It is not easy to form a clear cut of past and futur

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Yang Wang
+1 (binding) Best, Yang Xintong Song 于2022年2月7日周一 10:25写道: > +1 (binding) > > Thank you~ > > Xintong Song > > > > On Mon, Feb 7, 2022 at 12:52 AM Márton Balassi > wrote: > > > +1 (binding) > > > > On Sat, Feb 5, 2022 at 5:35 PM Israel Ekpo wrote: > > > > > I am very excited to see this. > > >

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Xintong Song
+1 (binding) Thank you~ Xintong Song On Mon, Feb 7, 2022 at 12:52 AM Márton Balassi wrote: > +1 (binding) > > On Sat, Feb 5, 2022 at 5:35 PM Israel Ekpo wrote: > > > I am very excited to see this. > > > > Thanks for driving the effort > > > > +1 (non-binding) > > > > > > On Sat, Feb 5, 2022

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Xintong Song
Sorry for the late reply. We were out due to the public holidays in China. @Thomas, The intention is to support application management through operator and CR, > which means there won't be any 2 step submission process, which as you > allude to would defeat the purpose of this project. The CR exa

Re: [DISCUSS] Checkpointing (partially) failing jobs

2022-02-06 Thread Caizhi Weng
Hi Gyula! Thanks for raising this discussion. I agree that this will be an interesting feature but I actually have some doubts about the motivation and use case. If there are multiple individual subgraphs in the same job, why not just distribute them to multiple jobs so that each job contains only

[DISCUSS] Checkpointing (partially) failing jobs

2022-02-06 Thread Gyula Fóra
Hi all! At the moment checkpointing only works for healthy jobs with all running (or some finished) tasks. This sounds reasonable in most cases but there are a few applications where it would make sense to checkpoint failing jobs as well. Due to how the checkpointing mechanism works, subgraphs th

[jira] [Created] (FLINK-25965) Add a Couchbase sink

2022-02-06 Thread Nir Tsruya (Jira)
Nir Tsruya created FLINK-25965: -- Summary: Add a Couchbase sink Key: FLINK-25965 URL: https://issues.apache.org/jira/browse/FLINK-25965 Project: Flink Issue Type: New Feature Components

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Gyula Fóra
Thank you Alexis, Will definitely check this out. You are right, Kotlin makes it difficult to adopt pieces of this code directly but I think it will be good to get inspiration for the architecture and look at how particular problems have been solved. It will be a great help for us I am sure. Chee

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-06 Thread Márton Balassi
+1 (binding) On Sat, Feb 5, 2022 at 5:35 PM Israel Ekpo wrote: > I am very excited to see this. > > Thanks for driving the effort > > +1 (non-binding) > > > On Sat, Feb 5, 2022 at 10:53 AM Shqiprim Bunjaku < > shqiprimbunj...@gmail.com> > wrote: > > > +1 (non-binding) > > > > > > > > On Sat, Feb