Flink job manager conditional start of flink jobs

2023-04-11 Thread naga sudhakar
Hi Team, Greetings!! Just wanted to know when job manager or task manager is being restarted, is there a way to run the existing flink jobs based on a condition? Same query when I am starting flink job fresh also. Please let me know if any more information is required from my side. Thanks & Regar

Re: Flink job manager conditional start of flink jobs

2023-04-11 Thread Gen Luo
Hi, Is the job you want to start running or already finished? If the job is running, this is simply a failover or a JM failover case. While if the job has finished, there's no such feature that can restart the job automatically, AFAIK. The job has to be submitted again. On Wed, Apr 12, 2023 at 1

Re: Flink job manager conditional start of flink jobs

2023-04-12 Thread naga sudhakar
Thanks for your email. I am looking more in terms of running these flinkk jobs in multi names pace environment and make sure only one namespace flink jobs are running. So on the Job manager when i try to start a flink job, it has to check if it's allowed to run in this namespace or not and acco

Re: Flink job manager conditional start of flink jobs

2023-04-12 Thread Shammon FY
Hi naga Could you provide a specific description of your scene? It sounds like your requirement requires a uniqueness check to ensure that there are no multiple identical jobs running simultaneously, right? Best, Shammon FY On Wed, Apr 12, 2023 at 4:08 PM naga sudhakar wrote: > Thanks for your

Re: Flink job manager conditional start of flink jobs

2023-04-12 Thread naga sudhakar
Hi, Thanks for your reply. It is slightly different, would be happy to have any suggestion for the scenario you mentioned. My scenario: I have 2 namespaces say ns1,ns2. I have to make sure only one of ns1 or ns2 should run my flink jobs. Say initially ns1 is running flink jobs, later planned to mo

Re: Flink job manager conditional start of flink jobs

2023-04-12 Thread Shammon FY
Hi The job in ns2 has the permission to stop the job in ns1? How about managing the relationship in your `Job Submission Service` if it exists. The service can check and stop the job in ns1 before it submitting the job to ns2, what do you think? Best, Shammon FY On Thu, Apr 13, 2023 at 10:50 AM

Re: Flink job manager conditional start of flink jobs

2023-04-13 Thread Hang Ruan
Hi, nage, I agree to the Shammon's suggestion after reading the context. Maybe you need a 'Job Management Service' to manage all jobs among different namespaces. I think the job management is not suitable for implementation in the Flink engine. Best, Hang Shammon FY 于2023年4月13日周四 11:34写道: > Hi