[jira] [Commented] (FLINK-35417) JobManager and TaskManager support merging and run in a single process

2024-05-22 Thread Jing Ge (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-35417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848706#comment-17848706
 ] 

Jing Ge commented on FLINK-35417:
-

[~jeyhunkarimov] assigned you, please pay attention to comments above, thanks!

> JobManager and TaskManager support merging and run in a single process
> --
>
> Key: FLINK-35417
> URL: https://issues.apache.org/jira/browse/FLINK-35417
> Project: Flink
>  Issue Type: Improvement
>  Components: API / Core
>Reporter: melin
>Assignee: Jeyhun Karimov
>Priority: Major
>
> flink is widely used in data integration scenarios, where a single 
> concurrency is not high, and in many cases a single concurrency can run a 
> task. Consider the high availability, application mode, and large number of 
> JobManger nodes that cost a lot of resources. If the Session mode is used, 
> the stability is not high.
> In application mode, JobManager and TaskManager can be run together to 
> achieve reliability and save resources.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-35417) JobManager and TaskManager support merging and run in a single process

2024-05-22 Thread melin (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-35417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848676#comment-17848676
 ] 

melin commented on FLINK-35417:
---

The current deployment model has encountered challenges in many projects. I 
just want to send an issue to discuss, perhaps a mailing list is more 
appropriate.
There is currently no ability to implement new deployment patterns. 
[~jeyhunkarimov] You are welcome to drive this issue. Thank you very much!

> JobManager and TaskManager support merging and run in a single process
> --
>
> Key: FLINK-35417
> URL: https://issues.apache.org/jira/browse/FLINK-35417
> Project: Flink
>  Issue Type: Improvement
>  Components: API / Core
>Reporter: melin
>Priority: Major
>
> flink is widely used in data integration scenarios, where a single 
> concurrency is not high, and in many cases a single concurrency can run a 
> task. Consider the high availability, application mode, and large number of 
> JobManger nodes that cost a lot of resources. If the Session mode is used, 
> the stability is not high.
> In application mode, JobManager and TaskManager can be run together to 
> achieve reliability and save resources.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-35417) JobManager and TaskManager support merging and run in a single process

2024-05-22 Thread Jeyhun Karimov (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-35417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848618#comment-17848618
 ] 

Jeyhun Karimov commented on FLINK-35417:


Hi [~melin], I echo the comments above. In case, you are not planning to work 
on this, I can spend some cycles to drive this issue.

> JobManager and TaskManager support merging and run in a single process
> --
>
> Key: FLINK-35417
> URL: https://issues.apache.org/jira/browse/FLINK-35417
> Project: Flink
>  Issue Type: Improvement
>  Components: API / Core
>Reporter: melin
>Priority: Major
>
> flink is widely used in data integration scenarios, where a single 
> concurrency is not high, and in many cases a single concurrency can run a 
> task. Consider the high availability, application mode, and large number of 
> JobManger nodes that cost a lot of resources. If the Session mode is used, 
> the stability is not high.
> In application mode, JobManager and TaskManager can be run together to 
> achieve reliability and save resources.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-35417) JobManager and TaskManager support merging and run in a single process

2024-05-22 Thread Xintong Song (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-35417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848514#comment-17848514
 ] 

Xintong Song commented on FLINK-35417:
--

Hi [~melin],

Please help me understand. Are you willing to make a contribution on the 
proposed feature? Or are you only expressing a demand but has no intention to 
work on it?

If you'd like to contribute, I agree with [~martijnvisser] that this is a bit 
too big for a Jira ticket. Please propose this on the mailing list. My gut 
feeling this would introduce a new deployment mode, as well as public interface 
changes, which means it needs to go through a FLIP process.

If you're only making a demand, then I think this is a valid demand. We have 
similar observations in our production, and I have even thought about similar 
ideas. Unfortunately, I haven't yet find enough time and resources to work on 
it.

> JobManager and TaskManager support merging and run in a single process
> --
>
> Key: FLINK-35417
> URL: https://issues.apache.org/jira/browse/FLINK-35417
> Project: Flink
>  Issue Type: Improvement
>  Components: API / Core
>Reporter: melin
>Priority: Major
>
> flink is widely used in data integration scenarios, where a single 
> concurrency is not high, and in many cases a single concurrency can run a 
> task. Consider the high availability, application mode, and large number of 
> JobManger nodes that cost a lot of resources. If the Session mode is used, 
> the stability is not high.
> In application mode, JobManager and TaskManager can be run together to 
> achieve reliability and save resources.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-35417) JobManager and TaskManager support merging and run in a single process

2024-05-22 Thread Martijn Visser (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-35417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848485#comment-17848485
 ] 

Martijn Visser commented on FLINK-35417:


Per https://flink.apache.org/how-to-contribute/contribute-code/#consensus this 
should first be a discussion on the Dev mailing list before opening a Jira 
ticket. 

> JobManager and TaskManager support merging and run in a single process
> --
>
> Key: FLINK-35417
> URL: https://issues.apache.org/jira/browse/FLINK-35417
> Project: Flink
>  Issue Type: Improvement
>  Components: API / Core
>Reporter: melin
>Priority: Major
>
> flink is widely used in data integration scenarios, where a single 
> concurrency is not high, and in many cases a single concurrency can run a 
> task. Consider the high availability, application mode, and large number of 
> JobManger nodes that cost a lot of resources. If the Session mode is used, 
> the stability is not high.
> In application mode, JobManager and TaskManager can be run together to 
> achieve reliability and save resources.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)