GitHub user luoluoyuyu added a comment to the discussion: Elements Service Load Balancer
Hi @bossenti I would expect that adding a load balancer would not change the existing user operation logic. Regarding the separation of the pipeline it should be invisible for the user, just in CORE dividing the pipeline in branches as allocation units. For the extension service (the pipeline is also not visible, it only knows to run one of the elements), so this part can only be done in core. There are several reasons for pipeline separation: 1, as shown in the above allocation, the elements that can be run by each extension service may be different, which may result in fewer extension services being available for allocation in pipeline-based allocation units, and if this occurs for a large number of pipelines it can result in a serious overload of one service and low utilisation of other services. 2. When a pipeline consumes a large number of server resources, the pipeline can be divided into two parts, which can better average the computing resources. Regarding the separated units of the pipeline, it is only necessary to add additional data structures to save them. GitHub link: https://github.com/apache/streampipes/discussions/2552#discussioncomment-8784092 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
