Hey Alex, I can't really paste the source, but I can help with pseudo-code:
- We have 3 categories of nodes, let's call them Workers, Scheduler, and Seeds (There are NodeAttributes that set each node in a particular group) - Most nodes are Workers and roughly 20 nodes are Scheduler - and Seeds has only 3 nodes and is a bit irrelevant in this case - We have roughly 1,000 services deployed across this cluster - most of them are cluster singletons, few are node singletons - Within those 1,000 services, most go to the Schedulers group, via a node attribute filter (.forAttribute) - We would like a solutions that could round-robin the service deployment for us (So that each Scheduler node would get 900 / 20 services ~), if possible, because when services congregate in a particular node, it gets overloaded and crashes - Using affinity key is problematic because the cache could rebalance and redeploy services with no real reason Would love your assistance ! - Danny On Tue, May 16, 2017 at 4:01 PM afedotov [via Apache Ignite Users] < [email protected]> wrote: > Danny, just one more question. > Are all the services deployed as cluster singletons? > > Kind regards, > Alex. > > On Tue, May 16, 2017 at 1:10 PM, afedotov <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=12888&i=0>> wrote: > >> Hi Danny, >> >> Could you please share your node filter/cluster group pedicate? >> >> Kind regards, >> Alex. >> >> On Mon, May 15, 2017 at 10:49 PM, dany74q [via Apache Ignite Users] <[hidden >> email] <http:///user/SendEmail.jtp?type=node&node=12883&i=0>> wrote: >> >>> Hey Alex. >>> Thanks for the prompt response ! >>> We actually do employ a node filter - lets say we have N nodes, we >>> deploy our services to a group B, being roughly N/10 of size (20~ nodes). >>> >>> Our services are quite extensive, and so deploying them unevenly across >>> B causes GCs, stalls and other unfortunate results. >>> >>> We have implemented a round robin approach, where we create a cluster >>> group consisting of one of the nodes in B (node at position counter % >>> B_size). >>> But it has many shortcomings - >>> their deployment predicate is always fixed on that specific node, and it >>> doesn't scale when adding additional nodes with the same node attribute. >>> >>> We're looking for a different solution. >>> Question - is it possible to have an affinity key deployment strategy, >>> but not having the services "rebalance" when a node leaves / joins ? >>> Any other solutions ? >>> On Mon, May 15, 2017 at 10:12 PM afedotov [via Apache Ignite Users] <[hidden >>> email] <http:///user/SendEmail.jtp?type=node&node=12861&i=0>> wrote: >>> >>>> Danny, in this case, it looks like you are right and the random don't >>>> provide even distribution. >>>> To avoid this you could devise a solution based on a node filter if >>>> affinity deployment doesn't do the trick for you >>>> >>>> https://apacheignite.readme.io/v1.8/docs/service-grid#section-node-filter-based-deployment >>>> >>>> Kind regards, >>>> Alex. >>>> >>>> On Mon, May 15, 2017 at 8:34 PM, dany74q <[hidden email] >>>> <http:///user/SendEmail.jtp?type=node&node=12860&i=0>> wrote: >>>> Hey Alex, >>>> >>>> The services are deployed dynamically after all nodes are started. >>>> It could be the case that the random distribution isn't "random enough" >>>> - and most of the services are concentrated on specific nodes. >>>> >>>> Could we somehow control the way those services are deployed within >>>> their cluster group, but without an affinity key ? >>>> >>>> Would love that ability ! >>>> >>>> On Mon, May 15, 2017 at 4:08 PM afedotov [via Apache Ignite Users] <[hidden >>> email] <http:///user/SendEmail.jtp?type=node&node=12858&i=0>> wrote: >>> Hi Danny. >>> >>> At the moment, cluster singleton services are deployed on a random basis. >>> How many Ignite nodes do you run? >>> How did you deploy services: after all the nodes had been started or you >>> have the services specified in configuration? >>> >>> Kind regards, >>> Alex. >>> >>> On Sat, May 13, 2017 at 1:06 AM, dany74q [via Apache Ignite Users] <[hidden >>> email] <http:///user/SendEmail.jtp?type=node&node=12845&i=0>> wrote: >>> Hi everyone, >>> >>> We use ignite's (1.7.4) service grid to deploy hundreds of (different) >>> services in our cluster. >>> There's a behavior we've noticed, once deploying that many services, >>> that they tend do deploy unevenly in the cluster, >>> that means that we can get to a point where a great number of services >>> are deployed on a single node, instead of spreading them out more evenly. >>> >>> Is there any way to guarantee a random or even distribution, or a >>> round-robin deployment policy on our nodes, without using affinity keys >>> (Partitions may shift, and so services can get cancelled and redeployed >>> with no reason) ? >>> >>> Thanks ! >>> - Danny >>> >>> ------------------------------ >>> If you reply to this email, your message will be added to the discussion >>> below: >>> >>> http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674.html >>> To start a new topic under Apache Ignite Users, email [hidden email] >>> <http:///user/SendEmail.jtp?type=node&node=12845&i=1> >>> To unsubscribe from Apache Ignite Users, click here. >>> NAML >>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >>> >>> >>> ------------------------------ >>> If you reply to this email, your message will be added to the discussion >>> below: >>> >>> http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12845.html >>> To unsubscribe from How to deploy ignite service evenly across the >>> cluster ?, click here. >>> NAML >>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >>> ------------------------------ >>> View this message in context: Re: How to deploy ignite service evenly >>> across the cluster ? >>> <http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12858.html> >>> >>> Sent from the Apache Ignite Users mailing list archive >>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. >>> If you reply to this email, your message will be added to the discussion >>> below: >>> >>> http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12860.html >>> To unsubscribe from How to deploy ignite service evenly across the >>> cluster ?, click here. >>> NAML >>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >>> >>> ------------------------------ >>> If you reply to this email, your message will be added to the discussion >>> below: >>> >>> http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12861.html >>> To start a new topic under Apache Ignite Users, email [hidden email] >>> <http:///user/SendEmail.jtp?type=node&node=12883&i=1> >>> To unsubscribe from Apache Ignite Users, click here. >>> NAML >>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >> >> >> ------------------------------ >> View this message in context: Re: How to deploy ignite service evenly >> across the cluster ? >> <http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12883.html> >> Sent from the Apache Ignite Users mailing list archive >> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. >> > > If you reply to this email, your message will be added to the discussion > below: > > http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12888.html > To unsubscribe from How to deploy ignite service evenly across the cluster > ?, click here > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=12674&code=ZGFueTc0cUBnbWFpbC5jb218MTI2NzR8LTEzNTAyNDczNjE=> > . > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p13020.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
