Hi Erwin, Yes thats possible. Good idea.
thanks, Kishore G On Fri, Jun 14, 2013 at 3:42 PM, Erwin Karbasi <[email protected]> wrote: > Kishore, > > Thank you a lot for detailed explanation. > > I thought to define in configuration of each participant whether it will > act as controller or Zookeeper as well. All participants will know those > controllers and zookeepers that defined. > For example I will define in 5 participants that they also should act as > controller and zookeepers as well on the same process of the participants. > Is it possible? > > Cheers, > Erwin > On Jun 14, 2013 6:21 PM, "kishore g" <[email protected]> wrote: > >> Hi Erwin, >> >> Here are the options: >> >> 1. participant controller and zookeeper on one process on all 50 boxes: >> This is not possible, you need only 3 or 5 zookeepers running and all >> participant/controllers need to know the machinename: port of the zookeeper >> boxes upfront. >> >> 2. separate zookeeper but participant and controller in one process: This >> is possible, controller can be started as part of all participants but only >> one controller will be active. This is fine until you have like 100 nodes, >> after which its better to have a separate controller. >> >> 3. zookeeper and controller on same box but separate process, >> participants in different: This is possible, note that once you this >> configuration all additional clusters you deploy can share the same >> zookeeper/controller. So if you plan to have many clusters this is a better >> option and much easier to manage the clusters. You can use standard >> commands to start zookeeper and helix controller. >> >> 4. zookeeper and controller on same box and in one process: This also >> works and the advantages of 3 holds good here as well. You will have to >> start >> both of them programmatically. Any time you restart a process, both >> zookeeper and controller will be upgraded. >> >> >> My preference is 3) 4) 2), you cannot do 1). >> >> I agree with you that it requires additional monitoring but actual >> experience suggests that co-locating zookeeper with participants causes >> un-wanted problems. For example, the participants are heavily loaded and >> your application might have large GC pauses/consume al cpu/memory on that >> box. This will adversely impact the functionality of zookeeper. >> >> Thanks, >> Kishore G >> >> >> >> >> >> On Fri, Jun 14, 2013 at 5:44 AM, Erwin Karbasi <[email protected]>wrote: >> >>> My question is, whether is it possible to collocate participant, >>> controller and zookeeper on one process, is there any limitations in term >>> of Helix or Zookeeper? >>> Any direction regarding above approach would be appreciated. >>> On Jun 14, 2013 3:23 PM, "Ming Fang" <[email protected]> wrote: >>> >>>> That's not possible at this time. >>>> ZAC currently must run as a standalone process and only allow one >>>> instance. >>>> >>>> Sent from my iPad >>>> >>>> On Jun 14, 2013, at 7:35 AM, Erwin Karbasi <[email protected]> wrote: >>>> >>>> If I have 40 app server instances (JVMs) could I run participants on >>>> all of them, controller and zookeeper just on 5 of then, but on the same >>>> process of the 5 participants? >>>> On Jun 14, 2013 2:29 PM, "Ming Fang" <[email protected]> wrote: >>>> >>>>> ZAC currently runs Zookeeper in standalone mode. >>>>> Therefore you can only run one instance. >>>>> If you also run the participant in it then that limits you to one >>>>> instance of the participant also. >>>>> >>>>> >>>>> Sent from my iPad >>>>> >>>>> On Jun 14, 2013, at 7:23 AM, Erwin Karbasi <[email protected]> wrote: >>>>> >>>>> Hi Ming, >>>>> >>>>> Thank you for fast turnaround response. >>>>> >>>>> Could I run my app that uses Helix (participant), controller and >>>>> zookeeper on the same process? >>>>> >>>>> Cheers, >>>>> Erwin >>>>> On Jun 14, 2013 2:16 PM, "Ming Fang" <[email protected]> wrote: >>>>> >>>>>> Erwin >>>>>> >>>>>> Here is an example of running Zookeeper, and Controller in the same >>>>>> process, >>>>>> >>>>>> https://github.com/mingfang/apache-helix/blob/master/helix-example/src/main/java/org/apache/helix/examples/ZAC.java >>>>>> We have this in production. >>>>>> >>>>>> Sent from my iPad >>>>>> >>>>>> On Jun 14, 2013, at 3:34 AM, Erwin Karbasi <[email protected]> >>>>>> wrote: >>>>>> >>>>>> I know that it's better that participants, controller and zookeeper >>>>>> will have separate java processes but we have more then 40 boxes with app >>>>>> server that we'd like to manage them with Helix Leader election and >>>>>> cannot >>>>>> add more processes because each additional java process requires >>>>>> management >>>>>> (monitoring, alarms, KPI, etc...), so we prefer to avoid the additional >>>>>> java processes. We'll not manage to approve additional processes >>>>>> >>>>>> Your insight would appreciated. >>>>>> >>>>>> Thanks, >>>>>> Erwin >>>>>> On Jun 14, 2013 9:58 AM, "kishore g" <[email protected]> wrote: >>>>>> >>>>>>> Hi Erwin, >>>>>>> >>>>>>> This is how I would setup your cluster. Allocate 3 boxes that only >>>>>>> run zookeeper and Helix controller. Its better that they are two >>>>>>> separate >>>>>>> process. These boxes need not be powerful but isolating zookeeper and >>>>>>> helix >>>>>>> controller from actual server is a good deployment practice. See >>>>>>> http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServerSetupon >>>>>>> how to use zookeeper clustering ensemble. >>>>>>> >>>>>>> >>>>>>> Run the real servers on the rest of the boxes. Do not start >>>>>>> zookeeper as part of these processes, instead simply provide the >>>>>>> zookeeper >>>>>>> address (zk1:port1,zk2:port2,zk3:port3) when you invoke Helix Api. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Kishore G >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Jun 13, 2013 at 10:05 PM, Erwin Karbasi >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi Kishore, >>>>>>>> >>>>>>>> We have more then 50 nodes. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Erwin >>>>>>>> On Jun 14, 2013 1:51 AM, "kishore g" <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Erwin, >>>>>>>>> >>>>>>>>> How many nodes do you have in your cluster ? >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Kishore G >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Jun 13, 2013 at 1:43 PM, Erwin Karbasi <[email protected] >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> Hello Maters, >>>>>>>>>> >>>>>>>>>> We don't want to have separate/specific JAVA process for >>>>>>>>>> Zookeeper. >>>>>>>>>> As I figured out from the Master-Slave sample we can start the >>>>>>>>>> Zookeeper programmatically using the Zookeeper API. >>>>>>>>>> >>>>>>>>>> I have several questions: >>>>>>>>>> >>>>>>>>>> 1. If we are using the Zookeeper API for initializing the >>>>>>>>>> Zookeeper from the Helix app (out app that uses Helix) would we have >>>>>>>>>> specific process for Zookeeper? Would we have one process for our >>>>>>>>>> app and >>>>>>>>>> another for the Zookeeper? >>>>>>>>>> What happens if we'll not initialize the Zookeeper from the Helix >>>>>>>>>> app programmatically but using the commands, whether then would we >>>>>>>>>> have >>>>>>>>>> specific process for Zookeeper? >>>>>>>>>> >>>>>>>>>> 2. If we'll decide to initialize Zookeeper from Helix app, >>>>>>>>>> programmatically, How can we use Zookeeper clustering (ensemble) , >>>>>>>>>> is there >>>>>>>>>> any API for Zookeeper clustering, any direction would highly >>>>>>>>>> appreciated. >>>>>>>>>> >>>>>>>>>> 3. Is it possible to start the Zookeeper by using commands and >>>>>>>>>> not initializing specific process for it but using the Helix app >>>>>>>>>> (our app >>>>>>>>>> that uses Helix API)? >>>>>>>>>> >>>>>>>>>> I hope my questions are enough clear. >>>>>>>>>> >>>>>>>>>> Thanks in advance, >>>>>>>>>> Erwin Karbasi >>>>>>>>>> AT&T, Senior Software Architect >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>
