Kishore - I tried different versions of Helix going back to 0.8.0 in combinations with different JDK versions and the behavior is the same. It feels like we are missing something obvious. Are there any other insights on your side?
On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: > Everything looks right to me. There is something really silly that we are > missing. There can’t be any bugs/ regression in this code - it’s core to > Helix. > > Lei, can you spot anything wrong here? > > > > On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> > wrote: > >> Kishore, >> >> Thanks again... I had to modify you snippet a little bit assuming it's a >> version thing... >> >> System.out.println(manager.getClusterManagmentTool() >> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >> >> Output: >> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >> STATE_MODEL_DEF_REF=MasterSlave, >> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >> localhost_7002=MASTER, localhost_7003=SLAVE}, data2_1={localhost_7001=SLAVE, >> localhost_7002=SLAVE, localhost_7003=MASTER}, >> data2_2={localhost_7001=MASTER, localhost_7002=SLAVE, localhost_7003=SLAVE}, >> data2_3={localhost_7001=SLAVE, localhost_7002=SLAVE, localhost_7003=MASTER}, >> data2_4={localhost_7001=MASTER, localhost_7002=SLAVE, localhost_7003=SLAVE}, >> data2_5={localhost_7001=SLAVE, localhost_7002=MASTER, >> localhost_7003=SLAVE}}{} >> >> >> >> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> wrote: >> >>> The code looks good to me. Can you dump the external view from the code?. >>> >>> manager.getClusterManagementTool().getExternalView("data2"); >>> >>> >>> >>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>> wrote: >>> >>>> Kishore, >>>> >>>> Thanks for the reply. When I tried to implement, the instances always >>>> come back with an empty list. Can you let me know if I am missing >>>> something? >>>> >>>> Regards, >>>> >>>> Rob >>>> >>>> >>>> ExternalView for data2: >>>> { >>>> "id" : "data2", >>>> "mapFields" : { >>>> "data2_0" : { >>>> "localhost_7002" : "MASTER", >>>> "localhost_7003" : "SLAVE" >>>> }, >>>> "data2_1" : { >>>> "localhost_7002" : "SLAVE", >>>> "localhost_7003" : "MASTER" >>>> }, >>>> "data2_2" : { >>>> "localhost_7002" : "SLAVE", >>>> "localhost_7003" : "MASTER" >>>> }, >>>> "data2_3" : { >>>> "localhost_7002" : "SLAVE", >>>> "localhost_7003" : "MASTER" >>>> }, >>>> "data2_4" : { >>>> "localhost_7002" : "SLAVE", >>>> "localhost_7003" : "MASTER" >>>> }, >>>> "data2_5" : { >>>> "localhost_7002" : "MASTER", >>>> "localhost_7003" : "SLAVE" >>>> } >>>> }, >>>> "listFields" : { >>>> }, >>>> "simpleFields" : { >>>> "BUCKET_SIZE" : "0", >>>> "IDEAL_STATE_MODE" : "AUTO", >>>> "NUM_PARTITIONS" : "6", >>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>> "REPLICAS" : "3", >>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>> } >>>> } >>>> >>>> >>>> Code Snippet: >>>> >>>> try { >>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>> Conf.CLUSTER_NAME, >>>> "spec_1", >>>> InstanceType.SPECTATOR, >>>> Conf.ZK_ADDRESS); >>>> >>>> manager.connect(); >>>> >>>> >>>> RoutingTableProvider routingTableProvider = new RoutingTableProvider(); >>>> manager.addExternalViewChangeListener(routingTableProvider); >>>> >>>> List<InstanceConfig> list = routingTableProvider.getInstances("data2", >>>> "data2_0", >>>> "MASTER"); >>>> >>>> System.out.println(list.size()); // This is coming back as zero >>>> >>>> >>>> } catch (Exception e) { >>>> e.printStackTrace(); >>>> } >>>> >>>> >>>> >>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> wrote: >>>> >>>>> That's right. >>>>> >>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>> instanceName, >>>>> InstanceType.SPECTATOR, zkAddr); >>>>> zkAddress = zkAddr; >>>>> helixClusterName = clusterName; >>>>> >>>>> manager.connect(); >>>>> routingTableProvider = new RoutingTableProvider(); >>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>> >>>>> >>>>> to route >>>>> >>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>> state);// state can be MASTER in your case >>>>> >>>>> >>>>> >>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon <[email protected]> >>>>> wrote: >>>>> >>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and I >>>>>> have an http request that needs to get data from a specific partition. >>>>>> What >>>>>> is the pattern to make sure requests are routed to the correct node, in >>>>>> this case a node that is the master of that particular partition? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rob >>>>>> >>>>>
