Is it possible to put your code on github and send me the link? On Fri, Oct 5, 2018 at 4:34 AM Rob McKinnon <[email protected]> wrote:
> Kishore - I tried different versions of Helix going back to 0.8.0 in > combinations with different JDK versions and the behavior is the same. It > feels like we are missing something obvious. Are there any other insights > on your side? > > On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: > >> Everything looks right to me. There is something really silly that we are >> missing. There can’t be any bugs/ regression in this code - it’s core to >> Helix. >> >> Lei, can you spot anything wrong here? >> >> >> >> On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> >> wrote: >> >>> Kishore, >>> >>> Thanks again... I had to modify you snippet a little bit assuming it's a >>> version thing... >>> >>> System.out.println(manager.getClusterManagmentTool() >>> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >>> >>> Output: >>> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >>> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >>> STATE_MODEL_DEF_REF=MasterSlave, >>> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >>> localhost_7002=MASTER, localhost_7003=SLAVE}, >>> data2_1={localhost_7001=SLAVE, localhost_7002=SLAVE, >>> localhost_7003=MASTER}, data2_2={localhost_7001=MASTER, >>> localhost_7002=SLAVE, localhost_7003=SLAVE}, data2_3={localhost_7001=SLAVE, >>> localhost_7002=SLAVE, localhost_7003=MASTER}, >>> data2_4={localhost_7001=MASTER, localhost_7002=SLAVE, >>> localhost_7003=SLAVE}, data2_5={localhost_7001=SLAVE, >>> localhost_7002=MASTER, localhost_7003=SLAVE}}{} >>> >>> >>> >>> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> wrote: >>> >>>> The code looks good to me. Can you dump the external view from the >>>> code?. >>>> >>>> manager.getClusterManagementTool().getExternalView("data2"); >>>> >>>> >>>> >>>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>>> wrote: >>>> >>>>> Kishore, >>>>> >>>>> Thanks for the reply. When I tried to implement, the instances always >>>>> come back with an empty list. Can you let me know if I am missing >>>>> something? >>>>> >>>>> Regards, >>>>> >>>>> Rob >>>>> >>>>> >>>>> ExternalView for data2: >>>>> { >>>>> "id" : "data2", >>>>> "mapFields" : { >>>>> "data2_0" : { >>>>> "localhost_7002" : "MASTER", >>>>> "localhost_7003" : "SLAVE" >>>>> }, >>>>> "data2_1" : { >>>>> "localhost_7002" : "SLAVE", >>>>> "localhost_7003" : "MASTER" >>>>> }, >>>>> "data2_2" : { >>>>> "localhost_7002" : "SLAVE", >>>>> "localhost_7003" : "MASTER" >>>>> }, >>>>> "data2_3" : { >>>>> "localhost_7002" : "SLAVE", >>>>> "localhost_7003" : "MASTER" >>>>> }, >>>>> "data2_4" : { >>>>> "localhost_7002" : "SLAVE", >>>>> "localhost_7003" : "MASTER" >>>>> }, >>>>> "data2_5" : { >>>>> "localhost_7002" : "MASTER", >>>>> "localhost_7003" : "SLAVE" >>>>> } >>>>> }, >>>>> "listFields" : { >>>>> }, >>>>> "simpleFields" : { >>>>> "BUCKET_SIZE" : "0", >>>>> "IDEAL_STATE_MODE" : "AUTO", >>>>> "NUM_PARTITIONS" : "6", >>>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>>> "REPLICAS" : "3", >>>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>>> } >>>>> } >>>>> >>>>> >>>>> Code Snippet: >>>>> >>>>> try { >>>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>>> Conf.CLUSTER_NAME, >>>>> "spec_1", >>>>> InstanceType.SPECTATOR, >>>>> Conf.ZK_ADDRESS); >>>>> >>>>> manager.connect(); >>>>> >>>>> >>>>> RoutingTableProvider routingTableProvider = new >>>>> RoutingTableProvider(); >>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>> >>>>> List<InstanceConfig> list = >>>>> routingTableProvider.getInstances("data2", "data2_0", >>>>> "MASTER"); >>>>> >>>>> System.out.println(list.size()); // This is coming back as zero >>>>> >>>>> >>>>> } catch (Exception e) { >>>>> e.printStackTrace(); >>>>> } >>>>> >>>>> >>>>> >>>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> wrote: >>>>> >>>>>> That's right. >>>>>> >>>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>>> instanceName, >>>>>> InstanceType.SPECTATOR, zkAddr); >>>>>> zkAddress = zkAddr; >>>>>> helixClusterName = clusterName; >>>>>> >>>>>> manager.connect(); >>>>>> routingTableProvider = new RoutingTableProvider(); >>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>> >>>>>> >>>>>> to route >>>>>> >>>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>>> state);// state can be MASTER in your case >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and I >>>>>>> have an http request that needs to get data from a specific partition. >>>>>>> What >>>>>>> is the pattern to make sure requests are routed to the correct node, in >>>>>>> this case a node that is the master of that particular partition? >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Rob >>>>>>> >>>>>>
