https://github.com/rmckinnon/helix-poc
On Fri, Oct 5, 2018 at 8:49 AM kishore g <[email protected]> wrote: > Is it possible to put your code on github and send me the link? > > On Fri, Oct 5, 2018 at 4:34 AM Rob McKinnon <[email protected]> > wrote: > >> Kishore - I tried different versions of Helix going back to 0.8.0 in >> combinations with different JDK versions and the behavior is the same. It >> feels like we are missing something obvious. Are there any other insights >> on your side? >> >> On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: >> >>> Everything looks right to me. There is something really silly that we >>> are missing. There can’t be any bugs/ regression in this code - it’s core >>> to Helix. >>> >>> Lei, can you spot anything wrong here? >>> >>> >>> >>> On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> >>> wrote: >>> >>>> Kishore, >>>> >>>> Thanks again... I had to modify you snippet a little bit assuming it's >>>> a version thing... >>>> >>>> System.out.println(manager.getClusterManagmentTool() >>>> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >>>> >>>> Output: >>>> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >>>> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >>>> STATE_MODEL_DEF_REF=MasterSlave, >>>> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >>>> localhost_7002=MASTER, localhost_7003=SLAVE}, >>>> data2_1={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>> localhost_7003=MASTER}, data2_2={localhost_7001=MASTER, >>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>> data2_3={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>> localhost_7003=MASTER}, data2_4={localhost_7001=MASTER, >>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>> data2_5={localhost_7001=SLAVE, localhost_7002=MASTER, >>>> localhost_7003=SLAVE}}{} >>>> >>>> >>>> >>>> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> wrote: >>>> >>>>> The code looks good to me. Can you dump the external view from the >>>>> code?. >>>>> >>>>> manager.getClusterManagementTool().getExternalView("data2"); >>>>> >>>>> >>>>> >>>>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>>>> wrote: >>>>> >>>>>> Kishore, >>>>>> >>>>>> Thanks for the reply. When I tried to implement, the instances always >>>>>> come back with an empty list. Can you let me know if I am missing >>>>>> something? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rob >>>>>> >>>>>> >>>>>> ExternalView for data2: >>>>>> { >>>>>> "id" : "data2", >>>>>> "mapFields" : { >>>>>> "data2_0" : { >>>>>> "localhost_7002" : "MASTER", >>>>>> "localhost_7003" : "SLAVE" >>>>>> }, >>>>>> "data2_1" : { >>>>>> "localhost_7002" : "SLAVE", >>>>>> "localhost_7003" : "MASTER" >>>>>> }, >>>>>> "data2_2" : { >>>>>> "localhost_7002" : "SLAVE", >>>>>> "localhost_7003" : "MASTER" >>>>>> }, >>>>>> "data2_3" : { >>>>>> "localhost_7002" : "SLAVE", >>>>>> "localhost_7003" : "MASTER" >>>>>> }, >>>>>> "data2_4" : { >>>>>> "localhost_7002" : "SLAVE", >>>>>> "localhost_7003" : "MASTER" >>>>>> }, >>>>>> "data2_5" : { >>>>>> "localhost_7002" : "MASTER", >>>>>> "localhost_7003" : "SLAVE" >>>>>> } >>>>>> }, >>>>>> "listFields" : { >>>>>> }, >>>>>> "simpleFields" : { >>>>>> "BUCKET_SIZE" : "0", >>>>>> "IDEAL_STATE_MODE" : "AUTO", >>>>>> "NUM_PARTITIONS" : "6", >>>>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>>>> "REPLICAS" : "3", >>>>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> Code Snippet: >>>>>> >>>>>> try { >>>>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>>>> Conf.CLUSTER_NAME, >>>>>> "spec_1", >>>>>> InstanceType.SPECTATOR, >>>>>> Conf.ZK_ADDRESS); >>>>>> >>>>>> manager.connect(); >>>>>> >>>>>> >>>>>> RoutingTableProvider routingTableProvider = new >>>>>> RoutingTableProvider(); >>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>> >>>>>> List<InstanceConfig> list = >>>>>> routingTableProvider.getInstances("data2", "data2_0", >>>>>> "MASTER"); >>>>>> >>>>>> System.out.println(list.size()); // This is coming back as zero >>>>>> >>>>>> >>>>>> } catch (Exception e) { >>>>>> e.printStackTrace(); >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> wrote: >>>>>> >>>>>>> That's right. >>>>>>> >>>>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>>>> instanceName, >>>>>>> InstanceType.SPECTATOR, zkAddr); >>>>>>> zkAddress = zkAddr; >>>>>>> helixClusterName = clusterName; >>>>>>> >>>>>>> manager.connect(); >>>>>>> routingTableProvider = new RoutingTableProvider(); >>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>> >>>>>>> >>>>>>> to route >>>>>>> >>>>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>>>> state);// state can be MASTER in your case >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and I >>>>>>>> have an http request that needs to get data from a specific partition. >>>>>>>> What >>>>>>>> is the pattern to make sure requests are routed to the correct node, in >>>>>>>> this case a node that is the master of that particular partition? >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rob >>>>>>>> >>>>>>>
