Changing the *RoutingTableBuilder rb = new RoutineTableBuilder(); *to *RoutingTableProvider routingTableProvider = new RoutingTableProvider(manager, PropertyType.EXTERNALVIEW);*
worked for me. Looks like that there was an enhancement done recently which allows updating the routing table either based on current state or external view, but it broke the existing api :( @Lei Xia <[email protected]> , any reason for breaking the existing functionality. thanks, Kishore G On Fri, Oct 5, 2018 at 9:28 AM Rob McKinnon <[email protected]> wrote: > https://github.com/rmckinnon/helix-poc > > On Fri, Oct 5, 2018 at 8:49 AM kishore g <[email protected]> wrote: > >> Is it possible to put your code on github and send me the link? >> >> On Fri, Oct 5, 2018 at 4:34 AM Rob McKinnon <[email protected]> >> wrote: >> >>> Kishore - I tried different versions of Helix going back to 0.8.0 in >>> combinations with different JDK versions and the behavior is the same. It >>> feels like we are missing something obvious. Are there any other insights >>> on your side? >>> >>> On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: >>> >>>> Everything looks right to me. There is something really silly that we >>>> are missing. There can’t be any bugs/ regression in this code - it’s core >>>> to Helix. >>>> >>>> Lei, can you spot anything wrong here? >>>> >>>> >>>> >>>> On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> >>>> wrote: >>>> >>>>> Kishore, >>>>> >>>>> Thanks again... I had to modify you snippet a little bit assuming it's >>>>> a version thing... >>>>> >>>>> System.out.println(manager.getClusterManagmentTool() >>>>> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >>>>> >>>>> Output: >>>>> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >>>>> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >>>>> STATE_MODEL_DEF_REF=MasterSlave, >>>>> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >>>>> localhost_7002=MASTER, localhost_7003=SLAVE}, >>>>> data2_1={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>> localhost_7003=MASTER}, data2_2={localhost_7001=MASTER, >>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>> data2_3={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>> localhost_7003=MASTER}, data2_4={localhost_7001=MASTER, >>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>> data2_5={localhost_7001=SLAVE, localhost_7002=MASTER, >>>>> localhost_7003=SLAVE}}{} >>>>> >>>>> >>>>> >>>>> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> wrote: >>>>> >>>>>> The code looks good to me. Can you dump the external view from the >>>>>> code?. >>>>>> >>>>>> manager.getClusterManagementTool().getExternalView("data2"); >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Kishore, >>>>>>> >>>>>>> Thanks for the reply. When I tried to implement, the instances >>>>>>> always come back with an empty list. Can you let me know if I am missing >>>>>>> something? >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Rob >>>>>>> >>>>>>> >>>>>>> ExternalView for data2: >>>>>>> { >>>>>>> "id" : "data2", >>>>>>> "mapFields" : { >>>>>>> "data2_0" : { >>>>>>> "localhost_7002" : "MASTER", >>>>>>> "localhost_7003" : "SLAVE" >>>>>>> }, >>>>>>> "data2_1" : { >>>>>>> "localhost_7002" : "SLAVE", >>>>>>> "localhost_7003" : "MASTER" >>>>>>> }, >>>>>>> "data2_2" : { >>>>>>> "localhost_7002" : "SLAVE", >>>>>>> "localhost_7003" : "MASTER" >>>>>>> }, >>>>>>> "data2_3" : { >>>>>>> "localhost_7002" : "SLAVE", >>>>>>> "localhost_7003" : "MASTER" >>>>>>> }, >>>>>>> "data2_4" : { >>>>>>> "localhost_7002" : "SLAVE", >>>>>>> "localhost_7003" : "MASTER" >>>>>>> }, >>>>>>> "data2_5" : { >>>>>>> "localhost_7002" : "MASTER", >>>>>>> "localhost_7003" : "SLAVE" >>>>>>> } >>>>>>> }, >>>>>>> "listFields" : { >>>>>>> }, >>>>>>> "simpleFields" : { >>>>>>> "BUCKET_SIZE" : "0", >>>>>>> "IDEAL_STATE_MODE" : "AUTO", >>>>>>> "NUM_PARTITIONS" : "6", >>>>>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>>>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>>>>> "REPLICAS" : "3", >>>>>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>>>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> >>>>>>> Code Snippet: >>>>>>> >>>>>>> try { >>>>>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>>>>> Conf.CLUSTER_NAME, >>>>>>> "spec_1", >>>>>>> InstanceType.SPECTATOR, >>>>>>> Conf.ZK_ADDRESS); >>>>>>> >>>>>>> manager.connect(); >>>>>>> >>>>>>> >>>>>>> RoutingTableProvider routingTableProvider = new >>>>>>> RoutingTableProvider(); >>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>> >>>>>>> List<InstanceConfig> list = >>>>>>> routingTableProvider.getInstances("data2", "data2_0", >>>>>>> "MASTER"); >>>>>>> >>>>>>> System.out.println(list.size()); // This is coming back as zero >>>>>>> >>>>>>> >>>>>>> } catch (Exception e) { >>>>>>> e.printStackTrace(); >>>>>>> } >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> That's right. >>>>>>>> >>>>>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>>>>> instanceName, >>>>>>>> InstanceType.SPECTATOR, zkAddr); >>>>>>>> zkAddress = zkAddr; >>>>>>>> helixClusterName = clusterName; >>>>>>>> >>>>>>>> manager.connect(); >>>>>>>> routingTableProvider = new RoutingTableProvider(); >>>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>>> >>>>>>>> >>>>>>>> to route >>>>>>>> >>>>>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>>>>> state);// state can be MASTER in your case >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and I >>>>>>>>> have an http request that needs to get data from a specific >>>>>>>>> partition. What >>>>>>>>> is the pattern to make sure requests are routed to the correct node, >>>>>>>>> in >>>>>>>>> this case a node that is the master of that particular partition? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rob >>>>>>>>> >>>>>>>>
