Kishore - Adding PropertyType.EXTERNALVIEW to the RoutingTableProvider constructor did indeed fix the problem. Thank you for the assistance.
Regards, Rob On Fri, Oct 5, 2018 at 1:55 PM kishore g <[email protected]> wrote: > Changing the *RoutingTableBuilder rb = new RoutineTableBuilder(); *to > > *RoutingTableProvider routingTableProvider = new > RoutingTableProvider(manager, PropertyType.EXTERNALVIEW);* > > worked for me. > > Looks like that there was an enhancement done recently which allows > updating the routing table either based on current state or external view, > but it broke the existing api :( > > @Lei Xia <[email protected]> , any reason for breaking the existing > functionality. > > thanks, > Kishore G > > On Fri, Oct 5, 2018 at 9:28 AM Rob McKinnon <[email protected]> > wrote: > >> https://github.com/rmckinnon/helix-poc >> >> On Fri, Oct 5, 2018 at 8:49 AM kishore g <[email protected]> wrote: >> >>> Is it possible to put your code on github and send me the link? >>> >>> On Fri, Oct 5, 2018 at 4:34 AM Rob McKinnon <[email protected]> >>> wrote: >>> >>>> Kishore - I tried different versions of Helix going back to 0.8.0 in >>>> combinations with different JDK versions and the behavior is the same. It >>>> feels like we are missing something obvious. Are there any other insights >>>> on your side? >>>> >>>> On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: >>>> >>>>> Everything looks right to me. There is something really silly that we >>>>> are missing. There can’t be any bugs/ regression in this code - it’s core >>>>> to Helix. >>>>> >>>>> Lei, can you spot anything wrong here? >>>>> >>>>> >>>>> >>>>> On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> >>>>> wrote: >>>>> >>>>>> Kishore, >>>>>> >>>>>> Thanks again... I had to modify you snippet a little bit assuming >>>>>> it's a version thing... >>>>>> >>>>>> System.out.println(manager.getClusterManagmentTool() >>>>>> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >>>>>> >>>>>> Output: >>>>>> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >>>>>> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >>>>>> STATE_MODEL_DEF_REF=MasterSlave, >>>>>> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >>>>>> localhost_7002=MASTER, localhost_7003=SLAVE}, >>>>>> data2_1={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>>> localhost_7003=MASTER}, data2_2={localhost_7001=MASTER, >>>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>>> data2_3={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>>> localhost_7003=MASTER}, data2_4={localhost_7001=MASTER, >>>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>>> data2_5={localhost_7001=SLAVE, localhost_7002=MASTER, >>>>>> localhost_7003=SLAVE}}{} >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> wrote: >>>>>> >>>>>>> The code looks good to me. Can you dump the external view from the >>>>>>> code?. >>>>>>> >>>>>>> manager.getClusterManagementTool().getExternalView("data2"); >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Kishore, >>>>>>>> >>>>>>>> Thanks for the reply. When I tried to implement, the instances >>>>>>>> always come back with an empty list. Can you let me know if I am >>>>>>>> missing >>>>>>>> something? >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rob >>>>>>>> >>>>>>>> >>>>>>>> ExternalView for data2: >>>>>>>> { >>>>>>>> "id" : "data2", >>>>>>>> "mapFields" : { >>>>>>>> "data2_0" : { >>>>>>>> "localhost_7002" : "MASTER", >>>>>>>> "localhost_7003" : "SLAVE" >>>>>>>> }, >>>>>>>> "data2_1" : { >>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>> "localhost_7003" : "MASTER" >>>>>>>> }, >>>>>>>> "data2_2" : { >>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>> "localhost_7003" : "MASTER" >>>>>>>> }, >>>>>>>> "data2_3" : { >>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>> "localhost_7003" : "MASTER" >>>>>>>> }, >>>>>>>> "data2_4" : { >>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>> "localhost_7003" : "MASTER" >>>>>>>> }, >>>>>>>> "data2_5" : { >>>>>>>> "localhost_7002" : "MASTER", >>>>>>>> "localhost_7003" : "SLAVE" >>>>>>>> } >>>>>>>> }, >>>>>>>> "listFields" : { >>>>>>>> }, >>>>>>>> "simpleFields" : { >>>>>>>> "BUCKET_SIZE" : "0", >>>>>>>> "IDEAL_STATE_MODE" : "AUTO", >>>>>>>> "NUM_PARTITIONS" : "6", >>>>>>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>>>>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>>>>>> "REPLICAS" : "3", >>>>>>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>>>>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> Code Snippet: >>>>>>>> >>>>>>>> try { >>>>>>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>>>>>> Conf.CLUSTER_NAME, >>>>>>>> "spec_1", >>>>>>>> InstanceType.SPECTATOR, >>>>>>>> Conf.ZK_ADDRESS); >>>>>>>> >>>>>>>> manager.connect(); >>>>>>>> >>>>>>>> >>>>>>>> RoutingTableProvider routingTableProvider = new >>>>>>>> RoutingTableProvider(); >>>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>>> >>>>>>>> List<InstanceConfig> list = >>>>>>>> routingTableProvider.getInstances("data2", "data2_0", >>>>>>>> "MASTER"); >>>>>>>> >>>>>>>> System.out.println(list.size()); // This is coming back as zero >>>>>>>> >>>>>>>> >>>>>>>> } catch (Exception e) { >>>>>>>> e.printStackTrace(); >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> That's right. >>>>>>>>> >>>>>>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>>>>>> instanceName, >>>>>>>>> InstanceType.SPECTATOR, zkAddr); >>>>>>>>> zkAddress = zkAddr; >>>>>>>>> helixClusterName = clusterName; >>>>>>>>> >>>>>>>>> manager.connect(); >>>>>>>>> routingTableProvider = new RoutingTableProvider(); >>>>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>>>> >>>>>>>>> >>>>>>>>> to route >>>>>>>>> >>>>>>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>>>>>> state);// state can be MASTER in your case >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and >>>>>>>>>> I have an http request that needs to get data from a specific >>>>>>>>>> partition. >>>>>>>>>> What is the pattern to make sure requests are routed to the correct >>>>>>>>>> node, >>>>>>>>>> in this case a node that is the master of that particular partition? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Rob >>>>>>>>>> >>>>>>>>>
