Can you please file a jira. We need to fix this. On Fri, Oct 5, 2018 at 1:48 PM Rob McKinnon <[email protected]> wrote:
> Kishore - Adding PropertyType.EXTERNALVIEW to the RoutingTableProvider > constructor did indeed fix the problem. Thank you for the assistance. > > Regards, > > Rob > > On Fri, Oct 5, 2018 at 1:55 PM kishore g <[email protected]> wrote: > >> Changing the *RoutingTableBuilder rb = new RoutineTableBuilder(); *to >> >> *RoutingTableProvider routingTableProvider = new >> RoutingTableProvider(manager, PropertyType.EXTERNALVIEW);* >> >> worked for me. >> >> Looks like that there was an enhancement done recently which allows >> updating the routing table either based on current state or external view, >> but it broke the existing api :( >> >> @Lei Xia <[email protected]> , any reason for breaking the existing >> functionality. >> >> thanks, >> Kishore G >> >> On Fri, Oct 5, 2018 at 9:28 AM Rob McKinnon <[email protected]> >> wrote: >> >>> https://github.com/rmckinnon/helix-poc >>> >>> On Fri, Oct 5, 2018 at 8:49 AM kishore g <[email protected]> wrote: >>> >>>> Is it possible to put your code on github and send me the link? >>>> >>>> On Fri, Oct 5, 2018 at 4:34 AM Rob McKinnon <[email protected]> >>>> wrote: >>>> >>>>> Kishore - I tried different versions of Helix going back to 0.8.0 in >>>>> combinations with different JDK versions and the behavior is the same. It >>>>> feels like we are missing something obvious. Are there any other insights >>>>> on your side? >>>>> >>>>> On Wed, Oct 3, 2018 at 12:12 AM kishore g <[email protected]> wrote: >>>>> >>>>>> Everything looks right to me. There is something really silly that we >>>>>> are missing. There can’t be any bugs/ regression in this code - it’s core >>>>>> to Helix. >>>>>> >>>>>> Lei, can you spot anything wrong here? >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Oct 2, 2018 at 5:18 PM Rob McKinnon <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Kishore, >>>>>>> >>>>>>> Thanks again... I had to modify you snippet a little bit assuming >>>>>>> it's a version thing... >>>>>>> >>>>>>> System.out.println(manager.getClusterManagmentTool() >>>>>>> .getResourceExternalView(Conf.CLUSTER_NAME, "data2")); >>>>>>> >>>>>>> Output: >>>>>>> data2, {BUCKET_SIZE=0, IDEAL_STATE_MODE=AUTO, NUM_PARTITIONS=6, >>>>>>> REBALANCE_MODE=SEMI_AUTO, REBALANCE_STRATEGY=DEFAULT, REPLICAS=3, >>>>>>> STATE_MODEL_DEF_REF=MasterSlave, >>>>>>> STATE_MODEL_FACTORY_NAME=DEFAULT}{data2_0={localhost_7001=SLAVE, >>>>>>> localhost_7002=MASTER, localhost_7003=SLAVE}, >>>>>>> data2_1={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>>>> localhost_7003=MASTER}, data2_2={localhost_7001=MASTER, >>>>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>>>> data2_3={localhost_7001=SLAVE, localhost_7002=SLAVE, >>>>>>> localhost_7003=MASTER}, data2_4={localhost_7001=MASTER, >>>>>>> localhost_7002=SLAVE, localhost_7003=SLAVE}, >>>>>>> data2_5={localhost_7001=SLAVE, localhost_7002=MASTER, >>>>>>> localhost_7003=SLAVE}}{} >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 2, 2018 at 6:40 PM kishore g <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> The code looks good to me. Can you dump the external view from the >>>>>>>> code?. >>>>>>>> >>>>>>>> manager.getClusterManagementTool().getExternalView("data2"); >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 2, 2018 at 3:20 PM Rob McKinnon <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Kishore, >>>>>>>>> >>>>>>>>> Thanks for the reply. When I tried to implement, the instances >>>>>>>>> always come back with an empty list. Can you let me know if I am >>>>>>>>> missing >>>>>>>>> something? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rob >>>>>>>>> >>>>>>>>> >>>>>>>>> ExternalView for data2: >>>>>>>>> { >>>>>>>>> "id" : "data2", >>>>>>>>> "mapFields" : { >>>>>>>>> "data2_0" : { >>>>>>>>> "localhost_7002" : "MASTER", >>>>>>>>> "localhost_7003" : "SLAVE" >>>>>>>>> }, >>>>>>>>> "data2_1" : { >>>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>>> "localhost_7003" : "MASTER" >>>>>>>>> }, >>>>>>>>> "data2_2" : { >>>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>>> "localhost_7003" : "MASTER" >>>>>>>>> }, >>>>>>>>> "data2_3" : { >>>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>>> "localhost_7003" : "MASTER" >>>>>>>>> }, >>>>>>>>> "data2_4" : { >>>>>>>>> "localhost_7002" : "SLAVE", >>>>>>>>> "localhost_7003" : "MASTER" >>>>>>>>> }, >>>>>>>>> "data2_5" : { >>>>>>>>> "localhost_7002" : "MASTER", >>>>>>>>> "localhost_7003" : "SLAVE" >>>>>>>>> } >>>>>>>>> }, >>>>>>>>> "listFields" : { >>>>>>>>> }, >>>>>>>>> "simpleFields" : { >>>>>>>>> "BUCKET_SIZE" : "0", >>>>>>>>> "IDEAL_STATE_MODE" : "AUTO", >>>>>>>>> "NUM_PARTITIONS" : "6", >>>>>>>>> "REBALANCE_MODE" : "SEMI_AUTO", >>>>>>>>> "REBALANCE_STRATEGY" : "DEFAULT", >>>>>>>>> "REPLICAS" : "3", >>>>>>>>> "STATE_MODEL_DEF_REF" : "MasterSlave", >>>>>>>>> "STATE_MODEL_FACTORY_NAME" : "DEFAULT" >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> Code Snippet: >>>>>>>>> >>>>>>>>> try { >>>>>>>>> HelixManager manager = HelixManagerFactory.getZKHelixManager( >>>>>>>>> Conf.CLUSTER_NAME, >>>>>>>>> "spec_1", >>>>>>>>> InstanceType.SPECTATOR, >>>>>>>>> Conf.ZK_ADDRESS); >>>>>>>>> >>>>>>>>> manager.connect(); >>>>>>>>> >>>>>>>>> >>>>>>>>> RoutingTableProvider routingTableProvider = new >>>>>>>>> RoutingTableProvider(); >>>>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>>>> >>>>>>>>> List<InstanceConfig> list = >>>>>>>>> routingTableProvider.getInstances("data2", "data2_0", >>>>>>>>> "MASTER"); >>>>>>>>> >>>>>>>>> System.out.println(list.size()); // This is coming back as zero >>>>>>>>> >>>>>>>>> >>>>>>>>> } catch (Exception e) { >>>>>>>>> e.printStackTrace(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Oct 2, 2018 at 9:15 AM kishore g <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> That's right. >>>>>>>>>> >>>>>>>>>> manager = HelixManagerFactory.getZKHelixManager(clusterName, >>>>>>>>>> instanceName, >>>>>>>>>> InstanceType.SPECTATOR, zkAddr); >>>>>>>>>> zkAddress = zkAddr; >>>>>>>>>> helixClusterName = clusterName; >>>>>>>>>> >>>>>>>>>> manager.connect(); >>>>>>>>>> routingTableProvider = new RoutingTableProvider(); >>>>>>>>>> manager.addExternalViewChangeListener(routingTableProvider); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> to route >>>>>>>>>> >>>>>>>>>> instances = routingTable.getInstances(resourceName, partionName, >>>>>>>>>> state);// state can be MASTER in your case >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Oct 2, 2018 at 5:51 AM Rob McKinnon < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Lets say that I have a cluster of 5 nodes with 10 partitions and >>>>>>>>>>> I have an http request that needs to get data from a specific >>>>>>>>>>> partition. >>>>>>>>>>> What is the pattern to make sure requests are routed to the correct >>>>>>>>>>> node, >>>>>>>>>>> in this case a node that is the master of that particular partition? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Rob >>>>>>>>>>> >>>>>>>>>>
