Re: Storm - field grouping question for Class instance tuples
Thanks Srinath, I have already started using this approach. ( i.e emitting multiple tuples , and doing field grouping in respective bolts for my required string key). Thought there could be some other way too. On Wed, Mar 19, 2014 at 9:31 PM, Srinath C srinat...@gmail.com wrote: You can have Bolt1 do: public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance.class1Key1, class1Instance)); // emit 2 values - class1Key1 and class1Instance } Then do field grouping for class1Key1 into Bolt2. On Wed, Mar 19, 2014 at 9:26 PM, Binita Bharati binita.bhar...@gmail.com wrote: Anyone has any idea on this please ? Thanks On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati binita.bhar...@gmail.com wrote: Code sample : Topology : === TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(epd-spout,new MyJmsSpout()); builder.setBolt(epd-CPY001-bolt1, new Bolt1(CPY001), parallelismHintMap.get(CPY001)) .shuffleGrouping(epd-spout); builder.setBolt(epd-CPY001-bolt2, new Bolt2(CPY001), parallelismHintMap.get(CPY001)) .fieldsGrouping(epd-CPY001-bolt1, new Fields(Bolt1Tuple)); Bolt1: public void declareOutputFields(OutputFieldsDeclarer declarer) { // TODO Auto-generated method stub declarer.declare(new Fields(Bolt1Tuple)); } public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance)); } Bolt2: == public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(Bolt1Tuple); } On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati binita.bhar...@gmail.com wrote: Hi , I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has a String field - class1Key1. I need to segregate all tuples having same value for a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2. How do I go about defining such a topology ? I have seen egs where tuples emitted were simple string values. But, my emitted tuple is a Class instance , and the field on which I want to do fieldGrouping is a particular field of that Class. (I tried defining a equals and hashcode method in a.b.c.MyClass1. Didn't work as expected..) Thanks Binita
Re: Storm - field grouping question for Class instance tuples
You can have Bolt1 do: public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance.class1Key1, class1Instance)); // emit 2 values - class1Key1 and class1Instance } Then do field grouping for class1Key1 into Bolt2. On Wed, Mar 19, 2014 at 9:26 PM, Binita Bharati binita.bhar...@gmail.comwrote: Anyone has any idea on this please ? Thanks On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati binita.bhar...@gmail.com wrote: Code sample : Topology : === TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(epd-spout,new MyJmsSpout()); builder.setBolt(epd-CPY001-bolt1, new Bolt1(CPY001), parallelismHintMap.get(CPY001)) .shuffleGrouping(epd-spout); builder.setBolt(epd-CPY001-bolt2, new Bolt2(CPY001), parallelismHintMap.get(CPY001)) .fieldsGrouping(epd-CPY001-bolt1, new Fields(Bolt1Tuple)); Bolt1: public void declareOutputFields(OutputFieldsDeclarer declarer) { // TODO Auto-generated method stub declarer.declare(new Fields(Bolt1Tuple)); } public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance)); } Bolt2: == public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(Bolt1Tuple); } On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati binita.bhar...@gmail.com wrote: Hi , I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has a String field - class1Key1. I need to segregate all tuples having same value for a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2. How do I go about defining such a topology ? I have seen egs where tuples emitted were simple string values. But, my emitted tuple is a Class instance , and the field on which I want to do fieldGrouping is a particular field of that Class. (I tried defining a equals and hashcode method in a.b.c.MyClass1. Didn't work as expected..) Thanks Binita
Re: Storm - field grouping question for Class instance tuples
Anyone has any idea on this please ? Thanks On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati binita.bhar...@gmail.com wrote: Code sample : Topology : === TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(epd-spout,new MyJmsSpout()); builder.setBolt(epd-CPY001-bolt1, new Bolt1(CPY001), parallelismHintMap.get(CPY001)) .shuffleGrouping(epd-spout); builder.setBolt(epd-CPY001-bolt2, new Bolt2(CPY001), parallelismHintMap.get(CPY001)) .fieldsGrouping(epd-CPY001-bolt1, new Fields(Bolt1Tuple)); Bolt1: public void declareOutputFields(OutputFieldsDeclarer declarer) { // TODO Auto-generated method stub declarer.declare(new Fields(Bolt1Tuple)); } public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance)); } Bolt2: == public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(Bolt1Tuple); } On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati binita.bhar...@gmail.com wrote: Hi , I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has a String field - class1Key1. I need to segregate all tuples having same value for a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2. How do I go about defining such a topology ? I have seen egs where tuples emitted were simple string values. But, my emitted tuple is a Class instance , and the field on which I want to do fieldGrouping is a particular field of that Class. (I tried defining a equals and hashcode method in a.b.c.MyClass1. Didn't work as expected..) Thanks Binita
Re: Storm - field grouping question for Class instance tuples
Code sample : Topology : === TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(epd-spout,new MyJmsSpout()); builder.setBolt(epd-CPY001-bolt1, new Bolt1(CPY001), parallelismHintMap.get(CPY001)) .shuffleGrouping(epd-spout); builder.setBolt(epd-CPY001-bolt2, new Bolt2(CPY001), parallelismHintMap.get(CPY001)) .fieldsGrouping(epd-CPY001-bolt1, new Fields(Bolt1Tuple)); Bolt1: public void declareOutputFields(OutputFieldsDeclarer declarer) { // TODO Auto-generated method stub declarer.declare(new Fields(Bolt1Tuple)); } public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(JmsSpoutTuple); collector.emit(new Values(class1Instance)); } Bolt2: == public void execute(Tuple input) { Class1 class1Instance = (Class1)input.getValueByField(Bolt1Tuple); } On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati binita.bhar...@gmail.com wrote: Hi , I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has a String field - class1Key1. I need to segregate all tuples having same value for a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2. How do I go about defining such a topology ? I have seen egs where tuples emitted were simple string values. But, my emitted tuple is a Class instance , and the field on which I want to do fieldGrouping is a particular field of that Class. (I tried defining a equals and hashcode method in a.b.c.MyClass1. Didn't work as expected..) Thanks Binita