Hi Balázs, The idea sounds good.
Only whitelisted configs can be overridden at component level. The java configs are converted to clojure variables by replacing _ (underscore) with - (hyphen) and you need to add and entry like [1] in executor.clj to get it working. - Arun [1] https://github.com/apache/storm/blob/master/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L114 On 5/30/16, 11:26 PM, "Balázs Kossovics" <balazs.kossov...@s4m.io> wrote: >Hey, > >I'm trying to implement an alterative behaviour in case of late tuples >for windowing. Currently late tuples are just logged (and acknowledged >in the coming 1.0.2), but in my usecase it would be desirable to emit >them onto a user defined stream. One could define a bolt with a stream >for late tuples like this: > >new MyWindowedBolt() > .withTimestampField("timestamp") >* .withLateTupleStream("late_tuples") > .withWindow( > new BaseWindowedBolt.Duration(1, TimeUnit.MINUTES), > new BaseWindowedBolt.Duration(1, TimeUnit.SECONDS) > ) > >I made a quick patch >(https://github.com/kosii/storm/commit/216c991da3c5b6c6cac1b25182b86507c3fb5e9e) > >to test the idea, where the indended behaviour would be something like this: >1, the withLateTupleStream builder method puts a new key into the >windowConfiguration Map, >2, in WindowedBoltExecutor. declareOutputFields a new stream gets declared, >3, in WindowedBoltExecutor.prepare we store the stream's name in a >private variable, >4, if this variable is not null, then in the execute method we emit each >late tuple onto the new stream. > >The problem is in the 3rd step >(https://github.com/kosii/storm/commit/216c991da3c5b6c6cac1b25182b86507c3fb5e9e#diff-32eff130ad25f4b7b6c069e8d42245acR170), > >where the stormConf map doesn't contain anymore my key, which was still >present in the 2nd step. I'm out of ideas, so I'd really appreciate if >someone could explain me what's happening here. > >I'm also interested in your ideas concerning the feature, and what would >you like to eventually see in the upstream. > >Best regards, >Balazs > >
