Here's a situation when doing so leads to an issue: when combining the aggregator with an opaque state, doing an in place update in the aggregator implies that the "previous" and "current" references point to the same java instance, so essentially the previous value is lost and we lose the robustness of the opaque approach. On Dec 22, 2013 3:58 AM, "Nathan Marz" <[email protected]> wrote:
> Actually, nevermind, those cases aren't problems since these objects are > contained within the operation of the combiner aggregator. But I still > wouldn't do it as it goes against the spirit of the API. > > > On Sat, Dec 21, 2013 at 6:56 PM, Nathan Marz <[email protected]>wrote: > >> They could potentially be re-used if the stream forks at that point or if >> there are multiple aggregators chained together. >> >> >> On Sat, Dec 21, 2013 at 4:24 PM, Adam Lewis <[email protected]> wrote: >> >>> Okay...what I've suggested isn't the cleanest seeming and I'm happy to >>> move to a less...smelly solution, but I'm unclear on how those values would >>> get reused. I'm not touching objects pointed to by the original tuple, >>> only mutating the object created in the init method of the >>> CombinerAggregator. >>> >>> Just to better my understanding of how storm/trident works, in what >>> circumstances could those values be reused? >>> >>> Thanks. >>> >>> >>> On Sat, Dec 21, 2013 at 5:49 PM, Nathan Marz <[email protected]>wrote: >>> >>>> You shouldn't do that. Those input values could potentially be re-used >>>> for other operations. I would recommend looking into using persistent data >>>> structures. >>>> >>>> >>>> On Sat, Dec 21, 2013 at 6:42 AM, Adam Lewis <[email protected]> wrote: >>>> >>>>> Hi All, >>>>> >>>>> When implementing CombinerAggregator in Trident, >>>>> is >>>>> there any issue with aggregating in >>>>> place? That is, something like this: >>>>> >>>>> >>>>> public class >>>>> MyAgg >>>>> implements CombinerAggregator< >>>>> MyMutable >>>>> > { >>>>> >>>>> >>>>> public MyMutable combine(MyMutable val1, MyMutable val2) { >>>>> >>>>> val1.mergeInPlace(val2); >>>>> return val1; >>>>> >>>>> } >>>>> >>>>> ... >>>>> >>>>> } >>>>> >>>>> In my use case the values are somewhat elaborate structurally, and >>>>> combining them in place is a lot more efficient...this seems to be working >>>>> fine in local mode, but I can't find any indication if this is a sound >>>>> practice before I commit to it. >>>>> >>>>> >>>>> Thanks, >>>>> Adam >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Twitter: @nathanmarz >>>> http://nathanmarz.com >>>> >>> >>> >> >> >> -- >> Twitter: @nathanmarz >> http://nathanmarz.com >> > > > > -- > Twitter: @nathanmarz > http://nathanmarz.com >
