Re: how to set timestamp field in trident window?
Strange problem, I can't receive reply of my question to email list. I read
ansewer from strom website. Thanks for
Stig Rohde Døssing. That's very helpful.
-- Original --
From: "ChenBo";
Date: Fri, Nov 23, 2018 11:51 AM
To: "user";
Subject: how to set timestamp field in trident window?
Hi All, I use storm-core 1.2.2, and there is my code:
spoutStream.window(TumblingDurationWindow.of(new BaseWindowedBolt.Duration(10,
TimeUnit.SECONDS)),
new Fields("value"), new CountAsAggregator(), new
Fields("record"));
I can't find any API to specify window timestamp field.
Can anyone tell me how to set timestamp field in trident TumblingDurationWindow?
Any help is appreciated. Thanks,
ChenBo
Re: how to set timestamp field in trident window?
Also you should be aware of
https://issues.apache.org/jira/browse/STORM-3280 if you intend to use
Trident based windowing.
Den fre. 23. nov. 2018 kl. 12.10 skrev Stig Rohde Døssing <
stigdoess...@gmail.com>:
> I don't believe you can. The Trident windowing code seems to use
> System.currentTimeMillis without allowing you to set a timestamp extractor.
> If you want to add this functionality, you should look at
> https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/trident/windowing/InMemoryTridentWindowManager.java#L53,
> and likely also the other implementation of ITridentWindowManager, and make
> them use the windowManager.add(event, ts) method instead of
> windowManager.add(event). You can consider using
> https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/topology/WindowedBoltExecutor.java#L310
> for reference to how it works in regular Storm topologies.
>
> Note that there doesn't seem to be a watermark event generator for the
> Trident code either, so you may want to look at adding that as well.
>
> Den fre. 23. nov. 2018 kl. 04.51 skrev 731635762 :
>
>> Hi All, I use storm-core 1.2.2, and there is my code:
>>
>> spoutStream.window(TumblingDurationWindow.of(new
>> BaseWindowedBolt.Duration(10, TimeUnit.SECONDS)),
>> new Fields("value"), new CountAsAggregator(), new
>> Fields("record"));
>>
>> I can't find any API to specify window timestamp field.
>>
>> Can anyone tell me how to set timestamp field in trident
>> TumblingDurationWindow?
>>
>>
>> Any help is appreciated.
>>
>> Thanks,
>> ChenBo
>>
>>
Re: how to set timestamp field in trident window?
I don't believe you can. The Trident windowing code seems to use
System.currentTimeMillis without allowing you to set a timestamp extractor.
If you want to add this functionality, you should look at
https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/trident/windowing/InMemoryTridentWindowManager.java#L53,
and likely also the other implementation of ITridentWindowManager, and make
them use the windowManager.add(event, ts) method instead of
windowManager.add(event). You can consider using
https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/topology/WindowedBoltExecutor.java#L310
for reference to how it works in regular Storm topologies.
Note that there doesn't seem to be a watermark event generator for the
Trident code either, so you may want to look at adding that as well.
Den fre. 23. nov. 2018 kl. 04.51 skrev 731635762 :
> Hi All, I use storm-core 1.2.2, and there is my code:
>
> spoutStream.window(TumblingDurationWindow.of(new
> BaseWindowedBolt.Duration(10, TimeUnit.SECONDS)),
> new Fields("value"), new CountAsAggregator(), new
> Fields("record"));
>
> I can't find any API to specify window timestamp field.
>
> Can anyone tell me how to set timestamp field in trident
> TumblingDurationWindow?
>
>
> Any help is appreciated.
>
> Thanks,
> ChenBo
>
>
