It will be better to return an instance of a class that implements
hash() and equals() as a key instead of a String. Even for a key that
has two integers benefit over million tuples may be significant.
Looking at the source code, it looks that support for {$} notation was
removed. I did not test it, but if that is the case, it will be good to
return it back.
Thank you,
Vlad
On 10/25/17 15:43, Munagala Ramanath wrote:
The mapping: tuple -> dedup-key needs to be 1-1; if multiple tuples
are mapped to the same dedup key you'll see problems like this. In
your case multiple tuples can be mapped to the same value of "id +
id1". For example, all tuples with (id, id1) being any of these pairs
will all map to a value of 3: (0, 3), (1, 2), (2, 1), (3, 0).
A simple way to get a unique dedep key is to convert all of your dedup
fields to strings and catenate them. So in your event/tuple class,
define a getDedupKey() method and within it, compute this string and
return it. Then, you can use
the expression {$}.getDedupKey(). Something along those lines should work.
Ram
On Wednesday, October 25, 2017, 12:21:59 PM PDT, Vivek Bhide
<vivek.bh...@target.com> wrote:
Thank you everyone for reply. Solution that chinmay suggested is
working but
then I see one more discrepancy.
After adding more that 1 fields as a dedup key, my expectation was to have
the dedup decision made on combination of these 2 keys. I did run the test
case multiple times with BoundedDedupOperator and what i found is, events
are marked as Duplicate but if search for the corresponding Unique
entry in
sysout then that entry is no where to be found. Its not happening for all
but for most of entires marked as Duplicate
Is my expectation of dedup behavior is correct and is this a right way to
validate if its working as expected?
Dedup_test_case_output.txt
<http://apache-apex-users-list.78494.x6.nabble.com/file/t127/Dedup_test_case_output.txt>
Sample entries :
Present as Unique and Duplicate :
Duplicate: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:44 PDT
2017]
Unique: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:18 PDT 2017]
Only present at Duplicate :
Duplicate: TestEvent [id=23, id1=77, eventTime=Wed Oct 25 12:09:04 PDT
2017]
Duplicate: TestEvent [id=44, id1=63, eventTime=Wed Oct 25 12:09:40 PDT
2017]
--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/
