Thank you everyone for reply. Solution that chinmay suggested is working but
then I see one more discrepancy. 

After adding more that 1 fields as a dedup key, my expectation was to have
the dedup decision made on combination of these 2 keys. I did run the test
case multiple times with BoundedDedupOperator and what i found is, events
are marked as Duplicate but if search for the corresponding Unique entry in
sysout then that entry is no where to be found. Its not happening for all
but for most of entires marked as Duplicate

Is my expectation of dedup behavior is correct and is this a right way to
validate if its working as expected?
Dedup_test_case_output.txt
<http://apache-apex-users-list.78494.x6.nabble.com/file/t127/Dedup_test_case_output.txt>
  
Sample entries :

Present as Unique and Duplicate : 
Duplicate: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:44 PDT 2017]
Unique: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:18 PDT 2017]

Only present at Duplicate : 
Duplicate: TestEvent [id=23, id1=77, eventTime=Wed Oct 25 12:09:04 PDT 2017]
Duplicate: TestEvent [id=44, id1=63, eventTime=Wed Oct 25 12:09:40 PDT 2017]




--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/

Reply via email to