Hi,
I am using the BoundedDedupOperator and with default value of numBuckets
(46340) the container is failing with below bucket conflict exception
2017-06-08 17:52:10,140 INFO stram.StreamingContainerParent
(StreamingContainerParent.java:log(170)) - child msg: Stopped running due to
an exception. java.lang.IllegalArgumentException: bucket conflict -27877
27877
at
org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.handleBucketConflict(AbstractManagedStateImpl.java:320)
at
org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.prepareBucket(AbstractManagedStateImpl.java:269)
at
org.apache.apex.malhar.lib.state.managed.AbstractManagedStateImpl.getValueFromBucketAsync(AbstractManagedStateImpl.java:305)
at
org.apache.apex.malhar.lib.state.managed.ManagedTimeStateImpl.getAsync(ManagedTimeStateImpl.java:76)
at
org.apache.apex.malhar.lib.dedup.BoundedDedupOperator.getAsyncManagedState(BoundedDedupOperator.java:151)
at
org.apache.apex.malhar.lib.dedup.AbstractDeduper.processTuple(AbstractDeduper.java:163)
at
org.apache.apex.malhar.lib.dedup.BoundedDedupOperator$1.process(BoundedDedupOperator.java:91)
at com.datatorrent.api.DefaultInputPort.put(DefaultInputPort.java:79)
at
com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.sweep(BufferServerSubscriber.java:288)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:269)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1428)
I tried changing the numBucket value to something else but it still fails
with same error. I was planning to use it with just 1 bucket but I think
that will slow the operator too much since the unique keys in our
applications are very high
Has anybody else encountered the same problem?
Regards
Vivek
--
View this message in context:
http://apache-apex-users-list.78494.x6.nabble.com/BoundedDedupOperator-failing-with-java-lang-IllegalArgumentException-bucket-conflict-tp1698.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
