Hey guys,
I have a druid extension which supports a metric that stores image pixel
values and generate a heatmap of the clicks on each pixel at the end, and
we have a custom aggregation logic. I am using bytebuffer (direct) for all
the bit manipulations. But I am getting the below readonlybuffer exception
in the aggregation. I used ByteBuffer.allocate() and wherever possible I am
wrapping the bytebuffer and deep copying. Any pointers on how to fix this?
Two caveats:
1. I am using the real time kafka ingestion.
2. The events I am ingesting are randomly spread over last couple days(So
lot of re-indexing)!
io.druid.segment.realtime.appenderator.AppenderatorImpl - Failed to push
merged index for
segment[heatmap_events_2018-09-28T00:00:00.000Z_2018-09-28T01:00:00.000Z_2018-10-09T15:53:31.871Z_83].
java.nio.ReadOnlyBufferException
at java.nio.DirectByteBufferR.put(DirectByteBufferR.java:309)
~[?:1.8.0_181]
at
HeatmapCountMinSketch.addTwoByteValues(HeatmapCountMinSketch.java:283)
~[?:?]
at fHeatmapCountMinSketch.addSketch(HeatmapCountMinSketch.java:253)
~[?:?]
at HeatmapAggregatorFactory.combine(HeatmapAggregatorFactory.java:98)
~[?:?]
at
io.druid.segment.IndexMerger$RowboatMergeFunction.apply(IndexMerger.java:396)
~[druid-processing-0.12.3.jar:0.12.3]
at
io.druid.segment.IndexMerger$RowboatMergeFunction.apply(IndexMerger.java:365)
~[druid-processing-0.12.3.jar:0.12.3]
at io.druid.collections.CombiningIterator.next(CombiningIterator.java:81)
~[druid-common-0.12.3.jar:0.12.3]
at
io.druid.segment.IndexMergerV9.mergeIndexesAndWriteColumns(IndexMergerV9.java:456)
~[druid-processing-0.12.3.jar:0.12.3]
at io.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:209)
~[druid-processing-0.12.3.jar:0.12.3]
at io.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:837) ~[druid
-processing-0.12.3.jar:0.12.3]
at
io.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:710)
~[druid-processing-0.12.3.jar:0.12.3]
at
io.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:688)
~[druid-processing-0.12.3.jar:0.12.3]
at
io.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:659)
~[druid-server-0.12.3.jar:0.12.3]
at
io.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$0(AppenderatorImpl.java:563)
~[druid-server-0.12.3.jar:0.12.3]
at com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
[guava-16.0.1.jar:?]
at
com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
[guava-16.0.1.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_181]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
2018-10-24T01:26:10,225 ERROR [publish-0]
io.druid.indexing.kafka.KafkaIndexTask
- Error while publishing segments for
--
-spurthi
