[
https://issues.apache.org/jira/browse/PHOENIX-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985223#comment-15985223
]
Samarth Jain commented on PHOENIX-3806:
---------------------------------------
After discussing this yesterday I realized that we cannot rely on the max
versions attribute that is set on HTableDescriptor. We *need* to look at all
the versions of all the rows that have been changed since the index was
disabled. This is because for index maintenance we need to issue delete markers
for those index rows where the index column was updated in the corresponding
data table row. This actually exposes a hole in our mutable index
implementation. If the max versions on a table is set to a lower number (like
1) and if the major compaction runs before index is rebuild, we will end up
losing these mutations on the data table rows. This will make the index
inconsistent.
> IndexUpdateManager spending a lot of time sorting mutations on Index rebuild
> ----------------------------------------------------------------------------
>
> Key: PHOENIX-3806
> URL: https://issues.apache.org/jira/browse/PHOENIX-3806
> Project: Phoenix
> Issue Type: Bug
> Reporter: Lars Hofhansl
>
> Here's the stack trace. The Array contains 50001 Delete Mutations in this
> case.
> It seems the code is sorting this over and over again.
> {code}
> Thread 170 (B.DefaultRpcServer.handler=67,queue=7,port=60020):
> State: RUNNABLE
> Blocked count: 220598
> Waited count: 377933
> Stack:
> java.util.TimSort.binarySort(TimSort.java:296)
> java.util.TimSort.sort(TimSort.java:239)
> java.util.Arrays.sort(Arrays.java:1438)
>
> org.apache.phoenix.hbase.index.covered.update.SortedCollection.iterator(SortedCollection.java:78)
>
> org.apache.phoenix.hbase.index.covered.update.IndexUpdateManager.fixUpCurrentUpdates(IndexUpdateManager.java:128)
>
> org.apache.phoenix.hbase.index.covered.update.IndexUpdateManager.addIndexUpdate(IndexUpdateManager.java:115)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addCurrentStateMutationsForBatch(NonTxIndexBuilder.java:333)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addUpdateForGivenTimestamp(NonTxIndexBuilder.java:258)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addMutationsForBatch(NonTxIndexBuilder.java:231)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.batchMutationAndAddUpdates(NonTxIndexBuilder.java:109)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.getIndexUpdate(NonTxIndexBuilder.java:71)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:137)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:133)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
>
> com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:61)
>
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:58)
>
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager.getIndexUpdate(IndexBuildManager.java:144)
>
> org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:324)
> Thread 169 (B.DefaultRpcServer.handler=66,queue=6,port=60020):
> {code}
> [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
