[
https://issues.apache.org/jira/browse/PHOENIX-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani reassigned PHOENIX-8053:
-------------------------------------
Assignee: Viraj Jasani
> Concurrency bugs related to conditional UPSERTs on same row
> -----------------------------------------------------------
>
> Key: PHOENIX-8053
> URL: https://issues.apache.org/jira/browse/PHOENIX-8053
> Project: Phoenix
> Issue Type: Bug
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
>
> Concurrent conditional UPSERT on the same row can fail, return uncommitted
> data, or lose updates.
> While one update on the data table is writing to its indexes, the update
> releases the row lock. A later concurrent update on the same row builds on
> the earlier update's pending state in memory (not from disk). However, this
> tracking of the pending updates can cause issues.
> Some of the issues as of today:
> * *Non-retryable error after a no-op:* A conditional upsert (ON DUPLICATE
> KEY) whose condition is false changes nothing and needs to return early, but
> it stays recorded as the row's latest pending update. The next concurrent
> UPSERT on the row finds it as completed and fails with a non-retryable
> {*}IllegalArgumentException: the phase of the last batch cannot be POST{*}.
> * *No-op result based on uncommitted data:* A conditional upsert whose
> condition is false returns to the client right away. Its result, including
> the returned row, can be based on the earlier update's pending state in
> memory. If that update fails, the client was provided result based on
> uncommitted data.
> * *Lost update after a failed update:* The row's latest pending update fails
> while an earlier update is still pending. The next update reads the row from
> disk (latest update failed) and misses the earlier pending write (which has
> not completed yet), and then overwrites it once both commit.
> * *Pending tracking removed too early:* Some updates never register as
> pending on the row: whole-row upserts on an immutable table with a global or
> uncovered index, or on a mutable table whose only indexes are uncovered. When
> such an update completes, it still removes the row's pending entry, even the
> one that a later update created. The next update reads from disk and misses a
> write that is still in progress.
> Also, the batches chained on a busy row (the pending ones and the ones
> already done) stay in memory as long as the row has updates in flight.
> Sample Exception for first case:
>
> {code:java}
> Caused by: org.apache.phoenix.execute.CommitException:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException:
> Failed to build index for unexpected reason! at
> org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:218)
> at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:706)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:1002)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:999)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:999)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3859)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4577)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4515)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4437)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1036)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp(RSRpcServices.java:954)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2897)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45039)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:416) at
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) at
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) at
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)Caused by:
> java.lang.IllegalArgumentException: the phase of the last batch cannot be
> POST at
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:141)
> at
> org.apache.phoenix.hbase.index.IndexRegionObserver.getCurrentRowStates(IndexRegionObserver.java:1264)
> at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:2029)
> at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:703)
> ... 18 more
> {code}
>
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
