[
https://issues.apache.org/jira/browse/PHOENIX-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjeet Malhotra updated PHOENIX-7231:
--------------------------------------
Description:
Steps to reproduce:
# Create a table.
# Upsert one row into it.
# Change column encoding scheme to transform table.
# Run transform tool and make sure cutover is successful.
# Upsert one more row to original Phoenix table which now points to new
physical table.
# Run delete from <table name>
Following error is thrown:
{{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException:
org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to
build index for unexpected reason!}}
{{ at
org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}}
{{ at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}}
{{ at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}}
{{ at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}}
{{ at
org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}}
{{ at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
{{ at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
{{ at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}}
{{ at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}}
{{ at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}}
{{ at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}}
{{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}}
{{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}}
{{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}}
{{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}}
{{Caused by: java.lang.NullPointerException}}
{{ at
org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}}
{{ ... 29 more}}
{{Debugging analysis:}}
{{Seems like its related to instance variable inheritance and visibility scope
of instance variables in TransformMaintainer class. As per error, its seems
like TransformMaintainer#buildRowDeleteMutation}} was called.
{{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}} and
is trying to access to access private instance variable:
{{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the
parent and child classes and is private in both so,
{{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while
TransformMaintainer.emptyKeyValueCFPtr is not non-null.}}
This Jira aims to fix following:
* Fix delete from table for transformed tables.
* Fix TransformTool related tests which are currently broken and not running
on master pipeline as Category annotations are missing.
* Handle physical and logical table name mismatch in general like for table
level max lookback age.
was:
Steps to reproduce:
# Create a table.
# Upsert one row into it.
# Change column encoding scheme to transform table.
# Run transform tool and make sure cutover is successful.
# Upsert one more row to original Phoenix table which now points to new
physical table.
# Run delete from <table name>
Following error is thrown:
{{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException:
org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to
build index for unexpected reason!}}
{{ at
org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}}
{{ at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}}
{{ at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}}
{{ at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}}
{{ at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}}
{{ at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}}
{{ at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}}
{{ at
org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}}
{{ at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
{{ at
org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
{{ at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}}
{{ at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}}
{{ at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}}
{{ at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}}
{{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}}
{{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}}
{{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}}
{{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}}
{{Caused by: java.lang.NullPointerException}}
{{ at
org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}}
{{ at
org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}}
{{ ... 29 more}}
{{Debugging analysis:}}
{{Seems like its related to instance variable inheritance and visibility scope
of instance variables in TransformMaintainer class. As per error, its seems
like {{TransformMaintainer#buildRowDeleteMutation}} was called.
{{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}} and
is trying to access to access private instance variable:
{{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the
parent and child classes and is private in both so,
}}{{{}{{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while
TransformMaintainer.emptyKeyValueCFPtr is not non-null.}}{}}}{{{}{}}}
{{{{Additionally, TransformIT tests need to fixed as they have test cases for
this scenario but are broken currently and not running in master branch
pipeline due to missing Category annotations.}}}}
> Delete from table for transformed table is failing
> --------------------------------------------------
>
> Key: PHOENIX-7231
> URL: https://issues.apache.org/jira/browse/PHOENIX-7231
> Project: Phoenix
> Issue Type: Bug
> Reporter: Sanjeet Malhotra
> Assignee: Sanjeet Malhotra
> Priority: Major
> Fix For: 5.2.0
>
>
> Steps to reproduce:
> # Create a table.
> # Upsert one row into it.
> # Change column encoding scheme to transform table.
> # Run transform tool and make sure cutover is successful.
> # Upsert one more row to original Phoenix table which now points to new
> physical table.
> # Run delete from <table name>
> Following error is thrown:
> {{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException:
> org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed
> to build index for unexpected reason!}}
> {{ at
> org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}}
> {{ at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}}
> {{ at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}}
> {{ at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}}
> {{ at
> org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}}
> {{ at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}}
> {{ at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}}
> {{ at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}}
> {{ at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}}
> {{ at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}}
> {{ at
> org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}}
> {{ at
> org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
> {{ at
> org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}}
> {{ at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}}
> {{ at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}}
> {{ at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}}
> {{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}}
> {{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}}
> {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}}
> {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}}
> {{Caused by: java.lang.NullPointerException}}
> {{ at
> org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}}
> {{ at
> org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}}
> {{ at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}}
> {{ at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}}
> {{ at
> org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}}
> {{ ... 29 more}}
> {{Debugging analysis:}}
> {{Seems like its related to instance variable inheritance and visibility
> scope of instance variables in TransformMaintainer class. As per error, its
> seems like TransformMaintainer#buildRowDeleteMutation}} was called.
> {{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}}
> and is trying to access to access private instance variable:
> {{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the
> parent and child classes and is private in both so,
> {{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while
> TransformMaintainer.emptyKeyValueCFPtr is not non-null.}}
>
> This Jira aims to fix following:
> * Fix delete from table for transformed tables.
> * Fix TransformTool related tests which are currently broken and not running
> on master pipeline as Category annotations are missing.
> * Handle physical and logical table name mismatch in general like for table
> level max lookback age.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
