[ https://issues.apache.org/jira/browse/PHOENIX-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjeet Malhotra updated PHOENIX-7231: -------------------------------------- Description: Steps to reproduce: # Create a table. # Upsert one row into it. # Change column encoding scheme to transform table. # Run transform tool and make sure cutover is successful. # Upsert one more row to original Phoenix table which now points to new physical table. # Run delete from <table name> Following error is thrown: {{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to build index for unexpected reason!}} {{ at org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}} {{ at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}} {{ at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}} {{ at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}} {{ at org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}} {{ at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} {{ at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} {{ at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}} {{ at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}} {{ at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}} {{ at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}} {{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}} {{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}} {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}} {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}} {{Caused by: java.lang.NullPointerException}} {{ at org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}} {{ ... 29 more}} {{Debugging analysis:}} {{Seems like its related to instance variable inheritance and visibility scope of instance variables in TransformMaintainer class. As per error, its seems like TransformMaintainer#buildRowDeleteMutation}} was called. {{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}} and is trying to access to access private instance variable: {{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the parent and child classes and is private in both so, {{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while TransformMaintainer.emptyKeyValueCFPtr is not non-null.}} This Jira aims to fix following: * Fix delete from table for transformed tables. * Fix TransformTool related tests which are currently broken and not running on master pipeline as Category annotations are missing. * Handle physical and logical table name mismatch in general like for table level max lookback age. was: Steps to reproduce: # Create a table. # Upsert one row into it. # Change column encoding scheme to transform table. # Run transform tool and make sure cutover is successful. # Upsert one more row to original Phoenix table which now points to new physical table. # Run delete from <table name> Following error is thrown: {{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to build index for unexpected reason!}} {{ at org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}} {{ at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}} {{ at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}} {{ at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}} {{ at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}} {{ at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}} {{ at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}} {{ at org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}} {{ at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} {{ at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} {{ at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}} {{ at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}} {{ at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}} {{ at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}} {{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}} {{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}} {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}} {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}} {{Caused by: java.lang.NullPointerException}} {{ at org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}} {{ at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}} {{ ... 29 more}} {{Debugging analysis:}} {{Seems like its related to instance variable inheritance and visibility scope of instance variables in TransformMaintainer class. As per error, its seems like {{TransformMaintainer#buildRowDeleteMutation}} was called. {{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}} and is trying to access to access private instance variable: {{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the parent and child classes and is private in both so, }}{{{}{{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while TransformMaintainer.emptyKeyValueCFPtr is not non-null.}}{}}}{{{}{}}} {{{{Additionally, TransformIT tests need to fixed as they have test cases for this scenario but are broken currently and not running in master branch pipeline due to missing Category annotations.}}}} > Delete from table for transformed table is failing > -------------------------------------------------- > > Key: PHOENIX-7231 > URL: https://issues.apache.org/jira/browse/PHOENIX-7231 > Project: Phoenix > Issue Type: Bug > Reporter: Sanjeet Malhotra > Assignee: Sanjeet Malhotra > Priority: Major > Fix For: 5.2.0 > > > Steps to reproduce: > # Create a table. > # Upsert one row into it. > # Change column encoding scheme to transform table. > # Run transform tool and make sure cutover is successful. > # Upsert one more row to original Phoenix table which now points to new > physical table. > # Run delete from <table name> > Following error is thrown: > {{org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: > org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed > to build index for unexpected reason!}} > {{ at > org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)}} > {{ at > org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)}} > {{ at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)}} > {{ at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)}} > {{ at > org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)}} > {{ at > org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)}} > {{ at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)}} > {{ at > org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)}} > {{ at > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)}} > {{ at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)}} > {{ at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)}} > {{ at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)}} > {{ at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)}} > {{ at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)}} > {{ at > org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)}} > {{ at > org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} > {{ at > org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)}} > {{ at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)}} > {{ at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)}} > {{ at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)}} > {{ at > org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)}} > {{ at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)}} > {{ at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)}} > {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)}} > {{ at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)}} > {{Caused by: java.lang.NullPointerException}} > {{ at > org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)}} > {{ at > org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)}} > {{ at > org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)}} > {{ at > org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)}} > {{ at > org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)}} > {{ ... 29 more}} > {{Debugging analysis:}} > {{Seems like its related to instance variable inheritance and visibility > scope of instance variables in TransformMaintainer class. As per error, its > seems like TransformMaintainer#buildRowDeleteMutation}} was called. > {{buildRowDeleteMutation}} is defined in parent class: {{IndexMaintainer}} > and is trying to access to access private instance variable: > {{{}emptyKeyValueCFPtr{}}}. As the instance variable is declared in both the > parent and child classes and is private in both so, > {{IndexMaintainer.emptyKeyValueCFPtr gets used which is null while > TransformMaintainer.emptyKeyValueCFPtr is not non-null.}} > > This Jira aims to fix following: > * Fix delete from table for transformed tables. > * Fix TransformTool related tests which are currently broken and not running > on master pipeline as Category annotations are missing. > * Handle physical and logical table name mismatch in general like for table > level max lookback age. -- This message was sent by Atlassian Jira (v8.20.10#820010)