[
https://issues.apache.org/jira/browse/PHOENIX-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated PHOENIX-7950:
-----------------------------------------
Description:
{{ALTER TABLE <mt_table> SET IMMUTABLE_STORAGE_SCHEME =
SINGLE_CELL_ARRAY_WITH_OFFSETS, COLUMN_ENCODED_BYTES = 2}} on a MULTI_TENANT
immutable base table succeeds, but the expected {{SystemTransformRecord}} is
never found afterwards.
Altering a multi-tenant base table can trigger a stale metadata re-resolution
that re-executes the entire {{TransformClient.addTransform}} path twice, and
the transform record UPSERT performed inside {{addTransform}} is left
uncommitted in the client {{MutationState}} and is discarded by the retry
before the statement's final commit.
{noformat}
17:45:25,330 TransformClient(290): Creating transforming table via CREATE
TABLE N000006_1(...) <- addTransform attempt #1
17:45:25,331 FromCompiler$BaseColumnResolver(839): Re-resolved stale table
N000006 ... 3 columns <- creating N000006_1 expired N000006 in the cache
17:45:25,337 TransformClient(290): Creating transforming table via CREATE
TABLE N000006_1(...) <- addTransform attempt #2 (whole statement retried)
17:45:25,339 ... CreateTableProcedure table=N000006_1 ...
<- new physical table actually created
17:45:29,626 TransformClient(386): Adding transform type: ...
logicalTableName: N000006, status: CREATED <- upsertTransform() executes
17:45:29,631 task.Task(132): Adding task type: TRANSFORM_MONITOR , tableName:
N000006 <- monitor task (separate server-side connection)
17:45:29,646 MetaDataEndpointImpl(4029): Column(s) added successfully,
tableName: N000006 <- base-table ALTER committed
17:45:29,653 TransformClient(125): Could not find System.Transform record ...
LOGICAL_TABLE_NAME ='N000006' <- test lookup finds nothing
{noformat}
Proposed fix:
Make the transform record write durable by committing immediately after
{{upsertTransform}} inside {{addTransform}}, mirroring the auto-commit already
performed by the preceding {{CREATE TABLE}}. Must verify idempotency or fix the
double execution. (The new physical table is currently created twice.)
OR
Prevent the multi-tenant ALTER from re-resolving/re-executing the transform
creation (make {{addTransform}} a no-op when an active record for the same
logical table already exists during the same statement).
was:
{{ALTER TABLE <mt_table> SET IMMUTABLE_STORAGE_SCHEME =
SINGLE_CELL_ARRAY_WITH_OFFSETS, COLUMN_ENCODED_BYTES = 2}} on a MULTI_TENANT
immutable base table succeeds, but the expected {{SystemTransformRecord}} is
never found afterwards.
Altering a multi-tenant base table can trigger a stale metadata re-resolution
that re-executes the entire {{TransformClient.addTransform}} path twice, and
the transform record UPSERT performed inside {{addTransform}} is left
uncommitted in the client {{MutationState}} and is discarded by the retry
before the statement's final commit.
{noformat}
17:45:25,330 TransformClient(290): Creating transforming table via CREATE
TABLE N000006_1(...) <- addTransform attempt #1
17:45:25,331 FromCompiler$BaseColumnResolver(839): Re-resolved stale table
N000006 ... 3 columns <- creating N000006_1 expired N000006 in the cache
17:45:25,337 TransformClient(290): Creating transforming table via CREATE
TABLE N000006_1(...) <- addTransform attempt #2 (whole statement retried)
17:45:25,339 ... CreateTableProcedure table=N000006_1 ...
<- new physical table actually created
17:45:29,626 TransformClient(386): Adding transform type: ...
logicalTableName: N000006, status: CREATED <- upsertTransform() executes
17:45:29,631 task.Task(132): Adding task type: TRANSFORM_MONITOR , tableName:
N000006 <- monitor task (separate server-side connection)
17:45:29,646 MetaDataEndpointImpl(4029): Column(s) added successfully,
tableName: N000006 <- base-table ALTER committed
17:45:29,653 TransformClient(125): Could not find System.Transform record ...
LOGICAL_TABLE_NAME ='N000006' <- test lookup finds nothing
{noformat}
Proposed fix:
Make the transform record write durable by committing immediately after
{{upsertTransform}} inside {{addTransform}}, mirroring the auto-commit already
performed by the preceding {{CREATE TABLE}}. Must verify idempotency or fix the
double execution. (The new physical table is currently created twice.)
OR
Prevent the multi-tenant ALTER from re-resolving/re-executing the transform
creation (make `addTransform` a no-op when an active record for the same
logical table already exists during the same statement).
> Multi-tenant base table transform does not persist its SYSTEM.TRANSFORM record
> ------------------------------------------------------------------------------
>
> Key: PHOENIX-7950
> URL: https://issues.apache.org/jira/browse/PHOENIX-7950
> Project: Phoenix
> Issue Type: Bug
> Components: core
> Reporter: Andrew Kyle Purtell
> Assignee: Andrew Kyle Purtell
> Priority: Major
> Fix For: 5.4.0, 5.3.2
>
>
> {{ALTER TABLE <mt_table> SET IMMUTABLE_STORAGE_SCHEME =
> SINGLE_CELL_ARRAY_WITH_OFFSETS, COLUMN_ENCODED_BYTES = 2}} on a MULTI_TENANT
> immutable base table succeeds, but the expected {{SystemTransformRecord}} is
> never found afterwards.
> Altering a multi-tenant base table can trigger a stale metadata re-resolution
> that re-executes the entire {{TransformClient.addTransform}} path twice, and
> the transform record UPSERT performed inside {{addTransform}} is left
> uncommitted in the client {{MutationState}} and is discarded by the retry
> before the statement's final commit.
> {noformat}
> 17:45:25,330 TransformClient(290): Creating transforming table via CREATE
> TABLE N000006_1(...) <- addTransform attempt #1
> 17:45:25,331 FromCompiler$BaseColumnResolver(839): Re-resolved stale table
> N000006 ... 3 columns <- creating N000006_1 expired N000006 in the cache
> 17:45:25,337 TransformClient(290): Creating transforming table via CREATE
> TABLE N000006_1(...) <- addTransform attempt #2 (whole statement retried)
> 17:45:25,339 ... CreateTableProcedure table=N000006_1 ...
> <- new physical table actually created
> 17:45:29,626 TransformClient(386): Adding transform type: ...
> logicalTableName: N000006, status: CREATED <- upsertTransform() executes
> 17:45:29,631 task.Task(132): Adding task type: TRANSFORM_MONITOR ,
> tableName: N000006 <- monitor task (separate
> server-side connection)
> 17:45:29,646 MetaDataEndpointImpl(4029): Column(s) added successfully,
> tableName: N000006 <- base-table ALTER committed
> 17:45:29,653 TransformClient(125): Could not find System.Transform record
> ... LOGICAL_TABLE_NAME ='N000006' <- test lookup finds nothing
> {noformat}
> Proposed fix:
> Make the transform record write durable by committing immediately after
> {{upsertTransform}} inside {{addTransform}}, mirroring the auto-commit
> already performed by the preceding {{CREATE TABLE}}. Must verify idempotency
> or fix the double execution. (The new physical table is currently created
> twice.)
> OR
> Prevent the multi-tenant ALTER from re-resolving/re-executing the transform
> creation (make {{addTransform}} a no-op when an active record for the same
> logical table already exists during the same statement).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
