[
https://issues.apache.org/jira/browse/HIVE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825297#comment-17825297
]
Rui Li commented on HIVE-26882:
-------------------------------
bq. What do you see as an issue with that?
The issue is each alter table operation updates more than just the metadata
location. For example, when we change iceberg table schema, JDO will update
both the iceberg metadata location, and the HMS storage descriptor. If we use
direct SQL, then either we follow JDO to generate all the SQL statements, or we
allow storage descriptor to be out of sync with iceberg metadata.
bq. The API only allows a single checked property, would it be enough to check
the change of that?
Not sure I understand the question. You can execute multiple update statements
in the transaction and check the affected rows for each of them. In our PoC, we
update current and previous metadata location, and leave all other fields out
of sync.
bq. Would READ COMMITTED serialization level enough for this solution?
I haven't tried that, but seems it will work.
bq. Is this a general solution which would work on all of the supported
databases?
I only verified it for MariaDB. Not sure about other databases. But I think it
works as long as the number of affected rows can be decided reliably.
I ran similar test with MS SQL Server 2017 [docker
image|https://hub.docker.com/_/microsoft-mssql-server], and same as Postgres,
it throws exception for concurrent writes at REPEATABLE_READ. I didn't find a
working docker image for Oracle.
> Allow transactional check of Table parameter before altering the Table
> ----------------------------------------------------------------------
>
> Key: HIVE-26882
> URL: https://issues.apache.org/jira/browse/HIVE-26882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10, 4.0.0-beta-1
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> We should add the possibility to transactionally check if a Table parameter
> is changed before altering the table in the HMS.
> This would provide an alternative, less error-prone and faster way to commit
> an Iceberg table, as the Iceberg table currently needs to:
> - Create an exclusive lock
> - Get the table metadata to check if the current snapshot is not changed
> - Update the table metadata
> - Release the lock
> After the change these 4 HMS calls could be substituted with a single alter
> table call.
> Also we could avoid cases where the locks are left hanging by failed processes
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
