[
https://issues.apache.org/jira/browse/IMPALA-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254192#comment-17254192
]
Vihang Karajgaonkar commented on IMPALA-2513:
---------------------------------------------
Having locks at the table level has performance considerations as well.
Concurrent statements overwriting each other from different systems like Hive
and Impala is a complex problem especially making sure that performance is not
hit significantly. Recently we have added support for ACID tables which support
transactional inserts and insert overwrites. This is significantly better
performance vise. I think the general direction we would like to go here to
support such functionality on table formats which support transactions (e.g
Iceberg). For Hdfs tables we will still have to rely on making sure that the
ETL jobs running on the tables are controlled better accross systems. Another
aspect of this is even if solve this by taking lock, there is still no
guarantee that the problem will go away because a third system like Spark can
do the same without taking locks. This behavior should be controlled from a
storage level and HMS which can be shared by all the systems.
> Add Table Locking Capability
> ----------------------------
>
> Key: IMPALA-2513
> URL: https://issues.apache.org/jira/browse/IMPALA-2513
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Affects Versions: Impala 2.2.4
> Reporter: Jacob Davis
> Priority: Major
> Labels: catalog-server
>
> When running multiple concurrent INSERT OVERWRITE statements on a table, the
> result is that only the last to be executed is done successfully. Running
> the same process via Hive will result in each statement running sequentially
> due to the locking behavior.
> The request is to add similar locking functionality within Impala. Ideally,
> it would work across with Hive so that queries executed from Hive and Impala
> would both honor the locks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
