[
https://issues.apache.org/jira/browse/HIVE-29520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K updated HIVE-29520:
-------------------------------------
Description:
*[Background]*
With the increasing adoption of the Apache Iceberg table format, many
deployments no longer rely on ACID (transactional) tables provided by Apache
Hive. In such environments, ACID-related functionality in the Hive Metastore
becomes unnecessary.
Currently, Hive Metastore initializes and runs several background threads and
services that are specifically designed to support ACID operations (e.g.,
compaction handling, transaction management, cleanup tasks). These components
continue to run regardless of whether ACID tables are actually used in the
deployment.
Running ACID-related services in environments that do not use ACID tables leads
to:
* Unnecessary resource consumption (CPU, memory, threads)
* Increased operational overhead
* Additional complexity in debugging and monitoring
* Longer startup times and maintenance burden
Providing a mechanism to disable ACID functionality would streamline such
deployments and improve efficiency.
*[Proposal]*
Introduce a configuration-driven mechanism in Hive Metastore to explicitly
enable or disable ACID functionality.
When ACID support is {*}disabled{*}:
* ACID-specific background threads (e.g., compactor, cleaner, transaction
handlers) should not be initialized or started
* Transaction-related APIs and workflows can throw exception or made inactive.
When ACID support is {*}enabled{*}:
* Existing behavior should remain unchanged to ensure backward compatibility
A possible approach could be introducing a configuration property
({{{}hive.metastore.support.acid{}}}) to control this behavior at metastore
startup.
was:
With the increasing adoption of the Apache Iceberg table format, many
deployments no longer rely on ACID (transactional) tables provided by Apache
Hive. In such environments, ACID-related functionality in the Hive Metastore
becomes unnecessary.
Currently, Hive Metastore initializes and runs several background threads and
services that are specifically designed to support ACID operations (e.g.,
compaction handling, transaction management, cleanup tasks). These components
continue to run regardless of whether ACID tables are actually used in the
deployment.
Running ACID-related services in environments that do not use ACID tables leads
to:
* Unnecessary resource consumption (CPU, memory, threads)
* Increased operational overhead
* Additional complexity in debugging and monitoring
* Longer startup times and maintenance burden
Providing a mechanism to disable ACID functionality would streamline such
deployments and improve efficiency.
> Add configuration to enable/disable ACID functionality in Hive Metastore
> ------------------------------------------------------------------------
>
> Key: HIVE-29520
> URL: https://issues.apache.org/jira/browse/HIVE-29520
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
> Reporter: Venugopal Reddy K
> Assignee: Venugopal Reddy K
> Priority: Major
>
> *[Background]*
> With the increasing adoption of the Apache Iceberg table format, many
> deployments no longer rely on ACID (transactional) tables provided by Apache
> Hive. In such environments, ACID-related functionality in the Hive Metastore
> becomes unnecessary.
> Currently, Hive Metastore initializes and runs several background threads and
> services that are specifically designed to support ACID operations (e.g.,
> compaction handling, transaction management, cleanup tasks). These components
> continue to run regardless of whether ACID tables are actually used in the
> deployment.
> Running ACID-related services in environments that do not use ACID tables
> leads to:
> * Unnecessary resource consumption (CPU, memory, threads)
> * Increased operational overhead
> * Additional complexity in debugging and monitoring
> * Longer startup times and maintenance burden
> Providing a mechanism to disable ACID functionality would streamline such
> deployments and improve efficiency.
>
> *[Proposal]*
> Introduce a configuration-driven mechanism in Hive Metastore to explicitly
> enable or disable ACID functionality.
> When ACID support is {*}disabled{*}:
> * ACID-specific background threads (e.g., compactor, cleaner, transaction
> handlers) should not be initialized or started
> * Transaction-related APIs and workflows can throw exception or made
> inactive.
> When ACID support is {*}enabled{*}:
> * Existing behavior should remain unchanged to ensure backward compatibility
> A possible approach could be introducing a configuration property
> ({{{}hive.metastore.support.acid{}}}) to control this behavior at metastore
> startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
