[ANNOUNCE] Apache Hive 4.0.0 Released

[Denys Kuzmenko]

The Apache Hive team is proud to announce the release of Apache Hive
version 4.0.0.

The Apache Hive (TM) data warehouse software facilitates querying and
managing large datasets residing in distributed storage. Built on top
of Apache Hadoop (TM), it provides, among others:

* Tools to enable easy data extract/transform/load (ETL)

* A mechanism to impose structure on a variety of data formats

* Access to files stored either directly in Apache HDFS (TM) or in other
  data storage systems such as Apache HBase (TM)

* Query execution via Apache Hadoop MapReduce, Apache Tez and Apache
Spark frameworks. (MapReduce is deprecated, and Spark has been removed
so the text needs to be modified depending on the release version)

For Hive release details and downloads, please
visit:https://hive.apache.org/downloads.html

Hive 4.0.0 Release Notes are available here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12343343=Text=12310843

We would like to thank the many contributors who made this release
possible.

Regards,

The Apache Hive Team

Re: Re: Hive's performance for querying the Iceberg table is very poor.

[Denys Kuzmenko]

`hive.iceberg.stats.source` controls where the stats should be sourced from. 
When it's set to iceberg (default), we should go directly to iceberg and bypass 
HMS. 

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

[Denys Kuzmenko]

DbTxnManager doesn't have support for explicit locking. 
You could use HMSClient to issue an explicit lock request or create a lock 
record manually in the backend DB `HIVE_LOCKS` table.
Maybe you could temporarily restrict access to those tables via Ranger or drop 
them from the HMS while performing maintenance?

Re: [ANNOUNCE] New committer for Apache Hive: Alessandro Solimando

[Denys Kuzmenko]

Congrats, Alessandro! Very well deserved.

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

[Denys Kuzmenko]

Unfortunately no. DummyTxnManager doesn't support DB locks, only Zookeeper and 
in-memory. So you won't be able to transparently lock the external table from 
other non-DummyTxtManager's.   

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

[Denys Kuzmenko]

DbTxnManager comes with DbLockManager implementation that doesn't support 
explicit lock/unlock functionality. 
It was designed to be used only for ACID tables, however, due to code refactor 
challenges is reused for external tables as well. This comes with few drawbacks 
like suboptimal behavior (opens a txn for an ext table) and reduced 
functionality.
That's something we plan to address in a future. For now you can configure a 
dedicated HS2 instance with DummyTxnManager to have explicit locking 
functionality for ext tables.  

Re: Odp: Re: hive 4.0.0-alpha2 external table locks

[Denys Kuzmenko]

If you are using external tables only, you could try switching to 
DummyTxnManager.  It supports explicit lock requests.

[ANNOUNCE] Apache Hive 4.0.0-alpha-2 Released

[Denys Kuzmenko]

The Apache Hive team is proud to announce the release of Apache Hive
version 4.0.0-alpha-2

The Apache Hive (TM) data warehouse software facilitates querying and
managing large datasets residing in distributed storage. Built on top
of Apache Hadoop (TM), it provides, among others:

* Tools to enable easy data extract/transform/load (ETL)

* A mechanism to impose structure on a variety of data formats

* Access to files stored either directly in Apache HDFS (TM) or in other
  data storage systems such as Apache HBase (TM)

* Query execution via Apache Hadoop MapReduce, Apache Tez and Apache
Spark frameworks.

For Hive release details and downloads, please
visit:https://hive.apache.org/downloads.html

Hive 4.0.0-alpha-2 Release Notes are available
here:https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351489=Html=12310843

We would like to thank the many contributors who made this release
possible.

Regards,

The Apache Hive Team