This is an automated email from the ASF dual-hosted git repository.
zabetak pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hive-site.git
The following commit(s) were added to refs/heads/main by this push:
new 07b3afd HIVE-29246: Migrate from EmbeddedDriver to AutoloadedDriver
since Derby 10.17.1.0 (#67)
07b3afd is described below
commit 07b3afd1f4aad11ff07565459ce9c49c64b09ded
Author: Raghav Aggarwal <[email protected]>
AuthorDate: Wed Oct 15 15:43:48 2025 +0530
HIVE-29246: Migrate from EmbeddedDriver to AutoloadedDriver since Derby
10.17.1.0 (#67)
1. Unify copy-of-hive-schema-tool.md with hive-schema-tool.md
2. Updated the schematool output to show 4.2.0 output in hive-schema-tool.md
3. Delete copy-of-hive-schema-tool.md
---
content/Development/desingdocs/statsdev.md | 3 +-
.../docs/latest/admin/adminmanual-configuration.md | 1 -
.../docs/latest/admin/copy-of-hive-schema-tool.md | 226 ------------------
content/docs/latest/admin/hive-schema-tool.md | 255 ++++++++++++++-------
.../docs/latest/user/configuration-properties.md | 9 +-
5 files changed, 171 insertions(+), 323 deletions(-)
diff --git a/content/Development/desingdocs/statsdev.md
b/content/Development/desingdocs/statsdev.md
index d628200..017d11b 100644
--- a/content/Development/desingdocs/statsdev.md
+++ b/content/Development/desingdocs/statsdev.md
@@ -175,12 +175,11 @@ set hive.stats.dbclass=hbase;
```
-In case of JDBC implementations of temporary stored statistics (ex. Derby or
MySQL), the user should specify the appropriate connection string to the
database by setting the variable **[hive.stats.dbconnectionstring]({{< ref
"#hive-stats-dbconnectionstring" >}})**. Also the user should specify the
appropriate JDBC driver by setting the variable **[hive.stats.jdbcdriver]({{<
ref "#hive-stats-jdbcdriver" >}})**.
+In case of JDBC implementations of temporary stored statistics (ex. Derby or
MySQL), the user should specify the appropriate connection string to the
database by setting the variable **[hive.stats.dbconnectionstring]({{< ref
"#hive-stats-dbconnectionstring" >}})**.
```
set hive.stats.dbclass=jdbc:derby;
set
hive.stats.dbconnectionstring="jdbc:derby:;databaseName=TempStatsStore;create=true";
-set hive.stats.jdbcdriver="org.apache.derby.jdbc.EmbeddedDriver";
```
diff --git a/content/docs/latest/admin/adminmanual-configuration.md
b/content/docs/latest/admin/adminmanual-configuration.md
index cfd4491..9d412f9 100644
--- a/content/docs/latest/admin/adminmanual-configuration.md
+++ b/content/docs/latest/admin/adminmanual-configuration.md
@@ -126,7 +126,6 @@ Version information: Metrics
| hive.stats.autogather | A flag to gather statistics automatically during the
INSERT OVERWRITE command. (As of Hive
[0.7.0](https://issues.apache.org/jira/browse/HIVE-1361).) | true |
| hive.stats.dbclass | The default database that stores temporary hive
statistics. Valid values are `hbase` and `jdbc` while `jdbc` should have a
specification of the Database to use, separated by a colon (e.g. `jdbc:mysql`).
(As of Hive [0.7.0](https://issues.apache.org/jira/browse/HIVE-1361).) |
jdbc:derby |
| hive.stats.dbconnectionstring | The default connection string for the
database that stores temporary hive statistics. (As of Hive
[0.7.0](https://issues.apache.org/jira/browse/HIVE-1361).) |
jdbc:derby:;databaseName=TempStatsStore;create=true |
-| hive.stats.jdbcdriver | The JDBC driver for the database that stores
temporary hive statistics. (As of Hive
[0.7.0](https://issues.apache.org/jira/browse/HIVE-1361).) |
org.apache.derby.jdbc.EmbeddedDriver |
| hive.stats.reliable | Whether queries will fail because stats cannot be
collected completely accurately. If this is set to true, reading/writing
from/into a partition may fail becuase the stats could not be computed
accurately. (As of Hive
[0.10.0](https://issues.apache.org/jira/browse/HIVE-1653).) | false |
| hive.enforce.bucketing | If enabled, enforces inserts into bucketed tables
to also be bucketed. (Hive 0.6.0 through Hive
[1.x.x](https://issues.apache.org/jira/browse/HIVE-12331) only) | false |
| hive.variable.substitute | Substitutes variables in Hive statements which
were previously set using the `set` command, system variables or environment
variables. See [HIVE-1096](https://issues.apache.org/jira/browse/HIVE-1096) for
details. (As of Hive 0.7.0.) | true |
diff --git a/content/docs/latest/admin/copy-of-hive-schema-tool.md
b/content/docs/latest/admin/copy-of-hive-schema-tool.md
deleted file mode 100644
index 6d5c4b7..0000000
--- a/content/docs/latest/admin/copy-of-hive-schema-tool.md
+++ /dev/null
@@ -1,226 +0,0 @@
----
-title: "Apache Hive : Copy of Hive Schema Tool - [TODO: move it under a 4.0
admin manual page, find a proper name]"
-date: 2024-12-12
----
-
-# Apache Hive : Copy of Hive Schema Tool - [TODO: move it under a 4.0 admin
manual page, find a proper name]
-
-{{< toc >}}
-
-## About
-
-Schema tool helps to initialise and upgrade metastore database and hive sys
schema.
-
-## Metastore Schema Verification
-
-Hive records the schema version in the metastore database and verifies that
the metastore schema version is compatible with Hive binaries that are going to
access the metastore. Note that the Hive properties to implicitly create or
alter the existing schema are disabled by default. Hive will not attempt to
change the metastore schema implicitly. When you execute a Hive query against
an old schema, it will fail to access the metastore:
-
-```
-$ build/dist/bin/hive -e "show tables"
-FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to
instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
-
-```
-
-The log will contain an error about version information not found:
-
-```
-...
-Caused by: MetaException(message:Version information not found in metastore. )
-...
-
-```
-
-By default the configuration property **metastore.schema.verification** is
true. If we set it false, metastore implicitly writes the schema version if
it's not matching. To disable the strict schema verification, you need to set
this property to false in `hive-site.xml`.
-
-See [Hive Metastore Administration]({{< ref
"adminmanual-metastore-administration" >}}) for general information about the
metastore.
-
-## The Hive Schema Tool
-
-The Hive distribution now includes an offline tool for Hive metastore schema
manipulation. This tool can be used to initialize the metastore schema for the
current Hive version. It can also handle upgrading the schema from an older
version to current. It tries to find the current schema from the metastore if
it is available. This will be applicable to future upgrades like 0.12.0 to
0.13.0. In case of upgrades from older releases like 0.7.0 or 0.10.0, you can
specify the schema version of [...]
-
-The `schematool` figures out the SQL scripts required to initialize or upgrade
the schema and then executes those scripts against the backend database. The
metastore DB connection information like JDBC URL, JDBC driver and DB
credentials are extracted from the Hive configuration. You can provide
alternate DB credentials if needed.
-
-**Warning**: You should always initialize the metastore schema first and the
Hive schema second. The Hive schema initialization checks the metastore
database schema and if it is not initialized, it puts some objects into the
metastore database as well to make sure hive schema is running. That also means
if you want to run the metastore schema initialization after the hive one, it
will fail as it finds a database that already contains some objects.
-
-### The `schematool` Command
-
-The `schematool` command invokes the Hive schema tool with these options:
-
-```
-$ usage: schemaTool
- -alterCatalog <arg> Alter a catalog, requires
- --catalogLocation and/or
- --catalogDescription parameter as well
- -catalogDescription <arg> Description of new catalog
- -catalogLocation <arg> Location of new catalog, required when
- adding a catalog
- -createCatalog <arg> Create a catalog, requires
- --catalogLocation parameter as well
- -createLogsTable <arg> Create table for Hive
- warehouse/compute logs
- -createUser Create the Hive user, set hiveUser to
- the db admin user and the hive
- password to the db admin password with
- this
- -dbOpts <databaseOpts> Backend DB specific options
- -dbType <databaseType> Metastore database type
- -driver <driver> driver name for connection
- -dropAllDatabases Drop all Hive databases (with
- CASCADE). This will remove all managed
- data!
- -dryRun list SQL scripts (no execute)
- -fromCatalog <arg> Catalog a moving database or table is
- coming from. This is required if you
- are moving a database or table.
- -fromDatabase <arg> Database a moving table is coming
- from. This is required if you are
- moving a table.
- -help print this message
- -hiveDb <arg> Hive database (for use with
- createUser)
- -hivePassword <arg> Hive password (for use with
- createUser)
- -hiveUser <arg> Hive user (for use with createUser)
- -ifNotExists If passed then it is not an error to
- create an existing catalog
- -info Show config and schema details
- -initOrUpgradeSchema Initialize or upgrade schema to latest
- version
- -initSchema Schema initialization
- -initSchemaTo <initTo> Schema initialization to a version
- -mergeCatalog <arg> Merge databases from a catalog into
- other, Argument is the source catalog
- name Requires --toCatalog to indicate
- the destination catalog
- -metaDbType <metaDatabaseType> Used only if upgrading the system
- catalog for hive
- -moveDatabase <arg> Move a database between catalogs.
- Argument is the database name.
- Requires --fromCatalog and --toCatalog
- parameters as well
- -moveTable <arg> Move a table to a different database.
- Argument is the table name. Requires
- --fromCatalog, --toCatalog,
- --fromDatabase, and --toDatabase
- parameters as well.
- -passWord <password> Override config file password
- -retentionPeriod <arg> Specify logs table retention period
- -servers <serverList> a comma-separated list of servers used
- in location validation in the format
- of scheme://authority (e.g.
- hdfs://localhost:8000)
- -toCatalog <arg> Catalog a moving database or table is
- going to. This is required if you are
- moving a database or table.
- -toDatabase <arg> Database a moving table is going to.
- This is required if you are moving a
- table.
- -upgradeSchema Schema upgrade
- -upgradeSchemaFrom <upgradeFrom> Schema upgrade from a version
- -url <url> connection url to the database
- -userName <user> Override config file user name
- -validate Validate the database
- -verbose only print SQL statements
- -yes Don't ask for confirmation when using
- -dropAllDatabases.
-```
-
-The **dbType** is required and can be one of:
-
-```
- derby|mysql|postgres|oracle|mssql|hive
-```
-
-Note: dbType=hive only can be used on Hive sys schema. The others are
metastore db types and in case of dbType=hive, it is mandatory to set
metaDbType as well.
-
-### Usage Examples
-
-* Initialize to current schema for a new Hive setup:
-
-```
-$ schematool -dbType derby -initSchema Initializing the schema to: 4.0.0-beta-2
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting metastore schema initialization to 4.0.0-beta-2
-Initialization script hive-schema-4.0.0-beta-2.derby.sql
-Initialization script completed
-```
-* Get schema information:
-
-```
-$ schematool -dbType derby -info
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Hive distribution version: 4.0.0-beta-2
-Metastore schema version: 4.0.0-beta-2
-```
-* Init schema to for a given version:
-
-```
-$ schematool -dbType derby -initSchemaTo 3.1.0
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting metastore schema initialization to 3.1.0
-Initialization script hive-schema-3.1.0.derby.sql
-```
-* Upgrade schema from an 3.1.0 release by specifying the 'from' version:
-
-```
-$ schematool -dbType derby -upgradeSchemaFrom 3.1.0 Upgrading from the user
input version 3.1.0
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting upgrade metastore schema from version 3.1.0 to 4.0.0-beta-2
-Upgrade script upgrade-3.1.0-to-3.2.0.derby.sql
-Completed upgrade-3.1.0-to-3.2.0.derby.sql
-...
-Completed upgrade-4.0.0-beta-1-to-4.0.0-beta-2.derby.sql
-```
-* Upgrade dry run can be used to list the required scripts for the given
upgrade.
-
-```
-$ schematool -dbType derby -upgradeSchemaFrom 3.1.0 -dryRun Upgrading from the
user input version 3.1.0
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting upgrade metastore schema from version 3.1.0 to 4.0.0-beta-2
-Upgrade script upgrade-3.1.0-to-3.2.0.derby.sql
-Upgrade script upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql
-Upgrade script upgrade-4.0.0-alpha-1-to-4.0.0-alpha-2.derby.sql
-Upgrade script upgrade-4.0.0-alpha-2-to-4.0.0-beta-1.derby.sql
-Upgrade script upgrade-4.0.0-beta-1-to-4.0.0-beta-2.derby.sql
-```
-
-This is useful if you just want to find out all the required scripts for the
schema upgrade.
-* Initialise Hive sys schema
-
-```
-$ ./schematool -dbType hive -metaDbType derby -initSchema --verbose
-url="jdbc:hive2://localhost:10000"
-```
-
-Note 1: As
-* Moving a database and tables under it from default Hive catalog to a custom
spark catalog
-
-```
-build/dist/bin/schematool -moveDatabase db1 -fromCatalog hive -toCatalog spark
-
-```
-* Moving a table from Hive catalog to Spark Catalog
-
-```
-# Create the desired target database in spark catalog if it doesn't already
exist.
-beeline ... -e "create database if not exists newdb";
-schematool -moveDatabase newdb -fromCatalog hive -toCatalog spark
-
-# Now move the table to target db under the spark catalog.
-schematool -moveTable table1 -fromCatalog hive -toCatalog spark -fromDatabase
db1 -toDatabase newdb
-
-```
-
-
-
-
-
diff --git a/content/docs/latest/admin/hive-schema-tool.md
b/content/docs/latest/admin/hive-schema-tool.md
index 27f51d0..06b41b0 100644
--- a/content/docs/latest/admin/hive-schema-tool.md
+++ b/content/docs/latest/admin/hive-schema-tool.md
@@ -1,24 +1,26 @@
---
+
title: "Apache Hive : Hive Schema Tool"
-date: 2024-12-12
+date: 2025-10-14
---
# Apache Hive : Hive Schema Tool
{{< toc >}}
-## Metastore Schema Verification
+## About
-Version
+Schema tool helps to initialise and upgrade metastore database and hive sys
schema.
+
+## Metastore Schema Verification
Introduced in Hive 0.12.0. See
[HIVE-3764](https://issues.apache.org/jira/browse/HIVE-3764).
-Hive now records the schema version in the metastore database and verifies
that the metastore schema version is compatible with Hive binaries that are
going to accesss the metastore. Note that the Hive properties to implicitly
create or alter the existing schema are disabled by default. Hive will not
attempt to change the metastore schema implicitly. When you execute a Hive
query against an old schema, it will fail to access the metastore:
+Hive records the schema version in the metastore database and verifies that
the metastore schema version is compatible with Hive binaries that are going to
access the metastore. Note that the Hive properties to implicitly create or
alter the existing schema are disabled by default. Hive will not attempt to
change the metastore schema implicitly. When you execute a Hive query against
an old schema, it will fail to access the metastore:
```
$ build/dist/bin/hive -e "show tables"
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to
instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
-
```
The log will contain an error about version information not found:
@@ -27,150 +29,236 @@ The log will contain an error about version information
not found:
...
Caused by: MetaException(message:Version information not found in metastore. )
...
-
```
-By default the configuration property **hive.metastore.schema.verification**
is false and metastore to implicitly write the schema version if it's not
matching. To enable the strict schema verification, you need to set this
property to true in `hive-site.xml`.
+By default the configuration property **metastore.schema.verification** is
true. If we set it false, metastore implicitly writes the schema version if
it's not matching. To disable the strict schema verification, you need to set
this property to false in `hive-site.xml`.
See [Hive Metastore Administration]({{< ref
"adminmanual-metastore-administration" >}}) for general information about the
metastore.
## The Hive Schema Tool
-Version
-
Introduced in Hive 0.12.0. See
[HIVE-5301](https://issues.apache.org/jira/browse/HIVE-5301). (Also see
[HIVE-5449](https://issues.apache.org/jira/browse/HIVE-5449) for a bug fix.)
The Hive distribution now includes an offline tool for Hive metastore schema
manipulation. This tool can be used to initialize the metastore schema for the
current Hive version. It can also handle upgrading the schema from an older
version to current. It tries to find the current schema from the metastore if
it is available. This will be applicable to future upgrades like 0.12.0 to
0.13.0. In case of upgrades from older releases like 0.7.0 or 0.10.0, you can
specify the schema version of [...]
The `schematool` figures out the SQL scripts required to initialize or upgrade
the schema and then executes those scripts against the backend database. The
metastore DB connection information like JDBC URL, JDBC driver and DB
credentials are extracted from the Hive configuration. You can provide
alternate DB credentials if needed.
+**Warning**: You should always initialize the metastore schema first and the
Hive schema second. The Hive schema initialization checks the metastore
database schema and if it is not initialized, it puts some objects into the
metastore database as well to make sure hive schema is running. That also means
if you want to run the metastore schema initialization after the hive one, it
will fail as it finds a database that already contains some objects.
+
### The `schematool` Command
The `schematool` command invokes the Hive schema tool with these options:
```
-$ schematool -help
usage: schemaTool
+ -alterCatalog <arg> Alter a catalog, requires
+ --catalogLocation and/or
+ --catalogDescription parameter as well
+ -catalogDescription <arg> Description of new catalog
+ -catalogLocation <arg> Location of new catalog, required when
+ adding a catalog
+ -createCatalog <arg> Create a catalog, requires
+ --catalogLocation parameter as well
+ -createLogsTable <arg> Create table for Hive
+ warehouse/compute logs
+ -createUser Create the Hive user, set hiveUser to
+ the db admin user and the hive
+ password to the db admin password with
+ this
+ -dbOpts <databaseOpts> Backend DB specific options
-dbType <databaseType> Metastore database type
- -driver <driver> Driver name for connection
- -dryRun List SQL scripts (no execute)
- -help Print this message
+ -driver <driver> driver name for connection
+ -dropAllDatabases Drop all Hive databases (with
+ CASCADE). This will remove all managed
+ data!
+ -dryRun list SQL scripts (no execute)
+ -fromCatalog <arg> Catalog a moving database or table is
+ coming from. This is required if you
+ are moving a database or table.
+ -fromDatabase <arg> Database a moving table is coming
+ from. This is required if you are
+ moving a table.
+ -help print this message
+ -hiveDb <arg> Hive database (for use with
+ createUser)
+ -hivePassword <arg> Hive password (for use with
+ createUser)
+ -hiveUser <arg> Hive user (for use with createUser)
+ -ifNotExists If passed then it is not an error to
+ create an existing catalog
-info Show config and schema details
+ -initOrUpgradeSchema Initialize or upgrade schema to latest
+ version
-initSchema Schema initialization
-initSchemaTo <initTo> Schema initialization to a version
- -metaDbType <metaDatabaseType> Used only if upgrading the system catalog
for hive
+ -mergeCatalog <arg> Merge databases from a catalog into
+ other, Argument is the source catalog
+ name Requires --toCatalog to indicate
+ the destination catalog
+ -metaDbType <metaDatabaseType> Used only if upgrading the system
+ catalog for hive
+ -moveDatabase <arg> Move a database between catalogs.
+ Argument is the database name.
+ Requires --fromCatalog and --toCatalog
+ parameters as well
+ -moveTable <arg> Move a table to a different database.
+ Argument is the table name. Requires
+ --fromCatalog, --toCatalog,
+ --fromDatabase, and --toDatabase
+ parameters as well.
-passWord <password> Override config file password
+ -retentionPeriod <arg> Specify logs table retention period
+ -servers <serverList> a comma-separated list of servers used
+ in location validation in the format
+ of scheme://authority (e.g.
+ hdfs://localhost:8000)
+ -toCatalog <arg> Catalog a moving database or table is
+ going to. This is required if you are
+ moving a database or table.
+ -toDatabase <arg> Database a moving table is going to.
+ This is required if you are moving a
+ table.
-upgradeSchema Schema upgrade
-upgradeSchemaFrom <upgradeFrom> Schema upgrade from a version
- -url <url> Connection url to the database
+ -url <url> connection url to the database
-userName <user> Override config file user name
- -verbose Only print SQL statements
-(Additional catalog related options added in Hive 3.0.0 (HIVE-19135] release
are below.
- -createCatalog <catalog> Create catalog with given name
- -catalogLocation <location> Location of new catalog, required when
adding a catalog
- -catalogDescription <description> Description of new catalog
- -ifNotExists If passed then it is not an error to
create an existing catalog
- -moveDatabase <database> Move a database between
catalogs. All tables under it would still be under it as part of new catalog.
Argument is the database name. Requires --fromCatalog and --toCatalog
parameters as well
- -moveTable <table> Move a table to a different database.
Argument is the table name. Requires --fromCatalog, --toCatalog,
--fromDatabase, and --toDatabase
- -toCatalog <catalog> Catalog a moving database or table is
going to. This is required if you are moving a database or table.
- -fromCatalog <catalog> Catalog a moving database or table is
coming from. This is required if you are moving a database or table.
- -toDatabase <database> Database a moving table is going to. This
is required if you are moving a table.
- -fromDatabase <database> Database a moving table is coming from.
This is required if you are moving a table.
-
+ -validate Validate the database
+ -verbose only print SQL statements
+ -yes Don't ask for confirmation when using
+ -dropAllDatabases.
```
The **dbType** is required and can be one of:
```
- derby|mysql|postgres|oracle|mssql
+derby|mysql|postgres|oracle|mssql|hive
```
-Version
-
The dbType "`mssql`" was added in Hive 0.13.1 with
[HIVE-6862](https://issues.apache.org/jira/browse/HIVE-6862).
+Note: dbType=hive only can be used on Hive sys schema. The others are
metastore db types and in case of dbType=hive, it is mandatory to set
metaDbType as well i.e.
+
+```
+schematool -dbType derby -initSchema
+schematool -dbType hive -initSchema -metaDbType derby
+```
+
### Usage Examples
* Initialize to current schema for a new Hive setup:
```
$ schematool -dbType derby -initSchema
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting metastore schema initialization to 0.13.0
-Initialization script hive-schema-0.13.0.derby.sql
-Initialization script completed
-schemaTool completed
+Initializing the schema to: 4.2.0
+2025-10-14 16:21:20,427 INFO schematool.HiveSchemaHelper: Metastore connection
URL: jdbc:derby:metastore_db;create=true
+Metastore connection URL: jdbc:derby:metastore_db;create=true
+2025-10-14 16:21:20,427 INFO schematool.HiveSchemaHelper: Metastore connection
Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+Metastore connection Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+2025-10-14 16:21:20,427 INFO schematool.HiveSchemaHelper: Metastore connection
User: APP
+Metastore connection User: APP
+Starting metastore schema initialization to 4.2.0
+Initialization script hive-schema-4.2.0.derby.sql
+...
+Initialization script completed
```
+
* Get schema information:
```
$ schematool -dbType derby -info
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Hive distribution version: 0.13.0
-Metastore schema version: 0.13.0
-schemaTool completed
+2025-10-14 16:29:13,116 INFO schematool.HiveSchemaHelper: Metastore connection
URL: jdbc:derby:metastore_db;create=true
+Metastore connection URL: jdbc:derby:metastore_db;create=true
+2025-10-14 16:29:13,116 INFO schematool.HiveSchemaHelper: Metastore connection
Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+Metastore connection Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+2025-10-14 16:29:13,116 INFO schematool.HiveSchemaHelper: Metastore connection
User: APP
+Metastore connection User: APP
+Hive distribution version: 4.2.0
+Metastore schema version: 4.2.0
```
-* Attempt to get schema information with older metastore:
-```
-$ schematool -dbType derby -info
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Hive distribution version: 0.13.0
-org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema
version.
-*** schemaTool failed ***
+* Init schema to for a given version:
+```
+$ schematool -dbType derby -initSchemaTo 3.1.0
+
+2025-10-14 16:37:11,089 INFO schematool.HiveSchemaHelper: Metastore connection
URL: jdbc:derby:metastore_db;create=true
+Metastore connection URL: jdbc:derby:metastore_db;create=true
+2025-10-14 16:37:11,089 INFO schematool.HiveSchemaHelper: Metastore connection
Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+Metastore connection Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+2025-10-14 16:37:11,089 INFO schematool.HiveSchemaHelper: Metastore connection
User: APP
+Metastore connection User: APP
+Starting metastore schema initialization to 3.1.0
+Initialization script hive-schema-3.1.0.derby.sql
+...
+Initialization script completed
```
-Since the older metastore doesn't store the version information, the tool
reports an error retrieving it.
-* Upgrade schema from an 0.10.0 release by specifying the 'from' version:
+* Upgrade schema from an 3.1.0 release by specifying the 'from' version:
```
-$ schematool -dbType derby -upgradeSchemaFrom 0.10.0
-Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
-Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting upgrade metastore schema from version 0.10.0 to 0.13.0
-Upgrade script upgrade-0.10.0-to-0.11.0.derby.sql
-Completed upgrade-0.10.0-to-0.11.0.derby.sql
-Upgrade script upgrade-0.11.0-to-0.12.0.derby.sql
-Completed upgrade-0.11.0-to-0.12.0.derby.sql
-Upgrade script upgrade-0.12.0-to-0.13.0.derby.sql
-Completed upgrade-0.12.0-to-0.13.0.derby.sql
-schemaTool completed
-
+$ schematool -dbType derby -upgradeSchemaFrom 3.1.0
+
+Upgrading from the user input version 3.1.0
+2025-10-14 16:41:04,018 INFO schematool.HiveSchemaHelper: Metastore connection
URL: jdbc:derby:metastore_db;create=true
+Metastore connection URL: jdbc:derby:metastore_db;create=true
+2025-10-14 16:41:04,018 INFO schematool.HiveSchemaHelper: Metastore connection
Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+Metastore connection Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+2025-10-14 16:41:04,018 INFO schematool.HiveSchemaHelper: Metastore connection
User: APP
+Metastore connection User: APP
+Starting upgrade metastore schema from version 3.1.0 to 4.2.0
+Upgrade script upgrade-3.1.0-to-3.2.0.derby.sql
+2025-10-14 16:41:04,142 INFO conf.HiveConf: Found configuration file
file:/Users/raghav/Desktop/setup/hive/conf/hive-site.xml
+2025-10-14 16:41:04,143 INFO conf.HiveConf: Found configuration file null
+2025-10-14 16:41:04,143 INFO conf.HiveConf: Found configuration file null
+2025-10-14 16:41:04,143 INFO conf.HiveConf: Found configuration file null
+2025-10-14 16:41:04,173 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
+Completed upgrade-3.1.0-to-3.2.0.derby.sql
+Upgrade script upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql
+Completed upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql
+Upgrade script upgrade-4.0.0-alpha-1-to-4.0.0-alpha-2.derby.sql
+Completed upgrade-4.0.0-alpha-1-to-4.0.0-alpha-2.derby.sql
+Upgrade script upgrade-4.0.0-alpha-2-to-4.0.0-beta-1.derby.sql
+Completed upgrade-4.0.0-alpha-2-to-4.0.0-beta-1.derby.sql
+Upgrade script upgrade-4.0.0-beta-1-to-4.0.0.derby.sql
+Completed upgrade-4.0.0-beta-1-to-4.0.0.derby.sql
+Upgrade script upgrade-4.0.0-to-4.1.0.derby.sql
+Completed upgrade-4.0.0-to-4.1.0.derby.sql
+Upgrade script upgrade-4.1.0-to-4.2.0.derby.sql
+Completed upgrade-4.1.0-to-4.2.0.derby.sql
```
+
* Upgrade dry run can be used to list the required scripts for the given
upgrade.
```
-$ build/dist/bin/schematool -dbType derby -upgradeSchemaFrom 0.7.0 -dryRun
-Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
-Metastore connection User: APP
-Starting upgrade metastore schema from version 0.7.0 to 0.13.0
-Upgrade script upgrade-0.7.0-to-0.8.0.derby.sql
-Upgrade script upgrade-0.8.0-to-0.9.0.derby.sql
-Upgrade script upgrade-0.9.0-to-0.10.0.derby.sql
-Upgrade script upgrade-0.10.0-to-0.11.0.derby.sql
-Upgrade script upgrade-0.11.0-to-0.12.0.derby.sql
-Upgrade script upgrade-0.12.0-to-0.13.0.derby.sql
-schemaTool completed
-
+$ schematool -dbType derby -upgradeSchemaFrom 3.1.0 -dryRun
+
+Upgrading from the user input version 3.1.0
+2025-10-14 16:43:16,211 INFO schematool.HiveSchemaHelper: Metastore connection
URL: jdbc:derby:metastore_db;create=true
+Metastore connection URL: jdbc:derby:metastore_db;create=true
+2025-10-14 16:43:16,211 INFO schematool.HiveSchemaHelper: Metastore connection
Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+Metastore connection Driver : org.apache.derby.iapi.jdbc.AutoloadedDriver
+2025-10-14 16:43:16,211 INFO schematool.HiveSchemaHelper: Metastore connection
User: APP
+Metastore connection User: APP
+Starting upgrade metastore schema from version 3.1.0 to 4.2.0
+Upgrade script upgrade-3.1.0-to-3.2.0.derby.sql
+Upgrade script upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql
+Upgrade script upgrade-4.0.0-alpha-1-to-4.0.0-alpha-2.derby.sql
+Upgrade script upgrade-4.0.0-alpha-2-to-4.0.0-beta-1.derby.sql
+Upgrade script upgrade-4.0.0-beta-1-to-4.0.0.derby.sql
+Upgrade script upgrade-4.0.0-to-4.1.0.derby.sql
+Upgrade script upgrade-4.1.0-to-4.2.0.derby.sql
```
This is useful if you just want to find out all the required scripts for the
schema upgrade.
+
* Moving a database and tables under it from default Hive catalog to a custom
spark catalog
```
build/dist/bin/schematool -moveDatabase db1 -fromCatalog hive -toCatalog spark
-
```
+
* Moving a table from Hive catalog to Spark Catalog
```
@@ -180,10 +268,5 @@ schematool -moveDatabase newdb -fromCatalog hive
-toCatalog spark
# Now move the table to target db under the spark catalog.
schematool -moveTable table1 -fromCatalog hive -toCatalog spark -fromDatabase
db1 -toDatabase newdb
-
```
-
-
-
-
diff --git a/content/docs/latest/user/configuration-properties.md
b/content/docs/latest/user/configuration-properties.md
index 0e364d1..aeaf410 100644
--- a/content/docs/latest/user/configuration-properties.md
+++ b/content/docs/latest/user/configuration-properties.md
@@ -1848,7 +1848,7 @@ JDBC connect string for a JDBC metastore.
##### javax.jdo.option.ConnectionDriverName
-* Default Value: `org.apache.derby.jdbc.EmbeddedDriver`
+* Default Value: `org.apache.derby.iapi.jdbc.AutoloadedDriver`
* Added In: Hive 0.8.1
Driver class name for a JDBC metastore.
@@ -4174,13 +4174,6 @@ Statistics are not gathered for `LOAD DATA` statements.
Extends statistics autogathering to also collect column level statistics.
-##### hive.stats.jdbcdriver
-
-* Default Value: `org.apache.derby.jdbc.EmbeddedDriver`
-* Added In: Hive 0.7 with
[HIVE-1361](https://issues.apache.org/jira/browse/HIVE-1361)
-
-The JDBC driver for the database that stores temporary Hive statistics.
-
##### hive.stats.dbconnectionstring
* Default Value: `jdbc:derby:;databaseName=TempStatsStore;create=true`
