This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6cd1efd [SPARK-27737][SQL] Upgrade to Hive 2.3.5 for Hive Metastore
Client and Hadoop-3.2 profile
6cd1efd is described below
commit 6cd1efd0ae75ca780047f16e2d88759e04a78e31
Author: Yuming Wang <yumw...@ebay.com>
AuthorDate: Wed May 22 10:24:17 2019 +0900
[SPARK-27737][SQL] Upgrade to Hive 2.3.5 for Hive Metastore Client and
Hadoop-3.2 profile
## What changes were proposed in this pull request?
This PR aims to upgrade to Hive 2.3.5 for Hive Metastore Client and
Hadoop-3.2 profile.
Release Notes - Hive - Version 2.3.5
- [[HIVE-21536](https://issues.apache.org/jira/browse/HIVE-21536)] -
Backport HIVE-17764 to branch-2.3
- [[HIVE-21585](https://issues.apache.org/jira/browse/HIVE-21585)] -
Upgrade branch-2.3 to ORC 1.3.4
- [[HIVE-21639](https://issues.apache.org/jira/browse/HIVE-21639)] - Spark
test failed since HIVE-10632
- [[HIVE-21680](https://issues.apache.org/jira/browse/HIVE-21680)] -
Backport HIVE-17644 to branch-2 and branch-2.3
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345394&styleName=Text&projectId=12310843
## How was this patch tested?
This PR is tested in two ways.
- Pass the Jenkins with the default configuration for `Hive Metastore
Client` testing.
- Pass the Jenkins with `test-hadoop3.2` configuration for `Hadoop 3.2`
testing.
Closes #24620 from wangyum/SPARK-27737.
Authored-by: Yuming Wang <yumw...@ebay.com>
Signed-off-by: HyukjinKwon <gurwls...@apache.org>
---
docs/sql-data-sources-hive-tables.md | 2 +-
docs/sql-migration-guide-hive-compatibility.md | 2 +-
pom.xml | 4 ++--
.../apache/spark/sql/execution/datasources/orc/OrcColumnVector.java | 0
.../org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala | 0
.../org/apache/spark/sql/execution/datasources/orc/OrcShimUtils.scala | 0
.../apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala | 0
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala | 2 +-
.../scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala | 2 +-
.../src/main/scala/org/apache/spark/sql/hive/client/package.scala | 2 +-
10 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/docs/sql-data-sources-hive-tables.md
b/docs/sql-data-sources-hive-tables.md
index 0c51c11..3d58e94 100644
--- a/docs/sql-data-sources-hive-tables.md
+++ b/docs/sql-data-sources-hive-tables.md
@@ -130,7 +130,7 @@ The following options can be used to configure the version
of Hive that is used
<td><code>1.2.1</code></td>
<td>
Version of the Hive metastore. Available
- options are <code>0.12.0</code> through <code>2.3.4</code> and
<code>3.1.0</code> through <code>3.1.1</code>.
+ options are <code>0.12.0</code> through <code>2.3.5</code> and
<code>3.1.0</code> through <code>3.1.1</code>.
</td>
</tr>
<tr>
diff --git a/docs/sql-migration-guide-hive-compatibility.md
b/docs/sql-migration-guide-hive-compatibility.md
index 5602f53..4a8076d 100644
--- a/docs/sql-migration-guide-hive-compatibility.md
+++ b/docs/sql-migration-guide-hive-compatibility.md
@@ -25,7 +25,7 @@ license: |
Spark SQL is designed to be compatible with the Hive Metastore, SerDes and
UDFs.
Currently, Hive SerDes and UDFs are based on Hive 1.2.1,
and Spark SQL can be connected to different versions of Hive Metastore
-(from 0.12.0 to 2.3.4 and 3.1.0 to 3.1.1. Also see [Interacting with Different
Versions of Hive
Metastore](sql-data-sources-hive-tables.html#interacting-with-different-versions-of-hive-metastore)).
+(from 0.12.0 to 2.3.5 and 3.1.0 to 3.1.1. Also see [Interacting with Different
Versions of Hive
Metastore](sql-data-sources-hive-tables.html#interacting-with-different-versions-of-hive-metastore)).
#### Deploying in Existing Hive Warehouses
diff --git a/pom.xml b/pom.xml
index 2e88e19..ff891ba 100644
--- a/pom.xml
+++ b/pom.xml
@@ -128,7 +128,7 @@
<hive.classifier></hive.classifier>
<!-- Version used in Maven Hive dependency -->
<hive.version>1.2.1.spark2</hive.version>
- <hive23.version>2.3.4</hive23.version>
+ <hive23.version>2.3.5</hive23.version>
<!-- Version used for internal directory structure -->
<hive.version.short>1.2.1</hive.version.short>
<!-- note that this should be compatible with Kafka brokers version 0.10
and up -->
@@ -2833,7 +2833,7 @@
<hive.group>org.apache.hive</hive.group>
<hive.classifier>core</hive.classifier>
<hive.version>${hive23.version}</hive.version>
- <hive.version.short>2.3.4</hive.version.short>
+ <hive.version.short>2.3.5</hive.version.short>
<!-- Do not need parquet-hadoop-bundle because we already have
parquet-common, parquet-column and parquet-hadoop -->
<hive.parquet.scope>provided</hive.parquet.scope>
diff --git
a/sql/core/v2.3.4/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java
b/sql/core/v2.3.5/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java
similarity index 100%
rename from
sql/core/v2.3.4/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java
rename to
sql/core/v2.3.5/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java
diff --git
a/sql/core/v2.3.4/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala
b/sql/core/v2.3.5/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala
similarity index 100%
rename from
sql/core/v2.3.4/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala
rename to
sql/core/v2.3.5/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala
diff --git
a/sql/core/v2.3.4/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcShimUtils.scala
b/sql/core/v2.3.5/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcShimUtils.scala
similarity index 100%
rename from
sql/core/v2.3.4/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcShimUtils.scala
rename to
sql/core/v2.3.5/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcShimUtils.scala
diff --git
a/sql/core/v2.3.4/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
b/sql/core/v2.3.5/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
similarity index 100%
rename from
sql/core/v2.3.4/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
rename to
sql/core/v2.3.5/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
index 773bf31..38ad061 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
@@ -63,7 +63,7 @@ private[spark] object HiveUtils extends Logging {
val HIVE_METASTORE_VERSION = buildConf("spark.sql.hive.metastore.version")
.doc("Version of the Hive metastore. Available options are " +
- "<code>0.12.0</code> through <code>2.3.4</code> and " +
+ "<code>0.12.0</code> through <code>2.3.5</code> and " +
"<code>3.1.0</code> through <code>3.1.1</code>.")
.stringConf
.createWithDefault(builtinHiveVersion)
diff --git
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
index 4e779da..f5268ad 100644
---
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
+++
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
@@ -101,7 +101,7 @@ private[hive] object IsolatedClientLoader extends Logging {
case "2.0" | "2.0.0" | "2.0.1" => hive.v2_0
case "2.1" | "2.1.0" | "2.1.1" => hive.v2_1
case "2.2" | "2.2.0" => hive.v2_2
- case "2.3" | "2.3.0" | "2.3.1" | "2.3.2" | "2.3.3" | "2.3.4" => hive.v2_3
+ case "2.3" | "2.3.0" | "2.3.1" | "2.3.2" | "2.3.3" | "2.3.4" | "2.3.5" =>
hive.v2_3
case "3.1" | "3.1.0" | "3.1.1" => hive.v3_1
case version =>
throw new UnsupportedOperationException(s"Unsupported Hive Metastore
version ($version). " +
diff --git
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
index 040f445..e9c13e8 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
@@ -89,7 +89,7 @@ package object client {
// Since HIVE-14496, Hive materialized view need calcite-core.
// For spark, only VersionsSuite currently creates a hive materialized
view for testing.
- case object v2_3 extends HiveVersion("2.3.4",
+ case object v2_3 extends HiveVersion("2.3.5",
exclusions = Seq("org.apache.calcite:calcite-druid",
"org.apache.calcite.avatica:avatica",
"org.apache.curator:*",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
