[hive] branch branch-3 updated: HIVE-27623: Backport of HIVE-26081: Upgrade ant to 1.10.9 (Ashish Sharma, reviewed by Adesh Rao)
This is an automated email from the ASF dual-hosted git repository. sankarh pushed a commit to branch branch-3 in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/branch-3 by this push: new ffc32b76032 HIVE-27623: Backport of HIVE-26081: Upgrade ant to 1.10.9 (Ashish Sharma, reviewed by Adesh Rao) ffc32b76032 is described below commit ffc32b760327681d58e7107a29a82558cf9550ec Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com> AuthorDate: Mon Aug 21 10:51:01 2023 +0530 HIVE-27623: Backport of HIVE-26081: Upgrade ant to 1.10.9 (Ashish Sharma, reviewed by Adesh Rao) Signed-off-by: Sankar Hariappan Closes (#4605) --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 617ad5b1648..b66a73711a6 100644 --- a/pom.xml +++ b/pom.xml @@ -117,7 +117,7 @@ 1.7.3 -1.9.1 +1.10.9 3.5.2 1.5.7
[hive] branch branch-3 updated: HIVE-27621: Backport of HIVE-25697: Upgrade commons-compress to 1.21 (Ramesh Kumar Thangarajan reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository. sankarh pushed a commit to branch branch-3 in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/branch-3 by this push: new a1ee99ec217 HIVE-27621: Backport of HIVE-25697: Upgrade commons-compress to 1.21 (Ramesh Kumar Thangarajan reviewed by Zoltan Haindrich) a1ee99ec217 is described below commit a1ee99ec217c1969b533db7bf52ae07d7f213aa6 Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com> AuthorDate: Mon Aug 21 10:49:02 2023 +0530 HIVE-27621: Backport of HIVE-25697: Upgrade commons-compress to 1.21 (Ramesh Kumar Thangarajan reviewed by Zoltan Haindrich) Signed-off-by: Sankar Hariappan Closes (#4603) --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index cfd9260b534..617ad5b1648 100644 --- a/pom.xml +++ b/pom.xml @@ -133,7 +133,7 @@ 1.2 1.15 3.2.2 -1.19 +1.21 1.1 2.6 2.6
[hive] branch branch-3 updated: HIVE-27620: Backport of HIVE-25945: Upgrade H2 database version to 2.1.210 (Stamatis Zampetakis, reviewed by Zhihua Deng)
This is an automated email from the ASF dual-hosted git repository.
sankarh pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new 573d8763176 HIVE-27620: Backport of HIVE-25945: Upgrade H2 database
version to 2.1.210 (Stamatis Zampetakis, reviewed by Zhihua Deng)
573d8763176 is described below
commit 573d8763176aa40681e37cdd04e49e38b860c81d
Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com>
AuthorDate: Mon Aug 21 10:46:41 2023 +0530
HIVE-27620: Backport of HIVE-25945: Upgrade H2 database version to 2.1.210
(Stamatis Zampetakis, reviewed by Zhihua Deng)
Signed-off-by: Sankar Hariappan
Closes (#4600)
---
pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 76c93c0c123..cfd9260b534 100644
--- a/pom.xml
+++ b/pom.xml
@@ -147,7 +147,7 @@
1.2.0-3f79e055
19.0
2.4.11
-1.3.166
+2.1.210
3.1.0
${basedir}/${hive.path.to.root}/testutils/hadoop
1.3
[hive] branch branch-3 updated: HIVE-27614 : Backport of HIVE-21009: Adding ability for user to set bind user (#4594)
This is an automated email from the ASF dual-hosted git repository.
sankarh pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new edd0d46408f HIVE-27614 : Backport of HIVE-21009: Adding ability for
user to set bind user (#4594)
edd0d46408f is described below
commit edd0d46408f7f9226e2fad2bc55f4d3b435d69c6
Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com>
AuthorDate: Mon Aug 21 10:42:47 2023 +0530
HIVE-27614 : Backport of HIVE-21009: Adding ability for user to set bind
user (#4594)
Signed-off-by: Sankar Hariappan
Closes (#4594)
---
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 10 ++
service/pom.xml| 11 ++
.../auth/LdapAuthenticationProviderImpl.java | 32 +-
.../auth/TestLdapAuthenticationProviderImpl.java | 113 +
service/src/test/resources/creds/test.jceks| Bin 0 -> 534 bytes
5 files changed, 164 insertions(+), 2 deletions(-)
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index f9a47324473..33796a24d19 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -3386,6 +3386,16 @@ public class HiveConf extends Configuration {
"For example:
(&(objectClass=group)(objectClass=top)(instanceType=4)(cn=Domain*)) \n" +
"(&(objectClass=person)(|(sAMAccountName=admin)(|(memberOf=CN=Domain
Admins,CN=Users,DC=domain,DC=com)" +
"(memberOf=CN=Administrators,CN=Builtin,DC=domain,DC=com"),
+
HIVE_SERVER2_PLAIN_LDAP_BIND_USER("hive.server2.authentication.ldap.binddn",
null,
+"The user with which to bind to the LDAP server, and search for the
full domain name " +
+"of the user being authenticated.\n" +
+"This should be the full domain name of the user, and should have
search access across all " +
+"users in the LDAP tree.\n" +
+"If not specified, then the user being authenticated will be used as
the bind user.\n" +
+"For example: CN=bindUser,CN=Users,DC=subdomain,DC=domain,DC=com"),
+
HIVE_SERVER2_PLAIN_LDAP_BIND_PASSWORD("hive.server2.authentication.ldap.bindpw",
null,
+"The password for the bind user, to be used to search for the full
name of the user being authenticated.\n" +
+"If the username is specified, this parameter must also be
specified."),
HIVE_SERVER2_CUSTOM_AUTHENTICATION_CLASS("hive.server2.custom.authentication.class",
null,
"Custom authentication class. Used when property\n" +
"'hive.server2.authentication' is set to 'CUSTOM'. Provided class\n" +
diff --git a/service/pom.xml b/service/pom.xml
index e44d1244e52..7f93efe0c04 100644
--- a/service/pom.xml
+++ b/service/pom.xml
@@ -34,6 +34,17 @@
+
+ org.apache.hive
+ hive-common
+ ${project.version}
+
+
+ org.eclipse.jetty.aggregate
+ jetty-all
+
+
+
org.apache.hive
hive-exec
diff --git
a/service/src/java/org/apache/hive/service/auth/LdapAuthenticationProviderImpl.java
b/service/src/java/org/apache/hive/service/auth/LdapAuthenticationProviderImpl.java
index 73bbb6bdf8a..0120513b515 100644
---
a/service/src/java/org/apache/hive/service/auth/LdapAuthenticationProviderImpl.java
+++
b/service/src/java/org/apache/hive/service/auth/LdapAuthenticationProviderImpl.java
@@ -18,9 +18,10 @@
package org.apache.hive.service.auth;
import javax.security.sasl.AuthenticationException;
-
+import javax.naming.NamingException;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
+import java.io.IOException;
import java.util.Iterator;
import java.util.List;
import org.apache.commons.lang.StringUtils;
@@ -68,9 +69,36 @@ public class LdapAuthenticationProviderImpl implements
PasswdAuthenticationProvi
@Override
public void Authenticate(String user, String password) throws
AuthenticationException {
DirSearch search = null;
+String bindUser =
this.conf.getVar(HiveConf.ConfVars.HIVE_SERVER2_PLAIN_LDAP_BIND_USER);
+String bindPassword = null;
+try {
+ char[] rawPassword =
this.conf.getPassword(HiveConf.ConfVars.HIVE_SERVER2_PLAIN_LDAP_BIND_PASSWORD.toString());
+ if (rawPassword != null) {
+bindPassword = new String(rawPassword);
+ }
+} catch (IOException e) {
+ bindPassword = null;
+}
+boolean usedBind = bindUser != null && bindPassword != null;
+if (!usedBind) {
+ // If no bind user or bind password was specified,
+ // we assume the user we are authenticating has the ability to search
+ // the LDAP tree, so we use it as the "binding" account.
+ // This is the way it worked before bind
[hive] branch branch-3 updated: HIVE-27609: Backport HIVE-22115: Prevent the creation of query routing appender if property is set to false (Slim Bouguerra reviewed by Gopal V)
This is an automated email from the ASF dual-hosted git repository.
sankarh pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new d4adc0734bc HIVE-27609: Backport HIVE-22115: Prevent the creation of
query routing appender if property is set to false (Slim Bouguerra reviewed by
Gopal V)
d4adc0734bc is described below
commit d4adc0734bc336ad38d8aa689522b9b0e0ff4535
Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com>
AuthorDate: Sun Aug 20 20:44:11 2023 +0530
HIVE-27609: Backport HIVE-22115: Prevent the creation of query routing
appender if property is set to false (Slim Bouguerra reviewed by Gopal V)
Signed-off-by: Sankar Hariappan
Closes (#4588)
---
ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java | 4
1 file changed, 4 insertions(+)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java
b/ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java
index 0105fd51c66..dd25f622c73 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java
@@ -172,6 +172,10 @@ public class LogDivertAppender {
* @param conf the configuration for HiveServer2 instance
*/
public static void
registerRoutingAppender(org.apache.hadoop.conf.Configuration conf) {
+if (!HiveConf.getBoolVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED, false)) {
+ // spare some resources, do not register logger if it is not enabled .
+ return;
+}
String loggingLevel = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL);
OperationLog.LoggingLevel loggingMode =
OperationLog.getLoggingLevel(loggingLevel);
String layout = loggingMode == OperationLog.LoggingLevel.VERBOSE ?
verboseLayout : nonVerboseLayout;
[hive] branch branch-3 updated: HIVE-27608: Backport HIVE-22106: Remove cross-query synchronization for the partition-eval (Slim B via Gopal V)
This is an automated email from the ASF dual-hosted git repository.
sankarh pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new 9e350c6adfe HIVE-27608: Backport HIVE-22106: Remove cross-query
synchronization for the partition-eval (Slim B via Gopal V)
9e350c6adfe is described below
commit 9e350c6adfe718008567bde096d7de0cf3169682
Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com>
AuthorDate: Sun Aug 20 20:41:58 2023 +0530
HIVE-27608: Backport HIVE-22106: Remove cross-query synchronization for the
partition-eval (Slim B via Gopal V)
Signed-off-by: Sankar Hariappan
Closes (#4587)
---
.../org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java
b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java
index 691e9428d2c..508d207293e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java
@@ -52,7 +52,7 @@ public class PartExprEvalUtils {
* @return value returned by the expression
* @throws HiveException
*/
- static synchronized public Object evalExprWithPart(ExprNodeDesc expr,
+ static public Object evalExprWithPart(ExprNodeDesc expr,
Partition p, List vcs,
StructObjectInspector rowObjectInspector) throws HiveException {
LinkedHashMap partSpec = p.getSpec();
@@ -103,7 +103,7 @@ public class PartExprEvalUtils {
.getPrimitiveJavaObject(evaluateResultO);
}
- static synchronized public ObjectPair prepareExpr(
+ static public ObjectPair
prepareExpr(
ExprNodeGenericFuncDesc expr, List partColumnNames,
List partColumnTypeInfos) throws HiveException {
// Create the row object
@@ -120,7 +120,7 @@ public class PartExprEvalUtils {
return ObjectPair.create((PrimitiveObjectInspector)evaluateResultOI,
evaluator);
}
- static synchronized public Object evaluateExprOnPart(
+ static public Object evaluateExprOnPart(
ObjectPair pair, Object
partColValues)
throws HiveException {
return
pair.getFirst().getPrimitiveJavaObject(pair.getSecond().evaluate(partColValues));
[hive] branch branch-3 updated: HIVE-27603: Backport of HIVE-22498: Schema tool enhancements to merge catalogs (Naveen Gangam, reviewed by Sam An)
This is an automated email from the ASF dual-hosted git repository.
sankarh pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new 17991250316 HIVE-27603: Backport of HIVE-22498: Schema tool
enhancements to merge catalogs (Naveen Gangam, reviewed by Sam An)
17991250316 is described below
commit 179912503161715c281ba78d43cdbd2c2dd5e540
Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com>
AuthorDate: Sun Aug 20 20:38:58 2023 +0530
HIVE-27603: Backport of HIVE-22498: Schema tool enhancements to merge
catalogs (Naveen Gangam, reviewed by Sam An)
Signed-off-by: Sankar Hariappan
Closes (#4582)
---
.../hive/metastore/tools/MetastoreSchemaTool.java | 2 +
.../metastore/tools/SchemaToolCommandLine.java | 13 +-
.../tools/SchemaToolTaskMergeCatalog.java | 174 +
3 files changed, 188 insertions(+), 1 deletion(-)
diff --git
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/MetastoreSchemaTool.java
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/MetastoreSchemaTool.java
index c2018f42199..85f9c1f4e2a 100644
---
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/MetastoreSchemaTool.java
+++
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/MetastoreSchemaTool.java
@@ -422,6 +422,8 @@ public class MetastoreSchemaTool {
task = new SchemaToolTaskCreateCatalog();
} else if (cmdLine.hasOption("alterCatalog")) {
task = new SchemaToolTaskAlterCatalog();
+ } else if (cmdLine.hasOption("mergeCatalog")) {
+task = new SchemaToolTaskMergeCatalog();
} else if (cmdLine.hasOption("moveDatabase")) {
task = new SchemaToolTaskMoveDatabase();
} else if (cmdLine.hasOption("moveTable")) {
diff --git
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolCommandLine.java
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolCommandLine.java
index 7eba2b7a6dd..cde8b36f025 100644
---
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolCommandLine.java
+++
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolCommandLine.java
@@ -56,6 +56,11 @@ public class SchemaToolCommandLine {
.hasArg()
.withDescription("Alter a catalog, requires --catalogLocation and/or
--catalogDescription parameter as well")
.create("alterCatalog");
+Option mergeCatalog = OptionBuilder
+.hasArg()
+.withDescription("Merge databases from a catalog into other, Argument
is the source catalog name " +
+"Requires --toCatalog to indicate the destination catalog")
+.create("mergeCatalog");
Option moveDatabase = OptionBuilder
.hasArg()
.withDescription("Move a database between catalogs. Argument is the
database name. " +
@@ -81,6 +86,7 @@ public class SchemaToolCommandLine {
.addOption(validateOpt)
.addOption(createCatalog)
.addOption(alterCatalog)
+ .addOption(mergeCatalog)
.addOption(moveDatabase)
.addOption(moveTable)
.addOption(createUserOpt);
@@ -255,6 +261,11 @@ public class SchemaToolCommandLine {
printAndExit("ifNotExists may be set only for createCatalog");
}
+if (cl.hasOption("mergeCatalog") &&
+(!cl.hasOption("toCatalog"))) {
+ printAndExit("mergeCatalog and toCatalog must be set for mergeCatalog");
+}
+
if (cl.hasOption("moveDatabase") &&
(!cl.hasOption("fromCatalog") || !cl.hasOption("toCatalog"))) {
printAndExit("fromCatalog and toCatalog must be set for moveDatabase");
@@ -266,7 +277,7 @@ public class SchemaToolCommandLine {
printAndExit("fromCatalog, toCatalog, fromDatabase and toDatabase must
be set for moveTable");
}
-if ((!cl.hasOption("moveDatabase") && !cl.hasOption("moveTable")) &&
+if ((!cl.hasOption("moveDatabase") && !cl.hasOption("moveTable") &&
!cl.hasOption("mergeCatalog")) &&
(cl.hasOption("fromCatalog") || cl.hasOption("toCatalog"))) {
printAndExit("fromCatalog and toCatalog may be set only for moveDatabase
and moveTable");
}
diff --git
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolTaskMergeCatalog.java
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolTaskMergeCatalog.java
new file mode 100644
index 000..bae06d15069
--- /dev/null
+++
b/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/SchemaToolTaskMergeCatalog.java
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional
[hive] branch branch-3 updated: HIVE-27622: Backport of HIVE-26078: Upgrade gson to 2.8.9 (Ashish Sharma, reviewed by Adesh Rao) (#4604)
This is an automated email from the ASF dual-hosted git repository. dengzh pushed a commit to branch branch-3 in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/branch-3 by this push: new 6009c5809f5 HIVE-27622: Backport of HIVE-26078: Upgrade gson to 2.8.9 (Ashish Sharma, reviewed by Adesh Rao) (#4604) 6009c5809f5 is described below commit 6009c5809f5d38d1422b122e65f1cc25b31e8325 Author: Aman Raj <104416558+amanraj2...@users.noreply.github.com> AuthorDate: Sun Aug 20 16:33:49 2023 +0530 HIVE-27622: Backport of HIVE-26078: Upgrade gson to 2.8.9 (Ashish Sharma, reviewed by Adesh Rao) (#4604) --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 0b9b9209e1e..76c93c0c123 100644 --- a/pom.xml +++ b/pom.xml @@ -215,7 +215,7 @@ 2.12.0 3.0.0 0.6.0 -2.2.4 +2.8.9
