spark git commit: [SPARK-9491] Avoid fetching HBase tokens when not needed.
Repository: spark
Updated Branches:
refs/heads/master 6688ba6e6 - df733cbea
[SPARK-9491] Avoid fetching HBase tokens when not needed.
Look at HBase's configuration to make sure it's configured for
Kerberos. If the HBase configuration is missing, or if HBase is
configured for non-kerberos authentication, then skip getting
tokens.
Reference: http://hbase.apache.org/book.html#security.prerequisites
Author: Marcelo Vanzin van...@cloudera.com
Closes #7810 from vanzin/SPARK-9491 and squashes the following commits:
a57c776 [Marcelo Vanzin] [SPARK-9491] Avoid fetching HBase tokens when not
needed.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/df733cbe
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/df733cbe
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/df733cbe
Branch: refs/heads/master
Commit: df733cbeae7a53826e89574af5463fa018329a22
Parents: 6688ba6
Author: Marcelo Vanzin van...@cloudera.com
Authored: Sat Aug 1 13:06:50 2015 -0700
Committer: Marcelo Vanzin van...@cloudera.com
Committed: Sat Aug 1 13:06:50 2015 -0700
--
.../main/scala/org/apache/spark/deploy/yarn/Client.scala | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/df733cbe/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
--
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index 4ac3397..fc11bbf 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -1295,11 +1295,12 @@ object Client extends Logging {
logDebug(Attempting to fetch HBase security token.)
-val hbaseConf = confCreate.invoke(null, conf)
-val token = obtainToken.invoke(null,
hbaseConf).asInstanceOf[Token[TokenIdentifier]]
-credentials.addToken(token.getService, token)
-
-logInfo(Added HBase security token to credentials.)
+val hbaseConf = confCreate.invoke(null,
conf).asInstanceOf[Configuration]
+if (kerberos == hbaseConf.get(hbase.security.authentication)) {
+ val token = obtainToken.invoke(null,
hbaseConf).asInstanceOf[Token[TokenIdentifier]]
+ credentials.addToken(token.getService, token)
+ logInfo(Added HBase security token to credentials.)
+}
} catch {
case e: java.lang.NoSuchMethodException =
logInfo(HBase Method not found: + e)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-9491] Avoid fetching HBase tokens when not needed.
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b53ca247d - f904b46c9
[SPARK-9491] Avoid fetching HBase tokens when not needed.
Look at HBase's configuration to make sure it's configured for
Kerberos. If the HBase configuration is missing, or if HBase is
configured for non-kerberos authentication, then skip getting
tokens.
Reference: http://hbase.apache.org/book.html#security.prerequisites
Author: Marcelo Vanzin van...@cloudera.com
Closes #7810 from vanzin/SPARK-9491 and squashes the following commits:
a57c776 [Marcelo Vanzin] [SPARK-9491] Avoid fetching HBase tokens when not
needed.
(cherry picked from commit df733cbeae7a53826e89574af5463fa018329a22)
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f904b46c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f904b46c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f904b46c
Branch: refs/heads/branch-1.4
Commit: f904b46c9d032bd91b2bee747d74d1134d0f2d43
Parents: b53ca24
Author: Marcelo Vanzin van...@cloudera.com
Authored: Sat Aug 1 13:06:50 2015 -0700
Committer: Marcelo Vanzin van...@cloudera.com
Committed: Sat Aug 1 13:10:53 2015 -0700
--
.../main/scala/org/apache/spark/deploy/yarn/Client.scala | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/f904b46c/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
--
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index e33b822..335c0b9 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -1182,11 +1182,12 @@ object Client extends Logging {
logDebug(Attempting to fetch HBase security token.)
-val hbaseConf = confCreate.invoke(null, conf)
-val token = obtainToken.invoke(null,
hbaseConf).asInstanceOf[Token[TokenIdentifier]]
-credentials.addToken(token.getService, token)
-
-logInfo(Added HBase security token to credentials.)
+val hbaseConf = confCreate.invoke(null,
conf).asInstanceOf[Configuration]
+if (kerberos == hbaseConf.get(hbase.security.authentication)) {
+ val token = obtainToken.invoke(null,
hbaseConf).asInstanceOf[Token[TokenIdentifier]]
+ credentials.addToken(token.getService, token)
+ logInfo(Added HBase security token to credentials.)
+}
} catch {
case e: java.lang.NoSuchMethodException =
logInfo(HBase Method not found: + e)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
