[
https://issues.apache.org/jira/browse/SPARK-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154251#comment-15154251
]
Minwoo Kang edited comment on SPARK-13394 at 2/19/16 2:09 PM:
--------------------------------------------------------------
I am so sorry for doesn't read the url before post this issue.
Do I have to delete this issue?
Yes, I look SimpleUserGroupsMapping class up in hadoop sources. But
SimpleUserGroupsMapping does not exist.
I do not have no idea. Why does spark need SimpleUserGroupsMapping class.
was (Author: minwoo.kang):
I am so sorry for doesn't read the url before post this issue.
Do I have to delete this issue?
Yes, I look SimpleUserGroupsMapping class up in hadoop sources. But
SimpleUserGroupsMapping does not exist.
> What is "java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.security.SimpleUserGroupsMapping not found"
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-13394
> URL: https://issues.apache.org/jira/browse/SPARK-13394
> Project: Spark
> Issue Type: Question
> Components: Spark Shell
> Affects Versions: 1.6.0
> Environment: Hadoop 2.7.1, r2
> Reporter: Minwoo Kang
> Priority: Minor
>
> I want to deploy spark on yarn.
> So I execute follow script
> {code}
> ./bin/spark-shell --master yarn-client
> {code}
> But Spark shell throw RuntimeException
> Error Description
> {code}
> Exception in thread "main" java.lang.RuntimeException:
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.security.SimpleUserGroupsMapping not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2106)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:70)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:66)
> at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:248)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:763)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
> at
> org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2136)
> at
> org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2136)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2136)
> at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:214)
> at org.apache.spark.repl.SparkIMain.<init>(SparkIMain.scala:118)
> at
> org.apache.spark.repl.SparkILoop$SparkILoopInterpreter.<init>(SparkILoop.scala:187)
> at
> org.apache.spark.repl.SparkILoop.createInterpreter(SparkILoop.scala:217)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:949)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> at
> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
> at
> org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
> at org.apache.spark.repl.Main$.main(Main.scala:31)
> at org.apache.spark.repl.Main.main(Main.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.security.SimpleUserGroupsMapping not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2098)
> ... 33 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.security.SimpleUserGroupsMapping not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1980)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> ... 34 more
> 16/02/19 22:29:51 INFO util.ShutdownHookManager: Shutdown hook called
> 16/02/19 22:29:51 INFO util.ShutdownHookManager: Deleting directory
> /tmp/spark-
> {code}
> Would you mind explaining how to fix this error and how to deploy spark on
> yarn step by step?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
