Running Spark application from Tomcat

[Andreas Koch]

I have a Spark java application that I run in local-mode. As such it runs
without any issues.

Now, I would like to run it as a webservice from Tomcat. The first issue I
had with this was that the spark-assembly jar contains javax.servlet, which
Tomcat does not allow. Therefore I removed javax.servlet from the jar file.

Now, I get an Exception like this:

java.lang.RuntimeException: class
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not
org.apache.hadoop.security.GroupMappingServiceProvider
        org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1921)
        org.apache.hadoop.security.Groups.<init>(Groups.java:64)
        
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
        
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
        
org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:283)
        org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:36)
        
org.apache.spark.deploy.SparkHadoopUtil$.<init>(SparkHadoopUtil.scala:109)
        org.apache.spark.deploy.SparkHadoopUtil$.<clinit>(SparkHadoopUtil.scala)
        org.apache.spark.SparkContext.<init>(SparkContext.scala:228)
        
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:53)


Any ideas?


Thanks,

Andreas