Getting following error stack The Spark session could not be created in the cluster: > at org.apache.hadoop.security.*UserGroupInformation.doAs* > (UserGroupInformation.java:1671) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:160) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ) > at org.*apache.hadoop.hive.metastore.HiveMetaStoreClient* > .open(HiveMetaStoreClient.java:466) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:234) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) > ... 35 more
My understanding that hive.server2.enable.impersonation and hive.server2.enable.doAs should be enabled to make UserGroupInformation.doAs() work? When I try to enable these parameters, Cloudera Manager shows error Hive Impersonation is enabled for Hive Server2 role 'HiveServer2 > (hostname)'. > Hive Impersonation should be disabled to enable Hive authorization using > Sentry So Spark-Hive conflicts with Sentry!? Environment: Hue 3.9 Spark Notebooks + Livy Server (built from master). CDH 5.5. This is a kerberized cluster with Sentry. I was using hue's keytab as hue user is normally (by default in CDH) is allowed to impersonate to other users. So very convenient for Spark Notebooks. Any information to help solve this will be highly appreciated. -- Ruslan Dautkhanov
