[jira] [Updated] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

2015-10-02 Thread Sean Owen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Owen updated SPARK-6882:
-
Assignee: Steve Loughran

> Spark ThriftServer2 Kerberos failed encountering 
> java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
> are: [auth-int, auth-conf, auth]
> 
>
> Key: SPARK-6882
> URL: https://issues.apache.org/jira/browse/SPARK-6882
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 1.2.1, 1.3.0, 1.4.0
> Environment: * Apache Hadoop 2.4.1 with Kerberos Enabled
> * Apache Hive 0.13.1
> * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
> * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
>Reporter: Andrew Lee
>Assignee: Steve Loughran
> Fix For: 1.5.1
>
>
> When Kerberos is enabled, I get the following exceptions. 
> {code}
> 2015-03-13 18:26:05,363 ERROR 
> org.apache.hive.service.cli.thrift.ThriftCLIService 
> (ThriftBinaryCLIService.java:run(93)) - Error: 
> java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
> are: [auth-int, auth-conf, auth]
> {code}
> I tried it in
> * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
> * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
> with
> * Apache Hive 0.13.1
> * Apache Hadoop 2.4.1
> Build command
> {code}
> mvn -U -X -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Phive-thriftserver 
> -Dhadoop.version=2.4.1 -Dyarn.version=2.4.1 -Dhive.version=0.13.1 -DskipTests 
> install
> {code}
> When starting Spark ThriftServer in {{yarn-client}} mode, the command to 
> start thriftserver looks like this
> {code}
> ./start-thriftserver.sh --hiveconf hive.server2.thrift.port=2 --hiveconf 
> hive.server2.thrift.bind.host=$(hostname) --master yarn-client
> {code}
> {{hostname}} points to the current hostname of the machine I'm using.
> Error message in {{spark.log}} from Spark 1.2.1 (1.2 rc1)
> {code}
> 2015-03-13 18:26:05,363 ERROR 
> org.apache.hive.service.cli.thrift.ThriftCLIService 
> (ThriftBinaryCLIService.java:run(93)) - Error: 
> java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
> are: [auth-int, auth-conf, auth]
> at org.apache.hive.service.auth.SaslQOP.fromString(SaslQOP.java:56)
> at 
> org.apache.hive.service.auth.HiveAuthFactory.getSaslProperties(HiveAuthFactory.java:118)
> at 
> org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:133)
> at 
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:43)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> I'm wondering if this is due to the same problem described in HIVE-8154 
> HIVE-7620 due to an older code based for the Spark ThriftServer?
> Any insights are appreciated. Currently, I can't get Spark ThriftServer2 to 
> run against a Kerberos cluster (Apache 2.4.1).
> My hive-site.xml looks like the following for spark/conf.
> The kerberos keytab and tgt are configured correctly, I'm able to connect to 
> metastore, but the subsequent steps failed due to the exception.
> {code}
> 
>   hive.semantic.analyzer.factory.impl
>   org.apache.hcatalog.cli.HCatSemanticAnalyzerFactory
> 
> 
>   hive.metastore.execute.setugi
>   true
> 
> 
>   hive.stats.autogather
>   false
> 
> 
>   hive.session.history.enabled
>   true
> 
> 
>   hive.querylog.location
>   /tmp/home/hive/log/${user.name}
> 
> 
>   hive.exec.local.scratchdir
>   /tmp/hive/scratch/${user.name}
> 
> 
>   hive.metastore.uris
>   thrift://somehostname:9083
> 
> 
> 
>   hive.server2.authentication
>   KERBEROS
> 
> 
>   hive.server2.authentication.kerberos.principal
>   ***
> 
> 
>   hive.server2.authentication.kerberos.keytab
>   ***
> 
> 
>   hive.server2.thrift.sasl.qop
>   auth
>   Sasl QOP value; one of 'auth', 'auth-int' and 
> 'auth-conf'
> 
> 
>   hive.server2.enable.impersonation
>   Enable user impersonation for HiveServer2
>   true
> 
> 
> 
>   hive.metastore.sasl.enabled
>   true
> 
> 
>   hive.metastore.kerberos.keytab.file
>   ***
> 
> 
>   hive.metastore.kerberos.principal
>   ***
> 
> 
>   hive.metastore.cache.pinobjtypes
>   Table,Database,Type,FieldSchema,Order
> 
> 
>   hdfs_sentinel_file
>   ***
> 
> 
>   hive.metastore.warehouse.dir
>   /hive
> 
> 
>   hive.metastore.client.socket.timeout
>   600
> 
> 
>   hive.warehouse.subdir.inherit.perms
>   true
> 
> {code}
> Here, I'm attaching a more detail logs from Spark 1.3 rc1.
> {code}
> 2015-04-13 16:37:20,688 INFO  org.apache.hadoop.security.UserGroupInformation 
> (UserGroupInformation.java:loginUserFromKeytab(893)) - Login successful for 
> user

[jira] [Updated] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

2015-07-06 Thread Andrew Lee (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lee updated SPARK-6882:
--
Affects Version/s: 1.4.0

 Spark ThriftServer2 Kerberos failed encountering 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 

 Key: SPARK-6882
 URL: https://issues.apache.org/jira/browse/SPARK-6882
 Project: Spark
  Issue Type: Bug
  Components: SQL
Affects Versions: 1.2.1, 1.3.0, 1.4.0
 Environment: * Apache Hadoop 2.4.1 with Kerberos Enabled
 * Apache Hive 0.13.1
 * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
 * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
Reporter: Andrew Lee

 When Kerberos is enabled, I get the following exceptions. 
 {code}
 2015-03-13 18:26:05,363 ERROR 
 org.apache.hive.service.cli.thrift.ThriftCLIService 
 (ThriftBinaryCLIService.java:run(93)) - Error: 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 {code}
 I tried it in
 * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
 * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
 with
 * Apache Hive 0.13.1
 * Apache Hadoop 2.4.1
 Build command
 {code}
 mvn -U -X -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Phive-thriftserver 
 -Dhadoop.version=2.4.1 -Dyarn.version=2.4.1 -Dhive.version=0.13.1 -DskipTests 
 install
 {code}
 When starting Spark ThriftServer in {{yarn-client}} mode, the command to 
 start thriftserver looks like this
 {code}
 ./start-thriftserver.sh --hiveconf hive.server2.thrift.port=2 --hiveconf 
 hive.server2.thrift.bind.host=$(hostname) --master yarn-client
 {code}
 {{hostname}} points to the current hostname of the machine I'm using.
 Error message in {{spark.log}} from Spark 1.2.1 (1.2 rc1)
 {code}
 2015-03-13 18:26:05,363 ERROR 
 org.apache.hive.service.cli.thrift.ThriftCLIService 
 (ThriftBinaryCLIService.java:run(93)) - Error: 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 at org.apache.hive.service.auth.SaslQOP.fromString(SaslQOP.java:56)
 at 
 org.apache.hive.service.auth.HiveAuthFactory.getSaslProperties(HiveAuthFactory.java:118)
 at 
 org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:133)
 at 
 org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:43)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 I'm wondering if this is due to the same problem described in HIVE-8154 
 HIVE-7620 due to an older code based for the Spark ThriftServer?
 Any insights are appreciated. Currently, I can't get Spark ThriftServer2 to 
 run against a Kerberos cluster (Apache 2.4.1).
 My hive-site.xml looks like the following for spark/conf.
 The kerberos keytab and tgt are configured correctly, I'm able to connect to 
 metastore, but the subsequent steps failed due to the exception.
 {code}
 property
   namehive.semantic.analyzer.factory.impl/name
   valueorg.apache.hcatalog.cli.HCatSemanticAnalyzerFactory/value
 /property
 property
   namehive.metastore.execute.setugi/name
   valuetrue/value
 /property
 property
   namehive.stats.autogather/name
   valuefalse/value
 /property
 property
   namehive.session.history.enabled/name
   valuetrue/value
 /property
 property
   namehive.querylog.location/name
   value/tmp/home/hive/log/${user.name}/value
 /property
 property
   namehive.exec.local.scratchdir/name
   value/tmp/hive/scratch/${user.name}/value
 /property
 property
   namehive.metastore.uris/name
   valuethrift://somehostname:9083/value
 /property
 !-- HIVE SERVER 2 --
 property
   namehive.server2.authentication/name
   valueKERBEROS/value
 /property
 property
   namehive.server2.authentication.kerberos.principal/name
   value***/value
 /property
 property
   namehive.server2.authentication.kerberos.keytab/name
   value***/value
 /property
 property
   namehive.server2.thrift.sasl.qop/name
   valueauth/value
   descriptionSasl QOP value; one of 'auth', 'auth-int' and 
 'auth-conf'/description
 /property
 property
   namehive.server2.enable.impersonation/name
   descriptionEnable user impersonation for HiveServer2/description
   valuetrue/value
 /property
 !-- HIVE METASTORE --
 property
   namehive.metastore.sasl.enabled/name
   valuetrue/value
 /property
 property
   namehive.metastore.kerberos.keytab.file/name
   value***/value
 /property
 property
   namehive.metastore.kerberos.principal/name
   value***/value
 /property
 property
   namehive.metastore.cache.pinobjtypes/name

[jira] [Updated] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

2015-04-13 Thread Sean Owen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Owen updated SPARK-6882:
-
Component/s: SQL

 Spark ThriftServer2 Kerberos failed encountering 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 

 Key: SPARK-6882
 URL: https://issues.apache.org/jira/browse/SPARK-6882
 Project: Spark
  Issue Type: Bug
  Components: SQL
Affects Versions: 1.2.1, 1.3.0
 Environment: * Apache Hadoop 2.4.1 with Kerberos Enabled
 * Apache Hive 0.13.1
 * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
 * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
Reporter: Andrew Lee

 When Kerberos is enabled, I get the following exceptions. 
 {code}
 2015-03-13 18:26:05,363 ERROR 
 org.apache.hive.service.cli.thrift.ThriftCLIService 
 (ThriftBinaryCLIService.java:run(93)) - Error: 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 {code}
 I tried it in
 * Spark 1.2.1 git commit b6eaf77d4332bfb0a698849b1f5f917d20d70e97
 * Spark 1.3.0 rc1 commit label 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
 with
 * Apache Hive 0.13.1
 * Apache Hadoop 2.4.1
 Build command
 {code}
 mvn -U -X -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Phive-thriftserver 
 -Dhadoop.version=2.4.1 -Dyarn.version=2.4.1 -Dhive.version=0.13.1 -DskipTests 
 install
 {code}
 When starting Spark ThriftServer in {{yarn-client}} mode, the command to 
 start thriftserver looks like this
 {code}
 ./start-thriftserver.sh --hiveconf hive.server2.thrift.port=2 --hiveconf 
 hive.server2.thrift.bind.host=$(hostname) --master yarn-client
 {code}
 {{hostname}} points to the current hostname of the machine I'm using.
 Error message in {{spark.log}} from Spark 1.2.1 (1.2 rc1)
 {code}
 2015-03-13 18:26:05,363 ERROR 
 org.apache.hive.service.cli.thrift.ThriftCLIService 
 (ThriftBinaryCLIService.java:run(93)) - Error: 
 java.lang.IllegalArgumentException: Unknown auth type: null Allowed values 
 are: [auth-int, auth-conf, auth]
 at org.apache.hive.service.auth.SaslQOP.fromString(SaslQOP.java:56)
 at 
 org.apache.hive.service.auth.HiveAuthFactory.getSaslProperties(HiveAuthFactory.java:118)
 at 
 org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:133)
 at 
 org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:43)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 I'm wondering if this is due to the same problem described in HIVE-8154 
 HIVE-7620 due to an older code based for the Spark ThriftServer?
 Any insights are appreciated. Currently, I can't get Spark ThriftServer2 to 
 run against a Kerberos cluster (Apache 2.4.1).
 My hive-site.xml looks like the following for spark/conf.
 The kerberos keytab and tgt are configured correctly, I'm able to connect to 
 metastore, but the subsequent steps failed due to the exception.
 {code}
 property
   namehive.semantic.analyzer.factory.impl/name
   valueorg.apache.hcatalog.cli.HCatSemanticAnalyzerFactory/value
 /property
 property
   namehive.metastore.execute.setugi/name
   valuetrue/value
 /property
 property
   namehive.stats.autogather/name
   valuefalse/value
 /property
 property
   namehive.session.history.enabled/name
   valuetrue/value
 /property
 property
   namehive.querylog.location/name
   value/tmp/home/hive/log/${user.name}/value
 /property
 property
   namehive.exec.local.scratchdir/name
   value/tmp/hive/scratch/${user.name}/value
 /property
 property
   namehive.metastore.uris/name
   valuethrift://somehostname:9083/value
 /property
 !-- HIVE SERVER 2 --
 property
   namehive.server2.authentication/name
   valueKERBEROS/value
 /property
 property
   namehive.server2.authentication.kerberos.principal/name
   value***/value
 /property
 property
   namehive.server2.authentication.kerberos.keytab/name
   value***/value
 /property
 property
   namehive.server2.thrift.sasl.qop/name
   valueauth/value
   descriptionSasl QOP value; one of 'auth', 'auth-int' and 
 'auth-conf'/description
 /property
 property
   namehive.server2.enable.impersonation/name
   descriptionEnable user impersonation for HiveServer2/description
   valuetrue/value
 /property
 !-- HIVE METASTORE --
 property
   namehive.metastore.sasl.enabled/name
   valuetrue/value
 /property
 property
   namehive.metastore.kerberos.keytab.file/name
   value***/value
 /property
 property
   namehive.metastore.kerberos.principal/name
   value***/value
 /property
 property
   namehive.metastore.cache.pinobjtypes/name
   valueTable,Database,Type,FieldSchema,Order/value