[jira] [Commented] (RANGER-2128) Implement SparkSQL plugin
[
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544410#comment-16544410
]
Don Bosco Durai commented on RANGER-2128:
-
{quote}You can try Spark 2.3.1, because this change
https://github.com/apache/spark/blob/branch-2.3/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala#L129
ensures the SessionState that contains the thrift user body will be
reused.{quote}
[~Qin Yao] thanks. I was able to get the user from the State. In the AuthzImpl,
the following code gave the session user. I will update the Ranger Authorizer
to use this user
{code:java}
clientImpl.state.getUserName()
{code}
Regarding the ClassCastException, "org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:
org.apache.hadoop.hive.metastore.ObjectStore cannot be cast to
org.apache.hadoop.hive.metastore.RawStore", I got this error even after I
removed all HiveAuthorizer related code. Do you think this is because of
different class loader that might be used when we do
"ext.injectOptimizerRule(Authorizer)" in the case of Thrift Server? You are
familiar with Spark, any clue is appreciated. Thanks
> Implement SparkSQL plugin
> -
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
> Issue Type: New Feature
> Components: plugins, Ranger
>Affects Versions: 1.1.0
>Reporter: t oo
>Assignee: Kent Yao
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: support_ranger11.tgz
>
>
> Implement SparkSQL plugin
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (RANGER-2153) Supply the function of reverting policy history version.
[ https://issues.apache.org/jira/browse/RANGER-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-2153: Attachment: 0001-RANGER-2153-Supply-the-function-of-reverting-policy-.patch > Supply the function of reverting policy history version. > > > Key: RANGER-2153 > URL: https://issues.apache.org/jira/browse/RANGER-2153 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Affects Versions: 1.0.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang >Priority: Major > Attachments: > 0001-RANGER-2153-Supply-the-function-of-reverting-policy-.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2153) Supply the function of reverting policy history version.
[ https://issues.apache.org/jira/browse/RANGER-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-2153: Issue Type: New Feature (was: Bug) > Supply the function of reverting policy history version. > > > Key: RANGER-2153 > URL: https://issues.apache.org/jira/browse/RANGER-2153 > Project: Ranger > Issue Type: New Feature > Components: Ranger >Affects Versions: 1.0.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2153) Supply the function of reverting policy history version.
Qiang Zhang created RANGER-2153: --- Summary: Supply the function of reverting policy history version. Key: RANGER-2153 URL: https://issues.apache.org/jira/browse/RANGER-2153 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 1.0.0 Reporter: Qiang Zhang Assignee: Qiang Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2128) Implement SparkSQL plugin
[ https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Don Bosco Durai updated RANGER-2128: Attachment: support_ranger11.tgz > Implement SparkSQL plugin > - > > Key: RANGER-2128 > URL: https://issues.apache.org/jira/browse/RANGER-2128 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Affects Versions: 1.1.0 >Reporter: t oo >Assignee: Kent Yao >Priority: Major > Fix For: 2.0.0 > > Attachments: support_ranger11.tgz > > > Implement SparkSQL plugin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2128) Implement SparkSQL plugin
[
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544118#comment-16544118
]
Don Bosco Durai commented on RANGER-2128:
-
[~Qin Yao] I managed to get your code working with Ranger master branch. I have
attached the updates I have done. Can you try it on your side and see whether
it works for you?
It works with Spark Shell, but with Thrift Server I am getting an exception.
Have you seen this before? I have enabled Ranger in thrift server by adding the
following line in sparkconf.conf
{code:java}
spark.sql.extensions
org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension
{code}
Error opening session:
org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:
org.apache.hadoop.hive.metastore.ObjectStore cannot be cast to
org.apache.hadoop.hive.metastore.RawStore;
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:106)
at
org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
at
org.apache.spark.sql.internal.SharedState.globalTempViewManager$lzycompute(SharedState.scala:138)
at
org.apache.spark.sql.internal.SharedState.globalTempViewManager(SharedState.scala:133)
at
org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$2.apply(HiveSessionStateBuilder.scala:54)
at
org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$2.apply(HiveSessionStateBuilder.scala:54)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.globalTempViewManager$lzycompute(SessionCatalog.scala:91)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.globalTempViewManager(SessionCatalog.scala:91)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.setCurrentDatabase(SessionCatalog.scala:252)
at
org.apache.spark.sql.execution.command.SetDatabaseCommand.run(databases.scala:59)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3253)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3252)
at org.apache.spark.sql.Dataset.(Dataset.scala:190)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:638)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.openSession(SparkSQLSessionManager.scala:70)
at
org.apache.hive.service.cli.CLIService.openSession(CLIService.java:194)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:354)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:246)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:53)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
> Implement SparkSQL plugin
> -
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
> Issue Type: New Feature
> Components: plugins, Ranger
>Affects Versions: 1.1.0
>Reporter: t oo
>Assignee: Kent Yao
>Priority: Major
> Fix For: 2.0.0
>
>
> Implement SparkSQL plugin
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
