[
https://issues.apache.org/jira/browse/HIVE-25174?focusedWorklogId=603418&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603418
]
ASF GitHub Bot logged work on HIVE-25174:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/May/21 07:39
Start Date: 28/May/21 07:39
Worklog Time Spent: 10m
Work Description: symious opened a new pull request #2327:
URL: https://github.com/apache/hive/pull/2327
### What changes were proposed in this pull request?
When Using Ranger on Hive MetaStore, we met an issue that users without
permission to table's HDFS path succeeded in running "msck repair table
TABLENAME".
This command is not authorized when we use `StorageBasedAuthorizer`, after
checking the code, we found `StorageBasedAuthorizer` would check the permission
of table's HDFS path, while `HiveMetastoreAuthorizer` used by Ranger won't when
dealing with the event of `AlterTableEvent`.
This ticket is to add the URI permission check on AlterTableEvent for
`HiveMetastoreAuthorizer`.
### Why are the changes needed?
When using `StorageBasedAuthorizer`, the command of `msck repair table`
would fail if the user don't have write permission to the table's path. But
when using `HiveMetastoreAuthorizer` with Ranger, the command would succeed
even the user don't have write permission to the table's path.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Can be manually test with `alter table` command. Need to set Ranger as
Authorizer for Hive MetaStore. Before the test, we need to ensure test user
doesn't have write permission on the table's path.
* before applying patch
```
spark-sql>
> alter table yiyang_people add columns(id int);
Time taken: 2.379 seconds
21/05/28 15:33:17 INFO SparkSQLCLIDriver: Time taken: 2.379 seconds
spark-sql>
```
* after applying patch
```
spark-sql>
>
> alter table yiyang_people add columns(id int);
21/05/28 15:30:59 WARN HiveExternalCatalog: Could not alter schema of table
`default`.`yiyang_people` in a Hive compatible way. Updating Hive metastore in
Spark SQL specific format.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.sql.hive.client.Shim_v0_12.alterTable(HiveShim.scala:400)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$alterTableDataSchema$1.apply$mcV$sp(HiveClientImpl.scala:536)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$alterTableDataSchema$1.apply(HiveClientImpl.scala:515)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$alterTableDataSchema$1.apply(HiveClientImpl.scala:515)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:277)
at
org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:215)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:214)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:260)
at
org.apache.spark.sql.hive.client.HiveClientImpl.alterTableDataSchema(HiveClientImpl.scala:515)
at
org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$alterTableDataSchema$1.apply$mcV$sp(HiveExternalCatalog.scala:664)
at
org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$alterTableDataSchema$1.apply(HiveExternalCatalog.scala:650)
at
org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$alterTableDataSchema$1.apply(HiveExternalCatalog.scala:650)
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
at
org.apache.spark.sql.hive.HiveExternalCatalog.alterTableDataSchema(HiveExternalCatalog.scala:650)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.alterTableDataSchema(ExternalCatalogWithListener.scala:124)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.alterTableDataSchema(SessionCatalog.scala:391)
at
org.apache.spark.sql.execution.command.AlterTableAddColumnsCommand.run(tables.scala:203)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
at
org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
at
org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$withAction(Dataset.scala:3369)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:643)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:62)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:371)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:274)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter
table. Permission denied: user [yiyang.zhou] does not have [READ_WRITE]
privilege on [hdfs://NN_IP:8020/user/yiyang.zhou/people]
at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:499)
at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:485)
... 52 more
Caused by: MetaException(message:Permission denied: user [yiyang.zhou] does
not have [READ_WRITE] privilege on [hdfs://NN_IP:8020/user/yiyang.zhou/people])
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$alter_table_with_cascade_result$alter_table_with_cascade_resultStandardScheme.read(ThriftHiveMetastore.java:41136)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$alter_table_with_cascade_result$alter_table_with_cascade_resultStandardScheme.read(ThriftHiveMetastore.java:41113)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$alter_table_with_cascade_result.read(ThriftHiveMetastore.java:41055)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_alter_table_with_cascade(ThriftHiveMetastore.java:1382)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.alter_table_with_cascade(ThriftHiveMetastore.java:1366)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table(HiveMetaStoreClient.java:341)
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.alter_table(SessionHiveMetaStoreClient.java:251)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
at com.sun.proxy.$Proxy5.alter_table(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:497)
... 53 more
Error in query: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
alter table. Permission denied: user [yiyang.zhou] does not have [READ_WRITE]
privilege on [hdfs://NN_IP:8020/user/yiyang.zhou/people];
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 603418)
Remaining Estimate: 0h
Time Spent: 10m
> HiveMetastoreAuthorizer didn't check URI permission for AlterTableEvent
> -----------------------------------------------------------------------
>
> Key: HIVE-25174
> URL: https://issues.apache.org/jira/browse/HIVE-25174
> Project: Hive
> Issue Type: Improvement
> Reporter: Janus Chow
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When Using Ranger on Hive MetaStore, we met an issue that users without
> permission to table's HDFS path succeeded in running "msck repair table
> TABLENAME".
> This command is not authorized when we use `StorageBasedAuthorizer`, after
> checking the code, we found `StorageBasedAuthorizer` would check the
> permission of table's HDFS path, while `HiveMetastoreAuthorizer` used by
> Ranger won't when dealing with the event of `AlterTableEvent`.
> This ticket is to add the URI permission check on AlterTableEvent for
> `HiveMetastoreAuthorizer`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
