[ https://issues.apache.org/jira/browse/SUBMARINE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Su updated SUBMARINE-834: ------------------------------- Fix Version/s: 0.7.0 Target Version: 0.7.0 > Ranger support for Spark Datasources > ------------------------------------ > > Key: SUBMARINE-834 > URL: https://issues.apache.org/jira/browse/SUBMARINE-834 > Project: Apache Submarine > Issue Type: New Feature > Components: Security > Reporter: Adesh Kumar Rao > Priority: Major > Fix For: 0.7.0 > > > Currently, the spark-security module is tightly coupled with > "HiveTableRelation" and "MetastoreRelation" for authorization of > non-transactional hive tables. > This does not work for Hive transactional (ACID enabled) table. Since Hive > has changed the way data/metadata is stored for transactional tables when > compared to non-transactional tables. Therefore, Spark can not read Hive > transactional tables directly. > So even if security module may enforce security on such tables, spark can't > actually read anything. > > Reading hive's transactional tables in spark, needs > [Spark-Acid|https://github.com/qubole/spark-acid], implemented as Spark > Datasource. Since security module is tightly coupled with "HiveTableRelation" > and "MetastoreRelation", it does not provides authorization support for any > datasource. > > The idea is to support Spark datasource authorization (which has > db/table/column/partitions analogous to hive). > > We can create generic interfaces for datasource, which each datasource can > implement and then it can be authorized using the existing codebase. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org