shfshihuafeng opened a new pull request, #3056: URL: https://github.com/apache/drill/pull/3056
# [DRILL-8548](https://issues.apache.org/jira/browse/DRILL-8548): Integrate Apache Ranger authorization for Drill Integrate Apache Ranger authorization for Drill ## Description ### Motivation Apache Drill is a federated query engine that concurrently accesses heterogeneous data sources such as relational databases (MySQL), file systems (local/HDFS), and search engines (Elasticsearch). However, each data source has its own independent permission model with varying granularities – relational databases support schema/table/column level controls, file systems often lack column‑level privileges, and Elasticsearch only provides index‑level authorization. This forces administrators to maintain separate permission schemes for each data source individually, resulting in cumbersome configuration, high operational overhead, and the inability to achieve cross‑source unified auditing. Therefore, it is necessary to build a unified, fine‑grained permission governance system for Drill, which is primarily embodied as follows: - **Unified Policy Management**: All data sources accessible via Drill can be governed through a single Ranger console. A single Ranger policy applies simultaneously across multiple heterogeneous sources, eliminating the need for per‑storage‑system configuration and greatly simplifying administrative complexity. - **Column‑Level Fine‑Grained Access Control**: Permissions can be restricted down to individual columns within a table, satisfying the need to isolate sensitive fields (e.g., ID numbers, salaries, phone numbers) and achieving true field‑level security protection. ### Introduction This PR introduces Apache Ranger as a pluggable authorization framework for Drill, enabling centralized table-level and column-level access control for Drill queries. It is a substantial feature spanning three layers: a new `drill-ranger` module (authorization plugin + Ranger Admin service plugin), integration hooks in `exec/java-exec`, and distribution packaging. The design follows Drill's existing `AccessAuthorizer` SPI and Calcite's `RelShuttle` mechanism so that column-level checks happen in the `toRel` phase before physical planning. - **Consistent Enforcement at the Federation Layer**: Permission checks are performed uniformly during Drill's `toRel` phase (before logical plan generation), ensuring that the same set of policies takes effect regardless of whether the query is a single‑table scan, a cross‑source JOIN, or a nested subquery. There is no possibility of bypassing Ranger through other storage plugins, guaranteeing complete integrity of mandatory enforcement. - **Audit & Compliance Readiness**: Every access decision is recorded and pushed into Ranger's audit pipeline, providing a complete audit trail of "who queried what and when". This natively supports compliance reporting requirements. ## Documentation This PR ships a developer-facing quick start guide, [RangerAuthorization.md](docs/dev/RangerAuthorization.md), covering the end-to-end architecture, configuration, deployment, and authorization behavior of the Ranger integration. It is intended both for contributors extending/debugging the integration and for operators who want to understand the column-level authorization flow. ## Testing ### `drill-ranger-service` (1 file, 28 cases, JUnit 5 + Mockito) - **`RangerServiceDrillTest`** — `buildBaseUrl` normalization, `escapeSql` SQL-injection guard, `firstHint`/`extractFirstColumnValues` JSON parsing, `lookupResource`/`validateConfig` edge cases. ### `exec/java-exec` (4 files, 27 cases, JUnit 4 + Mockito, extends `BaseTest`) - **`AccessAuthorizerFactoryTest`** — config-driven reflective loading, caching, failure modes. - **`NoOpAccessAuthorizerTest`** — fail-open contract. - **`RangerAccessAuthorizerTest`** — delegation to `DrillAccessControl` with `DrillAccessType` enum normalization (null/unknown → `SELECT`). - **`DrillCalciteCatalogReaderTest`** — `getDefaultSchemaByDataSource` package-private static helper. All tests pass on Windows with `-DforkCount=0` (the default `forkCount=1` fails with `CreateProcess error=5` on locked-down Windows hosts). `mockito-inline` (already declared in the root POM) is used for `mockStatic` of `RangerBaseAuthorizer`, `DrillAccessControl`, and `UserGroupInformation`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
