okumin commented on code in PR #6535: URL: https://github.com/apache/hive/pull/6535#discussion_r3541032725
########## THREAT_MODEL.md: ########## @@ -0,0 +1,404 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to you under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +# Apache Hive — Threat Model (v0 draft) + +> **Status:** v0 draft produced by the ASF Security team (Michael Scovetta +> rubric, run with Claude Opus) for the Apache Hive PMC to review, correct, +> and own. Every non-trivial claim is provenance-tagged +> *(documented)* / *(maintainer)* / *(inferred)*; the *(inferred)* tags are +> the producer's hypotheses and each has a matching question in §14. Written +> against `master`; revise on a new public-facing surface (a new server +> endpoint, auth mechanism, file format, or execution engine), not on +> internal refactors. +> +> As of the PMC's 2026-07 review, **this in-repo model is the canonical Hive +> security model**. Security information scattered across +> <https://hive.apache.org/> predates it and may be out of date; where they +> disagree, this document wins. *(maintainer — okumin, §14 Q14)* + +## §1 — Purpose and consumers + +This document describes the **implicit security contract** between Apache +Hive and its downstream operators: what Hive assumes about its environment, +what it upholds given those assumptions, what it leaves to the operator, and +which "syntactically possible" misuses fall outside the intended design. It +serves the **integrator/operator** (which threats they own) and the +**triager** (classifying a scanner/AI/CVE-style finding as valid, out of +model, or disclaimed by design — cite the section). + +## §2 — What Hive is + +Apache Hive is a **SQL data-warehouse layer over Apache Hadoop** *(documented: +README)*. The in-scope component families: + +| Component | Role | Primary surface | +| --- | --- | --- | +| **HiveServer2 (HS2)** | the SQL front door — accepts queries over a Thrift/binary or HTTP transport, authenticates the session, compiles + authorizes + executes | network (the highest-value untrusted boundary) *(inferred — §14 Q1)* | +| **Hive Metastore (HMS)** | a Thrift service holding table/partition/schema metadata + storage locations | network (intra-cluster) *(inferred — §14 Q1)* | +| **Query compiler + execution** | parse → plan → run on Tez / MapReduce / Spark; reads/writes HDFS, HBase, object stores | depends on the configured engine *(documented: README)* | +| **UDF / SerDe / file-format layer** | user-supplied or built-in functions and (de)serializers invoked during execution | in-JVM code execution *(inferred — §14 Q2)* | +| **JDBC/ODBC drivers + Beeline** | client-side connectors | client trust domain | + +Hive is **not** a standalone secured appliance: it is a clustered service +deployed behind an operator-controlled perimeter, depending on Hadoop (HDFS, +YARN), a metastore RDBMS, an external authorization provider (typically Apache +Ranger in production deployments, though SQL-standard authorization may also be +used), and a KDC for Kerberos — all treated as trusted dependencies. +*(maintainer — okumin, §14 Q3)* + +## §3 — Adversaries in and out of scope + +**In scope**: + +1. **Untrusted clients at Hive's service boundaries** — a **SQL client** + submitting statements to HiveServer2, and a **client accessing the Hive + Metastore through its supported APIs** — attempting to read/modify data + outside their authorization, or to reach the host through query features. + *(maintainer — okumin, §14 Q4)* +2. A **network MITM** on the client↔HS2 or HS2↔HMS path, **in scope when TLS + (or equivalent transport protection) is enabled**; the operator is + responsible for configuring TLS correctly. *(maintainer — okumin, §14 Q4)* +3. A **direct Metastore (HMS) client.** Some external services (e.g. Apache + Spark) connect to the Hive Metastore directly, so HMS is expected to enforce + caller authorization at the **application level** — it is not merely an + intra-cluster service shielded by a network perimeter. A client reaching HMS + outside its authorization is in-model. *(maintainer — okumin, §14 Q1.)* + +**Out of scope** *(maintainer — okumin, §14 Q5)*: + +4. **An operator with `root` / the Hadoop superuser / direct HDFS or metastore-DB + access.** Anyone who already controls the storage layer or the cluster + processes is not an adversary Hive defends against → `OUT-OF-MODEL: + adversary-not-in-scope`. +5. **A trusted authenticated admin** performing an authorized action (creating a + function, changing config, granting a role). A new path to a privilege the + principal already holds is `OUT-OF-MODEL: equivalent-harm`. +6. **Bugs in the dependencies Hive orchestrates** — Hadoop/HDFS, YARN, Tez, + the metastore RDBMS, Ranger, the KDC, the JVM. Report upstream → + `OUT-OF-MODEL: unsupported-component`. + +## §4 — Trust boundaries + +- **Client → HiveServer2** is the primary boundary. The session is + authenticated (Kerberos / LDAP / PAM / custom / none) and every statement is + authorization-checked before execution. From HS2's point of view the **SQL + text, JDBC connection properties, and session-configuration overrides are all + untrusted**; Hive rejects non-acceptable operations through the authorization + plugin or through the configured deny lists — `hive.conf.restricted.list` + (settings an untrusted user may not change), `hive.conf.locked.list`, and + `hive.conf.hidden.list` (secret values an untrusted user may not read). + *(maintainer — okumin, §14 Q6)* +- **HiveServer2 → Metastore** and **HS2 → execution engine / HDFS** are + intra-cluster boundaries. The Metastore is protected at the **application + level** (it enforces caller authorization), because external services such as + Spark talk to HMS directly (§3.3); network isolation is defense-in-depth, not + the primary control *(maintainer — okumin, §14 Q1)*. The HS2 → engine / HDFS + path is assumed inside an operator-controlled perimeter *(maintainer — okumin, §14 Q3)*. +- **`doAs` impersonation:** when enabled, HS2 executes work as the connected + end user against HDFS rather than as the Hive service principal; when + disabled, all access runs as the Hive principal and authorization is fully + delegated to the SQL-layer authorizer. Because the recommended posture uses + an authorization plugin (typically Apache Ranger), **`hive.server2.enable.doAs=false` + is the typical / expected configuration**: HS2 enforces authorization itself + rather than pushing it down to per-user HDFS permissions. *(maintainer — + okumin, §14 Q7)* + +## §5 — What Hive upholds (given §3/§4 assumptions) + +Given valid input and a secure configuration, Hive is expected to uphold the +following. **These properties are configuration-dependent** — they hold when the +operator has configured authentication, authorization, and transport protection +per §6/§8, not by default. *(maintainer — okumin, §14 Q8)* + +- **Authentication.** HiveServer2 and the Hive Metastore can require clients to + authenticate using the configured mechanism (e.g. Kerberos, LDAP), depending + on the endpoint. +- **Authorization scoping.** Requests are authorized as the authenticated Hive + session / metastore user through the configured authorization manager + (typically Apache Ranger). Decisions are scoped to Hive objects and operations + — databases, tables, partitions, columns, functions, and the relevant metadata + operations. +- **Metastore authorization.** When the Hive Metastore is directly exposed to + clients, metastore-side authorization can be enforced through + `HiveMetaStoreAuthorizer` as a pre-event listener, and metadata read/list + results can be filtered server-side via the metastore filter hook. +- **Transport protection.** HiveServer2, the Hive Metastore, and the HS2 Web UI + can protect traffic confidentiality and integrity when TLS/SSL is enabled and + correctly configured. +- **Configuration protection.** Sensitive or security-critical configuration + values can be restricted, locked, or hidden (§4/§8 deny lists) so untrusted + users cannot change security posture at runtime or read secret values. +- **Credential handling.** Secrets can be externalized through Hadoop credential + providers rather than stored in cleartext configuration files. +- **Memory safety on well-formed input** to the extent the JVM provides it; + Hive is Java, so classic memory-corruption is out of the language model. + +Hive does **not** by itself guarantee authorization for a caller who bypasses +Hive and accesses the underlying storage or metastore database directly; those +paths must be protected separately by HDFS / object-store / IAM / database +controls. *(maintainer — okumin)* + +## §6 — What Hive leaves to the operator + +*(maintainer — okumin, §14 Q9)* + +- **Transport security (TLS)** on the HS2, Metastore, and HS2 Web UI endpoints, + and the KDC / LDAP server's own security. +- **Choosing and configuring an authorization model.** **Apache Ranger is the + primary authorization plugin** for Hive; storage-based and SQL-standard + authorization give materially different guarantees. The operator owns the + choice (and whether "no authorization configured" is a supported production + mode — it is not, for an internet- or multi-tenant-exposed deployment). +- **Network isolation** of the Metastore, the metastore RDBMS, and the + execution cluster from untrusted networks. +- **Vetting UDFs / SerDes / aux JARs.** Adding a function or SerDe is adding + code to the execution JVM (see §9). + +## §7 — Properties Hive does *not* uphold (by design) + +*(maintainer — okumin, §14 Q10: confirmed)* + +- **A sandbox around UDFs, SerDes, custom InputFormats, or `TRANSFORM`/script + operators.** Code a principal is authorized to register or invoke runs with + the privileges of the execution process; this is a feature, not a + containment boundary. `BY-DESIGN: property-disclaimed`. The model assumes + authentication and authorization are configured; against that baseline + *(maintainer — okumin)*: + - **Built-in UDFs** are generally safe, but Hive ships a few that allow + arbitrary code execution (`reflect`, `reflect2`, `java_method`, + `in_file`). A Hive administrator must block these via + `hive.server2.builtin.udf.blacklist` (directly or through the + authorization plugin); major plugins such as the Ranger authorizer + configure this blacklist properly. + - **Custom UDFs** — the administrator must restrict, via access policies + (e.g. Ranger), who may add UDF jars or register UDFs. Those trusted users + are responsible for safe UDFs; Hive cannot guarantee safety when a + trusted user adds a compromised UDF. + - **SerDe / InputFormat / OutputFormat** — only administrators can install + the jars; they are responsible for deploying safe implementations, and + Hive trusts them. Hive cannot guarantee safety against a compromised one + an admin installs. + - **`TRANSFORM`** — in a secure deployment it must be prohibited. Major + authorization plugins add + `org.apache.hadoop.hive.ql.security.authorization.plugin.DisallowTransformHook` + to `hive.exec.pre.hooks`; administrators are responsible for using such a + plugin or configuring the hook themselves. +- **Protection against an operator who controls the underlying storage, + metastore DB, or cluster processes** (see §3 item 4). +- **Resource fairness / DoS protection as a hard guarantee.** Hive accepts + arbitrary HiveQL, so bounding the impact of a pathological query is an + operator responsibility, not a Hive bug: operators use HiveServer2 limits + (e.g. `hive.query.max.length`) and YARN resource pools, and — where stronger + isolation than HS2 + YARN can provide is required — separate HS2 instances or + separate Hadoop/YARN clusters. *(maintainer — okumin, §14 Q11)* + +## §8 — Key configuration levers (load-bearing) + +The security posture is configuration-dependent; the levers below are what an +operator sets to reach the §5 properties. Organized by component per the PMC's +structure preference (§14 Q15). *(maintainer — okumin, §14 Q12)* + +### §8.1 — HiveServer2 + +- **Authentication:** `hive.server2.authentication` and the more-specific + `hive.server2.authentication.*` parameters must be configured properly. +- **Authorization:** `hive.security.authorization.enabled=true` and + `hive.security.authorization.manager` (typically Apache Ranger). With Ranger, + `hive.security.authenticator.manager` should be + `org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator` to authorize + as the HS2 session user, and `hive.server2.enable.doAs` is typically `false`. +- **TLS:** `hive.server2.use.SSL=true` plus the more-specific TLS parameters. +- **Config protection:** a Hive administrator should protect security-related + settings via `hive.conf.restricted.list`, `hive.conf.locked.list`, and + `hive.conf.hidden.list` so untrusted users cannot update sensitive settings or + view secret values. +- **Code-execution levers:** built-in UDF blacklist + `hive.server2.builtin.udf.blacklist` blocks the code-exec built-ins + (`reflect`, `reflect2`, `java_method`, `in_file`); `DisallowTransformHook` in + `hive.exec.pre.hooks` prohibits `TRANSFORM`/script operators. Major + authorization plugins (e.g. Ranger) configure both. + +Reference: <https://hive.apache.org/docs/latest/admin/setting-up-hiveserver2/#authenticationsecurity-configuration> + +### §8.2 — HiveServer2 Web UI + +If the UI port (default `10002`) is exposed to untrusted users: + +- **Authentication:** at least one of `hive.server2.webui.use.spnego=true`, + `hive.server2.webui.use.pam=true`, or `hive.server2.webui.auth.method=LDAP` + (plus the more-specific parameters). +- **TLS:** `hive.server2.webui.use.ssl=true` plus the more-specific TLS + parameters. + +### §8.3 — Hive Metastore + +If a Hive administrator exposes the Metastore to untrusted users or systems +(e.g. Spark, Flink): + +- **Authentication:** `metastore.authentication` (e.g. `KERBEROS` or `LDAP`) + plus the more-specific parameters (`metastore.sasl.enabled`, + `metastore.kerberos.*` / `metastore.authentication.*`). +- **Authorization:** `hive.security.authorization.manager` set to + `RangerHiveAuthorizerFactory` (typical), `metastore.pre.event.listeners` + including + `org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer`, + and `metastore.server.filter.enabled=true` with `metastore.filter.hook` set to + the same `HiveMetaStoreAuthorizer` for server-side result filtering. +- **TLS:** `metastore.use.SSL` plus the more-specific TLS parameters. + +### §8.4 — Credentials (all components) + +All credentials / secrets should be managed through the Hadoop +`CredentialProviderAPI`, configured via `hadoop.security.credential.provider.path` +(<https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html>). +Hive does **not** guarantee that the content of `hive-site.xml`, +`metastore-site.xml`, environment variables, or system properties is never +exposed — secrets placed there are the administrator's risk (see §11a). +*(maintainer — okumin)* + +## §11a — Known non-findings (seed for scanner/AI triage) + +These are the dispositions the PMC has recorded; confirm and extend +*(maintainer — okumin, §14 Q13)*: + Review Comment: ```suggestion - Hive defaults are not a complete secure-production baseline; some defaults prioritize ease of adoption, compatibility, or trusted/local deployments. Operators are responsible for enabling and correctly configuring authentication, authorization, transport security, network isolation, and related controls for internet-exposed or multi-tenant production deployments. `OUT-OF-MODEL: operator responsibility` ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
