[
https://issues.apache.org/jira/browse/HIVE-29671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Hemanth Gantasala updated HIVE-29671:
-----------------------------------------
Security: (was: Non-Public)
> Fix SSRF in Avro SerDe via avro.schema.url and add schema-path authorization
> ----------------------------------------------------------------------------
>
> Key: HIVE-29671
> URL: https://issues.apache.org/jira/browse/HIVE-29671
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Security, Serializers/Deserializers
> Affects Versions: 4.2.0
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Critical
> Fix For: 4.2.1
>
> Attachments: screenshot-1.png
>
>
> *Summary*
> Hive's Avro SerDe resolves table schemas from {{avro.schema.url}} using
> {{url.openStream()}} when the URL is not a Hadoop FileSystem URI. An
> authenticated user can set this property on an external Avro table to make
> the Hive server (HiveServer2, Metastore, or Tez/MR workers) fetch arbitrary
> URLs — including cloud metadata endpoints
> ({{{}[http://169.254.169.254/]...{}}}), internal services, or local files
> ({{{}[file:///etc/passwd]{}}}).
> This issue adds scheme-based restrictions at the shared schema-resolution
> chokepoint, optional opt-in support for HTTP schema URLs with a host
> allowlist, and DFS_URI authorization of filesystem schema paths at both DDL
> and query time.
> *Problem*
> {{AvroSerdeUtils.determineSchemaOrThrowException()}} falls back to
> {{getSchemaFor(new URL(schemaString))}} when {{getSchemaFromFS()}} returns
> null, with no scheme or host validation. Schema resolution can occur at:
> * DDL ({{{}CREATE TABLE{}}} without columns)
> * Query compilation ({{{}Table.getDeserializer(){}}})
> * Query execution ({{{}AvroGenericRecordReader{}}}, {{MapOperator}} SerDe
> init)
> Because the fetch runs as the server/job identity, standard table {{SELECT}}
> authorization does not prevent SSRF via HTTP or {{[file://|file:///]}} URLs.
> {{avro.schema.url}} was also not included in the metastore {{DFS_URI}}
> authorization model.
> *Example attack:*
> CREATE EXTERNAL TABLE malicious_avro (id INT, name STRING)
> STORED AS AVRO
> TBLPROPERTIES
> ('avro.schema.url'='http://169.254.169.254/latest/meta-data/iam/security-credentials/');
>
> SELECT * FROM malicious_avro; – Hive server fetches attacker-controlled URL
>
> *Root Cause*
> * HIVE-5224 intentionally added a non-filesystem URL fallback for
> {{avro.schema.url}}
> * HIVE-13294 fixed a connection leak in {{getSchemaFor(URL)}} but did not
> add SSRF protections
> * No validation or authorization existed for {{avro.schema.url}} at schema
> fetch time
> *Solution*
> 1. Scheme allowlist (primary SSRF fix)
> In {{{}AvroSerdeUtils.determineSchemaOrThrowException(){}}}, validate
> {{avro.schema.url}} before any fetch:
> * Allow scheme-less URIs (default FS) and configurable filesystem schemes
> ({{{}hdfs{}}}, {{{}s3a{}}}, {{{}abfs{}}}, {{{}gs{}}}, etc.)
> * Block {{{}[file://|file:///]{}}}, {{{}ftp://{}}}, and other non-filesystem
> schemes unconditionally
> * Block {{http://}} / {{https://}} by default; allow only when explicitly
> enabled via configuration and host allowlist
> 2. New configuration ({{{}HiveConf{}}})
> ||Property||Default||Purpose||
> |{{hive.avro.schema.url.allowed.schemes}}|{{hdfs,s3,s3a,...}}|Permitted
> filesystem schemes|
> |{{hive.avro.schema.url.remote.http.enabled}}|{{false}}|Opt-in HTTP(S) schema
> fetch|
> |{{hive.avro.schema.url.http.allowed.hosts}}|{{""}}|Host allowlist when HTTP
> is enabled|
> 3. DDL-time DFS_URI authorization
> {{CreateTableEvent}} and {{AlterTableEvent}} add {{avro.schema.url}} as a
> {{DFS_URI}} input privilege object for Avro tables using filesystem schema
> URLs.
> 4. Query-time DFS_URI authorization
> {{AuthorizationUtils.addAvroSchemaUrlInputForReadEntity()}} adds a DFS
> {{ReadEntity}} when a direct table/partition read will fetch a filesystem
> {{{}avro.schema.url{}}}. Hooked from {{PlanUtils.addInput()}} and
> {{CommandAuthorizerV2}} so {{SELECT}} requires read access to the schema
> path, including for tables created before this fix.
> 5. Resolve-once optimization
> After a successful schema fetch, materialize the resolved schema into
> {{avro.schema.literal}} to avoid repeated per-split fetches on workers
> (aligns with HIVE-14792).
> *Backward Compatibility*
> * HTTP(S) {{avro.schema.url}} values stop working by default
> * To restore HTTP schema URLs: set
> {{hive.avro.schema.url.remote.http.enabled=true}} and configure
> {{hive.avro.schema.url.http.allowed.hosts}}
> * Scheme-less and {{hdfs://}} / object-store URLs continue to work unchanged
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
