HyukjinKwon opened a new pull request, #58605:
URL: https://github.com/apache/spark/pull/58605
### What changes were proposed in this pull request?
Add opt-in controls, all defaulting to the current behavior:
- `spark.sql.kafka.disallowedOptions` (default empty): reject the listed
Kafka client option names
when passed via SQL options.
- `spark.sql.avro.schemaUrlAllowedSchemes` (default empty = unrestricted):
restrict the URI schemes
accepted for `avroSchemaUrl`.
- Hive metastore serde/InputFormat class names are resolved with `initialize
= false` (static
initialization is deferred to instantiation at scan time).
### Why are the changes needed?
Gives operators optional controls over which data-source client options and
schema-URL schemes are
accepted and defers metastore class initialization. The option-based
controls default to empty
(disabled), so there is no behavior change unless configured.
### Does this PR introduce _any_ user-facing change?
No by default. When the new options are set, the corresponding Kafka options
/ `avroSchemaUrl`
schemes are rejected.
### How was this patch tested?
`hive/compile` and `sql-kafka-0-10/compile`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac
This pull request and its description were written by Isaac.
Co-authored-by: Isaac <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
