Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228320944 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -647,4 +647,42 @@ package object config { .stringConf .toSequence .createWithDefault(Nil) + + private[spark] val KAFKA_DELEGATION_TOKEN_ENABLED = + ConfigBuilder("spark.kafka.delegation.token.enabled") + .doc("Set to 'true' for obtaining delegation token from kafka.") + .booleanConf + .createWithDefault(false) + + private[spark] val KAFKA_BOOTSTRAP_SERVERS = + ConfigBuilder("spark.kafka.bootstrap.servers") --- End diff -- Given the defaults the tokens fetch would be attempted if only `spark.kafka.bootstrap.servers` is defined right ? And the spark-sql-kafka libraries needs to be in the class path as well ? Better mention these in the docs. And make the `spark.security.credentials.kafka.enabled` default to false if it makes sense. Also consider prefixing all the configs with `spark.security.credentials.kafka` instead of `spark.kafka` (like `spark.security.credentials.kafka.bootstrap.servers`) to make it explicit that these are security related settings required for fetching kafka delegation tokens.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org