Manu Zhang created SPARK-39278:
----------------------------------
Summary: Alternative configs of Hadoop Filesystems to access break
backward compatibility
Key: SPARK-39278
URL: https://issues.apache.org/jira/browse/SPARK-39278
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.3.0
Reporter: Manu Zhang
Before [https://github.com/apache/spark/pull/23698,]
The precedence of configuring Hadoop Filesystems to access is
{code:java}
spark.yarn.access.hadoopFileSystems -> spark.yarn.access.namenodes{code}
Afterwards, it's
{code:java}
spark.kerberos.access.hadoopFileSystems -> spark.yarn.access.namenodes ->
spark.yarn.access.hadoopFileSystems{code}
When both spark.yarn.access.hadoopFileSystems and spark.yarn.access.namenodes
are configured with different values, the PR breaks backward compatibility and
cause application failure.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
