[
https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704877#comment-16704877
]
Sean Owen commented on SPARK-26043:
-----------------------------------
I see. Typically you'd just instantiate Configuration and it would pick up all
the Hadoop config from your cluster env. I see you're setting the props locally
on the command line though. Configuration is "Writable" but not directly
"Serializable" unfortunately, or else you could just use it in the closure you
send. You could dump the Configuration key-values into a Map and use that I
suppose? Probably a couple lines of code.
> Make SparkHadoopUtil private to Spark
> -------------------------------------
>
> Key: SPARK-26043
> URL: https://issues.apache.org/jira/browse/SPARK-26043
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Marcelo Vanzin
> Assignee: Sean Owen
> Priority: Minor
> Labels: release-notes
> Fix For: 3.0.0
>
>
> This API contains a few small helper methods used internally by Spark, mostly
> related to Hadoop configs and kerberos.
> It's been historically marked as "DeveloperApi". But in reality it's not very
> useful for others, and changes a lot to be considered a stable API. Better to
> just make it private to Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
