The only way I ever got it to work with spark standalone is via web hdfs.
See
https://issues.apache.org/jira/browse/SPARK-5158?focusedCommentId=16516856=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16516856
On Fri, 8 Jan 2021 at 18:49, Sudhir Babu Pothineni
wrote:
Well it is now...
The RDD had a repartition call on it.
When I removed repartition it it it would work,
When i did not remove the repartition but called
called rdd.partitions.length on it it would also work!
I looked into the partitions method and in it some instance variables get
initialized,
