I get warnings:
SparkContext: Requesting executors is only supported in coarse-grained mode
ExecutorAllocationManager: Unable to reach the cluster manager to request 2
total executors
I get info messages:
INFO ContextCleaner: Cleaned accumulator 4
Then my "job" just seems to hang - I don't
I have data that I pull in using a sql context and then I convert to an rdd.
The problem is that the type in the rdd is [Any, Iterable[Any]]
And I need to have the type RDD[Array[String]] -- convert the Iterable to an
Array.
Here’s more detail:
val zdata = sqlContext.read.parquet("s3://..