I'm using Spark 1.5.1
When I turned on DEBUG, I don't see anything that looks useful. Other than
the INFO outputs, there is a ton of RPC message related logs, and this bit:
16/01/13 05:53:43 DEBUG ClosureCleaner: +++ Cleaning closure
(org.apache.spark.rdd.RDD$$anonfun$count$1) +++
16/01/13 05:53
Which release of Spark are you using ?
Can you turn on DEBUG logging to see if there is more clue ?
Thanks
On Tue, Jan 12, 2016 at 6:37 PM, AlexG wrote:
> I transpose a matrix (colChunkOfA) stored as a 200-by-54843210 as an array
> of
> rows in Array[Array[Float]] format into another matrix (r
