Thanks TD and Marco for the feedback.
The directory referenced by SPARK_LOCAL_DIRS did not exist. After creating
that directory, it worked.
This was the first time I was trying to run spark on standalone cluster, so
I missed it.
Thanks
On Fri, Feb 17, 2017 at 12:35 PM, Tathagata Das wrote:
>
Seems like an issue with the HDFS you are using for checkpointing. Its not
able to write data properly.
On Thu, Feb 16, 2017 at 2:40 PM, shyla deshpande
wrote:
> Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException):
> File
> /checkpoint/11ea8862-122c-4614-bc7e-f761bb57ba23/rdd-
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
/checkpoint/11ea8862-122c-4614-bc7e-f761bb57ba23/rdd-347/.part-1-attempt-3
could only be replicated to 0 nodes instead of minReplication (=1). There
are 0 datanode(s) running and no node(s) are excluded in this operati
