Re: Spark crashes with Filesystem recovery

2016-05-17 Thread Jeff Zhang
I don't think this related with file system recovery. spark.deploy.recoveryDirectory is standalone configuration which take effect in standalone mode, but you are in local mode. Can you just start pyspark using "bin/pyspark --master local[4]" ? On Wed, May 11, 2016 at 3:52 AM, Imran Akbar

Spark crashes with Filesystem recovery

2016-05-10 Thread Imran Akbar
I have some Python code that consistently ends up in this state: ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server Traceback (most recent call last): File "/home/ubuntu/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 690, in start