: HA support for Spark
I think that if we were able to maintain the full set of created RDDs as
well as some scheduler and block manager state, it would be enough for most
apps to recover.
On Wed, Dec 10, 2014 at 5:30 AM, Jun Feng Liu liuj...@cn.ibm.com wrote:
Well, it should not be mission
China
Tathagata Das tathagata.das1...@gmail.com
2014/12/11 20:20
To
Jun Feng Liu/China/IBM@IBMCN,
cc
Sandy Ryza sandy.r...@cloudera.com, dev@spark.apache.org
dev@spark.apache.org, Reynold Xin r...@databricks.com
Subject
Re: HA support for Spark
Spark Streaming essentially does
This would be plausible for specific purposes such as Spark streaming or
Spark SQL, but I don't think it is doable for general Spark driver since it
is just a normal JVM process with arbitrary program state.
On Wed, Dec 10, 2014 at 12:25 AM, Jun Feng Liu liuj...@cn.ibm.com wrote:
Do we have any
BLD 28,ZGC Software Park
No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193
China
Reynold Xin r...@databricks.com
2014/12/10 16:30
To
Jun Feng Liu/China/IBM@IBMCN,
cc
dev@spark.apache.org dev@spark.apache.org
Subject
Re: HA support for Spark
This would be plausible
@spark.apache.org dev@spark.apache.org
Subject
Re: HA support for Spark
This would be plausible for specific purposes such as Spark streaming or
Spark SQL, but I don't think it is doable for general Spark driver since it
is just a normal JVM process with arbitrary program state.
On Wed, Dec 10
...@databricks.com,
dev@spark.apache.org
dev@spark.apache.org
Subject
Re: HA support for Spark
