Use several jobs and orchestrate them, e.g. Via Ozzie. These jobs then can save intermediate results to disk and load them from there. Alternatively (or additionally!) you may use persist (to memory and disk), but I am not sure this is suitable for such long running applications.
> On 12. May 2017, at 08:54, issues solution <issues.solut...@gmail.com> wrote: > > Hi , > > often we preform a grid search and Cross validation under pyspark to find > best perameters , > but when you have in error not related to computation but to networks or any > think else . > > > HOW WE CAN SAVE INTERMADAITE RESULT ,particulary when you have a large > process during 3 or 4 days ???? > > > THX --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
