Hi,
Alluxio will allow you to share or cache data in-memory between different
Spark contexts by storing RDDs or Dataframes as a file in the Alluxio
system. The files can then be accessed by any Spark job like a file in any
other distributed storage system.
These two blogs do a good job of
Hello Guys,
What would be approach to accomplish Spark Multiple Shared Context without
Alluxio and with with Alluxio , and what would be best practice to achieve
parallelism and concurrency for spark jobs.
Thanks.
--
Yours Aye,
Chetan Khatri.
M.+91 7 80574
Data Science Researcher
INDIA
