This blog / tutorial <https://alluxio.com/blog/getting-started-with-alluxio-and-spark> maybe helpful to run Spark in the Cloud with Alluxio.
Best regards, Haoyuan On Mon, Nov 20, 2017 at 2:12 PM, lucas.g...@gmail.com <lucas.g...@gmail.com> wrote: > That sounds like allot of work and if I understand you correctly it sounds > like a piece of middleware that already exists (I could be wrong). Alluxio? > > Good luck and let us know how it goes! > > Gary > > On 20 November 2017 at 14:10, Jim Carroll <jimfcarr...@gmail.com> wrote: > >> Thanks. In the meantime I might just write a custom file system that maps >> writes to parquet file parts to their final locations and then skips the >> move. >> >> >> >> -- >> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> >
