Re: Can I set the Alluxio WriteType in Spark applications?

2019-09-19 Thread Bin Fan
Hi Mark, You can follow the instructions here: https://docs.alluxio.io/os/user/stable/en/compute/Spark.html#customize-alluxio-user-properties-for-individual-spark-jobs Something like this: $ spark-submit \--conf 'spark.driver.extraJavaOptions=-Dalluxio.user.file.writetype.default=CACHE_THROUGH'

Can I set the Alluxio WriteType in Spark applications?

2019-09-17 Thread Mark Zhao
Hi, If Spark applications write data into alluxio, can WriteType be configured? Thanks, Mark