Re: In yarn-cluster mode, provide system prop to the client jvm

2016-06-16 Thread Jacek Laskowski
Hi, You could use --properties-file to point to the properties file with properties or use spark.driver.extraJavaOptions. Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark http://bit.ly/mastering-apache-spark Follow me at

In yarn-cluster mode, provide system prop to the client jvm

2016-06-16 Thread Ellis, Tom (Financial Markets IT)
Hi, I was wondering if it was possible to submit a java system property to the JVM that does the submission of a yarn-cluster application, for instance, -Dlog4j.configuration. I believe it will default to using the SPARK_CONF_DIR's log4j.properties, is it possible to override this, as I do not