Re: Spark Standalone - Failing to pass extra java options to the driver in cluster mode

Thanks Jungtaek Lim, I upgraded the cluster to 2.4.3 and it worked fine. Thanks, Alex On Mon, Aug 19, 2019 at 10:01 PM Jungtaek Lim wrote: > Hi Alex, > > you seem to hit SPARK-26606 [1] which has been fixed in 2.4.1. Could you > try it out with latest version? > > Thanks, > Jungtaek Lim

Re: Spark Standalone - Failing to pass extra java options to the driver in cluster mode

Hi Alex, you seem to hit SPARK-26606 [1] which has been fixed in 2.4.1. Could you try it out with latest version? Thanks, Jungtaek Lim (HeartSaVioR) 1. https://issues.apache.org/jira/browse/SPARK-26606 On Tue, Aug 20, 2019 at 3:43 AM Alex Landa wrote: > Hi, > > We are using Spark Standalone

Spark Standalone - Failing to pass extra java options to the driver in cluster mode

Hi, We are using Spark Standalone 2.4.0 in production and publishing our Scala app using cluster mode. I saw that extra java options passed to the driver don't actually pass. A submit example: *spark-submit --deploy-mode cluster --master spark://:7077 --driver-memory 512mb --conf