[ https://issues.apache.org/jira/browse/SPARK-28095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emma Dickson updated SPARK-28095: --------------------------------- Summary: Pyspark with kubernetes doesn't parse arguments with spaces as expected. (was: Pyspark on kubernetes doesn't parse arguments with spaces as expected.) > Pyspark with kubernetes doesn't parse arguments with spaces as expected. > ------------------------------------------------------------------------ > > Key: SPARK-28095 > URL: https://issues.apache.org/jira/browse/SPARK-28095 > Project: Spark > Issue Type: Bug > Components: Kubernetes, PySpark > Affects Versions: 2.4.3 > Environment: Python 2.7.13 > Spark 2.4.3 > Kubernetes > > Reporter: Emma Dickson > Priority: Minor > Labels: newbie, usability > > When passing in arguments to a bash script that calls a python file which > sets up a pyspark context strings with spaces are processed as individual > strings even when encased in double quotes, using backslashes or unicode > escape characters. > > Example > Command entered > {code:java} > ./scripts/spark-k8s.sh v0.0.32 --job-args > "cos://waas-logentries.mycos/Logentries/IBM-b634032e/Github/Load Balancer" > --job pages{code} > > Error Message > > {code:java} > + exec /sbin/tini -s -- /opt/spark/bin/spark-submit --conf > spark.driver.bindAddress=172.30.83.253 --deploy-mode client --properties-file > /opt/spark/conf/spark.properties --class org.apache.spark.deploy.PythonRunner > /opt/spark/work-dir/main.py --job-args > cos://waas-logentries.mycos/Logentries/IBM-b634032e/Github/Load Balancer > --job pages > 19/06/18 19:28:35 WARN Utils: Kubernetes master URL uses HTTP instead of > HTTPS. > 19/06/18 19:28:36 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > usage: main.py [-h] --job JOB --job-args JOB_ARGS > main.py: error: unrecognized arguments: Balancer > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org