Nicholas Chammas created SPARK-17976:
----------------------------------------
Summary: Global options to spark-submit should not be
position-sensitive
Key: SPARK-17976
URL: https://issues.apache.org/jira/browse/SPARK-17976
Project: Spark
Issue Type: Improvement
Components: Deploy
Affects Versions: 2.0.1, 2.0.0
Reporter: Nicholas Chammas
Priority: Minor
It is maddening that this does what you expect:
{code}
spark-submit --packages graphframes:graphframes:0.2.0-spark2.0-s_2.11 \
file.py
{code}
whereas this doesn't because {{--packages}} is totally ignored:
{code}
spark-submit file.py \
--packages graphframes:graphframes:0.2.0-spark2.0-s_2.11
{code}
Ideally, global options should be valid no matter where they are specified.
If that's too much work, then I think at the very least {{spark-submit}} should
display a warning that some input is being ignored. (Ideally, it should error
out, but that's probably not possible for backwards-compatibility reasons at
this point.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
