[jira] [Commented] (SPARK-1606) spark-submit needs `--arg` for every application parameter

2014-05-03 Thread Mridul Muralidharan (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988756#comment-13988756
 ] 

Mridul Muralidharan commented on SPARK-1606:


Crap, got to this too late.
We really should not have added this - and I would have -1'ed it.

We have had too many issues with trying to parse the user command line and 
getting into all sorts of issues which are entirely avoidable by simply being 
explicit about what the user wants to pass.
Trying to pass strings which contain escape characters and/or whitespace, etc 
is just going to be a nightmare about this change.

> spark-submit needs `--arg` for every application parameter
> --
>
> Key: SPARK-1606
> URL: https://issues.apache.org/jira/browse/SPARK-1606
> Project: Spark
>  Issue Type: Improvement
>  Components: Spark Core
>Affects Versions: 1.0.0
>Reporter: Xiangrui Meng
>Assignee: Patrick Wendell
>Priority: Blocker
> Fix For: 1.0.0
>
>
> If the application has a few parameters, the spark-submit command looks like 
> the following:
> {code}
> spark-submit --master yarn-cluster --class main.Class --arg --numPartitions 
> --arg 8 --arg --kryo --arg true
> {code}
> It is a little bit hard to read and modify. Maybe it is okay to treat all 
> arguments after `main.Class` as application parameters.
> {code}
> spark-submit --master yarn-cluster --class main.Class --numPartitions 8 
> --kryo true
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (SPARK-1606) spark-submit needs `--arg` for every application parameter

2014-04-28 Thread Guoqiang Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13982797#comment-13982797
 ] 

Guoqiang Li  commented on SPARK-1606:
-

[SparkSubmitArguments.scala#L287|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L287]
 
{code}--arg ARG Argument to be passed to your application's main class. This 
option can be specified multiple times for multiple args.
{code} Should be deleted

> spark-submit needs `--arg` for every application parameter
> --
>
> Key: SPARK-1606
> URL: https://issues.apache.org/jira/browse/SPARK-1606
> Project: Spark
>  Issue Type: Improvement
>  Components: Spark Core
>Affects Versions: 1.0.0
>Reporter: Xiangrui Meng
>Assignee: Patrick Wendell
>Priority: Blocker
> Fix For: 1.0.0
>
>
> If the application has a few parameters, the spark-submit command looks like 
> the following:
> {code}
> spark-submit --master yarn-cluster --class main.Class --arg --numPartitions 
> --arg 8 --arg --kryo --arg true
> {code}
> It is a little bit hard to read and modify. Maybe it is okay to treat all 
> arguments after `main.Class` as application parameters.
> {code}
> spark-submit --master yarn-cluster --class main.Class --numPartitions 8 
> --kryo true
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (SPARK-1606) spark-submit needs `--arg` for every application parameter

2014-04-25 Thread Patrick Wendell (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13981851#comment-13981851
 ] 

Patrick Wendell commented on SPARK-1606:


I submitted a PR here that adds the following syntax.

{code}
./bin/spark-submit [options] user.jar [user options]
{code}

> spark-submit needs `--arg` for every application parameter
> --
>
> Key: SPARK-1606
> URL: https://issues.apache.org/jira/browse/SPARK-1606
> Project: Spark
>  Issue Type: Improvement
>  Components: Spark Core
>Affects Versions: 1.0.0
>Reporter: Xiangrui Meng
>Assignee: Patrick Wendell
>Priority: Blocker
>
> If the application has a few parameters, the spark-submit command looks like 
> the following:
> {code}
> spark-submit --master yarn-cluster --class main.Class --arg --numPartitions 
> --arg 8 --arg --kryo --arg true
> {code}
> It is a little bit hard to read and modify. Maybe it is okay to treat all 
> arguments after `main.Class` as application parameters.
> {code}
> spark-submit --master yarn-cluster --class main.Class --numPartitions 8 
> --kryo true
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)