[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009133#comment-16009133
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao closed the pull request at:

https://github.com/apache/flink/pull/1121


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973986#comment-14973986
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-151092573
  
hi @rmetzger ,
I believe I find the reason why my test failed.
A test will fail if we don’t specify “-t” argument(_which is an optional 
argument_) in the command when call runWithArgs function, and we will get 
exception here:
```
Exception in thread "main" java.lang.NoClassDefFoundError: 
org/apache/log4j/Level
at org.apache.hadoop.mapred.JobConf.(JobConf.java:357)
```
It seems some jar files relevant to log callback are needed.

One more thing, I noticed a segment of code as follows in the file 
“FlinkYarnSessionCli.java”:
```
File logback = new File(confDirPath + File.pathSeparator + 
CONFIG_FILE_LOGBACK_NAME);
if (logback.exists()) {
shipFiles.add(logback);
flinkYarnClient.setFlinkLoggingConfigurationPath(new 
Path(logback.toURI()));
}
```
And I wonder if “**File.pathSeparator**” should be “**File.separator**” 
here.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973216#comment-14973216
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-150919886
  
Hi rmetzger,
I have updated this PR.
I generated a flink-conf.yaml in a directory, which I then used as the 
configuration directory for the test. 


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973262#comment-14973262
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-150928188
  
Sorry for the failed tests, I'll fix them.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951383#comment-14951383
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-147009266
  
That's indeed a good question. I don't have much time currently, but I'm 
trying to answer you within the next few days.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947842#comment-14947842
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-146371570
  
@rmetzger : I'm sorry for the late reply, I have been on vacation last 
three weeks.
I failed to find a nice way to "generate a flink-conf.yaml just for the 
test", could you please give some tips on this?


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791811#comment-14791811
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-141016403
  
Sorry for the late response, I didn't see this new pull request.

The failed test is okay. The test is known for being unstable.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791826#comment-14791826
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on a diff in the pull request:

https://github.com/apache/flink/pull/1121#discussion_r39725238
  
--- Diff: 
flink-yarn-tests/src/main/java/org/apache/flink/yarn/YARNSessionFIFOITCase.java 
---
@@ -111,6 +111,20 @@ public void testClientStartup() {
}
 
/**
+* Test configuration parameter for default number of yarn containers
+*/
+   @Test
+   public void testDefaultNumberOfTaskManagers() {
+   LOG.info("Starting testDefaultNumberOfTaskManagers()");
--- End diff --

I would generate a flink-conf.yaml just for this test to test the behavior


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791828#comment-14791828
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1121#issuecomment-141019208
  
For others who follow this PR, there was already some discussion on this 
change here: https://github.com/apache/flink/pull/1107


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791824#comment-14791824
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on a diff in the pull request:

https://github.com/apache/flink/pull/1121#discussion_r39725184
  
--- Diff: flink-dist/src/main/resources/flink-conf.yaml ---
@@ -130,6 +130,15 @@ state.backend: jobmanager
 
 
 
#==
+# YARN

+#==
+
+# Default number of YARN container to allocate (=Number of Task Managers)
+
+yarn.defaultNumberOfTaskManagers: 1
--- End diff --

Can you remove the configuration value from the default configuration again?

I would like to force new users to specify the number of yarn containers 
when they start Flink on YARN.
The configuration value is meant for production users which want to control 
everything using configuration values.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740690#comment-14740690
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao closed the pull request at:

https://github.com/apache/flink/pull/1107


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740689#comment-14740689
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-139531959
  
I'm sorry that something went wrong with my original repository. So, I open 
a new PR, and this one will be closed.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740682#comment-14740682
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

GitHub user willmiao opened a pull request:

https://github.com/apache/flink/pull/1121

[FLINK-2591] Add configuration parameter for default number of yarn 
containers



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/willmiao/flink FLINK-2591

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1121.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1121


commit 6c6005d6ea92177b4a99183f2509a480718dabb2
Author: Will Miao 
Date:   2015-09-10T23:34:35Z

[FLINK-2591]Add configuration parameter for default number of yarn 
containers

commit 154e486620469ffadb9fa9caed0f27f69fbe04ad
Author: Will Miao 
Date:   2015-09-11T12:09:09Z

[FLINK-2591] Add configuration parameter for default number of yarn 
containers




> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736706#comment-14736706
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-138885088
  
I agree with your proposal! If `yarn.defaultNumberOfTaskManagers` is set to 
an invalid value, we should make `-n` required.


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734980#comment-14734980
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

GitHub user willmiao opened a pull request:

https://github.com/apache/flink/pull/1107

[FLINK-2591]Add configuration parameter for default number of yarn 
containers

https://issues.apache.org/jira/browse/FLINK-2591
Add configuration parameter for default number of yarn containers

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/willmiao/flink master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1107.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1107


commit c1d65cae4e301a1483929b360ddcc96ad34eff31
Author: Will Miao 
Date:   2015-09-08T14:58:38Z

FLINK-2591

https://issues.apache.org/jira/browse/FLINK-2591




> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734990#comment-14734990
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-138600722
  
Thanks a lot for opening the pull request.
Can you change the behavior of this a little bit to make the `-n` argument 
a required argument again? (if the config param is not set)
I think this behavior is the best:
if `yarn.numberOfTaskManagers` == -1 (default value): users have to supply 
the `-n` argument.
If `yarn.numberOfTaskManagers` is set to `>= 1`, the value will be used by 
default.
If `yarn.numberOfTaskManagers` is set to `>= 1` and the user specified 
`-n`, the value of `-n` will have priority.

Can you rename the configuration value to 
`yarn.defaultNumberOfTaskManagers`?

Can you add a test into the `flink-yarn-tests` which is using a 
configuration file with `yarn.defaultNumberOfTaskManagers` set to `1` WITHOUT a 
`-n` argument to see if everything is working as expected?


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735013#comment-14735013
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-138603922
  
Thanks for your advices, I'll fix it


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736118#comment-14736118
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-138772421
  
Hi rmetzger, I've got a question: what if yarn.defaultNumberOfTaskManagers 
is set to a invalid value, say -2? Should user have to supply the -n argument?


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736127#comment-14736127
 ] 

ASF GitHub Bot commented on FLINK-2591:
---

Github user willmiao commented on the pull request:

https://github.com/apache/flink/pull/1107#issuecomment-138774833
  
Hi rmetzger,
 I've got a question: what if yarn.defaultNumberOfTaskManagers is set to a 
invalid value, say -2? Should users have to supply the -n argument?
I was thinking this behavior may be appropriate:
If yarn.defaultNumberOfTaskManagers < 1 (not set or invalide value): users 
have to supply the -n argument.
else if the user specified -n, the value of -n will have priority.
else, yarn.defaultNumberOfTaskManagers value will be used by default.

What do you think?


> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[Will Miao (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734977#comment-14734977
 ] 

Will Miao commented on FLINK-2591:
--

Hi, if no one is working on this issue, I would like to work.

> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[Robert Metzger (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734983#comment-14734983
 ] 

Robert Metzger commented on FLINK-2591:
---

Thank you. I'll assign you to the JIRA.

> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2591) Add configuration parameter for default number of yarn containers

[Will Miao (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734987#comment-14734987
 ] 

Will Miao commented on FLINK-2591:
--

Thanks, and I've submitted a pull request on this. Would you please review it 
when you have a time?

> Add configuration parameter for default number of yarn containers
> -
>
> Key: FLINK-2591
> URL: https://issues.apache.org/jira/browse/FLINK-2591
> Project: Flink
>  Issue Type: Improvement
>  Components: YARN Client
>Reporter: Robert Metzger
>Assignee: Will Miao
>Priority: Minor
>  Labels: starter
>
> A user complained about the requirement to always specify the number of yarn 
> containers (-n) when starting a job.
> Adding a configuration value with a default value will allow users to set a 
> default ;)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)