[jira] [Resolved] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Neumann resolved TWILL-208.
---
Resolution: Fixed
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840748#comment-15840748
]
ASF GitHub Bot commented on TWILL-208:
--
Github user asfgit closed the pull request at:
https://github.com/apache/twill/pull/26
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user asfgit closed the pull request at: https://github.com/apache/twill/pull/26 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840747#comment-15840747
]
ASF GitHub Bot commented on TWILL-208:
--
Github user anew commented on the issue:
https://github.com/apache/twill/pull/26
squashing one more time
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user anew commented on the issue: https://github.com/apache/twill/pull/26 squashing one more time --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user poornachandra commented on the issue: https://github.com/apache/twill/pull/26 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840710#comment-15840710
]
ASF GitHub Bot commented on TWILL-208:
--
Github user poornachandra commented on the issue:
https://github.com/apache/twill/pull/26
LGTM
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840683#comment-15840683
]
ASF GitHub Bot commented on TWILL-208:
--
Github user chtyim commented on a diff in the pull request:
https://github.com/apache/twill/pull/26#discussion_r98118427
--- Diff:
twill-yarn/src/test/java/org/apache/twill/filesystem/FileContextLocationTest.java
---
@@ -17,26 +17,36 @@
*/
package org.apache.twill.filesystem;
+import com.google.common.base.Throwables;
import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.hadoop.security.UserGroupInformation;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import java.io.IOException;
+import java.security.PrivilegedAction;
/**
*
*/
public class FileContextLocationTest extends LocationTestBase {
public static MiniDFSCluster dfsCluster;
+ private static UserGroupInformation testUGI;
@BeforeClass
public static void init() throws IOException {
Configuration conf = new Configuration();
-conf.setBoolean("hdfs.permissions", true);
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR,
tmpFolder.newFolder().getAbsolutePath());
dfsCluster = new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
+// make rooot world-writable so that we can create all location
factories as unprivileged user
--- End diff --
root
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user chtyim commented on a diff in the pull request:
https://github.com/apache/twill/pull/26#discussion_r98118427
--- Diff:
twill-yarn/src/test/java/org/apache/twill/filesystem/FileContextLocationTest.java
---
@@ -17,26 +17,36 @@
*/
package org.apache.twill.filesystem;
+import com.google.common.base.Throwables;
import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.hadoop.security.UserGroupInformation;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import java.io.IOException;
+import java.security.PrivilegedAction;
/**
*
*/
public class FileContextLocationTest extends LocationTestBase {
public static MiniDFSCluster dfsCluster;
+ private static UserGroupInformation testUGI;
@BeforeClass
public static void init() throws IOException {
Configuration conf = new Configuration();
-conf.setBoolean("hdfs.permissions", true);
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR,
tmpFolder.newFolder().getAbsolutePath());
dfsCluster = new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
+// make rooot world-writable so that we can create all location
factories as unprivileged user
--- End diff --
root
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[
https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840427#comment-15840427
]
ASF GitHub Bot commented on TWILL-181:
--
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98088342
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
Instance ids for runnable instances go from `0` to `num-instances - 1`. If
an instance gets restarted, then it retains the same instance id. Using this
instance id we can track the number of times a particular instance was
restarted.
You can get the instance id in `handleCompleted` method by using
`getInstanceId(RunId runId)` method.
> Control the maximum number of retries for failed application starts
> ---
>
> Key: TWILL-181
> URL: https://issues.apache.org/jira/browse/TWILL-181
> Project: Apache Twill
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 0.7.0-incubating
>Reporter: Martin Serrano
>Assignee: Martin Serrano
> Fix For: 0.10.0
>
>
> If an application consistently exits with a non-zero code, twill will
> attempt to restart indefinitely. I ran into this issue and a list search
> also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4].
> There should be a mechanism to specify the maximum number of retries until
> the application fails. Ideally by default there would be a non-infinite
> maximum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #23: (TWILL-181) allow setting the maximum number of retr...
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98088342
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
Instance ids for runnable instances go from `0` to `num-instances - 1`. If
an instance gets restarted, then it retains the same instance id. Using this
instance id we can track the number of times a particular instance was
restarted.
You can get the instance id in `handleCompleted` method by using
`getInstanceId(RunId runId)` method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[
https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840382#comment-15840382
]
ASF GitHub Bot commented on TWILL-181:
--
Github user serranom commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98083899
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
from my analysis the association of instance ids does not necessarily
correspond to specific processes. it looked like there is a pool of requests,
a new request being serviced gets the lowest instance id and a failed request
gets put back on the queue. this is why i went with an instance adjusted
number of retries. did i miss something?
> Control the maximum number of retries for failed application starts
> ---
>
> Key: TWILL-181
> URL: https://issues.apache.org/jira/browse/TWILL-181
> Project: Apache Twill
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 0.7.0-incubating
>Reporter: Martin Serrano
>Assignee: Martin Serrano
> Fix For: 0.10.0
>
>
> If an application consistently exits with a non-zero code, twill will
> attempt to restart indefinitely. I ran into this issue and a list search
> also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4].
> There should be a mechanism to specify the maximum number of retries until
> the application fails. Ideally by default there would be a non-infinite
> maximum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #23: (TWILL-181) allow setting the maximum number of retr...
Github user serranom commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98083899
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
from my analysis the association of instance ids does not necessarily
correspond to specific processes. it looked like there is a pool of requests,
a new request being serviced gets the lowest instance id and a failed request
gets put back on the queue. this is why i went with an instance adjusted
number of retries. did i miss something?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840321#comment-15840321
]
ASF GitHub Bot commented on TWILL-208:
--
Github user anew commented on the issue:
https://github.com/apache/twill/pull/26
Squashing commits now.
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user anew commented on the issue: https://github.com/apache/twill/pull/26 Squashing commits now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840320#comment-15840320
]
ASF GitHub Bot commented on TWILL-208:
--
Github user poornachandra commented on the issue:
https://github.com/apache/twill/pull/26
LGTM
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user poornachandra commented on the issue: https://github.com/apache/twill/pull/26 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user poornachandra commented on the issue: https://github.com/apache/twill/pull/26 Just one comment on javadoc, please fix and merge ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840317#comment-15840317
]
ASF GitHub Bot commented on TWILL-208:
--
Github user poornachandra commented on the issue:
https://github.com/apache/twill/pull/26
Just one comment on javadoc, please fix and merge 👍
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[
https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840301#comment-15840301
]
ASF GitHub Bot commented on TWILL-181:
--
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98075925
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
On further thought, I think if we track the restarts per instance id then
it would simplify the contract for max retries. This will be consistent with
what the javadoc says too -
```
Sets the maximum number of times (per instance) a runnable will be retried
if it exits without success. The default behavior is to retry indefinitely.
```
> Control the maximum number of retries for failed application starts
> ---
>
> Key: TWILL-181
> URL: https://issues.apache.org/jira/browse/TWILL-181
> Project: Apache Twill
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 0.7.0-incubating
>Reporter: Martin Serrano
>Assignee: Martin Serrano
> Fix For: 0.10.0
>
>
> If an application consistently exits with a non-zero code, twill will
> attempt to restart indefinitely. I ran into this issue and a list search
> also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4].
> There should be a mechanism to specify the maximum number of retries until
> the application fails. Ideally by default there would be a non-infinite
> maximum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #23: (TWILL-181) allow setting the maximum number of retr...
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r98075925
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
On further thought, I think if we track the restarts per instance id then
it would simplify the contract for max retries. This will be consistent with
what the javadoc says too -
```
Sets the maximum number of times (per instance) a runnable will be retried
if it exits without success. The default behavior is to retry indefinitely.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840260#comment-15840260
]
ASF GitHub Bot commented on TWILL-208:
--
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/26#discussion_r98070266
--- Diff:
twill-common/src/main/java/org/apache/twill/filesystem/LocalLocation.java ---
@@ -251,6 +251,34 @@ public boolean mkdirs() throws IOException {
return file.mkdirs();
}
+ @Override
+ public boolean mkdirs(String permission) throws IOException {
--- End diff --
I see, in that case I think it would be good to add this to the javadoc of
the method.
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/26#discussion_r98070266
--- Diff:
twill-common/src/main/java/org/apache/twill/filesystem/LocalLocation.java ---
@@ -251,6 +251,34 @@ public boolean mkdirs() throws IOException {
return file.mkdirs();
}
+ @Override
+ public boolean mkdirs(String permission) throws IOException {
--- End diff --
I see, in that case I think it would be good to add this to the javadoc of
the method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840171#comment-15840171
]
ASF GitHub Bot commented on TWILL-208:
--
Github user gokulavasan commented on the issue:
https://github.com/apache/twill/pull/26
LGTM 👍
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill issue #26: (TWILL-208) add Location.mkdirs(String permissions)
Github user gokulavasan commented on the issue: https://github.com/apache/twill/pull/26 LGTM ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[ https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839679#comment-15839679 ] ASF GitHub Bot commented on TWILL-181: -- Github user serranom commented on the issue: https://github.com/apache/twill/pull/23 @hsaputra, since I have a bug to fix, the updated request will trigger the rebuild. thanks. > Control the maximum number of retries for failed application starts > --- > > Key: TWILL-181 > URL: https://issues.apache.org/jira/browse/TWILL-181 > Project: Apache Twill > Issue Type: Improvement > Components: yarn >Affects Versions: 0.7.0-incubating >Reporter: Martin Serrano >Assignee: Martin Serrano > Fix For: 0.10.0 > > > If an application consistently exits with a non-zero code, twill will > attempt to restart indefinitely. I ran into this issue and a list search > also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4]. > There should be a mechanism to specify the maximum number of retries until > the application fails. Ideally by default there would be a non-infinite > maximum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] twill issue #23: (TWILL-181) allow setting the maximum number of retries per...
Github user serranom commented on the issue: https://github.com/apache/twill/pull/23 @hsaputra, since I have a bug to fix, the updated request will trigger the rebuild. thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill pull request #23: (TWILL-181) allow setting the maximum number of retr...
Github user serranom commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r97994162
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
Good catch. Yes. I missed that. I will update the pull request with a
fix. When the number of instances increases, `deltaInstances * maxRetries`
will be added to this instance adjusted maximum. When the number of instances
decreases, the `deltaInstance*maxRetries` will be subtracted from the instance
adjusted maximum. Additionally upon a decrease, if the number of retries
exceeds the instance adjusted maximum it will be set to the instance adjusted
maximum.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[
https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839677#comment-15839677
]
ASF GitHub Bot commented on TWILL-181:
--
Github user serranom commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r97994162
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
Good catch. Yes. I missed that. I will update the pull request with a
fix. When the number of instances increases, `deltaInstances * maxRetries`
will be added to this instance adjusted maximum. When the number of instances
decreases, the `deltaInstance*maxRetries` will be subtracted from the instance
adjusted maximum. Additionally upon a decrease, if the number of retries
exceeds the instance adjusted maximum it will be set to the instance adjusted
maximum.
> Control the maximum number of retries for failed application starts
> ---
>
> Key: TWILL-181
> URL: https://issues.apache.org/jira/browse/TWILL-181
> Project: Apache Twill
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 0.7.0-incubating
>Reporter: Martin Serrano
>Assignee: Martin Serrano
> Fix For: 0.10.0
>
>
> If an application consistently exits with a non-zero code, twill will
> attempt to restart indefinitely. I ran into this issue and a list search
> also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4].
> There should be a mechanism to specify the maximum number of retries until
> the application fails. Ideally by default there would be a non-infinite
> maximum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] twill pull request #23: (TWILL-181) allow setting the maximum number of retr...
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r97944230
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
The max retries count in this map will have to be updated when the instance
count of a runnable changes, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts
[
https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839417#comment-15839417
]
ASF GitHub Bot commented on TWILL-181:
--
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/twill/pull/23#discussion_r97944230
--- Diff:
twill-yarn/src/main/java/org/apache/twill/internal/appmaster/RunningContainers.java
---
@@ -113,9 +117,11 @@ public Integer apply(BitSet input) {
private final Location applicationLocation;
private final Set runnableNames;
private final Map> logLevels;
+ private final Map maxRetries;
--- End diff --
The max retries count in this map will have to be updated when the instance
count of a runnable changes, right?
> Control the maximum number of retries for failed application starts
> ---
>
> Key: TWILL-181
> URL: https://issues.apache.org/jira/browse/TWILL-181
> Project: Apache Twill
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 0.7.0-incubating
>Reporter: Martin Serrano
>Assignee: Martin Serrano
> Fix For: 0.10.0
>
>
> If an application consistently exits with a non-zero code, twill will
> attempt to restart indefinitely. I ran into this issue and a list search
> also reveals [others| http://markmail.org/message/dehx7r6tpqgcmjh4].
> There should be a mechanism to specify the maximum number of retries until
> the application fails. Ideally by default there would be a non-infinite
> maximum.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
