[jira] [Created] (TWILL-241) Allow specifying reserved off-heap memory and extra JVM options per runnable
Andreas Neumann created TWILL-241: - Summary: Allow specifying reserved off-heap memory and extra JVM options per runnable Key: TWILL-241 URL: https://issues.apache.org/jira/browse/TWILL-241 Project: Apache Twill Issue Type: Improvement Components: api, yarn Affects Versions: 0.11.0 Reporter: Andreas Neumann Assignee: Terence Yim Fix For: 0.12.0 Sometimes, a particular runnable needs a lot more off-heap memory than others. It would therefore be useful to specify the amount of reserved non-heap memory per runnable. Similarly, for example, for debugging purposes, it may be necessary to add a JVM option for one of the runnables without affecting the other runnables of the same application. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (TWILL-61) Second launch attempt of AM always failed
[
https://issues.apache.org/jira/browse/TWILL-61?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Neumann reopened TWILL-61:
--
Seems that this is not fixed yet. We are seeing this in 0.8:
{noformat}
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists for
/SERVICENAME.REDACTED/144ef027-672b-4a87-b88e-6b7abcf35c2c/runnables
at
com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at
org.apache.twill.internal.appmaster.ApplicationMasterService.doStart(ApplicationMasterService.java:240)
at
org.apache.twill.internal.AbstractTwillService.startUp(AbstractTwillService.java:171)
at
com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47)
{noformat}
> Second launch attempt of AM always failed
> -
>
> Key: TWILL-61
> URL: https://issues.apache.org/jira/browse/TWILL-61
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Reporter: Terence Yim
>Assignee: Terence Yim
> Fix For: 0.5.0-incubating
>
>
> YARN would make multiple attempts to launch an application. Currently second
> or above attempts would always fail due to creation of /runId/state node in
> ZK fail (node exists) because runId is generated on client side and doesn't
> change between attempts.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (TWILL-208) Location should have a way to set permissions when creating directories
[
https://issues.apache.org/jira/browse/TWILL-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Neumann resolved TWILL-208.
---
Resolution: Fixed
> Location should have a way to set permissions when creating directories
> ---
>
> Key: TWILL-208
> URL: https://issues.apache.org/jira/browse/TWILL-208
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Andreas Neumann
>Assignee: Andreas Neumann
> Fix For: 0.10.0
>
>
> That is, we need to introduce a method
> {code}
> boolean mkdirs(String permissions);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TWILL-209) Location test should validate permissions
Andreas Neumann created TWILL-209:
-
Summary: Location test should validate permissions
Key: TWILL-209
URL: https://issues.apache.org/jira/browse/TWILL-209
Project: Apache Twill
Issue Type: Improvement
Components: yarn
Affects Versions: 0.10.0
Reporter: Andreas Neumann
The HDFSLocationtest and FileContextLocationTest use MiniDFSCluster. This does
not seem to enforce permissions when creating directories. The followng test
should throw an exception due to lack of permission but it does not:
{code}
public void testPermissions() throws IOException {
// create a directory that does not permit anything
dfsCluster.getFileSystem().mkdir(new Path("/a"),
FsPermission.valueOf("--"));
// creating a subdir should fail because even the owner has no write
permission
dfsCluster.getFileSystem().mkdir(new Path("/a/b"),
FsPermission.valueOf("--"));
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TWILL-208) Location should have a way to set permissions when creating directories
Andreas Neumann created TWILL-208:
-
Summary: Location should have a way to set permissions when
creating directories
Key: TWILL-208
URL: https://issues.apache.org/jira/browse/TWILL-208
Project: Apache Twill
Issue Type: Bug
Reporter: Andreas Neumann
Assignee: Andreas Neumann
Fix For: 0.10.0
That is, we need to introduce a method
{code}
boolean mkdirs(String permissions);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TWILL-205) Location should have a way to set the group ownership
Andreas Neumann created TWILL-205: - Summary: Location should have a way to set the group ownership Key: TWILL-205 URL: https://issues.apache.org/jira/browse/TWILL-205 Project: Apache Twill Issue Type: Bug Components: core Affects Versions: 0.9.0 Reporter: Andreas Neumann Assignee: Andreas Neumann Fix For: 0.10.0 This is needed to control access to files, which is often done by giving group permissions. Location already has a way to set the permissions, but it would be good if the group id could be changed, too. Also it would be useful to get the owner and group of a location. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
