[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365480#comment-16365480 ] lovekesh bansal commented on YARN-7328: --- [~leftnoteasy] Uploading the second patch with the test cases. Please review. Also Not sure but auto build was not triggered after the first upload, am I missing something? > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch, YARN-7328_trunk.002.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7328: -- Attachment: YARN-7328_trunk.002.patch > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch, YARN-7328_trunk.002.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365476#comment-16365476 ] lovekesh bansal commented on YARN-7901: --- [~leftnoteasy] Can you please review? Thanks. > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.0.1 > > Attachments: YARN-7901_trunk.001.patch > > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7901: -- Attachment: (was: YARN-7328_trunk.002.patch) > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.0.1 > > Attachments: YARN-7901_trunk.001.patch > > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7901: -- Attachment: YARN-7328_trunk.002.patch > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.0.1 > > Attachments: YARN-7901_trunk.001.patch > > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365387#comment-16365387 ] lovekesh bansal edited comment on YARN-7328 at 2/15/18 11:13 AM: - Thanks [~leftnoteasy] Ya I'll add test cases shortly. was (Author: lovekesh.bansal): [~leftnoteasy] Ya I'll add test cases shortly. > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365387#comment-16365387 ] lovekesh bansal commented on YARN-7328: --- [~leftnoteasy] Ya I'll add test cases shortly. > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363540#comment-16363540 ] lovekesh bansal edited comment on YARN-7328 at 2/14/18 6:16 AM: I have attached the first patch with just disabling memory-mb and vcores. And since we are throwing exception for all mandatory resources, we should addMandatoryResources only after checkMandatoryResources in getNodeResourceInformation. Please let me know your thoughts, accordingly I'll change the test cases. was (Author: lovekesh.bansal): I have attached the first patch with just disabling memory-mb and vcores. And since we are throwing exception for all mandatory resources, we should addMandatoryResources only after checkMandatoryResources in getNodeResourceInformation. > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363540#comment-16363540 ] lovekesh bansal commented on YARN-7328: --- I have attached the first patch with just disabling memory-mb and vcores. And since we are throwing exception for all mandatory resources, we should addMandatoryResources only after checkMandatoryResources in getNodeResourceInformation. > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal reassigned YARN-7328: - Assignee: lovekesh bansal > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Assignee: lovekesh bansal >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7328: -- Attachment: YARN-7328_trunk.001.patch > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Priority: Critical > Attachments: YARN-7328_trunk.001.patch > > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[
https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363536#comment-16363536
]
lovekesh bansal commented on YARN-7328:
---
[~templedf] [~leftnoteasy] I think some of the things have changed since when
the Jira was filed:
# We are currently throwing that exception in checkMandatoryResources for if
it contains memory. By making this change we will be limiting it to be defined
in both resource-types.xml and node-resources.xml. Is that the intent here?
# Now we have some more reosurces in MANDATORY_RESOURCES, so should not allow
of them to be defined in node-resources.xml?
# ResourceUtils.getNodeResourceInformation(conf) is called by
NodeManagerHardwareUtils.
getNodeResources which after calling function checks if(memResInfo.getValue()
== 0) \{ret.setMemorySize(getContainerMemoryMB(conf))}
So then if check would become redundant and we would always be setting the
memory by calling
ret.setMemorySize(getContainerMemoryMB(conf)) .
Let me know your thoughts?
> ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to
> override yarn.nodemanager.resource.memory-mb and .cpu-vcores
> --
>
> Key: YARN-7328
> URL: https://issues.apache.org/jira/browse/YARN-7328
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Affects Versions: 3.1.0
>Reporter: Daniel Templeton
>Priority: Critical
>
> We will throw an exception if yarn.nodemanager.resource-types.memory is
> configured, but not if .memory-mb or .vcores is configured. We should be
> consistent. We should not allow resource types to redefine something for
> which we already have a property to set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360786#comment-16360786 ] lovekesh bansal commented on YARN-7328: --- [~leftnoteasy] I'll do it in this week most probably by Wednesday. > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Priority: Critical > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal reassigned YARN-7901: - Assignee: lovekesh bansal Fix Version/s: 3.0.1 Submitting the patch > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.0.1 > > Attachments: YARN-7901_trunk.001.patch > > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7901: -- Attachment: YARN-7901_trunk.001.patch > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Priority: Minor > Fix For: 3.0.1 > > Attachments: YARN-7901_trunk.001.patch > > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
[ https://issues.apache.org/jira/browse/YARN-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7901: -- Issue Type: Sub-task (was: Bug) Parent: YARN-7069 > Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy > > > Key: YARN-7901 > URL: https://issues.apache.org/jira/browse/YARN-7901 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: lovekesh bansal >Priority: Minor > > in the splitIndividualAny method while creating the resourceRequest we are > not setting the profile capability. > ResourceRequest.newInstance(originalResourceRequest.getPriority(), > originalResourceRequest.getResourceName(), > originalResourceRequest.getCapability(), > originalResourceRequest.getNumContainers(), > originalResourceRequest.getRelaxLocality(), > originalResourceRequest.getNodeLabelExpression(), > originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7901) Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy
lovekesh bansal created YARN-7901: - Summary: Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy Key: YARN-7901 URL: https://issues.apache.org/jira/browse/YARN-7901 Project: Hadoop YARN Issue Type: Bug Reporter: lovekesh bansal in the splitIndividualAny method while creating the resourceRequest we are not setting the profile capability. ResourceRequest.newInstance(originalResourceRequest.getPriority(), originalResourceRequest.getResourceName(), originalResourceRequest.getCapability(), originalResourceRequest.getNumContainers(), originalResourceRequest.getRelaxLocality(), originalResourceRequest.getNodeLabelExpression(), originalResourceRequest.getExecutionTypeRequest()); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7722) Rename variables in MockNM, MockRM for better clarity
[ https://issues.apache.org/jira/browse/YARN-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321192#comment-16321192 ] lovekesh bansal commented on YARN-7722: --- Thanks [~sunilg] for committing. You are right, it is because of the YARN-7237 which is merged in trunk but not in 3.0. > Rename variables in MockNM, MockRM for better clarity > - > > Key: YARN-7722 > URL: https://issues.apache.org/jira/browse/YARN-7722 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Trivial > Fix For: 3.1.0 > > Attachments: YARN-7722_trunk.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7722) Correcting the capability in MockNM, MockRM test classes
[ https://issues.apache.org/jira/browse/YARN-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal reassigned YARN-7722: - Assignee: lovekesh bansal Target Version/s: 3.0.0 [~sunilg] Request you to please review. > Correcting the capability in MockNM, MockRM test classes > > > Key: YARN-7722 > URL: https://issues.apache.org/jira/browse/YARN-7722 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Attachments: YARN-7722_trunk.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7722) Correcting the capability in MockNM, MockRM test classes
[ https://issues.apache.org/jira/browse/YARN-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7722: -- Attachment: YARN-7722_trunk.001.patch attaching patching for the same. > Correcting the capability in MockNM, MockRM test classes > > > Key: YARN-7722 > URL: https://issues.apache.org/jira/browse/YARN-7722 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: lovekesh bansal >Priority: Minor > Attachments: YARN-7722_trunk.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7722) Correcting the capability in MockNM, MockRM test classes
lovekesh bansal created YARN-7722: - Summary: Correcting the capability in MockNM, MockRM test classes Key: YARN-7722 URL: https://issues.apache.org/jira/browse/YARN-7722 Project: Hadoop YARN Issue Type: Improvement Reporter: lovekesh bansal Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7328) ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores
[ https://issues.apache.org/jira/browse/YARN-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309466#comment-16309466 ] lovekesh bansal commented on YARN-7328: --- [~templedf] If you are not working on it, can I contribute here? > ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to > override yarn.nodemanager.resource.memory-mb and .cpu-vcores > -- > > Key: YARN-7328 > URL: https://issues.apache.org/jira/browse/YARN-7328 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0 >Reporter: Daniel Templeton >Priority: Critical > > We will throw an exception if yarn.nodemanager.resource-types.memory is > configured, but not if .memory-mb or .vcores is configured. We should be > consistent. We should not allow resource types to redefine something for > which we already have a property to set. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6971) Clean up different ways to create resources
[
https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309324#comment-16309324
]
lovekesh bansal edited comment on YARN-6971 at 1/3/18 9:01 AM:
---
[~sunilg] [~yufeigu] Since 3926 is already completed, What is the current
status of this and are you working on the same? If not, can I work on the same?
was (Author: lovekesh.bansal):
@Yufei Gu are you working on this? If not, can I work on the same?
> Clean up different ways to create resources
> ---
>
> Key: YARN-6971
> URL: https://issues.apache.org/jira/browse/YARN-6971
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager, scheduler
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>Priority: Minor
> Labels: newbie
>
> There are several ways to create a {{resource}} object, e.g.,
> BuilderUtils.newResource() and Resources.createResource(). These methods not
> only cause confusing but also performance issues, for example
> BuilderUtils.newResource() is significant slow than
> Resources.createResource().
> We could merge them some how, and replace most BuilderUtils.newResource()
> with Resources.createResource().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6971) Clean up different ways to create resources
[
https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309324#comment-16309324
]
lovekesh bansal commented on YARN-6971:
---
@Yufei Gu are you working on this? If not, can I work on the same?
> Clean up different ways to create resources
> ---
>
> Key: YARN-6971
> URL: https://issues.apache.org/jira/browse/YARN-6971
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager, scheduler
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>Priority: Minor
> Labels: newbie
>
> There are several ways to create a {{resource}} object, e.g.,
> BuilderUtils.newResource() and Resources.createResource(). These methods not
> only cause confusing but also performance issues, for example
> BuilderUtils.newResource() is significant slow than
> Resources.createResource().
> We could merge them some how, and replace most BuilderUtils.newResource()
> with Resources.createResource().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5589) Update CapacitySchedulerConfiguration minimum and maximum calculations to consider all resource types
[ https://issues.apache.org/jira/browse/YARN-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-5589: -- Attachment: YARN-5589_trunk.001.patch uploading patch with code changes. [~sunilg] Can you please review, accordingly I'll change the test cases. Thanks. > Update CapacitySchedulerConfiguration minimum and maximum calculations to > consider all resource types > - > > Key: YARN-5589 > URL: https://issues.apache.org/jira/browse/YARN-5589 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Varun Vasudev > Attachments: YARN-5589_trunk.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[
https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226937#comment-16226937
]
lovekesh bansal commented on YARN-7316:
---
I think passing Resource calculator as an argument should have been included
for some reason, not sure but might be scalability. Otherwise we could have
initialized resource calculator in one single place, be it in {{Resources}}
class or the calculator class. +1 for that we should have all operations on
resources in one single class and but calling functions on calculator class
seems unintuitive to me, Resources class seems appropriate for that purpose.
What do you suggest?
> Cleaning up the usage of Resources and ResourceCalculator
> -
>
> Key: YARN-7316
> URL: https://issues.apache.org/jira/browse/YARN-7316
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.1.0
>Reporter: lovekesh bansal
>Assignee: lovekesh bansal
>Priority: Minor
> Fix For: 3.1.0
>
> Attachments: YARN-7316_trunk.001.patch
>
>
> Cleaning up and Uniformizing the usage the usage of Resources and
> ResourceCalculator.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218450#comment-16218450 ] lovekesh bansal commented on YARN-7316: --- [~sunilg] Yes, It is cleaned up everywhere and Resources.api is being used everywhere. > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218434#comment-16218434 ] lovekesh bansal commented on YARN-7316: --- [~sunilg] Thanks Sunil. Yes all the usage was cleaned up while submitting the patch, though not sure about code checked in after that. Will cross check once. > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5589) Update CapacitySchedulerConfiguration minimum and maximum calculations to consider all resource types
[ https://issues.apache.org/jira/browse/YARN-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16205178#comment-16205178 ] lovekesh bansal commented on YARN-5589: --- [~vvasudev] I am new to the open source protocols. The JIRA is assigned to you. Wanted to know if you Are you actively working on it? If not, may I chip in if you are ok with it? > Update CapacitySchedulerConfiguration minimum and maximum calculations to > consider all resource types > - > > Key: YARN-5589 > URL: https://issues.apache.org/jira/browse/YARN-5589 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7316: -- Issue Type: Sub-task (was: Task) Parent: YARN-7069 > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal reassigned YARN-7316: - Assignee: lovekesh bansal > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Task >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7316: -- Fix Version/s: 3.1.0 > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7316: -- Attachment: YARN-7316_trunk.001.patch Uploading the patch related to YARN-7069 > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7316: -- Affects Version/s: 3.1.0 > Cleaning up the usage of Resources and ResourceCalculator > - > > Key: YARN-7316 > URL: https://issues.apache.org/jira/browse/YARN-7316 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal >Priority: Minor > Fix For: 3.1.0 > > Attachments: YARN-7316_trunk.001.patch > > > Cleaning up and Uniformizing the usage the usage of Resources and > ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7316) Cleaning up the usage of Resources and ResourceCalculator
lovekesh bansal created YARN-7316: - Summary: Cleaning up the usage of Resources and ResourceCalculator Key: YARN-7316 URL: https://issues.apache.org/jira/browse/YARN-7316 Project: Hadoop YARN Issue Type: Task Reporter: lovekesh bansal Priority: Minor Cleaning up and Uniformizing the usage the usage of Resources and ResourceCalculator. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198875#comment-16198875 ] lovekesh bansal commented on YARN-7312: --- Ok thanks for guiding me. And I will wait for your comments. > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198825#comment-16198825 ] lovekesh bansal commented on YARN-7312: --- [~sunilg] Ok sure, I could see some of issues very specifically related to Fair scheduler but not covering these test cases. And I am new to apache community. Just for my understanding, in this case, we'll continue with committing the patches or fix those test cases first? > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198789#comment-16198789 ] lovekesh bansal commented on YARN-7312: --- [~sunilg] Thanks for quick reply. Have created the bug YARN-7314 and linked to the parent umbrella. > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7314) Multiple Resource manager test cases failing
[ https://issues.apache.org/jira/browse/YARN-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198775#comment-16198775 ] lovekesh bansal commented on YARN-7314: --- Linking to the parent umbrella for resource profiles. > Multiple Resource manager test cases failing > > > Key: YARN-7314 > URL: https://issues.apache.org/jira/browse/YARN-7314 > Project: Hadoop YARN > Issue Type: Bug >Reporter: lovekesh bansal >Priority: Critical > > Following 14 unit tests are failing in the trunk: > > org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService.testResourceTypes > > > org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testSchedulerRecovery[FAIR] > > > org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testDynamicQueueRecovery[FAIR] > > > org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForFairSche > > > org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForCapacitySche > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt.testHeadroomWithBlackListedNodes > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShare > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testMultipleCompletedEvent > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRefreshQueuesWhenRMHA > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareWithContainerReservation > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testComputeMaxAMResource > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareDefault > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testReservationMetrics > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRequestAMResourceInZeroFairShareQueue > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7314) Multiple Resource manager test cases failing
lovekesh bansal created YARN-7314: - Summary: Multiple Resource manager test cases failing Key: YARN-7314 URL: https://issues.apache.org/jira/browse/YARN-7314 Project: Hadoop YARN Issue Type: Bug Reporter: lovekesh bansal Priority: Critical Following 14 unit tests are failing in the trunk: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService.testResourceTypes org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testSchedulerRecovery[FAIR] org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testDynamicQueueRecovery[FAIR] org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForFairSche org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForCapacitySche org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt.testHeadroomWithBlackListedNodes org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShare org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testMultipleCompletedEvent org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRefreshQueuesWhenRMHA org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareWithContainerReservation org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testComputeMaxAMResource org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareDefault org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testReservationMetrics org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRequestAMResourceInZeroFairShareQueue -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198644#comment-16198644 ] lovekesh bansal commented on YARN-7312: --- [~sunilg] I checked and found that these unit tests have already been failing in the trunk as against it seems to be failing because of the patch. Not sure how to go forward. Can you please guide? > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal reassigned YARN-7312: - Assignee: lovekesh bansal > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7296) convertToProtoFormat(Resource r) is not setting for all resource types
[ https://issues.apache.org/jira/browse/YARN-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198252#comment-16198252 ] lovekesh bansal commented on YARN-7296: --- Thanks [~sunilg] and [~templedf] for reviewing and taking it forward. > convertToProtoFormat(Resource r) is not setting for all resource types > -- > > Key: YARN-7296 > URL: https://issues.apache.org/jira/browse/YARN-7296 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal >Assignee: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7296_YARN-3926.001.patch, > YARN-7296_trunk.001.patch, YARN-7296_trunk.002.patch > > > convertToProtoFormat(Resource r) is not setting all resources in a couple of > files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198186#comment-16198186 ] lovekesh bansal edited comment on YARN-7312 at 10/10/17 5:40 AM: - V1 Patch uploaded for umbrella YARN-7069. was (Author: lovekesh.bansal): V1 Patch uploaded. > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7312: -- Attachment: YARN-7312_trunk.001.patch V1 Patch uploaded. > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7312_trunk.001.patch > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7312: -- Affects Version/s: 3.1.0 > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
[ https://issues.apache.org/jira/browse/YARN-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7312: -- Fix Version/s: 3.1.0 > [Resource Profiles] getMaximumAllocationPerQueue to account for all resources > - > > Key: YARN-7312 > URL: https://issues.apache.org/jira/browse/YARN-7312 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > > getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be > made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7312) [Resource Profiles] getMaximumAllocationPerQueue to account for all resources
lovekesh bansal created YARN-7312: - Summary: [Resource Profiles] getMaximumAllocationPerQueue to account for all resources Key: YARN-7312 URL: https://issues.apache.org/jira/browse/YARN-7312 Project: Hadoop YARN Issue Type: Task Reporter: lovekesh bansal getMaximumAllocationPerQueue has memory and vcores hardcoded. So it should be made generic for all resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7296) [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources
[ https://issues.apache.org/jira/browse/YARN-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7296: -- Attachment: YARN-7296_trunk.002.patch > [Resource Profiles] convertToProtoFormat(Resource r) is not setting all > resources > - > > Key: YARN-7296 > URL: https://issues.apache.org/jira/browse/YARN-7296 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: YARN-3926 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7296_YARN-3926.001.patch, > YARN-7296_trunk.001.patch, YARN-7296_trunk.002.patch > > > convertToProtoFormat(Resource r) is not setting all resources in a couple of > files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7296) [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources
[ https://issues.apache.org/jira/browse/YARN-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7296: -- Attachment: YARN-7296_trunk.001.patch [~sunilg] Thanks for updating me. Please find attached the Patch against the trunk. Request you to please review. > [Resource Profiles] convertToProtoFormat(Resource r) is not setting all > resources > - > > Key: YARN-7296 > URL: https://issues.apache.org/jira/browse/YARN-7296 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: YARN-3926 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7296_YARN-3926.001.patch, YARN-7296_trunk.001.patch > > > convertToProtoFormat(Resource r) is not setting all resources in a couple of > files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7296) [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources
[ https://issues.apache.org/jira/browse/YARN-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7296: -- Description: convertToProtoFormat(Resource r) is not setting all resources in a couple of files i.e. ProfileCapabilityPBImpl.java (was: In the convertToProtoFormat(Resource r) is not setting all resources in a couple of files i.e. ProfileCapabilityPBImpl.java) > [Resource Profiles] convertToProtoFormat(Resource r) is not setting all > resources > - > > Key: YARN-7296 > URL: https://issues.apache.org/jira/browse/YARN-7296 > Project: Hadoop YARN > Issue Type: Task >Affects Versions: YARN-3926 >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7296_YARN-3926.001.patch > > > convertToProtoFormat(Resource r) is not setting all resources in a couple of > files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7296) [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources
[ https://issues.apache.org/jira/browse/YARN-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal updated YARN-7296: -- Attachment: YARN-7296_YARN-3926.001.patch This patch is for resource profiles branch YARN-3926 > [Resource Profiles] convertToProtoFormat(Resource r) is not setting all > resources > - > > Key: YARN-7296 > URL: https://issues.apache.org/jira/browse/YARN-7296 > Project: Hadoop YARN > Issue Type: Task >Reporter: lovekesh bansal > Fix For: 3.1.0 > > Attachments: YARN-7296_YARN-3926.001.patch > > > In the convertToProtoFormat(Resource r) is not setting all resources in a > couple of files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7296) [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources
lovekesh bansal created YARN-7296: - Summary: [Resource Profiles] convertToProtoFormat(Resource r) is not setting all resources Key: YARN-7296 URL: https://issues.apache.org/jira/browse/YARN-7296 Project: Hadoop YARN Issue Type: Task Reporter: lovekesh bansal Fix For: 3.1.0 In the convertToProtoFormat(Resource r) is not setting all resources in a couple of files i.e. ProfileCapabilityPBImpl.java -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7184) Test Jira
[ https://issues.apache.org/jira/browse/YARN-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lovekesh bansal resolved YARN-7184. --- Resolution: Fixed > Test Jira > - > > Key: YARN-7184 > URL: https://issues.apache.org/jira/browse/YARN-7184 > Project: Hadoop YARN > Issue Type: Task > Environment: Test variable is missing in the test class >Reporter: lovekesh bansal > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7184) Test Jira
lovekesh bansal created YARN-7184: - Summary: Test Jira Key: YARN-7184 URL: https://issues.apache.org/jira/browse/YARN-7184 Project: Hadoop YARN Issue Type: Task Environment: Test variable is missing in the test class Reporter: lovekesh bansal -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
