[jira] [Commented] (YARN-8595) [UI2] Container diagnostic missing from container page
[ https://issues.apache.org/jira/browse/YARN-8595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565008#comment-16565008 ] Sunil Govindan commented on YARN-8595: -- Looks good to me. Will commit shortly. Thanks [~akhilpb]. > [UI2] Container diagnostic missing from container page > -- > > Key: YARN-8595 > URL: https://issues.apache.org/jira/browse/YARN-8595 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8595.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8595) [UI2] Container diagnostic information is missing from container page
[ https://issues.apache.org/jira/browse/YARN-8595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8595: - Summary: [UI2] Container diagnostic information is missing from container page (was: [UI2] Container diagnostic missing from container page) > [UI2] Container diagnostic information is missing from container page > - > > Key: YARN-8595 > URL: https://issues.apache.org/jira/browse/YARN-8595 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8595.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8397) Potential thread leak in ActivitiesManager
[
https://issues.apache.org/jira/browse/YARN-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan updated YARN-8397:
-
Summary: Potential thread leak in ActivitiesManager (was: Thread leak in
ActivitiesManager)
> Potential thread leak in ActivitiesManager
> --
>
> Key: YARN-8397
> URL: https://issues.apache.org/jira/browse/YARN-8397
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8397.01.patch
>
>
> It is observed while using MiniYARNCluster, MiniYARNCluster#stop doesn't stop
> JVM.
> Thread dump shows that ActivitiesManager is in timed_waiting state.
> {code}
> "Thread-43" #66 prio=5 os_prio=31 tid=0x7ffea09fd000 nid=0xa103 waiting
> on condition [0x76f1]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager$1.run(ActivitiesManager.java:142)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7159) Normalize unit of resource objects in RM and avoid to do unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563322#comment-16563322 ] Sunil Govindan commented on YARN-7159: -- Thanks [~maniraj...@gmail.com]. [~snemeth] I was waiting for a response from [~templedf]. It seems his comments is already taken care by Mani. We are moving forward. [~maniraj...@gmail.com], cud u pls help to confirm whether latest patch is good. I ll rerun jenkins if its fine. > Normalize unit of resource objects in RM and avoid to do unit conversion in > critical path > - > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Show the current logged in user in UI2
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565205#comment-16565205 ] Sunil Govindan commented on YARN-8594: -- [~akhilpb] pls help to rebase patch against trunk. > [UI2] Show the current logged in user in UI2 > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8593) Add new RM web service endpoint to get cluster user info
[ https://issues.apache.org/jira/browse/YARN-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565207#comment-16565207 ] Sunil Govindan commented on YARN-8593: -- Looks good. Pending jenkins > Add new RM web service endpoint to get cluster user info > > > Key: YARN-8593 > URL: https://issues.apache.org/jira/browse/YARN-8593 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8593.001.patch, YARN-8593.002.patch, > YARN-8593.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8318) IP address in component page shows N/A
[ https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566875#comment-16566875 ] Sunil Govindan commented on YARN-8318: -- This patch looks good and straightforward. Some how fell out my radar. Committing shortly. > IP address in component page shows N/A > -- > > Key: YARN-8318 > URL: https://issues.apache.org/jira/browse/YARN-8318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8318.001.patch > > > Component page shows IP address value as N/A. It should print IP address of > docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8318) [UI2] IP address in component page shows N/A
[ https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8318: - Summary: [UI2] IP address in component page shows N/A (was: IP address in component page shows N/A) > [UI2] IP address in component page shows N/A > > > Key: YARN-8318 > URL: https://issues.apache.org/jira/browse/YARN-8318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8318.001.patch > > > Component page shows IP address value as N/A. It should print IP address of > docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Show the current logged in user in UI2
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566385#comment-16566385 ] Sunil Govindan commented on YARN-8594: -- +1. Looks good. Will commit shortly > [UI2] Show the current logged in user in UI2 > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-8594) [UI2] Show the current logged in user in UI2
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8594: - Comment: was deleted (was: +1. Looks good. Will commit shortly) > [UI2] Show the current logged in user in UI2 > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Show the current logged in user in UI2
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566386#comment-16566386 ] Sunil Govindan commented on YARN-8594: -- One more minor nit # There are some commented code, pls remove the same. > [UI2] Show the current logged in user in UI2 > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8593) Add RM web service endpoint to get user information
[ https://issues.apache.org/jira/browse/YARN-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8593: - Summary: Add RM web service endpoint to get user information (was: Add new RM web service endpoint to get cluster user info) > Add RM web service endpoint to get user information > --- > > Key: YARN-8593 > URL: https://issues.apache.org/jira/browse/YARN-8593 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8593.001.patch, YARN-8593.002.patch, > YARN-8593.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8613) Old RM UI shows wrong vcores total value
[ https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566422#comment-16566422 ] Sunil Govindan commented on YARN-8613: -- This looks like a QueuMetrics issue. cc [~rohithsharma] [~leftnoteasy] [~bibinchundatt] > Old RM UI shows wrong vcores total value > > > Key: YARN-8613 > URL: https://issues.apache.org/jira/browse/YARN-8613 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Priority: Major > Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot > 2018-08-02 at 12.16.53 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Display current logged in user
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566434#comment-16566434 ] Sunil Govindan commented on YARN-8594: -- Looks good +1 > [UI2] Display current logged in user > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch, YARN-8594.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7159) Normalize unit of resource objects in RM and avoid to do unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566419#comment-16566419 ] Sunil Govindan commented on YARN-7159: -- Yes. Latest patch is good to go. Will commit shortly > Normalize unit of resource objects in RM and avoid to do unit conversion in > critical path > - > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8594) [UI2] Display current logged in user
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8594: - Summary: [UI2] Display current logged in user (was: [UI2] Show the current logged in user in UI2) > [UI2] Display current logged in user > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch, YARN-8594.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8592) [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser
[
https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566589#comment-16566589
]
Sunil Govindan commented on YARN-8592:
--
Thanks [~akhilpb]. This looks fine to me.
I tested in a chrome browser and looks fine.
> [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome
> browser
>
>
> Key: YARN-8592
> URL: https://issues.apache.org/jira/browse/YARN-8592
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Akhil S Naik
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8592.001.patch
>
>
> after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating
> Sytem .
> But, When we connect to Resource Manager UI(v2), web browser shows a blank
> page.
> In a chrome development tool, this error is shown
> Stack trace..
> {code:java}
> TypeError: Cannot read property 'join's null
> at new i (hostname:8088/ui2/assets/vendor.js:5598:40988)
> at l (hostname:8088/ui2/assets/vendor.js:5598:41338)
> at p (hostname:8088/ui2/assets/vendor.js:5598:42035)
> at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235)
> at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445)
> at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656)
> at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127)
> at Class.internalNormalizeSingleResponse
> (hostname:8088/ui2/assets/yarn-ui.js:287:73)
> at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024)
> at Array.map (native)
> {code}
> The Error is hit in the underlying framework that Yarn-UI uses :
> moment-timezone-with-data-2010-2020.min.js
> this is the discussion happening in framework github for the same :
> https://github.com/moment/moment-timezone/issues/294
> Please find the discussion on this bug here :
> https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html
> It works well in firefox browser in windows Operating systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8592) [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser
[
https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan updated YARN-8592:
-
Summary: [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and
Chrome browser (was: [UI2] Resource Manager Web UI V2 shows a blank page in
windows OS And Chrome browser)
> [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome
> browser
>
>
> Key: YARN-8592
> URL: https://issues.apache.org/jira/browse/YARN-8592
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Akhil S Naik
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8592.001.patch
>
>
> after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating
> Sytem .
> But, When we connect to Resource Manager UI(v2), web browser shows a blank
> page.
> In a chrome development tool, this error is shown
> Stack trace..
> {code:java}
> TypeError: Cannot read property 'join's null
> at new i (hostname:8088/ui2/assets/vendor.js:5598:40988)
> at l (hostname:8088/ui2/assets/vendor.js:5598:41338)
> at p (hostname:8088/ui2/assets/vendor.js:5598:42035)
> at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235)
> at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445)
> at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656)
> at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127)
> at Class.internalNormalizeSingleResponse
> (hostname:8088/ui2/assets/yarn-ui.js:287:73)
> at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024)
> at Array.map (native)
> {code}
> The Error is hit in the underlying framework that Yarn-UI uses :
> moment-timezone-with-data-2010-2020.min.js
> this is the discussion happening in framework github for the same :
> https://github.com/moment/moment-timezone/issues/294
> Please find the discussion on this bug here :
> https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html
> It works well in firefox browser in windows Operating systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8559) Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566711#comment-16566711 ] Sunil Govindan commented on YARN-8559: -- Thanks [~cheersyang]. Looks good. Only concern was that because we use initWritableEndPoint is called from GET api. However as [~leftnoteasy] mentioned, its better to protect these for admin for now. Thanks for adding some more comments there. If no objections, I can help to commit the same tomorrow. > Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint > > > Key: YARN-8559 > URL: https://issues.apache.org/jira/browse/YARN-8559 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Anna Savarin >Assignee: Weiwei Yang >Priority: Major > Attachments: YARN-8559.001.patch, YARN-8559.002.patch, > YARN-8559.003.patch, YARN-8559.004.patch > > > All Hadoop services provide a set of common endpoints (/stacks, /logLevel, > /metrics, /jmx, /conf). In the case of the Resource Manager, part of the > configuration comes from the scheduler being used. Currently, these > configuration key/values are not exposed through the /conf endpoint, thereby > revealing an incomplete configuration picture. > Make an improvement and expose the scheduling configuration info through the > RM's /conf endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8591) [ATSv2] NPE while checking for entity acl in non-secure cluster
[
https://issues.apache.org/jira/browse/YARN-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan updated YARN-8591:
-
Summary: [ATSv2] NPE while checking for entity acl in non-secure cluster
(was: [ATSv2] NPE error while checking for entity acl in non-secure cluster)
> [ATSv2] NPE while checking for entity acl in non-secure cluster
> ---
>
> Key: YARN-8591
> URL: https://issues.apache.org/jira/browse/YARN-8591
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelinereader, timelineserver
>Reporter: Akhil PB
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8591.01.patch
>
>
> {code:java}
> GET
> http://ctr-e138-1518143905142-417433-01-04.hwx.site:8198/ws/v2/timeline/apps/application_1532578985272_0002/entities/YARN_CONTAINER?fields=ALL&_=1532670071899{code}
> {code:java}
> 2018-07-27 05:32:03,468 WARN webapp.GenericExceptionHandler
> (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
> javax.ws.rs.WebApplicationException: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.handleException(TimelineReaderWebServices.java:196)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:624)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:474)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
> org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1604)
>
[jira] [Updated] (YARN-7991) Use ServiceState values to publish to ATS
[ https://issues.apache.org/jira/browse/YARN-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7991: - Issue Type: Bug (was: Sub-task) Parent: (was: YARN-7957) > Use ServiceState values to publish to ATS > - > > Key: YARN-7991 > URL: https://issues.apache.org/jira/browse/YARN-7991 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services >Reporter: Gour Saha >Assignee: Gour Saha >Priority: Major > > Add the state DELETED to ServiceState and then use ServiceState values to > publish to ATS (instead of FinalApplicationStatus). > Refer to parent issue for more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7993) [UI2] yarn-service page need to consider ServiceState to show stop/delete buttons
[ https://issues.apache.org/jira/browse/YARN-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7993: - Issue Type: Bug (was: Sub-task) Parent: (was: YARN-7957) > [UI2] yarn-service page need to consider ServiceState to show stop/delete > buttons > - > > Key: YARN-7993 > URL: https://issues.apache.org/jira/browse/YARN-7993 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Sunil Govindan >Assignee: Akhil PB >Priority: Major > > yarn service page has stop/delete buttons. These buttons has to be > shown/hidden based on ServiceState of each app from ATS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8561) [Submarine] Add initial implementation: training job submission and job history retrieve.
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576613#comment-16576613 ] Sunil Govindan commented on YARN-8561: -- Thanks [~leftnoteasy], Overall looks good to me Will create additional jiras as discussed in this ticket. If there are no objections, I will commit this patch tomorrow. +1 Thanks > [Submarine] Add initial implementation: training job submission and job > history retrieve. > - > > Key: YARN-8561 > URL: https://issues.apache.org/jira/browse/YARN-8561 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-8561.001.patch, YARN-8561.002.patch, > YARN-8561.003.patch, YARN-8561.004.patch, YARN-8561.005.patch > > > Added following parts: > 1) New subcomponent of YARN, under applications/ project. > 2) Tensorflow training job submission, including training (single node and > distributed). > - Supported Docker container. > - Support GPU isolation. > - Support YARN registry DNS. > 3) Retrieve job history. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576395#comment-16576395 ] Sunil Govindan commented on YARN-7957: -- Thanks [~akhilpb] Makes sense to me. Pls help to implement same. > [UI2] Yarn service delete option disappears after stopping application > -- > > Key: YARN-7957 > URL: https://issues.apache.org/jira/browse/YARN-7957 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Akhil PB >Priority: Critical > Attachments: YARN-7957.001.patch > > > Steps: > 1) Launch yarn service > 2) Go to service page and click on Setting button->"Stop Service". The > application will be stopped. > 3) Refresh page > Here, setting button disappears. Thus, user can not delete service from UI > after stopping application > Expected behavior: > Setting button should be present on UI page after application is stopped. If > application is stopped, setting button should only have "Delete Service" > action available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.14.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576604#comment-16576604 ] Sunil Govindan commented on YARN-7494: -- As discussed, removed updating multiNodePolicyName in Queue interface. This is changed to CSQueue. [~cheersyang] pls help to review. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8426) Upgrade jquery-ui to 1.12.1 in YARN
[ https://issues.apache.org/jira/browse/YARN-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8426: - Fix Version/s: 3.1.2 > Upgrade jquery-ui to 1.12.1 in YARN > --- > > Key: YARN-8426 > URL: https://issues.apache.org/jira/browse/YARN-8426 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8426.001.patch > > > In align to HADOOP-15483, upgrade jquery-ui for YARN common package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.12.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application
[
https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574674#comment-16574674
]
Sunil Govindan commented on YARN-7957:
--
Thanks [~akhilpb]. Patch almost looks good.
Pls check any more states to be added to {{const serviceStates = ['ACCEPTED',
'STARTED', 'STABLE', 'RUNNING'];}}
> [UI2] Yarn service delete option disappears after stopping application
> --
>
> Key: YARN-7957
> URL: https://issues.apache.org/jira/browse/YARN-7957
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
>Affects Versions: 3.1.0
>Reporter: Yesha Vora
>Assignee: Akhil PB
>Priority: Critical
> Attachments: YARN-7957.001.patch
>
>
> Steps:
> 1) Launch yarn service
> 2) Go to service page and click on Setting button->"Stop Service". The
> application will be stopped.
> 3) Refresh page
> Here, setting button disappears. Thus, user can not delete service from UI
> after stopping application
> Expected behavior:
> Setting button should be present on UI page after application is stopped. If
> application is stopped, setting button should only have "Delete Service"
> action available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574680#comment-16574680 ] Sunil Govindan commented on YARN-7494: -- Thanks [~cheersyang] and [~leftnoteasy] Almost fixed all comments. 5) It's better to add getMultiNodeSortingPolicyName CSQueue, inside FiCaSchedulerApp, you can check the instanceof. With this we can limit the changes to CS. Sunil: We initially had it for only CS. But during our discussion with [~cheersyang] we thought its better be generic so that FS can use with ease. And less if checks in FicaSchedulerApp. 6) RegularContainerAllocator#allocate, when the change needed? Earlier for each node heartbeat, we were doing precheck. and then allocate was been invoked. In multi-node mode, we should try to do precheck and allocate in same loop of sorted nodes so that if one check fails,next node could be looked sooner. 7) Instead of adding MultiNodeSortingManager to RMContext, can we limit changes inside Scheduler? Here also We initially had it for only CS. But during this patch review with [~cheersyang], we thought this might be better for a common approach. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574366#comment-16574366 ] Sunil Govindan commented on YARN-8633: -- Committed to trunk. Cherry-pick to branch-3.1 is failing. [~akhilpb] pls help to share branch-3.1 patch. > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574353#comment-16574353 ] Sunil Govindan commented on YARN-8633: -- Thanks [~msingh] for confirming. I will commit this patch now. Will take care of the whitespace while committing.Thanks [~akhilpb] > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8426) Upgrade jquery-ui to 1.12.1 in YARN
[ https://issues.apache.org/jira/browse/YARN-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574375#comment-16574375 ] Sunil Govindan commented on YARN-8426: -- I also back ported to branch-3.1. Thanks > Upgrade jquery-ui to 1.12.1 in YARN > --- > > Key: YARN-8426 > URL: https://issues.apache.org/jira/browse/YARN-8426 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8426.001.patch > > > In align to HADOOP-15483, upgrade jquery-ui for YARN common package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574380#comment-16574380 ] Sunil Govindan commented on YARN-8633: -- I pulled YARN-8426 to branch-3.1. This will help to cherrypick this patch to branch-3.1 without additional rebase. > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574382#comment-16574382 ] Sunil Govindan commented on YARN-8633: -- Thanks [~akhilpb] for the patch. And thanks [~msingh] for additional reviews. Committed to trunk/branch-3.1 > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8561) [Submarine] Add initial implementation: training job submission and job history retrieve.
[
https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574994#comment-16574994
]
Sunil Govindan commented on YARN-8561:
--
Thanks [~leftnoteasy]
>> I'm not quite sure about this suggestion, it seems to me that we should add
>> the getServiceResourceFromYarnResource method to service.Resource instead. I
>> don't want to touch any service classes in this patch. Should we do it in a
>> separate JIRA?
Yes. This makes sense.
>> I think we can push this to the future patch, one possible solution is to
>>include a yaml file to describe job configs and user can reuse it instead of
>>passing 10+ params to CLI.
This could be a followup jira. I agree.
Few more comments:
# In {{Cli}} class, could we use Options for CLI parsing. This will help to do
paring and add help much better.
# We have a bunch of constants defined in {{CliConstants}}> going forward we
will more here. I am not sure whether this is a good idea. Could we load this
from a config file something like resource-types.xml where all such commands
can be loaded. A new command can be added with a definition in this spec file
without code change.
# RemoteDirectoryManager holds information about dirs and fs. Currently its
only in HDFS, or could we also support local file?
# CliUtils#replacePatternsInLaunchCommand
{code:java}
65 String newCli = specifiedCli;
66 for (Map.Entry replace : replacePattern.entrySet()) {
67 newCli = newCli.replace(replace.getKey(), replace.getValue());
68 }{code}
I didnt understand this very cleanly. We want to replace the value in
the specifiedCli to newCli, correct? Key will be the string which starts with %.
5. I think StringUtils#strip is better to trim chars like '[' and ']' in
{{parseResourcesString}}
6. {{!resource.matches("^[^=]+=\\d+\\s?\\w*$"}} In parseResourcesString, its
better to put this in const string
7. Yes, UnitsConversionUtil is more exhaustive and might have a bit diff
meaning. But could we refactor the unit conversion code in
parseResourcesString to a util, as i think it might help for some other apps
too.
8. Resource profile support in cli? May help to avoid specify whole resources?
9. Submarine cli might need to support app priority, app timeout, queue mapping
etc ?
10. Submarine kerberos support will be in subsequent jira, correct?
11. In cli, CliConstants.ENV helps to add envs to job. But we also need to
specify ENV's per component level, correct?
12. May be checksum verification needed for FSBasedSubmarineStorageImpl
13. I think we need a translation from ServiceState to JobState. Because more
and more states are added native service, and it will be tough to map to
submarine.
14. In general i think this is a great effort in getting cli, job tracking,
store etc in to one framework. Parameter validation and error handling is
always tougher, but is there a way where we can cleanly show error comes from
native service if the image doesnt exist or corrupted or no permission etc to
be popped up in submarine level.
> [Submarine] Add initial implementation: training job submission and job
> history retrieve.
> -
>
> Key: YARN-8561
> URL: https://issues.apache.org/jira/browse/YARN-8561
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Major
> Attachments: YARN-8561.001.patch, YARN-8561.002.patch,
> YARN-8561.003.patch, YARN-8561.004.patch
>
>
> Added following parts:
> 1) New subcomponent of YARN, under applications/ project.
> 2) Tensorflow training job submission, including training (single node and
> distributed).
> - Supported Docker container.
> - Support GPU isolation.
> - Support YARN registry DNS.
> 3) Retrieve job history.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update JQuery version references in yarn-common
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573273#comment-16573273 ] Sunil Govindan commented on YARN-8633: -- Tested this locally. Old UI pages are loading good. Committing shortly. > Update JQuery version references in yarn-common > --- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8633: - Summary: Update DataTables version in yarn-common in line with JQuery 3 upgrade (was: Update JQuery version references in yarn-common) > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573281#comment-16573281 ] Sunil Govindan commented on YARN-8633: -- Accidentally pushed this. I have reverted now. [~akhilpb], i thought the test case were not failing locally. But in my linux machine, its failing still. Cud u pls check. Thank you > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8633) Update DataTables version in yarn-common in line with JQuery 3 upgrade
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573283#comment-16573283 ] Sunil Govindan commented on YARN-8633: -- [~msingh] cud u pls help to see whether this test failures are related? > Update DataTables version in yarn-common in line with JQuery 3 upgrade > -- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue
[
https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579199#comment-16579199
]
Sunil Govindan commented on YARN-8657:
--
Hi [~leftnoteasy]
Thanks for the patch. Some doubts on this.
{{computeUserLimitAndSetHeadroom}} is invoked under readLock in few places and
writeLock under {{updateClusterResource}}. With this patch, this method is now
called under readLock. However other than
{{metrics.setAvailableResourcesToUser(nodePartition, user, headroom);}}, all
other setters are under respective locks. I think this also to be protected as
I have seen some recent issues in metrics in queue metrics. I suspect this is
one reason.
{{computeUserLimitAndSetHeadroom}} has this annotation. I think this need to
revisited ? *@Lock(\{LeafQueue.class})*
> User limit calculation should be read-lock-protected within LeafQueue
> -
>
> Key: YARN-8657
> URL: https://issues.apache.org/jira/browse/YARN-8657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sumana Sathish
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8657.001.patch
>
>
> When async scheduling is enabled, user limit calculation could be wrong:
> It is possible that scheduler calculated a user_limit, but inside
> {{canAssignToUser}} it becomes staled.
> We need to protect user limit calculation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.005.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583362#comment-16583362 ] Sunil Govindan commented on YARN-7863: -- For now, i removed AttributeSpec class. Using same placement_spec. However i have already done a separate parse etc for a clener impl of node attribute. Hence retaining all same. cc [~Naganarasimha] [~cheersyang] > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583360#comment-16583360
]
Sunil Govindan commented on YARN-7863:
--
Attaching an end-to-end working patch. (only anti-affinity support in
Scheduler, once [~cheersyang] patch is in, affinity will be supported for
placement spec as whole including node attributes
Test steps:
*Configuration*
{code:java}
yarn.resourcemanager.placement-constraints.handler
scheduler
{code}
*Add attributes:*
{{yarn nodeattributes -add localhost:java=1.8}}
*Run DS*
{{yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar
../share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0-SNAPSHOT.jar
-shell_command sleep -shell_args 10 -num_containers 2 -placement_spec
NOTIN,java=1.8}}
> Modify placement constraints to support node attributes
> ---
>
> Key: YARN-7863
> URL: https://issues.apache.org/jira/browse/YARN-7863
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Sunil Govindan
>Assignee: Sunil Govindan
>Priority: Major
> Attachments: YARN-7863-YARN-3409.002.patch,
> YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch,
> YARN-7863-YARN-3409.005.patch, YARN-7863.v0.patch
>
>
> This Jira will track to *Modify existing placement constraints to support
> node attributes.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8613) Old RM UI shows wrong vcores total value
[ https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571429#comment-16571429 ] Sunil Govindan commented on YARN-8613: -- Thanks [~Sen Zhao] for the patch. This fixes issue which I think is fine. However my worry is why QueueMetrics is failing. I think we need to debug more on how Queue Metrics got corrupted else same issue will come via metrics etc as well. [~bibinchundatt] [~rohithsharma] pls share ur thoughts > Old RM UI shows wrong vcores total value > > > Key: YARN-8613 > URL: https://issues.apache.org/jira/browse/YARN-8613 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Sen Zhao >Priority: Major > Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot > 2018-08-02 at 12.16.53 PM.png, YARN-8613.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8630) ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled
[ https://issues.apache.org/jira/browse/YARN-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571432#comment-16571432 ] Sunil Govindan commented on YARN-8630: -- +1 on this patch. Its straight forward fix. Thanks [~rohithsharma] > ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster > when ACls are enabled > --- > > Key: YARN-8630 > URL: https://issues.apache.org/jira/browse/YARN-8630 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8630.01.patch > > > It is observed that ATSv2 REST endpoints are not honoring > *yarn.webapp.filter-entity-list-by-user* in non-secure cluster when ACLs are > enabled. > The issue can be seen if static web app filter is not configured in > non-secure cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8633) Update JQuery version references in yarn-common
[ https://issues.apache.org/jira/browse/YARN-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8633: - Summary: Update JQuery version references in yarn-common (was: [BlackDuck] [Hadoop Yarn Common] Update JQuery version references) > Update JQuery version references in yarn-common > --- > > Key: YARN-8633 > URL: https://issues.apache.org/jira/browse/YARN-8633 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8633.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8561) [Submarine] Add initial implementation: training job submission and job history retrieve.
[
https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571719#comment-16571719
]
Sunil Govindan commented on YARN-8561:
--
Thanks [~leftnoteasy] for the effort. I have tried to look through the approach
and code.
Few comments which is mixed or major and minor :)
1. I think we can used same CLI model of client where CLI extends Configured
and implements Tool. This helps for tests. Also this helps to avoid abstract
run method as its Tool.
2. We could also stop a job from CLI, correct? In that case, do we need to do
some thing more extra than a simple yarn app -kill appId ?
3. I think we can use UnitsConversionUtil for unit convertion.
CliUtils#parseResourcesString
4. In CapSchedConfig for absolute resource, we used a pattern match code.
{code}
public static final String PATTERN_FOR_ABSOLUTE_RESOURCE = "^\\[[\\w\\.,\\-_=\\
/]+\\]$";
private static final Pattern RESOURCE_PATTERN =
Pattern.compile(PATTERN_FOR_ABSOLUTE_RESOURCE);
{code}
Could we use same in CLI as well?
5. May be rename JobState to SubmarineJobState
6. Commandline options looks very clean and thorough. I think as we go forward,
more CLI options will be added. and it will become more complex. Could we load
a profile to submarine and then use the profile get 80% of such config items.
Given a profile, may be user might need to fill 1 or 2 variable arguments.
7. DevelopperGuide.md ==> DeveloperGuide.md
> [Submarine] Add initial implementation: training job submission and job
> history retrieve.
> -
>
> Key: YARN-8561
> URL: https://issues.apache.org/jira/browse/YARN-8561
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Major
> Attachments: YARN-8561.001.patch
>
>
> Added following parts:
> 1) New subcomponent of YARN, under applications/ project.
> 2) Tensorflow training job submission, including training (single node and
> distributed).
> - Supported Docker container.
> - Support GPU isolation.
> - Support YARN registry DNS.
> 3) Retrieve job history.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8561) [Submarine] Add initial implementation: training job submission and job history retrieve.
[
https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571719#comment-16571719
]
Sunil Govindan edited comment on YARN-8561 at 8/7/18 3:00 PM:
--
Thanks [~leftnoteasy] for the effort. I have tried to look through the approach
and code.
Few comments which is mixed or major and minor :)
1. I think we can used same CLI model of client where CLI extends Configured
and implements Tool. This helps for tests. Also this helps to avoid abstract
run method as its Tool.
2. We could also stop a job from CLI, correct? In that case, do we need to do
some thing more extra than a simple yarn app -kill appId ?
3. I think we can use UnitsConversionUtil for unit convertion.
CliUtils#parseResourcesString
4. In CapSchedConfig for absolute resource, we used a pattern match code.
{code}
public static final String PATTERN_FOR_ABSOLUTE_RESOURCE = "^\\[[\\w\\.,\\-_=\\
/]+\\]$";
private static final Pattern RESOURCE_PATTERN =
Pattern.compile(PATTERN_FOR_ABSOLUTE_RESOURCE);
{code}
Could we use same in CLI as well?
5. May be rename JobState to SubmarineJobState
6. Commandline options looks very clean and thorough. I think as we go forward,
more CLI options will be added. and it will become more complex. Could we load
a profile to submarine and then use the profile get 80% of such config items.
Given a profile, may be user might need to fill 1 or 2 variable arguments.
7. DevelopperGuide.md ==> DeveloperGuide.md
8. In getServiceResourceFromYarnResource, I think we should get the resource
list from ResourceUtils. Also it might be better to use a common client/server
util method to create resource. something like
Resource.newInstance(yarnResource) or Resources.createResource(yarnResource)
9. In verbose or debug mode, may be in YarnServiceJobSubmitter could dump all
contents of \{{FileWriter fw}}
10. It might be better to add a shutdown signal or interrupt signal to break
out from JobMonitor#waitTrainingFinal, if job is faulty.
11. In fromServiceState, service state STOPPED is considered as
JobState.SUCCEEDED;
12. Some commented code in JobStatusBuilder
13. How could we increase number of workers on a running job?
was (Author: sunilg):
Thanks [~leftnoteasy] for the effort. I have tried to look through the approach
and code.
Few comments which is mixed or major and minor :)
1. I think we can used same CLI model of client where CLI extends Configured
and implements Tool. This helps for tests. Also this helps to avoid abstract
run method as its Tool.
2. We could also stop a job from CLI, correct? In that case, do we need to do
some thing more extra than a simple yarn app -kill appId ?
3. I think we can use UnitsConversionUtil for unit convertion.
CliUtils#parseResourcesString
4. In CapSchedConfig for absolute resource, we used a pattern match code.
{code}
public static final String PATTERN_FOR_ABSOLUTE_RESOURCE = "^\\[[\\w\\.,\\-_=\\
/]+\\]$";
private static final Pattern RESOURCE_PATTERN =
Pattern.compile(PATTERN_FOR_ABSOLUTE_RESOURCE);
{code}
Could we use same in CLI as well?
5. May be rename JobState to SubmarineJobState
6. Commandline options looks very clean and thorough. I think as we go forward,
more CLI options will be added. and it will become more complex. Could we load
a profile to submarine and then use the profile get 80% of such config items.
Given a profile, may be user might need to fill 1 or 2 variable arguments.
7. DevelopperGuide.md ==> DeveloperGuide.md
> [Submarine] Add initial implementation: training job submission and job
> history retrieve.
> -
>
> Key: YARN-8561
> URL: https://issues.apache.org/jira/browse/YARN-8561
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Major
> Attachments: YARN-8561.001.patch
>
>
> Added following parts:
> 1) New subcomponent of YARN, under applications/ project.
> 2) Tensorflow training job submission, including training (single node and
> distributed).
> - Supported Docker container.
> - Support GPU isolation.
> - Support YARN registry DNS.
> 3) Retrieve job history.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.16.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577817#comment-16577817 ] Sunil Govindan commented on YARN-7494: -- Thanks [~cheersyang] for quick review. Yes, I modified manager class to have a wrapper to do this extra check. I tried to make the scheduler lookup path as lean as possible. Pls help to review. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.15.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577651#comment-16577651 ] Sunil Govindan commented on YARN-7494: -- Updating v15 patch after correcting default class name. cc [~cheersyang] > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8561) [Submarine] Initial implementation: Training job submission and job history retrieval
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8561: - Summary: [Submarine] Initial implementation: Training job submission and job history retrieval (was: [Submarine] Add initial implementation: training job submission and job history retrieve.) > [Submarine] Initial implementation: Training job submission and job history > retrieval > - > > Key: YARN-8561 > URL: https://issues.apache.org/jira/browse/YARN-8561 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-8561.001.patch, YARN-8561.002.patch, > YARN-8561.003.patch, YARN-8561.004.patch, YARN-8561.005.patch > > > Added following parts: > 1) New subcomponent of YARN, under applications/ project. > 2) Tensorflow training job submission, including training (single node and > distributed). > - Supported Docker container. > - Support GPU isolation. > - Support YARN registry DNS. > 3) Retrieve job history. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8561) [Submarine] Initial implementation: Training job submission and job history retrieval
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577971#comment-16577971 ] Sunil Govindan commented on YARN-8561: -- Test case errors are not related. Whitespace issue is taken care while committing. Pushed to trunk. Thanks [~leftnoteasy] for the effort. > [Submarine] Initial implementation: Training job submission and job history > retrieval > - > > Key: YARN-8561 > URL: https://issues.apache.org/jira/browse/YARN-8561 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-8561.001.patch, YARN-8561.002.patch, > YARN-8561.003.patch, YARN-8561.004.patch, YARN-8561.005.patch > > > Added following parts: > 1) New subcomponent of YARN, under applications/ project. > 2) Tensorflow training job submission, including training (single node and > distributed). > - Supported Docker container. > - Support GPU isolation. > - Support YARN registry DNS. > 3) Retrieve job history. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.13.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575974#comment-16575974 ] Sunil Govindan commented on YARN-7494: -- Thanks [~cheersyang]. bq.my opinion is if this causes burden ending up with changing a lot of files, it seems better to limit this feature in CS. Just let the sorting manager be a common service would be enough. With current approach, we can actually have a cleaner implementation generic. Though it touches few files, it is not much causing problem. Also we can avoid few instance of check which I think is always better to have a cleaner impl. Other issue are addressed and uploaded a new patch. Pls help to review > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579323#comment-16579323 ] Sunil Govindan commented on YARN-7494: -- Updated v17 patch. [~cheersyang] pls help to review. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.17.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584281#comment-16584281 ] Sunil Govindan commented on YARN-7863: -- Updating a refined patch. Thanks [~cheersyang] for quick offline comments. > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8678) Queue Management API - rephrase error messages
[
https://issues.apache.org/jira/browse/YARN-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan updated YARN-8678:
-
Issue Type: Sub-task (was: Bug)
Parent: YARN-5734
> Queue Management API - rephrase error messages
> --
>
> Key: YARN-8678
> URL: https://issues.apache.org/jira/browse/YARN-8678
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Akhil PB
>Assignee: Akhil PB
>Priority: Major
>
> 1. When stopping a running parent queue, error message thrown by API was not
> meaningful.
> For example: When tried to stop root queue, error message thrown was
> {{Failed to re-init queues : The parent queue:root state is STOPPED, child
> queue:default state cannot be RUNNING.}}
> It is evident that root queue update failed, but the message says
> {{queue:root state is STOPPED}}.
> 2. While tried to delete a running leaf queue, error message thrown by API
> was not meaningful.
> For example: Error message was {{Failed to re-init queues : root.default.prod
> is deleted from the new capacity scheduler configuration, but the queue is
> not yet in stopped state. Current State : RUNNING}}.
> Clearly deletion of queue root.default.prod failed with error, but the
> message says {{queues : root.default.prod is deleted from the new capacity
> scheduler configuration}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8677) Queue Management API - no errors thrown for wrong properties
[ https://issues.apache.org/jira/browse/YARN-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8677: - Issue Type: Sub-task (was: Bug) Parent: YARN-5734 > Queue Management API - no errors thrown for wrong properties > > > Key: YARN-8677 > URL: https://issues.apache.org/jira/browse/YARN-8677 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > > While updating the queue using API with the wrong properties, there are no > errors thrown for wrong properties. Instead, API gives 200 OK and there are > no changes affected. API should throw 400 for request with wrong properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.006.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.19.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.19.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586978#comment-16586978 ] Sunil Govindan commented on YARN-7494: -- Fixed checkstyle whichever is possible. [~cheersyang] pls help to check. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.19.patch, YARN-7494.v0.patch, > YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.20.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.19.patch, YARN-7494.20.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.007.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587389#comment-16587389 ] Sunil Govindan commented on YARN-7494: -- [~cheersyang] Fixed checkstyles which are possible. Some lines length cant be done as it is name etc. Also that class doesnt need a setter and getter. Pls check. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.19.patch, YARN-7494.20.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587371#comment-16587371 ] Sunil Govindan commented on YARN-7863: -- Thanks [~cheersyang] [~Naganarasimha] *TestCases* could be added for DS in another patch i think. It ll cover all DS level cases. I ll add AND & OR cases in this one. Other cases are covered. > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8015) Complete placement constraint support for Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588274#comment-16588274 ] Sunil Govindan commented on YARN-8015: -- Thanks [~cheersyang]. Looks fine to me. Committing shortly. I think we can commit this only to trunk alone, correct? > Complete placement constraint support for Capacity Scheduler > > > Key: YARN-8015 > URL: https://issues.apache.org/jira/browse/YARN-8015 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Critical > Attachments: YARN-8015.001.patch, YARN-8015.002.patch, > YARN-8015.003.patch, YARN-8015.004.patch > > > AppPlacementAllocator currently only supports intra-app anti-affinity > placement constraints, once YARN-8002 and YARN-8013 are resolved, it needs to > support inter-app constraints too. Also, this may require some refactoring on > the existing code logic. Use this JIRA to track. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8691) AMRMClient unregisterApplicationMaster Api's appMessage should have a maximum size
[
https://issues.apache.org/jira/browse/YARN-8691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan reassigned YARN-8691:
Assignee: Yicong Cai
> AMRMClient unregisterApplicationMaster Api's appMessage should have a maximum
> size
> --
>
> Key: YARN-8691
> URL: https://issues.apache.org/jira/browse/YARN-8691
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.3
>Reporter: Yicong Cai
>Assignee: Yicong Cai
>Priority: Critical
> Fix For: 2.7.7
>
>
> SparkSQL AM Codegen ERROR，then call unregister AM API and send the error
> message to RM, RM receive the AM state and update to RMStateStore. The
> Codegen error message maybe is huge, (Our case is about 200MB). If the
> RMStateStore is ZKRMStateStore, it causes the same exception as YARN-6125,
> but YARN-6125 doesn't cover the unregisterApplicationMaster's message cut.
>
> SparkSQL Codegen error message show below:
> 18/08/18 08:34:54 ERROR codegen.CodeGenerator: failed to compile:
> org.codehaus.janino.JaninoRuntimeException: Constant pool has grown past JVM
> limit of 0x
> /* 001 */ public java.lang.Object generate(Object[] references)
> { /* 002 */ return new SpecificSafeProjection(references); /* 003 */ }
> /* 004 */
> /* 005 */ class SpecificSafeProjection extends
> org.apache.spark.sql.catalyst.expressions.codegen.BaseProjection {
> ..
> about 2 million lines.
> ..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8691) AMRMClient unregisterApplicationMaster Api's appMessage should have a maximum size
[
https://issues.apache.org/jira/browse/YARN-8691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589647#comment-16589647
]
Sunil Govindan commented on YARN-8691:
--
Thank [~caiyicong], assigned to u.
> AMRMClient unregisterApplicationMaster Api's appMessage should have a maximum
> size
> --
>
> Key: YARN-8691
> URL: https://issues.apache.org/jira/browse/YARN-8691
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.3
>Reporter: Yicong Cai
>Assignee: Yicong Cai
>Priority: Critical
> Fix For: 2.7.7
>
>
> SparkSQL AM Codegen ERROR，then call unregister AM API and send the error
> message to RM, RM receive the AM state and update to RMStateStore. The
> Codegen error message maybe is huge, (Our case is about 200MB). If the
> RMStateStore is ZKRMStateStore, it causes the same exception as YARN-6125,
> but YARN-6125 doesn't cover the unregisterApplicationMaster's message cut.
>
> SparkSQL Codegen error message show below:
> 18/08/18 08:34:54 ERROR codegen.CodeGenerator: failed to compile:
> org.codehaus.janino.JaninoRuntimeException: Constant pool has grown past JVM
> limit of 0x
> /* 001 */ public java.lang.Object generate(Object[] references)
> { /* 002 */ return new SpecificSafeProjection(references); /* 003 */ }
> /* 004 */
> /* 005 */ class SpecificSafeProjection extends
> org.apache.spark.sql.catalyst.expressions.codegen.BaseProjection {
> ..
> about 2 million lines.
> ..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.008.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587849#comment-16587849 ] Sunil Govindan commented on YARN-7863: -- Updated v8 patch. cc [~cheersyang] [~Naganarasimha] > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.009.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863-YARN-3409.009.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7863) Modify placement constraints to support node attributes
[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592627#comment-16592627
]
Sunil Govindan edited comment on YARN-7863 at 8/25/18 3:32 PM:
---
Thanks [~cheersyang] and [~Naganarasimha]
Improved syntax in latest patch. Now user can say like {{-placement_spec
foo=4,java=1.8}}. By this user can easily specify constraint as EQ or NE. These
are the 2 op codes which are supported now.
Kindly help to check the patch.
was (Author: sunilg):
Thanks [~cheersyang] and [~Naganarasimha]
Improved syntax in latest patch. Now user can say like {{-placement_spec
foo=4,java=1.8}}. By this user can easily specify constraint and EQ or NE.
These are the 2 op codes which are supported now.
Kindly help to check the patch.
> Modify placement constraints to support node attributes
> ---
>
> Key: YARN-7863
> URL: https://issues.apache.org/jira/browse/YARN-7863
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Sunil Govindan
>Assignee: Sunil Govindan
>Priority: Major
> Attachments: YARN-7863-YARN-3409.002.patch,
> YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch,
> YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch,
> YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch,
> YARN-7863-YARN-3409.009.patch, YARN-7863.v0.patch
>
>
> This Jira will track to *Modify existing placement constraints to support
> node attributes.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592627#comment-16592627
]
Sunil Govindan commented on YARN-7863:
--
Thanks [~cheersyang] and [~Naganarasimha]
Improved syntax in latest patch. Now user can say like {{-placement_spec
foo=4,java=1.8}}. By this user can easily specify constraint and EQ or NE.
These are the 2 op codes which are supported now.
Kindly help to check the patch.
> Modify placement constraints to support node attributes
> ---
>
> Key: YARN-7863
> URL: https://issues.apache.org/jira/browse/YARN-7863
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Sunil Govindan
>Assignee: Sunil Govindan
>Priority: Major
> Attachments: YARN-7863-YARN-3409.002.patch,
> YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch,
> YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch,
> YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch,
> YARN-7863-YARN-3409.009.patch, YARN-7863.v0.patch
>
>
> This Jira will track to *Modify existing placement constraints to support
> node attributes.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592785#comment-16592785 ] Sunil Govindan commented on YARN-7863: -- Since we use an enum, proto class with default to first value of enum (here its EQ). And this is coming in normal placement constraint (tags). So i added NO_OP as default enum to avoid such issues and to have a clean impl for pb. > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863-YARN-3409.009.patch, YARN-7863-YARN-3409.010.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592784#comment-16592784 ] Sunil Govindan commented on YARN-7863: -- Thanks [~cheersyang]. Fixed these issues and uploaded a new patch. > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863-YARN-3409.009.patch, YARN-7863-YARN-3409.010.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.010.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863-YARN-3409.009.patch, YARN-7863-YARN-3409.010.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592955#comment-16592955 ] Sunil Govindan commented on YARN-7863: -- Thanks [~Naganarasimha]. Updating latest patch addressing comments > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch, > YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch, > YARN-7863-YARN-3409.009.patch, YARN-7863-YARN-3409.010.patch, > YARN-7863-YARN-3409.011.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8718) Merge related work for YARN-3409
[ https://issues.apache.org/jira/browse/YARN-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8718: - Attachment: YARN-8718.003.patch > Merge related work for YARN-3409 > > > Key: YARN-8718 > URL: https://issues.apache.org/jira/browse/YARN-8718 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Priority: Major > Attachments: YARN-3409.001.patch, YARN-3409.002.patch, > YARN-8718.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted
[
https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595890#comment-16595890
]
Sunil Govindan commented on YARN-8709:
--
Thanks [~Tao Yang] for raising this issue.
Yes, I agree with [~eepayne]. I think its better we clear
partitionToUnderServedQueues before PreemptionCandidatesSelectors are called.
> intra-queue preemption checker always fail since one under-served queue was
> deleted
> ---
>
> Key: YARN-8709
> URL: https://issues.apache.org/jira/browse/YARN-8709
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler preemption
>Affects Versions: 3.2.0
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Major
>
> After some queues deleted, the preemption checker in SchedulingMonitor was
> always skipped because of YarnRuntimeException for every run.
> Error logs:
> {noformat}
> ERROR [SchedulingMonitor (ProportionalCapacityPreemptionPolicy)]
> org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor:
> Exception raised while executing preemption checker, skip this run...,
> exception=
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: This shouldn't
> happen, cannot find TempQueuePerPartition for queueName=1535075839208
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.getQueueByPartition(ProportionalCapacityPreemptionPolicy.java:701)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.IntraQueueCandidatesSelector.computeIntraQueuePreemptionDemand(IntraQueueCandidatesSelector.java:302)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.IntraQueueCandidatesSelector.selectCandidates(IntraQueueCandidatesSelector.java:128)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:514)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:348)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:99)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PolicyInvoker.run(SchedulingMonitor.java:111)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:186)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
> at java.lang.Thread.run(Thread.java:834)
> {noformat}
> I think there is something wrong with partitionToUnderServedQueues field in
> ProportionalCapacityPreemptionPolicy. Items of partitionToUnderServedQueues
> can be add but never be removed, except rebuilding this policy. For example,
> once under-served queue "a" is added into this structure, it will always be
> there and never be removed, intra-queue preemption checker will try to get
> all queues info for partitionToUnderServedQueues in
> IntraQueueCandidatesSelector#selectCandidates and will throw
> YarnRuntimeException if not found. So that after queue "a" is deleted from
> queue structure, the preemption checker will always fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8286) Add NMClient callback on container relaunch
[ https://issues.apache.org/jira/browse/YARN-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595920#comment-16595920 ] Sunil Govindan commented on YARN-8286: -- Ping [~billie.rinaldi] As this jira is marked for 3.2 as a critical, cud u pls help to take this forward or move out if its not feasible to finish in coming weeks. 3.2 code freeze date is nearby in a weeks. Kindly help to check the same. > Add NMClient callback on container relaunch > --- > > Key: YARN-8286 > URL: https://issues.apache.org/jira/browse/YARN-8286 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Priority: Critical > > The AM may need to perform actions when a container has been relaunched. For > example, the service AM would want to change the state it has recorded for > the container and retrieve new container status for the container, in case > the container IP has changed. (The NM would also need to remove the IP it has > stored for the container, so container status calls don't return an IP for a > container that is not currently running.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8340) Capacity Scheduler Intra Queue Preemption Should Work When 3rd or more resources enabled.
[ https://issues.apache.org/jira/browse/YARN-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595923#comment-16595923 ] Sunil Govindan commented on YARN-8340: -- [~leftnoteasy] [~Zian Chen] [~eepayne] As this jira is marked for 3.2 as a critical, cud u pls help to take this forward or move out if its not feasible to finish in coming weeks. 3.2 code freeze date is nearby in a weeks. Kindly help to check the same. > Capacity Scheduler Intra Queue Preemption Should Work When 3rd or more > resources enabled. > - > > Key: YARN-8340 > URL: https://issues.apache.org/jira/browse/YARN-8340 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Priority: Critical > > Refer to comment from [~eepayne] and discussion below that: > https://issues.apache.org/jira/browse/YARN-8292?focusedCommentId=16482689=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16482689 > for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7505) RM REST endpoints generate malformed JSON
[
https://issues.apache.org/jira/browse/YARN-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595928#comment-16595928
]
Sunil Govindan commented on YARN-7505:
--
[~templedf]
As this jira is marked for 3.2 as a critical, cud u pls help to take this
forward or move out if its not feasible to finish in coming weeks. 3.2 code
freeze date is nearby in a weeks. Kindly help to check the same.
> RM REST endpoints generate malformed JSON
> -
>
> Key: YARN-7505
> URL: https://issues.apache.org/jira/browse/YARN-7505
> Project: Hadoop YARN
> Issue Type: Bug
> Components: restapi
>Affects Versions: 3.0.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: YARN-7505.001.patch, YARN-7505.002.patch
>
>
> For all endpoints that return DAOs that contain maps, the generated JSON is
> malformed. For example:
> % curl 'http://localhost:8088/ws/v1/cluster/apps'
> {"apps":{"app":[{"id":"application_1510777276702_0001","user":"daniel","name":"QuasiMonteCarlo","queue":"root.daniel","state":"RUNNING","finalStatus":"UNDEFINED","progress":5.0,"trackingUI":"ApplicationMaster","trackingUrl":"http://dhcp-10-16-0-181.pa.cloudera.com:8088/proxy/application_1510777276702_0001/","diagnostics":"","clusterId":1510777276702,"applicationType":"MAPREDUCE","applicationTags":"","priority":0,"startedTime":1510777317853,"finishedTime":0,"elapsedTime":21623,"amContainerLogs":"http://dhcp-10-16-0-181.pa.cloudera.com:8042/node/containerlogs/container_1510777276702_0001_01_01/daniel","amHostHttpAddress":"dhcp-10-16-0-181.pa.cloudera.com:8042","amRPCAddress":"dhcp-10-16-0-181.pa.cloudera.com:63371","allocatedMB":5120,"allocatedVCores":4,"reservedMB":0,"reservedVCores":0,"runningContainers":4,"memorySeconds":49820,"vcoreSeconds":26,"queueUsagePercentage":62.5,"clusterUsagePercentage":62.5,"resourceSecondsMap":{"entry":{"key":"test2","value":"0"},"entry":{"key":"test","value":"0"},"entry":{"key":"memory-mb","value":"49820"},"entry":{"key":"vcores","value":"26"}},"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0,"preemptedMemorySeconds":0,"preemptedVcoreSeconds":0,"preemptedResourceSecondsMap":{},"resourceRequests":[{"priority":20,"resourceName":"dhcp-10-16-0-181.pa.cloudera.com","capability":{"memory":1024,"vCores":1},"numContainers":8,"relaxLocality":true,"nodeLabelExpression":"","executionTypeRequest":{"executionType":"GUARANTEED","enforceExecutionType":true},"enforceExecutionType":false},{"priority":20,"resourceName":"/default-rack","capability":{"memory":1024,"vCores":1},"numContainers":8,"relaxLocality":true,"nodeLabelExpression":"","executionTypeRequest":{"executionType":"GUARANTEED","enforceExecutionType":true},"enforceExecutionType":false},{"priority":20,"resourceName":"*","capability":{"memory":1024,"vCores":1},"numContainers":8,"relaxLocality":true,"nodeLabelExpression":"","executionTypeRequest":{"executionType":"GUARANTEED","enforceExecutionType":true},"enforceExecutionType":false}],"logAggregationStatus":"DISABLED","unmanagedApplication":false,"amNodeLabelExpression":"","timeouts":{"timeout":[{"type":"LIFETIME","expiryTime":"UNLIMITED","remainingTimeInSeconds":-1}]}}]}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue
[
https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595937#comment-16595937
]
Sunil Govindan commented on YARN-8657:
--
I think there are no changes in ordering. Only annotation was bit confusing.
And existing trunk code was doings some updates in readLock which is out of
scope.
I think we can revisit the same in another patch also. I ll check this patch in
details and share comments if any. cc [~cheersyang]
> User limit calculation should be read-lock-protected within LeafQueue
> -
>
> Key: YARN-8657
> URL: https://issues.apache.org/jira/browse/YARN-8657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sumana Sathish
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8657.001.patch
>
>
> When async scheduling is enabled, user limit calculation could be wrong:
> It is possible that scheduler calculated a user_limit, but inside
> {{canAssignToUser}} it becomes staled.
> We need to protect user limit calculation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8680) YARN NM: Implement Iterable Abstraction for LocalResourceTrackerstate
[ https://issues.apache.org/jira/browse/YARN-8680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595939#comment-16595939 ] Sunil Govindan commented on YARN-8680: -- Hi [~pradeepambati] As this jira is marked for 3.2 as a blocker, cud u pls help to take this forward or move out if its not feasible to finish in coming weeks. 3.2 code freeze date is nearby in a weeks. Kindly help to check the same. > YARN NM: Implement Iterable Abstraction for LocalResourceTrackerstate > - > > Key: YARN-8680 > URL: https://issues.apache.org/jira/browse/YARN-8680 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Reporter: Pradeep Ambati >Assignee: Pradeep Ambati >Priority: Critical > Attachments: YARN-8680.00.patch, YARN-8680.01.patch > > > Similar to YARN-8242, implement iterable abstraction for > LocalResourceTrackerState to load completed and in progress resources when > needed rather than loading them all at a time for a respective state. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8220) Running Tensorflow on YARN with GPU and Docker - Examples
[ https://issues.apache.org/jira/browse/YARN-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595942#comment-16595942 ] Sunil Govindan commented on YARN-8220: -- hi [~leftnoteasy] As Submarine is in, i think this work is not very important to go in for tensorflow. If u agree, i can cancel patch and close the same. > Running Tensorflow on YARN with GPU and Docker - Examples > - > > Key: YARN-8220 > URL: https://issues.apache.org/jira/browse/YARN-8220 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Critical > Attachments: YARN-8220.001.patch, YARN-8220.002.patch, > YARN-8220.003.patch, YARN-8220.004.patch > > > Tensorflow could be run on YARN and could leverage YARN's distributed > features. > This spec fill will help to run Tensorflow on yarn with GPU/docker -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8718) Merge related work for YARN-3409
[ https://issues.apache.org/jira/browse/YARN-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8718: - Attachment: YARN-3409.004.patch > Merge related work for YARN-3409 > > > Key: YARN-8718 > URL: https://issues.apache.org/jira/browse/YARN-8718 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Priority: Major > Attachments: YARN-3409.001.patch, YARN-3409.002.patch, > YARN-3409.004.patch, YARN-8718.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8718) Merge related work for YARN-3409
[ https://issues.apache.org/jira/browse/YARN-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595953#comment-16595953 ] Sunil Govindan commented on YARN-8718: -- Thanks [~leftnoteasy]. Yes, i took from a wrong commit id earlier. Attaching correct one. > Merge related work for YARN-3409 > > > Key: YARN-8718 > URL: https://issues.apache.org/jira/browse/YARN-8718 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Priority: Major > Attachments: YARN-3409.001.patch, YARN-3409.002.patch, > YARN-8718.003.patch, YARN-8718.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8718) Merge related work for YARN-3409
[ https://issues.apache.org/jira/browse/YARN-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8718: - Attachment: YARN-8718.004.patch > Merge related work for YARN-3409 > > > Key: YARN-8718 > URL: https://issues.apache.org/jira/browse/YARN-8718 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Priority: Major > Attachments: YARN-3409.001.patch, YARN-3409.002.patch, > YARN-8718.003.patch, YARN-8718.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8718) Merge related work for YARN-3409
[ https://issues.apache.org/jira/browse/YARN-8718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8718: - Attachment: (was: YARN-3409.004.patch) > Merge related work for YARN-3409 > > > Key: YARN-8718 > URL: https://issues.apache.org/jira/browse/YARN-8718 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Priority: Major > Attachments: YARN-3409.001.patch, YARN-3409.002.patch, > YARN-8718.003.patch, YARN-8718.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8726) [UI2] YARN UI2 is not accessible when config.env file failed to load
[
https://issues.apache.org/jira/browse/YARN-8726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596104#comment-16596104
]
Sunil Govindan commented on YARN-8726:
--
Seems jenkins is down.
> [UI2] YARN UI2 is not accessible when config.env file failed to load
>
>
> Key: YARN-8726
> URL: https://issues.apache.org/jira/browse/YARN-8726
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
>Reporter: Akhil PB
>Assignee: Akhil PB
>Priority: Critical
> Attachments: YARN-8726.001.patch
>
>
> It is observed that yarn UI2 is not accessible. When UI2 is inspected, it
> gives below error
> {code:java}
> index.html:1 Refused to execute script from
> 'http://ctr-e138-1518143905142-456429-01-05.hwx.site:8088/ui2/config/configs.env'
> because its MIME type ('text/plain') is not executable, and strict MIME type
> checking is enabled.
> yarn-ui.js:219 base url:
> vendor.js:1978 ReferenceError: ENV is not defined
> at updateConfigs (yarn-ui.js:212)
> at Object.initialize (yarn-ui.js:218)
> at vendor.js:824
> at vendor.js:825
> at visit (vendor.js:3025)
> at Object.visit [as default] (vendor.js:3024)
> at DAG.topsort (vendor.js:750)
> at Class._runInitializer (vendor.js:825)
> at Class.runInitializers (vendor.js:824)
> at Class._bootSync (vendor.js:823)
> onerrorDefault @ vendor.js:1978
> trigger @ vendor.js:2967
> (anonymous) @ vendor.js:3006
> invoke @ vendor.js:626
> flush @ vendor.js:629
> flush @ vendor.js:619
> end @ vendor.js:642
> run @ vendor.js:648
> join @ vendor.js:648
> run.join @ vendor.js:1510
> (anonymous) @ vendor.js:1512
> fire @ vendor.js:230
> fireWith @ vendor.js:235
> ready @ vendor.js:242
> completed @ vendor.js:242
> vendor.js:823 Uncaught ReferenceError: ENV is not defined
> at updateConfigs (yarn-ui.js:212)
> at Object.initialize (yarn-ui.js:218)
> at vendor.js:824
> at vendor.js:825
> at visit (vendor.js:3025)
> at Object.visit [as default] (vendor.js:3024)
> at DAG.topsort (vendor.js:750)
> at Class._runInitializer (vendor.js:825)
> at Class.runInitializers (vendor.js:824)
> at Class._bootSync (vendor.js:823)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8220) Running Tensorflow on YARN with GPU and Docker - Examples
[ https://issues.apache.org/jira/browse/YARN-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8220: - Target Version/s: (was: 3.2.0) > Running Tensorflow on YARN with GPU and Docker - Examples > - > > Key: YARN-8220 > URL: https://issues.apache.org/jira/browse/YARN-8220 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Critical > Attachments: YARN-8220.001.patch, YARN-8220.002.patch, > YARN-8220.003.patch, YARN-8220.004.patch > > > Tensorflow could be run on YARN and could leverage YARN's distributed > features. > This spec fill will help to run Tensorflow on yarn with GPU/docker -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-8220) Running Tensorflow on YARN with GPU and Docker - Examples
[ https://issues.apache.org/jira/browse/YARN-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan resolved YARN-8220. -- Resolution: Done With Submarine, we have a better implementation for this. Hence let us close this and migrate the enhancements to Submarine. YARN-8135 > Running Tensorflow on YARN with GPU and Docker - Examples > - > > Key: YARN-8220 > URL: https://issues.apache.org/jira/browse/YARN-8220 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Critical > Attachments: YARN-8220.001.patch, YARN-8220.002.patch, > YARN-8220.003.patch, YARN-8220.004.patch > > > Tensorflow could be run on YARN and could leverage YARN's distributed > features. > This spec fill will help to run Tensorflow on yarn with GPU/docker -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8015) Complete placement constraint support for Capacity Scheduler
[
https://issues.apache.org/jira/browse/YARN-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585209#comment-16585209
]
Sunil Govindan commented on YARN-8015:
--
Thanks for the patch [~cheersyang]
Some quick questions:
# To support affinity for node partition, i think current patch supports to
fetch the partition from incoming expression. Setting in targetNodePartition is
enough?
# I just saw that {{if (!(ac instanceof
PlacementConstraint.SingleConstraint))}} is removed. But this check is needed,
correct?
> Complete placement constraint support for Capacity Scheduler
>
>
> Key: YARN-8015
> URL: https://issues.apache.org/jira/browse/YARN-8015
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Critical
> Attachments: YARN-8015.001.patch, YARN-8015.002.patch,
> YARN-8015.003.patch, YARN-8015.004.patch
>
>
> AppPlacementAllocator currently only supports intra-app anti-affinity
> placement constraints, once YARN-8002 and YARN-8013 are resolved, it needs to
> support inter-app constraints too. Also, this may require some refactoring on
> the existing code logic. Use this JIRA to track.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585620#comment-16585620 ] Sunil Govindan commented on YARN-7494: -- Fixed test case. Attaching new patch, > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7494: - Attachment: YARN-7494.18.patch > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch, > YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch, > YARN-7494.18.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, > multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8718) Merge related work for YARN-3409
Sunil Govindan created YARN-8718: Summary: Merge related work for YARN-3409 Key: YARN-8718 URL: https://issues.apache.org/jira/browse/YARN-8718 Project: Hadoop YARN Issue Type: Sub-task Reporter: Sunil Govindan -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
