[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969748#comment-14969748
]
zhihai xu commented on YARN-4256:
-
committed it to trunk and branch-2, Thanks [~Prabhu Joseph] for reporting this
issue, thanks [~hex108] for the patch and thanks [~brahmareddy] for additional
review!
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969840#comment-14969840
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #570 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/570/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969857#comment-14969857
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8691 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8691/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969921#comment-14969921
]
Hudson commented on YARN-4256:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #1306 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1306/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969973#comment-14969973
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #585 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/585/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970180#comment-14970180
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2517 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2517/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970225#comment-14970225
]
Jun Gong commented on YARN-4256:
Thanks [~zxu] for the review and commit!
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970270#comment-14970270
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #528 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/528/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970237#comment-14970237
]
Hudson commented on YARN-4256:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/])
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed (zxu:
rev 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967355#comment-14967355
]
zhihai xu commented on YARN-4256:
-
+1 LGTM, Will commit tomorrow if no one objects.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967418#comment-14967418
]
Brahma Reddy Battula commented on YARN-4256:
[~Prabhu Joseph] nice finding..[~hex108] thanks for patch..LGTM..
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965146#comment-14965146
]
Jun Gong commented on YARN-4256:
[~zxu], could you please help review it?
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965488#comment-14965488
]
zhihai xu commented on YARN-4256:
-
Thanks for reporting this issue [~Prabhu Joseph]! Thanks for the patch
[~hex108]! The patch looks most good. Can we change '+' to '*'
(\\.\\d+)? => (\\.\\d*)? So we can relax the condition to support 1024. mb.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966068#comment-14966068
]
Jun Gong commented on YARN-4256:
Thanks [~zxu] for the review and comments.
Attach a new patch to address it and add a new test for the case '1024. mb'.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966208#comment-14966208
]
Hadoop QA commented on YARN-4256:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 26s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 9m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 47s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 26s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 57s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 48s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 38s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 36s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 64m 35s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 110m 49s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12767706/YARN-4256.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0c4af0f |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9504/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9504/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9504/console |
This message was automatically generated.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch, YARN-4256.002.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960049#comment-14960049
]
Prabhu Joseph commented on YARN-4256:
-
Thanks Jun Gong.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956594#comment-14956594
]
Jun Gong commented on YARN-4256:
Thanks [~Prabhu Joseph] for reporting. How about considering the value before
decimal? If OK, I will attach a patch.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Priority: Minor
> Fix For: 2.7.2
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4256) YARN fair scheduler vcores with decimal values
[
https://issues.apache.org/jira/browse/YARN-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956833#comment-14956833
]
Hadoop QA commented on YARN-4256:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 4s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 48s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 47s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 55s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 41s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 40s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 63m 3s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 108m 4s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766517/YARN-4256.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d6c8bad |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9440/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9440/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9440/console |
This message was automatically generated.
> YARN fair scheduler vcores with decimal values
> --
>
> Key: YARN-4256
> URL: https://issues.apache.org/jira/browse/YARN-4256
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Jun Gong
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-4256.001.patch
>
>
> When the queue with vcores is in decimal value, the value after the decimal
> point is taken as vcores by FairScheduler.
> For the below queue,
> 2 mb,20 vcores,20.25 disks
> 3 mb,40.2 vcores,30.25 disks
> When many applications submitted parallely into queue, all were in PENDING
> state as the vcores is taken as 2 skipping the value 40.
> The code FairSchedulerConfiguration.java to Pattern match the vcores has to
> be improved in such a way either throw
> AllocationConfigurationException("Missing resource") or consider the value
> before decimal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
