[jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
[
https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425718#comment-15425718
]
Hadoop QA commented on YARN-4376:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color}
| {color:red} YARN-4376 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12773610/YARN-4376.2.patch |
| JIRA Issue | YARN-4376 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/12819/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Memory Timeline Store return incorrect results on fromId paging
> ---
>
> Key: YARN-4376
> URL: https://issues.apache.org/jira/browse/YARN-4376
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: YARN-4376.2.patch
>
>
> As pointed out correctly by [~jlowe].
> https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831
> The MemoryTimelineStore cannot page correctly when using fromId. This is due
> switching between data structures that apparently have different natural
> sorting. In addition, the approach of creating a new data structure every
> time from scratch is costly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
[ https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118184#comment-15118184 ] Jason Lowe commented on YARN-4376: -- Patch looks ok -- do you have any performance numbers? Wondering how expensive it is to maintain the treeset. Also this will need to be reconciled with the proposed changes in YARN-4219. I believe that proposed change also fixes the issue, although it's creating the treeset on demand which could be slow for answering getEntities queries on a large dataset. I think it's straightforward to reconcile, just need explicit valueSetIterator overrides in the memory timeline store map adapters. > Memory Timeline Store return incorrect results on fromId paging > --- > > Key: YARN-4376 > URL: https://issues.apache.org/jira/browse/YARN-4376 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: YARN-4376.2.patch > > > As pointed out correctly by [~jlowe]. > https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831 > The MemoryTimelineStore cannot page correctly when using fromId. This is due > switching between data structures that apparently have different natural > sorting. In addition, the approach of creating a new data structure every > time from scratch is costly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
[ https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109332#comment-15109332 ] Jonathan Eagles commented on YARN-4376: --- [~jlowe], can you take a look at this jira for review. > Memory Timeline Store return incorrect results on fromId paging > --- > > Key: YARN-4376 > URL: https://issues.apache.org/jira/browse/YARN-4376 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: YARN-4376.2.patch > > > As pointed out correctly by [~jlowe]. > https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831 > The MemoryTimelineStore cannot page correctly when using fromId. This is due > switching between data structures that apparently have different natural > sorting. In addition, the approach of creating a new data structure every > time from scratch is costly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
[
https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063691#comment-15063691
]
Hadoop QA commented on YARN-4376:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 12s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
(total was 11, now 12). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 36s {color}
| {color:red} hadoop-yarn-server-applicationhistoryservice in the patch failed
with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 33s {color}
| {color:red} hadoop-yarn-server-applicationhistoryservice in the patch failed
with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests |
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryClientService
|
| JDK v1.7.0_91 Failed junit tests |
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryClientService
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
[jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
[
https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018856#comment-15018856
]
Jonathan Eagles commented on YARN-4376:
---
According the
[PriorityQueue|http://docs.oracle.com/javase/7/docs/api/java/util/PriorityQueue.html]
java doc.
{code}
The Iterator provided in method iterator() is not guaranteed to traverse the
elements of the priority queue in any particular order. If you need ordered
traversal, consider using Arrays.sort(pq.toArray()).
{code}
> Memory Timeline Store return incorrect results on fromId paging
> ---
>
> Key: YARN-4376
> URL: https://issues.apache.org/jira/browse/YARN-4376
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>
> As pointed out correctly by [~jlowe].
> https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831
> The MemoryTimelineStore cannot page correctly when using fromId. This is due
> switching between data structures that apparently have different natural
> sorting. In addition, the approach of creating a new data structure every
> time from scratch is costly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
