[jira] [Commented] (YARN-3726) Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
[
https://issues.apache.org/jira/browse/YARN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369798#comment-15369798
]
Hudson commented on YARN-3726:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10074/])
YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing (sjlee:
rev db891f51d0b76a199a85c98319764bb115626a47)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineWriterImpl.java
> Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
> --
>
> Key: YARN-3726
> URL: https://issues.apache.org/jira/browse/YARN-3726
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Fix For: YARN-2928
>
> Attachments: YARN-3726-YARN-2928.001.patch
>
>
> There is a very fascinating bug that was introduced by the test data in the
> metrics time series check in the unit test in TestHBaseTimelineWriterImpl in
> YARN-3411.
> The unit test failure seen is
> {code}
> Error Message
> expected:<1> but was:<6>
> Stacktrace
> java.lang.AssertionError: expected:<1> but was:<6>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.checkMetricsTimeseries(TestHBaseTimelineWriterImpl.java:219)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.testWriteEntityToHBase(TestHBaseTimelineWriterImpl.java:204)
> {code}
> The test data had 6 timestamps that belonged to 22nd April 2015. When the
> patch in YARN-3411 was submitted and tested by Hadoop QA on May 19th, the
> unit test was working fine. Fast forward a few more days and the test started
> failing. There has been no relevant code change or package version change
> interim. The change that is triggering the unit test failure is the passage
> of time.
> The reason for test failure is that the metrics time series data lives in a
> column family which has a TTL set to 30 days. Metrics time series data was
> written to the mini hbase cluster with cell timestamps set to April 22nd.
> Based on the column family configuration, hbase started deleting the data
> that was older than 30 days and the test started failing. The last value is
> retained, hence there is one value fetched from hbase.
> Will submit a patch with the test case fixed shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3726) Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
[
https://issues.apache.org/jira/browse/YARN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563222#comment-14563222
]
Joep Rottinghuis commented on YARN-3726:
Aside from the immediate fix for the unit test, this makes me wonder if we
should protect against this in the writer itself for production cases.
If somebody tries to write an entity (metric) with a date older than the TTL,
should we simply refuse and throw an IOException?
That way at least folks notice, otherwise we simply add load and silently
ignore, as what happened with the unit test. If was only the confirmation that
the data was there that failed, not the code trying to write something that
clearly didn't make sense.
> Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
> --
>
> Key: YARN-3726
> URL: https://issues.apache.org/jira/browse/YARN-3726
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Fix For: YARN-2928
>
> Attachments: YARN-3726-YARN-2928.001.patch
>
>
> There is a very fascinating bug that was introduced by the test data in the
> metrics time series check in the unit test in TestHBaseTimelineWriterImpl in
> YARN-3411.
> The unit test failure seen is
> {code}
> Error Message
> expected:<1> but was:<6>
> Stacktrace
> java.lang.AssertionError: expected:<1> but was:<6>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.checkMetricsTimeseries(TestHBaseTimelineWriterImpl.java:219)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.testWriteEntityToHBase(TestHBaseTimelineWriterImpl.java:204)
> {code}
> The test data had 6 timestamps that belonged to 22nd April 2015. When the
> patch in YARN-3411 was submitted and tested by Hadoop QA on May 19th, the
> unit test was working fine. Fast forward a few more days and the test started
> failing. There has been no relevant code change or package version change
> interim. The change that is triggering the unit test failure is the passage
> of time.
> The reason for test failure is that the metrics time series data lives in a
> column family which has a TTL set to 30 days. Metrics time series data was
> written to the mini hbase cluster with cell timestamps set to April 22nd.
> Based on the column family configuration, hbase started deleting the data
> that was older than 30 days and the test started failing. The last value is
> retained, hence there is one value fetched from hbase.
> Will submit a patch with the test case fixed shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3726) Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
[
https://issues.apache.org/jira/browse/YARN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562165#comment-14562165
]
Hadoop QA commented on YARN-3726:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 6m 25s | Pre-patch YARN-2928 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 10m 17s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 28s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 40s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 53s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 45s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 42s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 1m 16s | Tests passed in
hadoop-yarn-server-timelineservice. |
| | | 22m 32s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12735742/YARN-3726-YARN-2928.001.patch
|
| Optional Tests | javac unit findbugs checkstyle |
| git revision | YARN-2928 / e19566a |
| hadoop-yarn-server-timelineservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8106/artifact/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8106/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8106/console |
This message was automatically generated.
> Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
> --
>
> Key: YARN-3726
> URL: https://issues.apache.org/jira/browse/YARN-3726
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Attachments: YARN-3726-YARN-2928.001.patch
>
>
> There is a very fascinating bug that was introduced by the test data in the
> metrics time series check in the unit test in TestHBaseTimelineWriterImpl in
> YARN-3411.
> The unit test failure seen is
> {code}
> Error Message
> expected:<1> but was:<6>
> Stacktrace
> java.lang.AssertionError: expected:<1> but was:<6>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.checkMetricsTimeseries(TestHBaseTimelineWriterImpl.java:219)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.testWriteEntityToHBase(TestHBaseTimelineWriterImpl.java:204)
> {code}
> The test data had 6 timestamps that belonged to 22nd April 2015. When the
> patch in YARN-3411 was submitted and tested by Hadoop QA on May 19th, the
> unit test was working fine. Fast forward a few more days and the test started
> failing. There has been no relevant code change or package version change
> interim. The change that is triggering the unit test failure is the passage
> of time.
> The reason for test failure is that the metrics time series data lives in a
> column family which has a TTL set to 30 days. Metrics time series data was
> written to the mini hbase cluster with cell timestamps set to April 22nd.
> Based on the column family configuration, hbase started deleting the data
> that was older than 30 days and the test started failing. The last value is
> retained, hence there is one value fetched from hbase.
> Will submit a patch with the test case fixed shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3726) Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
[
https://issues.apache.org/jira/browse/YARN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562136#comment-14562136
]
Sangjin Lee commented on YARN-3726:
---
LGTM too. Once the jenkins comes back green (and unless there is an objection),
I'll commit the patch.
> Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
> --
>
> Key: YARN-3726
> URL: https://issues.apache.org/jira/browse/YARN-3726
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Attachments: YARN-3726-YARN-2928.001.patch
>
>
> There is a very fascinating bug that was introduced by the test data in the
> metrics time series check in the unit test in TestHBaseTimelineWriterImpl in
> YARN-3411.
> The unit test failure seen is
> {code}
> Error Message
> expected:<1> but was:<6>
> Stacktrace
> java.lang.AssertionError: expected:<1> but was:<6>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.checkMetricsTimeseries(TestHBaseTimelineWriterImpl.java:219)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.testWriteEntityToHBase(TestHBaseTimelineWriterImpl.java:204)
> {code}
> The test data had 6 timestamps that belonged to 22nd April 2015. When the
> patch in YARN-3411 was submitted and tested by Hadoop QA on May 19th, the
> unit test was working fine. Fast forward a few more days and the test started
> failing. There has been no relevant code change or package version change
> interim. The change that is triggering the unit test failure is the passage
> of time.
> The reason for test failure is that the metrics time series data lives in a
> column family which has a TTL set to 30 days. Metrics time series data was
> written to the mini hbase cluster with cell timestamps set to April 22nd.
> Based on the column family configuration, hbase started deleting the data
> that was older than 30 days and the test started failing. The last value is
> retained, hence there is one value fetched from hbase.
> Will submit a patch with the test case fixed shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3726) Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
[
https://issues.apache.org/jira/browse/YARN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562115#comment-14562115
]
Joep Rottinghuis commented on YARN-3726:
+1 applying the patch resolves the unit test failure (in Eclipse) for me.
> Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data
> --
>
> Key: YARN-3726
> URL: https://issues.apache.org/jira/browse/YARN-3726
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Attachments: YARN-3726-YARN-2928.001.patch
>
>
> There is a very fascinating bug that was introduced by the test data in the
> metrics time series check in the unit test in TestHBaseTimelineWriterImpl in
> YARN-3411.
> The unit test failure seen is
> {code}
> Error Message
> expected:<1> but was:<6>
> Stacktrace
> java.lang.AssertionError: expected:<1> but was:<6>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.checkMetricsTimeseries(TestHBaseTimelineWriterImpl.java:219)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineWriterImpl.testWriteEntityToHBase(TestHBaseTimelineWriterImpl.java:204)
> {code}
> The test data had 6 timestamps that belonged to 22nd April 2015. When the
> patch in YARN-3411 was submitted and tested by Hadoop QA on May 19th, the
> unit test was working fine. Fast forward a few more days and the test started
> failing. There has been no relevant code change or package version change
> interim. The change that is triggering the unit test failure is the passage
> of time.
> The reason for test failure is that the metrics time series data lives in a
> column family which has a TTL set to 30 days. Metrics time series data was
> written to the mini hbase cluster with cell timestamps set to April 22nd.
> Based on the column family configuration, hbase started deleting the data
> that was older than 30 days and the test started failing. The last value is
> retained, hence there is one value fetched from hbase.
> Will submit a patch with the test case fixed shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
