[
https://issues.apache.org/jira/browse/YARN-9551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382157#comment-17382157
]
Andras Gyori commented on YARN-9551:
------------------------------------
Unfortunately it is really not guaranteed for the async pushes to be merged.
The intermittent failure is due to the fact, that the third entity is put into
the queue after the the polling on TimelineV2ClientImpl#515. Since it is a non
blocking operation, the nextEntityInQueue is null and therefore the second
entity is published before it has a chance to merge with the third entity.
Either we would need a guarantee (eg. wait on the queue instead of polling), or
we just eliminate this wrong assumption.
I have chosen the latter, since I am reluctant to change the client behaviour
just for the sake of a test.
> TestTimelineClientV2Impl.testSyncCall fails intermittent
> --------------------------------------------------------
>
> Key: YARN-9551
> URL: https://issues.apache.org/jira/browse/YARN-9551
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2, test
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestTimelineClientV2Impl.testSyncCall fails intermittent
> {code:java}
> Failed
> org.apache.hadoop.yarn.client.api.impl.TestTimelineClientV2Impl.testSyncCall
> Failing for the past 1 build (Since #24083 )
> Took 1.5 sec.
> Error Message
> TimelineEntities not published as desired expected:<3> but was:<4>
> Stacktrace
> java.lang.AssertionError: TimelineEntities not published as desired
> expected:<3> but was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at
> org.apache.hadoop.yarn.client.api.impl.TestTimelineClientV2Impl.testSyncCall(TestTimelineClientV2Impl.java:251)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Standard Output
> 2019-05-13 15:33:46,596 WARN [main] util.NativeCodeLoader
> (NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library
> for your platform... using builtin-java classes where applicable
> 2019-05-13 15:33:47,763 INFO [main] impl.TestTimelineClientV2Impl
> (TestTimelineClientV2Impl.java:printReceivedEntities(413)) - Entities
> Published @ index 0 : 1,
> 2019-05-13 15:33:47,764 INFO [main] impl.TestTimelineClientV2Impl
> (TestTimelineClientV2Impl.java:printReceivedEntities(413)) - Entities
> Published @ index 1 : 2,
> 2019-05-13 15:33:47,764 INFO [main] impl.TestTimelineClientV2Impl
> (TestTimelineClientV2Impl.java:printReceivedEntities(413)) - Entities
> Published @ index 2 : 3,
> 2019-05-13 15:33:47,764 INFO [main] impl.TestTimelineClientV2Impl
> (TestTimelineClientV2Impl.java:printReceivedEntities(413)) - Entities
> Published @ index 3 : 4,
> 2019-05-13 15:33:47,765 INFO [main] impl.TimelineV2ClientImpl
> (TimelineV2ClientImpl.java:stop(563)) - Stopping TimelineClient.
> 2019-05-13 15:33:47,765 INFO [pool-1-thread-1] impl.TimelineV2ClientImpl
> (TimelineV2ClientImpl.java:run(429)) - Timeline dispatcher thread was
> interrupted
> 2019-05-13 15:33:47,766 INFO [main] impl.TimelineV2ClientImpl
> (TimelineV2ClientImpl.java:stop(563)) - Stopping TimelineClient.
> 2019-05-13 15:33:47,766 INFO [pool-2-thread-1] impl.TimelineV2ClientImpl
> (TimelineV2ClientImpl.java:run(429)) - Timeline dispatcher thread was
> interrupted {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
