[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985422#comment-15985422
 ] 

ASF GitHub Bot commented on KAFKA-5124:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/2921


> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-26 Thread Armin Braun (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984980#comment-15984980
 ] 

Armin Braun commented on KAFKA-5124:


[~mjsax] I think the same fix I just added here 
https://github.com/apache/kafka/pull/2921 should also work for KAFKA-5005.

> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984973#comment-15984973
 ] 

ASF GitHub Bot commented on KAFKA-5124:
---

GitHub user original-brownbear opened a pull request:

https://github.com/apache/kafka/pull/2921

KAFKA-5124: autocommit reset earliest fixes race condition

Fixes 
`org.apache.kafka.streams.integration.utils.IntegrationTestUtils#readKeyValues` 
potentially starting to `poll` for stream output after the stream finished 
sending the test data and hence missing it when working with `latest` offsets.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/original-brownbear/kafka KAFKA-5124

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/2921.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2921


commit bdd06b97c57ec06714f1f654d5b2f2621143224f
Author: Armin Braun 
Date:   2017-04-26T15:10:44Z

KAFKA-5124: autocommit reset earliest fixes race condition




> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-25 Thread Armin Braun (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983539#comment-15983539
 ] 

Armin Braun commented on KAFKA-5124:


@msjax certainly looks like it at least. I only found about 30 mins for this 
today (haven't quite reached the root yet) but should be able to figure it out 
tomorrow morning after some sleep :)

What I have learnt so far is:
* I have some indication this could be a "leakish" kind of thing ... once I ran 
like 500 iterations to get the test to fail in this way, it fails like every 
10th run, while the initial 500 runs are green. I could never make it fail 
without a bunch of "warmup" runs so far.
* Not a timeout issue ... test normally takes ~ 1s -> 30s are plenty -> setting 
Long.MAX as timeout makes it go on forever.
* This would be the thread dump with Long.MAX for the timeout (in case you can 
spot something my tired eyes can't at this point :D):

{code}
"main@1" prio=5 tid=0x1 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
  at sun.nio.ch.EPollArrayWrapper.epollWait(EPollArrayWrapper.java:-1)
  at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
  at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
  - locked <0x2553> (a sun.nio.ch.EPollSelectorImpl)
  - locked <0x2554> (a java.util.Collections$UnmodifiableSet)
  - locked <0x2555> (a sun.nio.ch.Util$3)
  at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
  at org.apache.kafka.common.network.Selector.select(Selector.java:493)
  at org.apache.kafka.common.network.Selector.poll(Selector.java:302)
  at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:359)
  at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:230)
  - locked <0x2556> (a 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient)
  at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:206)
  at 
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1055)
  at 
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1002)
  at 
org.apache.kafka.streams.integration.utils.IntegrationTestUtils.readKeyValues(IntegrationTestUtils.java:94)
  at 
org.apache.kafka.streams.integration.utils.IntegrationTestUtils$1.conditionMet(IntegrationTestUtils.java:199)
  at 
org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:256)
  at 
org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
  at 
org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:220)
  at 
org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:141)
  at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source:-1)
  at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
  at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
  at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
  at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
  at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
  at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
  at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
  at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
  at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
  at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:137)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-25 Thread Matthias J. Sax (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983498#comment-15983498
 ] 

Matthias J. Sax commented on KAFKA-5124:


I guess is related to https://issues.apache.org/jira/browse/KAFKA-5005 -- 
assuming both issues do have the same root cause. WDYT? 

> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-25 Thread Eno Thereska (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982917#comment-15982917
 ] 

Eno Thereska commented on KAFKA-5124:
-

Thanks [~original-brownbear]

> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5124) shouldInnerLeftJoin unit test fails

2017-04-25 Thread Armin Braun (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982909#comment-15982909
 ] 

Armin Braun commented on KAFKA-5124:


Trying this one, I can reproduce it locally :)

> shouldInnerLeftJoin unit test fails
> ---
>
> Key: KAFKA-5124
> URL: https://issues.apache.org/jira/browse/KAFKA-5124
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Eno Thereska
>Assignee: Armin Braun
> Fix For: 0.11.0.0
>
>
> Unit test on trunk gives occasional failure:
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest > 
> shouldInnerLeftJoin FAILED
> java.lang.AssertionError: Condition not met within timeout 3. 
> Expecting 1 records from topic output- while only received 0: []
> at 
> org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:265)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:207)
> at 
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:176)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.verifyKTableKTableJoin(KTableKTableJoinIntegrationTest.java:222)
> at 
> org.apache.kafka.streams.integration.KTableKTableJoinIntegrationTest.shouldInnerLeftJoin(KTableKTableJoinIntegrationTest.java:143)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)