[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383653#comment-17383653
]
Walker Carlson commented on KAFKA-13010:
It looks like the issue only shows up between cooperative rebalances and just
retrying will fix it. I will make a PR with a fix shortly
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383620#comment-17383620
]
Walker Carlson commented on KAFKA-13010:
I ran it for the commit before KIP-744 and it did fail. I am going to do a
binary search from when the test was introduced to see where it started failing
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383564#comment-17383564
]
Josep Prat commented on KAFKA-13010:
I only mentioned because it seems to be the only other change in that PR that
is not just moving implementations around.
If youbrun the test before the KIP-744 changes were introduced, does it also
fail after, let's say 200 iterations?
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383560#comment-17383560
]
Walker Carlson commented on KAFKA-13010:
This doesn't use the StreamsMetadata so I don't think that would be related
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383557#comment-17383557
]
Josep Prat commented on KAFKA-13010:
The only other difference left now is in StreamsMetadataImpl where the
immutable collections were created within the getters instead of the
constructor. A diff between StreamsMetadataImpl and the Deprecated
StreamsMetadata class will show the places where it changed
I guess it's another long shot
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383553#comment-17383553
]
Walker Carlson commented on KAFKA-13010:
it took about 126 runs but it still failed
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383537#comment-17383537
]
Josep Prat commented on KAFKA-13010:
If this is the reason why we are seeing this bug, by replacing line 60 in
ThreadMetadataImpl the following might cause the test to not fail:
{code:java}
this.producerClientIds = producerClientIds;
{code}
As I can't reproduce the test locally would you be able to try this?
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383533#comment-17383533
]
Walker Carlson commented on KAFKA-13010:
That sounds likely as the ThreadMetadata is retrieved using
`metadataForLocalThreads()`
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383531#comment-17383531
]
Josep Prat commented on KAFKA-13010:
One thing that was modified under that PR that might cause some race conditions
is the fact that StreamsMetadataImpl now saves all collections as immutable
ones during creation instead of doing it inside the getters. Similar thing
ThreadMetadataImpl that now saves producerClientIds as an immutable collection
within the constructor.
However, TaskMetadataImpl behaves the same in regards of immutable collections.
I doubt it is related, but it's worth a shot.
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383516#comment-17383516
]
Walker Carlson commented on KAFKA-13010:
That does seem to be the case. I think somehow those "existing active tasks"
are getting excluded from the `activeTasks()` list in the `ThreadMetadata`
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383513#comment-17383513
]
Josep Prat commented on KAFKA-13010:
To add more to the mystery, I'm running the test until failure and I'm already
over 250 repetitions and can't reproduce the bug (on a i7 CPU 4 cores, 8
threads).
Maybe what I'm about to say it's pretty obvious, but according to the logs you
provide, it seems that when the test fails, TaskManager prints:
{code:java}
Handle new assignment with:
New active tasks: []
New standby tasks: []
Existing active tasks: [0_0]
Existing standby tasks: []
{code}
instead of:
{code:java}
Handle new assignment with:
New active tasks: [0_0]
New standby tasks: []
Existing active tasks: []
Existing standby tasks: []
{code}
(or at least it's what's been printed on my machine when test run
successfully). Notice the "0_0" task being in the "existing active tasks" when
failing instead of being in "new active tasks".
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383504#comment-17383504
]
A. Sophie Blee-Goldman commented on KAFKA-13010:
Yeah it's honestly pretty hard to imagine how that could have introduced a bug
like this, but the timing definitely is suspicious. Seeing as Walker was able
to reproduce it after a "reasonable" number of retries, it should be easy to
confirm the suspicion by just running the test again with sufficient repeats on
the commit just before KIP-744 was merged.
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383484#comment-17383484
]
Josep Prat commented on KAFKA-13010:
If it happened after KIP-744 probably I introduced the regression .
I will also try to look if I can see where is the mistake. If I remember
correctly, most of the code I wrote was just copied over from one place to
another.
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383474#comment-17383474
]
Walker Carlson commented on KAFKA-13010:
It is possible. I ran till failure (75 runs). And it seems that the metadata
was reporting no tasks, so I would agree.
{code:java}
[ThreadMetadata{threadName=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-2,
threadState=RUNNING, activeTasks=[], standbyTasks=[],
consumerClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-2-consumer,
restoreConsumerClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-2-restore-consumer,
producerClientIds=[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-2-producer],
adminClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-admin},
ThreadMetadata{threadName=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-1,
threadState=RUNNING, activeTasks=[], standbyTasks=[],
consumerClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-1-consumer,
restoreConsumerClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-1-restore-consumer,
producerClientIds=[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-StreamThread-1-producer],
adminClientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-c1a902ee-54b1-4b8e-a195-98d4a6143c1e-admin}]
{code}
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383445#comment-17383445
]
A. Sophie Blee-Goldman commented on KAFKA-13010:
It's worth noting that this test seemed to be pretty stable for roughly a full
release cycle, then started failing pretty frequently right after
[KIP-744|https://cwiki.apache.org/confluence/display/KAFKA/KIP-744%3A+Migrate+TaskMetadata+and+ThreadMetadata+to+an+interface+with+internal+implementation]
/ KAFKA-12849 was merged (June 25th). [~wcarlson5] I wonder if the changes to
the metadata hierarchy could have introduced a potentially serious bug?
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383431#comment-17383431
]
Walker Carlson commented on KAFKA-13010:
This failure should will effect both tests. When it is getting the task list it
asserts that there is only one task in the very simple topology. I don't know
if this is really a problem with the test or even the feature that the test is
targeted at. It looks like the tasks created are not consistent. The task
behavior might be intentional but I don't think so. I will see if I can
reproduce and look at the logs.
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: Walker Carlson
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380939#comment-17380939
]
A. Sophie Blee-Goldman commented on KAFKA-13010:
Not the exact same test, but I did manage to reproduce this same "only one
task" failure in
TaskMetadataIntegrationTest.shouldReportCorrectEndOffsetInformation:
{code:java}
java.lang.AssertionError: only one task
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
at
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:163)
at
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectEndOffsetInformation(TaskMetadataIntegrationTest.java:144)
{code}
I saved the logs since they should actually be the full, un-truncated logs –
hope this helps:
[^TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf]
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Priority: Major
> Labels: flaky-test
> Attachments:
> TaskMetadataIntegrationTest#shouldReportCorrectEndOffsetInformation.rtf
>
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380838#comment-17380838
]
A. Sophie Blee-Goldman commented on KAFKA-13010:
[~wcarl...@confluent.io] I'm guessing you wrote this test so you have the most
context, can you reproduce this locally and take a minute or two to look
through the logs and see if anything jumps out at you?
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Priority: Major
> Labels: flaky-test
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380444#comment-17380444
]
Bruno Cadonna commented on KAFKA-13010:
---
For the future assignee of this ticket: I was able to reproduce this failure
multiple times by running the test in IntelliJ in the until failure mode. It
failed quite quickly after approx. 25 runs.
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Priority: Major
> Labels: flaky-test
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380212#comment-17380212
]
A. Sophie Blee-Goldman commented on KAFKA-13010:
Failed again
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-10985/6/testReport/junit/org.apache.kafka.streams.integration/TaskMetadataIntegrationTest/Build___JDK_11_and_Scala_2_13___shouldReportCorrectCommittedOffsetInformation/
> Flaky test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
> ---
>
> Key: KAFKA-13010
> URL: https://issues.apache.org/jira/browse/KAFKA-13010
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bruno Cadonna
>Priority: Major
> Labels: flaky-test
>
> Integration test {{test
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
> sometimes fails with
> {code:java}
> java.lang.AssertionError: only one task
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
> at
> org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()
[
https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372977#comment-17372977
]
Bruno Cadonna commented on KAFKA-13010:
---
Some logs that might be interesting:
{code}
[2021-06-29 12:19:40,200] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2-consumer]
Finished unstable assignment of tasks, a followup rebalance will be scheduled.
(org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor:818)
[2021-06-29 12:19:40,200] WARN [Consumer
clientId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2-consumer,
groupId=TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation]
The following subscribed topics are not assigned to any members:
[inputTaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation]
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:611)
[2021-06-29 12:19:40,200] INFO [GroupCoordinator 0]: Assignment received from
leader
TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2-consumer-0a548162-9e3f-4003-98c5-54ece6f5e1b8
for group
TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation
for generation 2. The group has 2 members, 0 of which are static.
(kafka.coordinator.group.GroupCoordinator:66)
[2021-06-29 12:19:40,201] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-1-consumer]
Requested to schedule immediate rebalance for new tasks to be safely revoked
from current owner.
(org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor:1300)
[2021-06-29 12:19:40,201] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
State transition from RUNNING to PARTITIONS_REVOKED
(org.apache.kafka.streams.processor.internals.StreamThread:229)
[2021-06-29 12:19:40,201] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-1]
Handle new assignment with:
New active tasks: []
New standby tasks: []
Existing active tasks: []
Existing standby tasks: []
(org.apache.kafka.streams.processor.internals.TaskManager:263)
[2021-06-29 12:19:40,201] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-1]
State transition from STARTING to PARTITIONS_ASSIGNED
(org.apache.kafka.streams.processor.internals.StreamThread:229)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
task [0_0] Suspended RUNNING
(org.apache.kafka.streams.processor.internals.StreamTask:1187)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
task [0_0] Suspended running
(org.apache.kafka.streams.processor.internals.StreamTask:300)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
partition revocation took 1 ms.
(org.apache.kafka.streams.processor.internals.StreamThread:97)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2-consumer]
No followup rebalance was requested, resetting the rebalance schedule.
(org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor:1306)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
Handle new assignment with:
New active tasks: []
New standby tasks: []
Existing active tasks: [0_0]
Existing standby tasks: []
(org.apache.kafka.streams.processor.internals.TaskManager:263)
[2021-06-29 12:19:40,202] INFO stream-thread
[TaskMetadataTest_TaskMetadataIntegrationTestshouldReportCorrectCommittedOffsetInformation-6889e6c9-c4fa-427c-83bf-469b33a34bb5-StreamThread-2]
task [0_0] Closing record collector clean
(org.apache.kafka.streams.processor.internals.RecordCollectorImpl:268)
[2021-06-29
