[ https://issues.apache.org/jira/browse/KAFKA-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruno Cadonna updated KAFKA-10148: ---------------------------------- Description: System test {{StreamsEosTest.test_failure_and_recovery}} for eos-beta exposes a bug that results in wrong results in the output topic. The cause seems to be a too low end offset during restoration of a state store. Example: The system test computes a minimum aggregate over records in an input topic and writes the results to an output topic. The input topic partition {{data-1}} contains the following records among others: {code} ... offset: 125 CreateTime: 1591690264681 keysize: 5 valuesize: 4 sequence: 125 headerKeys: [] key: 14920 payload: 9215 ... offset: 1611 CreateTime: 1591690297424 keysize: 5 valuesize: 4 sequence: 1611 headerKeys: [] key: 14920 payload: 1595 ... offset: 2104 CreateTime: 1591690308542 keysize: 5 valuesize: 4 sequence: 2104 headerKeys: [] key: 14920 payload: 9274 ... {code} The output topic partition {{min-1}} contains: {code} ... offset: 125 CreateTime: 1591690264681 keysize: 5 valuesize: 4 sequence: 125 headerKeys: [] key: 14920 payload: 9215 ... offset: 1828 CreateTime: 1591690297424 keysize: 5 valuesize: 4 sequence: 1213 headerKeys: [] key: 14920 payload: 1595 ... offset: 2324 CreateTime: 1591690308542 keysize: 5 valuesize: 4 sequence: 10 headerKeys: [] key: 14920 payload: 9215 ... {code} The last record is obviously wrong because 1595 is less than 9215. was: System test {{StreamsEosTest.test_failure_and_recovery}} for eos-beta exposes a bug that results in wrong results in the output topic. The cause seems to be a too low end offset during restoration of a state store. df > Kafka Streams Restores too few Records with eos-beta Enabled > ------------------------------------------------------------- > > Key: KAFKA-10148 > URL: https://issues.apache.org/jira/browse/KAFKA-10148 > Project: Kafka > Issue Type: Bug > Components: streams > Reporter: Bruno Cadonna > Assignee: Bruno Cadonna > Priority: Blocker > Fix For: 2.6.0 > > > System test {{StreamsEosTest.test_failure_and_recovery}} for eos-beta exposes > a bug that results in wrong results in the output topic. The cause seems to > be a too low end offset during restoration of a state store. > Example: > The system test computes a minimum aggregate over records in an input topic > and writes the results to an output topic. The input topic partition > {{data-1}} contains the following records among others: > {code} > ... > offset: 125 CreateTime: 1591690264681 keysize: 5 valuesize: 4 sequence: 125 > headerKeys: [] key: 14920 payload: 9215 > ... > offset: 1611 CreateTime: 1591690297424 keysize: 5 valuesize: 4 sequence: 1611 > headerKeys: [] key: 14920 payload: 1595 > ... > offset: 2104 CreateTime: 1591690308542 keysize: 5 valuesize: 4 sequence: 2104 > headerKeys: [] key: 14920 payload: 9274 > ... > {code} > The output topic partition {{min-1}} contains: > {code} > ... > offset: 125 CreateTime: 1591690264681 keysize: 5 valuesize: 4 sequence: 125 > headerKeys: [] key: 14920 payload: 9215 > ... > offset: 1828 CreateTime: 1591690297424 keysize: 5 valuesize: 4 sequence: 1213 > headerKeys: [] key: 14920 payload: 1595 > ... > offset: 2324 CreateTime: 1591690308542 keysize: 5 valuesize: 4 sequence: 10 > headerKeys: [] key: 14920 payload: 9215 > ... > {code} > The last record is obviously wrong because 1595 is less than 9215. -- This message was sent by Atlassian Jira (v8.3.4#803005)