[
https://issues.apache.org/jira/browse/SAMZA-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018898#comment-14018898
]
Chris Riccomini commented on SAMZA-224:
---------------------------------------
bq. I find it quite surprising that there are two different config properties —
systems.*.samza.offset.default and systems.*.consumer.auto.offset.reset which
seem to do almost the same thing (to someone who doesn't know all the
internals).
Yea. The nuance here is that samza.offset.default defines what to do when there
is no checkpoint for your input stream. auto.offset.reset defines what to do
when you fall of the edge of a Kafka topic. These are two different things. You
might want to start at the beginning of a stream the first time your job runs,
but fail explicitly if you fall of the edge of a topic. Having two separate
settings allows this.
In the case you've found, in addition to there being no checkpoint, there's
also no valid oldest/newest offset for the stream because it's empty. Perhaps a
better behavior is to say, if samza.offset.default resolves to an offset that's
null, we use the "upcoming" offset (which will always be non-null) instead?
This would fix the test, as well. It also seems like maybe a more intuitive
behavior.
bq. Perhaps that's something we can tidy up, but it's probably out of scope for
this ticket (and 0.7.0).
Agree. Feel free to open a second ticket.
> TestStatefulTask.testShouldStartAndRestore fails intermittently
> ---------------------------------------------------------------
>
> Key: SAMZA-224
> URL: https://issues.apache.org/jira/browse/SAMZA-224
> Project: Samza
> Issue Type: Bug
> Components: test
> Affects Versions: 0.6.0
> Reporter: Martin Kleppmann
> Assignee: Martin Kleppmann
> Fix For: 0.7.0
>
>
> [~closeuris] reported the following issue on SAMZA-185. I'm now
> intermittently seeing this problem too, so I'm opening a new issue for it (as
> I'm not sure the issue has anything to do with the Kafka 0.8.1 upgrade).
> {noformat}
> org.apache.samza.test.integration.TestStatefulTask >
> testShouldStartAndRestore FAILED
> java.lang.AssertionError at TestStatefulTask.scala:356
> 2 tests completed, 1 failed
> :samza-test_2.10:test FAILED
> {noformat}
> Logs provided by Yan (mine look similar):
> Failed tests: http://pastebin.com/3nEDw9jC
> Standard output: http://pastebin.com/20vZbk7c
> Standard error: http://pastebin.com/gU6cTNAr
--
This message was sent by Atlassian JIRA
(v6.2#6252)
