[
https://issues.apache.org/jira/browse/STORM-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351308#comment-16351308
]
Alexandre Vermeerbergen commented on STORM-2914:
------------------------------------------------
Hello [~hmclouro]
+1 for your proposal to get things decided quickly !
In your 7 questions lists, my vote is:
+1 for item #5 : because I want to trust you guys on the meaning of "identical"
in this sentence: "_Document that ProcessingGuarantee.NO_GUARANTEES has a
behavior identical to auto.commit.enable=true but disregards this property
altogether (if auto.commit.enable is set the KafkaSpoutConfig will throw an
exception)_".
And I have no problem changing the way I configure Kafka spout to get this
"identical" behavior if documentation is crystal clear about it. The
documentation needs to be very clear about how to replace legacy use of
auto.commit.enable and related setting by newer ones.
+ 0 for other items, meaning I am neutral there, as I'm selfishly concerned by
my need to keep a behavior "identical" to auto.commit.enable=true.
I just hope that "identical" means "no noticeable difference in performance".
Best regards,
Alexandre Vermeerbergen
PS: I also hope that in next RC, the toollib/ directory will be cleaned up from
*src* and *javadoc* artifacts, which seems to break the display of Kafka spout
statistics in Nimbus UI in 1.2.0 RC2 (see my previous comment about this in
this thread).**
> Remove enable.auto.commit support from storm-kafka-client
> ---------------------------------------------------------
>
> Key: STORM-2914
> URL: https://issues.apache.org/jira/browse/STORM-2914
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka-client
> Affects Versions: 2.0.0, 1.2.0
> Reporter: Stig Rohde Døssing
> Assignee: Stig Rohde Døssing
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The enable.auto.commit option causes the KafkaConsumer to periodically commit
> the latest offsets it has returned from poll(). It is convenient for use
> cases where messages are polled from Kafka and processed synchronously, in a
> loop.
> Due to https://issues.apache.org/jira/browse/STORM-2913 we'd really like to
> store some metadata in Kafka when the spout commits. This is not possible
> with enable.auto.commit. I took at look at what that setting actually does,
> and it just causes the KafkaConsumer to call commitAsync during poll (and
> during a few other operations, e.g. close and assign) with some interval.
> Ideally I'd like to get rid of ProcessingGuarantee.NONE, since I think
> ProcessingGuarantee.AT_MOST_ONCE covers the same use cases, and is likely
> almost as fast. The primary difference between them is that AT_MOST_ONCE
> commits synchronously.
> If we really want to keep ProcessingGuarantee.NONE, I think we should make
> our ProcessingGuarantee.NONE setting cause the spout to call commitAsync
> after poll, and never use the enable.auto.commit option. This allows us to
> include metadata in the commit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
