I agree with Jason's notion that
> implicit use of the empty group.id to commit offsets is more likely to
be causing users unexpected problems than actually providing a useful
capability.
I was initially confused that this is the behavior when investigating a
new-ish JIRA issue <https://issues.apache.org/jira/browse/KAFKA-6758> about
the same topic.
So, +1 to deprecating "" as a group.id
The question after that becomes what the *default* value should be - should
we:
a) treat an unconfigured group.id consumer as a sort of intermittent
consumer where you don't store offsets at all (thereby making the user
explicitly sign up for them)
b) have a default value which makes use of them? I sort of like the former.
@Dhruvil, thinking about it at a high-level - yes. I can't think of a
situation where it makes sense to name something an empty string as far as
I'm aware - to me it seems like potential for confusion
On Fri, Jul 20, 2018 at 10:22 AM Rajini Sivaram <rajinisiva...@gmail.com>
wrote:
> +1 to deprecate use of "" as group.id since it is odd to have a resource
> name that you cannot set ACLs for. Agree, we have to support older clients
> though.
>
> Thanks,
>
> Rajini
>
> On Fri, Jul 20, 2018 at 5:25 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hi Vahid,
> >
> > Sorry for getting to this so late. I think there are two things here:
> >
> > 1. The use of "" as a groupId has always been a dubious practice at best.
> > We definitely ought to deprecate its use in the client. Perhaps in the
> next
> > major release, we can remove support completely. However, since older
> > clients depend on it, we may have to continue letting the broker support
> it
> > to some extent. Perhaps we just need to bump the OffsetCommit request API
> > and only accept the offset commit for older versions. You probably have
> to
> > do this anyway if you want to introduce the new error code since old
> > clients will not expect it.
> >
> > 2. There should be a way for the consumer to indicate that it has no
> group
> > id and will not commit offsets. This is an explicit instruction that the
> > consumer should not bother with coordinator lookup and such. We currently
> > have some brittle logic in place to let users avoid the coordinator
> lookup,
> > but it is a bit error-prone. I was hoping that we could change the
> default
> > value of group.id to be null so that the user had to take an explicit
> > action to opt into coordinator management (groups or offsets). However,
> it
> > is true that some users may be unknowingly depending on offset storage if
> > they are using both the default group.id and the default
> > enable.auto.commit. Perhaps one option is to disable enable.auto.commit
> > automatically if no group.id is specified? I am not sure if there are
> any
> > drawbacks, but my feeling is that implicit use of the empty group.id to
> > commit offsets is more likely to be causing users unexpected problems
> than
> > actually providing a useful capability.
> >
> > Thoughts?
> >
> > Thanks,
> > Jason
> >
> >
> >
> >
> > On Mon, May 28, 2018 at 9:50 AM, Vahid S Hashemian <
> > vahidhashem...@us.ibm.com> wrote:
> >
> > > Hi Viktor,
> > >
> > > Thanks for sharing your opinion.
> > > So you're in favor of disallowing the empty ("") group id altogether
> > (even
> > > for fetching).
> > > Given that ideally no one should be using the empty group id (at least
> in
> > > a production setting) I think the impact would be minimal in either
> case.
> > >
> > > But as you said, let's hear what others think and I'd be happy to
> modify
> > > the KIP if needed.
> > >
> > > Regards.
> > > --Vahid
> > >
> > >
> > >
> > >
> > > From: Viktor Somogyi <viktorsomo...@gmail.com>
> > > To: dev <dev@kafka.apache.org>
> > > Date: 05/28/2018 05:18 AM
> > > Subject: Re: [DISCUSS] KIP-289: Improve the default group id
> > > behavior in KafkaConsumer
> > >
> > >
> > >
> > > Hi Vahid,
> > >
> > > (with the argument that using the default group id for offset commit
> > > should not be the user's intention in practice).
> > >
> > > Yea, so in my opinion too this use case doesn't seem too practical.
> Also
> > I
> > > think breaking the offset commit is not smaller from this perspective
> > than
> > > breaking fetch and offset fetch. If we suppose that someone uses the
> > > default group id and we break the offset commit then that might be
> harder
> > > to detect than breaking the whole thing altogether. (If we think about
> an
> > > upgrade situation.)
> > > So since we think it is not a practical use case, I think it would be
> > > better to break altogether but ofc that's just my 2 cents :). Let's
> > gather
> > > other's input as well.
> > >
> > > Cheers,
> > > Viktor
> > >
> > > On Fri, May 25, 2018 at 5:43 PM, Vahid S Hashemian <
> > > vahidhashem...@us.ibm.com> wrote:
> > >
> > > > Hi Victor,
> > > >
> > > > Thanks for reviewing the KIP.
> > > >
> > > > Yes, to minimize the backward compatibility impact, there would be no
> > > harm
> > > > in letting a stand-alone consumer consume messages under a "" group
> id
> > > (as
> > > > long as there is no offset commit).
> > > > It would have to knowingly seek to an offset or rely on the
> > > > auto.offset.reset config for the starting offset.
> > > > This way the existing functionality would be preserved for the most
> > part
> > > > (with the argument that using the default group id for offset commit
> > > > should not be the user's intention in practice).
> > > >
> > > > Does it seem reasonable?
> > > >
> > > > Thanks.
> > > > --Vahid
> > > >
> > > >
> > > >
> > > >
> > > > From: Viktor Somogyi <viktorsomo...@gmail.com>
> > > > To: dev <dev@kafka.apache.org>
> > > > Date: 05/25/2018 04:56 AM
> > > > Subject: Re: [DISCUSS] KIP-289: Improve the default group id
> > > > behavior in KafkaConsumer
> > > >
> > > >
> > > >
> > > > Hi Vahid,
> > > >
> > > > When reading your KIP I coldn't fully understand why did you decide
> at
> > > > failing with "offset_commit" in case #2? Can't we fail with an empty
> > > group
> > > > id even in "fetch" or "fetch_offset"? What was the reason for
> deciding
> > > to
> > > > fail at "offset_commit"? Was it because of upgrade compatibility
> > > reasons?
> > > >
> > > > Thanks,
> > > > Viktor
> > > >
> > > > On Thu, May 24, 2018 at 12:06 AM, Ted Yu <yuzhih...@gmail.com>
> wrote:
> > > >
> > > > > Looks good to me.
> > > > > -------- Original message --------From: Vahid S Hashemian <
> > > > > vahidhashem...@us.ibm.com> Date: 5/23/18 11:19 AM (GMT-08:00)
> To:
> > > > > dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-289: Improve the
> > > default
> > > > > group id behavior in KafkaConsumer
> > > > > Hi Ted,
> > > > >
> > > > > Thanks for reviewing the KIP. I updated the KIP and introduced an
> > > error
> > > > > code for the scenario described.
> > > > >
> > > > > --Vahid
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > From: Ted Yu <yuzhih...@gmail.com>
> > > > > To: dev@kafka.apache.org
> > > > > Date: 04/27/2018 04:31 PM
> > > > > Subject: Re: [DISCUSS] KIP-289: Improve the default group id
> > > > > behavior in KafkaConsumer
> > > > >
> > > > >
> > > > >
> > > > > bq. If they attempt an offset commit they will receive an error.
> > > > >
> > > > > Can you outline what specific error would be encountered ?
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Fri, Apr 27, 2018 at 2:17 PM, Vahid S Hashemian <
> > > > > vahidhashem...@us.ibm.com> wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I have drafted a proposal for improving the behavior of
> > > KafkaConsumer
> > > > > when
> > > > > > using the default group id:
> > > > > >
> > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >
> > > >
> > > > >
> > > > > > 289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer
> > > > > > The proposal based on the issue and suggestion reported in
> > > KAFKA-6774.
> > > > > >
> > > > > > Your feedback is welcome!
> > > > > >
> > > > > > Thanks.
> > > > > > --Vahid
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
>
--
Best,
Stanislav
