Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[James Cheng]

+1

I agree with Gwen that this needs to be well communicated to users. Anyone who 
is using the current default may see their brokers start to go exit when they 
upgrade to this new version.

On a somewhat related subject, occasionally I'll need to look up how my Kafka 
brokers are configured. Typically, I will either

1) Look at the broker's server.log to see what is printed out at startup
OR
2a) look in the broker's config file for explicit overrides
2b) look in at the online docs to figure out defaults
2c) Do some not-that-hard mental layering to figure out which value is the 
right one.

Option 1 may not be available, if the server logs have been rotated out.
Option 2 is a multi-step process.

I wonder if it might be useful to have a "dry-run" mode where Kafka would parse 
its config file, calculate its config (using the defaults and overrides), and 
then print out the result and exit. This would help with detecting changes in 
config values from one version to another, without requiring actually launching 
the broker or closely tracking the docs.

And/or, a way to query the running broker and have it dump out its config on 
demand.

The same type of thing also applies to topic configuration. 
./bin/kafka-configs.sh --describe can be used to look up overrides, but I still 
have to cross reference the broker config (by doing the above) to figure out 
how a topic is actually configured.

Users could then use that functionality to quickly get an idea of how a new 
version's configuration has changed, compared to what they are currently 
running.

I know such a thing is out of scope for this KIP. I just wanted to mention it, 
because it's related to changing of default values (and being aware of changes).

-James

> On Jan 4, 2017, at 11:51 AM, Rajini Sivaram  wrote:
> 
> +1
> 
> On Wed, Jan 4, 2017 at 1:49 AM, Becket Qin  wrote:
> 
>> +1
>> 
>> On Tue, Jan 3, 2017 at 5:41 PM, Joel Koshy  wrote:
>> 
>>> +1
>>> 
>>> On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford  wrote:
>>> 
 Hi All
 
 Please find the below KIP which proposes changing the setting
 unclean.leader.election.enabled from true to false. The motivation for
 this change is that it catches out new Kafka users who don’t realise
>> the
 default favours availability over data loss.
 
 This would mean clusters wishing to continue with unclean leader
>> election
 enabled would need to add the appropriate configuration on upgrade.
 
 Please let me know if you foresee any issue with this change, agree or
 don’t agree.
 
 https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+
 KIP-106+-+Change+Default+unclean.leader.election.
 enabled+from+True+to+False >> 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
>>> election.enabled+from+True+to+False>
> 
 
 Thanks
 
 B
 
 Ben Stopford
 Confluent, http://www.confluent.io 
 
 
 
 
>>> 
>> 

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Rajini Sivaram]

+1

On Wed, Jan 4, 2017 at 1:49 AM, Becket Qin  wrote:

> +1
>
> On Tue, Jan 3, 2017 at 5:41 PM, Joel Koshy  wrote:
>
> > +1
> >
> > On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford  wrote:
> >
> > > Hi All
> > >
> > > Please find the below KIP which proposes changing the setting
> > > unclean.leader.election.enabled from true to false. The motivation for
> > > this change is that it catches out new Kafka users who don’t realise
> the
> > > default favours availability over data loss.
> > >
> > > This would mean clusters wishing to continue with unclean leader
> election
> > > enabled would need to add the appropriate configuration on upgrade.
> > >
> > > Please let me know if you foresee any issue with this change, agree or
> > > don’t agree.
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+
> > > KIP-106+-+Change+Default+unclean.leader.election.
> > > enabled+from+True+to+False  > > luence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+uncle
> > > an.leader.election.enabled+from+True+to+False
> > >  > 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> > election.enabled+from+True+to+False>
> > > >
> > >
> > > Thanks
> > >
> > > B
> > >
> > > Ben Stopford
> > > Confluent, http://www.confluent.io 
> > >
> > >
> > >
> > >
> >
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Becket Qin]

+1

On Tue, Jan 3, 2017 at 5:41 PM, Joel Koshy  wrote:

> +1
>
> On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford  wrote:
>
> > Hi All
> >
> > Please find the below KIP which proposes changing the setting
> > unclean.leader.election.enabled from true to false. The motivation for
> > this change is that it catches out new Kafka users who don’t realise the
> > default favours availability over data loss.
> >
> > This would mean clusters wishing to continue with unclean leader election
> > enabled would need to add the appropriate configuration on upgrade.
> >
> > Please let me know if you foresee any issue with this change, agree or
> > don’t agree.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+
> > KIP-106+-+Change+Default+unclean.leader.election.
> > enabled+from+True+to+False  > luence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+uncle
> > an.leader.election.enabled+from+True+to+False
> >  5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> election.enabled+from+True+to+False>
> > >
> >
> > Thanks
> >
> > B
> >
> > Ben Stopford
> > Confluent, http://www.confluent.io 
> >
> >
> >
> >
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Joel Koshy]

+1

On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford  wrote:

> Hi All
>
> Please find the below KIP which proposes changing the setting
> unclean.leader.election.enabled from true to false. The motivation for
> this change is that it catches out new Kafka users who don’t realise the
> default favours availability over data loss.
>
> This would mean clusters wishing to continue with unclean leader election
> enabled would need to add the appropriate configuration on upgrade.
>
> Please let me know if you foresee any issue with this change, agree or
> don’t agree.
>
> https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+
> KIP-106+-+Change+Default+unclean.leader.election.
> enabled+from+True+to+False  luence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+uncle
> an.leader.election.enabled+from+True+to+False
> 
> >
>
> Thanks
>
> B
>
> Ben Stopford
> Confluent, http://www.confluent.io 
>
>
>
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Onur Karaman]

+1

On Tue, Jan 3, 2017 at 1:12 PM, Sriram Subramanian  wrote:

> +1
>
> On Tue, Jan 3, 2017 at 11:56 AM, Ian Wrigley  wrote:
>
> > +1 from me too. When I talk to people in training classes, who are
> > typically much newer to Kafka, they tend to be surprised (and
> > scared/horrified) that the default is true. Much safer to set it to false
> > and let people change it when they really understand the tradeoffs.
> >
> > Ian.
> >
> > ---
> > Ian Wrigley
> > Director, Education Services
> > Confluent, Inc
> >
> > > On Jan 3, 2017, at 1:22 PM, Tom Crayford  wrote:
> > >
> > > +1. We've been running it in production for a long time and it's the
> > right
> > > default.
> > >
> > > On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma  wrote:
> > >
> > >> Thanks for the KIP, +1 from me.
> > >>
> > >> Ismael
> > >>
> > >> On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:
> > >>
> > >>> Hi All
> > >>>
> > >>> Please find the below KIP which proposes changing the setting
> > >>> unclean.leader.election.enabled from true to false. The motivation
> for
> > >>> this change is that it catches out new Kafka users who don’t realise
> > the
> > >>> default favours availability over data loss.
> > >>>
> > >>> This would mean clusters wishing to continue with unclean leader
> > election
> > >>> enabled would need to add the appropriate configuration on upgrade.
> > >>>
> > >>> Please let me know if you foresee any issue with this change, agree
> or
> > >>> don’t agree.
> > >>>
> > >>> https://cwiki.apache.org/confluence/display/KAFKA/%
> > >>> 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> > >>> election.enabled+from+True+to+False  > >>> confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
> > >>> unclean.leader.election.enabled+from+True+to+False>
> > >>>
> > >>> Thanks
> > >>>
> > >>> B
> > >>>
> > >>> Ben Stopford
> > >>> Confluent, http://www.confluent.io 
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> >
> >
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Sriram Subramanian]

+1

On Tue, Jan 3, 2017 at 11:56 AM, Ian Wrigley  wrote:

> +1 from me too. When I talk to people in training classes, who are
> typically much newer to Kafka, they tend to be surprised (and
> scared/horrified) that the default is true. Much safer to set it to false
> and let people change it when they really understand the tradeoffs.
>
> Ian.
>
> ---
> Ian Wrigley
> Director, Education Services
> Confluent, Inc
>
> > On Jan 3, 2017, at 1:22 PM, Tom Crayford  wrote:
> >
> > +1. We've been running it in production for a long time and it's the
> right
> > default.
> >
> > On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma  wrote:
> >
> >> Thanks for the KIP, +1 from me.
> >>
> >> Ismael
> >>
> >> On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:
> >>
> >>> Hi All
> >>>
> >>> Please find the below KIP which proposes changing the setting
> >>> unclean.leader.election.enabled from true to false. The motivation for
> >>> this change is that it catches out new Kafka users who don’t realise
> the
> >>> default favours availability over data loss.
> >>>
> >>> This would mean clusters wishing to continue with unclean leader
> election
> >>> enabled would need to add the appropriate configuration on upgrade.
> >>>
> >>> Please let me know if you foresee any issue with this change, agree or
> >>> don’t agree.
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/%
> >>> 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> >>> election.enabled+from+True+to+False  >>> confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
> >>> unclean.leader.election.enabled+from+True+to+False>
> >>>
> >>> Thanks
> >>>
> >>> B
> >>>
> >>> Ben Stopford
> >>> Confluent, http://www.confluent.io 
> >>>
> >>>
> >>>
> >>>
> >>
>
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Ian Wrigley]

+1 from me too. When I talk to people in training classes, who are typically 
much newer to Kafka, they tend to be surprised (and scared/horrified) that the 
default is true. Much safer to set it to false and let people change it when 
they really understand the tradeoffs.

Ian.

---
Ian Wrigley
Director, Education Services
Confluent, Inc

> On Jan 3, 2017, at 1:22 PM, Tom Crayford  wrote:
> 
> +1. We've been running it in production for a long time and it's the right
> default.
> 
> On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma  wrote:
> 
>> Thanks for the KIP, +1 from me.
>> 
>> Ismael
>> 
>> On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:
>> 
>>> Hi All
>>> 
>>> Please find the below KIP which proposes changing the setting
>>> unclean.leader.election.enabled from true to false. The motivation for
>>> this change is that it catches out new Kafka users who don’t realise the
>>> default favours availability over data loss.
>>> 
>>> This would mean clusters wishing to continue with unclean leader election
>>> enabled would need to add the appropriate configuration on upgrade.
>>> 
>>> Please let me know if you foresee any issue with this change, agree or
>>> don’t agree.
>>> 
>>> https://cwiki.apache.org/confluence/display/KAFKA/%
>>> 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
>>> election.enabled+from+True+to+False >> confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
>>> unclean.leader.election.enabled+from+True+to+False>
>>> 
>>> Thanks
>>> 
>>> B
>>> 
>>> Ben Stopford
>>> Confluent, http://www.confluent.io 
>>> 
>>> 
>>> 
>>> 
>> 

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Neha Narkhede]

Big +1.

On Tue, Jan 3, 2017 at 11:22 AM Tom Crayford  wrote:

> +1. We've been running it in production for a long time and it's the right
> default.
>
> On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma  wrote:
>
> > Thanks for the KIP, +1 from me.
> >
> > Ismael
> >
> > On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:
> >
> > > Hi All
> > >
> > > Please find the below KIP which proposes changing the setting
> > > unclean.leader.election.enabled from true to false. The motivation for
> > > this change is that it catches out new Kafka users who don’t realise
> the
> > > default favours availability over data loss.
> > >
> > > This would mean clusters wishing to continue with unclean leader
> election
> > > enabled would need to add the appropriate configuration on upgrade.
> > >
> > > Please let me know if you foresee any issue with this change, agree or
> > > don’t agree.
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/%
> > > 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> > > election.enabled+from+True+to+False  > > confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
> > > unclean.leader.election.enabled+from+True+to+False>
> > >
> > > Thanks
> > >
> > > B
> > >
> > > Ben Stopford
> > > Confluent, http://www.confluent.io 
> > >
> > >
> > >
> > >
> >
>
-- 
Thanks,
Neha

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Tom Crayford]

+1. We've been running it in production for a long time and it's the right
default.

On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma  wrote:

> Thanks for the KIP, +1 from me.
>
> Ismael
>
> On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:
>
> > Hi All
> >
> > Please find the below KIP which proposes changing the setting
> > unclean.leader.election.enabled from true to false. The motivation for
> > this change is that it catches out new Kafka users who don’t realise the
> > default favours availability over data loss.
> >
> > This would mean clusters wishing to continue with unclean leader election
> > enabled would need to add the appropriate configuration on upgrade.
> >
> > Please let me know if you foresee any issue with this change, agree or
> > don’t agree.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/%
> > 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> > election.enabled+from+True+to+False  > confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
> > unclean.leader.election.enabled+from+True+to+False>
> >
> > Thanks
> >
> > B
> >
> > Ben Stopford
> > Confluent, http://www.confluent.io 
> >
> >
> >
> >
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Ismael Juma]

Thanks for the KIP, +1 from me.

Ismael

On 3 Jan 2017 6:54 pm, "Ben Stopford"  wrote:

> Hi All
>
> Please find the below KIP which proposes changing the setting
> unclean.leader.election.enabled from true to false. The motivation for
> this change is that it catches out new Kafka users who don’t realise the
> default favours availability over data loss.
>
> This would mean clusters wishing to continue with unclean leader election
> enabled would need to add the appropriate configuration on upgrade.
>
> Please let me know if you foresee any issue with this change, agree or
> don’t agree.
>
> https://cwiki.apache.org/confluence/display/KAFKA/%
> 5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.
> election.enabled+from+True+to+False  confluence/display/KAFKA/[WIP]+KIP-106+-+Change+Default+
> unclean.leader.election.enabled+from+True+to+False>
>
> Thanks
>
> B
>
> Ben Stopford
> Confluent, http://www.confluent.io 
>
>
>
>

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Gwen Shapira]

I strongly support this. I've seen many users and customers
accidentally lose data because they didn't know this configuration
exists. I always support defaulting to "Not lose data" and let the
users who prefer high availability do some extra research. This will
need to be very well documented - both the default trade-off, the
alternative and the fact that things changed.

On Tue, Jan 3, 2017 at 10:54 AM, Ben Stopford  wrote:
> Hi All
>
> Please find the below KIP which proposes changing the setting 
> unclean.leader.election.enabled from true to false. The motivation for this 
> change is that it catches out new Kafka users who don’t realise the default 
> favours availability over data loss.
>
> This would mean clusters wishing to continue with unclean leader election 
> enabled would need to add the appropriate configuration on upgrade.
>
> Please let me know if you foresee any issue with this change, agree or don’t 
> agree.
>
> https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.election.enabled+from+True+to+False
>  
> 
>
> Thanks
>
> B
>
> Ben Stopford
> Confluent, http://www.confluent.io 
>
>
>



-- 
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog

[DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

[Ben Stopford]

Hi All

Please find the below KIP which proposes changing the setting 
unclean.leader.election.enabled from true to false. The motivation for this 
change is that it catches out new Kafka users who don’t realise the default 
favours availability over data loss.

This would mean clusters wishing to continue with unclean leader election 
enabled would need to add the appropriate configuration on upgrade. 

Please let me know if you foresee any issue with this change, agree or don’t 
agree.

https://cwiki.apache.org/confluence/display/KAFKA/%5BWIP%5D+KIP-106+-+Change+Default+unclean.leader.election.enabled+from+True+to+False
 


Thanks 

B

Ben Stopford
Confluent, http://www.confluent.io