Re: Kafka Retention Policy to Indefinite

2017-03-15 Thread Joe San
> > I am saying that replication quotas will mitigate one of the potential > downsides of setting an infinite retention policy. I was just interested in all of the possible potential downsides! Could you please point me to a documentation that has more information on this? On Tue, Mar 14, 2017 a

Re: Kafka Retention Policy to Indefinite

2017-03-14 Thread Hans Jespersen
I am saying that replication quotas will mitigate one of the potential downsides of setting an infinite retention policy. There is no clear set yes/no best practice rule for setting an extremely large retention policy. It is clearly a valid configuration and there are people who run this way. The

Re: Kafka Retention Policy to Indefinite

2017-03-14 Thread Joe San
So that means with replication quotas, I can set the retention policy to be infinite? On Tue, Mar 14, 2017 at 6:25 PM, Hans Jespersen wrote: > You might want to use the new replication quotas mechanism (i.e. network > throttling) to make sure that replication traffic doesn't negatively impact >

Re: Kafka Retention Policy to Indefinite

2017-03-14 Thread Hans Jespersen
You might want to use the new replication quotas mechanism (i.e. network throttling) to make sure that replication traffic doesn't negatively impact your production traffic. See for details: https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas This feature was added in 0.10

Kafka Retention Policy to Indefinite

2017-03-14 Thread Joe San
Dear Kafka Users, What are the arguments against setting the retention plociy on a Kafka topic to infinite? I was in an interesting discussion with one of my colleagues where he was suggesting to set the retention policy for a topic to be indefinite. So how does this play up when adding new broke