Re: Recommended max number of topics (and data separation)

5 AM, David Espinosa <espi...@gmail.com> wrote: > I used: > -Djute.maxbuffer=50111000 > and the gain I had is that I could increment number of topics from 70k to > 100k :P > > 2018-01-30 23:25 GMT+01:00 Andrey Falko <afa...@salesforce.com>: > >> On Tue, Jan 30

Re: Recommended max number of topics (and data separation)

I think the longer the topic names > are the sooner jute.maxbuffer overflows. I see; what value(s) have you tried with and how much gain did you you see? > David > > > 2018-01-30 4:40 GMT+01:00 Andrey Falko <afa...@salesforce.com>: > >> On Sun, Jan 28, 2018 at 8:45 AM, Da

Re: Recommended max number of topics (and data separation)

On Sun, Jan 28, 2018 at 8:45 AM, David Espinosa wrote: > Hi Monty, > > I'm also planning to use a big amount of topics in Kafka, so recently I > made a test within a 3 nodes kafka cluster where I created 100k topics with > one partition. Sent 1M messages in total. Are your

Unrecoverable kafka cluster after failed rolling restart

and/or starts failing to meet various SLAs we've made with our users. I'll try to "correct" the original mistake that I made, however, I hope that an operator error like mine doesn't take out a production cluster this like this :). Best regards, Andrey Falko Salesforce.com

Re: Kafka Replicated Partition Limits

n > > On Wed, Jan 3, 2018 at 3:00 PM, Andrey Falko <afa...@salesforce.com> wrote: > >> Ben Wood: >> 1. We have 5 ZK nodes. >> 2. I only tracked outstanding requests thus far from ZK-side of >> things. At 9.5k topics, I recorded about 5k outstanding requests.

Re: Kafka Replicated Partition Limits

e are now discussing KIP-227, which could reduce the > overhead for replication and consumption when there are many partitions. > > Jun > > On Wed, Jan 3, 2018 at 1:48 PM, Andrey Falko <afa...@salesforce.com> wrote: > >> Hi everyone, >> >> We are seeing more

Kafka Replicated Partition Limits

to have kafka support more than that? Best regards, Andrey Falko Salesforce.com