Hi all!
We're trying to use Kafka to deliver messages to thousands of clients
(different data for each client).
Actually we want to get behavior pretty much like message broker,
because we have to work with continious flow of small data pieces.
Suppose, we should create a partition for each client. Is it right?
We stuck with problems of infinite shrink/expand ISR and the
"FETCH_SESSION_ID_NOT_FOUND" error.
My configuration:
5 servers (8 vCPU, 16GB RAM)
20k partitions
2000 consumers
-replication-factor - 2
queued.max.requests = 2000
replica.fetch.wait.max.ms = 5000
replica.lag.time.max.ms = 20000
num.replica.fetchers = 5
num.network.threads=9
num.io.threads=24
--
Best Regards, Denis Romanenko
Head of ESB Development,
PJSC Magnit
