Hi. We've suffered a single node HW failure (broker_id 4)
with at least 2 replicas of each topic partition, but some
topic parts are now leaderless (all were across 4,5):
Topic: topic.with.two.replicas Partition: 0Leader: -1
Replicas: 4,5 Isr:
on broker 5, we see warnings like this in
Hi all. I saw something weird yesterday on our leaf instances
which run kafka 0.7.2 (and mirror to kafka 0.8 via our custom code).
I fully realize everyone's instinctual response is upgrade, already.,
but I'd like to have an internals discussion to better understand
what happened, as I suspect
compressed? Did you try to run a consumer through the topic's
data or alternately the DumpLogSegments tool?
On Thu, Nov 6, 2014 at 12:56 PM, Neil Harkins nhark...@gmail.com wrote:
Hi all. I saw something weird yesterday on our leaf instances
which run kafka 0.7.2 (and mirror to kafka 0.8 via our
I've been thinking about this recently.
If kafka provided cmdline hooks to be executed on segment rotation,
similar to postgres' wal 'archive_command', configurations could store
only the current segments and all their random i/o on flash, then once
rotated, copy them sequentially onto
Hi. I've got a 5 node cluster running Kafka 0.8.1,
with 4697 partitions (2 replicas each) across 564 topics.
I'm sending it about 1% of our total messaging load now,
and several times a day there is a period where 1~1500
partitions have one replica not in sync. Is this normal?
If a consumer is
On Tue, Oct 21, 2014 at 2:10 PM, Todd Palino tpal...@gmail.com wrote:
As far as the number of partitions a single broker can handle, we've set
our cap at 4000 partitions (including replicas). Above that we've seen some
performance and stability issues.
How many brokers? I'm curious: what kinds
On Tue, Oct 7, 2014 at 3:56 PM, Josh J joshjd...@gmail.com wrote:
Is there a way to monitor the rate of rates to a particular topic? I wish
to monitor the frequency of incoming tuples in order to consume from the
topic in particular ways depending on the incoming write throughput.
we've got a
No compression is configured, the spool files
can be /usr/bin/strings'd to show the msgs.
On Tue, Aug 5, 2014 at 2:13 PM, Guozhang Wang wangg...@gmail.com wrote:
Hello Neil,
Did you use any compression on your producers?
Guozhang
On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins nhark
