Hello,
We're having the following issue with Kafka and/or Zookeeper:
If a broker (id=1) is running, and you start another broker with id=1, the new
broker will exit saying A broker is already registered on the path
/brokers/ids/1. However, I noticed when I query zookeeper /brokers/ids/1
Hello,
First, is there a limit to how many Kafka brokers you can have?
Second, if a Kafka broker node fails and I start a new broker on a new node, is
it correct to assume that the cluster will copy data to that node to satisfy
the replication factor specified for a given topic? In other
Hello,
We're looking into using Kafka for a improved version of a system and the
question of how to scale Kafka came up. Specifically, we want to try to make
the system scale as transparently as possible. The concern was that if we go
from N to N*2 consumers that we would have some that are
Hello,
We're using Kafka 0.8.1.1 and we're trying to run it in a Docker container.
For the most part, this has been fine, however one of the containers has
stopped a couple times and when I look at the log output from Docker (E.g.
Kafka STDOUT), I don't see any errors. At one point it states
of a few operations) that requires a broker restart can be done
in a rolling manner.
On Wed, Oct 15, 2014 at 7:16 PM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
We're looking into deploying Kafka and Zookeeper into an environment where
we want things to be as easy to stand
Roger,
My understanding of both, beyond what Zookeeper already does, are:
1. Consul can be used to monitor a service and report it's status. This can be
very useful for knowing if a service, such as Zookeeper of Kafka, goes down.
This can be done through a built-in web interface.
2. Confd
Hello,
We're looking into deploying Kafka and Zookeeper into an environment where we
want things to be as easy to stand up and administer. To do this, we're
looking into using Consul, or similar, and Confd to try to make this as
automatic as possible. I was wondering if anyone had an
Nevermind...I just found it in the docs and it looks like it has been looked
into.
Casey Sybrandy MSWE
Sr. Software Engineer
CACI/Six3Systems
301-206-6000 (Office)
301-206-6020 (Fax)
11820 West Market Place
Suites N-P
Fulton, MD. 20759
From: Sybrandy, Casey
Sent
the
output and also the log of the consumer here?
Guozhang
On Tue, Dec 10, 2013 at 12:26 PM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
First, I'm using version 0.7.2.
I'm trying to read some messages from a broker, but looking at wireshark,
it appears that only part
-Myconsumerseemstohavestopped%2Cwhy%3F
?
If that doesn't help, could you file a jira and attach your log?
Apache
mailing list doesn't support attachments.
Thanks,
Jun
On Wed, Dec 11, 2013 at 6:15 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
No, the entire log file isn't bigger
on in the broker?
Guozhang
On Wed, Dec 11, 2013 at 8:43 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
First, I saw the partial message looking at raw network traffic via
Wireshark, not the output of the iterator as the iterator never seems to
provide me any data. That's where the code
Hello,
How can I have the brokers log errors to a file? Do I just have to configure
something like log4j or is something else used?
Thanks.
Casey
.
Thanks,
Neha
On Wed, Nov 6, 2013 at 8:48 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
How can I have the brokers log errors to a file? Do I just have to
configure something like log4j or is something else used?
Thanks.
Casey
--
-- Guozhang
:30 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
We're seeing an issue running 0.7.0 where one or more of our consumers
are pausing after about an hour when we have a lot of threads
configured. Our setup is as follows:
* 4 brokers configured for 32 threads
Hello,
We're seeing an issue running 0.7.0 where one or more of our consumers are
pausing after about an hour when we have a lot of threads configured. Our
setup is as follows:
* 4 brokers configured for 32 threads and 32 partitions on each broker.
* 2 consumers each
In the past there was some discussion about having a C client for non-JVM
languages. Is this still planned as well? Being able to work with Kafka from
other languages would be a great thing. Where I work, we interact with Kafka
via Java and Ruby (producer), so having an official C library
...@incubator.apache.org
Subject: Re: Duplicate Messages on the Consumer
Have you actually examined the Kafka files on disk, to make sure those dupes
are really there? Or is this a case of reading the same message more than once?
Philip
On Thu, Jul 18, 2013 at 8:55 AM, Sybrandy, Casey
casey.sybra...@six3systems.com
Hello,
We recently started seeing duplicate messages appearing at our consumers.
Thankfully, the database is set up so that we don't store the dupes, but it is
annoying. It's not every message, only about 1% of them. We are running 0.7.0
for the broker with Zookeeper 3.3.4 from Cloudera and
@kafka.apache.org
Subject: Re: NoBrokersForPartitionException
Could you try 0.7.2?
Thanks,
Jun
On Wed, Jul 10, 2013 at 11:38 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Hello,
Apologies for bringing this back from the dead, but I'm getting the
same exception using Kafka 0.7.0. What could
IIRC, I think I tried to use stunnel with Kafka once and it worked fine and the
configuration wasn't too bad, at least for a simple configuration.
-Original Message-
From: Dragos Manolescu [mailto:dragos.manole...@servicenow.com]
Sent: Friday, June 07, 2013 4:51 PM
To:
That's what I would have assumed. And no, we're not using compression.
Thanks.
From: Jun Rao [mailto:jun...@gmail.com]
Sent: Wednesday, May 08, 2013 11:26 AM
To: users@kafka.apache.org
Cc: Sybrandy, Casey
Subject: Re: Binary Data and Kafka
No. Kafka broker stores the binary data
Hello,
IIRC, no, it does not. Where I work, one team had the same issue and built
some custom code to handle the encryption and decryption of messages at the
producer and consumer. However, you have to take key management into account
as once a message is written to the broker, you can't
Apologies for not responding sooner. My mail client must have been
malfunctioning at the time as I never saw your responses until today.
As for the error, it looks like it's a bug on my part that just didn't click
until I read Jim's responses. I have a config file that I specify the options
are using
DefaultEventHandler, there is a trace level logging that tells you which
broker a request is sent to.
Thanks,
Jun
On Mon, Dec 10, 2012 at 8:10 AM, Sybrandy, Casey
casey.sybra...@six3systems.com wrote:
Is it at least possible to see which broker a message is sent to? I'm
using a Zookeeper
24 matches
Mail list logo
