Why we can't find Kafka in public maven repositories ?
What's obstacle to publish into public maven repositories ?
Thanks,
Choi
Hi all,
In our application, we are currently starting a Kafka Consumer with the
following lines of code:
connector = Consumer.createJavaConsumerConnector(consumerConfig);
streams = connector .createMessageStreams(map);
Then, each KafkaStream is processed in a dedicated thread per topic and
On our current project we want to have a single shell script that:
first starts kafka with zookeeper and second strarts tomcat server
with our webapp.
We run kafka and tomcat on single machine, use kafka 0.8
We set num.partitions=4 in server.properties.
In webapp we start 4 threads that do
Is that in 0.7 or 0.8? JMX won't allow a bean with the same name to be
registered twice. Do you see 2 beans with similar names? What are the exact
bean names?
Thanks,
Jun
On Thu, Oct 10, 2013 at 2:42 AM, Tanguy tlrx tlrx@gmail.com wrote:
Hi all,
In our application, we are currently
Could you try the latest code in the 0.8 branch? We recently fixed a
consumer side issue (KAFKA-1030) that seems related.
Thanks,
Jun
On Thu, Oct 10, 2013 at 6:36 AM, Viktor Kolodrevskiy
viktor.kolodrevs...@gmail.com wrote:
On our current project we want to have a single shell script that:
It's in 0.8.
The JMX names are not exactly the same, but I see 2 beans with similar
names, something like:
my_consumer_group-my_consumer-mytopic-my_consumer_hostname.thing.com-1381416134138-3573c8bf-7-FetchQueueSize
-- Tanguy
2013/10/10 Jun Rao jun...@gmail.com
Is that in 0.7 or 0.8? JMX
Each time we create a new consumer connector, we assign a random consumer
id by default. You can try setting consumer.id to use a fixed consumer
id. In any case, we probably should deregister those beans when shutting
down the connector. Could you file a jira?
Thanks,
Jun
On Thu, Oct 10, 2013
Thanks guys.
Since we have some time constraints we are going with a Java based REST
endpoint within our already existing API to act as a proxy for the message.
Since it's a low volume of messages coming from our PHP side to kafka we
are willing to accept the additional overhead of doubling the
Do you see the following warn messages on the producer ?
Fetching topic metadata with correlation id %d for topics [%s] from broker
[%s] failed
fetching topic metadata for topics [%s] from broker [%s] failed
On the server side -
1. could you enable DEBUG on kafka.server.KafkaApis and see if
Is anyone out there running a single broker kafka setup?
How about with only 8 GB RAM?
I'm looking at one of the better dedicated server prodivers, and a 8GB
server is pretty much what I want to spend at the moment, would it make
sense going this route?
This same server would also potentially be
A 10/10/2013, às 23:14, S Ahmed sahmed1...@gmail.com escreveu:
Is anyone out there running a single broker kafka setup?
How about with only 8 GB RAM?
I'm looking at one of the better dedicated server prodivers, and a 8GB
server is pretty much what I want to spend at the moment, would it
Hi,
Following the _kafka documentation_
http://kafka.apache.org/documentation.html#quickstart, I succeeded in
Step 1: Download the code, Step 2: Start the server and Step 3: Create a
topic in three different terminals.
While I tried Step 4: Send some messages, the problem happened, the
Hi, I run into kafka issue recently, when I try to produce a message, it
gives me follow error
[2013-10-11 00:30:26,740] INFO Fetching metadata from broker
id:0,host:localhost,port:9092 with correlation id 47 for 1 topic(s)
Set(my-replicated-topic) (kafka.client.ClientUtils$)
[2013-10-11
can anyone help me? Thanks a lot
2013/10/10 Jiang Jacky jiang0...@gmail.com
Hi, I run into kafka issue recently, when I try to produce a message, it
gives me follow error
[2013-10-11 00:30:26,740] INFO Fetching metadata from broker
id:0,host:localhost,port:9092 with correlation id 47 for 1
Are you trying to produce these messages to a newly created topic?
On Thu, Oct 10, 2013 at 6:15 PM, Jiang Jacky jiang0...@gmail.com wrote:
can anyone help me? Thanks a lot
2013/10/10 Jiang Jacky jiang0...@gmail.com
Hi, I run into kafka issue recently, when I try to produce a message, it
yes, even I cannot create new topic, same error, leadernotavailable.
My zookeeper and storm all are working very well, except of Kafka.
2013/10/10 Neha Narkhede neha.narkh...@gmail.com
Are you trying to produce these messages to a newly created topic?
On Thu, Oct 10, 2013 at 6:15 PM, Jiang
Another thing need is my instance comes with very small memory, just 683M,
I am not sure if it is related. I will try to use 8G memory instance.
2013/10/10 Jiang Jacky jiang0...@gmail.com
yes, even I cannot create new topic, same error, leadernotavailable.
My zookeeper and storm all are
did anyone experience same issue? can you share with me?
Thanks a lot.
2013/10/7 Jun Rao jun...@gmail.com
Kafka port is documented in
http://kafka.apache.org/documentation.html#brokerconfigs
Thanks,
Jun
On Sat, Oct 5, 2013 at 12:05 PM, Jiang Jacky jiang0...@gmail.com wrote:
Hi, I
it may be that the broker is reporting its internal ip address to zookeeper
which then may interfere with your ability to submit the the broker with a
producer that isn't on AWS's network. you can make the broker report the
public dns name instead of the internal host name which i believe it
another thing to note is that your brokers may be running on port 6667 instead
of 9092. it appears that the default recently changed from 9092 to 6667 so make
sure if you have firewall rules for 9092 that you specifically set the port
setting in your configuration to what you expect. -Joe
Hi Jiang
Just thinking out loud, it could be because Kafka can't connect to
Zookeeper. Can you check Kafka logs to validate the same?
On 11 October 2013 08:22, Jiang Jacky jiang0...@gmail.com wrote:
Hi, Everyone, I use the larger memory instance to replace that small one,
for now, I can
*Hi, Guys,*
*I am currently running into the kafka server issue. *
*I have a 5 nodes cluster and zookeeper running without any problem. when I
manually boot each node by using* *JMX_PORT=9997 bin/kafka-server-start.sh
config/server-x.properties * command.
*The scenario is:*
*Then, first node, it
Hi, Aniket
Thanks for your reply
I checked the log from /tmp/kafka-logs/
There are some empty files, likes .index,
.log. I cannot find any useful information there.
However, i found some other interesting things, I have posted here
Any error in the controller and state-change log?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:38 PM, Jiang Jacky jiang0...@gmail.com wrote:
Hi, Aniket
Thanks for your reply
I checked the log from /tmp/kafka-logs/
There are some empty files, likes .index,
Yes, it just says INFO Reconnect due to socket error
But why and how come it comes? my zookeeper and storm have no any problem
to collaborate each other.
2013/10/11 Jun Rao jun...@gmail.com
The log you posted for the second broker didn't say why it crashed. Is that
all you got?
Thanks,
Hi, Thanks Joe
I have disabled the firewall, but I did not tried that again, I changed
another way by using Netty sending the socket package to kafka server, then
it will produce in the local. But recently I suddenly ran into below issue,
can you have a look?
Thanks Jun Rao
Where can I find those logs?
2013/10/11 Jun Rao jun...@gmail.com
Any error in the controller and state-change log?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:38 PM, Jiang Jacky jiang0...@gmail.com wrote:
Hi, Aniket
Thanks for your reply
I checked the log from
Is that reproducible?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:54 PM, Jiang Jacky jiang0...@gmail.com wrote:
Yes, it just says INFO Reconnect due to socket error
But why and how come it comes? my zookeeper and storm have no any problem
to collaborate each other.
2013/10/11 Jun Rao
under logs/.
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:56 PM, Jiang Jacky jiang0...@gmail.com wrote:
Thanks Jun Rao
Where can I find those logs?
2013/10/11 Jun Rao jun...@gmail.com
Any error in the controller and state-change log?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:38 PM,
Yes, everytime, I send message, it throws that error.
2013/10/11 Jun Rao jun...@gmail.com
Is that reproducible?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:54 PM, Jiang Jacky jiang0...@gmail.com wrote:
Yes, it just says INFO Reconnect due to socket error
But why and how come it comes? my
Sorry, even I don't send message, it shows up that message about every 1
minute
2013/10/11 Jiang Jacky jiang0...@gmail.com
Yes, everytime, I send message, it throws that error.
2013/10/11 Jun Rao jun...@gmail.com
Is that reproducible?
Thanks,
Jun
On Thu, Oct 10, 2013 at 9:54 PM,
no, it is not reproducible. Thank you.
2013/10/11 Jun Rao jun...@gmail.com
I meant the broker crash. Is that reproducible?
Thanks,
Jun
On Thu, Oct 10, 2013 at 10:10 PM, Jiang Jacky jiang0...@gmail.com wrote:
Yes, everytime, I send message, it throws that error.
2013/10/11 Jun
Hi Snehalata
state-change log are automatically created if you are using kafka-0.8
version.
On Wed, Oct 9, 2013 at 10:42 AM, Snehalata Nagaje
snehalata.nag...@harbingergroup.com wrote:
Actually state-change log setting is not on. So could not find the
state-change log.
Hi guys,
I'm trying to maintain a bunch of simple kafka consumer to consume messages
from brokers. I know there is a way to send TopicMetadataRequest to broker
and get the response from the broker. But you have to specify the broker
list to query the information. But broker might not be available
34 matches
Mail list logo
