Re: install kafka and hadoop on the same cluster?

I have had Kafka installed with Hadoop/Spark/etc in a comparable cluster, except that I didn't have Kafka brokers installed in the nodes with zookeeper. As long as physical disk space in the rest of the nodes is not a concern, you will be OK. I don't have enough background with Cassandra. But I

install kafka and hadoop on the same cluster?

Hi, All I have a 9-node cluster where I already installed cloudera Hadoop/spark, now I want to install kafka in this cluster too, is it a good idea I install kafka on each of 9 node? If so, any potential risk for that? Also I am thinking to install cassandra on each of this node too, basically

Need to understand the graph

Hello, I have an issue with the timing of Kafka. As and when the time increases with load testing, we see the increase in "request queue time". What is the "request queue time" ? Some basic config we are using 2 Kafka Nodes(CPU - 8 CPU's [Thread(s) per core:2], Mem - 32 GB Ram)

Re: Fetching meta from Kafka continuously.

Problem solved. There was a DNS problem for one of the brokers. Kim On Mon, Feb 29, 2016 at 12:56 AM, Jens Rantil wrote: > Hi, > > Could it be that you need rebalance your topics, perhaps? > > Cheers, > Jens > > On Wed, Feb 24, 2016 at 7:25 PM, Kim Chew

Connecting to secure Kafka

So, my Kafka is kerberized and all works well However when trying to connect with missing SASL properties Kafka doesn’t fail. It simply hangs indefinitely. Here is the example code; Properties props = new Properties(); props.put("bootstrap.servers",

Re: Kafka node liveness check

Crap, forgot to remove my signature.. I guess my e-mail will now get spammed forever :( On Mon, Feb 29, 2016 at 3:14 PM, Elias Abacioglu < elias.abacio...@deltaprojects.com> wrote: > We've setup jmxtrans and use it to check these two values. > UncleanLeaderElectionsPerSec >

Re: Kafka node liveness check

We've setup jmxtrans and use it to check these two values. UncleanLeaderElectionsPerSec UnderReplicatedPartitions Here is our shinken/nagios configuration: define command { command_name check_kafka_underreplicated command_line $USER1$/check_jmx -U

Re: Kafka node liveness check

Thanks Jens. What I want to achieve is to check every broker within a cluster functions probably. The way you suggest can identify the liveness of a cluster but it doesn't necessarily mean every broker in the cluster is alive. In order to achieve that I can either create a topic with number of

Re: Unable to start cluster after crash (0.8.2.2)

Double post. Please keep discussion in the other thread. Cheers, Jens On Wed, Feb 24, 2016 at 4:39 PM, Anthony Sparks wrote: > Hello, > > Our Kafka cluster (3 servers, each server has Zookeeper and Kafka installed > and running) crashed, and actually out of the 6

Re: Is it possible to configure Kafka Mirror to specify fixed ports to connect to Remote DC

Hi Munir, Are you referring to outbound or inbound ports from/to the Mirror tool? Cheers, Jens On Wed, Feb 24, 2016 at 6:01 PM, Munir Khan (munkhan) wrote: > Hi, > I am trying out Kafka Mirror for moving kafka message between DC. In our > case we have to make it work

Re: Fetching meta from Kafka continuously.

Hi, Could it be that you need rebalance your topics, perhaps? Cheers, Jens On Wed, Feb 24, 2016 at 7:25 PM, Kim Chew wrote: > We have shut down some nodes from our cluster yesterday and now we are > seeing tons of these in the log, > > > 2016-02-24 18:11:23 INFO

Re: Kafka node liveness check

Hi, I assume you first want to ask yourself what liveness you would like to check for. I guess the most realistic check is to put a "ping" message on the broken and make sure that you can consume it. Cheers, Jens On Fri, Feb 26, 2016 at 12:38 PM, tao xiao wrote: > Hi