Hi,
I am looking to get information about individual brokers in kafka cluster.
The information that I am looking for is:
- List of topics in a broker
- Partitions for each topic in a broker
- Metrics like BytesIn/Out Per min, Messages In/Min per topic
- ...
I have tried looking into
Hello Sumit,
Not yet. AFAIK few topic management requests got introduced in broker side
(see https://issues.apache.org/jira/browse/KAFKA-2229 ) but not yet in
client APIs. Querying/listing topics request doesn't seem even to be
planned yet.
AdminUtils which talks with ZooKeeper via ZkClient is Ka
Hi team,
I am very new to kafka. Through some tutorials I bring up two clusters (two
instances in separate servers). I want to replicate the topic between these
servers using mirrormaker. But I cant find any useful material to set my
configurations. Can you please help me in setting mirror mak
Hi Ravi,
a very basic config we are using with kafka 0.10 and kafka release in
/opt/kafka:
Process to start:
JAVA_HOME=/opt/java /opt/kafka/bin/kafka-mirror-maker.sh --new.consumer
--consumer.config /opt/kafka/config/consumer.properties --producer.config
/opt/kafka/config/producer.properties -
I think what you're describing could be handled in KStreams by a "global"
KTable. This functionality is currently being discussed/voted on in a KIP
discussion: https://cwiki.apache.org/confluence/pages/viewpa
ge.action?pageId=67633649 The list of interests would be a global KTable
(shared globally
Hi!
I have written a guide how to calculate the minimal speed of Kafka
consumers - so you can avoid the situation where your Kafka consumers are
to slow, and because of this you lose data.
The guide takes into account three key factors - producer speed, retention
period of your data in Kafka and t
Hi,
Does adding more partitions to kafka will help in scaling ?
*Regards,*
*Laxmi Narayan Patel*
*MCA NIT Durgapur (2011-2014)*
*Mob:-9741292048,8345847473*
Hi Sumit,
JMX Metrics will give you lot of in depth information on Kafka.
Just try this.
https://github.com/srijiths/kafka-connectors/tree/master/kafka-connect-jmx
If you use above , then you should have a custom UI to show the metrics.
Also you can try open source kafka monitoring tool
https:
I don't think so because more partition can led to unavailability but yes
can led to higher throughput. But it cause more problems like increasing
end to end latency, requires more open file handler and require more memory
at client side.
*Thanks, Kunal*
*+91-9958189589*
*Data Analyst*
*First Pape
Hi ,
If I have kafka cluster along with replication .
Say cluster of 3 kafka-server and replication factor of message with 2.
Que 1:
I have 1-consumer group but 2 consumer in one group reading from 1-Topic,
If I enable auto commit true in consumer , will same message will be read
twice ?
Que
For your Question 1 No, Message will be read only once by anyone of the
consumer in a consumer group
For your Question 2 Yes, Because each consumer group will read a message
exactly once. Kafka is meant for Guaranteed Message Processing
There are three ways of Guaranteed Message Processing 1) Exa
where does kafka runs RAM or SSD ?
My question how to delete message frm Kafka topics.
Is message expire is only way ?
*Regards,*
*Laxmi Narayan Patel*
*MCA NIT Durgapur (2011-2014)*
*Mob:-9741292048,8345847473*
On Tue, Jan 3, 2017 at 12:46 PM, Kunal Gupta wrote:
> For your Questio
12 matches
Mail list logo
