I upgrade two small test cluster and I had two small issues but I'm, not
clear yet as to if those were an issue due to us using ansible to configure
and deploy the cluster.
The first issue could be us doing something bad when distributing the
update (I updated, not reinstalled) but it should be
Svante,
I tested this out locally and the mbeans for those metrics do show up on
startup. Can you reproduce the issue reliably? Also, is what you saw an
issue with the mbean itself or graphite?
Thanks,
Jun
On Fri, Jan 16, 2015 at 4:38 AM, svante karlsson s...@csi.se wrote:
I upgrade two
Hi ,
I have a general query -
As per the code in Kafka producer the serialization happens before
partitioning , Is my understanding correct ? If yes whats the reason for it
?
Regards,
Liju John
Those errors are expected - if broker 10.0.0.11 went down, it will
reset the connection and the other broker will close the socket.
However, it looks like 10.0.0.11 crashes every two minutes?
Do you have the logs from 10.0.0.11?
On Thu, Jan 15, 2015 at 9:51 PM, Tousif tousif.pa...@gmail.com
Jun,
I don't know if it was an issue with graphite or the mbean but I have not
seen it since - and we have tried of several cases of failover and this
problem has only been seen once.
That said, I have the feeling that it was a kafka issue and I'm a bit
suspicious about the new mbeans.
I attach
Hi Rajiv,
Thanks for this proposal, it would be great if you can upload some
implementation patch for the CAS idea and show some memory usage / perf
differences.
Guozhang
On Sun, Dec 14, 2014 at 9:27 PM, Rajiv Kurian ra...@signalfuse.com wrote:
Resuscitating this thread. I've done some more
Tousif,
I meant to say if kafka broker is going down often its better to
analyze whats the root of cause of the crash. Using supervisord
to monitor kafka broker is fine, sorry about the confusion.
-Harsha
On Fri, Jan 16, 2015, at 11:25 AM, Gwen Shapira wrote:
Those errors
Disregard the previous message, it was send accidently..
Jun,
I don't know if it was an issue with graphite or the mbean and have not
seen it since - and we have tried of several cases of failover.
That said, I have the feeling that it was a kafka issue and I'm a bit
suspicious about the new
Hi,
I was wondering if anyone can compare and contrast KOYA and Samza?
Thanks,
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr Elasticsearch Support * http://sematext.com/
Hey Otis,
I'm not terribly familiar with KOYA, but my understanding is that it's a
tool for deploying Kafka brokers to YARN, and administering them. I don't
think that it has any stream processing functionality built into it. As
such, it seems to me that KOYA and Samza could be used together: you
Hi,
I am following this link:
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
for a consumer design using kafka_2.9.2 version 0.8.1.1 (what I can find in
maven central). I have a couple of questions about the consumer. I
checked the archives and didn't see these
Apache mailing list doesn't allow attachments. Could you file a jira and
attach the graph and the description there?
Thanks,
Jun
On Fri, Jan 16, 2015 at 12:09 PM, svante karlsson s...@csi.se wrote:
Disregard the previous message, it was send accidently..
Jun,
I don't know if it was an
I did some quick tests and the mbean values look reasonable. On the
producer side, produce msg/sec is actually for all brokers.
Thanks,
Jun
On Fri, Jan 16, 2015 at 12:09 PM, svante karlsson s...@csi.se wrote:
Disregard the previous message, it was send accidently..
Jun,
I don't know if it
Hm. My understanding was that both are aimed at basically the same thing -
Kafka on YARN. From Samza site:
Apache Samza is a distributed stream processing framework. It uses Apache
Kafka http://kafka.apache.org/ for messaging, and Apache Hadoop YARN
An analogy I use is that in comparison to classic MapReduce with its
storage (HDFS) and compute (MR), Kafka is the storage and Samza is the
compute. But HDFS does not run through YARN...
So, in the same way that HDFS is run and launched separately from YARN
(and hence Map-Reduce), Kafka is also
I appologize in advance for a noob question, just getting started with
kafka, and trying to get JMX data from it.
So, I had thought that running the JXMTool with no arguments would dump all
the data, but it doesn't seem to return.
I do know I can query for a specific Mbean name seems to work.
Here are some more tools for that
https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters depending
on what you have in place and what you are trying todo different options
exist.
A lot of folks like JMX Trans.
My favorite quick out of the box is using
Hi,
1. In SimpleConsumer, you must keep track of the offsets in your
application.
In the example code, readOffset variable can be saved in
redis/zookeeper.
You should plugin this logic in your code. High Level Consumer stores
the last
read offset information in ZooKeeper.
2. You will
Thanks, I actually ran into those already. I was hoping just to be able to
dump the JMX data plain and simple. I can consume it with other tools but I
am mostly just trying to get the metrics in some format... any format.
I have some limitations on what I can build/run, so hoping I can just
You can use the DumpLogSegments tool to see if a log segment is indeed
corrupted.
Thanks,
Jun
On Mon, Jan 12, 2015 at 2:04 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com
wrote:
Hi Kafka Team,
I am trying to find out Kafka Internal and how a message can be corrupted
or lost at brokers side.
Perhaps you can see if you can reproduce this issue and then file a jira.
Thanks,
Jun
On Fri, Jan 16, 2015 at 4:39 PM, svante karlsson s...@csi.se wrote:
Hmm, produce msg/sec in rate seems to be per broker and produce msg/per
sec should also be per broker and thus be related. The problem is
Hello Everyone,
Thank you for the time and help. I had the Kafka Producer running, but am
having some trouble now.
1) Using Maven, I wrote a Kafka Producer similar to the one found here:
https://github.com/pppsunil/HelloKafka/blob/master/src/main/java/com/spnotes/kafka/HelloKafkaProducer.java
Pl check your classpath. Some jars might be missing.
On Sat, Jan 17, 2015 at 7:41 AM, Su She suhsheka...@gmail.com wrote:
Hello Everyone,
Thank you for the time and help. I had the Kafka Producer running, but am
having some trouble now.
1) Using Maven, I wrote a Kafka Producer similar to
23 matches
Mail list logo
