Re: Kafka metrics: Bytesout greater than Bytesin??

2014-04-30 Thread Arnaud Lawson
s, > > Jun > > > On Tue, Apr 29, 2014 at 9:51 PM, Arnaud Lawson >wrote: > > > Thank you Jun, > > > > What could a partial message be? A consumer fetching for messages on an > > empty queue and receiving a message representing that fact? > > > >

Re: Kafka metrics: Bytesout greater than Bytesin??

2014-04-29 Thread Arnaud Lawson
will be ignored in the > consumer though. > > Thanks, > > Jun > > > On Tue, Apr 29, 2014 at 2:13 PM, Arnaud Lawson >wrote: > > > We are using Kafka 0.7.1 and i believe replication isn't available for > this > > version (correct me if i am wrong).

Re: Kafka metrics: Bytesout greater than Bytesin??

2014-04-29 Thread Arnaud Lawson
ctly double of BytesIn. > > > On Tue, Apr 29, 2014 at 1:26 PM, Arnaud Lawson >wrote: > > > Hello, > > > > After graphing the cumulative values of Bytesin and Bytesout for one of > our > > Kafka topics, we've noticed that the Bytesout values are sometimes >

Kafka metrics: Bytesout greater than Bytesin??

2014-04-29 Thread Arnaud Lawson
been put on the queue for that topic. Any explanations? Thanks. -- ​Arnaud Lawson​ , ​ Systems Operations Engineer *VELOS* Accelerating Machine Learning. 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001 C: (240) 393 - 6703 F: 646.349.4063 E: arnaud.law...@velos.io W: www.velos.io

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
Ok thank you. Arnaud- On Tue, Apr 1, 2014 at 10:50 AM, Jun Rao wrote: > Search for MaxLag in http://kafka.apache.org/documentation.html#monitoring > . > Note that this is only available in 0.8.0 and above. > > Thanks, > > Jun > > > On Tue, Apr 1, 2014 at

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
e a metric in the consumer that measures the # of messages a > consumer is behind. > > Thanks, > > Jun > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson >wrote: > > > Hi, > > > > Does anyone know what the JMX metric(s) are for determining the depth o

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
ffset of messages on the brokers v.s. the consumed last offsets > of the specified consumer on certain topics. > > Guozhang > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson >wrote: > > > Hi, > > > > Does anyone know what the JMX metric(s) are for dete

Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
Hi, Does anyone know what the JMX metric(s) are for determining the depth of the kafka queue? I basically want to know the amount of messages that are on the queue at a certain point in time. Please let me know if there is a way to find that out. Thanks. -- Arnaud Lawson , Systems Operations

Migration Process from Kafka 0.7 to 0.8

2013-12-09 Thread Arnaud Lawson
/confluence/display/KAFKA/Migrating+from+0.7+to+0.8#Migratingfrom0.7to0.8-IsshallowiterationsupportedfortheMigrationtool%27sconsumerconfig%3F -- Available at these partners: [image: CloudFlare | shopify | Bigcommerce] -- ARNAUD LAWSON, SYSTEMS OPERATIONS ENGINEER *SOCIOCAST* Simple. Powerful. Predictions