Can anyone help with this question? On Tue, Jun 14, 2016 at 1:45 PM, allen chan <allen.michael.c...@gmail.com> wrote:
> Thanks for answer Otis. > The producer that i use (Logstash) does not track message sizes. > > I already loaded all the metrics from JMX into my monitoring system. > I just need to confirm that "record" is equivalent to an individual log > message. > > > > On Tue, Jun 14, 2016 at 1:27 PM, Otis Gospodnetić < > otis.gospodne...@gmail.com> wrote: > >> Hi, >> >> Do you control the producers? If so, couldn't you measure the message >> sizes there? >> Alternatively, you can use something like SPM for Kafka or other Kafka >> monitoring tools that expose relevant metrics. >> >> For example, I think you can compute avg msg size based on metrics shown >> in >> these charts: >> https://apps.sematext.com/spm-reports/s/T1aD50s1Xp >> https://apps.sematext.com/spm-reports/s/x5WcTA470A >> >> Otis >> -- >> Monitoring - Log Management - Alerting - Anomaly Detection >> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >> >> >> On Mon, Jun 13, 2016 at 4:43 PM, allen chan <allen.michael.c...@gmail.com >> > >> wrote: >> >> > In JMX for Kafka producer there are metrics for both request, record, >> and >> > batch size Max + Avg. >> > >> > What is the difference between these concepts? >> > >> > In the logging use case: I assume record is the single log line, batch >> is >> > multiple log lines together and request is the batch wrapped with the >> > metadata to be sent. >> > >> > Can someone confirm or correct this assumption? >> > I need to be able to see how big a message is so i can properly tune >> this >> > setting: message.max.bytes. The confusion i have is do i use the >> request or >> > the record metric. >> > >> > >> > Thanks, >> > -- >> > Allen Michael Chan >> > >> > > > > -- > Allen Michael Chan > -- Allen Michael Chan
