Are you using snappy compression? There was a bug with snappy that caused
corrupt messages.
Sent from my iPhone
> On Mar 29, 2016, at 8:15 AM, sunil kalva wrote:
>
> Hi
> Do we store message crc also on disk, and server verifies same when we are
> reading messages back from disk?
> And how to
Hi
Do we store message crc also on disk, and server verifies same when we are
reading messages back from disk?
And how to handle errors when we use async publish ?
On Fri, Mar 25, 2016 at 4:17 AM, Becket Qin wrote:
> You mentioned that you saw few corrupted messages, (< 0.1%). If so are you
> ab
You mentioned that you saw few corrupted messages, (< 0.1%). If so are you
able to see some corrupted messages if you produce, say, 10M messages?
On Wed, Mar 23, 2016 at 9:40 PM, sunil kalva wrote:
> I am using java client and kafka 0.8.2, since events are corrupted in
> kafka broker i cant rea
I am using java client and kafka 0.8.2, since events are corrupted in
kafka broker i cant read and replay them again.
On Thu, Mar 24, 2016 at 9:42 AM, Becket Qin wrote:
> Hi Sunil,
>
> The messages in Kafka has a CRC stored with each of them. When consumer
> receives a message, it will compute
Hi Sunil,
The messages in Kafka has a CRC stored with each of them. When consumer
receives a message, it will compute the CRC from the message bytes and
compare it to the stored CRC. If the computed CRC and stored CRC does not
match, that indicates the message has corrupted. I am not sure in your
can some one help me out here.
On Wed, Mar 23, 2016 at 7:36 PM, sunil kalva wrote:
> Hi
> I am seeing few messages getting corrupted in kafka, It is not happening
> frequently and percentage is also very very less (less than 0.1%).
>
> Basically i am publishing thrift events in byte array format
Hi
I am seeing few messages getting corrupted in kafka, It is not happening
frequently and percentage is also very very less (less than 0.1%).
Basically i am publishing thrift events in byte array format to kafka
topics(with out encoding like base64), and i also see more events than i
publish (i c
