I use lz4 compression for topic with JSON data, don't remember exact
numbers but lz4 compression ratio is higher than gzip with lower CPU load.
Not a huge difference but still significant.
But I found one serious issue - lz4 compression is not compatible with
Spark 1.5 or higher. Spark 1.5 updated
The LZ4 implementation "works" but has a framing bug that can make third
party client use difficult. See KAFKA-3160. If you only plan to use the
official Java client then that issue shouldn't be a problem.
-Dana
On Mar 21, 2016 12:26 PM, "Pete Wright" wrote:
>
>
> On 03/17/2016 04:03 PM, Virendr
On 03/17/2016 04:03 PM, Virendra Pratap Singh wrote:
More like getting a feel from the community about using lz4 for compression?
Has anyone used in the kafka setup.
I am aware that gzip and snappy are more older implementation and regressed.
Given that lz4 has better compression/decompressio
More like getting a feel from the community about using lz4 for compression?
Has anyone used in the kafka setup.
I am aware that gzip and snappy are more older implementation and regressed.
Given that lz4 has better compression/decompression cycles (though slightly
less compression ratio), was t
