[
https://issues.apache.org/jira/browse/SPARK-2399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Bowyer updated SPARK-2399:
-------------------------------
Description:
LZ4 is a compression codec of the same ideas as googles snappy, but has some
advantages:
* It is faster than snappy with a similar compression ratio
* The implementation is Apache licensed and not GPL
It has shown promise in both the lucene and hadoop communities, and it looks
like its a really easy add to spark io compression.
Attached is a patch that does this
was:
LZ4 is a compression codec of the same ideas as googles snappy, but has some
advantages:
* It is faster than snappy with a similar compression ration
* The implementation is Apache licensed and not GPL
It has shown promise in both the lucene and hadoop communities, and it looks
like its a really easy add to spark io compression.
Attached is a patch that does this
> Add support for LZ4 compression
> -------------------------------
>
> Key: SPARK-2399
> URL: https://issues.apache.org/jira/browse/SPARK-2399
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Greg Bowyer
> Labels: compression, lz4
> Attachments: SPARK-2399-Make-spark-compression-able-to-use-LZ4.patch
>
>
> LZ4 is a compression codec of the same ideas as googles snappy, but has some
> advantages:
> * It is faster than snappy with a similar compression ratio
> * The implementation is Apache licensed and not GPL
> It has shown promise in both the lucene and hadoop communities, and it looks
> like its a really easy add to spark io compression.
> Attached is a patch that does this
--
This message was sent by Atlassian JIRA
(v6.2#6252)
