[
https://issues.apache.org/jira/browse/APEXMALHAR-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999009#comment-15999009
]
ASF GitHub Bot commented on APEXMALHAR-2487:
--------------------------------------------
GitHub user ilganeli opened a pull request:
https://github.com/apache/apex-malhar/pull/616
APEXMALHAR-2487 Added support for Snappy compression in FilterStreamProvider
* Based on existing code to output Gzip or CipherText this patch adds
support for writing data out as Hadoop-readable Snappy format
* Added unit tests which validate both the provider and the simpler
SnappyStream functionality.
* This patch reuses some code from existing tests where possible.
https://issues.apache.org/jira/browse/APEXMALHAR-2487
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/incubator-apex-malhar APEXMALHAR-2487
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/616.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #616
----
commit e756309c676778917ae0464c1ec44a9cf84fbe2b
Author: Ilya Ganelin <[email protected]>
Date: 2017-04-29T04:54:06Z
Added support for Snappy compression in FilterStreamProvider, which in turn
enables Snappy output.
commit 9ed1e7e3e0bb357f4cdad0b3cfc34cc40404d9f3
Author: Ilya Ganelin <[email protected]>
Date: 2017-04-29T04:58:55Z
Added additional check for presence of native Snappy libraries.
commit 9e442c8f41284d13fef1f2ede99a17f7e105c06d
Author: Ilya Ganelin <[email protected]>
Date: 2017-04-29T05:18:46Z
Checkstyle fixes.
commit 4f19d45d98f88d17dcde2e3b040b68f9f8a7f1b8
Author: Ilya Ganelin <[email protected]>
Date: 2017-04-29T05:41:10Z
Fixed header.
commit 70a30e6ef855afb5cb5399a84c271fa93ef23c05
Author: Ilya Ganelin <[email protected]>
Date: 2017-05-05T21:48:14Z
Adressed PR comments.
----
> Malhar should support outputting data in Snappy compression
> -----------------------------------------------------------
>
> Key: APEXMALHAR-2487
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2487
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Reporter: Ilya Ganelin
> Assignee: Ilya Ganelin
>
> At present, the default file output operator (AbstractFileOutputOperator)
> supports compression by setting the FilterStreamProvider. However, Malhar
> presently only includes two FilterStreamProvider - one to Cipher data, and
> one for Gzip.
> Snappy offers substantially improved performance over Gzip in terms of
> compression and decompression speed at the expense of compression ratio. In
> certain applications this is useful. Thus, it would be helpful to add a
> Snappy FilterStreamProvider.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
