[
https://issues.apache.org/jira/browse/FLINK-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Hutchison updated FLINK-6185:
----------------------------------
Description:
File sources (such as {{env#readCsvFile()}}) and sinks (such as
{{FileOutputFormat}} and its subclasses, and methods such as
{{DataSet#writeAsText()}}) need the ability to transparently decompress and
compress files. Primarily gzip would be useful, but it would be nice if this
were pluggable to support bzip2, xz, etc.
There could be options for autodetect (based on file extension and/or file
content), which could be the default, as well as no compression or a selected
compression method.
was:
File sources (such as {{env#readCsvFile()}}) and sinks (such as
FileOutputFormat and its subclasses, and methods such as
{{DataSet#writeAsText()}}) need the ability to transparently decompress and
compress files. Primarily gzip would be useful, but it would be nice if this
were pluggable to support bzip2, xz, etc.
There could be options for autodetect (based on file extension and/or file
content), which could be the default, as well as no compression or a selected
compression method.
> Input readers and output writers/formats need to support gzip
> -------------------------------------------------------------
>
> Key: FLINK-6185
> URL: https://issues.apache.org/jira/browse/FLINK-6185
> Project: Flink
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.2.0
> Reporter: Luke Hutchison
> Priority: Minor
>
> File sources (such as {{env#readCsvFile()}}) and sinks (such as
> {{FileOutputFormat}} and its subclasses, and methods such as
> {{DataSet#writeAsText()}}) need the ability to transparently decompress and
> compress files. Primarily gzip would be useful, but it would be nice if this
> were pluggable to support bzip2, xz, etc.
> There could be options for autodetect (based on file extension and/or file
> content), which could be the default, as well as no compression or a selected
> compression method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
