[
https://issues.apache.org/jira/browse/HDFS-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039159#comment-13039159
]
Harsh J Chouraria commented on HDFS-1640:
-----------------------------------------
@Option 2 -- This can be done at the writer level itself. Much too expensive
maintaining a per-file property for compression and implementing the same to be
guaranteed as well.
Besides, compressing on the fly for tx would still invoke the same amount of
IO. No savings, right?
> Transmission of large files in compressed format will save the network
> traffic and can improve the data transfer time.
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-1640
> URL: https://issues.apache.org/jira/browse/HDFS-1640
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, hdfs client
> Reporter: Uma Maheswara Rao G
>
> *In Write scenario:*
> DFSClient can Compress the data when transmitting it over the network, Data
> Nodes can forward the same compressed data to other Data Nodes in pipeline
> and as well as it can decompress that data and write on to their local disks.
> *In Read Scenario:*
> Data Node can compress the data when transmitting it over the network.
> DFSClient can decompress it and write on to the local stream.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
