[ https://issues.apache.org/jira/browse/IMPALA-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-3766: ---------------------------------- Summary: Optionally compress spilled data before writing it do disk (was: LZ4 compress spill data before writing it do disk) > Optionally compress spilled data before writing it do disk > ---------------------------------------------------------- > > Key: IMPALA-3766 > URL: https://issues.apache.org/jira/browse/IMPALA-3766 > Project: IMPALA > Issue Type: New Feature > Components: Backend > Affects Versions: Impala 2.7.0 > Reporter: Mostafa Mokhtar > Assignee: Tim Armstrong > Priority: Minor > Labels: performance > > Evaluate compressing the buffers before writing them to disk for spilling > operators. > Applying LZ4 on row batches before sending them over the network as part of > exchange provides around 2x compression. > {code} > - BytesSent: 612.87 MB (642635712) > - NetworkThroughput(*): 1.88 GB/sec > - OverallThroughput: 1.21 GB/sec > - PeakMemoryUsage: 51.00 KB (52224) > - RowsReturned: 360.00K (360000) > - SerializeBatchTime: 176.002ms > - TransmitDataRPCTime: 319.005ms > - UncompressedRowBatchSize: 1.47 GB (1573356320) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org