Attila Jeges has uploaded a new patch set (#2). Change subject: IMPALA-5407: Fix crash in HdfsSequenceTableWriter ......................................................................
IMPALA-5407: Fix crash in HdfsSequenceTableWriter The following use of sequence file writer can lead to a crash: > set compression_codec=gzip; > set seq_compression_mode=record; > set allow_unsupported_formats=1; > create table seq_tbl like tbl stored as sequencefile; > insert into seq_tbl select * from tbl; The issue is that HdfsSequenceTableWriter::Flush() frees the per-file memory pool, but the buffer used for compression is still in use after Flush() returns. If there are additional rows to process after calling Flush(), impalad might crash. This fix removes the FreeAll() call from Flush(). It was tested on a local minicluster with a table containing around 100 million rows. Change-Id: Ida0b9f189175358ae54149d0e1af7caa06ae3bec --- M be/src/exec/hdfs-sequence-table-writer.cc 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/7394/2 -- To view, visit http://gerrit.cloudera.org:8080/7394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida0b9f189175358ae54149d0e1af7caa06ae3bec Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Attila Jeges <atti...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com>
