Jason Dere created ORC-536:
------------------------------
Summary: "expectedEntries should be > 0" error when bloom filters
enabled but rowIndexStride set to 0
Key: ORC-536
URL: https://issues.apache.org/jira/browse/ORC-536
Project: ORC
Issue Type: Bug
Components: Java
Reporter: Jason Dere
{noformat}
java.lang.IllegalArgumentException: expectedEntries should be > 0
at
shadeorcorg.apache.orc.util.BloomFilter.checkArgument(BloomFilter.java:54)
at shadeorcorg.apache.orc.util.BloomFilter.<init>(BloomFilter.java:63)
at
shadeorcorg.apache.orc.util.BloomFilterUtf8.<init>(BloomFilterUtf8.java:30)
at
shadeorcorg.apache.orc.impl.writer.TreeWriterBase.<init>(TreeWriterBase.java:116)
at
shadeorcorg.apache.orc.impl.writer.StringBaseTreeWriter.<init>(StringBaseTreeWriter.java:64)
at
shadeorcorg.apache.orc.impl.writer.StringTreeWriter.<init>(StringTreeWriter.java:33)
at
shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:129)
at
shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
at
shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
at
shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
at
shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
at shadeorcorg.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:180)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.WriterImpl.<init>(WriterImpl.java:94)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.OrcFile.createWriter(OrcFile.java:378)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.initWriter(OrcRecordUpdater.java:603)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSimpleEvent(OrcRecordUpdater.java:424)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSplitUpdateEvent(OrcRecordUpdater.java:433)
at
shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.insert(OrcRecordUpdater.java:485)
at
org.apache.hive.streaming.AbstractRecordWriter.write(AbstractRecordWriter.java:439)
at
org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.write(HiveStreamingConnection.java:794)
{noformat}
Pretty easy to repro with TestWriterImpl:
{code}
public void testNoBFIfNoIndex() throws Exception {
// overriding the flag should result in a successful write (no exception)
conf.set(OrcConf.OVERWRITE_OUTPUT_FILE.getAttribute(), "true");
// Enable bloomfilter, but disable index
conf.set(OrcConf.ROW_INDEX_STRIDE.getAttribute(), "0");
conf.set(OrcConf.BLOOM_FILTER_COLUMNS.getAttribute(), "*");
Writer w = OrcFile.createWriter(testFilePath,
OrcFile.writerOptions(conf).setSchema(schema));
w.close();
}
{code}
cc [~prasanth_j]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
