Re: Review Request 26887: StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26887/ --- (Updated Oct. 17, 2014, 7:10 p.m.) Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q to reproduce it. Simply, this query does three data loads which generates three base orc files. ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 07efb8f Diff: https://reviews.apache.org/r/26887/diff/ Testing --- passed alter_merge_stats_orc.q Thanks, XIAOBING ZHOU
Review Request 26887: StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26887/ --- Review request for hive. Repository: hive-git Description --- run the test mvn -Phadoop-2 test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q to reproduce it. Simply, this query does three data loads which generates three base orc files. ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 07efb8f Diff: https://reviews.apache.org/r/26887/diff/ Testing --- passed alter_merge_stats_orc.q Thanks, XIAOBING ZHOU
