Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/12741 )
Change subject: IMPALA-6625: Skip computing parquet conjuncts for non-Parquet
scans
......................................................................
Patch Set 1:
Conflicts:
testdata/bin/create-load-data.sh
302 function copy-and-load-dependent-tables {
303 # COPY
304 # TODO: The multi-format table will move these files. So we need to copy
them to a
305 # temporary location for that table to use. Should find a better way to
handle this.
306 echo COPYING AND LOADING DATA FOR DEPENDENT TABLES
307 hadoop fs -rm -r -f /test-warehouse/alltypesmixedformat \
308 /tmp/alltypes_rc /tmp/alltypes_seq /tmp/alltypes_parquet
309 hadoop fs -mkdir -p /tmp/alltypes_seq/year=2009 \
310 <<<<<<< HEAD
311 /tmp/alltypes_rc/year=2009
312 =======
313 /tmp/alltypes_rc/year=2009 /tmp/alltypes_parquet/year=2009
314
315 # The file written by hive to /test-warehouse will be strangely
replicated rather than
316 # erasure coded if EC is not set in /tmp
317 if [[ -n "${HDFS_ERASURECODE_POLICY:-}" ]]; then
318 hdfs ec -setPolicy -policy "${HDFS_ERASURECODE_POLICY}" -path
"/tmp/alltypes_rc"
319 hdfs ec -setPolicy -policy "${HDFS_ERASURECODE_POLICY}" -path
"/tmp/alltypes_seq"
320 hdfs ec -setPolicy -policy "${HDFS_ERASURECODE_POLICY}" -path
"/tmp/alltypes_parquet"
321 fi
322
323 >>>>>>> c6f9b61... IMPALA-6625: Skip computing parquet conjuncts for
non-Parquet scans
324 hadoop fs -cp /test-warehouse/alltypes_seq/year=2009/month=2/
/tmp/alltypes_seq/year=2009
325 hadoop fs -cp /test-warehouse/alltypes_rc/year=2009/month=3/
/tmp/alltypes_rc/year=2009
326 hadoop fs -cp /test-warehouse/alltypes_parquet/year=2009/month=4/
/tmp/alltypes_parquet/year=2009
--
To view, visit http://gerrit.cloudera.org:8080/12741
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d6c26d42db090c8a15c602f6419ad6399c329e7
Gerrit-Change-Number: 12741
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Pooja Nilangekar <pooja.nilange...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2019 04:56:19 +0000
Gerrit-HasComments: No
