lipeng...@sensorsdata.cn has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/19114 )
Change subject: IMPALA-11607: Support for TPC-H and TPC-DS test datasets stored as Iceberg tables ...................................................................... IMPALA-11607: Support for TPC-H and TPC-DS test datasets stored as Iceberg tables Impala 4.1.0 has officially supported Iceberg tables, but we don't have tools to generate TPC-H and TPC-DS test datasets for Iceberg tables. We should use load-data.py to generate test datasets, e.g: "${IMPALA_HOME}/bin/load-data.py -w tpcds -f -s 10 --table_formats=text/none,iceberg/none". Testing: - load-data.py -w tpch -s 1 --table_formats=text/none,parquet/none, iceberg/none -e exhaustive -f - load-data.py -w tpcds -s 1 --table_formats=text/none,parquet/zstd, iceberg/zstd -e exhaustive -f - single_node_perf_run.py --workloads=tpch --scale=30 --iterations=3 -- table_formats=parquet/lz4,iceberg/lz4 --num_impalads=1 --load 3ee3dc847 Workload: TPCH, Scale Factor: 30 Table Format: parquet/lz4/block +----------+----------------+-----------+ | Query | Time Taken (s) | Client ID | +----------+----------------+-----------+ | TPCH-Q1 | 7.73 | 1 | | TPCH-Q1 | 7.50 | 1 | | TPCH-Q1 | 7.55 | 1 | +----------+----------------+-----------+ | TPCH-Q10 | 3.65 | 1 | | TPCH-Q10 | 3.70 | 1 | | TPCH-Q10 | 3.69 | 1 | +----------+----------------+-----------+ | TPCH-Q11 | 0.68 | 1 | | TPCH-Q11 | 0.68 | 1 | | TPCH-Q11 | 0.67 | 1 | +----------+----------------+-----------+ | TPCH-Q12 | 1.33 | 1 | | TPCH-Q12 | 1.08 | 1 | | TPCH-Q12 | 1.08 | 1 | +----------+----------------+-----------+ | TPCH-Q13 | 10.63 | 1 | | TPCH-Q13 | 10.36 | 1 | | TPCH-Q13 | 10.52 | 1 | +----------+----------------+-----------+ | TPCH-Q14 | 1.63 | 1 | | TPCH-Q14 | 1.79 | 1 | | TPCH-Q14 | 1.63 | 1 | +----------+----------------+-----------+ | TPCH-Q15 | 1.44 | 1 | | TPCH-Q15 | 1.54 | 1 | | TPCH-Q15 | 1.45 | 1 | +----------+----------------+-----------+ | TPCH-Q16 | 2.35 | 1 | | TPCH-Q16 | 2.35 | 1 | | TPCH-Q16 | 2.40 | 1 | +----------+----------------+-----------+ | TPCH-Q17 | 1.20 | 1 | | TPCH-Q17 | 1.13 | 1 | | TPCH-Q17 | 1.10 | 1 | +----------+----------------+-----------+ | TPCH-Q18 | 17.50 | 1 | | TPCH-Q18 | 17.04 | 1 | | TPCH-Q18 | 14.43 | 1 | +----------+----------------+-----------+ | TPCH-Q19 | 1.61 | 1 | | TPCH-Q19 | 1.56 | 1 | | TPCH-Q19 | 1.56 | 1 | +----------+----------------+-----------+ | TPCH-Q2 | 0.68 | 1 | | TPCH-Q2 | 0.68 | 1 | | TPCH-Q2 | 0.68 | 1 | +----------+----------------+-----------+ | TPCH-Q20 | 1.09 | 1 | | TPCH-Q20 | 1.09 | 1 | | TPCH-Q20 | 1.15 | 1 | +----------+----------------+-----------+ | TPCH-Q21 | 24.94 | 1 | | TPCH-Q21 | 26.42 | 1 | | TPCH-Q21 | 25.62 | 1 | +----------+----------------+-----------+ | TPCH-Q22 | 3.69 | 1 | | TPCH-Q22 | 3.58 | 1 | | TPCH-Q22 | 3.24 | 1 | +----------+----------------+-----------+ | TPCH-Q3 | 3.58 | 1 | | TPCH-Q3 | 3.68 | 1 | | TPCH-Q3 | 3.89 | 1 | +----------+----------------+-----------+ | TPCH-Q4 | 1.37 | 1 | | TPCH-Q4 | 1.37 | 1 | | TPCH-Q4 | 1.32 | 1 | +----------+----------------+-----------+ | TPCH-Q5 | 1.68 | 1 | | TPCH-Q5 | 1.69 | 1 | | TPCH-Q5 | 1.65 | 1 | +----------+----------------+-----------+ | TPCH-Q6 | 0.46 | 1 | | TPCH-Q6 | 0.47 | 1 | | TPCH-Q6 | 0.47 | 1 | +----------+----------------+-----------+ | TPCH-Q7 | 2.54 | 1 | | TPCH-Q7 | 2.49 | 1 | | TPCH-Q7 | 2.54 | 1 | +----------+----------------+-----------+ | TPCH-Q8 | 1.90 | 1 | | TPCH-Q8 | 1.90 | 1 | | TPCH-Q8 | 1.90 | 1 | +----------+----------------+-----------+ | TPCH-Q9 | 12.72 | 1 | | TPCH-Q9 | 12.02 | 1 | | TPCH-Q9 | 12.67 | 1 | +----------+----------------+-----------+ Table Format: iceberg/lz4/block +----------+----------------+-----------+ | Query | Time Taken (s) | Client ID | +----------+----------------+-----------+ | TPCH-Q1 | 7.51 | 1 | | TPCH-Q1 | 7.61 | 1 | | TPCH-Q1 | 7.60 | 1 | +----------+----------------+-----------+ | TPCH-Q10 | 3.79 | 1 | | TPCH-Q10 | 3.73 | 1 | | TPCH-Q10 | 3.67 | 1 | +----------+----------------+-----------+ | TPCH-Q11 | 0.75 | 1 | | TPCH-Q11 | 0.88 | 1 | | TPCH-Q11 | 0.69 | 1 | +----------+----------------+-----------+ | TPCH-Q12 | 1.18 | 1 | | TPCH-Q12 | 1.13 | 1 | | TPCH-Q12 | 1.04 | 1 | +----------+----------------+-----------+ | TPCH-Q13 | 10.11 | 1 | | TPCH-Q13 | 9.76 | 1 | | TPCH-Q13 | 10.31 | 1 | +----------+----------------+-----------+ | TPCH-Q14 | 1.64 | 1 | | TPCH-Q14 | 1.64 | 1 | | TPCH-Q14 | 1.53 | 1 | +----------+----------------+-----------+ | TPCH-Q15 | 1.44 | 1 | | TPCH-Q15 | 1.39 | 1 | | TPCH-Q15 | 1.35 | 1 | +----------+----------------+-----------+ | TPCH-Q16 | 2.24 | 1 | | TPCH-Q16 | 2.15 | 1 | | TPCH-Q16 | 2.15 | 1 | +----------+----------------+-----------+ | TPCH-Q17 | 1.00 | 1 | | TPCH-Q17 | 1.00 | 1 | | TPCH-Q17 | 0.99 | 1 | +----------+----------------+-----------+ | TPCH-Q18 | 15.14 | 1 | | TPCH-Q18 | 14.94 | 1 | | TPCH-Q18 | 15.04 | 1 | +----------+----------------+-----------+ | TPCH-Q19 | 1.63 | 1 | | TPCH-Q19 | 1.66 | 1 | | TPCH-Q19 | 1.60 | 1 | +----------+----------------+-----------+ | TPCH-Q2 | 0.69 | 1 | | TPCH-Q2 | 0.75 | 1 | | TPCH-Q2 | 0.69 | 1 | +----------+----------------+-----------+ | TPCH-Q20 | 1.10 | 1 | | TPCH-Q20 | 1.09 | 1 | | TPCH-Q20 | 1.21 | 1 | +----------+----------------+-----------+ | TPCH-Q21 | 25.38 | 1 | | TPCH-Q21 | 25.48 | 1 | | TPCH-Q21 | 25.18 | 1 | +----------+----------------+-----------+ | TPCH-Q22 | 3.54 | 1 | | TPCH-Q22 | 3.59 | 1 | | TPCH-Q22 | 3.49 | 1 | +----------+----------------+-----------+ | TPCH-Q3 | 3.95 | 1 | | TPCH-Q3 | 3.96 | 1 | | TPCH-Q3 | 3.79 | 1 | +----------+----------------+-----------+ | TPCH-Q4 | 1.43 | 1 | | TPCH-Q4 | 1.43 | 1 | | TPCH-Q4 | 1.43 | 1 | +----------+----------------+-----------+ | TPCH-Q5 | 1.84 | 1 | | TPCH-Q5 | 1.80 | 1 | | TPCH-Q5 | 1.71 | 1 | +----------+----------------+-----------+ | TPCH-Q6 | 0.56 | 1 | | TPCH-Q6 | 0.47 | 1 | | TPCH-Q6 | 0.53 | 1 | +----------+----------------+-----------+ | TPCH-Q7 | 2.75 | 1 | | TPCH-Q7 | 2.59 | 1 | | TPCH-Q7 | 2.49 | 1 | +----------+----------------+-----------+ | TPCH-Q8 | 1.90 | 1 | | TPCH-Q8 | 2.00 | 1 | | TPCH-Q8 | 1.91 | 1 | +----------+----------------+-----------+ | TPCH-Q9 | 13.34 | 1 | | TPCH-Q9 | 12.18 | 1 | | TPCH-Q9 | 12.71 | 1 | +----------+----------------+-----------+ Change-Id: Ic8bbc5209e7e649f67a48144a2844b35d9f9c7f1 --- M testdata/bin/generate-schema-statements.py M tests/common/test_dimensions.py 2 files changed, 33 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/19114/9 -- To view, visit http://gerrit.cloudera.org:8080/19114 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic8bbc5209e7e649f67a48144a2844b35d9f9c7f1 Gerrit-Change-Number: 19114 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward <lipeng...@sensorsdata.cn> Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>