[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these claim minor performance improvements. Testing: - Ran release exhaustive job and debug core job - Ran TPC-H scale 42 with Parquet/Snappy and Parquet/ZSTD. (ZSTD tests ran with default compression level.) Parquet/Snappy was unchanged. Parquet/ZSTD improved: +--++-++++ | Workload | File Format| Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +--++-++++ | TPCH(42) | parquet / zstd / block | 8.50| -2.10% | 5.46 | -2.63% | +--++-++++ Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Reviewed-on: http://gerrit.cloudera.org:8080/17254 Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 6 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 20:42:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7054/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 14:47:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7053/ -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 08:58:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8522/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 03:32:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7053/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 03:13:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 5: Code-Review+2 Went back to the toolchain without JWT. Carrying +2 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 08 Apr 2021 03:12:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17254 to look at the new patch set (#5). Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these claim minor performance improvements. Testing: - Ran release exhaustive job and debug core job - Ran TPC-H scale 42 with Parquet/Snappy and Parquet/ZSTD. (ZSTD tests ran with default compression level.) Parquet/Snappy was unchanged. Parquet/ZSTD improved: +--++-++++ | Workload | File Format| Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +--++-++++ | TPCH(42) | parquet / zstd / block | 8.50| -2.10% | 5.46 | -2.63% | +--++-++++ Change-Id: I858f82f773023bd0aea14543f18bd74071758468 --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/17254/5 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 07 Apr 2021 10:41:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 07 Apr 2021 06:56:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8511/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 07 Apr 2021 04:59:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7049/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 07 Apr 2021 04:44:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 4: Rebased and used a slightly newer toolchain build -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 07 Apr 2021 04:39:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17254 to look at the new patch set (#4). Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these claim minor performance improvements. Testing: - Ran release exhaustive job and debug core job - Ran TPC-H scale 42 with Parquet/Snappy and Parquet/ZSTD. (ZSTD tests ran with default compression level.) Parquet/Snappy was unchanged. Parquet/ZSTD improved: +--++-++++ | Workload | File Format| Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +--++-++++ | TPCH(42) | parquet / zstd / block | 8.50| -2.10% | 5.46 | -2.63% | +--++-++++ Change-Id: I858f82f773023bd0aea14543f18bd74071758468 --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/17254/4 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8492/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 16:30:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17254 to look at the new patch set (#3). Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these claim minor performance improvements. Testing: - Ran release exhaustive job and debug core job - Ran TPC-H scale 42 with Parquet/Snappy and Parquet/ZSTD. (ZSTD tests ran with default compression level.) Parquet/Snappy was unchanged. Parquet/ZSTD improved: +--++-++++ | Workload | File Format| Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +--++-++++ | TPCH(42) | parquet / zstd / block | 8.50| -2.10% | 5.46 | -2.63% | +--++-++++ Change-Id: I858f82f773023bd0aea14543f18bd74071758468 --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/17254/3 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8481/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 18:56:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17254 Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these have minor performance improvements. Testing: - Ran release exhaustive job and debug core job TODO: - Run performance job Change-Id: I858f82f773023bd0aea14543f18bd74071758468 --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/17254/1 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
