[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16645 ) Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7617/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 10 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 05 Nov 2020 06:45:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Hello Gabor Kaszab, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16645 to look at the new patch set (#10). Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. IMPALA-10279: Import CPC functionality from DataSketches This patch imports the functionality needed for CPC approximate algorithm from Apache DataSketches. First, I updated our existing snapshot of DataSketches to the following commit:2b84e213067b681b696ec883d245ddf911790ff2 "Merge pull request #172 from chufucun/const_cast" This affects files originated from cpc/ directories of the DataSketches repo. Then I copied all the files needed for CPC into our snapshot directory. Browse the source files here: https://github.com/apache/incubator-datasketches-cpp Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c --- M be/src/exprs/CMakeLists.txt M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/binomial_bounds.hpp A be/src/thirdparty/datasketches/ceiling_power_of_2.hpp M be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/compression_data.hpp A be/src/thirdparty/datasketches/conditional_back_inserter.hpp A be/src/thirdparty/datasketches/conditional_forward.hpp A be/src/thirdparty/datasketches/cpc_common.hpp A be/src/thirdparty/datasketches/cpc_compressor.hpp A be/src/thirdparty/datasketches/cpc_compressor_impl.hpp A be/src/thirdparty/datasketches/cpc_confidence.hpp A be/src/thirdparty/datasketches/cpc_sketch.hpp A be/src/thirdparty/datasketches/cpc_sketch_impl.hpp A be/src/thirdparty/datasketches/cpc_union.hpp A be/src/thirdparty/datasketches/cpc_union_impl.hpp A be/src/thirdparty/datasketches/cpc_util.hpp A be/src/thirdparty/datasketches/icon_estimator.hpp A be/src/thirdparty/datasketches/kxp_byte_lookup.hpp M be/src/thirdparty/datasketches/serde.hpp A be/src/thirdparty/datasketches/u32_table.hpp A be/src/thirdparty/datasketches/u32_table_impl.hpp 24 files changed, 10,273 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/16645/10 -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 10 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16645 ) Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7616/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 8 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 05 Nov 2020 05:47:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Fucun Chu has posted comments on this change. (
http://gerrit.cloudera.org:8080/16645 )
Change subject: IMPALA-10279: Import CPC functionality from DataSketches
..
Patch Set 8:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/16645/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16645/6//COMMIT_MSG@11
PS6, Line 11:
: First, I updated our existing snapshot of DataSketches to the
: following commit:2b84e213067b681b696ec883d245ddf911790ff2
> I checked manually the commits between the git hash we used previously and
The previous snapshot(commit:c1a6f8edb49699520f248d3d02019b87429b4241) already
contains the first 4 commits, see:
https://github.com/apache/incubator-datasketches-cpp/commits/2.1.x-incubating
The commit(c5be98a81c5947e5100868b90fb0f50aba4a9334) affects our snapshot files:
+41 ?0 {theta ? common}/include/binomial_bounds.hpp
+41 ?0 common/include/ceiling_power_of_2.hpp
+15 ?0 common/include/common_defs.hpp
+6 ?1 {theta ? common}/include/conditional_back_inserter.hpp
+70 ?0 common/include/conditional_forward.hpp
+15 ?15 common/include/serde.hpp
+3 ?15 hll/include/HllUtil.hpp
These files have been updated to our snapshot.
http://gerrit.cloudera.org:8080/#/c/16645/6/be/src/exprs/datasketches-test.cc
File be/src/exprs/datasketches-test.cc:
http://gerrit.cloudera.org:8080/#/c/16645/6/be/src/exprs/datasketches-test.cc@114
PS6, Line 114: // Like HLL, the order of the inputs fed to the sketches is
fix here so we get the
> thx! I just wanted to make sure that this result is deterministic and is th
The result is deterministic, comments have been added.
http://gerrit.cloudera.org:8080/#/c/16645/7/be/src/thirdparty/datasketches/README.md
File be/src/thirdparty/datasketches/README.md:
http://gerrit.cloudera.org:8080/#/c/16645/7/be/src/thirdparty/datasketches/README.md@16
PS7, Line 16: https://github.com/apache/incubator-datasketches-cpp/
> Could you provide a link here that points to the particular snapshot that y
Done
--
To view, visit http://gerrit.cloudera.org:8080/16645
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c
Gerrit-Change-Number: 16645
Gerrit-PatchSet: 8
Gerrit-Owner: Fucun Chu
Gerrit-Reviewer: Fucun Chu
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 05 Nov 2020 05:29:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Hello Gabor Kaszab, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16645 to look at the new patch set (#8). Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. IMPALA-10279: Import CPC functionality from DataSketches This patch imports the functionality needed for CPC approximate algorithm from Apache DataSketches. First, I updated our existing snapshot of DataSketches to the following commit:2b84e213067b681b696ec883d245ddf911790ff2 "Merge pull request #172 from chufucun/const_cast" This affects files originated from cpc/ directories of the DataSketches repo. Then I copied all the files needed for CPC into our snapshot directory. Browse the source files here: https://github.com/apache/incubator-datasketches-cpp Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c --- M be/src/exprs/CMakeLists.txt M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/binomial_bounds.hpp A be/src/thirdparty/datasketches/ceiling_power_of_2.hpp M be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/compression_data.hpp A be/src/thirdparty/datasketches/conditional_back_inserter.hpp A be/src/thirdparty/datasketches/conditional_forward.hpp A be/src/thirdparty/datasketches/cpc_common.hpp A be/src/thirdparty/datasketches/cpc_compressor.hpp A be/src/thirdparty/datasketches/cpc_compressor_impl.hpp A be/src/thirdparty/datasketches/cpc_confidence.hpp A be/src/thirdparty/datasketches/cpc_sketch.hpp A be/src/thirdparty/datasketches/cpc_sketch_impl.hpp A be/src/thirdparty/datasketches/cpc_union.hpp A be/src/thirdparty/datasketches/cpc_union_impl.hpp A be/src/thirdparty/datasketches/cpc_util.hpp A be/src/thirdparty/datasketches/icon_estimator.hpp A be/src/thirdparty/datasketches/kxp_byte_lookup.hpp M be/src/thirdparty/datasketches/serde.hpp A be/src/thirdparty/datasketches/u32_table.hpp A be/src/thirdparty/datasketches/u32_table_impl.hpp 24 files changed, 10,274 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/16645/8 -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 8 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 5: (3 comments) Looks good to me! http://gerrit.cloudera.org:8080/#/c/16599/5/fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java File fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java: http://gerrit.cloudera.org:8080/#/c/16599/5/fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java@59 PS5, Line 59: target nit: 'specify' sounds better. http://gerrit.cloudera.org:8080/#/c/16599/5/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/16599/5/fe/src/main/java/org/apache/impala/service/Frontend.java@1098 PS5, Line 1098: Set ancestorIds = Sets.newHashSet(SnapshotUtil.currentAncestors(table)) It looks like this call can be placed before the start of the FOR loop. http://gerrit.cloudera.org:8080/#/c/16599/5/testdata/data/README File testdata/data/README: http://gerrit.cloudera.org:8080/#/c/16599/5/testdata/data/README@536 PS5, Line 536: Created the table and inserted the data by Impala. nit: Is it in contradiction to the line at 528: "impala cannot insert iceberg table now"? -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 04 Nov 2020 20:37:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7615/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 12 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 20:20:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 133 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/12 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 12 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7614/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 19:19:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. IMPALA-10305: Sync Kudu's FIPS compliant changes kudu-3210 added FIPS compliant changes. This patch ports the following patches for kudu-3210 into Impala source tree: http://gerrit.cloudera.org:8080/16631/ http://gerrit.cloudera.org:8080/16657/ http://gerrit.cloudera.org:8080/16658/ http://gerrit.cloudera.org:8080/16659/ Testing: - Passed exhausive tests. Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 --- M be/src/kudu/security/CMakeLists.txt M be/src/kudu/security/ca/cert_management-test.cc M be/src/kudu/security/crypto.cc M be/src/kudu/security/openssl_util.cc M be/src/kudu/security/openssl_util.h M be/src/kudu/security/tls_context.cc M be/src/kudu/security/tls_handshake.cc M be/src/kudu/security/token-test.cc M be/src/kudu/util/flags.cc M be/src/kudu/util/flags.h M be/src/kudu/util/test_util.cc M be/src/kudu/util/test_util.h 12 files changed, 187 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/16684/5 -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou
