[kudu-CR](gh-pages) [site] update site for Kudu 1.12.0 release
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15922 ) Change subject: [site] update site for Kudu 1.12.0 release .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I5ef3550eec4d98df889effef4c873aad0046b17b Gerrit-Change-Number: 15922 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Hao Hao Gerrit-Comment-Date: Fri, 15 May 2020 05:26:22 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [site] update site for Kudu 1.12.0 release
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15922 ) Change subject: [site] update site for Kudu 1.12.0 release .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I5ef3550eec4d98df889effef4c873aad0046b17b Gerrit-Change-Number: 15922 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Hao Hao Gerrit-Comment-Date: Fri, 15 May 2020 05:26:15 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [site] update site for Kudu 1.12.0 release
Hao Hao has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15922 ) Change subject: [site] update site for Kudu 1.12.0 release .. [site] update site for Kudu 1.12.0 release Change-Id: I5ef3550eec4d98df889effef4c873aad0046b17b Reviewed-on: http://gerrit.cloudera.org:8080/15922 Reviewed-by: Hao Hao Tested-by: Hao Hao --- A releases/1.12.0/apidocs/allclasses-frame.html A releases/1.12.0/apidocs/allclasses-noframe.html A releases/1.12.0/apidocs/constant-values.html A releases/1.12.0/apidocs/deprecated-list.html A releases/1.12.0/apidocs/help-doc.html A releases/1.12.0/apidocs/index-all.html A releases/1.12.0/apidocs/index.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.ColumnSchemaBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.CompressionAlgorithm.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.Encoding.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnTypeAttributes.ColumnTypeAttributesBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnTypeAttributes.html A releases/1.12.0/apidocs/org/apache/kudu/Schema.html A releases/1.12.0/apidocs/org/apache/kudu/Type.html A releases/1.12.0/apidocs/org/apache/kudu/client/AbstractKuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AlterTableOptions.html A releases/1.12.0/apidocs/org/apache/kudu/client/AlterTableResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduClient.AsyncKuduClientBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduClient.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.AsyncKuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.ReadMode.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduSession.html A releases/1.12.0/apidocs/org/apache/kudu/client/ColumnRangePredicate.html A releases/1.12.0/apidocs/org/apache/kudu/client/CreateTableOptions.html A releases/1.12.0/apidocs/org/apache/kudu/client/Delete.html A releases/1.12.0/apidocs/org/apache/kudu/client/DeleteTableResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/ExternalConsistencyMode.html A releases/1.12.0/apidocs/org/apache/kudu/client/HasFailedRpcException.html A releases/1.12.0/apidocs/org/apache/kudu/client/Insert.html A releases/1.12.0/apidocs/org/apache/kudu/client/InsertIgnore.html A releases/1.12.0/apidocs/org/apache/kudu/client/IsAlterTableDoneResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/IsCreateTableDoneResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduClient.KuduClientBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduClient.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduException.OriginalException.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduException.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPartitioner.KuduPartitionerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPartitioner.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPredicate.ComparisonOp.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPredicate.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanToken.KuduScanTokenBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanToken.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanner.KuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanner.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScannerIterator.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduSession.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduTable.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduTableStatistics.html A releases/1.12.0/apidocs/org/apache/kudu/client/ListTablesResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/ListTabletServersResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/LocatedTablet.Replica.html A releases/1.12.0/apidocs/org/apache/kudu/client/LocatedTablet.html A releases/1.12.0/apidocs/org/apache/kudu/client/Operation.html A releases/1.12.0/apidocs/org/apache/kudu/client/OperationResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/PartialRow.html A releases/1.12.0/apidocs/org/apache/kudu/client/PleaseThrottleException.html A releases/1.12.0/apidocs/org/apache/kudu/client/RangePartitionBound.html A releases/1.12.0/apidocs/org/apache/kudu/client/ReplicaSelection.html A releases/1.12.0/apidocs/org/apache/kudu/client/ResourceMetrics.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowError.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowErrorsAndOverflowStatus.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowResult.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowResultIterator.htm
[kudu-CR](gh-pages) [site] update site for Kudu 1.12.0 release
Hao Hao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15922 Change subject: [site] update site for Kudu 1.12.0 release .. [site] update site for Kudu 1.12.0 release Change-Id: I5ef3550eec4d98df889effef4c873aad0046b17b --- A releases/1.12.0/apidocs/allclasses-frame.html A releases/1.12.0/apidocs/allclasses-noframe.html A releases/1.12.0/apidocs/constant-values.html A releases/1.12.0/apidocs/deprecated-list.html A releases/1.12.0/apidocs/help-doc.html A releases/1.12.0/apidocs/index-all.html A releases/1.12.0/apidocs/index.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.ColumnSchemaBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.CompressionAlgorithm.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.Encoding.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnSchema.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnTypeAttributes.ColumnTypeAttributesBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/ColumnTypeAttributes.html A releases/1.12.0/apidocs/org/apache/kudu/Schema.html A releases/1.12.0/apidocs/org/apache/kudu/Type.html A releases/1.12.0/apidocs/org/apache/kudu/client/AbstractKuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AlterTableOptions.html A releases/1.12.0/apidocs/org/apache/kudu/client/AlterTableResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduClient.AsyncKuduClientBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduClient.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.AsyncKuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.ReadMode.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.html A releases/1.12.0/apidocs/org/apache/kudu/client/AsyncKuduSession.html A releases/1.12.0/apidocs/org/apache/kudu/client/ColumnRangePredicate.html A releases/1.12.0/apidocs/org/apache/kudu/client/CreateTableOptions.html A releases/1.12.0/apidocs/org/apache/kudu/client/Delete.html A releases/1.12.0/apidocs/org/apache/kudu/client/DeleteTableResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/ExternalConsistencyMode.html A releases/1.12.0/apidocs/org/apache/kudu/client/HasFailedRpcException.html A releases/1.12.0/apidocs/org/apache/kudu/client/Insert.html A releases/1.12.0/apidocs/org/apache/kudu/client/InsertIgnore.html A releases/1.12.0/apidocs/org/apache/kudu/client/IsAlterTableDoneResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/IsCreateTableDoneResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduClient.KuduClientBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduClient.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduException.OriginalException.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduException.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPartitioner.KuduPartitionerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPartitioner.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPredicate.ComparisonOp.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduPredicate.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanToken.KuduScanTokenBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanToken.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanner.KuduScannerBuilder.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScanner.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduScannerIterator.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduSession.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduTable.html A releases/1.12.0/apidocs/org/apache/kudu/client/KuduTableStatistics.html A releases/1.12.0/apidocs/org/apache/kudu/client/ListTablesResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/ListTabletServersResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/LocatedTablet.Replica.html A releases/1.12.0/apidocs/org/apache/kudu/client/LocatedTablet.html A releases/1.12.0/apidocs/org/apache/kudu/client/Operation.html A releases/1.12.0/apidocs/org/apache/kudu/client/OperationResponse.html A releases/1.12.0/apidocs/org/apache/kudu/client/PartialRow.html A releases/1.12.0/apidocs/org/apache/kudu/client/PleaseThrottleException.html A releases/1.12.0/apidocs/org/apache/kudu/client/RangePartitionBound.html A releases/1.12.0/apidocs/org/apache/kudu/client/ReplicaSelection.html A releases/1.12.0/apidocs/org/apache/kudu/client/ResourceMetrics.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowError.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowErrorsAndOverflowStatus.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowResult.html A releases/1.12.0/apidocs/org/apache/kudu/client/RowResultIterator.html A releases/1.12.0/apidocs/org/apache/kudu/client/SessionConfiguration.FlushMode.html A releases/1.1
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: liusheng Gerrit-Comment-Date: Fri, 15 May 2020 04:44:44 + Gerrit-HasComments: No
[kudu-CR] [fs] limit memory could consume compacting low live ratio blocks during startup
wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/15830 ) Change subject: [fs] limit memory could consume compacting low live ratio blocks during startup .. Patch Set 6: (8 comments) Thx for point out. My original purpose was to limit the memory. http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG@7 PS4, Line 7: [fs] limit memory could consume compacting low live ratio blocks during startup : > Also I think this would be a clearer: Change the commit to limit the memory may consume. http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG@10 PS4, Line 10: nd the memory behave is not under control of > nit: I wasn't sure what "low live ratio metadata" referred to. Could you al It refers to all blocks in low ratio blocks. Changed to a clearer title. http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG@11 PS4, Line 11: d_bytes. > nit: "control" with one 'l' Done http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG@16 PS4, Line 16: memory > nit: "exceed" Done http://gerrit.cloudera.org:8080/#/c/15830/4//COMMIT_MSG@30 PS4, Line 30: > nit: "exceeds" Done http://gerrit.cloudera.org:8080/#/c/15830/4/src/kudu/fs/log_block_manager.cc File src/kudu/fs/log_block_manager.cc: http://gerrit.cloudera.org:8080/#/c/15830/4/src/kudu/fs/log_block_manager.cc@119 PS4, Line 119: "startup time"); > nit: can you put this up by log_container_live_metadata_before_compact_rati Done http://gerrit.cloudera.org:8080/#/c/15830/4/src/kudu/fs/log_block_manager.cc@120 PS4, Line 120: n > nit: spell out "number" Done http://gerrit.cloudera.org:8080/#/c/15830/4/src/kudu/fs/log_block_manager.cc@2668 PS4, Line 2668: }); > Looking at the definition of BlockRecordPB, it seems to have the following Done -- To view, visit http://gerrit.cloudera.org:8080/15830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifad1d6de4e61a1ddcfb35743abd71b57d6418896 Gerrit-Change-Number: 15830 Gerrit-PatchSet: 6 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Fri, 15 May 2020 03:23:51 + Gerrit-HasComments: Yes
[kudu-CR] [fs] limit memory could consume compacting low live ratio blocks during startup
Hello Kudu Jenkins, Andrew Wong, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15830 to look at the new patch set (#6). Change subject: [fs] limit memory could consume compacting low live ratio blocks during startup .. [fs] limit memory could consume compacting low live ratio blocks during startup During tserver startup, it loads all blocks of low live ratio metadata into memory for compacting, and the memory behave is not under control of FLAGS_memory_limit_hard_bytes. To those tservers which didn't compact well due to starvation of compaction, the metadata files inflated, size of all metadata files increases and may result in OOM in next startup. Sometimes the memory consumption for startup may exceed physical memory. E.g. In a node with 32G physical memory, we found about 90GiB metatatada files, to load all the low live ratio blocks into memory may take over 50g memory. It's far exceeds the memory limit of kudu and even physical limit. Once it failed due to some reason or restart it manually, it may never make it startup. Here introduce FlAGS_max_limit_hard_bytes_low_live_ratio_blocks_compacting to limit the memory may take by compacting low live blocks. Change-Id: Ifad1d6de4e61a1ddcfb35743abd71b57d6418896 --- M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 2 files changed, 16 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/15830/6 -- To view, visit http://gerrit.cloudera.org:8080/15830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifad1d6de4e61a1ddcfb35743abd71b57d6418896 Gerrit-Change-Number: 15830 Gerrit-PatchSet: 6 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [fs] limit memory could consumpt by low live ratio blocks compacting during startup
Hello Kudu Jenkins, Andrew Wong, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15830 to look at the new patch set (#5). Change subject: [fs] limit memory could consumpt by low live ratio blocks compacting during startup .. [fs] limit memory could consumpt by low live ratio blocks compacting during startup During tserver startup, it loads all blocks of low live ratio metadata into memory for compacting, and the memory behave is not under control of FLAGS_memory_limit_hard_bytes. To those tservers which didn't compact well due to starvation of compaction, the metadata files inflated, size of all metadata files increases and may result in OOM in next startup. Sometimes the memory consumption for startup may exceed physical memory. E.g. In a node with 32G physical memory, we found about 90GiB metatatada files, to load all the low live ratio blocks into memory may take over 50g memory. It's far exceeds the memory limit of kudu and even physical limit. Once it failed due to some reason or restart it manually, it may never make it startup. Here introduce FlAGS_max_limit_hard_bytes_low_live_ratio_blocks_compacting to limit the memory may take by compacting low live blocks. Change-Id: Ifad1d6de4e61a1ddcfb35743abd71b57d6418896 --- M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 2 files changed, 16 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/15830/5 -- To view, visit http://gerrit.cloudera.org:8080/15830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifad1d6de4e61a1ddcfb35743abd71b57d6418896 Gerrit-Change-Number: 15830 Gerrit-PatchSet: 5 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] KUDU-3112. Fix WaitForBind method for checking service status
liusheng has abandoned this change. ( http://gerrit.cloudera.org:8080/15893 ) Change subject: KUDU-3112. Fix WaitForBind method for checking service status .. Abandoned thanks, I have tested the https://gerrit.cloudera.org/#/c/15917/ on ARM mechine, it works ok. abandon this. -- To view, visit http://gerrit.cloudera.org:8080/15893 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ib79297e0eb59cc96a91c6e301f6a70ba123f4644 Gerrit-Change-Number: 15893 Gerrit-PatchSet: 4 Gerrit-Owner: liusheng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: liusheng
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 2: Code-Review+1 Thanks @Alexey Serbin, I have tested this patch in my ARM mechine, it works OK. Let this pick this up and abandon the https://gerrit.cloudera.org/#/c/15893/ LGTM +1 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: liusheng Gerrit-Comment-Date: Fri, 15 May 2020 02:51:47 + Gerrit-HasComments: No
[kudu-CR] KUDU-3112. Fix WaitForBind method for checking service status
liusheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/15893 ) Change subject: KUDU-3112. Fix WaitForBind method for checking service status .. Patch Set 4: > > Build Successful > > > > http://jenkins.kudu.apache.org/job/kudu-gerrit/21643/ : SUCCESS > > Thank you for addressing these comments. > > BTW, would the tests pass on aarch64 if you apply the following > patch: https://gerrit.cloudera.org/#/c/15917/ ? > > If that works, we could keep the original semantics of > WaitForBind() and this patch wouldn't be needed. Sure, let me try, thank you :) -- To view, visit http://gerrit.cloudera.org:8080/15893 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib79297e0eb59cc96a91c6e301f6a70ba123f4644 Gerrit-Change-Number: 15893 Gerrit-PatchSet: 4 Gerrit-Owner: liusheng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: liusheng Gerrit-Comment-Date: Fri, 15 May 2020 01:43:30 + Gerrit-HasComments: No
[kudu-CR] rebalancer: throttle logging when building cluster info
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/15920 )
Change subject: rebalancer: throttle logging when building cluster info
..
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15920/1/src/kudu/rebalance/rebalancer.cc
File src/kudu/rebalance/rebalancer.cc:
http://gerrit.cloudera.org:8080/#/c/15920/1/src/kudu/rebalance/rebalancer.cc@313
PS1, Line 313: KLOG_EVERY_N_SECS(INFO, 10) <<
: Substitute("skipping tablet server $0 ($1) because of
its "
: "non-HEALTHY status ($2)",
: s.uuid, s.address,
: ServerHealthToString(s.health));
I guess it's OK to do so if we are not interested to get information on all
tablet servers which are not in good health.
--
To view, visit http://gerrit.cloudera.org:8080/15920
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I93007b50213244c14e47fc6e913415fad7698057
Gerrit-Change-Number: 15920
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 15 May 2020 00:28:48 +
Gerrit-HasComments: Yes
[kudu-CR] rebalancer: throttle logging when building cluster info
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15920 ) Change subject: rebalancer: throttle logging when building cluster info .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/15920/1/src/kudu/rebalance/rebalancer.cc File src/kudu/rebalance/rebalancer.cc: http://gerrit.cloudera.org:8080/#/c/15920/1/src/kudu/rebalance/rebalancer.cc@317 PS1, Line 317: ; Why no THROTTLE_MSG here like below? http://gerrit.cloudera.org:8080/#/c/15920/1/src/kudu/rebalance/rebalancer.cc@327 PS1, Line 327: 10 Unless there is user intervention to increase number of replica tablets, the number wouldn't increase, right? If so, this can be much higher. -- To view, visit http://gerrit.cloudera.org:8080/15920 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93007b50213244c14e47fc6e913415fad7698057 Gerrit-Change-Number: 15920 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 15 May 2020 00:25:22 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.12.x) [C++ client] doxygenized C++ client API in columnar scan batch.h
Hao Hao has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15919 ) Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. [C++ client] doxygenized C++ client API in columnar_scan_batch.h This patch documents the missing parameters for APIs in columnar_scan_batch.h to satisfy the warnings from doxygen. Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Reviewed-on: http://gerrit.cloudera.org:8080/15918 Reviewed-by: Grant Henke Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin (cherry picked from commit 7d1f3fbb40984b6917e6e76f252bf6a0924fd70d) Reviewed-on: http://gerrit.cloudera.org:8080/15919 --- M src/kudu/client/columnar_scan_batch.h 1 file changed, 28 insertions(+), 13 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.12.x Gerrit-MessageType: merged Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15919 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] rebalancer: throttle logging when building cluster info
Hello Alexey Serbin, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/15920 to review the following change. Change subject: rebalancer: throttle logging when building cluster info .. rebalancer: throttle logging when building cluster info I enabled the auto-rebalancer on a cluster with many single-replica tablets and was met with hundreds of messages like the following in very short bursts: I0514 16:24:53.469353 151489 rebalancer.cc:325] tablet 0bc9157245694b57a313d1e032093393 of table 'default.loadgen_auto_24bd8866af594df9b9d2185c77ff0d5c' (0d852b00af434a1980b83e63da3d984a) has single replica, skipping I0514 16:24:53.469357 151489 rebalancer.cc:325] tablet 1f9507a576dd4115b445faac37b9fcf3 of table 'default.loadgen_auto_24bd8866af594df9b9d2185c77ff0d5c' (0d852b00af434a1980b83e63da3d984a) has single replica, skipping I0514 16:24:53.469362 151489 rebalancer.cc:325] tablet 65c794016e3a4e8eb635a3764e3e33a7 of table 'default.loadgen_auto_32c3519a594f42df9a9c5d753db437a9' (49599b5cee6f48308425dafd04519b02) has single replica, skipping I0514 16:24:53.469367 151489 rebalancer.cc:325] tablet 20910b83145f4e7c83b6d8c6885e423b of table 'default.loadgen_auto_32c3519a594f42df9a9c5d753db437a9' (49599b5cee6f48308425dafd04519b02) has single replica, skipping I0514 16:24:53.469373 151489 rebalancer.cc:325] tablet 577024c5dea74edd8f286de9d5b73b41 of table 'default.loadgen_auto_32c3519a594f42df9a9c5d753db437a9' (49599b5cee6f48308425dafd04519b02) has single replica, skipping This throttles such logs to display once every 10 seconds at most. Change-Id: I93007b50213244c14e47fc6e913415fad7698057 --- M src/kudu/rebalance/rebalancer.cc 1 file changed, 11 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/15920/1 -- To view, visit http://gerrit.cloudera.org:8080/15920 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I93007b50213244c14e47fc6e913415fad7698057 Gerrit-Change-Number: 15920 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 2: > Thanks! It'd be great if we could confirm with liusheng that this > unblocks their test runs. Yes, I asked liusheng2...@gmail.com to confirm this patch addresses the issue with mini-ranger-related tests on aarch64. -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 23:17:44 + Gerrit-HasComments: No
[kudu-CR](branch-1.12.x) [C++ client] doxygenized C++ client API in columnar scan batch.h
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15919 ) Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.12.x Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15919 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 23:18:09 + Gerrit-HasComments: No
[kudu-CR](branch-1.12.x) [C++ client] doxygenized C++ client API in columnar scan batch.h
Hao Hao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15919 Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. [C++ client] doxygenized C++ client API in columnar_scan_batch.h This patch documents the missing parameters for APIs in columnar_scan_batch.h to satisfy the warnings from doxygen. Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Reviewed-on: http://gerrit.cloudera.org:8080/15918 Reviewed-by: Grant Henke Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin (cherry picked from commit 7d1f3fbb40984b6917e6e76f252bf6a0924fd70d) --- M src/kudu/client/columnar_scan_batch.h 1 file changed, 28 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/15919/1 -- To view, visit http://gerrit.cloudera.org:8080/15919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.12.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15919 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao
[kudu-CR] [C++ client] doxygenized C++ client API in columnar scan batch.h
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15918 ) Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. [C++ client] doxygenized C++ client API in columnar_scan_batch.h This patch documents the missing parameters for APIs in columnar_scan_batch.h to satisfy the warnings from doxygen. Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Reviewed-on: http://gerrit.cloudera.org:8080/15918 Reviewed-by: Grant Henke Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M src/kudu/client/columnar_scan_batch.h 1 file changed, 28 insertions(+), 13 deletions(-) Approvals: Grant Henke: Looks good to me, approved Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/15918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15918 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [C++ client] doxygenized C++ client API in columnar scan batch.h
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15918 ) Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15918 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 14 May 2020 23:15:17 + Gerrit-HasComments: No
[kudu-CR] [C++ client] doxygenized C++ client API in columnar scan batch.h
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15918 ) Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15918 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 14 May 2020 22:49:31 + Gerrit-HasComments: No
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15913 ) Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. Patch Set 4: Code-Review+1 (1 comment) It's a good optimization point! http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc File src/kudu/client/predicate-test.cc: http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1449 PS3, Line 1449: // SKIP_IF_SLOW_NOT_ALLOWED(); remove ? -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 4 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Thu, 14 May 2020 22:36:38 + Gerrit-HasComments: Yes
[kudu-CR] [C++ client] doxygenized C++ client API in columnar scan batch.h
Hao Hao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15918 Change subject: [C++ client] doxygenized C++ client API in columnar_scan_batch.h .. [C++ client] doxygenized C++ client API in columnar_scan_batch.h This patch documents the missing parameters for APIs in columnar_scan_batch.h to satisfy the warnings from doxygen. Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 --- M src/kudu/client/columnar_scan_batch.h 1 file changed, 28 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/15918/1 -- To view, visit http://gerrit.cloudera.org:8080/15918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifb8b927bf55ff181b9923411c35729eed8bafa08 Gerrit-Change-Number: 15918 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15913 ) Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/15913/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15913/3//COMMIT_MSG@19 PS3, Line 19: 100M nit: 1M here too http://gerrit.cloudera.org:8080/#/c/15913/3//COMMIT_MSG@25 PS3, Line 25: Without perf change: nit: would be nice to describe what these are and why the results look the way they do, e.g. "The added test runs two scans against a single dataset, both of which have a range predicate and a bloom filter predicate. The first scan's range predicate yields no rows, while the second scan's range predicate yields all rows. With this patch, we see a notable decrease in the time taken to evaluate the first scan, since the range predicate is evaluated first." -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 4 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Thu, 14 May 2020 22:28:32 + Gerrit-HasComments: Yes
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/15913 )
Change subject: [perf] Check range predicate first while evaluating Bloom
filter predicate
..
Patch Set 4:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/15913/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15913/1//COMMIT_MSG@18
PS1, Line 18: Tests:
> Disregarding the data type of the column, I think this is the best case sce
I see, yeah I was more curious about perf in the case of string/binary.
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc
File src/kudu/client/predicate-test.cc:
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1287
PS3, Line 1287: inclusive_values_
nit: same here, with "included".
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1289
PS3, Line 1289: exclusive_values_
nit: I think "excluded" is a more appropriate word. "exclusive" doesn't make
sense IMO.
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1293
PS3, Line 1293: // Initialize table with "num_all_values" values.
: // "num_inclusive_values" is the number of values from the
table that'll be inserted in
: // BloomFilter and searched against all values in the table.
: // "num_exclusive_values" is similarly the number of values
that are not present in the table.
nit: I found myself reading through other tests to better understand what this
was doing and why. Maybe reword a bit to add a more complete description:
"Initialize the members with the values that will be inserted to the table and
bloom filters. The table will have 'num_all_values' unique values, and we'll
generate two bloom bloom filters: one with 'num_inclusive_values' values from
the table, and one with 'num_exclusive_values' values that aren't present in
the table."
Also would it make sense to call InsertAllValuesInTable() here?
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1305
PS3, Line 1305: CreateRandomUniqueIntegers
nit: not related to this patch, but consider renaming this to
GetRandomAvoidingValues() or somesuch. Without going back to the
implementation, this just seems like it's creating an unrestricted set of ints.
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1355
PS3, Line 1355: Counting rows when no rows expected
nit: might be easier on others who may look at this test output to have a more
implementation-specific description, e.g.
"counting rows with a range predicate less than the min value"
and
"counting rows with a range predicate selecting all between the min and max
values"
Also why is the CURRENT_TEST_NAME() useful? Can't we figure that out from the
test output? Or if you're looking to benchmark a specific test, why not use
--gtest_filter or something?
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1368
PS3, Line 1368: {i
nit: spacing
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1449
PS3, Line 1449: // SKIP_IF_SLOW_NOT_ALLOWED();
Uncomment?
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1451
PS3, Line 1451: // Following are results for scanning against 100M rows on 48
core
: // Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz with 94GB of
memory.
: //
: // Counting rows when no rows expected: real 0.725s user
0.001s sys 0.000s
: // Counting rows when range predicate doesn't prune: real
0.847s user 0.000s sys 0.000s
: //
: // Counting rows when no rows expected: real 0.664s user
0.000s sys 0.000s
: // Counting rows when range predicate doesn't prune: real
0.794s user 0.001s sys 0.000s
: //
: // Counting rows when no rows expected: real 0.706s user
0.001s sys 0.000s
: // Counting rows when range predicate doesn't prune: real
0.774s user 0.000s sys 0.000s
: //
: // Writing large number of rows sometimes leads to write
timeouts. Hence the test below
: // uses 1M rows instead.
nit: this probably isn't necessary, given it's not particularly useful as a
reader of this code. If people were really interested, it seems more
appropriate to go back to the commit message for this info.
http://gerrit.cloudera.org:8080/#/c/15913/3/src/kudu/client/predicate-test.cc@1471
PS3, Line 1471: auto* inclusive_predicate_clone =
inclusive_predicate->Clone();
nit: Why do we need to clone this? Aren't they used in a const manner anyway?
--
To view, visit http://gerrit.cloudera.org:8080/15913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15913 to look at the new patch set (#4). Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. [perf] Check range predicate first while evaluating Bloom filter predicate Range predicates can be specified along with Bloom filter predicate for the same column. It's cheaper to check against range predicate and exit early if the column value is out of bounds compared to computing hash and then looking up the value in Bloom filter. This case is common when Impala pushes down Bloom filter predicate as it'll likely be accompained by min-max filter (i.e. range predicate) on the same column. Tests: Added a test case that scans against 100M column values. Across iterations observed an improvement of 20-30% when the range predicate check prevents hash computation and Bloom filter lookup. Don't see any noticeable regression for the case where values are within range bounds. Without perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.953s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.767s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.899s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.775s user 0.000s sys 0.001s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.983s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.832s user 0.001s sys 0.000s With perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.725s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.847s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.664s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.794s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.706s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.774s user 0.000s sys 0.000s Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 --- M src/kudu/client/predicate-test.cc M src/kudu/common/column_predicate.h 2 files changed, 79 insertions(+), 43 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/15913/4 -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 4 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [metrics] fix MeanGauge::snapshot()
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15916 ) Change subject: [metrics] fix MeanGauge::snapshot() .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Gerrit-Change-Number: 15916 Gerrit-PatchSet: 2 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 21:50:51 + Gerrit-HasComments: No
[kudu-CR] [metrics] fix MeanGauge::snapshot()
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15916 ) Change subject: [metrics] fix MeanGauge::snapshot() .. [metrics] fix MeanGauge::snapshot() MeanGauge::snapshot() will always return a metric whose total_sum/total_count is ‘0' because there is a bug in the method, that leads to wrong values when MeanGauge metrics merged. Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Reviewed-on: http://gerrit.cloudera.org:8080/15916 Reviewed-by: Andrew Wong Tested-by: Andrew Wong --- M src/kudu/util/metrics-test.cc M src/kudu/util/metrics.cc 2 files changed, 13 insertions(+), 0 deletions(-) Approvals: Andrew Wong: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Gerrit-Change-Number: 15916 Gerrit-PatchSet: 2 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15882 ) Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@78 PS7, Line 78: private[kudu] val numInserts: MapAccumulator[String, Long] = > Hmm, good question. I hadn't seen that issue before, but it makes sense. I We didn't add a flag to disable it in that fix/change. Instead we lowered the overhead to the point it wasn't an issue. But that issue and this enhancement makes me think it would be good to have a flag. It can be a follow on change if you want to file a jira to track doing it seperately. -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 7 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 21:38:25 + Gerrit-HasComments: Yes
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15913 to look at the new patch set (#3). Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. [perf] Check range predicate first while evaluating Bloom filter predicate Range predicates can be specified along with Bloom filter predicate for the same column. It's cheaper to check against range predicate and exit early if the column value is out of bounds compared to computing hash and then looking up the value in Bloom filter. This case is common when Impala pushes down Bloom filter predicate as it'll likely be accompained by min-max filter (i.e. range predicate) on the same column. Tests: Added a test case that scans against 100M column values. Across iterations observed an improvement of 20-30% when the range predicate check prevents hash computation and Bloom filter lookup. Don't see any noticeable regression for the case where values are within range bounds. Without perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.953s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.767s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.899s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.775s user 0.000s sys 0.001s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.983s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.832s user 0.001s sys 0.000s With perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.725s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.847s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.664s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.794s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.706s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.774s user 0.000s sys 0.000s Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 --- M src/kudu/client/predicate-test.cc M src/kudu/common/column_predicate.h 2 files changed, 79 insertions(+), 43 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/15913/3 -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 3 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 21:13:40 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Brian McDevitt has posted comments on this change. ( http://gerrit.cloudera.org:8080/15882 ) Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@78 PS7, Line 78: private[kudu] val numInserts: MapAccumulator[String, Long] = > Should we provide a way to disable these accumulators all together if a use Hmm, good question. I hadn't seen that issue before, but it makes sense. I don't see the flag in KuduContext or the HdrHistogramAccumulator for enabling or disabling it. I see that the RowAccumulator has a flag. -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 7 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 21:10:43 + Gerrit-HasComments: Yes
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15913 to look at the new patch set (#2). Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. [perf] Check range predicate first while evaluating Bloom filter predicate Range predicates can be specified along with Bloom filter predicate for the same column. It's cheaper to check against range predicate and exit early if the column value is out of bounds compared to computing hash and then looking up the value in Bloom filter. This case is common when Impala pushes down Bloom filter predicate as it'll likely be accompained by min-max filter (i.e. range predicate) on the same column. Tests: Added a test case that scans against 100M column values. Across iterations observed an improvement of 20-30% when the range predicate check prevents hash computation and Bloom filter lookup. Don't see any noticeable regression for the case where values are within range bounds. Without perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.953s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.767s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.899s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.775s user 0.000s sys 0.001s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.983s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.832s user 0.001s sys 0.000s With perf change: Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.725s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.847s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.664s user 0.000s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.794s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when no rows expected: real 0.706s user 0.001s sys 0.000s Time spent TestKuduBloomFilterPredicateBenchmark: Counting rows when range predicate doesn't prune: real 0.774s user 0.000s sys 0.000s Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 --- M src/kudu/client/predicate-test.cc M src/kudu/common/column_predicate.h 2 files changed, 79 insertions(+), 43 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/15913/2 -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 2 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Bankim Bhavsar has posted comments on this change. (
http://gerrit.cloudera.org:8080/15913 )
Change subject: [perf] Check range predicate first while evaluating Bloom
filter predicate
..
Patch Set 1:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/15913/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15913/1//COMMIT_MSG@18
PS1, Line 18: Tests:
> nit: is this a best-case scenario? average scenario? worst case? If we tune
Disregarding the data type of the column, I think this is the best case
scenario considering range predicate checks prevents any checks to Bloom filter.
Hashing for larger string/binary values could give better numbers.
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc
File src/kudu/client/predicate-test.cc:
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1311
PS1, Line 1311: num_false_positive_values_ = num_all_values *
kBloomFilterFalsePositiveProb;
> warning: narrowing conversion from 'float' to 'int' [bugprone-narrowing-con
Done
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1350
PS1, Line 1350: auto *
> nit: revert the order?
Done
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1353
PS1, Line 1353: const auto* test_name =
::testing::UnitTest::GetInstance()->current_test_info()->name();
> warning: Value stored to 'test_name' during its initialization is never rea
Using CURRENT_TEST_NAME() macro instead.
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1354
PS1, Line 1354: strings::
> nit: add a `using` for this?
Done
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/common/column_predicate.h
File src/kudu/common/column_predicate.h:
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/common/column_predicate.h@326
PS1, Line 326: return false;
: }
: } else if (upper_ != nullptr) {
: if (DataTypeTraits::Compare(cell,
this->upper_) >= 0) {
: return false;
: }
: } else if (lower_ != nullptr) {
: if (DataTypeTraits::Compare(cell,
this->lower_) < 0) {
: return false;
:
> nit: spacing for 'return false'
Done
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/common/column_predicate.h@322
PS1, Line 322: // Check optional lower and upper bound.
: if (lower_ != nullptr && upper_ != nullptr) {
: if (DataTypeTraits::Compare(cell,
this->upper_) >= 0 ||
: DataTypeTraits::Compare(cell,
this->lower_) < 0) {
: return false;
: }
: } else if (upper_ != nullptr) {
: if (DataTypeTraits::Compare(cell,
this->upper_) >= 0) {
: return false;
: }
: } else if (lower_ != nullptr) {
: if (DataTypeTraits::Compare(cell,
this->lower_) < 0) {
: return false;
: }
: }
> nit: would this work:
Indeed!
http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/common/column_predicate.h@349
PS1, Line 349: return false;
> nit: spacing
Looks like simply importing Google style didn't update the spacing in IDE on
Mac. Fixed now.
--
To view, visit http://gerrit.cloudera.org:8080/15913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3
Gerrit-Change-Number: 15913
Gerrit-PatchSet: 1
Gerrit-Owner: Bankim Bhavsar
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 14 May 2020 21:05:49 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] fix MeanGauge::snapshot()
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15916 ) Change subject: [metrics] fix MeanGauge::snapshot() .. Patch Set 1: Verified+1 Code-Review+2 Nice find! -- To view, visit http://gerrit.cloudera.org:8080/15916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Gerrit-Change-Number: 15916 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:56:17 + Gerrit-HasComments: No
[kudu-CR] [metrics] fix MeanGauge::snapshot()
Andrew Wong has removed a vote on this change. Change subject: [metrics] fix MeanGauge::snapshot() .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/15916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Gerrit-Change-Number: 15916 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3112. Fix WaitForBind method for checking service status
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15893 ) Change subject: KUDU-3112. Fix WaitForBind method for checking service status .. Patch Set 4: > Build Successful > > http://jenkins.kudu.apache.org/job/kudu-gerrit/21643/ : SUCCESS Thank you for addressing these comments. BTW, would the tests pass on aarch64 if you apply the following patch: https://gerrit.cloudera.org/#/c/15917/ ? If that works, we could keep the original semantics of WaitForBind() and this patch wouldn't be needed. -- To view, visit http://gerrit.cloudera.org:8080/15893 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib79297e0eb59cc96a91c6e301f6a70ba123f4644 Gerrit-Change-Number: 15893 Gerrit-PatchSet: 4 Gerrit-Owner: liusheng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: liusheng Gerrit-Comment-Date: Thu, 14 May 2020 20:53:14 + Gerrit-HasComments: No
[kudu-CR] KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15899 ) Change subject: KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread .. Patch Set 2: I uploaded libunwind 1.4.0 and verified thirdparty builds successfully. You can update `LIBUNWIND_VERSION=1.4.0` in vars.sh in this patch now as well. -- To view, visit http://gerrit.cloudera.org:8080/15899 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icc722cd5e8ed4ed668d279f6ec831e4eeb69f955 Gerrit-Change-Number: 15899 Gerrit-PatchSet: 2 Gerrit-Owner: RuiChen Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: RuiChen Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 14 May 2020 20:51:29 + Gerrit-HasComments: No
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 2: Code-Review+1 Thanks! It'd be great if we could confirm with liusheng that this unblocks their test runs. -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:48:44 + Gerrit-HasComments: No
[kudu-CR] [perf] Check range predicate first while evaluating Bloom filter predicate
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15913 ) Change subject: [perf] Check range predicate first while evaluating Bloom filter predicate .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc File src/kudu/client/predicate-test.cc: http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1350 PS1, Line 1350: auto * nit: revert the order? http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1353 PS1, Line 1353: const auto* test_name = ::testing::UnitTest::GetInstance()->current_test_info()->name(); > warning: Value stored to 'test_name' during its initialization is never rea Should we be dereferencing the pointer in the Substitute() call? http://gerrit.cloudera.org:8080/#/c/15913/1/src/kudu/client/predicate-test.cc@1354 PS1, Line 1354: strings:: nit: add a `using` for this? -- To view, visit http://gerrit.cloudera.org:8080/15913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8451d6ddfe1fbdf307b3e9f2cc23a8d06e655ba3 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Thu, 14 May 2020 20:47:31 + Gerrit-HasComments: Yes
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15917/1//COMMIT_MSG Commit Message: PS1: > nit: it'd be nice if you also mentioned the motivation behind this patch. Done -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:35:25 + Gerrit-HasComments: Yes
[kudu-CR] [mini ranger] expect Ranger to bind at wildcard address
Hello Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15917 to look at the new patch set (#2). Change subject: [mini_ranger] expect Ranger to bind at wildcard address .. [mini_ranger] expect Ranger to bind at wildcard address This patch updates the address parameter for WaitForTcpBind() in MiniRanger::StartRanger(), expecting the Ranger process to bind to a wildcard address instead of local loopback address. The motivation for this change is the growing evidence that the actual bound address for Ranger is a wildcard address, not a loopback one. Before this patch, it was necessary to work around this issue at least for aarch64 architecture (see https://gerrit.cloudera.org/#/c/15893/). Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 --- M src/kudu/ranger/mini_ranger.cc 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/15917/2 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [master] basic master scale test scenario
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15865 ) Change subject: [master] basic master scale test scenario .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/15865/2/src/kudu/master/master-scale-test.cc File src/kudu/master/master-scale-test.cc: http://gerrit.cloudera.org:8080/#/c/15865/2/src/kudu/master/master-scale-test.cc@66 PS2, Line 66: colling nit: calling http://gerrit.cloudera.org:8080/#/c/15865/2/src/kudu/master/master-scale-test.cc@86 PS2, Line 86: static const string column_to_update = "col0"; nit: could define "table_$0" up here too http://gerrit.cloudera.org:8080/#/c/15865/2/src/kudu/master/master-scale-test.cc@254 PS2, Line 254: &req_counters[thread_idx], &err_counters[thread_idx] I'm not sure that ++i is threadsafe. Should these be made atomic? Or split into their own vectors? http://gerrit.cloudera.org:8080/#/c/15865/2/src/kudu/master/master-scale-test.cc@280 PS2, Line 280: auto req_total = accumulate(req_counters.begin(), req_counters.end(), 0UL); : ASSERT_LT(0, req_total); : : // Check for errors. : auto err_total = accumulate(err_counters.begin(), err_counters.end(), 0UL); : ASSERT_EQ(0, err_total); nit: maybe do the asserts after displaying the info about the run? -- To view, visit http://gerrit.cloudera.org:8080/15865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb1903d5ac23da883729e3a57e772f815a800cd7 Gerrit-Change-Number: 15865 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:20:13 + Gerrit-HasComments: Yes
[kudu-CR] [mini ranger] expect ranger to bind to wildcard address
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect ranger to bind to wildcard address .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15917/1//COMMIT_MSG Commit Message: PS1: nit: it'd be nice if you also mentioned the motivation behind this patch. -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:15:43 + Gerrit-HasComments: Yes
[kudu-CR] [mini ranger] expect ranger to bind to wildcard address
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15917 ) Change subject: [mini_ranger] expect ranger to bind to wildcard address .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:09:49 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15882 ) Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/15882/7/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@78 PS7, Line 78: private[kudu] val numInserts: MapAccumulator[String, Long] = Should we provide a way to disable these accumulators all together if a user doesn't want them? I am not sure how much overhead they cause, but we did need to reduce overhead in the histogram ones a while back. That might be a good defensive feature flag to have. See https://issues.apache.org/jira/browse/KUDU-3056 for reference on the histogram metric issue. -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 7 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:08:54 + Gerrit-HasComments: Yes
[kudu-CR] [docs] Add docs for configuring maintenance priority for kudu tables
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15915 ) Change subject: [docs] Add docs for configuring maintenance priority for kudu tables .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/15915/1/docs/configuration.adoc File docs/configuration.adoc: http://gerrit.cloudera.org:8080/#/c/15915/1/docs/configuration.adoc@152 PS1, Line 152: | kudu.table.maintenance_priority | integer | 0 | Priority level of a table for maintenance. It could be nice to add more elaborate description of these and when they would be used below this table. That could be a follow on change if you are interested. -- To view, visit http://gerrit.cloudera.org:8080/15915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56c96e1e86600a503fa817d62733aef3c0489915 Gerrit-Change-Number: 15915 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 20:01:34 + Gerrit-HasComments: Yes
[kudu-CR] [docs] Add docs for configuring maintenance priority for kudu tables
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15915 ) Change subject: [docs] Add docs for configuring maintenance priority for kudu tables .. [docs] Add docs for configuring maintenance priority for kudu tables Change-Id: I56c96e1e86600a503fa817d62733aef3c0489915 Reviewed-on: http://gerrit.cloudera.org:8080/15915 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke --- M docs/configuration.adoc 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: Kudu Jenkins: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/15915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I56c96e1e86600a503fa817d62733aef3c0489915 Gerrit-Change-Number: 15915 Gerrit-PatchSet: 2 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [mini ranger] expect ranger to bind to wildcard address
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15917 Change subject: [mini_ranger] expect ranger to bind to wildcard address .. [mini_ranger] expect ranger to bind to wildcard address This patch updates the address parameter for WaitForTcpBind(), expecting the Ranger process to bind to a wildcard address instead of local loopback address. Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 --- M src/kudu/ranger/mini_ranger.cc 1 file changed, 1 insertion(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/15917/1 -- To view, visit http://gerrit.cloudera.org:8080/15917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9fc3b0ac8ffcb74941321588f7110e1dbb61e27 Gerrit-Change-Number: 15917 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] [tools] add --ignore nonexistent for 'local replica delete'
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15911 ) Change subject: [tools] add --ignore_nonexistent for 'local_replica delete' .. [tools] add --ignore_nonexistent for 'local_replica delete' Added --ignore_nonexistent flag for the 'local_replica delete' tool. The motivation for this change is to make the real-world scripting scenarios easier if trying to clean up tablet servers of particular tablet replicas. Also added a test to verify the newly introduced functionality. Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 Reviewed-on: http://gerrit.cloudera.org:8080/15911 Reviewed-by: Andrew Wong Tested-by: Kudu Jenkins --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc 2 files changed, 39 insertions(+), 5 deletions(-) Approvals: Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 Gerrit-Change-Number: 15911 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tools] multiple tablet ids in 'local replica delete'
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15909 ) Change subject: [tools] multiple tablet ids in 'local_replica delete' .. [tools] multiple tablet ids in 'local_replica delete' This patch updates the 'local_replica delete' tool to allow for multiple tablet identifiers to be specified and processed at once. The rationale behind this change is that opening tablet server's metadata takes significant time, and it's possible to reduce the overall latency if removing multiple tablet replicas at once after the metadata has already been opened. A new test to verify the new functionality has been added as well. Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1 Reviewed-on: http://gerrit.cloudera.org:8080/15909 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_common.cc M src/kudu/tools/tool_action_common.h M src/kudu/tools/tool_action_local_replica.cc 4 files changed, 94 insertions(+), 17 deletions(-) Approvals: Kudu Jenkins: Verified Andrew Wong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/15909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1 Gerrit-Change-Number: 15909 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tools] multiple tablet ids in 'local replica delete'
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15909 ) Change subject: [tools] multiple tablet ids in 'local_replica delete' .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1 Gerrit-Change-Number: 15909 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 19:20:16 + Gerrit-HasComments: No
[kudu-CR] [tools] add --ignore nonexistent for 'local replica delete'
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15911 ) Change subject: [tools] add --ignore_nonexistent for 'local_replica delete' .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 Gerrit-Change-Number: 15911 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 19:18:14 + Gerrit-HasComments: No
[kudu-CR] [tools] add --ignore nonexistent for 'local replica delete'
Hello Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15911 to look at the new patch set (#3). Change subject: [tools] add --ignore_nonexistent for 'local_replica delete' .. [tools] add --ignore_nonexistent for 'local_replica delete' Added --ignore_nonexistent flag for the 'local_replica delete' tool. The motivation for this change is to make the real-world scripting scenarios easier if trying to clean up tablet servers of particular tablet replicas. Also added a test to verify the newly introduced functionality. Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc 2 files changed, 39 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/15911/3 -- To view, visit http://gerrit.cloudera.org:8080/15911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 Gerrit-Change-Number: 15911 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tools] add --ignore nonexistent for 'local replica delete'
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15911 ) Change subject: [tools] add --ignore_nonexistent for 'local_replica delete' .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ecd00dea7f19747566b11e3e2a545c97d5f8194 Gerrit-Change-Number: 15911 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 19:14:48 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Brian McDevitt has posted comments on this change. ( http://gerrit.cloudera.org:8080/15882 ) Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. Patch Set 7: Failure appears to be flaky test via ASAN. -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 7 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 19:08:52 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Brian McDevitt has posted comments on this change. ( http://gerrit.cloudera.org:8080/15882 ) Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. Patch Set 6: (2 comments) New patch set. Having a hard time getting a clean build test due to some MacOS crashes during the unit tests. http://gerrit.cloudera.org:8080/#/c/15882/6/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/15882/6/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@92 PS6, Line 92: , tableName: String = "*" > Based on the current usage, why do we need a default at all? Aren't all ope That's true, I was thinking someone might extend this class and now their usage would break. I think you're right though, and I'll remove the default value. http://gerrit.cloudera.org:8080/#/c/15882/6/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala File java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala: PS6: > Could you also add a test that writes to multiple tables and confirms that Yes, thanks for the suggestion. -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 6 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 14 May 2020 18:28:07 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3116] Enhance KuduContext row operation metrics
Hello Kudu Jenkins, Andrew Wong, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15882 to look at the new patch set (#7). Change subject: [KUDU-3116] Enhance KuduContext row operation metrics .. [KUDU-3116] Enhance KuduContext row operation metrics Adds the ability to track operation counts per table. Introduces the MapAccumulator to track these metrics in a single accumulator per operation type. Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala A java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/MapAccumulator.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 3 files changed, 157 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/82/15882/7 -- To view, visit http://gerrit.cloudera.org:8080/15882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie66dab95041310c27ef62dacccbcc0977a84857e Gerrit-Change-Number: 15882 Gerrit-PatchSet: 7 Gerrit-Owner: Brian McDevitt Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Brian McDevitt Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tools] multiple tablet ids in 'local replica delete'
Hello Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15909 to look at the new patch set (#2). Change subject: [tools] multiple tablet ids in 'local_replica delete' .. [tools] multiple tablet ids in 'local_replica delete' This patch updates the 'local_replica delete' tool to allow for multiple tablet identifiers to be specified and processed at once. The rationale behind this change is that opening tablet server's metadata takes significant time, and it's possible to reduce the overall latency if removing multiple tablet replicas at once after the metadata has already been opened. A new test to verify the new functionality has been added as well. Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1 --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_common.cc M src/kudu/tools/tool_action_common.h M src/kudu/tools/tool_action_local_replica.cc 4 files changed, 94 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/15909/2 -- To view, visit http://gerrit.cloudera.org:8080/15909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1 Gerrit-Change-Number: 15909 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tools] multiple tablet ids in 'local replica delete'
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/15909 )
Change subject: [tools] multiple tablet ids in 'local_replica delete'
..
Patch Set 1:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/15909/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15909/1//COMMIT_MSG@10
PS1, Line 10: specified and processed at once
> Good points.
Good idea. The tool is useful as it is already, so I think we can add the
improvement of reading identifiers from a file in a separate changelist.
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/kudu-tool-test.cc@2771
PS1, Line 2771: const
> not needed with constexpr
Done
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/kudu-tool-test.cc@2782
PS1, Line 2782:
ts->server()->tablet_manager()->GetTabletReplicas(&tablet_replicas);
> You probably need to clear this list before shutting down.
Indeed: that was not obvious and gdb on my linux build machine failed on me
while I was trying to debug the CHECK() issue in the ThreadPool. Thank you!
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/tool_action_common.cc
File src/kudu/tools/tool_action_common.cc:
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/tool_action_common.cc@199
PS1, Line 199: List
> lower case list
Done
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/tool_action_local_replica.cc@411
PS1, Line 411: vector tablet_ids = strings::Split(tablet_ids_str, ",",
strings::SkipEmpty());
: if (tablet_ids.empty()) {
: return Status::InvalidArgument("no tablet identifiers
provided");
: }
> nit: Split() is templatized and can return a set. Why not use that? Is orde
The idea was to inform on duplicates, if any. If putting the elements into a
set up from, it's hard to tell whether there were duplicates or not.
http://gerrit.cloudera.org:8080/#/c/15909/1/src/kudu/tools/tool_action_local_replica.cc@421
PS1, Line 421: LOG(INFO) << "removed duplicate tablet identifiers";
> nit: might it be worth displaying how many unique tablet IDs there are? Or
Done
--
To view, visit http://gerrit.cloudera.org:8080/15909
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If0e509d1775be2a728e4e3b10c724c1f15a96ec1
Gerrit-Change-Number: 15909
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 14 May 2020 15:17:20 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] fix MeanGauge::snapshot()
Yifan Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15916 Change subject: [metrics] fix MeanGauge::snapshot() .. [metrics] fix MeanGauge::snapshot() MeanGauge::snapshot() will always return a metric whose total_sum/total_count is ‘0' because there is a bug in the method, that leads to wrong values when MeanGauge metrics merged. Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 --- M src/kudu/util/metrics-test.cc M src/kudu/util/metrics.cc 2 files changed, 13 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/15916/1 -- To view, visit http://gerrit.cloudera.org:8080/15916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I15f5e0705759a4bdd513089885aaaea4f5332ce1 Gerrit-Change-Number: 15916 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang
[kudu-CR] KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread
RuiChen has posted comments on this change. ( http://gerrit.cloudera.org:8080/15899 ) Change subject: KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread .. Patch Set 2: https://github.com/apache/kudu/blob/master/thirdparty/vars.sh#L31 -- To view, visit http://gerrit.cloudera.org:8080/15899 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icc722cd5e8ed4ed668d279f6ec831e4eeb69f955 Gerrit-Change-Number: 15899 Gerrit-PatchSet: 2 Gerrit-Owner: RuiChen Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: RuiChen Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 14 May 2020 09:37:19 + Gerrit-HasComments: No
[kudu-CR] KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread
RuiChen has posted comments on this change. ( http://gerrit.cloudera.org:8080/15899 ) Change subject: KUDU-3096: Fix core dump when thread race between GetThreadStack and SuperviseThread .. Patch Set 2: > > Patch Set 1: > > > > Todd, thank you for replying to share these information, looks > like two things impact the part of code: 1.libunwind call malloc() > in thread stack trace collection, I find upstream have fixed it in > libunwind 1.4.0 https://github.com/libunwind/libunwind/pull/72 ; > 2.save waiting time for spawning thread. So I update the patch to > address #2, set tid after strings::Substitute that cause coredump > in patchset 2, I think we maybe update libunwind in the future. > > Why not update libunwind now? My point above is that, even if > you've fixed this particular core dump, you still have a potential > deadlock bug because of the call to malloc in libunwind. The libunwind source code package that Kudu depend on is in AWS S3, and I don't know how to push the 1.4.x release into it, so I commit the patch as workaround. Could you help to add a 1.4.x libunwind into S3? then I will update patch to apply it, thanks -- To view, visit http://gerrit.cloudera.org:8080/15899 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icc722cd5e8ed4ed668d279f6ec831e4eeb69f955 Gerrit-Change-Number: 15899 Gerrit-PatchSet: 2 Gerrit-Owner: RuiChen Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: RuiChen Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 14 May 2020 09:37:05 + Gerrit-HasComments: No
[kudu-CR] [docs] Add docs for configuring maintenance priority for kudu tables
Yifan Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15915 Change subject: [docs] Add docs for configuring maintenance priority for kudu tables .. [docs] Add docs for configuring maintenance priority for kudu tables Change-Id: I56c96e1e86600a503fa817d62733aef3c0489915 --- M docs/configuration.adoc 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/15915/1 -- To view, visit http://gerrit.cloudera.org:8080/15915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I56c96e1e86600a503fa817d62733aef3c0489915 Gerrit-Change-Number: 15915 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang
