[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 06:30:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. IMPALA-10519: Allow setting of num_reactors for KuduClient The KuduClient allows setting a number of reactor threads, which are used for sending rpcs to Kudu such as for scans. The defaults is 4 threads, which may be too few for many use cases, since there is a single KuduClient per impalad. This patch adds a flag --kudu_client_num_reactor_threads which allows users to set the number of threads. The default for the flag is 4, which reflects the current default within Kudu, to avoid possible regressions. Some follow up work could be to do performance experiments and set the default empirically. Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Reviewed-on: http://gerrit.cloudera.org:8080/17086 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/kudu-util.cc 1 file changed, 10 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8167/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 00:46:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 00:37:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6901/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 00:37:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 00:35:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/17086/2/be/src/exec/kudu-util.cc File be/src/exec/kudu-util.cc: http://gerrit.cloudera.org:8080/#/c/17086/2/be/src/exec/kudu-util.cc@48 PS2, Line 48: 4, > nit: maybe use the default 4 to be consistent with current setting, if we a Done -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 19 Feb 2021 00:27:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Hello Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17086 to look at the new patch set (#3). Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. IMPALA-10519: Allow setting of num_reactors for KuduClient The KuduClient allows setting a number of reactor threads, which are used for sending rpcs to Kudu such as for scans. The defaults is 4 threads, which may be too few for many use cases, since there is a single KuduClient per impalad. This patch adds a flag --kudu_client_num_reactor_threads which allows users to set the number of threads. The default for the flag is 4, which reflects the current default within Kudu, to avoid possible regressions. Some follow up work could be to do performance experiments and set the default empirically. Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff --- M be/src/exec/kudu-util.cc 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/17086/3 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 2: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/17086/2/be/src/exec/kudu-util.cc File be/src/exec/kudu-util.cc: http://gerrit.cloudera.org:8080/#/c/17086/2/be/src/exec/kudu-util.cc@48 PS2, Line 48: 16 nit: maybe use the default 4 to be consistent with current setting, if we are not sure what a good default will be (as it may vary depends on the workload.) -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 19 Feb 2021 00:21:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8164/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 18 Feb 2021 21:35:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17086 ) Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8163/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 18 Feb 2021 21:28:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Hello Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17086 to look at the new patch set (#2). Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. IMPALA-10519: Allow setting of num_reactors for KuduClient The KuduClient allows setting a number of reactor threads, which are used for sending rpcs to Kudu such as for scans. The defaults is 4 threads, which may be too few for many use cases, since there is a single KuduClient per impalad. This patch adds a flag --kudu_client_num_reactor_threads which allows users to set the number of threads. The default for the flag is 16, which is consistent with the number of threads used in other IO scenarios such as --num_s3_io_threads and --num_ozone_io_threads. Some follow up work could be to do performance experiments and set the default empirically. Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff --- M be/src/exec/kudu-util.cc 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/17086/2 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10519: Allow setting of num reactors for KuduClient
Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17086 Change subject: IMPALA-10519: Allow setting of num_reactors for KuduClient .. IMPALA-10519: Allow setting of num_reactors for KuduClient The KuduClient allows setting a number of reactor threads, which are used for sending rpcs to Kudu such as for scans. The defaults is 4 threads, which may be too few for many use cases, since there is a single KuduClient per impalad. This patch adds a flag --kudu_client_num_reactor_threads which allows users to set the number of threads. The default for the flag is 16, which is consistent with the number of threads used in other IO scenarios such as --num_s3_io_threads and --num_ozone_io_threads. Some follow up work could be to do performance experiments and set the default empirically. Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff --- M be/src/exec/kudu-util.cc 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/17086/1 -- To view, visit http://gerrit.cloudera.org:8080/17086 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If2ccb9659b9223c9a5de2416b946e6313a3239ff Gerrit-Change-Number: 17086 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall
