[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 7 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 24 Jul 2018 02:36:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. IMPALA-7212: Removes --use_krpc flag and remove old DataStream services This change removes the flag --use_krpc which allows users to fall back to using Thrift based implementation of DataStream services. This flag was originally added during development of IMPALA-2567. It has served its purpose. As we port more ImpalaInternalServices to use KRPC, it's becoming increasingly burdensome to maintain parallel implementation of the RPC handlers. Therefore, going forward, KRPC is always enabled. This change removes the Thrift based implemenation of DataStreamServices and also simplifies some of the tests which were skipped when KRPC is disabled. Testing done: core debug build. Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Reviewed-on: http://gerrit.cloudera.org:8080/10835 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/common/global-flags.cc M be/src/exec/data-sink.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/rpc/authentication.cc M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/thrift-server-test.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/backend-client.h D be/src/runtime/data-stream-mgr-base.h D be/src/runtime/data-stream-mgr.h D be/src/runtime/data-stream-recvr-base.h D be/src/runtime/data-stream-recvr.cc D be/src/runtime/data-stream-recvr.h D be/src/runtime/data-stream-sender.cc D be/src/runtime/data-stream-sender.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/scheduling/scheduler.cc M be/src/service/data-stream-service.cc M be/src/service/impala-internal-service.cc M be/src/service/impala-internal-service.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M bin/run-all-tests.sh M bin/start-impala-cluster.py M common/thrift/ImpalaInternalService.thrift M tests/common/custom_cluster_test_suite.py M tests/common/skip.py D tests/common/test_skip.py M tests/conftest.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_krpc_metrics.py M tests/custom_cluster/test_rpc_exception.py M tests/query_test/test_codegen.py M tests/webserver/test_web_pages.py 44 files changed, 88 insertions(+), 2,143 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 8 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 7 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 23 Jul 2018 23:30:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2851/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 7 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 23 Jul 2018 23:30:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 6: Code-Review+2 Fix clang-tidy error. Carry Sailesh's +2. -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 23 Jul 2018 23:29:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Hello Sailesh Mukil, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10835 to look at the new patch set (#6). Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. IMPALA-7212: Removes --use_krpc flag and remove old DataStream services This change removes the flag --use_krpc which allows users to fall back to using Thrift based implementation of DataStream services. This flag was originally added during development of IMPALA-2567. It has served its purpose. As we port more ImpalaInternalServices to use KRPC, it's becoming increasingly burdensome to maintain parallel implementation of the RPC handlers. Therefore, going forward, KRPC is always enabled. This change removes the Thrift based implemenation of DataStreamServices and also simplifies some of the tests which were skipped when KRPC is disabled. Testing done: core debug build. Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 --- M be/src/common/global-flags.cc M be/src/exec/data-sink.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/rpc/authentication.cc M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/thrift-server-test.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/backend-client.h D be/src/runtime/data-stream-mgr-base.h D be/src/runtime/data-stream-mgr.h D be/src/runtime/data-stream-recvr-base.h D be/src/runtime/data-stream-recvr.cc D be/src/runtime/data-stream-recvr.h D be/src/runtime/data-stream-sender.cc D be/src/runtime/data-stream-sender.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/scheduling/scheduler.cc M be/src/service/data-stream-service.cc M be/src/service/impala-internal-service.cc M be/src/service/impala-internal-service.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M bin/run-all-tests.sh M bin/start-impala-cluster.py M common/thrift/ImpalaInternalService.thrift M tests/common/custom_cluster_test_suite.py M tests/common/skip.py D tests/common/test_skip.py M tests/conftest.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_krpc_metrics.py M tests/custom_cluster/test_rpc_exception.py M tests/query_test/test_codegen.py M tests/webserver/test_web_pages.py 44 files changed, 88 insertions(+), 2,143 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/10835/6 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 23 Jul 2018 18:43:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Hello Sailesh Mukil, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10835 to look at the new patch set (#5). Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. IMPALA-7212: Removes --use_krpc flag and remove old DataStream services This change removes the flag --use_krpc which allows users to fall back to using Thrift based implementation of DataStream services. This flag was originally added during development of IMPALA-2567. It has served its purpose. As we port more ImpalaInternalServices to use KRPC, it's becoming increasingly burdensome to maintain parallel implementation of the RPC handlers. Therefore, going forward, KRPC is always enabled. This change removes the Thrift based implemenation of DataStreamServices and also simplifies some of the tests which were skipped when KRPC is disabled. Testing done: core debug build. Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 --- M be/src/common/global-flags.cc M be/src/exec/data-sink.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/rpc/authentication.cc M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/thrift-server-test.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/backend-client.h D be/src/runtime/data-stream-mgr-base.h D be/src/runtime/data-stream-mgr.h D be/src/runtime/data-stream-recvr-base.h D be/src/runtime/data-stream-recvr.cc D be/src/runtime/data-stream-recvr.h D be/src/runtime/data-stream-sender.cc D be/src/runtime/data-stream-sender.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/scheduling/scheduler.cc M be/src/service/data-stream-service.cc M be/src/service/impala-internal-service.cc M be/src/service/impala-internal-service.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M bin/run-all-tests.sh M bin/start-impala-cluster.py M common/thrift/ImpalaInternalService.thrift M tests/common/custom_cluster_test_suite.py M tests/common/skip.py D tests/common/test_skip.py M tests/conftest.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_krpc_metrics.py M tests/custom_cluster/test_rpc_exception.py M tests/query_test/test_codegen.py M tests/webserver/test_web_pages.py 44 files changed, 88 insertions(+), 2,143 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/10835/5 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 5: Code-Review+1 Rebased. Carry +1. -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 23 Jul 2018 18:22:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 4: Code-Review+1 This looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Wed, 18 Jul 2018 18:39:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Hello Sailesh Mukil, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10835 to look at the new patch set (#4). Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. IMPALA-7212: Removes --use_krpc flag and remove old DataStream services This change removes the flag --use_krpc which allows users to fall back to using Thrift based implementation of DataStream services. This flag was originally added during development of IMPALA-2567. It has served its purpose. As we port more ImpalaInternalServices to use KRPC, it's becoming increasingly burdensome to maintain parallel implementation of the RPC handlers. Therefore, going forward, KRPC is always enabled. This change removes the Thrift based implemenation of DataStreamServices and also simplifies some of the tests which were skipped when KRPC is disabled. Testing done: core debug build. Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 --- M be/src/common/global-flags.cc M be/src/exec/data-sink.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/rpc/authentication.cc M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/thrift-server-test.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/backend-client.h D be/src/runtime/data-stream-mgr-base.h D be/src/runtime/data-stream-mgr.h D be/src/runtime/data-stream-recvr-base.h D be/src/runtime/data-stream-recvr.cc D be/src/runtime/data-stream-recvr.h D be/src/runtime/data-stream-sender.cc D be/src/runtime/data-stream-sender.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/scheduling/scheduler.cc M be/src/service/data-stream-service.cc M be/src/service/impala-internal-service.cc M be/src/service/impala-internal-service.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M bin/run-all-tests.sh M bin/start-impala-cluster.py M common/thrift/ImpalaInternalService.thrift M tests/common/custom_cluster_test_suite.py M tests/common/skip.py D tests/common/test_skip.py M tests/conftest.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_krpc_metrics.py M tests/custom_cluster/test_rpc_exception.py M tests/query_test/test_codegen.py M tests/webserver/test_web_pages.py 44 files changed, 88 insertions(+), 2,143 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/10835/4 -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/10835/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10835/3//COMMIT_MSG@7 PS3, Line 7: Deprecate > Nit: I think it would be clearer to say "Remove" rather than "Deprecate". Done http://gerrit.cloudera.org:8080/#/c/10835/3//COMMIT_MSG@9 PS3, Line 9: deprecates > Same here. Done http://gerrit.cloudera.org:8080/#/c/10835/3/be/src/common/global-flags.cc File be/src/common/global-flags.cc: http://gerrit.cloudera.org:8080/#/c/10835/3/be/src/common/global-flags.cc@40 PS3, Line 40: DEFINE_int32_hidden(krpc_port, 27000, : "port on which KRPC based ImpalaInternalService is exported"); > Now that we're exposing this, we should probably document this (if we're no Internal doc bug filed. http://gerrit.cloudera.org:8080/#/c/10835/3/be/src/common/global-flags.cc@260 PS3, Line 260: REMOVED_FLAG(disable_mem_pools); > What are our rules about removing startup flags? Should use_krpc be here? ( Done http://gerrit.cloudera.org:8080/#/c/10835/1/tests/custom_cluster/test_rpc_exception.py File tests/custom_cluster/test_rpc_exception.py: http://gerrit.cloudera.org:8080/#/c/10835/1/tests/custom_cluster/test_rpc_exception.py@a73 PS1, Line 73: > So that goes to say that we won't benefit much from running with this for R Yes -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Wed, 18 Jul 2018 05:33:32 + Gerrit-HasComments: Yes
